{"repo":"angular/web-codegen-scorer","free":true,"listed":false,"github":"https://github.com/angular/web-codegen-scorer","clone":"git clone https://github.com/angular/web-codegen-scorer.git","description":"Web Codegen Scorer is a tool for evaluating the quality of web code generated by LLMs.","language":"TypeScript","stars":764,"topics":["benchmarking","codegen","llm-coding","evaluation"],"license":"MIT","category":"dev-tools","readme_excerpt":"Web Codegen Scorer Web Codegen Scorer is a tool for evaluating the quality of web code generated by Large Language Models (LLMs). You can use this tool to make evidence-based decisions relating to AI-generated code. For example: 🔄 Iterate on a system prompt to find most effective instructions for your project. ⚖️ Compare the code quality of code produced by different models. 📈 Monitor generated code quality over time as models and agents evolve. Web Codegen Scorer is different from other code benchmarks in that it focuses specifically on web code and relies primarily on well-established measures of code quality. Features ⚙️ Configure your evaluations with different models, frameworks, and tools. ✍️ Specify system instructions and add MCP servers. 📋 Use built-in checks for build success, runtime errors, accessibility, security, LLM rating, and coding best practices. (More built-in checks coming soon!) 🔧 Automatically attempt to repair issues detected during code generating. 📊 View and compare results with an intuitive report viewer UI. Setup 1. Install the package: 2. Set up your API keys: In order to run an eval, you have to specify an API keys for the relevant providers as environment variables: 3. Run an eval: You can run your first eval using our Angular example with the following command: 4. (Optional) Set up your own eval: If you want to set up a custom eval, instead of using our built-in examples, you can run the following command which will guide you through the p","default_branch":null,"files":null,"tree":[],"storefront":"/r/angular","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/angular/web-codegen-scorer/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}