Executes an evaluation pipeline for AI-generated code with deterministic and LLM-based graders.
npx skills add https://github.com/kaelig/react-craft --skill eval-runner
Please help me install this Agent Skill. Skill: eval-runner Source: https://github.com/kaelig/react-craft/tree/main/react-craft/skills/eval-runner Command: npx skills add https://github.com/kaelig/react-craft --skill eval-runner
This Skill automates the rigorous evaluation of AI-generated code against design specifications, ensuring quality, correctness, and adherence to standards.
Run the eval-runner skill to execute the react-craft eval suite against the fixture located at /path/to/fixture.
High-intent search queries and answers about installing and using this skill.
To evaluate AI-generated React components, run an automated evaluation suite that tests compilation, checks linting errors, verifies design token usage, and assesses accessibility compliance to produce a comprehensive quality score.
Benchmarking AI code generation involves executing code against fixtures and scoring it with deterministic and LLM-as-judge graders, which validates code quality, adherence to design specifications, and identifies potential issues through a multi-faceted grading system.
An LLM-as-judge grader works as part of a multi-faceted evaluation pipeline to semantically assess AI-generated code, complementing deterministic checks to validate correctness, design token adherence, and accessibility compliance in detailed benchmark reports.
Yes, automated evaluation pipelines can verify whether AI-generated code uses design tokens correctly, alongside checking compilation, linting errors, and accessibility compliance, providing a comprehensive quality score and detailed benchmark report.
Deterministic graders for AI code benchmarking are limited to fixed rule checks like compilation and linting, which is why an LLM-as-judge grader is also required to evaluate semantic correctness and complex design specification adherence.
aroidoCreate unit and integration tests for React components with Vitest and React Testing Library.
Hashzin-0Review TypeScript code for anti-patterns, security vulnerabilities, and completeness gaps.
ACuberoAutomate Python CI/CD pipelines with GitHub Actions for testing, linting, and releases.
majiayu000Quantify AI-generated code quality using pass@k and pass^k metrics.
