metric-creator

To generate a Fair-Forge metric scaffold, provide a metric name and optional description to produce a complete project structure including source code, schemas, tests, and fixtures for the new evaluation.

A Fair-Forge metric boilerplate includes pyproject dependencies, metric implementation templates, schemas, test fixtures, example data, and optional LLM-judge or guardian components to ensure project consistency.

Adding tests and fixtures for a custom evaluation metric involves using a scaffold generator that automatically produces standardized test files and example data alongside the metric implementation source code.

Yes, you can include an LLM-judge component when scaffolding a new metric. The generation process produces optional templates for LLM or guardian components to help teams adopt uniform evaluation patterns.

Yes, the metric scaffold enforces specific naming conventions and a standard project layout. This structure streamlines integration and ensures consistency across all newly created evaluation metrics.

Using a scaffold generator instead of manually creating evaluation metric files ensures proper structure, schema, and test coverage while providing ready-to-adapt boilerplate that maintains consistency with existing metrics.