lai-gen-evals

What problem does it solve?

This Skill helps you create an evaluation suite that measures whether an LLM produces Langium DSL code that is syntactically correct, semantically valid, matches user intent, and handles edge cases—so you can iterate reliably instead of relying on superficial “it parses” checks.

Core Features & Use Cases

• Expands eval coverage by category: organizes evals into focused suites like syntax, semantics, intent, edge cases, and understanding.
• Leverages real Langium validation: uses LangiumEvaluator to validate generated programs against your actual parser and validator.
• Supports robust scoring and debugging: guides you to compute meaningful 0–1 scores (including composites) and return useful diagnostics and extra metadata.

Quick Start

Use the lai-gen-evals skill to generate a comprehensive eval suite inside your project’s evals/ directory based on your existing descriptor, system prompt, and Langium language validation behavior.