gen-eval

Descriptor-driven scenario generation validates microservice behavior by auto-detecting YAML interface definitions and automatically generating test scenarios to detect regressions, coverage gaps, and incorrect behavior in live or local services.

Generator-evaluator testing validates CLI and SDK behavior by generating test scenarios from interface descriptors and evaluating execution results. It supports template-only, cli-augmented, and sdk-only modes to verify correct service behavior.

Yes, descriptor-driven scenario execution supports parallel test runs with configurable time and cost budgets across template-only, cli-augmented, and sdk-only modes to manage resource limits during validation.

Yes, descriptor-driven scenario generation requires a Python runtime with the evaluation module available. You also need a descriptor YAML file in the evaluation directory, plus optional CLI or SDK credentials for augmented modes.

Generator-evaluator test results integrate into triage workflows through deterministic markdown and JSON reporting. The system produces per-scenario metrics and exit-code pass/fail semantics to streamline failure validation and feature-exploration workflows.

Template-only generator-evaluator testing generates scenarios solely from interface descriptors without CLI or SDK augmentation. It cannot validate live service behavior requiring authentication credentials or runtime SDK interactions.