skills.rest
ThalesGroup
ThalesGroupOFFICIAL
Explore all skills by this author →
WebsiteGitHub

agilab-prompt-eval-regression

Convert AGILAB prompts and model responses into repeatable regression fixtures.

19|2|Updated Mar 14, 2025
Search Tags:#llm#regression#fixtures#notebooks#tests#agilab#prompt-eval
One-click install
npx skills add https://github.com/ThalesGroup/agilab --skill agilab-prompt-eval-regression
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agilab-prompt-eval-regression
Source: https://github.com/ThalesGroup/agilab/tree/main/.claude/skills/agilab-prompt-eval-regression
Command: npx skills add https://github.com/ThalesGroup/agilab --skill agilab-prompt-eval-regression

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps design and maintain deterministic regression evaluations for AGILAB prompts, local/remote LLM flows, notebook import classification, generated-code routing, agent-skill behavior, and prompt-driven repair or analysis features.

Core Features & Use Cases

  • Static prompt contracts: bounded, redacted prompts with enough task context to drive reliable tests.
  • Deterministic decision fixtures: fixtures that encode routing and classification decisions without querying live models.
  • Optional live validation: opt-in model checks with clear evidence and timeout controls for safe experimentation.

Quick Start

Run targeted regression tests across touched AGILAB components to produce repeatable fixtures and evidence.

Frequently Asked Questions about agilab-prompt-eval-regression

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I make LLM prompt workflows deterministic for regression testing?▼

To make LLM prompt workflows deterministic, you convert prompts and model responses into repeatable regression fixtures that encode routing and classification decisions without querying live models.

What is a static prompt contract for testing notebook import routing?▼

A static prompt contract is a bounded, redacted prompt containing enough task context to drive reliable tests for notebook import classification and generated-code routing decisions.

Can I validate prompt changes against live models without breaking test determinism?▼

Yes, you can use optional live validation to run opt-in model checks with clear evidence and timeout controls, ensuring safe experimentation without compromising deterministic fixtures.

How do I test agent-skill behavior when prompt templates or model defaults change?▼

You test agent-skill behavior changes by running targeted regression tests across touched components to produce repeatable fixtures and evidence for local and remote LLM flows.

Does regression testing for prompt-driven workflows support data redaction?▼

Yes, deterministic regression fixtures apply redaction and evidence schemas to ensure safety and reproducibility when evaluating prompt templates and model defaults.

When should I use deterministic decision fixtures instead of live model validation?▼

Use deterministic decision fixtures to encode routing and classification decisions for reliable tests, and use optional live validation only when you need safe experimentation with live models.

Related Skills

Explore More Software Engineering Skills→
ekremmkasapekremmkasap

turborepo-caching

Configure turbo.json pipelines and enable local and remote caching for monorepo builds.

Community
Advanced
ThalesGroupThalesGroup
19

agilab-testing

Execute targeted AGILAB tests across unit, smoke, and regression suites.

Official
Advanced
ThalesGroupThalesGroup
19

agilab-ui-robot-validation

Validate AGILAB Streamlit UI changes with automated browser robots.

Official
Intermediate
AlchemicationAlchemication
1

llm-evals

Convert observed LLM failures into structured regression test cases.

Community
Advanced