skills.rest
awslabs
awslabsOFFICIAL
Explore all skills by this author →
WebsiteGitHub

run-source-model-baseline

Generate live source-model baselines from golden prompt datasets for LLM migration evaluation.

15|20|Updated May 11, 2026
Search Tags:#gemini#openai#anthropic#bedrock#jsonl#baseline evaluation#llm migration
One-click install
npx skills add https://github.com/awslabs/startups --skill run-source-model-baseline
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: run-source-model-baseline
Source: https://github.com/awslabs/startups/tree/main/migrate/plugins/ai-to-aws/skills/run-source-model-baseline
Command: npx skills add https://github.com/awslabs/startups --skill run-source-model-baseline

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the problem of unreliable AI migration evaluations by generating real source-model responses instead of relying on agent-created baseline answers.

Core Features & Use Cases

  • Live Source Model Execution: Re-runs golden prompt datasets against OpenAI, Anthropic, or Gemini models to collect authentic source outputs.
  • Model Validation and Recovery: Resolves safe model variants, validates provider availability, handles partial resumes, and records failures without exposing credentials.
  • Migration Evaluation Support: Provides side-by-side source and Bedrock outputs for LLM migration quality comparisons and regression analysis.

Quick Start

Use the run-source-model-baseline skill to generate live source model responses from the golden dataset for migration evaluation.

Frequently Asked Questions about run-source-model-baseline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate live source model baselines for LLM migration evaluation?▼

Generate live source model baselines by re-running golden prompt datasets against OpenAI, Anthropic, or Gemini APIs. This collects authentic source outputs, producing evaluation-ready baseline records for LLM migration quality comparisons.

Why use live source model execution instead of agent-created baselines for AI migration?▼

Live source model execution solves unreliable AI migration evaluations by generating real source-model responses from OpenAI, Anthropic, or Gemini. This authentic baseline data prevents inaccuracies found in agent-created baseline answers.

Can I process JSONL golden prompt datasets to compare Bedrock outputs against OpenAI models?▼

Yes, you can process JSONL golden prompt datasets to compare Bedrock outputs against OpenAI, Anthropic, or Gemini models. The system securely executes APIs to produce side-by-side evaluation-ready baseline records.

Does baseline evaluation support partial resumes and model validation recovery?▼

Yes, baseline evaluation supports partial resumes and model validation recovery. It resolves safe model variants, validates provider availability, and records failures without exposing credentials during JSONL processing.

What provider credentials do I need for generating LLM migration baselines?▼

You need validated provider credentials for OpenAI, Anthropic, Gemini, or Bedrock to generate LLM migration baselines. The system executes secure API calls against these providers without exposing your credentials during processing.

Related Skills

Explore More Software Engineering Skills→
open-haxopen-hax
6

opencode-model-variant-management

Add new models, providers, or variants to OpenCode configurations with validation.

Official
Advanced
DowwieDowwie
5

comparative-matrix

Generate a structured decision matrix comparing analyzed frameworks and architectures.

Community
Basic
mmtftrmmtftr

convex-http-actions

Define HTTP routes and webhook handlers in Convex applications using httpRouter and httpAction.

Community
Advanced
MooseGoose0701MooseGoose0701
1.1k

trace-qa

Analyze agent execution traces to debug steps, LLM calls, and tool usage.

Community
Intermediate