gh-models

What problem does it solve?

gh-models eliminates the manual, error-prone workflow of running AI prompts and then forgetting to verify quality changes, by helping you execute and evaluate prompts directly from the terminal.

Core Features & Use Cases

• Run prompts with context: Inference using gh models run, including REPL usage for exploration and piping real repo content (issues/PRs/events) as input.
• Evaluate prompts with guardrails: Regression-style testing using gh models eval, producing structured pass/fail results and scores from test cases and scorers.
• Generate test suites from prompts: Automatically create robust prompt evaluations with gh models generate, then immediately run eval to catch regressions early (PromptPex methodology).

Quick Start

Use the gh-models skill to run a single prompt against a GitHub Models model and store the response into an output file by providing a prompt asset and redirecting the result from the command output.