review-usability

Tests idiomatic Python code against CPython and Monty to detect behavioral divergences.

8.1k|413|Updated May 28, 2023
One-click install
npx skills add https://github.com/pydantic/monty --skill review-usability
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: review-usability
Source: https://github.com/pydantic/monty/tree/main/.agents/skills/review-usability
Command: npx skills add https://github.com/pydantic/monty --skill review-usability

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

When developing the Monty Python interpreter, changes on a branch can silently break common Python idioms that LLMs typically write, causing wrong results instead of clean errors. This Skill systematically compares branch behavior against CPython to surface divergences before they reach users.

Core Features & Use Cases

  • Idiom Gap Analysis: Lists the obvious methods, argument forms, and builtin combinations a model would reach for first, including cases the branch does not handle.
  • Differential Testing: Writes real test files in the playground directory and runs each under both CPython (uv run) and Monty (cargo run) to diff outputs.
  • Silent Divergence Prioritization: Flags same-code-different-result cases as higher severity than clean AttributeError failures, since wrong answers are unrecoverable.
  • Use Case: After modifying dict method support on a Monty branch, use this Skill to verify that common patterns like dict.get with defaults and dict unpacking still produce identical output under both interpreters.

Quick Start

Review the current branch for usability by testing common Python idioms in the playground against both CPython and Monty, then report any divergences you find.

Frequently Asked Questions about review-usability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test if my Python interpreter branch breaks common code?

Write real test files exercising the idioms a model would use first, then run each under both CPython (uv run) and your interpreter (cargo run) and diff the outputs. Focus on the obvious methods and argument forms rather than exotic edge cases.

What is differential testing between two Python interpreters?

Differential testing runs identical code under two implementations, here CPython and Monty, and compares results. Divergences reveal bugs, with silent wrong answers prioritized over clean errors because users cannot recover from incorrect output.

Why is silent divergence worse than an AttributeError?

A missing feature that raises AttributeError is recoverable because the caller sees the failure and can adapt. Silent divergence returns a wrong answer with no signal, so the model has no way to know it must write different code.

Does this review modify the interpreter source code?

No, the review is report-only unless the user explicitly asks for fixes. It leaves playground test files in place and documents each divergence with the code, both outputs, and likelihood of occurrence.

What kinds of Python code should be tested for usability?

Test the most common patterns an LLM would plausibly write: obvious method calls, standard argument forms, and typical builtin combinations. Exotic corners matter less because real usage concentrates on ordinary idiomatic code.