repo-evals

Evaluate open-source repos against README promises and render bilingual HTML dossiers.

1|Updated Apr 5, 2026
One-click install
npx skills add https://github.com/zinan92/repo-evals --skill repo-evals
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: repo-evals
Source: https://github.com/zinan92/repo-evals/tree/main
Command: npx skills add https://github.com/zinan92/repo-evals --skill repo-evals

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, uuidgen, gh, python3, and includes scripts (resource) components.

What problem does it solve?

repo-evals turns the question “Is this repo/skill actually usable, or just marketing?” into a structured, auditable evaluation dossier with a 0–100 score and an editorial, bilingual product page. It emphasizes claim-first, evidence-backed scoring so readers can challenge specific numbers point-by-point.

Core Features & Use Cases

  • Claim-first evaluation: convert README/SKILL promises into a claim ledger, then score claim-by-claim outcomes.
  • Auditable 0–100 scoring: compute a score with a transparent, six-component breakdown and explicit ceiling rules.
  • Bilingual dossier output: render an HTML one-pager covering decision snapshot, benefits (persona/scenario/without/with), workflow diagram, and similar-repos comparison from the corpus.

Quick Start

Use the repo-evals skill when the user asks to evaluate a repo (e.g., “eval 一下这个 repo”) by providing the target GitHub URL; then follow the framework’s steps to scaffold an evaluation directory, write the claim map, run coverage and scoring, and render the bilingual HTML dossier.

Frequently Asked Questions about repo-evals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate an open-source repo against its README claims to see if it is actually usable?

To evaluate a repo against its README claims, you convert the stated promises into a claim ledger, then score claim-by-claim outcomes to produce a trustworthy adoption dossier with an auditable 0–100 score.

How does claim-first evidence tracking work for generating an auditable repository score?

Claim-first evidence tracking works by mapping README promises to a claim ledger, applying rule-guided static evaluation, and using deterministic verdict calculation to compute a transparent 0–100 score with a six-component breakdown.

Can I compare two repos on consistent dimensions and generate a bilingual HTML report?

Yes, you can compare two repos on consistent dimensions by applying rule-guided static evaluation and evidence provenance tracking, then render the results into a bilingual HTML one-pager covering the decision snapshot and comparison.

What dependencies do I need to run a repo evaluation and generate a dossier?

You need python3, pyyaml, uuidgen, and gh installed in your environment to run the repo evaluation scripts, track evidence provenance, and generate the bilingual HTML dossier.

What is the best way to surface claim-versus-reality gaps in an open-source framework?

The best way to surface claim-versus-reality gaps is to author a claim map from the framework's README, run rule-guided static evaluation against it, and track evidence point-by-point to challenge specific numbers.

When should I not use an automated repo evaluation approach for auditing tools?

You should not use automated repo evaluation when you cannot author a claim map from the README, as the framework requires converting stated promises into a claim ledger before applying rule-guided static evaluation and deterministic verdict calculation.