benchmark-summary

Generate combined benchmark analysis reports from GitHub issues for clinical trial design skills.

93|23|Updated Mar 4, 2026

Search Tags:#failure analysis #data normalization #GitHub API #benchmark analysis #clinical trial design

Ready to run

npx skills add https://github.com/RConsortium/pharma-skills --skill benchmark-summary

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: benchmark-summary
Source: https://github.com/RConsortium/pharma-skills/tree/main/_automation/benchmark-summary
Command: npx skills add https://github.com/RConsortium/pharma-skills --skill benchmark-summary

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires gh, jq, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill automates the generation of a combined benchmark analysis for group-sequential-design skills, ensuring a streamlined and consistent evaluation of performance over time.

Core Features & Use Cases

Benchmark Analysis Generation: Aggregates and evaluates benchmark results from GitHub issues, focusing on the latest completed runs.
Data Normalization: Applies normalization rules to model names for consistency in reporting.
Failure Pattern Analysis: Identifies common failure patterns and provides recommendations for improvement.
Incremental Updates: Allows for incremental updates to the benchmark analysis, ensuring the document grows with new data.

Quick Start

To generate the benchmark analysis, execute the skill with the command 'generate benchmark analysis'.

Frequently Asked Questions about benchmark-summary

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate benchmark analysis for clinical trial design skills?▼

Automating benchmark analysis for clinical trial design skills involves processing GitHub issues to evaluate performance. This skill aggregates completed runs, normalizes run metadata, and compiles a structured report for consistent evaluation over time.

What is data normalization for benchmark run metadata?▼

Data normalization for benchmark run metadata applies standardization rules to model names, ensuring consistency in reporting. This process formats varying model identifiers into uniform values before compiling the final benchmark analysis report.

Do I need GitHub API access to generate benchmark analysis reports?▼

Yes, generating benchmark analysis reports requires GitHub API access to fetch and process GitHub issues. The workflow depends on the gh tool to retrieve completed run data and compile the structured evaluation document.

How do I identify failure patterns in clinical trial design benchmark runs?▼

Identifying failure patterns in clinical trial design benchmark runs involves analyzing aggregated GitHub issues from completed runs. The skill processes this data to detect common failure trends and provides recommendations for improvement.

Can I apply incremental updates to a benchmark analysis document?▼

Yes, you can apply incremental updates to a benchmark analysis document. This functionality allows the report to grow continuously by appending new normalized run metadata and failure pattern data from recent completed runs.

Does this benchmark analysis skill work with gh and jq dependencies?▼

Yes, this benchmark analysis skill works with gh and jq dependencies. It uses gh for GitHub API access to retrieve issues and jq to parse and normalize the run metadata before compiling the structured report.