training-pipeline-audit

Audit ML training pipelines to extract research gaps, hypotheses, and IMRaD paper plans.

6|1|Updated May 11, 2026
One-click install
npx skills add https://github.com/yakeworld/Synthos --skill training-pipeline-audit-yakeworld
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: training-pipeline-audit
Source: https://github.com/yakeworld/Synthos/tree/main/skills/private/research/training-pipeline-audit
Command: npx skills add https://github.com/yakeworld/Synthos --skill training-pipeline-audit-yakeworld

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Researchers with completed ML training pipelines often struggle to convert their engineering work into publishable science. This Skill audits a training pipeline project (datasets, pipeline steps, training logs) and produces a structured research output: identified research gaps, falsifiable scientific hypotheses, and an IMRaD-structured paper plan with literature scope and timeline. ## Core Features & Use Cases - Pipeline-to-Methodology Decomposition: Scans project files and decomposes pipeline steps (e.g., pupil localization, region growing, SAM segmentation) into a formal methodology description. - Research Gap & Hypothesis Generation: Identifies at least 3 research gaps located in literature contradictions or method gaps, and generates at least 3 testable hypotheses with test conditions and falsification paths. - Paper Planning with Data Honesty Gate: Outputs a paper_plan.json with IMRaD structure, 15-30 reference literature scope, target journal tier, and timeline, while rejecting any metric (e.g., Val Dice) that cannot be traced to training logs. - Use Case: Given a K230 eye-tracking project with 901 frames, a 7-step CV pipeline, and MobileNetV2 training logs, the Skill produces 5 research gaps, 5 falsifiable hypotheses, and 2 SCI paper plans (Q1 methodology + Q2 empirical analysis). ## Quick Start Audit my training pipeline project with its datasets, pipeline steps, and training logs, then generate research gaps, testable hypotheses, and an IMRaD paper plan.

Frequently Asked Questions about training-pipeline-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I turn a machine learning training pipeline into a research paper?

Audit the pipeline by decomposing its steps into a methodology, identifying at least 3 research gaps tied to literature contradictions or method gaps, and generating falsifiable hypotheses. Then produce an IMRaD-structured paper plan with a literature list and timeline.

How to generate testable hypotheses from training results?

Each hypothesis must include a claim threshold, a test_condition describing an executable experiment, and a falsification_path stating what result would disprove it. For example, claiming segmentation error dominates total error requires error decomposition across pipeline steps.

What is IMRaD structure for a methods paper?

IMRaD organizes a paper into Introduction (background, gap, contribution), Methods (pipeline, algorithm, experiment design, data), Results (main metrics, ablation, comparisons), Discussion (strengths, limitations, literature comparison), and Conclusion.

What happens if training metrics cannot be traced to logs?

The data honesty gate rejects unsourced metrics such as a verbally reported Dice score. The metric is marked unverified with missing_source, excluded from the paper plan's Results section, and an error report with recovery suggestions is returned.

How many research gaps and hypotheses are required for a valid audit?

A passing audit requires at least 3 research gaps, each located in a specific literature contradiction or method gap, and at least 3 hypotheses with test conditions and falsification paths. Fewer than 3 of either fails the quality gate.