ds-roast

Audit ML projects for data leakage, bugs, and bad practices.

Updated Dec 21, 2021
One-click install
npx skills add https://github.com/dobraga/dotfiles --skill ds-roast
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ds-roast
Source: https://github.com/dobraga/dotfiles/tree/main/.claude/skills/ds-roast
Command: npx skills add https://github.com/dobraga/dotfiles --skill ds-roast

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It performs a ruthless, structured evaluation of data science and ML projects to surface bugs, data leakage, bad practices, and improvement opportunities.

Core Features & Use Cases

  • Exhaustive codebase reconnaissance of notebooks, pipelines, and production ML artifacts, with a concrete set of issues to address.
  • Systematic checks for data quality, leakage (temporal, target, group), modeling pitfalls, and MLOps gaps, plus concrete remediation steps.
  • Use Case: run this roast on a complete project to generate a prioritized action list that reduces risk before production deployment.

Quick Start

Provide the ML project source (notebooks, scripts, configs) to initiate a roast and receive a detailed technical critique.

Frequently Asked Questions about ds-roast

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check my ML project for data leakage before deployment?

A structured ML project critique systematically validates data quality and performs explicit temporal, target, and group leakage checks across your notebooks and pipelines. It generates a prioritized remediation list to fix hidden leakage before production deployment.

What is the best way to review a data science notebook for bad practices?

Reviewing a data science notebook for bad practices requires an exhaustive codebase reconnaissance applying a rigorous checklist. This surfaces modeling pitfalls and MLOps gaps, producing a detailed technical critique with prioritized, actionable improvement suggestions.

Can I evaluate production ML pipelines for bugs and data quality issues systematically?

You can systematically evaluate production ML pipelines by running a structured technical roast that performs data quality validations and leakage checks. This surfaces bugs and MLOps gaps, resulting in a prioritized action list with concrete remediation steps.

Does a rigorous code review help find MLOps gaps in machine learning codebases?

A rigorous code review identifies MLOps gaps in machine learning codebases by performing exhaustive reconnaissance of scripts and configs. It systematically checks for data quality, leakage, and modeling pitfalls, delivering clear remediation suggestions to reduce production risk.

How do I perform a structured critique of an ML model evaluation workflow?

Performing a structured critique of an ML model evaluation workflow requires a hard-nosed technical roast across development, evaluation, and deployment stages. This enforces explicit leakage checks and modeling appropriateness validations to generate targeted remediation actions.