math-critic

What problem does it solve?

This Skill provides a comprehensive reference for mathematical rigor in machine learning, deep learning, and language model code, addressing loss function correctness, gradient verification, numerical stability, and statistical validity.

Core Features & Use Cases

• Loss Function Correctness: Offers guidance on proper loss function usage and common pitfalls.
• Numerical Stability: Discusses patterns and best practices to maintain stability in computations.
• Gradient and Backpropagation: Provides a guide on the correct order and practices for training loops.
• Optimizer and Scheduler: Offers insights into the selection and tuning of optimizers and learning rate schedules.
• Initialization: Details best practices for initializing neural network layers and weights.
• Attention Mathematics: Explains the mathematics behind attention mechanisms.
• Sampling Strategies: Covers temperature scaling, top-k/p, and repetition penalty for LLMs.
• Embeddings and Similarity: Discusses cosine similarity, dot product, and their applications.
• RAG Scoring and Evaluation: Provides metrics and methods for evaluating RAG systems.
• Statistical Validity: Offers guidelines for hypothesis testing, effect size, and confidence intervals.
• Calibration: Discusses metrics and methods for calibrating neural network predictions.
• Reproducibility: Provides a guide on ensuring reproducibility in experiments.
• Common Bugs: Lists common bugs and their detection patterns.
• Canonical References: Provides citations for key references in the field.
• Use Case: When auditing ML/DL/LLM code, use this Skill to verify mathematical correctness and statistical validity before proceeding to code review.

Quick Start

Analyze the mathematical rigor of the loss function in the provided model by running the math-critic skill on the model's codebase.