ml-math-foundations

Define the network as a composition of layers with pre-activations and activations, write the loss, then apply the chain rule to get the output error term and the recurrence for earlier layers. Parameter gradients follow as outer products of error terms and previous activations, which is exactly reverse-mode automatic differentiation.

Restate the claim with explicit assumptions, declare notation and tensor shapes, give a one-paragraph intuition, then derive the result step by step with each step labeled as proof, informal argument, or empirical fact. Finish with a sanity check on a degenerate case and the practical interpretation.

It covers linear algebra, calculus, probability and statistics (MLE/MAP, bias-variance), optimization theory (convexity, gradient descent convergence), information theory (entropy, KL divergence, mutual information), and learning theory (generalization, VC/PAC intuition) as they apply to ML methods.

Yes, it handles convergence proofs for gradient-based optimization and generalization or PAC-style intuition, stating assumptions like convexity or i.i.d. data explicitly. If a full proof is too long or uncertain, it says so and cites an authoritative source instead of fabricating results.

Informal arguments can hide broken assumptions such as non-differentiability or dependence between samples, so conclusions may not hold in practice. This Skill mitigates that by labeling rigor levels per step, checking limiting cases, and never presenting hand-waving as proof.