auditing-notebook-narrative

To check if a notebook narrative matches rendered code outputs, apply markdown parsing to detect directional claims, then compare them against nearest rendered outputs for numeric or visual discrepancies. This flags stale prose before sharing notebooks, grading assignments, or publishing model cards.

A narrative audit for data science notebooks is a validation process that detects prose-versus-output mismatches by comparing markdown claims about trends, figures, or metrics against rendered code-cell outputs. It ensures written stories align with actual analysis results before publication or grading.

To catch reversed claims in a rerun Jupyter notebook, perform an output comparison that detects directional statements about increases or decreases and matches them to the nearest rendered output. This generates a verdict report separating clear numeric disagreements from cases needing human review.

Yes, markdown parsing can find mismatches between text and figures in graded assignments by identifying directional claims about best or worst results and convergence, then applying best-effort numeric or visual comparison against rendered code-cell outputs to flag inconsistencies with specific cell indices.

The limitation of automated notebook output comparison for model cards is that it relies on best-effort numeric or visual comparison, meaning it separates clear numeric disagreements from cases needing human review. Complex visual interpretations or ambiguous markdown claims may require manual verification.