skills.rest
raphaelmansuy
raphaelmansuyCOMMUNITY
Explore all skills by this author →
Website𝕏@raphaelmansuyGitHub

pdf-markdown-validator

Validate PDF to Markdown conversion quality using F1-scored table, style, robustness, and performance metrics.

2.1k|244|Updated Dec 21, 2025
Search Tags:#ci/cd#pdf#markdown#validation#regression testing#f1-score#conversion quality
One-click install
npx skills add https://github.com/raphaelmansuy/edgequake --skill pdf-markdown-validator-raphaelmansuy
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: pdf-markdown-validator
Source: https://github.com/raphaelmansuy/edgequake/tree/main/.github/skills/pdf-markdown-validator
Command: npx skills add https://github.com/raphaelmansuy/edgequake --skill pdf-markdown-validator-raphaelmansuy

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? PDF to Markdown converters often lose tables, formatting, and heading structure without any objective way to measure the damage. This Skill quantifies conversion fidelity against gold reference files so regressions and quality gaps become measurable instead of anecdotal. ## Core Features & Use Cases - Multi-Dimensional Scoring: Computes a composite 0-100 score from table accuracy (40%), style preservation for bold/italic/headings (40%), robustness (10%), and performance (10%) using token-level F1 metrics. - Failure and Drift Analysis: Categorizes failures by severity and type, and produces line-by-line diff reports showing exactly where generated Markdown diverges from gold references. - CI/CD Quality Gates: Emits JSON reports with exit codes and configurable pass thresholds for automated pipeline enforcement. - Use Case: After modifying a PDF extraction crate, run the validator against your test corpus, compare the new report to the baseline with compare_runs.py, and catch regressions before merging. ## Quick Start Ask the AI to validate the generated Markdown files in your test directory against the .gold.md reference files and summarize the composite quality score.

Frequently Asked Questions about pdf-markdown-validator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate PDF to Markdown conversion quality?▼

Place generated .md files and reference .gold.md files in a directory, then run validate.py with --pdf-dir and --gold-dir arguments. The script outputs a JSON report with table accuracy, style accuracy, robustness, performance, and a composite score out of 100.

How do I compare two PDF conversion validation runs?▼

Use compare_runs.py with the baseline and current report JSON files as arguments. It shows per-metric deltas, percent changes, and flags document-level improvements or regressions beyond a 2-point threshold.

What metrics measure Markdown conversion accuracy?▼

The validator uses token-level F1 scores: table detection and cell content F1 weighted at 40%, bold/italic/heading F1 macro-averaged at 40%, plus robustness and performance at 10% each. These combine into a single composite score from 0 to 100.

Can I enforce a minimum conversion quality score in CI/CD?▼

Yes, run validate.py with --ci-mode and --fail-below followed by your threshold, such as 75. The script exits with a non-zero code when the composite score falls below the threshold, failing the pipeline step.

Why is italic detection scoring low in Markdown validation?▼

Greedy regex patterns match asterisks inside bold text as italic, corrupting scores. The validator fixes this with negative lookahead and lookbehind anchors so single-asterisk patterns do not match inside double-asterisk bold spans.

What are the limitations of this PDF Markdown validator?▼

Table matching relies on cell count tolerance rather than spatial position, heading scoring requires exact level matches, and performance uses a hardcoded baseline instead of measured timing. Gold annotation coverage must be curated manually for each test document.

Related Skills

Explore More Software Engineering Skills→
memyselfandmmemyselfandm
8

scraping-documentation

Scrape documentation websites and convert pages into organized Markdown files.

Community
Intermediate
tak-brotak-bro

health

Aggregate type check, lint, test, and dead code metrics into a health dashboard.

Community
Advanced
OutlineDrivenOutlineDriven
5

perf-profile

Identify performance hotspots using flamegraphs, allocation data, and latency profiles.

Official
Advanced
doanchienthangdevdoanchienthangdev
4

evaluation-methodology

Evaluate AI model outputs with exact match, semantic similarity, LLM-as-judge, and ELO ranking.

Community
Intermediate