skills.rest
SpreadZhao
SpreadZhaoCOMMUNITY
Explore all skills by this author →
GitHub

ingest-paper

Normalizes academic papers from PDF, URL, or HTML into canonical Markdown with traceable source blocks.

4|Updated May 20, 2024
Search Tags:#ocr#markdown#pdf extraction#literature review#academic papers#arxiv#source traceability
One-click install
npx skills add https://github.com/SpreadZhao/spreadconfig --skill ingest-paper-spreadzhao
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ingest-paper
Source: https://github.com/SpreadZhao/spreadconfig/tree/main/plugins/paper-reading-workbench/skills/ingest-paper
Command: npx skills add https://github.com/SpreadZhao/spreadconfig --skill ingest-paper-spreadzhao

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Academic papers arrive in inconsistent formats (PDFs, DOI pages, arXiv links, HTML, plain text), and naive text extraction loses reading order, equations, figures, and provenance. This Skill converts any supported source into a verified, loss-aware canonical Markdown baseline with stable source-block IDs, page images, and an extraction report inside a Paper Reading Workbench workspace. ## Core Features & Use Cases - Multi-format ingestion: Accepts local PDFs, direct URLs, DOI/arXiv/publisher pages, HTML, Markdown, and text, preserving the immutable original under 00 Source/. - Loss-aware extraction: Renders PDF page images, extracts embedded figures, falls back to OCR (ocrmypdf or tesseract) when native text is sparse, and localizes HTML images. - Traceable source blocks: Assigns stable IDs and SHA-256 hashes to every paragraph, heading, equation, table, and figure in source-map.json, with an extraction report listing uncertain or unreadable regions. - Use Case: A researcher downloads an arXiv PDF and asks the agent to ingest it; the Skill renders page images, extracts text and figures, flags OCR-uncertain regions, and produces a verified paper.md ready for later segmentation and analysis. ## Quick Start Use $ingest-paper to ingest the paper at this URL into my active paper reading workspace and verify the extraction against the rendered page images.

Frequently Asked Questions about ingest-paper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert an academic PDF into structured Markdown?▼

Run the ingest_paper.py script with a workspace folder and the PDF path or URL. It extracts text with pdftotext, renders page images with pdftoppm, pulls embedded figures with pdfimages, and writes canonical Markdown with stable source-block IDs.

What tools extract text and images from PDF papers?▼

This workflow uses the poppler utilities: pdfinfo for metadata, pdftotext for layout-preserving text, pdftoppm for page images, and pdfimages for embedded figures. When native text is sparse, it falls back to ocrmypdf or tesseract for OCR.

Can I ingest a paper from a DOI or arXiv link?▼

Yes, the script downloads sources from http(s) URLs including DOI, arXiv, OpenReview, and publisher pages, saving the original response unchanged. It refuses to bypass paywalls or access controls and records the access failure instead.

Does PDF text extraction work on scanned papers?▼

Scanned PDFs lack a native text layer, so the script detects sparse text and attempts OCR via ocrmypdf or tesseract against rendered page images. Uncertain OCR regions are flagged in the extraction report rather than silently corrected.

Why does the ingestion require a workspace first?▼

The script validates an existing Paper Reading Workbench workspace containing workflow state, a paper ID, and directory structure before writing outputs. This ensures source blocks, metadata, and assets stay traceable across later reading phases.

Related Skills

Explore More Education & Research Skills→
Nitrogen216Nitrogen216

deepxiv

Search and progressively read open-access academic papers through the DeepXiv SDK.

Community
Intermediate
xiaoyu-opsxiaoyu-ops

academic-researcher

Aggregate, summarize, and synthesize scholarly papers for literature reviews.

Community
Advanced
trungkiet2005trungkiet2005

citation-management

Generate validated BibTeX bibliographies from searches across Google Scholar, PubMed, Crossref, arXiv, and more.

Community
Advanced
sakamoto-family-smilesakamoto-family-smile

content-hash-cache-pattern

Cache file processing results using SHA-256 content hashes.

Community
Intermediate