skills.rest
yangchuansheng
yangchuanshengCOMMUNITY
Explore all skills by this author →
Website𝕏@carsonyungosGitHub

PubMed / NCBI — Scraping & Data Extraction

Extract PubMed article metadata via NCBI E-utilities APIs.

9|Updated Apr 23, 2026
Search Tags:#ncbi#pubmed#data-extraction#esearch#esummary#efetch#eutils
One-click install
npx skills add https://github.com/yangchuansheng/browser-harness-rust --skill pubmed-ncbi-scraping-data-extraction
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: PubMed / NCBI — Scraping & Data Extraction
Source: https://github.com/yangchuansheng/browser-harness-rust/tree/main/domains/pubmed
Command: npx skills add https://github.com/yangchuansheng/browser-harness-rust --skill pubmed-ncbi-scraping-data-extraction

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill enables extraction of PubMed article metadata directly from the NCBI E-utilities APIs, eliminating brittle browser scraping and manual collection.

Core Features & Use Cases

  • API-driven data collection: perform structured queries with ESearch, fetch summaries with ESummary, and obtain full records with EFetch.
  • Batch processing for literature reviews: retrieve PMIDs in bulk, resolve DOIs and PMC IDs, and assemble a compact dataset for dashboards or meta-analyses.
  • Use Case: researchers can build a reproducible workflow to collect title, authors, journal, date, and DOI for thousands of PubMed articles.

Quick Start

Construct a topic query with ESearch, then pull metadata via ESummary/EFetch to assemble a PubMed dataset.

Frequently Asked Questions about PubMed / NCBI — Scraping & Data Extraction

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract PubMed article metadata in bulk without web scraping?▼

You can extract PubMed article metadata in bulk by using the NCBI E-utilities APIs, such as ESearch and EFetch, to retrieve structured JSON or XML payloads containing titles, authors, dates, journals, and DOIs.

What's the best way to handle NCBI E-utilities rate limits when fetching large result sets?▼

Handling NCBI E-utilities rate limits requires paging through large result sets systematically during batch queries, ensuring your API-driven retrieval workflow stays within request thresholds while extracting metadata.

Can I retrieve DOIs and PMC IDs using ESearch and ESummary?▼

Yes, you can retrieve DOIs and PMC IDs by using ESearch to find PMIDs and ESummary or EFetch to pull the corresponding full records, assembling a compact dataset for literature reviews or dashboards.

Does this API-driven approach work for systematic literature reviews and meta-analyses?▼

Yes, this approach works for systematic literature reviews by performing structured queries with ESearch and fetching summaries with ESummary, enabling reproducible workflows to collect metadata for thousands of PubMed articles.

How do I query PubMed articles by topic and generate a dataset for a research dashboard?▼

Query PubMed articles by constructing a topic query with ESearch, then pull metadata via ESummary or EFetch to assemble a comprehensive dataset suitable for research dashboards and meta-analyses.

Why should I use NCBI E-utilities instead of browser scraping for PubMed data collection?▼

Using NCBI E-utilities instead of browser scraping eliminates brittle HTML parsing, providing stable API-driven data collection with structured JSON or XML payloads for reliable literature data extraction.

Related Skills

Explore More Education & Research Skills→
gexijingexijin
1

literature-search

Search PubMed via E-utilities and return titles, authors, abstracts, and PMIDs.

Community
Intermediate
Victory-HugoVictory-Hugo
1

gene-database

Retrieve NCBI gene metadata by symbol or ID via E-utilities and Datasets API.

Community
Intermediate
AlterLab-IEUAlterLab-IEU
58

alterlab-pubmed

Search PubMed via NCBI E-utilities and retrieve XML/JSON results.

Community
Advanced
cookjohncookjohn
19

pm-paper-detail

Extract structured metadata from PubMed articles by PMID.

Community
Intermediate