firecrawl-research

Automating web scraping research for academic writing involves querying APIs like FireCrawl to search sources, scrape full content into markdown files, and generate BibTeX bibliographies. This process aggregates scraped sources into ready-to-use academic references.

Extracting research topics from markdown notes requires parsing headers and specific tags like [research] or [search]. The Skill then queries the FireCrawl /v1/search endpoint to find and scrape web sources for each extracted topic automatically.

Yes, you need a FireCrawl API key configured in a .env file to generate markdown research reports. The Skill relies on this key to authenticate requests to the FireCrawl /v1/search endpoint for scraping and aggregating web content.

The rate limit for scraping multiple research topics using FireCrawl is 5 results per topic with a mandatory 12-second delay between topics. This constraint prevents API overload during automated web scraping and content aggregation workflows.

Yes, you can use Pandoc and MyST templates for scholarly writing workflows with scraped markdown content. The Skill provides these academic templates to streamline the conversion of topic-specific markdown reports into formatted scholarly documents.