polymer-pay-x402endpoints-firecrawl

Pay-per-use web scraping routes your requests through a proxy that charges per call, letting you scrape pages without a dedicated account. You POST your target URL to the proxy endpoint to execute scraping operations on demand.

Yes, you can scrape JavaScript-rendered pages with per-call pricing. The proxy handles JavaScript rendering and anti-bot measures automatically when you send your scrape request.

Yes, you need a Polymer Pay API key to extract structured data. You must sign up for a key and include it in your POST requests to the proxy URL to authenticate your scraping and extraction operations.

To map a website, send a request to the /v1/map endpoint via the proxy. This discovers all available URLs on the target site, which you can then pass to the /v1/crawl endpoint to crawl pages asynchronously.

Asynchronous crawling requires a mandatory limit on the number of pages and charges per page crawled. You must specify this limit upfront when calling /v1/crawl to control costs and scope.

The best way to get on-demand web data without a dedicated scraping account is using a pay-per-use proxy. It handles site mapping, crawling, and structured data extraction through per-call payments via a single API key.