backfilling-atproto

Backfill ATProto records by querying the PDS directly to enumerate and fetch historical data across custom collections, bypassing public API limitations. Direct PDS access successfully retrieves records from collections like network.comind.* that return 404 publicly.

The public API returns 404 for custom ATProto collections because it lacks coverage for non-standard namespaces. Direct PDS access works for all collections, allowing you to enumerate and backfill records across custom schemas that the public API cannot resolve.

The best way to sync historical activity is running a PDS-driven backfill process that enumerates records across multiple custom collections. It enforces pagination handling with cursors and idempotent processing to ensure complete, recoverable data synchronization without duplicate work.

Backfilling ATProto records handles pagination using cursors and enforces idempotent processing to prevent duplicate work. This ensures complete, recoverable backfills across custom collections while safely managing paging during data enumeration from the PDS.

Yes, you can scope PDS backfills to specific ATProto collections. The backfill process enforces collection scoping, allowing you to target and enumerate records within defined namespaces like stream.thought.* while ignoring irrelevant data during historical population.