migrating-to-checkpoints

Name: migrating-to-checkpoints
Availability: InStock
Author: Nango

Migrate Nango TypeScript createSync implementations to checkpoint-based progress models.

45|32|Updated Sep 23, 2024

Search Tags:#typescript #migration #checkpoint #sync #nango #batchsave #savecheckpoint

Ready to run

npx skills add https://github.com/NangoHQ/integration-templates --skill migrating-to-checkpoints

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: migrating-to-checkpoints
Source: https://github.com/NangoHQ/integration-templates/tree/main/.agents/skills/migrating-to-checkpoints
Command: npx skills add https://github.com/NangoHQ/integration-templates --skill migrating-to-checkpoints

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Migrates existing Nango TypeScript createSync implementations from using lastSyncDate, legacy incremental syncType, and non-resumable full refreshes to a checkpoint-based progress model, enabling durable progress across restarts.

Core Features & Use Cases

Migration of legacy syncs to checkpointed workflows with defined checkpoint schemas.
Enforcement of saveCheckpoint after every batchSave (and within paginate loops) to ensure progress is durable.
Guidance for zero YAML TypeScript syncs and handling of deletions in checkpointed incremental or full refresh scenarios.
Validation via dry runs using --checkpoint to simulate real progress windows.

Quick Start

Migrate your TypeScript createSync implementations to checkpoint-based progress by defining a checkpoint schema and updating exec loops to call getCheckpoint, batchSave, and saveCheckpoint after every page.

Frequently Asked Questions about migrating-to-checkpoints

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I migrate Nango TypeScript syncs from lastSyncDate to checkpoint-based progress?▼

To migrate Nango TypeScript syncs to checkpoint-based progress, define a flat checkpoint schema and update execution loops to call getCheckpoint, batchSave, and saveCheckpoint after every page. This replaces legacy lastSyncDate usage with durable, restartable progress tracking.

When do I need to use saveCheckpoint in a paginate loop?▼

You need to call saveCheckpoint inside paginate loops after every batchSave operation to ensure sync progress is durable. This checkpoint-based progress model guarantees that if a sync is interrupted, it resumes exactly where the last checkpoint was saved.

What is the correct way to handle clearCheckpoint during a full refresh?▼

The clearCheckpoint method should only be called after a full-refresh sync completes entirely. Using clearCheckpoint prematurely during a checkpointed full refresh will delete the progress state and prevent the Nango sync from resuming correctly if interrupted.

How do I test Nango syncs using checkpoint dry runs?▼

You test Nango syncs using checkpoint dry runs by executing the sync with the --checkpoint flag. This simulates real progress windows and validates that your getCheckpoint and saveCheckpoint calls are correctly structured across batchSave operations.

Can I use checkpoint schemas with zero YAML TypeScript syncs in Nango?▼

Yes, checkpoint schemas work with zero YAML TypeScript syncs in Nango. You define the flat checkpoint object structure directly in your createSync implementation, then manage it using getCheckpoint and saveCheckpoint throughout the execution loop.

Why does my checkpointed incremental sync fail to handle deletions correctly?▼

Checkpointed incremental syncs fail to handle deletions correctly if legacy deletion logic is not adjusted for the new progress model. You must update deletion handling to align with checkpoint boundaries so that records removed between sync runs are processed accurately.