tushare-duckdb-sync

To sync Tushare data to DuckDB incrementally, run the end-to-end ETL workflow that supports both full and incremental modes. It automatically logs sync states and updates DuckDB tables across multiple configured data sources.

The best way to automate Tushare ETL with data quality checks is using a structured pipeline that ingests data into DuckDB and automatically generates data quality snapshots, sync state logs, and metadata docs for governance.

Yes, you need pandas and loguru installed, along with tushare and duckdb. These dependencies are required to execute the script-based workflow, manage data frames, and handle logging during synchronization.

Yes, you can generate metadata documentation for DuckDB tables synced from Tushare by running the end-to-end workflow, which automatically produces schema metadata, provenance documentation, and data quality reports.

Yes, the Tushare to DuckDB pipeline supports cross-session sync state tracking by utilizing a mapping registry. This allows structured ETL pipelines to maintain synchronization logs and configurations across multiple sessions.