push-ingestion

You can automate push ingestion for Snowflake by generating ready-to-run scripts using the canonical pycarlo API, which handles metadata, lineage, and query-log pushes with batching and memory-safety checks.

Monte Carlo push ingestion is the process of pushing metadata, lineage, and query logs from warehouses like BigQuery, Databricks, and Redshift to Monte Carlo via the Ingestion API. You need it to ensure data observability across your pipelines.

Yes, push ingestion works with Databricks, Redshift, Snowflake, BigQuery, and Hive. The generated scripts provide template-driven workflows, with guidance for adapters when a specific warehouse template is missing.

You validate a push ingestion script by capturing the invocation_id returned by the Ingestion API and tracing it via GraphQL queries to ensure push visibility and successful metadata delivery in Monte Carlo.

Yes, pycarlo is required as the dependency for generating push ingestion workflows. The templates ensure scripts follow the canonical pycarlo API, including required environment variables and memory-safety checks.

Monte Carlo push ingestion scripts require specific environment variables for authentication and MC resource configuration. The generated templates ensure these required environment variables are properly set before execution.