clickzetta-oss-ingest-pipeline

To ingest data from object storage into a lakehouse, you can configure storage connections and external volumes, then create a PIPE to automate the data loading process.

Continuous ingestion uses LIST_PURGE and EVENT_NOTIFICATION modes for near real-time loading, while batch loading utilizes Volume and COPY or INSERT workflows for bulk imports into the lakehouse.

You set up continuous object storage ingestion pipelines by creating a PIPE configured with INGEST_MODE options like LIST_PURGE or EVENT_NOTIFICATION to enable near real-time loading from OSS, S3, or COS.

Yes, you can track load history and manage operational governance for object storage ingestion by monitoring active PIPEs, applying purge semantics, and reviewing load history records within the lakehouse environment.

Yes, object storage ingestion supports connections to OSS, S3, and COS, allowing you to configure external volumes and execute COPY or INSERT workflows to load data into the lakehouse.

You should use batch loading instead of continuous ingestion for lakehouse pipelines when performing bulk imports from object storage using Volume and COPY or INSERT workflows, rather than requiring near real-time event-driven loading.