skills.rest
starlake-ai
starlake-aiOFFICIAL
Explore all skills by this author →
GitHub

ingest

Ingest data from diverse sources into a data lake or warehouse.

209|30|Updated Sep 18, 2021
Search Tags:#etl#data-pipeline#batch-processing#ingest#data-ingestion#starlake
One-click install
npx skills add https://github.com/starlake-ai/starlake --skill ingest
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ingest
Source: https://github.com/starlake-ai/starlake/tree/main/.agent/skills/ingest
Command: npx skills add https://github.com/starlake-ai/starlake --skill ingest

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Ingest data from diverse sources into your data lake or data warehouse, simplifying loading for analytics and reporting.

Core Features & Use Cases

  • Ingest from multiple sources with domain, schema, and paths configuration.
  • Schedule and parameterize loads to support repeatable pipelines for analytics, migrations, and incremental updates.
  • Use cases include batch data loads, data migration, and incremental ingestion into your warehouse.

Quick Start

Ingest data from a specified path into the target domain and schema with a single command.

Frequently Asked Questions about ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I ingest data from multiple sources into a data warehouse?▼

Batch data ingestion from diverse sources into a data warehouse or data lake is configured using domain, schema, and paths settings to simplify loading for analytics and reporting pipelines.

Can I schedule repeatable batch data loads for incremental updates?▼

Yes, batch ingestion workflows support load scheduling and parameterization, enabling repeatable data pipelines for incremental ingestion, reporting, and data migrations into your warehouse.

What is the best way to load data into a data lake for analytics pipelines?▼

Loading data into a data lake for analytics is handled through batch ingestion workflows that satisfy domain, schema, paths, and scheduling configuration requirements in a single command.

Does this data ingestion method support data migration workflows?▼

Data migration is a supported use case, utilizing parameterized loads and batch ingestion to effectively transfer data from specified paths into target domains and schemas.

Do I need to configure schema and domain settings for batch data ingestion?▼

Yes, specifying the target domain, schema, and paths is required to define how diverse source data is structured and loaded into your data lake or warehouse during batch processing.

Related Skills

Explore More Data & Analytics Skills→
musujmusuj

phase-1-schema

Define project terminology, entities, relationships, and data structures.

Community
Intermediate
openyakopenyak
704

analyze

Analyze data questions with SQL or natural-language prompts and present validated results.

Official
Advanced
as4584as4584
2

airflow-dag-patterns

Generate Apache Airflow DAG patterns with task dependencies and testing examples.

Community
Advanced
QuestNova502QuestNova502
4

senior-data-engineer

Design and operate production-grade ETL/ELT data pipelines with Python, Spark, and Airflow.

Community
Advanced