skills.rest
rilldata
rilldataOFFICIAL
Explore all skills by this author →
Website𝕏@rilldataGitHub

rill-model

Develop Rill data models with ETL logic, transformations, and connector outputs.

47|9|Updated May 17, 2023
Search Tags:#data pipeline#data modeling#sql#data transformation#etl#rill
One-click install
npx skills add https://github.com/rilldata/rill-examples --skill rill-model
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: rill-model
Source: https://github.com/rilldata/rill-examples/tree/main/medicaid-provider-spending/.claude/skills/rill-model
Command: npx skills add https://github.com/rilldata/rill-examples --skill rill-model

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill provides comprehensive guidance on creating, configuring, and optimizing data models within the Rill data platform, enabling efficient data transformation and ingestion.

Core Features & Use Cases

  • Model Definition: Understand the structure and properties for defining Rill models, including SQL queries, materialization, and incremental loading.
  • Data Partitioning: Learn how to implement glob-based and SQL-based partitioning for scalable data ingestion.
  • Incremental Ingestion: Configure state-based and partition-based incremental strategies for efficient data updates.
  • Development Best Practices: Utilize dev partitions and understand performance considerations for faster iteration.
  • Use Case: A data engineer needs to build a daily aggregated sales report from raw transaction data. They would use this Skill to define a Rill model that incrementally loads new sales data, joins it with product information, and materializes it into a performant table for dashboard consumption.

Quick Start

Use the rill-model skill to learn how to define a materialized model that reads from a BigQuery table and outputs to ClickHouse.

Frequently Asked Questions about rill-model

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I configure incremental data ingestion in Rill models?▼

Rill models support incremental data ingestion through state-based and partition-based strategies. This enables efficient data updates by processing only new or changed records rather than reloading entire datasets during scheduled refreshes.

What is the best way to partition large datasets for transformation in Rill?▼

Partitioning datasets in Rill is achieved using glob-based or SQL-based partitioning methods. These techniques enable scalable data ingestion and can be optimized during development using specific dev partitions for faster iteration.

Does Rill data modeling support outputting transformed data to ClickHouse and DuckDB?▼

Rill data modeling supports outputting transformed data to ClickHouse and DuckDB. The Skill provides dialect-specific notes and guidance on staging connectors to ensure proper materialization into these target databases.

How do I schedule refreshes and configure retries for a Rill data pipeline?▼

Scheduled refreshes and retry configurations are defined directly within Rill model properties. Setting these parameters automates data transformation pipelines and ensures resilient execution against transient processing failures.

Can I reference other models when defining ETL logic in Rill?▼

You can reference other models when defining ETL logic in Rill. This allows building layered data transformation pipelines by chaining multiple models together before final materialization to staging connectors.

Related Skills

Explore More Data & Analytics Skills→
ContrastingSoundsContrastingSounds

rill-model

Define and deploy Rill data models with SQL, partitioning, and output configurations.

Community
Advanced
flag3flag3
1

clickhouse-io

Design and optimize ClickHouse analytics workloads with table engines, partitioning, and materialized views.

Community
Intermediate
ContrastingSoundsContrastingSounds

rill-metrics-view

Define Rill metrics_view YAML resources with schema and frontmatter requirements.

Community
Advanced
microwindmicrowind
65

data-pipeline

Ingest, transform, and load data from multiple sources into warehouses.

Official
Advanced