dagster-assets

What problem does it solve?

Dagster Software-Defined Assets (SDA) solve the problem of organizing data pipeline logic as explicit, dependency-tracked, materialized data objects instead of ad-hoc scripts and opaque job steps.

Core Features & Use Cases

• Asset-based pipeline structure: define persistent data objects (tables/files/models) with clear lineage between upstream and downstream assets.
• Partitioned materialization: implement daily, static, multi-dimensional, and dynamic partitions so you recompute only what’s needed.
• Automation and triggering: use sensors and schedules, plus declarative automation with AutomationCondition (eager/on_cron/on_missing) to materialize assets reliably.
• Custom IO management: control how asset outputs are stored/loaded (for example, Parquet on S3) and capture output metadata.
• Multi-asset outputs: produce multiple dependent assets from a single operation using @multi_asset.
• Resources and deployment wiring: inject external connections via ConfigurableResource and structure Definitions for deployment.

Quick Start

Activate this skill by telling your AI to generate a Dagster project that defines @asset and partitioned assets, wires a custom S3 Parquet IO manager, and adds sensors/schedules for materialization.