Design and validate YAML-based ETL pipelines for OctoMesh.
npx skills add https://github.com/meshmakers/octo-claude-skills --skill pipeline-expert
Please help me install this Agent Skill. Skill: pipeline-expert Source: https://github.com/meshmakers/octo-claude-skills/tree/main/skills/pipeline-expert Command: npx skills add https://github.com/meshmakers/octo-claude-skills --skill pipeline-expert
OctoMesh pipelines can be complex to design, read, and debug. This skill provides expert guidance for creating, reading, editing, and debugging YAML-defined ETL pipelines, with a focus on triggers, transformations, DataContext usage, and edge/mesh integration.
Write a minimal pipeline that triggers on FromHttpRequest@1, processes data with a ForEach transformation, and writes results to a target path.
High-intent search queries and answers about installing and using this skill.
To create an OctoMesh ETL pipeline triggered by HTTP requests, you write a YAML definition starting with a FromHttpRequest@1 trigger, process the payload using a ForEach transformation, and write the results to a target path.
ForEach iteration in OctoMesh YAML pipelines processes collections of data by applying transformations to each item individually, allowing you to manipulate records within the DataContext and output structured results to downstream mesh-edge targets.
The best way to debug complex YAML pipelines in OctoMesh is to validate the structure of your triggers, transformations, and DataContext operations step-by-step, ensuring proper mesh-edge integration and correct syntax in your ETL definitions.
Yes, you can use DataContext operations to manage state across transformations in OctoMesh pipelines, enabling you to store, read, and modify data variables throughout the ForEach iterations and mesh-edge integrations within your YAML definitions.
OctoMesh supports mesh-edge integration for real-world data flow scenarios by allowing YAML pipelines to route and transform data across distributed edge endpoints using configured triggers and DataContext operations.
Your OctoMesh YAML pipeline may fail to validate if it lacks required frontmatter fields like name and description, or if the optional references, scripts, and assets structure is incorrectly formatted within the self-contained skill unit.
leaderiopDefine package boundaries and API contracts for modular libraries.
astronomerTrigger Airflow DAG runs, diagnose failures, and inspect task logs.
aliciaplsDeploy Nexus workflows as a unified API, CLI, and MCP platform.
MantisWareDesign and configure Magic ETL dataflows with drag-and-drop tiles and scripted transformations.
