pipelines-as-code

Author and validate Grafana Alloy log and metric pipelines as typed YAML with JSONSchema enforcement.

3|Updated May 1, 2026
One-click install
npx skills add https://github.com/MaterializeInc/materialize-monitoring --skill pipelines-as-code-materializeinc
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: pipelines-as-code
Source: https://github.com/MaterializeInc/materialize-monitoring/tree/main/.claude/skills/pipelines-as-code
Command: npx skills add https://github.com/MaterializeInc/materialize-monitoring --skill pipelines-as-code-materializeinc

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Writing Grafana Alloy pipeline configuration by hand is error-prone: undocumented attributes slip through, ref-valued fields get quoted as strings, and alloy validate misses type mismatches that only fail at load time. This Skill provides the conventions, schema map, and invariants for Materialize's pipelines-as-code system, where YAML pipeline definitions are validated against embedded JSONSchemas and rendered to canonical .alloy config via typed Rust structs. ## Core Features & Use Cases - Typed pipeline authoring: Write log-processing (loki.process stages, discovery.kubernetes, relabel rules) and metric-processing (prometheus.*, otelcol.*) pipelines in YAML under packages/alloy-pipelines/, validated by schemas in packages/mzmon-lib/schemas/alloy/. - Schema extension guidance: Add new components, stages, attributes, or typed sub-blocks by extending the JSONSchemas and the Rust ComponentBlock sugar in lockstep, with the raw: escape hatch as a deliberate fallback. - Debugging rendered output: Diagnose mz-monitoring-build gen-pipelines failures, serde/validation drift, and capsule type mismatches using documented load-bearing invariants. - Use Case: When adding a new otelcol.processor to the gateway metrics pipeline, use this Skill to extend otelcol.schema.yaml, add the typed struct in components/otelcol.rs, wire it into gateway.yaml, and verify with make pipelines and cargo test -p mzmon-lib. ## Quick Start Ask the AI to add a new typed attribute or component to the Alloy pipeline schemas and render the updated gateway pipeline with make pipelines.

Frequently Asked Questions about pipelines-as-code

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new attribute to an Alloy pipeline component schema?

Add the attribute to the component's schema file under packages/mzmon-lib/schemas/alloy/ and widen the matching Rust struct field in packages/mzmon-lib/src/alloy/components/. If the schema admits an expression via anyOf, the Rust field must be Expressable; the two drift silently otherwise.

How do I render and validate Alloy pipelines from YAML?

Run make pipelines to render the YAML under packages/alloy-pipelines/ into .alloy files and run alloy validate per target. Use cargo test -p mzmon-lib for unit and oracle tests that pipe output through alloy fmt.

When should I use the raw escape hatch instead of a typed block?

Use raw only when an attribute or component is not yet covered by the typed schemas, since strict additionalProperties rejects undocumented fields. It is the last oneOf branch of every sub-block list and should be a deliberate, temporary choice, not the default.

Why does alloy validate pass but the config fail at load time?

alloy validate cannot see capsule type mismatches, such as list-wrapping a targets ref, which produces list(list(Target)) that Alloy rejects at load. Verify changes with a real alloy run or live cluster test, not validation alone.

Does the otelcol processor support require a specific Alloy version?

Yes, otelcol filter conditions blocks require Alloy 1.17 or newer, and the repo pins 1.17.1. Every otelcol component must also have a label, which is enforced as a required field in both the schema and the Rust structs.