skills.rest
MaterializeInc
MaterializeIncOFFICIAL
Explore all skills by this author →
Website𝕏@MaterializeIncGitHub

mz-release-signoff

Verify Materialize release candidates by comparing Grafana metrics across release boundaries.

6.4k|512|Updated Feb 22, 2019
Search Tags:#metrics#grafana#prometheus#PromQL#materialize#release verification#canary analysis
One-click install
npx skills add https://github.com/MaterializeInc/materialize --skill mz-release-signoff
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mz-release-signoff
Source: https://github.com/MaterializeInc/materialize/tree/main/.agents/skills/mz-release-signoff
Command: npx skills add https://github.com/MaterializeInc/materialize --skill mz-release-signoff

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Release sign-off requires judging whether a new Materialize release candidate regressed any of hundreds of Prometheus metrics across production canary and staging fleets, and doing it by hand invites mis-scoped queries, stale baselines, and empty panels misread as healthy zeros.

Core Features & Use Cases

  • Boundary-pinned comparison: Establishes exact upgrade boundaries from version series and zero-downtime pod doubling, then samples before and after windows that never straddle a boundary.
  • Batched PromQL measurement: Scripts build one batched range query per area from a metric roster, with the correct fleet selector for production canary namespaces or staging release candidates.
  • Per-area metric references: Curated references for compute, storage, persist, adapter, and reference dashboards record metric types, label quirks, invariants, dead panels, and known noise classes.
  • Missing-metric detection: A catalog diff script compares the checked-in metrics catalog between release tags, and a lint script keeps every metric name in the skill resolving against the catalog or an allowlist.
  • Use Case: When the release bot posts "please verify v26.39.0-rc.3" in #release, run the skill to sweep each area's dashboard, rule out confounders like restart age and fleet drift, and post a verdict with method and findings in the thread.

Quick Start

Ask the assistant to verify release v26.39.0-rc.3 on the Grafana dashboards and sign off in the release thread.

Frequently Asked Questions about mz-release-signoff

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify a Materialize release candidate on Grafana?▼

Pin the upgrade boundary from the v2_mz_compute_cluster_status version series, select the canary or staging namespaces, then compare batched PromQL range queries across the boundary for each area dashboard. Report the verdict in the release bot's Slack thread.

How do I compare Prometheus metrics across a release boundary?▼

Sample the previous release, each release candidate, and the boundary separately with a 6h step, never straddling the upgrade bucket. Batch metrics into one query using label_replace tags combined with or, using rate for counters and avg_over_time for gauges.

Why does an empty Grafana panel not mean a healthy metric?▼

Panels outlive the metrics they plot, so a renamed or deleted metric renders blank and reads as a healthy zero. Confirm the metric name exists with list_prometheus_metric_names or the metrics catalog diff before reporting zero.

Does this skill work without the Slack MCP server?▼

Yes. The Grafana MCP server against grafana.dev.materialize.com is required, but a user-pasted dashboard link or version string is enough to proceed without reading the release bot's Slack request.

What confounders cause false regression signals in release verification?▼

Fleet composition drift, restart-age differences in memory, bimodal arrangement gauges, and permanently unhealthy staging environments all produce steps unrelated to the release. Calibrate against three weeks of natural spread and compare at equal post-restart age.

Related Skills

Explore More Software Engineering Skills→
RefractSystemsRefractSystems

shipping-and-launch

Generate production deployment readiness checklists with feature flag and rollback plans.

Official
Advanced
JeremyDev87JeremyDev87
31

performance-optimization

Profile CPU, memory, and I/O to prioritize and prevent performance regressions.

Community
Advanced
opensearch-projectopensearch-project
34

correlation

Correlate OpenTelemetry traces, logs, and metrics using shared fields and PPL queries.

Official
Advanced
zhu637882-stackzhu637882-stack

canary

Monitor live apps for console errors, performance regressions, and page failures.

Community
Advanced