skills.rest
MaterializeInc
MaterializeIncOFFICIAL
Explore all skills by this author →
Website𝕏@MaterializeIncGitHub

mz-test

Runs and selects test frameworks for Materialize including sqllogictest, testdrive, mzcompose, and cargo test.

6.4k|512|Updated Feb 22, 2019
Search Tags:#Testing#materialize#regression test#Flaky tests#mzcompose#testdrive#sqllogictest
One-click install
npx skills add https://github.com/MaterializeInc/materialize --skill mz-test
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mz-test
Source: https://github.com/MaterializeInc/materialize/tree/main/.agents/skills/mz-test
Command: npx skills add https://github.com/MaterializeInc/materialize --skill mz-test

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Materialize has many test frameworks (cargo test, sqllogictest, testdrive, pgtest, mzcompose compositions), each with different invocation commands, rewrite flags, and conventions. This Skill removes the guesswork of picking the right framework and running it correctly, including rewriting expected results and reproducing flaky failures.

Core Features & Use Cases

  • Framework selection guidance: Maps the type of change (SQL correctness, sources/sinks, pgwire protocol, restarts/upgrades, concurrency, limits, OOM, benchmarks) to the correct test framework and file location.
  • Correct invocation commands: Provides exact commands such as bin/sqllogictest -- PATH, bin/mzcompose --find NAME run WORKFLOW, and bin/cargo-test -p mz-environmentd pgtest, plus result rewriting via --rewrite-results or REWRITE=1.
  • Flake and regression workflows: Covers reproducing flaky tests in loops, adding log_filter instrumentation, version-guarding testdrive output, and verifying a regression test fails without the fix before passing with it.
  • Use Case: After fixing a SQL bug, ask the Skill to add a regression test; it extends an existing .slt file, runs it against the unfixed code to confirm it fails, then confirms it passes with the fix.

Quick Start

Run the sqllogictest file test/sqllogictest/aggregates.slt and rewrite its expected results if the output changed.

Frequently Asked Questions about mz-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run sqllogictest files in Materialize?▼

Run sqllogictest files with `bin/sqllogictest -- PATH` where PATH is relative to the repo root, usually under test/sqllogictest/. Batch multiple files in one invocation to amortize startup cost, and rewrite expected results with `bin/sqllogictest -- --rewrite-results PATH`.

How do I run testdrive tests with mzcompose?▼

Run testdrive files with `bin/mzcompose --find testdrive run default -- FILENAME.td`, where the file is relative to test/testdrive/. Run `bin/mzcompose --find testdrive down` between runs since tests expect fresh state.

Which test framework should I use for a Materialize bug fix?▼

Use sqllogictest for SQL correctness, testdrive for sources/sinks and external systems, pgtest for raw pgwire protocol, Rust unit tests for pure logic, and Platform Checks for restart/upgrade issues. For functional bugs, aim for at least two frameworks that independently detect the regression.

Why does sqllogictest fail with InconsistentViewOutcome?▼

CI runs many SLTs with --auto-index-selects, which wraps SELECTs in an indexed view and compares results with the one-shot query. A query that behaves differently when wrapped fails this check; exempt the file via tests_without_views or tests_no_auto_index_selects in test/sqllogictest/mzcompose.py.

How do I reproduce a flaky Materialize test locally?▼

Run the mzcompose composition in a loop with `down` before each run until it fails, and add log_filter instrumentation via additional_system_parameter_defaults in the test's mzcompose.py. If it cannot be reproduced locally, trigger 10-20 CI runs via ci.dev.materialize.com/trigger.

How do I update snapshot files in Materialize cargo tests?▼

Do not manually edit *.snap files. Run `cargo test` followed by `cargo insta accept` to update snapshots, and use `REWRITE=1 cargo test ...` to rewrite datadriven test expectations.

Related Skills

Explore More Software Engineering Skills→
RedHatProductSecurityRedHatProductSecurity
47

cargo-fuzz

Automate fuzz testing of Cargo-based Rust projects with libFuzzer.

Official
Intermediate
ethereum-optimismethereum-optimism
6.5k

run-acceptance-tests

Build dependencies and run acceptance tests locally in the Optimism monorepo.

Official
Intermediate
Teng-bioTeng-bio
2

ci-failure-triage

Diagnose CI pipeline failures and distinguish deterministic failures from flaky tests.

Community
Intermediate
ewe-studiosewe-studios
1

Rust Testing Excellence

Create Rust unit, integration, and property-based tests with proptest.

Official
Advanced