reproducible-analysis

Create and maintain a JSONL journal capturing data operations for reproducible analysis.

3.7k|104|Updated Dec 11, 2020

Search Tags:#logging #data-analysis #journal #deterministic #jsonl #reproducible

Ready to run

npx skills add https://github.com/dathere/qsv --skill reproducible-analysis

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: reproducible-analysis
Source: https://github.com/dathere/qsv/tree/main/.claude/skills/skills/reproducible-analysis
Command: npx skills add https://github.com/dathere/qsv --skill reproducible-analysis

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Reproducible analysis is hard when steps are scattered across scripts and outputs; this skill provides a unified, machine-readable journal to capture every data operation so analyses can be replayed and verified end-to-end.

Core Features & Use Cases

Automatic generation of a <analysis-name>.journal.jsonl file that records each operation as a JSON object with fields like seq, ts, op, tool, input, input_sha256, output, output_sha256, duration_ms, and note.
Supports deterministic replay across environments, enabling auditors and teammates to reproduce results given the same inputs and toolchain.
Works with common data formats (CSV, JSON, Parquet) and multiple data-processing steps (indexing, filtering, joining, statistics).

Quick Start

Create the first reproducible journal entry by running an analysis step and appending its details to a file named <analysis-name>.journal.jsonl.

Frequently Asked Questions about reproducible-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I make my data analysis reproducible across different environments?▼

To make data analysis reproducible across environments, you need a machine-readable journal that records every operation. This skill automates capturing steps like filtering and joining in a stable JSONL format, enabling deterministic replay and end-to-end verification.

What is a JSONL journal format for deterministic replay of data operations?▼

A JSONL journal format for deterministic replay is a structured log where each line is a JSON object capturing a data operation. It stores fields like seq, ts, op, input, input_sha256, and output_sha256 to verify that analyses produce identical results across different toolchains.

How do I log data operations like filtering and joining for audit verification?▼

You log data operations for audit verification by appending each step's details to an analysis journal file. This skill automatically records the operation, input, output, duration, and cryptographic hashes to a JSONL file, ensuring teammates can reproduce results.

Can I use this reproducible journal with CSV, JSON, and Parquet file formats?▼

Yes, the reproducible journal works with common data formats including CSV, JSON, and Parquet. It records multiple data-processing steps such as indexing, filtering, joining, and statistics, capturing inputs and outputs regardless of the underlying file format.

What fields are required in a reproducible analysis journal entry?▼

A reproducible analysis journal entry requires a consistent schema storing fields such as seq, ts, op, input, input_sha256, output, output_sha256, duration_ms, and note. These fields ensure every data operation is verifiable and machine-readable.

Why does my data analysis pipeline fail to produce identical outputs when replayed?▼

Data analysis pipelines fail to produce identical outputs when steps are scattered across scripts without a unified log. A machine-readable journal solves this by capturing each operation with input and output hashes, enabling deterministic replay and verification.