data-engineer

Validate market data ingestion pipelines, calendars, and corporate actions for PIT correctness.

Updated Jun 24, 2025
One-click install
npx skills add https://github.com/gtnix/quant_b3_backtest --skill data-engineer-gtnix
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: data-engineer
Source: https://github.com/gtnix/quant_b3_backtest/tree/main/.cursor/skills/data-engineer
Command: npx skills add https://github.com/gtnix/quant_b3_backtest --skill data-engineer-gtnix

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Market data feeds are error-prone and require strict quality controls across ingestion, calendars, and survivorship-sensitive universes.

Core Features & Use Cases

  • Data quality gates and sanity checks for OHLCV and dividends
  • Corporate actions ledger with adjustments and anti-double-count policies
  • Point-in-time universe validation to avoid survivorship bias
  • Trading calendar handling and gap analysis across markets

Quick Start

Run an end-to-end data-quality validation for a given date range to verify PIT correctness.

Frequently Asked Questions about data-engineer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate point-in-time market data to avoid survivorship bias in backtests?

Validate point-in-time market data by enforcing universe correctness and timeline gap analysis to prevent survivorship bias in backtests. This skill applies strict quality gates to ensure auditable, snapshot-based data integrity across historical simulations.

What is the best way to handle corporate actions and OHLCV data quality checks?

The best way to handle corporate actions and OHLCV data quality is through a validation ledger that applies explicit dividend and split adjustments. This prevents double-counting and ensures price-volume integrity across ingestion pipelines.

Can I normalize trading calendars and timezones across B3 and US market feeds?

Yes, you can normalize trading calendars and timezones across B3 and US market feeds. The skill performs gap analysis and calendar validation to align market sessions, ensuring consistent timeline data across different regional exchanges.

Why does my market data pipeline have timeline gaps and point-in-time versioning errors?

Timeline gaps and point-in-time versioning errors occur when ingestion pipelines lack strict data quality gates. Implementing PIT versioning with snapshot IDs and checksums ensures auditable, survivorship-free data by validating timeline continuity.

Does this data quality validation support point-in-time universe checks for backtesting?

Yes, this data quality validation supports point-in-time universe checks specifically for backtesting. It enforces strict data quality gates and validates PIT versioning with snapshot IDs and checksums to guarantee survivorship-free historical simulations.

What are the limitations of running end-to-end OHLCV integrity checks across multiple markets?

Limitations of running OHLCV integrity checks across markets include requiring explicit timezone normalization and accurate corporate actions ledgers. Without proper calendar handling and anti-double-count policies, multi-market validation may produce misaligned timelines.