skills.rest
asimihsan
asimihsanCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

telemetry-redrive

Redrive telemetry data through the telemetry-parser-service Lambda to repair corrupted Parquet output.

Updated Aug 23, 2026
Search Tags:#aws#data quality#data pipeline#telemetry#s3#lambda#redrive#sqs
One-click install
npx skills add https://github.com/asimihsan/dotfiles --skill telemetry-redrive
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: telemetry-redrive
Source: https://github.com/asimihsan/dotfiles/tree/main/chezmoi/dot_claude/skills/telemetry-redrive
Command: npx skills add https://github.com/asimihsan/dotfiles --skill telemetry-redrive

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires go.

What problem does it solve?

This Skill addresses the critical need to reprocess telemetry data due to corruption, parsing errors, or schema changes. It automates the complex task of identifying and redriving specific files or entire days of data, while preventing common pitfalls like data duplication.

Core Features & Use Cases

  • Flexible Redrive Options: Reprocess single raw JSONL files, multiple files via S3 prefix, or surgically redrive based on Parquet files or specific days.
  • Automated Parquet Deletion: Prevent data duplication by automatically deleting old Parquet files before reprocessing, ensuring data integrity.
  • Surgical Day-Based Redrive: Efficiently reprocess all data for a specific day and type, ideal for bug fixes affecting historical data.
  • SQS Queue Monitoring: Includes safeguards to monitor the SQS queue, preventing overload during large redrive operations.
  • Use Case: A bug fix in the telemetry parser requires reprocessing all lock type data for a specific day. Use the surgical day-based redrive to automatically delete old Parquet files and re-ingest the correct data, ensuring data integrity without manual intervention.

Quick Start

Perform a dry run to redrive a single telemetry file located at 's3://stage-signal-data-lake-raw/raw/json_telemetry/year=2025/month=11/day=24/hour=21/telemetry-signals-lake-firehose-1-2025-11-24-21-09-35-xxx.gz' in the platform-stage environment.

Frequently Asked Questions about telemetry-redrive

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reprocess corrupted telemetry data in AWS S3?▼

Reprocess telemetry data by redriving raw JSONL files through the telemetry-parser-service Lambda. The Skill supports single-file, prefix-based, Parquet-driven, and day-based redrives with automatic Parquet deletion to prevent duplication and repair corrupted output across dev, stage, and prod environments.

Can I redrive telemetry data for a specific day after a parser fix?▼

Yes, the surgical day-based redrive automatically deletes old Parquet files and reprocesses all data for a specific day and type. This enables targeted fixes when parser changes affect historical telemetry without manual intervention or data duplication.

What safeguards prevent data duplication during telemetry redrives?▼

Automated Parquet deletion removes old corrupted files before reprocessing begins. The Skill enforces flag-based controls, dry-run mode for validation, and SQS queue monitoring to prevent overload and ensure data integrity throughout large redrive operations.

How do I validate a telemetry redrive before processing production data?▼

Run a dry-run operation first using the Skill's dry-run flag to preview the redrive without modifying data. This allows you to test the redrive strategy and parser changes in dev or stage environments before running against production telemetry.

Does telemetry redriving work across multiple AWS environments?▼

Yes, the Skill enables safe, auditable redrives across dev, stage, and prod environments with Lambda-version coordination and Athena querying. Configuration and flag-based controls enforce environment-specific safeguards for each deployment.

Can I redrive telemetry data using S3 prefix matching?▼

Yes, prefix-based redrives allow you to reprocess multiple raw JSONL files matching an S3 prefix pattern. This enables targeted fixes across batches of telemetry files without reprocessing your entire dataset.

Related Skills

Explore More Software Engineering Skills→
anyshift-ioanyshift-io
17

sqs-queue-auditor

Audit AWS SQS queue configurations for silent message loss and redelivery risks.

Official
Advanced
starlake-aistarlake-ai
209

parquet2csv

Convert Parquet files into CSV format for batch ETL pipelines.

Official
Intermediate
itsmostafaitsmostafa
1.1k

sqs

Set up and operate AWS SQS queues with DLQ and boto3.

Community
Advanced
DoiiarXDoiiarX
20

multipar-cli

Create, verify, and repair PAR2 archives with the MultiPar command-line tool.

Community
Intermediate