skills.rest
expanso-io
expanso-ioOFFICIAL
Explore all skills by this author →
GitHub

expanso-enrich-export

Enrich log data with lineage and quality metadata before exporting to Amazon S3.

1|Updated Feb 1, 2026
Search Tags:#logs#data pipeline#s3#export#enrichment#log processing
One-click install
npx skills add https://github.com/expanso-io/expanso-skills --skill expanso-enrich-export
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: expanso-enrich-export
Source: https://github.com/expanso-io/expanso-skills/tree/main/skills/recipes/enrich-export
Command: npx skills add https://github.com/expanso-io/expanso-skills --skill expanso-enrich-export

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill automates the process of enriching log data with essential metadata and then exporting it to a designated storage location, streamlining log management and analysis.

Core Features & Use Cases

  • Log Enrichment: Adds lineage, processing, and quality metadata to log entries.
  • Data Restructuring: Formats logs into a structured event/metadata schema.
  • S3 Export: Efficiently exports enriched logs to Amazon S3 with partitioning and batching.
  • Error Stream: Separates and exports error logs to a dedicated S3 bucket.
  • Use Case: Automatically enrich and export application logs to S3 for long-term storage and analysis, ensuring all relevant context is preserved.

Quick Start

Run the pipeline with sample data using the provided script.

Frequently Asked Questions about expanso-enrich-export

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I enrich log data with metadata before exporting to S3?▼

To enrich log data, this Skill adds lineage, processing, and quality metadata to log entries. It restructures logs into an event and metadata schema before efficiently exporting them to Amazon S3 for long-term storage.

What is the best way to export application logs to Amazon S3 for long-term archival?▼

Exporting application logs to S3 is handled with configurable batching, compression, and storage classes. This approach ensures efficient data archival while preserving all relevant context through metadata enrichment.

Can I separate error logs into a dedicated S3 bucket during data pipeline processing?▼

Yes, you can separate error logs into a dedicated S3 bucket. The pipeline features an error stream that isolates error logs and exports them independently from standard log entries.

Does the log enrichment pipeline support configurable batching and compression for S3 exports?▼

Yes, the log enrichment pipeline supports configurable batching and compression. These features are built into the S3 export process to optimize storage efficiency and manage data archival costs.

How does log restructuring into event and metadata formats work?▼

Log restructuring formats raw log data into a structured event and metadata schema. This process attaches lineage and quality metadata directly to the events, ensuring all relevant processing context is preserved.

Related Skills

Explore More Software Engineering Skills→
bahayonghangbahayonghang
3

kb-ingest

Register and normalize new or changed sources into raw/_manifest.yaml.

Community
Intermediate
david-2814david-2814
3

athena

Run SQL queries against S3 data using Amazon Athena.

Community
Advanced
timequitytimequity
6

pipeline-design

Design ETL/ELT pipeline architectures for batch and streaming data integration.

Official
Advanced
incidentfoxincidentfox
654

elasticsearch-analysis

Analyze Elasticsearch/OpenSearch logs to identify incident root causes.

Official
Intermediate