effect-patterns-streams-sinks

Compose Effect-TS streaming sinks for batching, persisting, and publishing data.

785|26|Updated Jun 22, 2025
One-click install
npx skills add https://github.com/PaulJPhilp/EffectPatterns --skill effect-patterns-streams-sinks
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: effect-patterns-streams-sinks
Source: https://github.com/PaulJPhilp/EffectPatterns/tree/main/config/.claude-plugin/plugins/effect-patterns/skills/effect-patterns-streams-sinks
Command: npx skills add https://github.com/PaulJPhilp/EffectPatterns --skill effect-patterns-streams-sinks

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides battle-tested sink patterns for streaming data in Effect-TS, enabling reliable batching, persistence, and distribution of stream records with resilience against failures.

Core Features & Use Cases

  • Batch Insert Stream Records into Database: batch records to reduce DB round-trips and improve throughput.
  • Write Stream Events to an Event Log: append events with metadata for auditability and replay.
  • Sink Pattern 3: Write Stream Lines to File: buffered file writes for logs and exports.
  • Sink Pattern 4: Send Stream Records to Message Queue: batch and publish with partitioning by key.
  • Sink Pattern 5: Fall Back to Alternative Sink on Failure: progressive degradation across cache, DB, and dead-letter targets.
  • Sink Pattern 6: Retry Failed Stream Operations: exponential backoff retries with jitter.

Quick Start

Run the streams-sinks sample to execute all six sink patterns against a simulated stream of records and observe batching, publishing, and retry behaviors.

Frequently Asked Questions about effect-patterns-streams-sinks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I batch stream records into database writes using Effect-TS?

Batch stream records into database writes using Effect-TS by applying sink patterns that group records, reducing DB round-trips and improving overall pipeline throughput.

What is the best way to handle stream failure with fallback sinks in Effect-TS?

Handle stream failure with fallback sinks in Effect-TS by implementing progressive degradation across cache, database, and dead-letter targets to ensure data persistence during outages.

How do I implement retry strategies with exponential backoff for Effect-TS streams?

Implement retry strategies for Effect-TS streams by applying sink patterns with exponential backoff and jitter to handle transient failures robustly during data processing.

Can I write streaming events to an event log and publish to message queues using Effect-TS?

Write streaming events to logs and publish to message queues using Effect-TS sink patterns that append metadata for auditability and batch records with partitioning by key.

Does Effect-TS support backpressure handling for streaming data pipelines?

Effect-TS supports backpressure handling for streaming data pipelines by composing robust sinks with built-in error handling to manage flow control during batch processing.

When should I use Effect-TS sink patterns for streaming data instead of simpler approaches?

Use Effect-TS sink patterns for streaming data when pipelines require deterministic batching, resilient error handling, fallback degradation, and reliable distribution across multiple targets.