effect-patterns-streams

Provide Effect-TS streaming patterns for backpressure, grouping, and error handling.

785|26|Updated Jun 22, 2025
One-click install
npx skills add https://github.com/PaulJPhilp/EffectPatterns --skill effect-patterns-streams
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: effect-patterns-streams
Source: https://github.com/PaulJPhilp/EffectPatterns/tree/main/config/.claude-plugin/plugins/effect-patterns/skills/effect-patterns-streams
Command: npx skills add https://github.com/PaulJPhilp/EffectPatterns --skill effect-patterns-streams

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a curated set of Effect-TS streaming patterns to help developers design robust, lazy, and composable data pipelines for streams. It reduces boilerplate and guides best practices for streaming with Effect-TS.

Core Features & Use Cases

  • Backpressure and throttling to control producer/consumer rates.
  • Windowing and grouping (tumbling/sliding windows, groupBy) for per-key aggregations.
  • Stateful stream analytics via scan and fold to maintain running statistics.
  • Resource-safe streams using bracket and scope to ensure cleanup.
  • Advanced transformations (demux, chunking, custom operators) for complex pipelines.
  • Error handling (catch, retry, fallback) to keep streams resilient.

Quick Start

Start by loading the Streams skill and running a simple pipeline that creates a Stream of events and applies map/filter, then uses scan to track running totals. No external setup is required; just reference the Skill in your project and follow the step-by-step example in the Skill guide.

Frequently Asked Questions about effect-patterns-streams

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I implement backpressure in Effect-TS streams to control producer and consumer rates?

To implement backpressure in Effect-TS streams, you apply throttling patterns that control the rates of producers and consumers. This prevents memory overload by ensuring consumers process data at a sustainable pace.

What is the best way to handle errors in Effect-TS streaming pipelines?

The best way to handle errors in Effect-TS streaming pipelines is using catch, retry, and fallback patterns. These mechanisms keep your data streams resilient by automatically recovering from transient failures during processing.

How do I perform per-key aggregations and windowing with Effect-TS streams?

You perform per-key aggregations and windowing in Effect-TS streams using tumbling and sliding window patterns alongside groupBy operators. These transformations enable targeted stateful analytics over specific time intervals.

Can I maintain running statistics in Effect-TS streams using scan and fold?

Yes, you can maintain running statistics in Effect-TS streams by applying the scan and fold operators. These stateful stream analytics patterns accumulate values continuously, tracking running totals throughout the data pipeline.

How do I ensure resource cleanup in Effect-TS streams when processing fails?

You ensure resource cleanup in Effect-TS streams by using bracket and scope patterns. These constructs guarantee that resources are safely released and cleaned up, even if the stream processing fails or terminates unexpectedly.

Does Effect-TS support custom operators and chunking for complex stream transformations?

Yes, Effect-TS supports custom operators and chunking for complex stream transformations. You can demux streams and build advanced pipeline operators to handle intricate data routing and processing requirements.