stream-processing

Process unbounded event streams with deterministic results and bounded state.

Updated Jun 5, 2026

Search Tags:#stream processing #exactly-once #real-time analytics #stateful processing #event stream

Ready to run

npx skills add https://github.com/hung-phan/system-skills --skill stream-processing-hung-phan

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: stream-processing
Source: https://github.com/hung-phan/system-skills/tree/main/skills/system-review/references/data-systems/stream-processing
Command: npx skills add https://github.com/hung-phan/system-skills --skill stream-processing-hung-phan

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of processing and analyzing unbounded sequences of events in real-time, ensuring accurate, timely insights without overwhelming state or resource constraints.

Core Features & Use Cases

Real-time Aggregation: Compute continuous aggregates over event streams.
Stateful Processing: Manage stateful operations such as joins and deduplication efficiently.
Exactly-once Semantics: Ensure end-to-end data consistency across sources, processing, and sinks.
Use Case: Use this Skill to analyze user behavior in real-time, detecting anomalies or tracking user sessions.

Quick Start

Use the stream-processing skill to analyze the user activity stream from the 'user-activity.log' file.

Frequently Asked Questions about stream-processing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I process unbounded event streams with exactly-once semantics?▼

To process unbounded event streams with exactly-once semantics, you need a system that manages bounded state and utilizes idempotent sinks to guarantee deterministic results without data duplication or loss.

What is stateful processing in real-time analytics?▼

Stateful processing in real-time analytics involves maintaining application state, such as user sessions or deduplication buffers, to compute continuous aggregates over unbounded event streams efficiently.

How do I handle event time windowing for real-time aggregation?▼

Handling event time windowing for real-time aggregation involves grouping unbounded event streams into logical time windows, allowing the system to process continuous aggregates with low latency despite out-of-order data arrival.

Can I use stream processing for anomaly detection in user activity logs?▼

Yes, stream processing is designed to analyze user activity logs in real-time, tracking user sessions and detecting anomalies by applying stateful processing and continuous aggregation to the event stream.

Why do I need idempotent sinks for exactly-once processing?▼

Idempotent sinks are required for exactly-once processing to ensure that replaying events during failure recovery does not duplicate output, thereby maintaining end-to-end data consistency across sources and sinks.

What is the best way to track user sessions in an event stream?▼

The best way to track user sessions in an event stream is through stateful processing, which manages bounded state to deduplicate events and compute continuous aggregates with low latency for real-time analytics.