message_queues

Implement asynchronous messaging patterns using Kafka, RabbitMQ, NATS, Redis Streams, Celery, BullMQ, and Temporal.

Updated Jan 14, 2026
One-click install
npx skills add https://github.com/jvsandhu/agentic-skills --skill message-queues-jvsandhu
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: message_queues
Source: https://github.com/jvsandhu/agentic-skills/tree/main/skills/message_queues
Command: npx skills add https://github.com/jvsandhu/agentic-skills --skill message-queues-jvsandhu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires confluent-kafka, celery, temporalio, aio-pika, redis, bullmq, kafkajs, and includes scripts (resource) and references (resource) components.

What problem does it solve? Long-running operations block HTTP requests, tightly coupled services fail together, and background jobs get lost without retries. This Skill provides decision frameworks and working patterns for building event-driven architectures, background job processing, and service decoupling with message brokers. ## Core Features & Use Cases - Broker Selection Guidance: Decision tree comparing Kafka (event streaming), RabbitMQ (complex routing), NATS (request-reply), Redis Streams (simple queues), and Temporal (workflow orchestration) with throughput and latency benchmarks. - Production Patterns: Event naming conventions, schema structure, dead letter queues, idempotency for exactly-once processing, and anti-patterns to avoid. - Multi-Language Examples: Working code for Celery (Python), BullMQ (TypeScript), Kafka producers/consumers, and Temporal saga workflows, plus frontend integration via SSE for job status updates. - Use Case: Offload a 5-minute report generation from your API by enqueueing a Celery task, then stream progress updates to a React component via Server-Sent Events. ## Quick Start Ask the agent to set up a background job queue with Celery and Redis for processing image uploads asynchronously, including retry logic and a dead letter queue.

Frequently Asked Questions about message_queues

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose between Kafka, RabbitMQ, and Redis for message queues?

Choose Kafka for event streaming and replay at 500K+ msg/s, RabbitMQ for complex routing with exchanges at 50K-100K msg/s, and Redis Streams for simple queues when you already run Redis. Use NATS for request-reply patterns and Temporal for multi-step workflow orchestration.

How do I process background jobs in Python without blocking API requests?

Use Celery with Redis as the broker to enqueue tasks and return a task ID immediately from your endpoint. Configure max_retries and route permanently failed messages to a dead letter queue for manual inspection.

Kafka vs RabbitMQ for event-driven microservices?

Kafka suits event sourcing, analytics pipelines, and high-throughput streaming with long-term retention and replay. RabbitMQ fits task queues and pub/sub needing complex routing via direct, topic, fanout, or headers exchanges.

How do I prevent duplicate message processing in consumers?

Make consumers idempotent by requiring an idempotency key and checking a Redis cache before processing. Store results with a TTL so retried or duplicated messages return the cached outcome instead of executing side effects twice.

When should I not use Kafka for messaging?

Avoid Kafka for request-reply or RPC patterns since it is asynchronous and lacks response correlation. Use NATS request-reply, HTTP, or gRPC instead when you need synchronous responses with sub-millisecond latency.