taskiq

Run asynchronous distributed background tasks and scheduled jobs in Python.

Updated Apr 21, 2023
One-click install
npx skills add https://github.com/tuliosouza99/CartolaPy --skill taskiq
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: taskiq
Source: https://github.com/tuliosouza99/CartolaPy/tree/main/.agents/skills/taskiq
Command: npx skills add https://github.com/tuliosouza99/CartolaPy --skill taskiq

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Taskiq removes the complexity of running asynchronous, distributed background work in Python applications by providing brokers, workers, scheduling, and dependency injection so developers can offload long-running jobs and schedule recurring tasks reliably.

Core Features & Use Cases

  • Brokers & Result Backends: Support for InMemory, RabbitMQ (aio-pika), NATS, and Redis-based queues plus pluggable result backends for storing task results.
  • Worker & Scheduler Management: Run worker processes, file-system discovery, reloadable workers, label and Redis-based scheduling, and lifecycle-aware startup/shutdown for frameworks like FastAPI.
  • Dependency Injection & Testing: Resolve FastAPI-style dependencies inside tasks, use TaskiqDepends for injection, and use InMemoryBroker patterns for deterministic testing and awaiting tasks.
  • Use Case: Offload email sending, data processing, or database writes from web requests, schedule periodic aggregation jobs, and run background pipelines across multiple worker nodes.

Quick Start

Set up a broker, declare an async task with @broker.task, start the broker in your app lifespan, and run a taskiq worker process to process queued jobs.

Frequently Asked Questions about taskiq

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run asynchronous background tasks in a FastAPI application?

Running asynchronous background tasks in a FastAPI application involves setting up a broker, declaring tasks with a decorator, and starting a worker process. Taskiq integrates with FastAPI lifespans to manage worker startup and shutdown while offloading jobs like email sending or data processing.

What is the best way to schedule recurring cron-like jobs in Python asyncio services?

Scheduling recurring cron-like jobs in Python asyncio services is supported through label and Redis-based scheduling mechanisms. This allows you to run periodic aggregation jobs and background pipelines across multiple worker nodes reliably without blocking the main event loop.

Can I use dependency injection for task parameters in a Python task queue?

Yes, you can use dependency injection for task parameters in a Python task queue. Taskiq supports resolving FastAPI-style dependencies inside tasks using TaskiqDepends, allowing you to inject required services and parameters directly into your distributed background job functions.

Does Taskiq support RabbitMQ and NATS brokers for distributed task queues?

Taskiq supports RabbitMQ and NATS brokers for distributed task queues, alongside Redis and InMemory options. These brokers enable asynchronous distributed background processing, allowing you to scale workers across multiple nodes for long-running jobs.

How do I test async background tasks deterministically?

Testing async background tasks deterministically uses an InMemoryBroker pattern to execute jobs without external dependencies. This approach allows you to await tasks directly during test runs, ensuring predictable and reliable verification of your background processing logic.

What are the limitations of using InMemory brokers for background task queues?

InMemory brokers for background task queues are limited to single-process execution and do not support distributed scaling across multiple worker nodes. They are designed for testing and local development rather than production workloads, where RabbitMQ or Redis brokers are required for reliability.