portal-export-jobs

Implement non-blocking background export jobs with FastAPI, Polars, and GCS signed URLs.

Updated Jul 27, 2026
One-click install
npx skills add https://github.com/ArthurZizumbo/karisma-data --skill portal-export-jobs-arthurzizumbo
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: portal-export-jobs
Source: https://github.com/ArthurZizumbo/karisma-data/tree/main/.claude/skills/portal-export-jobs
Command: npx skills add https://github.com/ArthurZizumbo/karisma-data --skill portal-export-jobs-arthurzizumbo

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires fastapi, polars, google-cloud-storage.

What problem does it solve? Large data exports block API requests and degrade the user experience. This Skill implements an asynchronous export pipeline where POST /api/export returns a job_id immediately while heavy Polars joins and serialization run in BackgroundTasks, keeping the rest of the API responsive. ## Core Features & Use Cases - Non-blocking export endpoint: POST /api/export validates the request, registers the job, and returns 202 with a job_id instantly; the worker runs in FastAPI BackgroundTasks. - Job state tracking: Jobs move through queued, running, done, and failed states persisted in the export_job table, with audit fields for user, filters, size, and duration. - GCS delivery with signed URLs: Finished CSV/XLSX files upload to GCS under exports/{user}/{job_id} with 24-hour signed URLs and a 7-day bucket lifecycle. - Use Case: An analyst requests a 1-million-row export as XLSX; the API responds immediately, the worker builds the file via the semantic layer, and the analyst polls GET /api/export/{job_id} until a signed download URL appears. ## Quick Start Ask the AI to implement the export endpoint and background worker so that POST /api/export returns a job_id immediately and the finished file is uploaded to GCS with a signed URL.

Frequently Asked Questions about portal-export-jobs

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run background tasks in FastAPI without blocking the request?

Use FastAPI's BackgroundTasks parameter: register the job in the endpoint, add the worker function with background_tasks.add_task, and return a 202 response with the job_id immediately. The heavy work executes after the response is sent.

How to generate signed URLs for Google Cloud Storage files?

Upload the file with blob.upload_from_filename, then call blob.generate_signed_url with version v4 and an expiration timedelta, such as 24 hours. The bucket lifecycle rule, for example 7 days, controls when objects are deleted.

Can Polars write Excel XLSX files directly?

Yes, Polars supports df.write_excel for XLSX output alongside df.write_csv for CSV. For large exports, collect the lazy frame in a thread with asyncio.to_thread so the event loop stays responsive.

How do I prevent users from accessing other users' export jobs?

Check ownership in the status endpoint: compare job.user_id with the current user's id and allow access only to the owner or users with the admin role, returning 403 otherwise. Store user_id on every job row for auditing.

Why does my API slow down during large data exports?

Blocking work like large joins or file serialization on the async event loop starves other requests. Move heavy computation to BackgroundTasks and offload CPU-bound steps with asyncio.to_thread so endpoints stay under 500 ms during exports.