skills.rest
doccker
docckerCOMMUNITY
Explore all skills by this author →
GitHub

cc-streaming-export-safety

Implements streaming export patterns for large Excel, CSV, JSON, and PDF files to prevent OOM.

1.0k|109|Updated Jan 4, 2026
Search Tags:#excel#PDF#CSV#large file#async task#OOM prevention#streaming export
One-click install
npx skills add https://github.com/doccker/cc-use-exp --skill cc-streaming-export-safety
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cc-streaming-export-safety
Source: https://github.com/doccker/cc-use-exp/tree/main/.codex/skills/cc-streaming-export-safety
Command: npx skills add https://github.com/doccker/cc-use-exp --skill cc-streaming-export-safety

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Large user-driven file exports built entirely in memory cause OOM crashes, Full GC storms, leftover temp files, HTTP thread blocking, and spreadsheet formula injection. This Skill enforces streaming and paginated export patterns when data volume exceeds 10,000 rows or 10 MB, or when the volume is unpredictable.

Core Features & Use Cases

  • Streaming API guidance per format and language: Covers Apache POI SXSSF, EasyExcel, openpyxl write_only, excelize StreamWriter, ExcelJS stream, Jackson JsonGenerator, OpenCSV, ReportLab, PDFKit, and more, with correct usage and known limitations.
  • Async export task pattern: Replaces synchronous HTTP export with task submission, background execution, object storage upload, and status polling to avoid gateway timeouts.
  • Safety checklist: Covers temp file cleanup in finally blocks, SXSSF dispose, disk quota checks, CSV/Excel formula injection escaping, and data volume caps.
  • Use Case: When asked to add an order export endpoint that may return hundreds of thousands of rows, the Skill guides you to use SXSSFWorkbook with pagination, run it asynchronously, upload to object storage, and escape user-controlled text cells.

Quick Start

Ask the AI to implement or review a large data export feature, such as "add an Excel export for all orders that may exceed 100k rows", and it will apply the streaming and async patterns from this Skill.

Frequently Asked Questions about cc-streaming-export-safety

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I export large Excel files in Java without OOM?▼

Use Apache POI SXSSFWorkbook with a small window size instead of XSSFWorkbook, which loads everything into memory. Write rows in paginated batches, call dispose() in a finally block, and note that flushed rows cannot be read back or edited.

What is the difference between XSSFWorkbook and SXSSFWorkbook?▼

XSSFWorkbook keeps the entire workbook in memory and causes OOM beyond roughly 10,000 rows, while SXSSFWorkbook streams rows to temp files outside a sliding window. SXSSF cannot randomly access flushed rows, evaluate formulas, or clone sheets.

How do I prevent CSV or Excel formula injection in exports?▼

Escape user-controlled text cells that start with =, +, -, or @ by prefixing them with a single quote or tab, applied centrally in the DTO or cell writer layer. Only process text fields so numbers, dates, and amounts remain intact.

Why does my export endpoint time out with a 504 error?▼

Synchronous exports taking over 30 seconds get cut off by Nginx or gateways and block HTTP threads. Convert the endpoint to return a task ID immediately, run the export asynchronously, upload the file to object storage, and let clients poll for status.

Does openpyxl support streaming writes for large Excel files?▼

Yes, openpyxl supports streaming writes with Workbook(write_only=True), where you append rows via ws.append() instead of ws.cell(). The limitation is that written rows cannot be modified afterward.

When should I use JSONL instead of a JSON array for exports?▼

Use JSONL when exporting very large datasets, since each line is an independent JSON object that downstream consumers can parse incrementally. Standard JSON arrays require reading the entire file before parsing, which defeats streaming.

Related Skills

Explore More Software Engineering Skills→
Swear4Swear4

data-performance-optimization

Optimize data retrieval with projections, cursor-based pagination, and N+1 elimination.

Community
Advanced
loongclaw-ailoongclaw-ai
641

lark-drive

Manage Feishu Drive files, folders, and comments via lark-cli commands.

Official
Advanced
AgentiveCityAgentiveCity
2

nestjs-validation-and-pipes

Configure NestJS global ValidationPipe with DTOs for request validation and transformation.

Official
Intermediate
stacksjsstacksjs
622

stacks-queue

Orchestrate background jobs and queues across Redis, database, and in-memory drivers.

Official
Advanced