What problem does it solve?
Handling large datasets and high-traffic scenarios often leads to performance bottlenecks, N+1 queries, and out-of-memory errors. This Skill provides proven patterns and tools within Koan Framework to build highly performant and scalable applications from the ground up.
Core Features & Use Cases
- Memory-Efficient Streaming: Process millions of records without exhausting memory by streaming data in batches instead of loading everything at once.
- Optimized Count Strategies: Get accurate or estimated record counts thousands of times faster using metadata-based "Fast" counts for UI, and "Exact" counts for critical logic.
- Bulk Operations: Perform mass create, update, or delete operations with a single, highly optimized database call, dramatically reducing execution time.
- Batch Retrieval: Eliminate N+1 query problems by fetching multiple entities by ID in a single, efficient database query.
- Pagination for APIs: Implement robust pagination for web APIs, providing total counts for rich user interfaces.
- Use Case: Process a daily batch of 100,000 sensor readings without memory issues using streaming, update thousands of product prices in a single transaction, or display a dashboard with real-time (estimated) counts of active users.
Quick Start
To stream all 'Todo' entities in batches of 1000 to avoid memory issues:
await foreach (var todo in Todo.AllStream(batchSize: 1000)) {
// Process todo
}
To get a fast, estimated count of all 'Todo' entities for a dashboard:
var fastCount = await Todo.Count.Fast();
