Optimize SQL scan and filter operations using predicate pushdown and SIMD filtering.
npx skills add https://github.com/SolidLao/GenDB --skill scan-optimization
Please help me install this Agent Skill. Skill: scan-optimization Source: https://github.com/SolidLao/GenDB/tree/main/.claude/skills/scan-optimization Command: npx skills add https://github.com/SolidLao/GenDB --skill scan-optimization
This Skill optimizes how data is read from and filtered within large tables in database queries, significantly speeding up query execution by reducing unnecessary data processing and I/O.
WHERE clause on a specific date range and a few other columns, this skill ensures only the relevant data blocks and columns are accessed, drastically reducing scan time.Use the scan-optimization skill to optimize a query with a WHERE clause on a large table.
High-intent search queries and answers about installing and using this skill.
To optimize SQL query scan performance on large tables, you can apply techniques like predicate pushdown, late materialization, selection vectors, and SIMD filtering to reduce unnecessary data processing and I/O during execution.
Predicate pushdown speeds up WHERE clause execution by evaluating filters as early as possible in the query planning phase, ensuring that only relevant data blocks are accessed and drastically reducing scan time.
Use late materialization when filtering multi-billion row fact tables to load only necessary columns for qualifying rows, minimizing I/O and accelerating query execution for low-selectivity filters.
Selection vectors improve low-selectivity query performance by efficiently gathering qualifying row indices, while SIMD filtering leverages CPU vector instructions to process data in parallel for rapid filtering.
Query scan optimization requires an understanding of hardware characteristics and data distributions to navigate query execution trade-offs effectively during planning, generation, and optimization phases.
rsmdtDiagnose performance issues by profiling CPU, memory, and I/O metrics.
kabirnayeem99Analyze Perfetto trace files to identify Android performance issue root causes.
spqwLoad Apache Arrow IPC and Parquet files in Observable Framework projects.
smith-hornOptimize AgentDB vector databases with quantization, HNSW indexing, and caching.