database-sharding

Database sharding partitions data horizontally across multiple database instances using a shard key. Hash, range, or directory-based strategies distribute records to specific shards, enabling horizontal scaling for billions of records while maintaining query isolation per shard.

A shard key must have high cardinality, remain immutable after insertion, and route single queries to one shard when possible. Consistent hashing minimizes data reshuffling during rebalancing, reducing migration overhead and maintaining even distribution across shards.

Cross-shard queries require routing logic that targets multiple shards and aggregates results. Two-phase commit or saga patterns handle distributed transactions, though latency increases. Dual-write migration strategies enable zero-downtime rebalancing when reshuffling data between shards.

Input validation on shard identifiers and strict parameterized queries prevent injection attacks in multi-tenant deployments. Directory-based sharding isolates tenant data by routing requests through validated shard keys before query execution.

Hash sharding distributes keys evenly across shards using consistent hashing, ideal for uniform load. Range sharding partitions sequential or time-series data by value ranges, simplifying range queries but requiring hotspot mitigation and manual rebalancing.

Robust monitoring tracks per-shard record counts, query latency, and storage distribution to identify hotspots. Rebalancing strategies redistribute data when skew exceeds thresholds, maintaining performance and preventing single-shard bottlenecks.