scalability-plan

Collect 30 days of metrics including average and p99 RPS, CPU, memory, and database connections. Then model growth scenarios (conservative, expected, optimistic) and calculate required servers based on a target CPU utilization of around 70 percent.

Use horizontal scaling for application servers because it scales linearly and provides redundancy, but it requires stateless applications and a load balancer. Use vertical scaling for databases initially since sharding adds significant complexity.

Define a HorizontalPodAutoscaler targeting CPU utilization around 70 percent and memory around 80 percent, with min and max replica bounds. You can also scale on custom metrics like requests per second, and tune scale-up and scale-down stabilization windows.

Use read replicas first since they handle roughly 10x read growth with far less complexity. Move to sharding only when write throughput or data size exceeds a single primary, since sharding requires application-level routing logic.

Run load tests with Locust at twice your expected peak traffic while watching HPA and pod counts. Confirm that rising CPU triggers scale-up, new pods start successfully, and utilization normalizes.

Use reserved instances for baseline capacity to save 40 to 60 percent, on-demand instances for predictable peak hours, and spot instances for interruptible batch jobs. A hybrid strategy can cut costs by over 50 percent versus all on-demand.