Load testing answers questions capacity planning cannot: where the system breaks, what breaks first, and whether it recovers. It also generates genuine traffic, which means it can cause a genuine outage if pointed carelessly.
Decide the question before the tooling
-
Capacity
How many concurrent users can we serve within our latency target?
-
Breakpoint
Where does it fail, and does it fail gracefully?
-
Soak
Does it degrade over hours – leaks, connection exhaustion, disk?
-
Spike
What happens when traffic arrives faster than autoscaling reacts?
Each needs a different profile. Running a generic script and looking at the graph afterwards rarely answers anything specific.
Test somewhere you can safely break
- Run against an environment that mirrors production shape, in its own account.
- Check every third-party dependency: sandbox endpoints, not live ones.
- Confirm no shared database, cache or queue is in the blast radius.
- Tell anyone who might see the alerts, in advance.
- Know how to stop the test immediately.
Generate load from the right place
Load generated from one machine in one region tests that machine’s network path as much as your service. Generate from multiple locations resembling your user distribution, and confirm the generator is not itself the bottleneck – a saturated load generator produces convincing but meaningless numbers.
What to record
| Metric | Why |
|---|---|
| Latency percentiles | The tail is what users experience. |
| Error rate by type | Timeouts and rejections mean different things. |
| Throughput at the latency limit | Your real capacity number. |
| Resource saturation | Identifies the binding constraint. |
| Downstream dependency behaviour | Often the actual bottleneck. |
| Recovery time after load stops | Whether it recovers unaided. |
Summary
Pick one question, run it in an isolated account against a production-shaped environment, generate load from realistic locations, and measure the tail. Then fix the first bottleneck and repeat.
