Before adding servers: finding the bottleneck that limits growth.

Start with the outcome

“We need to scale” is a direction, not a testable requirement. Replace it with a workload and an acceptable result: a particular mix of operations at a given arrival rate, within latency and error targets. Include data volume, payload sizes, and the duration you expect the system to sustain the load.

Concurrent users alone rarely describe the workload. Ten thousand people reading cached pages are different from ten thousand people issuing expensive searches. Capture the activities, think times, and bursts that matter to the product before selecting a load-test scenario.

Follow the work across the system

Map the request path: entry point, application workers, database calls, external dependencies, and any queues in between. At each step, distinguish time doing work from time waiting. This gives you a sequence of places to investigate instead of a collection of infrastructure graphs without context.

Brendan Gregg’s USE Method provides a useful resource-oriented check: examine utilization, saturation, and errors for each resource. A queue can reveal unmet demand even when a utilization metric is misleading or incomplete. Reference: The USE Method.

Ask whether another instance reaches the constraint

If requests are waiting on a shared dependency, adding application instances may leave that dependency unchanged. If the application itself is doing independent work and has spare downstream capacity, another instance may help. Both possibilities need measurement.

Before choosing the intervention, make a specific prediction. For example: “If worker capacity is the limiting factor, increasing it should reduce this queue without pushing the database beyond its acceptable operating range.” That prediction tells you which metrics to watch and what would disprove your theory.

Run a bounded experiment

Use an environment and dataset representative enough to answer the question. Increase load in controlled steps while recording completed work, latency, errors, queues, and resource behavior. Define stopping conditions before the test. Keep production testing inside an explicitly agreed scope.

Change one important variable at a time where practical. Repeat enough to distinguish a stable effect from warmup, background work, or noise. A single impressive peak throughput number does not describe sustained capacity under a mixed workload.

Consider the cheapest useful intervention

The evidence may point to an inefficient query, repeated work, unbounded concurrency, or a serial section. It may also confirm that additional infrastructure is the sensible solution. Compare implementation effort and operating cost with the amount of useful headroom each option is likely to provide.

Include the maintenance consequences. A cache introduces freshness questions. A new queue changes failure and recovery behavior. A more intricate algorithm can make future changes harder. An optimization should survive this broader review, not just improve one benchmark.

Write down the operating envelope

After the change, record the tested workload, achieved results, resource configuration, and remaining unknowns. Give the team a practical threshold for revisiting capacity. Keep the load scenario and measurement steps available so the next growth decision starts from evidence.

Your goal is a known operating envelope with useful headroom. Infrastructure spend can then become an intentional decision tied to demand rather than a repeated reaction to unexplained slowdowns.