What is Scalability Testing? Definition and Method

What is scalability testing?

Scalability testing measures how a system's performance changes as you add or remove resources: more pods, more nodes, bigger instances, more database replicas. The goal is to learn whether doubling the infrastructure roughly doubles the throughput (linear scaling), or whether gains flatten (sub-linear) or even reverse because of coordination overhead. It tells you what extra capacity really buys.

Scalability testing vs load testing

A load test measures behaviour at one fixed infrastructure size. A scalability test repeats the same load test at several sizes and compares the results. Related, but different, capacity testing asks how much load one configuration can carry before it breaks its targets.

Example with illustrative numbers: at 1 node the system serves 1,000 requests per second at a p95 latency of 600 ms. At 8 nodes, linear scaling would be 8,000 requests per second at the same latency. If you measure 6,000, scaling is sub-linear; if latency climbs to 1,200 ms at 4,000, coordination is eating the gains.

Horizontal and vertical scaling

• Horizontal: add instances. Run the same load against 1, 2, 4, 8 instances behind a load balancer.
• Vertical: give one instance more CPU, memory or disk throughput. Run the same load against small, medium and large sizes.

Stateless web tiers usually scale horizontally well. Databases and other stateful parts often scale vertically first and need replicas or sharding later, and those are exactly the parts worth testing.

How to run a scalability test

1. Pick a fixed workload. One scripted journey mix, one data set, the same load regions, so the infrastructure is the only variable.
2. Choose the scale points. For example 1, 2, 4 and 8 nodes, or three instance sizes.
3. Run each point to steady state. At each size, find the highest load that still meets your response time and error targets.
4. Plot the curve. Throughput, throughput per node and p95 latency against size.
5. Find the bottleneck where the curve bends. Check database connections, locks, shared caches, network and the load balancer.

What to measure

• Scaling factor: throughput at 2N divided by throughput at N. About 2.0 is linear.
• Throughput per node: should stay roughly constant. A drop signals overhead.
• p95 latency per size: should hold or improve, not climb.
• Cost per request: the practical result, since a sub-linear curve means each extra node costs more per unit of work.
• Autoscaling behaviour: how long new instances take to help, which spike testing exposes.

Common mistakes

• Changing the test between runs. A different script, region mix or data size makes the curve meaningless.
• Scaling only the app tier. The database or a shared cache becomes the ceiling and the curve flattens early.
• Starting from a cold system. Warm caches and connection pools first, or the first point looks worse than it is.
• Load generator limits. If the generator saturates, you measure the generator, not the system.
• Too few points. Two sizes show a ratio but not where the curve bends.

Tools and where to go next

The same scripts run at every scale point, so any load testing tool works; JMeter and k6 are the usual choices. LoadFocus runs both from 25 cloud regions, so load generation stays consistent across the whole series. For architecture-specific advice, see the guide to load testing microservices in the cloud (more on microservices) and the overview of performance testing types.

FAQ: scalability testing

What is the difference between scalability testing and load testing?

Load testing checks performance at one fixed size. Scalability testing runs a series of load tests at different sizes to see how performance changes with resources.

What is linear scalability?

It means throughput grows in proportion to resources: double the nodes, roughly double the requests per second at the same latency.

Horizontal or vertical scalability: which should I test?

Test the one your architecture relies on. Many systems use both: horizontal for stateless services, vertical (then replicas or sharding) for databases.

Does scalability testing validate autoscaling?

Partly. It shows whether added capacity is useful. To see how quickly autoscaling reacts, combine it with a spike test and watch latency while new instances start.

How many scale points do I need?

At least three or four, for example 1, 2, 4 and 8 nodes, so you can see the shape of the curve and not only one ratio.