Key takeaways
- A high standard deviation means inconsistent response times, which users notice more than a slow average.
- Two runs with the same mean can behave completely differently, and this is how you tell.
- Pair it with percentiles rather than reporting either alone.
- One extreme outlier can inflate it, and response times are skewed, so read it next to the median and the 95th percentile.
Standard deviation in performance testing measures how far individual response times spread around their average (the mean). A small value means most requests take about as long as the average, so the application behaves consistently. A large value means some requests are much faster or slower than the average, so the average describes few real users.
How Is Standard Deviation Calculated?
Take each response time, subtract the mean, square the result, average the squares, and take the square root. Squaring makes every deviation positive and gives big deviations more weight than small ones. The result is in the same unit as the response times (milliseconds).
There are two versions. The population standard deviation divides the sum of squares by n, the number of samples. The sample standard deviation divides by n - 1. For a load test with thousands of samples the two are practically identical; they differ visibly only on small data sets.
Worked Example With Two Runs That Have the Same Average
Two runs each send five requests. Response times in milliseconds:
| Run | Response times (ms) | Mean (ms) | Population std dev (ms) | Sample std dev (ms) |
|---|---|---|---|---|
| A | 100, 110, 90, 105, 95 | 100 | 7.07 | 7.91 |
| B | 20, 200, 10, 250, 20 | 100 | 103.34 | 115.54 |
For run A the deviations from the mean are 0, 10, -10, 5 and -5. Their squares add up to 250. Divided by 5 that is 50, and the square root is 7.07. Dividing by 4 instead gives 62.5 and a square root of 7.91.
For run B the deviations are -80, 100, -90, 150 and -80, whose squares add up to 53,400. Divided by 5 that is 10,680 and the square root is 103.34. Dividing by 4 gives 13,350 and 115.54.
Both runs report a mean of 100 ms, yet run A is steady and run B swings between very fast and very slow. Only the standard deviation shows the difference. A handy way to compare is the standard deviation divided by the mean: about 7% for run A and about 103% for run B.
What Is a High Standard Deviation?
There is no universal threshold, because it depends on the mean. A standard deviation of 50 ms is large next to a 40 ms mean and small next to a 2,000 ms mean. Compare the standard deviation with the mean (as above), and compare runs of the same test against each other. A rise between two runs of the same scenario, with the same load, points to a regression or to a resource that is becoming unstable.
Common causes of a high spread are garbage collection pauses, lock or connection pool contention, cache hits mixed with cache misses, a slow third-party dependency, and load generators that are themselves overloaded.
Why You Should Not Read It Alone
Standard deviation is sensitive to outliers because it squares the deviations. Take eight requests of 100 to 140 ms and one that times out at 5,000 ms: the median is 125 ms, but the mean jumps to about 664 ms and the population standard deviation to about 1,533 ms. One request moved both numbers a lot.
Response times are also skewed to the right: many fast requests and a long tail of slow ones. The familiar rule that about 95% of values fall within two standard deviations of the mean applies to normally distributed data, so do not use it to predict response times. The 95th and 99th percentiles say directly how slow the slow requests are, which is why this metric works best next to them. Why percentiles beat averages covers that side.
How Do JMeter, k6 and LoadFocus Report It?
- JMeter: the Summary Report listener shows a “Std. Dev.” column per sampler. JMeter computes it as the square root of the mean of the squares minus the square of the mean, which is the population form (jorphan
StatCalculator, apache/jmeter source, read October 2026). - k6: the end-of-test summary shows avg, min, med, max, p(90) and p(95) for trend metrics by default and does not list a standard deviation (
--summary-trend-statsdocumentation, grafana.com/docs/k6, read October 2026). If you need it, compute it from exported results. - LoadFocus: the result metrics of a load test include Standard Deviation next to the average, minimum and maximum response time and the 50th, 90th, 95th and 99th percentiles (LoadFocus docs, October 2026). It works the same whether you test a website, an API, or run a JMeter script in the cloud.
Because tools may use different formulas and different sample sets, compare standard deviations from the same tool. To try it on your own site, run a load test and read the spread next to the percentiles.
Frequently Asked Questions
What does a high standard deviation tell me?
That response times are inconsistent. Users notice inconsistency more than a uniformly slow site, because they cannot form an expectation of how long anything takes.
Two runs have the same average. Are they the same?
Not necessarily. Identical means can hide completely different distributions, one steady and one swinging between fast and unacceptable, as runs A and B above show.
Should I report it on its own?
Pair it with percentiles. Standard deviation says how spread out the results are; percentiles say where the slow tail actually sits.
Is it the population or the sample standard deviation?
It depends on the tool. JMeter uses the population form. With thousands of samples the difference is negligible, but on a handful of samples it can be 10% or more, as in the example above.
Sources
- Apache JMeter StatCalculator source (github.com, October 2026)
- k6 options reference: summary-trend-stats (grafana.com/docs/k6/latest/using-k6/k6-options/reference, October 2026)
Written by Bogdan Vazzolla.
