Key takeaways
- A mean with a large standard deviation describes almost none of your users.
- Percentiles show the tail, which is where the complaints come from.
- Report p95 or p99 alongside the median rather than the average alone.
Standard deviation and percentiles are very helpful when comes to calculating the response times in a load test or a performance test.
Is Your Infrastructure Ready for Global Traffic Spikes?
Unexpected load surges can disrupt your services. With LoadFocus’s cutting-edge Load Testing solutions, simulate real-world traffic from multiple global locations in a single test. Our advanced engine dynamically upscales and downscales virtual users in real time, delivering comprehensive reports that empower you to identify and resolve performance bottlenecks before they affect your users.
Standard deviation is just like it sounds: the routine deviation around the average. Standard Deviation can then be used as a gauge of longer response times.
For normal distributions, we know that roughly 70% of the response times will be within one standard deviation of the average; and that about 95% of the responses will be within two standard deviations.
The standard deviation should be a low number, if your load test is consistent in load you want to see this number low because it means most response times are close in number.
Think your website can handle a traffic spike?
Fair enough, but why leave it to chance? Uncover your website’s true limits with LoadFocus’s cloud-based Load Testing for Web Apps, Websites, and APIs. Avoid the risk of costly downtimes and missed opportunities—find out before your users do!
If the Standard Deviation number is too low/high you can use it to identify a potential bottleneck in your web application or website.
The mean is the average, which can be pulled higher or lower by a cluster of extremely fast or slow responses being collected
The Median or the 50th Percentile is great but only accounts for what 50% of your users will experience. Would be better to check the 90th(95th) percentiles values, rather than mean, to ensure only those 10% wildcard calls are ‘missed’.
Typically, we’re only interested in the poor performers, so we pick a few top percentiles, like the 85th, 90th, 95th, and 99th.
Load testing is an iterative process. Once you find one issue, you need to re-run the load test and find the next! Here are some load testing tips on how to run your load test.
LoadFocus.com has standard deviation, mean, median (50th percentile), 90th(99th) percentiles and minimum and maximum metrics to all load tests.
LoadFocus is an all-in-one Cloud Testing Platform for Websites and APIs for Load Testing, Apache JMeter Load Testing, Page Speed Monitoring and API Monitoring!
Frequently Asked Questions
Why is the average response time misleading?
Because a mean with a large standard deviation describes almost none of your users. Half of a bimodal distribution can be fast and half unusable, and the average sits in a gap nobody experienced.
What do percentiles tell me?
Where the tail is. The p95 and p99 are the requests people complain about, and they can be several times the median while the average barely moves.
What should I report?
The median with p95 or p99 alongside it. Either one on its own can be made to look good, and together they describe the actual spread.