{"id":673,"date":"2016-03-14T11:39:50","date_gmt":"2016-03-14T11:39:50","guid":{"rendered":"https:\/\/loadfocus.com\/blog\/?p=673"},"modified":"2016-03-11T15:40:41","modified_gmt":"2016-03-11T15:40:41","slug":"standard-deviation-and-percentiles-in-load-testing-metrics","status":"publish","type":"post","link":"https:\/\/loadfocus.com\/blog\/2016\/03\/standard-deviation-and-percentiles-in-load-testing-metrics","title":{"rendered":"Standard Deviation and Percentiles in Load Testing Metrics"},"content":{"rendered":"<span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\"><\/span> <span class=\"rt-time\"> 2<\/span> <span class=\"rt-label rt-postfix\">minutes read<\/span><\/span><p class=\"lead\"><!-- pn-tldr --><\/p>\n<h2>Key takeaways<\/h2>\n<ul>\n<li>A mean with a large standard deviation describes almost none of your users.<\/li>\n<li>Percentiles show the tail, which is where the complaints come from.<\/li>\n<li>Report p95 or p99 alongside the median rather than the average alone.<\/li>\n<\/ul>\n<p><!-- \/pn-tldr --><strong>Standard deviation<\/strong> and <strong>percentiles<\/strong> are very helpful when comes to calculating the response times in a load test or a performance test.<\/p>\n<p>Standard deviation is just like it sounds: the routine deviation around the average. Standard Deviation can then be used as a gauge of longer response times.<\/p>\n<p>For normal distributions, we know that roughly 70% of the response times will be within one standard deviation of the average; and that about 95% of the responses will be within two standard deviations.<\/p>\n<p>The standard deviation should be a low number, if your load test is consistent in load you want to see this number low because it means most response times are close in number.<\/p>\n<p>If the Standard Deviation number is too low\/high you can use it to identify a potential bottleneck in your web application or website.<br \/>\nThe mean is the average, which can be pulled higher or lower by a cluster of extremely fast or slow responses being collected<\/p>\n<p>The Median or the 50th Percentile is great but only accounts for what 50% of your users will experience. Would be better to check the 90th(95th) percentiles values, rather than mean, to ensure only those 10% wildcard calls are &#8216;missed&#8217;.<br \/>\nTypically, we\u2019re only interested in the poor performers, so we pick a few top percentiles, like the 85th, 90th, 95th, and 99th.<\/p>\n<p>Load testing is an iterative process. Once you find one issue, you need to re-run the load test and find the next! Here are <a href=\"https:\/\/loadfocus.com\/blog\/2015\/08\/20\/top-10-performance-testing-tips-and-tricks\/\">some load testing tips<\/a> on how to run your load test.<\/p>\n<p><a href=\"https:\/\/loadfocus.com\">LoadFocus.com<\/a> has standard deviation, mean, median (50th percentile), 90th(99th) percentiles and minimum and maximum metrics to all load tests.<!-- pn-faq --><\/p>\n<h2>Frequently Asked Questions<\/h2>\n<h3>Why is the average response time misleading?<\/h3>\n<p>Because a mean with a large standard deviation describes almost none of your users. Half of a bimodal distribution can be fast and half unusable, and the average sits in a gap nobody experienced.<\/p>\n<h3>What do percentiles tell me?<\/h3>\n<p>Where the tail is. The p95 and p99 are the requests people complain about, and they can be several times the median while the average barely moves.<\/p>\n<h3>What should I report?<\/h3>\n<p>The median with p95 or p99 alongside it. Either one on its own can be made to look good, and together they describe the actual spread.<\/p>\n<p><!-- \/pn-faq --><!-- pn-related-reading --><\/p>\n<h2>Related reading<\/h2>\n<ul>\n<li><a href=\"https:\/\/loadfocus.com\/blog\/2021\/02\/understanding-standard-deviation-in-performance-testing\">Understanding Standard Deviation in Performance Testing<\/a><\/li>\n<li><a href=\"https:\/\/loadfocus.com\/blog\/2021\/10\/why-are-percentiles-more-useful-than-averages-in-performance-testing\">Why Percentiles Beat Averages in Performance Testing<\/a><\/li>\n<li><a href=\"https:\/\/loadfocus.com\/blog\/2026\/06\/reducing-api-latency-distributed-load-testing-case-study\">Reducing API Latency With Distributed Load Testing<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\"><\/span> <span class=\"rt-time\"> 2<\/span> <span class=\"rt-label rt-postfix\">minutes read<\/span><\/span>Key takeaways A mean with a large standard deviation describes almost none of your users. Percentiles show the tail, which is where the complaints come from. Report p95 or p99 alongside the median rather than the average alone. Standard deviation and percentiles are very helpful when comes to calculating the response times in a load&#8230;  <a href=\"https:\/\/loadfocus.com\/blog\/2016\/03\/standard-deviation-and-percentiles-in-load-testing-metrics\" class=\"more-link\" title=\"Read Standard Deviation and Percentiles in Load Testing Metrics\">Read more &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":3731,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[109,111,110,108,112,107],"class_list":["post-673","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-load-testing","tag-90th-percentile","tag-mean","tag-median","tag-percentiles","tag-performance-metrics","tag-standard-deviation"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/posts\/673","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/comments?post=673"}],"version-history":[{"count":0,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/posts\/673\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/media\/3731"}],"wp:attachment":[{"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/media?parent=673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/categories?post=673"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/tags?post=673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}