{"id":2416,"date":"2022-03-28T05:56:51","date_gmt":"2022-03-28T05:56:51","guid":{"rendered":"https:\/\/loadfocus.com\/blog\/?p=2416"},"modified":"2024-06-19T10:28:46","modified_gmt":"2024-06-19T10:28:46","slug":"actively-measure-the-right-key-metrics-during-performance-testing","status":"publish","type":"post","link":"https:\/\/loadfocus.com\/blog\/2022\/03\/actively-measure-the-right-key-metrics-during-performance-testing","title":{"rendered":"How to Measure the Right Key Metrics During Performance Testing"},"content":{"rendered":"<span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\"><\/span> <span class=\"rt-time\"> 6<\/span> <span class=\"rt-label rt-postfix\">minutes read<\/span><\/span>\n<p class=\"lead\">During crucial performance testing session we need to monitor and measure various parameters\/metrics  to be able to analyze and understand why the application behaves in a certain way under a specific load.<\/p>\n\n\n\n<p>Below ones are the most used metrics collected during performance testing sessions.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Vital performance metrics:<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2022\/04\/what-is-cpu-utilization-in-performance-testing\/\">Processor Usage<\/a> \u2013<\/strong>&nbsp;an amount of time processor spends executing non-idle threads.<\/li><li><strong>Memory use \u2013<\/strong>&nbsp;amount of physical memory available to processes on a computer.<\/li><li><strong>Disk time \u2013&nbsp;<\/strong>amount of time disk is busy executing a read or write request.<\/li><li><strong>Bandwidth \u2013<\/strong>&nbsp;shows the bits per second used by a network interface.<\/li><li><strong>Private bytes \u2013<\/strong>&nbsp;number of bytes a process has allocated that can\u2019t be shared amongst other processes. These are used to measure memory leaks and usage.<\/li><li><strong>Committed memory \u2013<\/strong>&nbsp;amount of virtual memory used.<\/li><li><strong>Memory pages\/second \u2013<\/strong>&nbsp;number of pages written to or read from the disk in order to resolve hard page faults. Hard page faults are when code not from the current working set is called up from elsewhere and retrieved from a disk.<\/li><li><strong>Page faults\/second \u2013<\/strong>&nbsp;the overall rate in which fault pages are processed by the processor. This again occurs when a process requires code from outside its working set.<\/li><li><strong>CPU interrupts per second \u2013<\/strong>&nbsp;is the avg. number of hardware interrupts a processor is receiving and processing each second.<\/li><li><strong>Disk queue length \u2013<\/strong>&nbsp;is the avg. no. of read and write requests queued for the selected disk during a sample interval.<\/li><li><strong>Network output queue length \u2013<\/strong>&nbsp;length of the output packet queue in packets. Anything more than two means a delay and bottlenecking needs to be stopped.<\/li><li><strong>Network bytes total per second \u2013<\/strong>&nbsp;rate which bytes are sent and received on the interface including framing characters.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2013\/07\/what-is-response-time-in-performance-testing\/\" title=\"Response time\">Response time<\/a> \u2013<\/strong>&nbsp;time from when a user enters a request until the first character of the response is received.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2013\/08\/what-is-latency-in-load-and-performance-testing\/\" title=\"Latency\">Latency<\/a><\/strong> &#8211; the processing time that is needed on the server to process your request + the delay involved for the request to reach to server.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2014\/07\/errors-and-response-codes-in-load-testing-on-loadfocus-com\/\" title=\"Error rate\">Error rate<\/a><\/strong> &#8211; represent the number of errors compared to the total number of requests that were done during the test<strong>.<\/strong><\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2013\/07\/what-is-throughput-in-performance-testing\/\" title=\"Throughput\">Throughput<\/a> \u2013<\/strong>&nbsp;rate a computer or network receives requests per second.<\/li><li><strong>Concurrent Users<\/strong>&nbsp;is the number of virtual users that are active at any given point in time during a performance test cycle.<\/li><li><strong>Requests per Second<\/strong>&nbsp;is the number of requests sent to the target server during a performance test cycle, including HTML, stylesheets, JavaScript.<\/li><li><strong>Throughput<\/strong>&nbsp;is the number of kilobytes per second transmitted during the performance test cycle, which illustrates the amount of data flowing back and forth.<\/li><li><strong>Average Response Time<\/strong>&nbsp;is the roundtrip time that it takes for a request from the client to generate a response from the server.<\/li><li><strong>Peak Response Time<\/strong>&nbsp;is the longest response time that occurred within a given performance test cycle.<\/li><li><strong>Amount of connection pooling \u2013<\/strong>&nbsp;the number of user requests that are met by pooled connections. The more requests met by connections in the pool, the better the performance will be.<\/li><li><strong>Maximum active sessions \u2013<\/strong>&nbsp;the maximum number of sessions that can be active at once.<\/li><li><strong>Hit ratios \u2013<\/strong>&nbsp;This has to do with the number of<a href=\"https:\/\/www.guru99.com\/sql.html\" class=\"broken_link\">&nbsp;SQL&nbsp;<\/a>statements that are handled by cached data instead of expensive I\/O operations. This is a good place to start for solving bottlenecking issues.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2021\/04\/what-is-hits-per-second-in-performance-testing\/\" title=\"Hits per second\">Hits per second<\/a> \u2013<\/strong>&nbsp;the no. of hits on a web server during each second of a load test.<\/li><li><strong>Rollback segment \u2013<\/strong>&nbsp;the amount of data that can rollback at any point in time.<\/li><li><strong>Database locks \u2013<\/strong>&nbsp;locking of tables and databases needs to be monitored and carefully tuned.<\/li><li><strong>Top waits \u2013<\/strong>&nbsp;are monitored to determine what wait times can be cut down when dealing with the how fast data is retrieved from memory<\/li><li><strong>Thread counts \u2013<\/strong>&nbsp;An applications health can be measured by the no. of threads that are running and currently active.<\/li><li><strong>Garbage collection \u2013<\/strong>&nbsp;It has to do with returning unused memory back to the system. Garbage collection needs to be monitored for efficiency.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Introduction<\/h3>\n\n\n\n<p>Performance testing is crucial for ensuring that your application runs smoothly and efficiently. Whether you&#8217;re a non-technical business owner or a seasoned software engineer, understanding and measuring the right metrics can significantly impact your system&#8217;s performance and user experience. This guide will walk you through the essential metrics to monitor during <a href=\"https:\/\/loadfocus.com\/blog\/2022\/03\/actively-measure-the-right-key-metrics-during-performance-testing\">performance testing<\/a> and how to measure them effectively.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Understanding Performance Testing<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">What is Performance Testing?<\/h3>\n\n\n\n<p><a href=\"https:\/\/loadfocus.com\/blog\/2022\/03\/actively-measure-the-right-key-metrics-during-performance-testing\">Performance testing<\/a> is a process used to determine how a system performs under various conditions. It assesses speed, scalability, stability, and reliability by simulating different load scenarios. The primary goal is to identify and address performance bottlenecks before they impact end users.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Why Key Metrics Matter<\/h3>\n\n\n\n<p>Metrics are the lifeblood of performance testing. They provide quantifiable data that helps you evaluate how well your system performs and where improvements are needed. By focusing on key metrics, you can make informed decisions that enhance user satisfaction and business outcomes.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Non-Technical Overview of Key Metrics<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Response Time<\/h3>\n\n\n\n<p>Response time measures the duration between a user&#8217;s request and the system&#8217;s response. It\u2019s a critical metric because it directly affects user experience. Fast response times lead to happy users, while slow response times can frustrate and drive them away.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Throughput<\/h3>\n\n\n\n<p>Throughput refers to the number of transactions processed within a given timeframe. It indicates the system&#8217;s capacity to handle a large volume of requests. High throughput is essential for ensuring that your application can serve many users simultaneously without performance degradation.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Error Rate<\/h3>\n\n\n\n<p>The error rate measures the percentage of requests that result in errors. High error rates can indicate serious issues with your application, such as bugs or resource limitations. Monitoring error rates helps you maintain the reliability and robustness of your system.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Key Metrics for Technical Audiences<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Load Time<\/h3>\n\n\n\n<p><a href=\"https:\/\/loadfocus.com\/blog\/2023\/04\/best-online-load-testing-tools\">Load time<\/a> is the time it takes for a page or application to load completely. This metric is vital for both user experience and search engine rankings. Tools like Google PageSpeed Insights can help you measure and optimize load time.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>const startTime = performance.now();\n\n\/\/ Perform some operations\n\nconst endTime = performance.now();\nconst loadTime = endTime - startTime;\nconsole.log(`Load time: ${loadTime} milliseconds`);\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Latency<\/h3>\n\n\n\n<p>Latency is the delay before a transfer of data begins following an instruction. Lower latency is crucial for applications that require real-time interactions, such as video conferencing or online gaming.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>import time\n\nstart_time = time.time()\n\n# Simulate data transfer or processing\ntime.sleep(1)\n\nend_time = time.time()\nlatency = end_time - start_time\nprint(f\"Latency: {latency} seconds\")\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Scalability<\/h3>\n\n\n\n<p>Scalability measures a system&#8217;s ability to handle increased load without compromising performance. It\u2019s essential for applications expected to grow over time. Techniques like load balancing and horizontal scaling are commonly used to enhance scalability.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Advanced Metrics and Their Measurement<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Resource Utilization<\/h3>\n\n\n\n<p>Resource utilization tracks the usage of CPU, memory, and network resources during testing. This metric helps identify whether your system has sufficient resources or if it\u2019s being overutilized, which can lead to performance issues.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code># Example command to monitor CPU and memory usage on Linux\ntop\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Concurrent Users and Sessions<\/h3>\n\n\n\n<p>Testing for concurrent users and sessions is crucial for understanding how your application handles multiple users simultaneously. This metric helps ensure that your application can maintain performance under heavy user loads.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Peak Load Metrics<\/h3>\n\n\n\n<p>Peak load metrics assess how your system performs under maximum load conditions. This type of testing helps you prepare for high-traffic events, such as product launches or sales promotions.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tools and Techniques for Measuring Metrics<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Overview of Popular Tools<\/h3>\n\n\n\n<p>Several tools can help you measure performance metrics effectively:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2020\/09\/how-to-install-jmeter-on-mac\">JMeter<\/a><\/strong>: An open-source tool for load testing and measuring performance.<\/li><li><strong>LoadRunner<\/strong>: A comprehensive tool for testing applications under load.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Step-by-Step Guide to Using JMeter<\/h3>\n\n\n\n<p>Here&#8217;s a quick walkthrough of using <a href=\"https:\/\/loadfocus.com\/blog\/2024\/03\/generating-dynamic-ids-in-jmeter-for-enhanced-test-automation\">JMeter<\/a> for performance testing:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li><strong>Download and Install JMeter<\/strong>: Visit the <a href=\"https:\/\/jmeter.apache.org\/\">JMeter website<\/a> and download the latest version.<\/li><li><strong>Create a Test Plan<\/strong>: Open JMeter and create a new test plan.<\/li><li><strong>Add a Thread Group<\/strong>: Define the number of users, ramp-up time, and loop count.<\/li><li><strong>Add Samplers<\/strong>: Specify the requests to be sent to the server.<\/li><li><strong>Add Listeners<\/strong>: Configure listeners to collect and display test results.<\/li><li><strong>Run the Test<\/strong>: Execute the test plan and analyze the results.<\/li><\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">Best Practices for Accurate Measurement<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Planning and Designing Tests<\/h3>\n\n\n\n<p>Proper planning is critical for accurate performance testing. Define clear objectives and identify the key metrics you want to measure. Design your tests to simulate real-world usage as closely as possible.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Executing Tests and Collecting Data<\/h3>\n\n\n\n<p>Follow best practices for test execution to ensure accurate data collection. This includes using realistic data, running tests during off-peak hours, and repeating tests to validate results.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Analyzing Results<\/h3>\n\n\n\n<p>Analyzing performance data involves identifying trends, pinpointing bottlenecks, and determining areas for improvement. Use visualization tools and dashboards to make sense of the data and communicate findings effectively.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">FAQs on Metrics During Performance Testing<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">What Are the Metrics Monitored in Performance Testing?<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li><strong><a>Response Time<\/a><\/strong>: Measures how long it takes for a system to respond to a request.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2013\/07\/what-is-throughput-in-performance-testing\" title=\"What is Throughput in Performance Testing?\">Throughput<\/a><\/strong>: Tracks the number of transactions processed in a given time frame.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2014\/07\/errors-and-response-codes-in-load-testing-on-loadfocus-com\" title=\"Errors and Response Codes in Load Testing on LoadFocus.com\">Error Rate<\/a><\/strong>: Calculates the percentage of errors encountered during testing.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2023\/07\/googles-best-kept-secret-how-page-speed-insights-can-skyrocket-your-rankings\">Resource Utilization<\/a><\/strong>: Monitors CPU, memory, and network usage during the tests.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">What Are the KPIs for Performance Testing?<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>Response Time<\/strong>: Key indicator of user experience and system efficiency.<\/li><li><strong>Throughput<\/strong>: Critical for understanding system capacity and performance under load.<\/li><li><strong>Error Rate<\/strong>: Essential for ensuring reliability and robustness of the application.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2020\/09\/why-does-installing-jmeter-with-homebrew-gives-invalid-option-with-plugins\">Latency<\/a><\/strong>: Important for assessing the delay in communication and data processing.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">What Is 90% in Performance Testing?<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>90th Percentile Response Time<\/strong>: Indicates that 90% of the requests are completed within this time frame, providing insight into the worst-case performance experienced by most users.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">What Are the Parameters Measured in Performance Testing?<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2020\/09\/should-i-use-the-jmeter-gui-for-running-the-load-tests-against-my-application-api\">Response Time<\/a> and Latency<\/strong>: Measure how quickly a system responds to requests.<\/li><li><strong>Throughput and Bandwidth<\/strong>: Assess the volume of data and transactions handled.<\/li><li><strong>Error Rates and Success Rates<\/strong>: Track the reliability of the system.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2023\/04\/flaky-tests-in-software-testing\">Resource Utilization<\/a><\/strong>: Monitor usage of CPU, memory, and network resources.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">What Is KPI and SLA for Performance Testing?<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2013\/08\/how-to-compare-the-performance-of-different-builds-of-a-web-application\" title=\"Comparing Performance Across Web Application Builds\">KPI (Key Performance Indicator)<\/a><\/strong>: Metrics that reflect the performance and efficiency of the system, such as response time, throughput, and error rate.<\/li><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2022\/01\/how-to-compare-two-load-tests-using-the-baseline-comparison-feature\">SLA (Service Level Agreement)<\/a><\/strong>: A commitment between service provider and client, defining the expected performance level, such as maximum allowable response time or minimum throughput.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">How to Benchmark Performance Testing?<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li><strong><a href=\"https:\/\/loadfocus.com\/blog\/2022\/12\/i-want-to-load-test-my-new-website-where-do-i-begin\" title=\"I want to load test my new website. Where do I begin?\">Set Clear Objectives<\/a><\/strong>: Define what aspects of performance are most critical.<\/li><li><strong>Select Appropriate Metrics<\/strong>: Choose relevant metrics like response time, throughput, and resource utilization.<\/li><li><strong>Use Standard Tools and Methods<\/strong>: Employ industry-standard tools and consistent testing procedures.<\/li><li><strong>Compare Against Industry Standards<\/strong>: Benchmark your results against known standards or competitors to evaluate performance.<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p>Measuring the right key metrics during performance testing is essential for ensuring your application&#8217;s success. By understanding and monitoring these metrics, you can optimize your system&#8217;s performance, improve user experience, and achieve better business outcomes.<\/p>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">LoadFocus: Your Partner in Performance Testing<\/h3>\n\n\n\n<p><em><a href=\"https:\/\/loadfocus.com\/\">LoadFocus<\/a>&nbsp;is a modern cloud testing platform,&nbsp;a&nbsp;<a href=\"https:\/\/loadfocus.com\/load-testing\" target=\"_blank\" rel=\"noreferrer noopener\">load and stress testing cloud tool<\/a>&nbsp;which provides the infrastructure to run tests with thousands of concurrent users, from&nbsp;<a href=\"https:\/\/loadfocus.com\/locations\">multiple cloud locations<\/a>, in less than a few minutes, keep history of the results, compare different runs to inspect performance improvements or performance degradation.<\/em>&nbsp;<em>It also supports running&nbsp;<a href=\"https:\/\/loadfocus.com\/jmeter-load-testing\" target=\"_blank\" rel=\"noreferrer noopener\">JMeter load tests from the cloud<\/a><\/em>&nbsp;and&nbsp;<a href=\"https:\/\/loadfocus.com\/page-speed-monitoring\">monitoring and audit web and mobile performance<\/a>.<\/p>\n\n\n\n<p><em>Written by&nbsp;Chris R.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\"><\/span> <span class=\"rt-time\"> 6<\/span> <span class=\"rt-label rt-postfix\">minutes read<\/span><\/span>During crucial performance testing session we need to monitor and measure various parameters\/metrics to be able to analyze and understand why the application behaves in a certain way under a specific load. Below ones are the most used metrics collected during performance testing sessions. Vital performance metrics: Processor Usage \u2013&nbsp;an amount of time processor spends&#8230;  <a href=\"https:\/\/loadfocus.com\/blog\/2022\/03\/actively-measure-the-right-key-metrics-during-performance-testing\" class=\"more-link\" title=\"Read How to Measure the Right Key Metrics During Performance Testing\">Read more &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":2426,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9,6,48],"tags":[387,388],"class_list":["post-2416","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-load-testing","category-performance-testing","category-test-automation","tag-collect-performance-metrics","tag-measure-performance-parameters"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/posts\/2416","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/comments?post=2416"}],"version-history":[{"count":5,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/posts\/2416\/revisions"}],"predecessor-version":[{"id":3016,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/posts\/2416\/revisions\/3016"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/media\/2426"}],"wp:attachment":[{"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/media?parent=2416"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/categories?post=2416"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/loadfocus.com\/blog\/wp-json\/wp\/v2\/tags?post=2416"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}