Baseline Comparison Between Load Tests
Baseline Comparison Between Load Tests: An Essential Performance Analysis Tool
When it comes to load testing, establishing a standard for comparison is crucial. That's where the concept of a baseline comes into play. A baseline, in the context of load testing, represents a specific test run that's used as a reference point against which other test runs are compared. This benchmark allows testers and developers to detect variations in performance metrics over different test iterations.
Why Establish a Baseline?
Consistency: By setting a baseline, teams can ensure that they are comparing subsequent tests to a consistent standard. This consistency is pivotal for making informed decisions about any detected deviations in performance.
Performance Trends: With a defined baseline, it becomes easier to spot trends, whether it's an improvement due to optimizations or a decline possibly caused by new code changes or infrastructure adjustments.
Impact Analysis: Introducing new features or making backend changes? By comparing new test runs with the baseline, teams can quickly assess the impact of such alterations on performance.
Focused Debugging: If a certain test run exhibits issues, comparing it with the baseline can help pinpoint the specific areas where the test run deviated, making it easier to narrow down potential causes.
Set, change and remove a baseline
Baseline comparison is available from the Essential plan.
- Run the test, then open the results of the run you want as the reference.
- Click Set as baseline. The comparison percentages and the text under the test name confirm it worked.
- Every later run of the same test is now compared against that run automatically.
To change the baseline, open the results of a different run of the same test and click Set as baseline. The run you are viewing becomes the new baseline and the comparisons are recalculated.
To remove it, open the run that is the current baseline and click the green check that marks it. The test then has no baseline and no run shows a comparison.
Pick a run you trust and can reproduce, such as a known-good release, not the fastest number you ever recorded. A baseline set on an unusually good run makes every later release look like a regression.
See also baseline comparison and performance trend of load test results.