Key takeaways
- This is an opinion, and a plain one: generating load is the easy part. The hard part is making the load mean something.
- Four things make that hard: test data and realism, an environment that behaves like production, think time and the load model, and reading the numbers once you have them.
- A tool will run whatever script you hand it. It will not tell you the script was unrealistic.
Why generating load is not the hard part
Starting 500 virtual users is a solved problem. Apache JMeter, k6 and cloud platforms such as LoadFocus all do it. What no tool can do for you is decide which 500 users, doing what, against which system, and what a result means afterwards. That is where load tests go wrong, and it is why I think the hardest part of load and performance testing is realism, followed closely by interpretation.
If you are new to the vocabulary, the mistakes post lists the usual errors and the realistic behaviour guide covers modelling in detail. This post is the argument for why they are hard, not a second checklist.
Test data and realism: the part nobody enjoys
A script that logs in as one user and fetches one product page 10,000 times mostly tests your cache. Real traffic has many users, many products, different cart contents and different search terms, and every one of those changes which code path and which database rows get touched.
The tooling for this is simple and the work is not. JMeter’s own best-practices page says to put test data in a file and read it with the CSV Data Set element, which reads a new line for each thread. The hard bit is producing a data file that looks like production: enough distinct accounts, realistic spread of popular and rare items, and no reused single-use values such as one-time tokens. Our guide to using CSV data in JMeter tests shows the mechanics.
Environment parity: a smaller system fails differently
My opinion here is the least comfortable one: a test against a scaled-down staging copy answers a smaller question than most people think. A database with a fraction of production’s rows, a cache that is empty, one app server instead of an autoscaled group, and a third-party API stubbed out will all produce numbers, and none of those numbers transfer cleanly to production. Performance is rarely linear in server count or data size, so you cannot multiply a staging result by a ratio and call it a forecast.
What helps is being explicit about what the environment differs on, treating results as a comparison between builds on the same environment, and verifying anything critical with a short, careful test in the real thing. That last step needs agreement with whoever owns production. Our staging environment explainer covers what a staging copy is for.
Think time and the load model: the quiet distortion
Think time is the pause a real person takes between actions. In JMeter you model it with timer elements; in k6 you call sleep(), which the k6 docs describe as suspending VU execution for the specified duration. Leave it out and every virtual user fires requests back to back, so 100 virtual users can generate far more requests per second than 100 real people ever would. Put in a fixed pause and every user moves in lockstep, which is also unlike real traffic. A random spread around a realistic average is closer.
The deeper issue is the load model. Most scripts are a closed model: each virtual user waits for its response before the next iteration. k6’s documentation defines it as one where “VU iterations start only when the last iteration finishes”, and contrasts it with an open model where “VUs arrive independently of iteration completion”. Real visitors behave like the second: they keep arriving whether or not your site is struggling. In a closed model, a slowing system automatically slows the test, so it can look better than it is. k6’s docs note that this problem is known in some testing literature as coordinated omission, and that its arrival-rate executors (constant-arrival-rate, ramping-arrival-rate) implement the open model.
Gil Tene, who coined the term, argues in his talk “How NOT to Measure Latency” (QCon London, 2013) that when a test loop waits for each response before timing the next request, slow periods get under-counted, and the percentiles you report describe the tester’s pacing rather than what a user arriving at a random moment would see. JMeter’s best-practices page warns of the same trap in a different place: with too few threads for a target throughput, “you will face the Coordinated Omission problem”. I read the talk through slide summaries and secondary write-ups, not a full transcript, so go to the original for his exact claims.
To size a test in the first place, see calculating concurrent users.
Interpreting results: where most of the effort goes
After the run, you have response-time distributions, error counts and a throughput curve, and the real work starts. Three habits make this easier.
- Read percentiles, not just the average. An average hides the slow tail. Why percentiles beat averages explains the arithmetic.
- Check the load generator itself. A saturated generator makes the system under test look slow. JMeter’s best practices recommend avoiding View Results Tree and View Results in Table listeners during a load test and, for large loads, running multiple command-line instances in distributed mode.
- Decide the pass criteria before the run. Agree on thresholds such as a p95 limit and an error budget up front. Otherwise the result gets argued to fit the story afterwards.
Finding the actual bottleneck, meaning code, database, connection pool or a downstream service, needs server-side metrics beside the test results. The test says that something is slow; it rarely says why.
What to do about it
- Model two or three business-critical journeys well rather than twenty badly. The scenario design method walks through choosing them.
- Build the data file first, then the script.
- Write down every way the test environment differs from production and carry that list into the report.
- Prefer an open model when you care about a target arrival rate, and say which model you used.
- Run the same test after every significant change, so you compare like with like.
For running these on cloud generators, see LoadFocus’s JMeter and k6 pages.
Frequently Asked Questions
What is the hardest part of load testing?
In my view, making the test realistic: representative data, user behaviour with real think time, a load model that matches how traffic arrives, and an environment close enough to production. Running the load is the easy part.
Is load testing hard?
Starting a test is easy. Designing one whose results you can trust, and then explaining a bottleneck from the numbers, takes practice.
How do I decide how much load to use?
Start from real traffic: peak concurrent users or requests per second from analytics or server logs, plus a growth margin you can justify. Calculating concurrent users shows one way.
What does the 90 percent figure mean in performance testing?
Most likely the 90th percentile: the response time that 90% of requests were at or below. Teams often use the 95th or 99th for the slow tail.
What is the 80/20 rule in performance testing?
It is the Pareto idea applied informally: a small share of components often accounts for most of the delay. It is a rule of thumb, not a measured law, and only your profiling data can say whether it holds for your system.
Sources
- Grafana k6 documentation, open and closed models (grafana.com/docs/k6, read October 2026)
- Grafana k6 documentation,
sleep(grafana.com/docs/k6, read October 2026) - Apache JMeter, best practices (jmeter.apache.org/usermanual/best-practices.html, read October 2026)
- Gil Tene, “How NOT to Measure Latency”, QCon London 2013 (via slide summaries and secondary write-ups)
