Key takeaways
- Decide what you are measuring and why before configuring anything.
- Choose the metrics up front, or you will report whatever looks best afterwards.
- Change one variable at a time if you want the comparison to mean anything.
Below we listed what we consider 10 tips and tricks related to load and performance testing:
1. What exactly do you want to measure?
There is no general approach for running a performance test, so you need to define what output to expect from the load test and what questions are you trying to answer?
Start with a business needs analysis as you don’t want to pay for a tool you don’t actually need or paying extra for features you don’t need.
If your application uses APIs run performance tests against APIs, if you application has clients all around the world than run load tests from different locations in the world in order to find you latency issues. Also consider the countries and regions where you anticipate high usage.
Make these relevant to your business stakeholders. Agree what level of success you need and this will define your exit criteria.
2. Decide which metrics you want to measure in your load/performance test
Load testing is useful when is done properly. Try to allocate enough time and thought for each stage of your test:
- planning your testing scenarios and getting prerequisites
- execution of your performance testing scenarios
- understanding and results analysis – this phase would require a lot time to be allocated for truly understanding the results and finding the solutions for the found problems
3. Involve the right people for successful testing scenarios
Involve the right people in the testing effort, Developer, Network Engineer, DBA, Business Owner, QA – each will approach the problem from a different angle and you get an overview of the current performance of your application, website, API infrastructure, content and network.
4. Consider free and paid tools
Evaluate your technical needs not just for today, but also for the future. Do not use price to decide which options to evaluate.
Try to be as close to your actual users experience as possible when running a load test. If you can’t achieve this using a free or an open source load testing tool, then maybe a paid solution will solve your need. Load Testing should not start with a budget and go and spend it.
5. Inspect first results and decide which test scenarios you need to run in order to get most of your application behaviour
Results need to be looked at carefully to ensure bottlenecks/errors/weaknesses are really understood and that the proposed solutions are effective. Make sure to reach out to everyone and schedule adequate time.
6. Identify your production application performance results
Be ready to test your production application.
For most of the cases you don’t have a staging environment where you can run your performance tests, or your stating environment is not an exactly duplicate of the production environment (not exact duplicates of database data, or network bandwidth, or accessible only from inside the firewall).
7. Start small and go until you break your application, website or API
Start with running a test with just one user (client) per second, and check the performance metrics your are interested in.
Go with 5, 10, 50, 100, 500 , 1000 users (clients) per second from 1 minute up to 10 minutes.
Try to store the results in an easy to inspect manner, so that when you want to compare different test runs this would be a very easy task. This is where cloud load testing solution are also handy. They keep all your previous runs without losing valuable performance testing results for future inspections.
8. Analyse load tests results
Take your time and try to understand the results of your various tests, based on number of concurrent (parallel) users, duration, delay between clients start hitting the application, location etc.
Inspect performance metrics like response time, latency, hits per second, throughput per second, errors, CPU, memory, I/O usage (where you can measure).
9. Deploy new build and run same tests again
Try to monitor improvements or degradations of your new build by running same tests on same environment.
Also run performance tests again when new build is released (code changes), hardware is changed, application architecture is changed, changes are made on the server side or at the database level, traffic spikes are expected etc.
Improvements include: bug fixes, caching, code refactoring, database optimisation or hardware changes.
10. Schedule load and performance tests
Load testing your application on a scheduled manner will help you continually improve your product, your business and find errors or application bottlenecks before your clients will encounter them.
Performance testing doesn’t have to be time-consuming, hard to understand or difficult to run. If you do your performance tests right and part of every software cycle, you can avoid future disasters.
Frequently Asked Questions
What should I settle before configuring anything?
What you are measuring and why. There is no general-purpose performance test, so without a question the run produces numbers that cannot be right or wrong.
Why choose metrics in advance?
Because chosen afterwards you will report whatever looks best. Deciding first is what makes the result a finding rather than a summary.
How do I compare two runs meaningfully?
Change one variable at a time. With several changes in flight you can see that something moved but never what moved it.
