Filed under Apache JMeter, Load Testing, Performance Testing, Stress Testing.

Overview

We are going to present how to use Apache JMeter (open source load testing tool) in order to perform basic load and stress testing on your web application. We are going through each step and show how to use JMeter’s GUI (graphical user interface) in order to build a test plan, configure load tests, run test and inspect the results.

Apache JMeter is an open source Java application, so it can run on any OS (operating system) which supports JAVA 6 or later. JMeter is a load testing (performance testing) tool designed to run tests on any given URI endpoint, with configurable number of concurrent clients (users) and other useful test configurable metrics and inspect results. Test results can be viewed in charts, tables or can be exported in CSV or XML formats.

Prerequisites

In order to start using JMeter you need a machine that you can install and run JMeter on. By default JMeter runs as a GUI application, which requires more resources than a command line based application. In order to perform the load tests faster and without bottlenecks you can run JMeter in the non-graphical mode (details on how to start JMeter in non-graphical mode in the next post)
Please note that results can differ based on the web server where the web application is hosted. Factors that we need to take into account are:

  • system resources CPU, RAM Memory of the web server
  • system resources CPU, RAM Memory of the machine where JMeter is installed (from where the tests are executed)
  • location of the web server between the load test injector and the web server where the application is hosted (which affects the latency of the tests – use a cloud load testing tool in order to easily run load tests from different locations)

Install JMeter

Apache JMeter is very easy to install. Depending on the OS, there are different ways on how to do this.

In order to run JMeter, here is what you need to install:

Make sure that after installing JAVA you add it to the $PATH environment variable. Here you can find how to add the JAVA to the $PATH environment variable for every OS .

To start JMeter on Windows OS you need to go to the folder where you unarchived JMeter binaries and just run $JMETER_HOME/bin/jmeter.bat batch file. Please see here how you can run JMeter on Mac.

So, once all is installed and JMeter is running you can start creating your first load test. For this demo we are going to use Apache JMeter 2.11 on Mac.

Build Test Plan

In order to create a load test, add a Thread Group, in order to configure the thread properties (load test configuration):

apache jmeter test plan

Start by adding a Thread Group to your Test Plan by following the below steps:

  • Select the Test Plan on the left side, and Right-click it
  • Hover to Add -> Threads (Users) -> Thread Group
  • Click on Thread Group
apache jmeter thread group

Configure the following properties which will be used to run your load test:

  • Number of threads (users): The number of concurrent clients/users JMeter is going to simulate. Set it to 10.
  • Ramp-up Period: How long JMeter should take to get all the threads started. Set it to 10.
  • Loop Count: Number of times to perform the test case. Set it also to 10.
apache jmeter thread group configuration

Now we add a HTTP Request Sampler which represents an individual request made by each thread.
To do this Add a HTTP Request Sampler to your Thread Group by following the below steps:

  • Select on the Thread Group on the left side, then Right-click it
  • Hover (mouse over) to Add -> Sampler -> HTTP Request
  • Click on HTTP Request
apache jmeter http sampler

In the HTTP Request Sampler, under Web server, set the URL in the Server Name or IP field (website URL i.e www.example.com or public IP address).

If you want to execute more than one HTTP Request by each thread (user), simply duplicate the HTTP Request Sampler and update the Server Name or IP. Here are the steps to duplicate the HTTP Request Sampler.

  • Select HTTP Request Sampler, then Copy it (Cmd/Control + C)
  • Select Thread Group, then Paste it (Cmd/Control + V)

Note: In our JMeter test we will continue to use only one HTTP Request Sampler.

Add HTTP Cookie Manager (optional)

If your web application uses Cookies, JMeter allows to add cookies which will be sent with each HTTP request. You can do that simply by adding the HTTP Cookie Manager by following these steps:

  • Select Thread Group, then Right-click it
  • Hover (mouse-over) Add -> Config Element -> HTTP Cookie Manager

 

Add HTTP Header Manager (optional)

In case you need to send specific headers with every request, JMeter allows to add headers by adding the HTTP Header Manager by following these steps:

  • Select Thread Group, then Right-click it
  • Hover (mouse-over) Add -> Config Element -> HTTP Header Manager

Add View Results in Table

You need to use JMeter’s Listeners in order to view the output of a load or stress test. One of the most used Listeners is the View Results in Table, which we’ll use in the post and here is how to add it to your Test Plan:

  • Select Thread Group, then Right-click it
  • Hover (mouse-over) Add -> Listener -> View Results in Table
apache jmeter results listener

Run your Test Plan

After installing everything needed in order to start Apache JMeter and after configuring the Test Plan, you can start running your test.

First, save the test plan with the .jmx extension.
Run the test by pressing the green arrow button in the toolbar or by pressing Cmd/Control + R keys.

apache jmeter results in tree

You’ll starting seeing results in the View Results in Table, important columns are Sample Time (ms), Status, Bytes and Latency. To understand better the errors and responses you can check out this post.

  • Sample Time (ms): Represents the number of milliseconds that the server took to serve the request (response + latency)
  • Latency (ms): Represents the number of milliseconds that elapsed between when JMeter sent the request and when an initial response was received

At the bottom of the View Results in Table, you can see the Average Sample Time in ms and the Standard Deviation.
According to the results, after running 100 samples the Average Sample Time in milliseconds was 133 ms. Latency is going to be affected by the geographic distance between the place from where the request is made to the place where the web server is hosted.

apache jmeter 100 samples

So, no error occurred on our test when running the 10 requests X 10 threads (users) = 100 samples, and the Sample Time was low 133 ms.

Increase the Load by changing the Threads (users) values

In order to make our load test more realistic, let’s change the Threads (Users) field to 300.
Clear the previous JMeter results by pressing Cmd/Control + E or by clicking the Clear icon in JMeter’s toolbar.

Start the test, and check the results again. As you can see, the Sample Time increased almost by 50% to 178 ms, however no errors occurred while running our load test with JMeter.

apache jmeter 3000 samples

Also, while running the tests, server side monitoring will be helpful, looking at metrics like CPU or Memory. In Windows you can monitor these stats by opening the PerfMon tool or in Linux by typing the top command in terminal.

You can extend the load test up to 1000 threads (users), to check how your system behaves and discover potential bottlenecks, memory leaks, performance issues. LoadFocus.com is a cloud load testing service and you can run load tests with thousands of threads (users).

Conclusion

JMeter is a useful load testing tool, which helps you find load testing issues, discover bottlenecks and increate performance of your web application. Feel free to experiment different values for the load test in order to understand how your web server behaves under heavy load. In order to add more plugins to your default JMeter installation, you can check the JMeter plugins add-ons or check here how to add JMeter plugins to JMeter.

In order to run load tests with thousands of threads (users) and to run load tests from different world locations, you can use LoadFocus.com which offers a cloud load testing service, easy to use, which keeps all your tests and test runs in the history automatically.

Filed under Product Development.

Entrepreneurs always ask themselves which is the best way to grow a startup, paid ads or organic user search.
We believe that organic customers are a good way of identifying the product market fit of your MVP. Before investing into any paid ads you must first have a product, a product that would be loved and shared by your first customers.
Once you get the product market fit and you get some traction, you may find it not that risky to actually start spending money on paid advertising (paid ads, paid videos or paid posts).
Organic Growth
Here are a few ways to reach to you customers without spending startup money:
  • optimise your application for the search engines (SEO optimisation)
  • create a blog related to your startup activity (related to future product features, tech ideas)
  • create a Q/A are on your website to dynamically generate a FAQ
  • create guest blog posts and offer to post them on other websites without asking for anything in return
  • HackerNews is a good way to share your new idea to the world and get
Some of the metrics we consider are Bounce Rate, Daily Page Views per Visitor, Daily Time on Site. Creating goals for important actions on your website and segmenting the audience is something you may need to consider. Also to validate that you have a product market fit, 90 day retention numbers give you a good overview.
Useful growth resources that helped us can be found here: https://growth-hacking.zeef.com/david.arnoux

 

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.

Filed under Load Testing, Page Load Time, Performance Testing, Test Automation, UI Testing.

Your website or API is going to be tested by our testing team.

LoadFocus.com released a new pricing plan – the Business Plan. The big plus with this new plan is that we are going to provide every month a performance and a speed assessment of your web application or API.
We are going to provide a load testing assessment:

  • potential performance bottlenecks
  • errors and issues when a large number of users access your website
  • boundary testing to understand which are the limits of your system, or how many users/clients can access you website in parallel
  • full performance metrics: response time, latency, hits per second, throughput per second, percentiles, standard devition
  • charts showing how the applications behaves over a bigger period

We are going to provide a website speed testing assessment:

  • response time and performance score
  • potential errors and W3C validation issues
  • hand-on advice on how to improve the speed on your website: caching, minifying, compressing assets, resizing images

Overall, even if you have a website, a WordPress blog or you just want to check how your APIs behave, we’re going to provide valuable feedback regarding potential issues, things on how to improve them, hand-on advice on best practices at the moment on the web (client/server side). This will help you understand better how your application works, and how your users experience it.

Click here and checkout the Business Plan created specially for you, after you’ll try it one, you’ll never want to cancel. Check it now.

 

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.

Filed under Load Testing, Performance Testing.

In this post we are going to present why minifying javascript files is important.
People believe that compressing (gzipping) JavaScript files should be enough and there is no need to minify JavaScript files.
This is a common mistake and below we present an explanation of what is involved in the process of minifying a file and why we should still do it even though we gzip our files.

What are the processes that happen when you minify a file ?

  • code comments are removed
  • line breaks are removed
  • unnecessary spaces are removed
  • extraneous punctuation (such as parentheses and semicolons) and other whitespace are removed
  • some javascript minifiers do more advanced operations like shortening of variables, properties, arguments, classes, functions and method names (sometimes people call this to be obfuscation but is just a bi-product of minification)
  • some javascript minifiers wrap the code in a immediately executed function, with a lot of arguments like below that makes it possible to use these variables without declaring them with the var keyword, thus reducing the size of you code in some cases with tree bytes times the number var keywords
(function(a,b,c,d,e,f,g){/* ... */})();

In case the files just get gzipped all these extra bytes will be compressed. So by minifying our files we can save quite a few bytes on a big file. We minifying our files we reduce also the response time when that resource is requested by our application.

 

 

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.

Filed under Load Testing, Performance Testing.

Standard deviation and percentiles are very helpful when comes to calculating the response times in a load test or a performance test.

Standard deviation is just like it sounds: the routine deviation around the average. Standard Deviation can then be used as a gauge of longer response times.

For normal distributions, we know that roughly 70% of the response times will be within one standard deviation of the average; and that about 95% of the responses will be within two standard deviations.

The standard deviation should be a low number, if your load test is consistent in load you want to see this number low because it means most response times are close in number.

If the Standard Deviation number is too low/high you can use it to identify a potential bottleneck in your web application or website.
The mean is the average, which can be pulled higher or lower by a cluster of extremely fast or slow responses being collected

The Median or the 50th Percentile is great but only accounts for what 50% of your users will experience. Would be better to check the 90th(95th) percentiles values, rather than mean, to ensure only those 10% wildcard calls are ‘missed’.
Typically, we’re only interested in the poor performers, so we pick a few top percentiles, like the 85th, 90th, 95th, and 99th.

Load testing is an iterative process. Once you find one issue, you need to re-run the load test and find the next! Here are some load testing tips on how to run your load test.

LoadFocus.com has standard deviation, mean, median (50th percentile), 90th(99th) percentiles and minimum and maximum metrics to all load tests.

Filed under Page Load Time, Performance Testing, Website Speed Testing.

Today we are going to website speed test the landing pages of the most important car producers and sellers in the UK.

 

First tested site: UK website of Audi manufacturer

Screen Shot 2016-02-25 at 00.33.16

Next tested site: UK website of Vauxhall manufacturer

Screen Shot 2016-02-25 at 00.33.44

 

Next tested site: UK website of Ford manufacturer

Screen Shot 2016-02-25 at 00.34.08

 

Next tested site: UK website of Land Rover manufacturer

Screen Shot 2016-02-25 at 00.34.44

 

Next tested site: UK website of Mini manufacturer

Screen Shot 2016-02-25 at 00.35.07

 

Next tested site: UK website of Mercedes manufacturer

Screen Shot 2016-02-25 at 00.35.25

 

Next tested site: UK website of Toyota manufacturer

Screen Shot 2016-02-25 at 00.36.04

 

Last tested site: UK website of Honda manufacturer

Screen Shot 2016-02-25 at 00.36.28

 

In terms of page load time the order would be:

  1. Mercedes site – 865 ms
  2. Honda site – 2055 ms
  3. Vauxhall site – 2183 ms
  4. Audi site – 2420 ms
  5. Ford site – 2611 ms
  6. Mini site – 2649 ms
  7. Toyota site – 2821 ms
  8. Land Rover – 3959 ms

In terms of the performance score (the score given based on compression of different types of resources, caching …) the order would be:

  1. Vauxhall site – 68/100
  2. Honda site – 58/100
  3. Audi site – 55/100
  4. Ford site – 55/100
  5. Mini site – 52/100
  6. Land Rover – 41/100
  7. Toyota site – 31/100
  8. Mercedes site – 24/100

 

Seems that almost all of the sites have a lot ore to improve to achieve the score closer to 100. For this the section with the Advice from the LoadFocus platform gives a lot of hints on how to improve this. Improving the performance score will also improve the page load time of the pages measured.

The tests were run using the Website Speed Test functionality of the LoadFocus platform:

https://loadfocus.com/website-speed-testing

 

Enjoy. Make your sites perform better for your customers satisfaction.

 

 

Filed under Load Testing, Page Load Time, Website Speed Testing.

When do you need to load test or measure the loading time of your application on localhost:

  • before you deploy your application the first time and assign a public IP
  • during development of a new feature that you want to test in terms of performance metrics

Below are the steps that will help test your application on localhost:

  • Go to https://ngrok.com/
  • Create an account
  • Download the ngrok software on your localhost
  • Install the auth token that you have been assigned when you created the account on https://ngrok.com
    Ex. ./ngrok authtoken 2CN4XNu97Hk2KCbuKF9pp_2NqX7j8kBntMBSEuG9mwv – change the last part with your auth token
  • Start ngrok on your localhost like:
    Ex. ./ngrok http 80 – this will forward the requests to port 80 on your localhost (in case you need another port just change 80 to the desired port)
  • Once started in the command line you will see the url that you need to use in your load test (should be something like the following: http://b1455fgd.ngronk.io)
  • Log into your LoadFocus account and create your first test
  • Set Test name, the location from which you want to run the test and the url which in this case will be something like: http://adafce35.ngrok.io
Screen Shot 2016-02-24 at 20.15.14

That’s all. Enjoy.

P.S.: Ngrok software will have an impact on the results of the test but is a good starting point until you will deploy your application on the production environment.

 

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.