LoadFocus launches today the cloud load and performance testing service powerful enough to stress a web application or API with thousands of concurrent users.
Based on Apache JMeter open source project and hosted in the cloud, LoadFocus is designed for software developers, QA (Quality Assurance) engineers and product owners to run accurate load and performance tests with minimum costs.
As we can see nowadays there are numerous web applications and APIs that appear everyday which requires load and performance assessments.
However, the existing commercial and professional load testing solutions are too expensive and too complex to meet real world requirements. The open-source load testing tools, like JMeter, are limited in terms of scalability for the requirements of enterprise and high-traffic web applications.
Once you sign up with LoadFocus.com, the only cost you will have is running the injector machines (Amazon EC2 instances). The cost depends on the type of machine you’re using and the duration of the load test.
LoadFocus.com provides unlimited testing capacity, interactive real-time reporting and comprehensive result analysis integrated with mobile devices.
Let’s say that you want to access a web service or a web page. The response time that you can see with different tools is a sum of the server processing time of the request and the delay involved for your request to reach to server and back.
This is getting more important when remote data centers are hosting the web page or web service.
You can have a look at the latency vs. response time chart as you need to understand these different aspects of the system, and it is important for all the parties involved to know the difference between the two when they are doing a performance analysis.
You can use LoadFocus in order to stress test your application or API, inspect the latency and response time for all requests.
Performance testing the page load time of a web application can be a little tricky but pretty useful for simulating what the user experience of your customers.
The following solution uses the PhantomJS headless browser for measuring the needed timing information, Google charts for visualizing the results. The solution can also be used pretty much as it in the continuous integration environment based on Jenkins or any other system.
For what can I use this solution ?
page load time testing
size and number of resources retrieved for URL call
load time for each resource
The solution described below is composed of the following scripts and executables:
1. PhantomJS headerless browser 2. load.sh script: script used for running the starting PhantomJS and the performance measuring script 3. loadreport.js: script used for retrieving the timing information needed for performance measuring
Steps to make the solution work for you (Linux OS as this is what I have , similar on MacOS or Windows).
wait for PhantomJS to be downloaded and issue the command ”
tar -xvjpf phantomjs-1.9.1-linux-x86_64.tar.bz2
now PhantomJS is installed
2. Create the load.sh script and add the code from below (or git clone fromhttps://github.com/loadfocus/pageloadtime.git)
3. Create the loadreport.js script and add the code from below (or git clone from https://github.com/loadfocus/pageloadtime.git)
4. Running the load.sh script
got to the directory where the load.sh script is located and issue the following command in the terminal:
sh load.sh url_to_be_measured number_of_requests_to_be_made path_to_phantomjs
sh load.sh http://www.google.com 20 /home/user/tempphantom/phantomjs-1.9.1-linux-x86_64/bin/phantomjs
5. Visualizing the results – after the test finishes in the same directory you will see: results.html which contains the chart with the results reports directory which contains the json file with the actual results – open results.html in the browser of your choice
Apache JMeter is an open source load and performance tool written in Java and it’s available on almost any OS. This is an intro post about Apache JMeter tool and I’m trying to point out how it can help you in your daily Quality Engineer job. You can use Apache JMeter for performance, load, stress testing and memory-leak testing for accurate information. Also, JMeter can be used for understanding the performance of a web application, service, end-point etc.
I will present how to use Jmeter for creating a basic load test for a web application, below are Jmeter test configuration from the most basic one (without assertions and reporting), to the most advanced which includes Response Assertions, Reporting, Exporting Data and Chart Results and so on.
1. GET HTTP Request for https://www.example.com/ Description:
Add a new Thread Group to the Test Plan
Right Click on Test Plan and go to Add -> Threads (Users) -> Thread Group
Add a new HTTP GET Request to the Thread Group
Right Click on Thread Group and go to Add -> Sampler -> HTTP Request
This HTTP Request Sample is going to contain the configuration for your web request
Configure the GET HTTP Request Sampler
In order to make a performance test you need to configure the HTTP Sampler request with the necessary data
Change the name of the HTTP Request with a more descriptive one: GET Request www.example.com
In the Web Server -> Server Name of IP add the URL of the website, web application that you want to test: www.example.com
Do not add the protocol in front of the URL (I.e https://www.example.com or https://www.example.com), the protocol will be added only in the Protocol field
Add the Port Number of the web application: 80 for HTTP or 443 for HTTPS
Add the protocol in the Protocol [http] field: http
Select the HTTP Method from the Method drop down: GET
Add the Path of the request
For testing the home endpoint you can add only a / character in Path field
Or you can add a full URL: /index.html?query1=1&query2=2
Follow Redirects and Keep Alive checkboxes should be selected by default, if not please check them
Click the green Run icon from the toolbar or hit Ctrl+R to run the test
2. Add Assertions to the test:
Right Click the added HTTP Sampler and go to Add -> Assertions -> Response Assertion
Validate 200 response code for every request
From the Response Filed to Test select the Response Code radio button
From the Pattern Matching Rules select the Matches option
Click the Add button and add 200 value in the Patterns to Test field
Response code = 200 -> Pass
Response code different from 200 -> Fail
3. Adding Visual Reporting to the performance test:
Right Click the added HTTP Sampler and go to Add -> Listener -> View Results Tree
Click the View Results Tree listener from the left area
Run the test again (click the RUN icon from the toolbar or press Ctrl+R) and the test will appear in the list
Click on the executed request from the list in View Results Tree to see more details of the request
Green font color -> Pass; Red font color -> Fail
Clicking the result in the list reveals new data in the right area: Sampler result, Request and Response Data
Sampler result: shows overall data of the executed request, offer detail of the Thread Name, Latency, Size, Sample Count, Response Code, Message and Headers
Request: shows details of the HTTP request, type of the request, cookies, request headers etc
You can log only Errors or only Successes by clicking on the check boxes in the top-right area
Clean the list of runs by pressing the Ctrl+E shortcut
4. Adding Aggregate Reporting to the test
In order to get a better understanding of the request executed we can add a Aggregate Report to the HTTP Sampler
To do that Right Click HTTP Sampler and go to Add -> Listener -> Aggregate Report
Click on the Aggregate Report from the left area and execute the test again (click RUN icon form toolbar or press Ctrl+R)
In the Aggregate Report table you can see the test run results:
Right Click the Aggregate Report and go to Help from the menu for a full description of the table fields (also you can see what response time or throughput means from out earlier posts)
Same as for the View Results Tree you can export results to an external file, log only errors or successes in the table and in the external file
5. Exporting JMeter results data to an external CSV file
In the View Results Tree listener you can enter the name of the results file: I.e. results.csv
From the right side you can click on the Configure button to select what data to export in the CSV file
Now run the test again the a file named results.csv will be created containing the test results for each request
6. Change Threads and Loops for better performance tests
Click on the Thread Group
From the Number of Threads (users) field you can adjust the number of concurrent users executing the created request
Ramp-Up Period sets the delay between starting new threads (users), if set to 0, the test will try to ramp-up all the threads from the beginning of the test run
Loop Count sets the how many times each thread is going to execute the test
Also options when an error occurs may be set from the Thread Group
You can schedule a test run to execute between exact dates, for a preset interval and with a start up delay (seconds)
With this performance test configuration you can start having an initial view of how to create a performance test with Apache JMeter.
Here are the steps in images
1. Add a new Thread Group to the Test Plan 2. Add a new HTTP Sampler 3. Configure the GET HTTP Request 4. Add visual reports 5. Add response assertion for response code 6. Add in detail reporting 7. Configure performance test
In the next posts I will present details of performance testing expressions: response time, latency, hits per second, transactions per second, throughput, and size per second.
Throughput – indicates the number of transactions per second an application can handle, the amount of transactions produced over time during a test.
For every application there are lots of users performing lots of different requests. What you need to ensure is that your application meets the required capacity before it hits production or live. To ensure that, load and performance testing is the solution.
Choose different type of requests (frequent, critical, and intensive) and see how many pass successfully in an interval. To solve the problem you should try to simulate load through load testing tools like Load Focus (loadfocus.com) to pick a mix of scenario for load testing, define concurrent users, number of loops etc. The load test should be simulated with real live data (users, requests etc)
Throughput depends on different factors:
specifications of the host computer
processing overhead in the software
degree of parallelism both hardware and software support
Response Time measures the performance of an individual transaction or query. Response time is the amount of time from the moment that a user sends a request until the time that the application indicates that the request has completed. For example, let’s say you have API and you want to find how much time that API call takes to execute once requested, here you are measuring the response time. Response time is the total time it takes from when a user makes a request until they receive a response.
When doing a load or performance test you need to find out how is your application, website, API handling all the requests and how the response time increases with the load.
Response Time Load Testing
You can use cloud testing tools to run load and performance tests, tools likeLoadFocus (http://loadfocus.com)which is a – very easy to use and low-cost online service, has accurate results and is compatible with open source Apache JMeter.