Filed under Amazon Web Services, Apache JMeter, Cloud Services, Load Testing.

LoadFocus launches today the cloud load and performance testing service powerful enough to stress a web application or API with thousands of concurrent users.

Based on Apache JMeter open source project and hosted in the cloud, LoadFocus is designed for software developers, QA (Quality Assurance) engineers and product owners to run accurate load and performance tests with minimum costs.

load testing results

As we can see nowadays there are numerous web applications and APIs that appear everyday which requires load and performance assessments.

However, the existing commercial and professional load testing solutions are too expensive and too complex to meet real world requirements. The open-source load testing tools, like JMeter, are limited in terms of scalability for the requirements of enterprise and high-traffic web applications.

Once you sign up with LoadFocus.com, the only cost you will have is running the injector machines (Amazon EC2 instances).
The cost depends on the type of machine you’re using and the duration of the load test.

LoadFocus.com provides unlimited testing capacity, interactive real-time reporting and comprehensive result analysis integrated with mobile devices.

Click here to try it now!

Filed under Load Testing, Performance Testing.

Let’s say that you want to access a web service or a web page. The response time that you can see with different tools is a sum of the server processing time of the request and the delay involved for your request to reach to server and back.

This is getting more important when remote data centers are hosting the web page or web service.

Latency + Processing Time = Response Time

You can have a look at the latency vs. response time chart as you need to understand these different aspects of the system, and it is important for all the parties involved to know the difference between the two when they are doing a performance analysis.

You can use LoadFocus in order to stress test your application or API, inspect the latency and response time for all requests.

latency and response time

Filed under Apache JMeter, Load Testing, Performance Testing.

In order to create a new Cloud Apache JMeter Load Test with Load Focus all you need to do is follow the steps below:

1. Select New JMeter test

load test

2. Enter the name of the test and upload the Apache JMeter test script files (.jmx)

Here are the steps in more detail

– log into your account from https://loadfocus.com/login
– click on the Apache JMeter -> New Test menu item from left side of the https://loadfocus.com/admin web page
– on the https://loadfocus.com/newjmeter webpage insert a Test Run Name (i.e. First JMeter Script GET Load Test for www.example.com)
– upload your Apache JMeter .jmx script file that will execute the load test
– select the Location of the injector machine and the type of the machine
– you can upload a new file to execute in parallel a new .jmx script file

You can either Save the configuration or directly Execute the test.

If you are not familiar with Apache JMeter you can check the Apache JMeter Performance Testing tutorial or run a General Test with Load Focus.

Filed under Cloud Services, Load Testing, Performance Testing.

It is very easy to create a New General Load Test configuration, all you need to do is follow the steps below:

Click on the General Test Button

load test

Enter a new name for your General Test

Screen shot 2013-07-24 at 3.01.07 AM

Select the request type GET or POST and details for injector machines

Screen shot 2013-07-24 at 3.03.17 AM

Add Load Test details: threads, loops, URL and delay:

Screen shot 2013-07-24 at 3.03.49 AM

Here are the steps in more details:

  • log into your account
  • click on the General Test -> New Test from the https://loadfocus.com/admin URL
  • on the https://loadfocus.com/newgeneral URL enter the Test Run Name (i.e. First GET Load Test for www.example.com)
  • click on the “New Machine” button from the right side of the page
  • select the type of the request (GET or POST) from the New Machine area
  • you may select the geographical location of the injector machine – this will represent the location from where the users will hit your application
  • you may select also the Amazon Web Services machine type used for the test – the more concurrent users added the bigger the instance should be
  • GET Request, Request Label: GET www.example.com, URL: www.example.com, Threads (concurrent users): 100, Delay (seconds) 1, Loop count: 3
  • This configuration will execute 300 sample requests
  • you can duplicate this type of request to add other 100 thread x 3 loops in parallel
  • also you can add another type of requests: you can select a different location or type for the injector machine and a different number of threads, delay and loops

You can either Save the configuration or directly Execute the test.

Filed under Page Load Time, Performance Testing, Website Speed Testing.

Try the free website speed testing tool https://loadfocus.com/website-speed-testing, gives you page load time, performance score, insights tips, numbers and sizes of resources for CSS, JS, images files.

Performance testing the page load time of a web application can be a little tricky but pretty useful for simulating what the user experience of your customers.

The following solution uses the PhantomJS headless browser for measuring the needed timing information,
Google charts for visualizing the results. The solution can also be used pretty much as it in the continuous integration
environment based on Jenkins or any other system.

For what can I use this solution ?

  • page load time testing
  • size and number of resources retrieved for URL call
  • load time for each resource

The solution described below is composed of the following scripts and executables:

1. PhantomJS headerless browser
2. load.sh script: script used for running the starting PhantomJS and the performance measuring script
3. loadreport.js: script used for retrieving the timing information needed for performance measuring

Steps to make the solution work for you (Linux OS as this is what I have , similar on MacOS or Windows).

1. Install PhantomJS

  • browse to http://phantomjs.org/download.html
  • open a terminal window and issue
     wget https://phantomjs.googlecode.com/files/phantomjs-1.9.1-linux-x86_64.tar.bz2 
  • wait for PhantomJS to be downloaded and issue the command ”
     tar -xvjpf phantomjs-1.9.1-linux-x86_64.tar.bz2 

  • now PhantomJS is installed

2. Create the load.sh script and add the code from below (or git clone fromhttps://github.com/loadfocus/pageloadtime.git)

3. Create the loadreport.js script and add the code from below (or git clone from https://github.com/loadfocus/pageloadtime.git)

4. Running the load.sh script

  • got to the directory where the load.sh script is located and issue the following command in the terminal:
  • run script:
     sh load.sh url_to_be_measured number_of_requests_to_be_made path_to_phantomjs 
  • example:
 sh load.sh http://www.google.com 20 /home/user/tempphantom/phantomjs-1.9.1-linux-x86_64/bin/phantomjs 

5. Visualizing the results
– after the test finishes in the same directory you will see:
results.html which contains the chart with the results
reports directory which contains the json file with the actual results
– open results.html in the browser of your choice

The results should look like below:

load_graph_loadfocus slowest_resource_loadfocus largest_resource_loadfocus resource_loaded_with_size_loadfocus

Filed under Apache JMeter, Performance Testing.

Apache JMeter is an open source load and performance tool written in Java and it’s available on almost any OS. This is an intro post about Apache JMeter tool and I’m trying to point out how it can help you in your daily Quality Engineer job.
You can use Apache JMeter for performance, load, stress testing and memory-leak testing for accurate information. Also, JMeter can be used for understanding the performance of a web application, service, end-point etc.

I will present how to use Jmeter for creating a basic load test for a web application, below are Jmeter test configuration from the most basic one (without assertions and reporting), to the most advanced which includes Response Assertions, Reporting, Exporting Data and Chart Results and so on.

1. GET HTTP Request for http://www.example.com/
Description:

  • no assertions
  • no reporting

Steps:

  • Start Jmeter
  • Add a new Thread Group to the Test Plan
  • Right Click on Test Plan and go to Add -> Threads (Users) -> Thread Group
  • Add a new HTTP GET Request to the Thread Group
  • Right Click on Thread Group and go to Add -> Sampler -> HTTP Request
  • This HTTP Request Sample is going to contain the configuration for your web request
  • Configure the GET HTTP Request Sampler
  • In order to make a performance test you need to configure the HTTP Sampler request with the necessary data
  • Change the name of the HTTP Request with a more descriptive one: GET Request www.example.com
  • In the Web Server -> Server Name of IP add the URL of the website, web application that you want to test: www.example.com
  • Do not add the protocol in front of the URL (I.e http://www.example.com or https://www.example.com), the protocol will be added only in the Protocol field
  •  Add the Port Number of the web application: 80 for HTTP or 443 for HTTPS
  • Add the protocol in the Protocol [http] field: http
  • Select the HTTP Method from the Method drop down: GET
  • Add the Path of the request
  • For testing the home endpoint you can add only a / character in Path field
  • Or you can add a full URL: /index.html?query1=1&query2=2
  • Follow Redirects and Keep Alive checkboxes should be selected by default, if not please check them
  • Click the green Run icon from the toolbar or hit Ctrl+R to run the test

2. Add Assertions to the test:

Steps

  • Right Click the added HTTP Sampler and go to Add -> Assertions -> Response Assertion
  • Validate 200 response code for every request
  • From the Response Filed to Test select the Response Code radio button
  • From the Pattern Matching Rules select the Matches option
  • Click the Add button and add 200 value in the Patterns to Test field
  • Response code = 200 -> Pass
  • Response code different from 200 -> Fail

3. Adding Visual Reporting to the performance test:

Steps

  • Right Click the added HTTP Sampler and go to Add -> Listener -> View Results Tree
  • Click the View Results Tree listener from the left area
  • Run the test again (click the RUN icon from the toolbar or press Ctrl+R) and the test will appear in the list
  • Click on the executed request from the list in View Results Tree to see more details of the request
  • Green font color -> Pass; Red font color -> Fail
  • Clicking the result in the list reveals new data in the right area: Sampler result, Request and Response Data
  • Sampler result: shows overall data of the executed request, offer detail of the Thread Name, Latency, Size, Sample Count, Response Code, Message and Headers
  • Request: shows details of the HTTP request, type of the request, cookies, request headers etc
  • You can log only Errors or only Successes by clicking on the check boxes in the top-right area
  • Clean the list of runs by pressing the Ctrl+E shortcut

4. Adding Aggregate Reporting to the test

Steps

  • In order to get a better understanding of the request executed we can add a Aggregate Report to the HTTP Sampler
  • To do that Right Click HTTP Sampler and go to Add -> Listener -> Aggregate Report
  • Click on the Aggregate Report from the left area and execute the test again (click RUN icon form toolbar or press Ctrl+R)
  • In the Aggregate Report table you can see the test run results:
  • Right Click the Aggregate Report and go to Help from the menu for a full description of the table fields (also you can see what response time or throughput means from out earlier posts)
  • Same as for the View Results Tree you can export results to an external file, log only errors or successes in the table and in the external file

5. Exporting JMeter results data to an external CSV file

Steps

  • In the View Results Tree listener you can enter the name of the results file: I.e. results.csv
  • From the right side you can click on the Configure button to select what data to export in the CSV file
  • Now run the test again the a file named results.csv will be created containing the test results for each request

6. Change Threads and Loops for better performance tests

Steps

  • Click on the Thread Group
  • From the Number of Threads (users) field you can adjust the number of concurrent users executing the created request
  • Ramp-Up Period sets the delay between starting new threads (users), if set to 0, the test will try to ramp-up all the threads from the beginning of the test run
  • Loop Count sets the how many times each thread is going to execute the test
  • Also options when an error occurs may be set from the Thread Group
  • You can schedule a test run to execute between exact dates, for a preset interval and with a start up delay (seconds)
  • With this performance test configuration you can start having an initial view of how to create a performance test with Apache JMeter.

    Here are the steps in images

    1. Add a new Thread Group to the Test Plan jmeter1
    2. Add a new HTTP Sampler jmeter2
    3. Configure the GET HTTP Request jmeter3
    4. Add visual reports jmeter4
    5. Add response assertion for response code jmeter5
    6. Add in detail reporting jmeter6
    7. Configure performance test jmeter7

In the next posts I will present details of performance testing expressions: response time, latency, hits per second, transactions per second, throughput, and size per second.

Please share if you like. Thank you.



Filed under Load Testing, Performance Testing.

Throughput – indicates the number of transactions per second an application can handle, the amount of transactions produced over time during a test.

For every application there are lots of users performing lots of different requests. What you need to ensure is that your application meets the required capacity before it hits production or live. To ensure that, load and performance testing is the solution.

Choose different type of requests (frequent, critical, and intensive) and see how many pass successfully in an interval.
To solve the problem you should try to simulate load through load testing tools like Load Focus (loadfocus.com) to pick a mix of scenario for load testing, define concurrent users, number of loops etc. The load test should be simulated with real live data (users, requests etc)

Throughput depends on different factors:

  • specifications of the host computer
  • processing overhead in the software
  • degree of parallelism both hardware and software support
  • types of transactions being processed