Filed under Load Testing, Performance Testing, Test Automation, Uncategorized.

What are assertions ?

In case you are not familiar with the term, an assertion “is a test / a condition that must be tested to confirm conformance to a requirement”. Basically a test assertion is – a condition that compares the expected value with the actual value for a specified use case.

 

Why should I use assertions ?

Before we deliver a product, we need to run many test assertions to make sure the product meets the customer’s expectations. Also, you can check our intro blog post regarding how to run a load test with Apache JMeter.

 

 

Should we use assertions in the JMeter tests ?

Usually assertions are not used when we create the scripts for load or stress testing an application. Most of the tester that run
these kind of tests base their results only on the response status codes that the application under test returns. This might be a problem and might make the final results not accurate.

What are the cases in which not using an assertions may affect your final results ?

Not all the time a successfull response from the server (200 OK) means that the application responded as expected.
You may ask yourself and the a few cases are mentioned below:

  1. the application redirects to a maintaince page
  2. the application responds a 200 OK but sends an error message in the body of the response indicating a error message
  3. the application redirects to an error page and sends back a 200 OK with and error message indicating why the error returns

To avoid these kind of errors to affect your performance report is a very good idea to use assertions:

  • in GET/POST/PUT/PATCH methods
  • in API performance testing
  • in functional testing

Be aware that if your are creating a long running test or a test with a lot of requests is good not to use a lot of assertions as
they tend to consume quite a bit of memory which could make your JMeter tool crash and means starting the test again.

What type of assertions does JMeter support ?

There are several types of assertions that you can use in your test scripts.
The most used ones are:

  • Response Assertion“: the response assertion control panel lets you add pattern strings to be compared against various fields of the response
  • XPath Assertion“: the XPath Assertion tests a document for well formedness, has the option of validating against a DTD, or putting the document through JTidy and testing for an XPath (if that XPath exists, the Assertion is true)
  • HMTL Assertion“: the HTML Assertion allows the user to check the HTML syntax of the response data using JTidy.
  • XML Assertion“: tests that the response data consists of a formally correct XML document; it does not validate the XML based on a DTD or schema or do any further validation.
  • Compare Assertion“: can be used to compare sample results within its scope; either the contents or the elapsed time can be compared, and the contents can be filtered before comparison.
    Compare Assertion MUST NOT BE USED during load test as it consumes a lot of resources (memory and CPU). Use it only for either functional testing or during Test Plan debugging and Validation.
  • Size Assertion“: verifies that each response contains the right number of bytes in it; you can specify that the size be equal to,
    greater than, less than, or not equal to a given number of bytes

 

How much do assertions consume in terms of CPU and memory ?

If the test that your are using is a long running test or makes a very large number of requests you may encounter unexpected failures related to “out of memory” errors or similar.

In terms of resources the HMTL Assertion, XML Assertion or Compare Assertion consume a lot of resources so is a good idea to avoid using those in long running tests. The other assertions does not have a very bad impact on the resources so they can be used without a problem.

 

Which is the scope to which the assertions apply ?

There are 3 cases related to which scope an assertion applies to:

1. applies to all the child requests

Screen Shot 2016-04-11 at 11.05.53

 

2. applies to all the sibling requests

Screen Shot 2016-04-11 at 11.05.47

 

3. applies only to one specific request

Screen Shot 2016-04-11 at 11.05.38

 

How to use a “Response Assertion” in JMeter ?

Response Assertion is one of the most widely used assertion as it allows to verify the content of the response, status code or headers.

It can be configured to verify:

  • the different segments of the response like text (response body)
  • the document type like pdf, doc etc
  • the headers
  • the response code (2xx, 4xx)
  • response message (description of code) and response headers
  • tests the response against a pattern string which can be a plain text or a Perl5 regular expression; the specified pattern is matched against the selected segment of the response

 

Let’s start by adding a response assertion that expects that the response code is 200.

Screen Shot 2016-04-11 at 11.24.32

 

We will now add another assertion that expects that the text “wikipedia” is present in the response of the server.

 

Screen Shot 2016-04-11 at 11.26.17

 

Running the script will show that that the request succeeded without any problem.

Screen Shot 2016-04-11 at 11.26.34

 

Now we will change the text of the second assertion to “dummy_text_inexistent” and run the test again.

Screen Shot 2016-04-11 at 11.29.34

In the next tutorials we will show you how to use the rest of the assertions that are supported in JMeter.


You can also use LoadFocus to load test your websites, APIs or any web-app with thousands of users from AWS cloud locations.

 

Filed under Page Load Time, Website Speed Testing.

During Speed testing of your website or web application using LoadFocus (https://loadfocus.com/speedtests) on the Advice section you might get the tip that you need to enable compression like in the image below:

 

advice_enable_compression
The question is why and how do you do that ?

 

  • The response to the why is that enabling gzip compression can reduce the size of the transferred response by up to 80-90%;
    This has the following benefits:
  • significantly reduces the amount of time it takes to download the resources
  • reduces data usage for the client
  • improves the time to first render of your pages
  • a better user experience for your customers
  • a better monetization for your web application

You can enable gzip compression for CSS and JavaScript files.
Apart from these any text data or XML files will also benefit from gzip compression. Don’t use gzip for image or other binary files.

Image file formats supported by the web, as well as videos, PDFs and other binary formats, are already compressed; using gzip compression on them won’t provide any additional benefit.

By default all modern browsers automatically negotiate gzip compression for all HTTP requests that are done.
To help you enable gzip compression on your server you can check out the following project

https://github.com/h5bp/server-configs

that contains the sample configuration files for quite a few different servers.
After you’ve finishes to enable compression on your server run the LoadFocus speed test again. This time the tip regarding to Enable compression should say that everything is fine.

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.

Filed under Apache JMeter, Performance Testing.

In case you want to easily start Apache JMeter load testing tool on your MAC OS, here some things which you help you get started.

Once you downloaded Apache JMeter, just go to the PATH where you’ve dowloaded it. For example, I’ve downloaded JMeter in my Downloads folder. Here are full details on how to run your first load test with JMeter.

 

Steps to Add JMeter to the PATH environment variable:

  1. Open Command-Line (Terminal)
  2. Type jmeter and press Enter
  3. If JMeter is not in the PATH variable, you’ll get the following:
    -bash: jmeter: command not found

  4. In terminal, go to the bin folder, where you’ve downloaded JMeter
  5. Press pwd command to get the current working directory
  6. Update the PATH variable with the following command:
    export PATH=$PATH:/Users/user/Downloads/apache-jmeter-2.13/bin
  7. Now type jmeter in the terminal and Apache JMeter will start

This is what you should see in your terminal:

computer:bin user$ jmeter
-bash: jmeter: command not found
computer:bin user$ pwd
/Users/user/Downloads/apache-jmeter-2.13/bin
computer:bin user$ export PATH=$PATH:/Users/user/Downloads/apache-jmeter-2.13/bin
computer:bin user$ jmeter
org.apache.jmeter.testelement.TestPlan
org.apache.jmeter.control.gui.TestPlanGui