Filed under Load Testing, Website Speed Testing.

Get every month 5 free load tests and 5 free website speed tests to your Free plan by adding our code in your website.

Our Link Program lets you run 5 website load tests and 5 free website speed tests per month completely free!

Just add the LoadFocus link to your website to upgrade. Here is the link:

<a href="https://loadfocus.com/?utm_source=link" title="load testing tool" ></a>

Once the link is visible on your site, we’ll automatically give you 5 load tests and 5 website speed tests per month for free.

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.

Filed under Load Testing, Performance Testing, Test Automation, Uncategorized.

What are assertions ?

In case you are not familiar with the term, an assertion “is a test / a condition that must be tested to confirm conformance to a requirement”. Basically a test assertion is – a condition that compares the expected value with the actual value for a specified use case.

 

Why should I use assertions ?

Before we deliver a product, we need to run many test assertions to make sure the product meets the customer’s expectations. Also, you can check our intro blog post regarding how to run a load test with Apache JMeter.

 

 

Should we use assertions in the JMeter tests ?

Usually assertions are not used when we create the scripts for load or stress testing an application. Most of the tester that run
these kind of tests base their results only on the response status codes that the application under test returns. This might be a problem and might make the final results not accurate.

What are the cases in which not using an assertions may affect your final results ?

Not all the time a successfull response from the server (200 OK) means that the application responded as expected.
You may ask yourself and the a few cases are mentioned below:

  1. the application redirects to a maintaince page
  2. the application responds a 200 OK but sends an error message in the body of the response indicating a error message
  3. the application redirects to an error page and sends back a 200 OK with and error message indicating why the error returns

To avoid these kind of errors to affect your performance report is a very good idea to use assertions:

  • in GET/POST/PUT/PATCH methods
  • in API performance testing
  • in functional testing

Be aware that if your are creating a long running test or a test with a lot of requests is good not to use a lot of assertions as
they tend to consume quite a bit of memory which could make your JMeter tool crash and means starting the test again.

What type of assertions does JMeter support ?

There are several types of assertions that you can use in your test scripts.
The most used ones are:

  • Response Assertion“: the response assertion control panel lets you add pattern strings to be compared against various fields of the response
  • XPath Assertion“: the XPath Assertion tests a document for well formedness, has the option of validating against a DTD, or putting the document through JTidy and testing for an XPath (if that XPath exists, the Assertion is true)
  • HMTL Assertion“: the HTML Assertion allows the user to check the HTML syntax of the response data using JTidy.
  • XML Assertion“: tests that the response data consists of a formally correct XML document; it does not validate the XML based on a DTD or schema or do any further validation.
  • Compare Assertion“: can be used to compare sample results within its scope; either the contents or the elapsed time can be compared, and the contents can be filtered before comparison.
    Compare Assertion MUST NOT BE USED during load test as it consumes a lot of resources (memory and CPU). Use it only for either functional testing or during Test Plan debugging and Validation.
  • Size Assertion“: verifies that each response contains the right number of bytes in it; you can specify that the size be equal to,
    greater than, less than, or not equal to a given number of bytes

 

How much do assertions consume in terms of CPU and memory ?

If the test that your are using is a long running test or makes a very large number of requests you may encounter unexpected failures related to “out of memory” errors or similar.

In terms of resources the HMTL Assertion, XML Assertion or Compare Assertion consume a lot of resources so is a good idea to avoid using those in long running tests. The other assertions does not have a very bad impact on the resources so they can be used without a problem.

 

Which is the scope to which the assertions apply ?

There are 3 cases related to which scope an assertion applies to:

1. applies to all the child requests

Screen Shot 2016-04-11 at 11.05.53

 

2. applies to all the sibling requests

Screen Shot 2016-04-11 at 11.05.47

 

3. applies only to one specific request

Screen Shot 2016-04-11 at 11.05.38

 

How to use a “Response Assertion” in JMeter ?

Response Assertion is one of the most widely used assertion as it allows to verify the content of the response, status code or headers.

It can be configured to verify:

  • the different segments of the response like text (response body)
  • the document type like pdf, doc etc
  • the headers
  • the response code (2xx, 4xx)
  • response message (description of code) and response headers
  • tests the response against a pattern string which can be a plain text or a Perl5 regular expression; the specified pattern is matched against the selected segment of the response

 

Let’s start by adding a response assertion that expects that the response code is 200.

Screen Shot 2016-04-11 at 11.24.32

 

We will now add another assertion that expects that the text “wikipedia” is present in the response of the server.

 

Screen Shot 2016-04-11 at 11.26.17

 

Running the script will show that that the request succeeded without any problem.

Screen Shot 2016-04-11 at 11.26.34

 

Now we will change the text of the second assertion to “dummy_text_inexistent” and run the test again.

Screen Shot 2016-04-11 at 11.29.34

In the next tutorials we will show you how to use the rest of the assertions that are supported in JMeter.

You can also use LoadFocus to load test your websites, APIs or any web-app with thousands of users from AWS cloud locations.

 

Filed under Page Load Time, Website Speed Testing.

During Speed testing of your website or web application using LoadFocus (https://loadfocus.com/speedtests) on the Advice section you might get the tip that you need to enable compression like in the image below:

 

advice_enable_compression
The question is why and how do you do that ?

 

  • The response to the why is that enabling gzip compression can reduce the size of the transferred response by up to 80-90%;
    This has the following benefits:
  • significantly reduces the amount of time it takes to download the resources
  • reduces data usage for the client
  • improves the time to first render of your pages
  • a better user experience for your customers
  • a better monetization for your web application

You can enable gzip compression for CSS and JavaScript files.
Apart from these any text data or XML files will also benefit from gzip compression. Don’t use gzip for image or other binary files.

Image file formats supported by the web, as well as videos, PDFs and other binary formats, are already compressed; using gzip compression on them won’t provide any additional benefit.

By default all modern browsers automatically negotiate gzip compression for all HTTP requests that are done.
To help you enable gzip compression on your server you can check out the following project

https://github.com/h5bp/server-configs

that contains the sample configuration files for quite a few different servers.
After you’ve finishes to enable compression on your server run the LoadFocus speed test again. This time the tip regarding to Enable compression should say that everything is fine.

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.

Filed under Apache JMeter, Performance Testing.

In case you want to easily start Apache JMeter load testing tool on your MAC OS, here some things which you help you get started.

Once you downloaded Apache JMeter, just go to the PATH where you’ve dowloaded it. For example, I’ve downloaded JMeter in my Downloads folder. Here are full details on how to run your first load test with JMeter.

 

Steps to Add JMeter to the PATH environment variable:

  1. Open Command-Line (Terminal)
  2. Type jmeter and press Enter
  3. If JMeter is not in the PATH variable, you’ll get the following:
    -bash: jmeter: command not found
  4. In terminal, go to the bin folder, where you’ve downloaded JMeter
  5. Press pwd command to get the current working directory
  6. Update the PATH variable with the following command:
    export PATH=$PATH:/Users/user/Downloads/apache-jmeter-2.13/bin
  7. Now type jmeter in the terminal and Apache JMeter will start

This is what you should see in your terminal:

computer:bin user$ jmeter
-bash: jmeter: command not found
computer:bin user$ pwd
/Users/user/Downloads/apache-jmeter-2.13/bin
computer:bin user$ export PATH=$PATH:/Users/user/Downloads/apache-jmeter-2.13/bin
computer:bin user$ jmeter
org.apache.jmeter.testelement.TestPlan
org.apache.jmeter.control.gui.TestPlanGui

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.

Filed under Apache JMeter, Load Testing, Performance Testing, Stress Testing.

Overview

We are going to present how to use Apache JMeter (open source load testing tool) in order to perform basic load and stress testing on your web application. We are going through each step and show how to use JMeter’s GUI (graphical user interface) in order to build a test plan, configure load tests, run test and inspect the results.

Apache JMeter is an open source Java application, so it can run on any OS (operating system) which supports JAVA 6 or later. JMeter is a load testing (performance testing) tool designed to run tests on any given URI endpoint, with configurable number of concurrent clients (users) and other useful test configurable metrics and inspect results. Test results can be viewed in charts, tables or can be exported in CSV or XML formats.

Prerequisites

In order to start using JMeter you need a machine that you can install and run JMeter on. By default JMeter runs as a GUI application, which requires more resources than a command line based application. In order to perform the load tests faster and without bottlenecks you can run JMeter in the non-graphical mode (details on how to start JMeter in non-graphical mode in the next post)
Please note that results can differ based on the web server where the web application is hosted. Factors that we need to take into account are:

  • system resources CPU, RAM Memory of the web server
  • system resources CPU, RAM Memory of the machine where JMeter is installed (from where the tests are executed)
  • location of the web server between the load test injector and the web server where the application is hosted (which affects the latency of the tests – use a cloud load testing tool in order to easily run load tests from different locations)

Install JMeter

Apache JMeter is very easy to install. Depending on the OS, there are different ways on how to do this.

In order to run JMeter, here is what you need to install:

Make sure that after installing JAVA you add it to the $PATH environment variable. Here you can find how to add the JAVA to the $PATH environment variable for every OS .

To start JMeter on Windows OS you need to go to the folder where you unarchived JMeter binaries and just run $JMETER_HOME/bin/jmeter.bat batch file. Please see here how you can run JMeter on Mac.

So, once all is installed and JMeter is running you can start creating your first load test. For this demo we are going to use Apache JMeter 2.11 on Mac.

Build Test Plan

In order to create a load test, add a Thread Group, in order to configure the thread properties (load test configuration):

apache jmeter test plan

Start by adding a Thread Group to your Test Plan by following the below steps:

  • Select the Test Plan on the left side, and Right-click it
  • Hover to Add -> Threads (Users) -> Thread Group
  • Click on Thread Group
apache jmeter thread group

Configure the following properties which will be used to run your load test:

  • Number of threads (users): The number of concurrent clients/users JMeter is going to simulate. Set it to 10.
  • Ramp-up Period: How long JMeter should take to get all the threads started. Set it to 10.
  • Loop Count: Number of times to perform the test case. Set it also to 10.
apache jmeter thread group configuration

Now we add a HTTP Request Sampler which represents an individual request made by each thread.
To do this Add a HTTP Request Sampler to your Thread Group by following the below steps:

  • Select on the Thread Group on the left side, then Right-click it
  • Hover (mouse over) to Add -> Sampler -> HTTP Request
  • Click on HTTP Request
apache jmeter http sampler

In the HTTP Request Sampler, under Web server, set the URL in the Server Name or IP field (website URL i.e www.example.com or public IP address).

If you want to execute more than one HTTP Request by each thread (user), simply duplicate the HTTP Request Sampler and update the Server Name or IP. Here are the steps to duplicate the HTTP Request Sampler.

  • Select HTTP Request Sampler, then Copy it (Cmd/Control + C)
  • Select Thread Group, then Paste it (Cmd/Control + V)

Note: In our JMeter test we will continue to use only one HTTP Request Sampler.

Add HTTP Cookie Manager (optional)

If your web application uses Cookies, JMeter allows to add cookies which will be sent with each HTTP request. You can do that simply by adding the HTTP Cookie Manager by following these steps:

  • Select Thread Group, then Right-click it
  • Hover (mouse-over) Add -> Config Element -> HTTP Cookie Manager

 

Add HTTP Header Manager (optional)

In case you need to send specific headers with every request, JMeter allows to add headers by adding the HTTP Header Manager by following these steps:

  • Select Thread Group, then Right-click it
  • Hover (mouse-over) Add -> Config Element -> HTTP Header Manager

Add View Results in Table

You need to use JMeter’s Listeners in order to view the output of a load or stress test. One of the most used Listeners is the View Results in Table, which we’ll use in the post and here is how to add it to your Test Plan:

  • Select Thread Group, then Right-click it
  • Hover (mouse-over) Add -> Listener -> View Results in Table
apache jmeter results listener

Run your Test Plan

After installing everything needed in order to start Apache JMeter and after configuring the Test Plan, you can start running your test.

First, save the test plan with the .jmx extension.
Run the test by pressing the green arrow button in the toolbar or by pressing Cmd/Control + R keys.

apache jmeter results in tree

You’ll starting seeing results in the View Results in Table, important columns are Sample Time (ms), Status, Bytes and Latency. To understand better the errors and responses you can check out this post.

  • Sample Time (ms): Represents the number of milliseconds that the server took to serve the request (response + latency)
  • Latency (ms): Represents the number of milliseconds that elapsed between when JMeter sent the request and when an initial response was received

At the bottom of the View Results in Table, you can see the Average Sample Time in ms and the Standard Deviation.
According to the results, after running 100 samples the Average Sample Time in milliseconds was 133 ms. Latency is going to be affected by the geographic distance between the place from where the request is made to the place where the web server is hosted.

apache jmeter 100 samples

So, no error occurred on our test when running the 10 requests X 10 threads (users) = 100 samples, and the Sample Time was low 133 ms.

Increase the Load by changing the Threads (users) values

In order to make our load test more realistic, let’s change the Threads (Users) field to 300.
Clear the previous JMeter results by pressing Cmd/Control + E or by clicking the Clear icon in JMeter’s toolbar.

Start the test, and check the results again. As you can see, the Sample Time increased almost by 50% to 178 ms, however no errors occurred while running our load test with JMeter.

apache jmeter 3000 samples

Also, while running the tests, server side monitoring will be helpful, looking at metrics like CPU or Memory. In Windows you can monitor these stats by opening the PerfMon tool or in Linux by typing the top command in terminal.

You can extend the load test up to 1000 threads (users), to check how your system behaves and discover potential bottlenecks, memory leaks, performance issues. LoadFocus.com is a cloud load testing service and you can run load tests with thousands of threads (users).

Conclusion

JMeter is a useful load testing tool, which helps you find load testing issues, discover bottlenecks and increate performance of your web application. Feel free to experiment different values for the load test in order to understand how your web server behaves under heavy load. In order to add more plugins to your default JMeter installation, you can check the JMeter plugins add-ons or check here how to add JMeter plugins to JMeter.

In order to run load tests with thousands of threads (users) and to run load tests from different world locations, you can use LoadFocus.com which offers a cloud load testing service, easy to use, which keeps all your tests and test runs in the history automatically.

Filed under Product Development.

Entrepreneurs always ask themselves which is the best way to grow a startup, paid ads or organic user search.
We believe that organic customers are a good way of identifying the product market fit of your MVP. Before investing into any paid ads you must first have a product, a product that would be loved and shared by your first customers.
Once you get the product market fit and you get some traction, you may find it not that risky to actually start spending money on paid advertising (paid ads, paid videos or paid posts).
Organic Growth
Here are a few ways to reach to you customers without spending startup money:
  • optimise your application for the search engines (SEO optimisation)
  • create a blog related to your startup activity (related to future product features, tech ideas)
  • create a Q/A are on your website to dynamically generate a FAQ
  • create guest blog posts and offer to post them on other websites without asking for anything in return
  • HackerNews is a good way to share your new idea to the world and get
Some of the metrics we consider are Bounce Rate, Daily Page Views per Visitor, Daily Time on Site. Creating goals for important actions on your website and segmenting the audience is something you may need to consider. Also to validate that you have a product market fit, 90 day retention numbers give you a good overview.
Useful growth resources that helped us can be found here: https://growth-hacking.zeef.com/david.arnoux

 

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.

Filed under Load Testing, Page Load Time, Performance Testing, Test Automation, UI Testing.

Your website or API is going to be tested by our testing team.

LoadFocus.com released a new pricing plan – the Business Plan. The big plus with this new plan is that we are going to provide every month a performance and a speed assessment of your web application or API.
We are going to provide a load testing assessment:

  • potential performance bottlenecks
  • errors and issues when a large number of users access your website
  • boundary testing to understand which are the limits of your system, or how many users/clients can access you website in parallel
  • full performance metrics: response time, latency, hits per second, throughput per second, percentiles, standard devition
  • charts showing how the applications behaves over a bigger period

We are going to provide a website speed testing assessment:

  • response time and performance score
  • potential errors and W3C validation issues
  • hand-on advice on how to improve the speed on your website: caching, minifying, compressing assets, resizing images

Overall, even if you have a website, a WordPress blog or you just want to check how your APIs behave, we’re going to provide valuable feedback regarding potential issues, things on how to improve them, hand-on advice on best practices at the moment on the web (client/server side). This will help you understand better how your application works, and how your users experience it.

Click here and checkout the Business Plan created specially for you, after you’ll try it one, you’ll never want to cancel. Check it now.

 

LoadFocus.com is a Cloud Testing Platform used for Load and Performance Testing for Websites and APIs and Website Speed Testing with Analytics.