software release checklist

Integrations Mobile Testing Product Development Test Automation UI Testing

2 minutes read

In this new agile world, you want to improve your application every day by releasing early and often.

How do you manage to release often without major issues?

Create a checklist for your software releases, a list of steps to blindly follow every time release a new piece of software.

Here is an example of what steps your list should contain:

  • Is the code to be released tested? Tested should include the following types of tests: functional testing, load testing, speed testing, accessibility testing, exploratory testing, API testing, Unit testing and others.
  • Are all branches code reviewed and merged?
  • Is the code to be released tagged with the correct version?
  • Do we have approval from all teams to release this piece of functionality in front of live customers?
  • Has it been tested on local and staging environments?
  • Do we have a rollback plan in case something goes wrong? if yes, which are the steps to rollback?
  • Are all assets minified and optimized for production release?
  • Are All automated and manual tests passing on local and staging environments?
  • Do we have someone on-call?

This list can go on and on, by I would suggest that every team should have a similar checklist.

This checklist should be reviewed and updated every 1-2 weeks (doesn’t matter if you do Scrum or Kanban), most likely at the team retrospective meetings.

Create a Software Release Checklist

Make sure you starting working on this list and share it with your team as soon as possible. It will help you discover a lost of issues or bugs earlier on.

How does your checklist look like?

Written by Bogdan Vazzolla.

LoadFocus is a cloud performance, load and stress testing tool which provides the infrastructure and the ability to run all these tests with thousands of concurrent users, from multiple cloud locations, in less than a few minutes, keep history of the results, compare different runs to inspect performance improvements or performance degradation.

load testing online

Load Testing Performance Testing Stress Testing Test Automation

2 minutes read

First goal of load testing is “do not lose customers”, the second goal of load testing is… check the first goal.

Leaving movie quotes aside, load testing helps you understand how your application behaves under both normal and peak conditions, and discover what is causing the degradation. But truly, the main goal of load testing is not to lose customers.

You don’t do load testing for the sake of doing load testing.

You do it so that when loads of customers enter your Website, Mobile App, E-Commerce platform, you’ll be ready to handle all those visitors, and make their experience as seamless as it can be. That’s the main purpose of load testing.

Of course you have to talk to your team members, sometimes talk to other teams, understand KPIs and SLAs and how your customers use your app and from where is the world they use it.

Now that the Black Friday season is almost due, think about how did your app behave, was it anything else you could’ve done better to handle all the load in this busy period.

The idea behind load testing and testing in general is that you have to do it all year round, not only before the sale season, or before a huge campaign.

You never know when your app/website will be on the first page of YCombinator or ProductHunt.
So be ready to handle the spike of visitors, this event may happen only once and it may help you take your website / app to the next level, so define your objectives clearly, understand all expectations that you set in place, and start load testing.

Load testing will help you reduce the cost of failure, improve scalability, but most of all, improve customer satisfaction.

Written by Bogdan Vazzolla.

LoadFocus is a cloud performance, load and stress testing tool which provides the infrastructure and the ability to run all these tests with thousands of concurrent users, from multiple cloud locations, in less than a few minutes, keep history of the results, compare different runs to inspect performance improvements or performance degradation.

record jmeter tests in the browser

Apache JMeter Load Testing Performance Testing Stress Testing

< 1 minute read

It’s so easy to record a load test for your website, just add to Chrome Browser the JMeter in the Cloud Chrome Extension and you’ll be able to record and run a load in less than 60 seconds.

Let’s have a deeper look how to do this.

Steps to Record and Run Load Tests

  1. Add to Chrome the LoadFocus | JMeter Load Testing in the Cloud extension
  2. Open the extension and Sign In / Up (free account available)
  3. Click Record JMeter Test button (choose which requests you want to record by selecting the checkboxes below the Record button)
  4. Start navigating to your website and to your website pages (during this time your requests are recorded in a new load test)
  5. Stop the recording and view the generated load test

This is it, less than 60 seconds to record a load test.

No need to install or start Apache JMeter, recording is done in the browser, and running the test in the cloud.

Now you can:

More detail can be found in this video.

Install Chrome Extension from here.

Written by Bogdan Vazzolla.

LoadFocus is a cloud Performance, load and stress testing tool which provides the infrastructure and the ability to run all these tests with thousands of concurrent users, from multiple cloud locations, in less than a few minutes, keep history of the results, compare different runs to inspect performance improvements or performance degradation.

record jmeter tests chrome extension

Apache JMeter Chrome Extension Load Testing Performance Testing

5 minutes read

If you need to run a JMeter Load Test and it’s a burden to download Apache JMeter locally on your machine, open it and manually create your JMeter test file, now there is a much easier way to record browser interactions with JMeter.

The LoadFocus | JMeter Load Testing in the Cloud Chrome Extension for recording JMeter test script files (.JMX files) from LoadFocus is a free tool to easily record and run load tests in the cloud. Basically it allows to run JMeter tests without having JMeter installed, for free, everything with just a few clicks.

Chrome Extension supports:

  • recording JMX, JSON files (HTTP/S Requests, JavaScript (JS), CSS, Cookies, Images or Other requests)
  • downloading locally or uploading the test files to LoadFocus
  • running a load test in the cloud with thousands of users from multiple locations

It’s very easy to get started with recording JMeter tests with JMeter Load Testing Chrome extension from LoadFocus.

Let’s have a look at the steps in order to record and generate a JMeter test file.

Getting Started

  1. Go to Google App Store and Add to Chrome the “LoadFocus | JMeter Load Testing in the Cloud”
  2. Sign In into your LoadFocus account or create an account for free

If you can’t access the Google App Store, just download the CRX file (see link at the end of this post) and follow the instruction from here on how to install a Chrome Extension manually.

Record JMeter test

recording jmeter test
Recording your JMeter test
  1. Click the “Record JMeter test” button and the “Recording” text should be visible in the top right corner of your browser, in the extension, waiting for the first request.
  2. Start your test workflow by loading your website (i.e. enter https://example.com in the browser’s URL bar)
  3. Navigate to any webpage you want to be present in your JMeter test (requests are recorded automatically in your test)
  4. Once finished you can stop the JMeter Recording (the extension icon should update to Done)
jmx recorded test file
Recorded JMeter test

If you want to start again just click the “X” button next to the test name or click the “Start again” button and your current test will be discarded and you can start recording another JMeter performance test.

Also, you can select which requests you want to record? You can choose between: HTTP/S Requests, JavaScript (JS), CSS, Cookies, Images or Other. This allows you to filter which type of requests you want to be present in your JMX JMeter recording.

Edit the Recorded JMeter test

Click the Edit button next to the test name to edit the recorded JMeter test.

edit jmeter recorded test
Edit Recorded test by Inserting, Duplicating or Removing Requests
  1. You can edit the automatically generated test name (i.e. Test_Rec_Dec_17_2020_6_37_22_AM)
  2. You can expand and change the order of the requests by clicking and dragging up or down on the left icon of a request
  3. Click on the context menu for a request to Insert, Duplicate or Remove the selected request
  4. Click on the request and expand by click on the left arrow to:
    • change anything from the name of the label, URL, method, request type, headers etc.
  5. Export the amended test to JMX or JSON by clicking the top buttons Export JMX and Export JSON

Download the Recorded JMeter test (.JMX file)

  1. Click the “Download JMeter test” button
  2. Select which domains you want to include in your .JMX file (this option is visible only if your test has recorded requests coming from multiple domains)
  3. Download locally the generate JMeter test script file (.JMX file)

Run Recorded JMeter test in the cloud

To run the JMeter test in the cloud, you’ll need an account with LoadFocus, see plans here.

Run JMeter Load Test from multiple location with JMeter Load Testing
  1. Click the Run JMeter Test button and this will automatically start running the generated load test in the cloud using JMeter Cloud Load Testing Service.
  2. Or you can click the “Upload JMeter Test” button and upload the generated JMX file
  3. Select number of Virtual Users, Duration, Ramp Up Period and Ramp Up Steps
  4. Click Execute and view live-results from the automatically generated load test

As you can see, we’ve managed to generate a JMeter load test, edited the recorded requests without any need to install JMeter on our local machine. This comes in very handy when you need to generate a load test based on predefined workflows.

Of course you can download the JMeter recorded test script file locally and start updating your test with custom headers, load configuration from CSV files and add different samplers to your test.

With LoadFocus you can upload a CSV configuration file along with your JMX file.

View Previous JMeter Load Tests Results

Previous Test Runs with LoadFocus

In order to view the previous JMeter load test just click the “View Previous Test Runs” button and you can see a list with your previous test runs.

To go in-depth, click on the “View details” button and a short snapshot of the test will be visible, with an option to “View Details” which will take you to the main website where you can have a better understanding of the performance results, with interactive charts, errors and log files to inspect and download locally.

Hope this Chrome Extension is helpful and subscribe below in order to receive new product updates.

JMeter Load Test Results

Install Chrome Extension from here.

Written by Bogdan Vazzolla.

LoadFocus is a cloud Performance, load and stress testing tool which provides the infrastructure and the ability to run all these tests with thousands of concurrent users, from multiple cloud locations, in less than a few minutes, keep history of the results, compare different runs to inspect performance improvements or performance degradation.

writing testing articles

Apache JMeter Performance Testing Test Automation UI Testing

< 1 minute read

Wanted to help you make the most of our blogs and group in this article the list of blog posts we consider to be the most important, viewed and useful:

Here is the list:

How to Build, Schedule and Execute Performance Tests Before Black Friday and Cyber Week

Cloud JMeter Load Testing at Scale

How to Save Apache JMeter Results to a CSV file or XML file?

What is Throughput in Performance Testing?

How to start Apache JMeter in non-graphical (non-GUI) mode?

How to use User Defined Variables in Apache JMeter

How to Take Screenshots with Puppeteer using Headless Chrome Browser

Best Practices for Implementing E2E (End-to-End) UI Tests

Errors and Response Codes in Load Testing

Top 15 reasons why load testing is important to your website

Hope this list will structure the content a bit better and and get you closer to mastering testing and load testing.

Written by Bogdan Vazzolla.

LoadFocus is a Cloud Performance and Load Testing tool which provides the infrastructure and the ability to run all these tests with thousands of concurrent users, from multiple cloud locations, in less than a few minutes, keep history of the results, compare different runs to inspect performance improvements or performance degradation.

Load Test App for Black Friday

Automated Website Testing & Monitoring Cloud Services Load Testing Performance Testing Stress Testing Test Automation

8 minutes read

Well into the pandemic and with the E-commerce boom rapidly accelerating due to people needing to stay inside, you’ll need to make sure that your website and mobile applications are performing well under high load for the holiday shopping season and especially for Black Friday and Cyber Monday sale events.

We have written a handbook, in order to help businesses from small to medium and large with this assessment, handbook (or testing plan) that can be used for setting up and running such a performance testing session. This would be useful, so that no site crash happens for you and your team during the Black Friday or Cyber Monday events.

Why should I care about performance testing my web and mobile applications ?

The answer is simple: if you don’t consider performance testing it’s important, it may affect your customers experience, it may hurt your sales and revenue in a bad way.

With that in mind, let’s take a deep dive into performance testing.

Plan your Performance Assessment

By using performance testing as a tool, will allow to avoid crashes and system slowness, which in turn means more people will have a better experience accessing your application, better UX, which translates to bigger sales and higher revenue for your company.

There are some examples online with companies that did not bother to run a performance assessment for their website before a major event like Black Friday. Check the links more a more in-depth understanding:

The list is much longer, but let’s focus on what to do to prevent these undesirable situations.

When should I start load and stress testing my application?

The short and correct answer is, all the time.

In reality, performance testing is time consuming and most of the companies, or the teams of engineers inside most of the companies, just do it before major events when the load on their applications is expected to shoot up.

From our experience, if you are in this category (and most of the companies are) you should plan the load or stress tests ahead of the Black Friday event with at least 2-3 weeks, so that in case something pretty bad is found during the test the engineering teams have time to fix the issue and rerun the test session.

Who should be involved in the testing process?

For the testing session to be successful, you need to involve people from multiple departments and teams.

Teams involved in Planning Performance Testing Sessions
  • Developers (Backend, Frontend and Mobile Developers): they are the one closest to the application internals, they know the ins and outs of how the application is implemented; they are the first people to know if some performance issue arises as a result of an application fault or wrong configuration
  • QA engineers: they represent the main part of the entire process as they are involved not only in running the test session, but also in planning, discussing and agreeing with all the involved parties, the discussion should take into account:
    • which is the success criteria?
    • what scenarios should be run?
    • when is the best time to run the tests?
    • which environments should be used (a lot of times the real performance test can only be run in production as is not feasible to have a development or testing environment at the same scale with the production one)?
    • which are the tools that are needed?
    • which are the metrics and logging information that is needed?
  • DevOps: they are the best people to know what happens at the infrastructure level and the networking level, on cloud or on premise instances, they should handle all the provisioning necessary for meeting the performance session goals.
  • Product Managers: should be the ones that know the business goals and have data and metrics about normal traffic and traffic from previous Black Friday or Cyber Monday periods, and can help with potential insights on current year’s expected levels of traffic based on marketing campaigns and growth.

What type of performance tests should we run?

An important first step for getting a head start would be for the QA team to have an initial plan proposition for the type of tests that they think should be run based on mapping them to the business goals agreed with the product managers.

Below is of load test types that can be performed during the assessment session:

  • Load testing: gives information on how the system performs under an expected load
  • Soak testing: gives information about how the system behaves under an expected load for a continuous period of time
  • Stress Testing: gives information about the maximum capacity limits of the application / system
  • Breakpoint testing: these is similar to stress testing; an incremental load is applied and the system is monitored for predetermined failure conditions; this type of testing is also referred to as Capacity testing because it determines the maximum capacity under which the system will perform to its required specification or SLAs.The results of the breakpoint tests gives information about the initial capacity allocation and the scaling strategy that is required.
  • Configuration testing: this type of testing is performed to determine the effects of configuration changes to the system’s components on the system’s performance and behaviour; an example would be trying different memory allocations for applications, different methods for load balancing and check the performance impact
  • Isolation testing: this type of testing involves repeating a test execution that resulted in a system problem as this testing can often isolate and confirm the fault domain

Which are the systems that will be under load and which are the ways of monitoring their behaviour?

Now that the team has agreed on the type of tests that will be run and the business goals that the test session needs to answer to, the next important step is to identify which are the system parts that will be under a heavy load and make sure that there is monitoring in place that will be able to provide to us insights into what is happening with our system in real time (you would like this to be true especially in the case of testing in production as you would not want to take down the entire production environment – even though probably less expensive than to be taken down during Black Friday).

Which is the information that we need to have access during the test session?

  • Relational and NoSql databases metrics
  • Physical machine metrics
  • Network metrics
  • Load balancer metrics 
  • Application metrics
  • Logging information

Most of the metrics from above can be achieved by using products like this can be achieved using products like Datadog, SolarWinds, DynaTrace, NewRelic, AppDynamics, CloudWatch and others; all these give you the option to send information to a central point where all that information can be easily viewed on charts and dashboards. 

The logging information is normally present in systems like Splunk, ElasticSearch; be sure to have queries, charts and dashboards already set up before the tests are running.

Normally a fairly mature company has all these systems set up way before getting to performance testing.

The information that is already on the dashboards before the test is started gives the team the baseline that they need to compare against when running the tests.

Which are the exact metrics that we should be considered during performance testing ?

This is quite a long discussion and can be checked out in a future blog post. 

How can one person know all this stuff ? 

Most of the time there is no person that knows all this stuff on his/her own. If there is such a person probably is someone from DevOps or the QA team, but most of the time the entire picture can be created from the information that Developers, QAs and DevOps hold collectively. So this is why you need someone from each of the teams present (make sure that is someone fairly knowledgeable in their area of expertise).

Which are the scenarios that you should consider?

Choosing the scenarios is not as straightforward as people might think. At a very simplistic view one would say, let’s test:

  • “Login / Sign In” flow
  • “Sign Up” flow
  • “Add to cart” flow
  • “Checkout” flow

And this is all correct, these should definitely be the first scenarios to be run when performance testing your application but there are more scenarios that you should take into consideration; below we have added some of the ones that should also be part of your test plan:

  • “Add to wish list” flow in case the user missed the product but you will have a future customer for that item
  • “Notifications” flow for new products available from your wish list: you would not want potential users to not know the products are back in stock
  • “Forget password” flow as you would not want your potential customers remain logged out
  • “Apply coupon” flow in case you web application has this functionality implemented

Which are the tools to be used to generate the needed load on the system?

There are different open source tools that the team can use for load testing. The most used ones are: Apache JMeter, Gatling, Locust. All these tools are similar with little variations.

JMeter Load Testing tool from LoadFocus is a cloud load testing platform the offers the possibility to define your tests using directly the web UI or by uploading your Apache JMeter scripts, and run these test at scale with thousands of concurrent users. 

Besides the very easy configuration of the tests the platform offers large scalability, the possibility to load test your application from more than 15 different cloud geolocations with custom or predefined scenarios.

LoadFocus Testing Platform

LoadFocus offers an easy way to understand the dashboards and reports The management and stakeholders can easily to understand the results of the tests ran by the engineering teams.

LoadFocus test report
LoadFocus Test Report
LoadFocus test report detailed
LoadFocus test report detailed
LoadFocus test report errors
LoadFocus test report errors

Which are the next steps ?

It’s fairly simple (in theory):

  • run the tests
  • analyze the results
  • rerun the tests once the fixes for issues found during the initial performance testing session are done
  • increase the load and rerun previous steps

More details on the actual running and analyzing the results in a separate post, as it is quite a lengthy discussion.

Ultimately, your customer experience and satisfaction comes down to a couple of things: UX, Performance, Site Reliability and Availability (taking into consideration the technical part of the business).

With this in mind, make sure that the Development, DevOps and QA teams are all aligned and have gone through the steps needed to make sure your application can withstand the unexpected events and will not crash or show slowness during the shopping season as this may hurt the bottom line of your business.

Written by Cristian Vazzolla.

LoadFocus is a cloud Performance, load and stress testing tool which provides the infrastructure and the ability to run all these tests with thousands of concurrent users, from multiple cloud locations, in less than a few minutes, keep history of the results, compare different runs to inspect performance improvements or performance degradation.

Apache JMeter

Apache JMeter

< 1 minute read

Apache JMeter is a Java open source tool.

If you want to run load or performance tests using Apache JMeter, you can use their Graphic User Interface (GUI) and you don’t need to know any programming language, follow the JMeter documentation and you are ready to go to design your API testing scenarios.

Depends a lot on your tests that you want to do with JMeter, at the moment there are a lot of Samplers and a lot of tasks come out of the box, or through other plugins that can be easily be installed with the JMeter Plugins Manager.

However, if you want to extend the functionality, you can use Groovy, which is the recommended scripting language for Apache JMeter.
Groovy is easy to use and flexible.

For assertions or samplers, you can use BeanShell programming scripts, but most of the tasks can be accomplished nowadays with builtin JMeter functionality.

Regular expression may come in handy also when creating your JMeter script assertions, so regex knowledge might be useful.

With JMeter Load Testing from LoadFocus you can run tests with thousands of concurrent users from multiple world regions in the cloud, all within minutes.

JMeter Load Testing from LoadFocus