UI Testing

In this article we will show how to create End to End (E2E) UI tests for the e-commerce ASOS for 3 of their markets (UK, IT, DE) in less than 30 minutes.

Why use TestCafe for running tests?

For creating the tests we will be using the TestCafe framework. TestCafe is an open source framework that was migrated from a commercial product.

Here are some of the advantages of TestCafe over WebDriver based frameworks:

  • less moving parts and no problem with the webdriver to browser compatibility as TestCafe does not use webdriver
  • the tests run much faster
  • the error message that we get when a test fails provide the exact reason of the fail unlike the webdriver based frameworks which provide an useless error message
  • implementing the tests is very easy once the boiler plate is all set up so the ROI is very high
  • the framework is Javascript based which makes it very easy to use by the front end developers

How to create the framework and run tests against the ecommerce platform?

 

The project is located at:

(this is just a demo project for learning purposes, we are no NOT affiliated with, funded, or in any way associated with ASOS brand):


https://github.com/loadfocusapp/asos-testcafe-e2e

Please clone the repository:


git clone https://github.com/loadfocusapp/asos-testcafe-e2e

Run the tests against asos.com, asos.de and asos.com/it websites.

For running the tests please go into the directory where you cloned the repository and run:

In order to run the tests you need to have NPM install on your machine, find here how to install NPM.

In this demo we use: Node v8.9.4 and NPM v5.6.0.

npm i
ENV=AsosCom npm run e2e:firefox
ENV=AsosDe npm run e2e:chrome
ENV=AsosIt npm run e2e:safari

Let us know if you need help setting this up for your own web application.

 

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

LoadFocus Team – offers test automation services to small and medium clients.

UI Testing

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

Writing UI tests that are resilient and easy maintainable is hard. Everyone knows that UI tests are slow and flaky but using a few best practices can make the job of implementing and maintaining UI tests a little more easier.

Here are some best practices that we found useful:

  1. use the page object model/robot pattern; this makes the tests easier to understand and update
  2. make the tests run independently from one another and still pass; not fulfilling this will make your tests flaky and give weird behaviours on different environments
  3. use data-*(data-test/data-testid) attributes so that we can keep the tests independent from the changes in the CSS or JS (ex. button id=”main” class=”btn btn-large” data-test=”submit-login-btn”); this will make your tests more resilient to changes in CSS or JS
  4. use more assertions in a single test as the framework should give you enough information related to which check failed; also UI tests take a long time to go to a page so it’s preferable to have more assertions on that page; this will take the number of tests lower and a side effect will also be the running time of the tests will go down
  5. clean up the state before each test run and at the end of the test using beforeEach / afterEach; this will improve reliability and make the tests less flaky
  6. do not use wait for arbitrary periods of time; instead use assertions that wait for a specific condition to be met; this will make the tests less flaky and avoids weird fails in the tests
  7. always set the start url in the context of your test; do not set as start url that needs an entire flow to be run so that we get to what we want to test; this will lower the execution time of the test suites and avoid fails triggered by bugs in different parts of the application which are not to be tested by the respective test suite
  8. talk to 3rd party services via their API not by visiting another web app; this will decrease flakiness of the tests and lower the execution time of the test suites

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

Native Device Testing UI Testing

What is OpenSTF ?

OpenSTF is an open source project that gives us the ability to create our own test farm with real devices.

Using OpenSTF we can create a device farm that gives us the possibility to share devices between team members without actually physically moving the device from the device farm.

Why is OpenSTF useful ?

The are quite a few advantages for using OpenSTF:

  • it’s open source
  • we do not need to physically move the devices around
  • we can easy install the application on the devices

How do we install OpenSTF on our server ?

  1. First we need to install the dependencies for STF using home brew by running the following from the command line:
    • brew install rethinkdb graphicsmagick zeromq protobuf yasm pkg-config
  2. If we want to run STF from source we need to:
    • Clone the repository from https://github.com/openstf/stf by running:
      git clone git@github.com:openstf/stf.git
    • build the application by running:
      npm install
    • link the module so that you’ll be able to access the stf command directly from the command line
      npm link
  3. If we just want to install STF without building the latest version we can simply run:
    npm install -g stf
  4. To start openstf we first need to start rethinkdb; for that we need to run the following command in the command line:
    rethinkdb
  5. Now we are ready to start STF; for that we run the following command in the command line:
    stf local
  6. Open your favourite browser and browse to:
    http://localhost:7100
  7. Enter a username and password (you can enter any username and password)
  8. You should now be able to see in the OpenSTF UI the devices that you have connected to your machine as in the picture below

  9. Double clicking the device will take you to the device and you will be able to interact with the device exactly like you would do on the real device
  10. Note: it does not support emulators or iOS devices

What do we need to buy to create our own device farm ?

  • a Mac Mini box
  • a couple of Android devices (ex. 5 devices)
  • a couple of USB hubs
  • a couple of USB cables

Which are the features that OpenSTF supports ?

  • Android OS support: Android 2.3 ~ 8.0 including Wear 5.1, FireOS, CyanogenMod
  • Keyboard & Mouse Input with Multi-Touch, even in Safari iOS
  • Copy & Paste to and from the device
  • Take Screenshots and resize them
  • Drag & Drop APK files
  • Install and launch apps in 1 step
  • Open URLs in any installed browser
  • Display Logs with realtime filtering
  • Run Shell Commands without leaving your browser
  • Debug Remotely with Android Studio, Chrome Debug Tools, as if it was plugged into your computer
  • Reverse Port Forwarding
  • Device Rotation
  • Play Store Automation

For a complete description of OpenSTF and the features please visit the project page: https://openstf.io/

 

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

Puppeteer UI Testing

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

Using Puppeteer to interact with the UI elements from web pages is quite simple. All you need to do is to open a new page with Puppeteer:

const page = await browser.newPage();

Then, all you have to do, is identifying the element you want to perform the action, click in our case:

await page.click('.clickButton');

Puppeteer comes with a set of options for the click action, all optional:
According to the documentation, you can have these options sent to the click action:

  • button left, right, or middle, defaults to left.
  • clickCount defaults to 1. See UIEvent.detail.
  • delay Time to wait between mouse down and mouse up in milliseconds. Defaults to 0.

Click method will scroll into view the selected element (if it’s found, and if the element is not in the viewport) and click in the center of the element.
If the element is not found, it will throw an error.

More details can be found on the documentation page from Puppeteer: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md

Check out more details of how to take screenshots with Puppeteer using Headless Chrome Browser.

 

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

JAVA

Why do we need to mock APIs ?

A lot of times when you develop integration tests there is a need to mock different APIs to test for negative cases to check how the rest of the platform behaves in that case and it is not feasible to change the API to return errors. This is especially true when you are trying to run the tests against a staging environment which simulates exactly the Production environment.

In these cases we can use WireMock to mock different APIs and to make the API return exactly what response we need for our tests.
Below we are going to run WireMock as a standalone server not as a Gradle or Maven dependency.

How to install WireMock ?
1. Browse to :
http://wiremock.org/docs/running-standalone/
2. Download the standalone jar from :
http://repo1.maven.org/maven2/com/github/tomakehurst/wiremock-standalone/2.14.0/wiremock-standalone-2.14.0.jar
3. After the jar finished downloaded, open the terminal where the jar has been downloaded and run the following command:
java -jar wiremock-standalone-2.14.0.jar

4. In the terminal you should see something like below:

➜ wiremock java -jar wiremock-standalone-2.14.0.jar
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
 /$$ /$$ /$$ /$$ /$$ /$$
| $$ /$ | $$|__/ | $$$ /$$$ | $$
| $$ /$$$| $$ /$$ /$$$$$$ /$$$$$$ | $$$$ /$$$$ /$$$$$$ /$$$$$$$| $$ /$$
| $$/$$ $$ $$| $$ /$$__ $$ /$$__ $$| $$ $$/$$ $$ /$$__ $$ /$$_____/| $$ /$$/
| $$$$_ $$$$| $$| $$ \__/| $$$$$$$$| $$ $$$| $$| $$ \ $$| $$ | $$$$$$/
| $$$/ \ $$$| $$| $$ | $$_____/| $$\ $ | $$| $$ | $$| $$ | $$_ $$
| $$/ \ $$| $$| $$ | $$$$$$$| $$ \/ | $$| $$$$$$/| $$$$$$$| $$ \ $$
|__/ \__/|__/|__/ \_______/|__/ |__/ \______/ \_______/|__/ \__/

port: 8080
enable-browser-proxying: false
no-request-journal: false
verbose: false

5. By the default the server will start on port 8080
6. Next we are going to create a mapping for our API using the JSON configuration.
For this we are going to create a JSON file “myApi.json” with the following contents:

{
 "request": {
 "method": "GET",
 "url": "/myApi"
 },
 "response": {
 "status": 200,
 "body": "My API custom response\n"
 }
 }

7. We are going to place the file in the mappings folder that was created in the same place where the wiremock-standalone-2.14.0.jar is located.
8. Restart the wiremock server by running:
java -jar wiremock-standalone-2.14.0.jar
9. Now let’s browse to: http://localhost:8080/test just to test that our API mapping has been picked up.
The page should display something like the below:


10. Browsing to our mapped API should respond with the configured response (status code 200 and with the body “My API custom response”)

JAVA JUnit

Why structure your integration tests ?

When the number of integration tests grow is a good practice to separate the tests based on the feature/api that they are testing.

The advantages of doing this are:

  • easier to maitain as the tests for a specific feature/API are grouped in a specific category
  • easier to run the tests only for that specific category when only that feature changes to get faster feedback
  • easier to group the tests when running them in Jenkins/Travis CI or any other CI so that multiple jobs can be run in parallel and also when a job fails imediatelly the team will know which feature/API has failed

How is this done using Java, JUnit and Maven ?

  1. Create a basic Maven project
  2. Create a new package in the src/main/java with the name “tutorial.junit.categories” in which the different categories will be added.
    The structure of the project should look like below:
  3. Create a few JUnit test cases in a few test classes. In our case we are going to create:
    •  LoginApiTestIT.java
    •  SearchApiTestIT.java
  4. Create a few categories in which we want to split the tests. In our case we are going to split them as follows:
    • LoginApiCategory – tests run against the login api
    • SearchApiCategory – tests run against the search api
    • SmokeTestCategory – which includes tests from either Login or Search API tests that we want to run as smoke tests
  5. A category is just an empty interface as in the example below:
  6. Setting the category for each of the tests. For setting the category we just need to add the @Category annotation either at the class or method level as  below.
  7. Example below:
    
    package tutorial.junit;
    
    import org.junit.Test;
     import org.junit.experimental.categories.Category;
    
    import tutorial.junit.categories.LoginApiCategory;
     import tutorial.junit.categories.SmokeTestCategory;
    
    @Category(LoginApiCategory.class)
     public class LoginApiTestIT {
    
    @Category(SmokeTestCategory.class)
     @Test
     public void shouldReturn200ForValidCredentials(){
     System.out.println("Running Login API tests - Positive case");
     }
    
    @Test
     public void shouldReturn404ForInvalidCredentials(){
     System.out.println("Running Login API tests - Negative case");
     }
     }
    
    
  8. Also for running just the integration tests we are going to create a Maven profile in our Maven pom.xml for the tests as below:
    • <profiles>
      
           <!-- The Configuration of the integration-test profile -->
      
           <profile>
      
              <id>integration-test</id>
      
              <build>
      
           <plugins>
      
            <plugin>
      
              <groupId>org.apache.maven.plugins</groupId>
      
              <artifactId>maven-failsafe-plugin</artifactId>
      
              <version>2.18.1</version>
      
              <executions>
      
                <execution>
      
                  <id>integration-test</id>
      
                  <goals>
      
                    <goal>integration-test</goal>
      
                  </goals>
      
                </execution>
      
                <execution>
      
                  <id>verify</id>
      
                  <goals>
      
                    <goal>verify</goal>
      
                  </goals>
      
                </execution>
      
              </executions>
      
            </plugin>
      
           </plugins>
      
         </build>
      
           </profile>
      
      </profiles>

       

  9. Running the tests based on the category is pretty simple using the Maven commands below. Open the Maven pom.xml file location in your terminal and run the following commands:
    • running the login tests
      mvn clean install -P integration-test -Dgroups="tutorial.junit.categories.LoginApiCategory"
    • running the search tests
      mvn clean install -P integration-test -Dgroups="tutorial.junit.categories.SearchApiCategory"
    • running the smoke tests which include tests from login and search
      mvn clean install -P integration-test -Dgroups="tutorial.junit.categories.SmokeTestCategory"

JAVA

Here is how to click a link by text with Selenium WebDriver in Java using the built in WebDriver helper methods or by XPath:

Click link by full text using Selenium WebDriver

WebElement linkByText = driver.findElement(By.linkText("My Link"));
linkByText.click();

Click link by partial text using Selenium WebDriver

WebElement linkByPartialText = driver.findElement(By.partialLinkText("First"))
linkByPartialText.click();

Click link by text using XPath in Selenium WebDriver

WebElement linkByTextUsingXPath = driver.findElement(By.xpath("//a[text()='First']"));
linkByTextUsingXPath.click();

Click link by partial text using XPath in Selenium WebDriver

WebElement linkByPartialTextUsingXPath = driver.findElement(By.xpath("//a[contains(text(),'ABC')]"));
linkByPartialTextUsingXPath.click();

More details here on how to locate elements with Selenium WebDriver.

 

LoadFocus is a cloud testing platform for: