Filed under UI Testing.

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

Writing UI tests that are resilient and easy maintainable is hard. Everyone knows that UI tests are slow and flaky but using a few best practices can make the job of implementing and maintaining UI tests a little more easier.

Here are some best practices that we found useful:

  1. use the page object model/robot pattern; this makes the tests easier to understand and update
  2. make the tests run independently from one another and still pass; not fulfilling this will make your tests flaky and give weird behaviours on different environments
  3. use data-*(data-test/data-testid) attributes so that we can keep the tests independent from the changes in the CSS or JS (ex. button id=”main” class=”btn btn-large” data-test=”submit-login-btn”); this will make your tests more resilient to changes in CSS or JS
  4. use more assertions in a single test as the framework should give you enough information related to which check failed; also UI tests take a long time to go to a page so it’s preferable to have more assertions on that page; this will take the number of tests lower and a side effect will also be the running time of the tests will go down
  5. clean up the state before each test run and at the end of the test using beforeEach / afterEach; this will improve reliability and make the tests less flaky
  6. do not use wait for arbitrary periods of time; instead use assertions that wait for a specific condition to be met; this will make the tests less flaky and avoids weird fails in the tests
  7. always set the start url in the context of your test; do not set as start url that needs an entire flow to be run so that we get to what we want to test; this will lower the execution time of the test suites and avoid fails triggered by bugs in different parts of the application which are not to be tested by the respective test suite
  8. talk to 3rd party services via their API not by visiting another web app; this will decrease flakiness of the tests and lower the execution time of the test suites

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

Filed under Native Device Testing, UI Testing.

What is OpenSTF ?

OpenSTF is an open source project that gives us the ability to create our own test farm with real devices.

Using OpenSTF we can create a device farm that gives us the possibility to share devices between team members without actually physically moving the device from the device farm.

Why is OpenSTF useful ?

The are quite a few advantages for using OpenSTF:

  • it’s open source
  • we do not need to physically move the devices around
  • we can easy install the application on the devices

How do we install OpenSTF on our server ?

  1. First we need to install the dependencies for STF using home brew by running the following from the command line:
    • brew install rethinkdb graphicsmagick zeromq protobuf yasm pkg-config
  2. If we want to run STF from source we need to:
    • Clone the repository from https://github.com/openstf/stf by running:
      git clone git@github.com:openstf/stf.git
    • build the application by running:
      npm install
    • link the module so that you’ll be able to access the stf command directly from the command line
      npm link
  3. If we just want to install STF without building the latest version we can simply run:
    npm install -g stf
  4. To start openstf we first need to start rethinkdb; for that we need to run the following command in the command line:
    rethinkdb
  5. Now we are ready to start STF; for that we run the following command in the command line:
    stf local
  6. Open your favourite browser and browse to:
    http://localhost:7100
  7. Enter a username and password (you can enter any username and password)
  8. You should now be able to see in the OpenSTF UI the devices that you have connected to your machine as in the picture below

  9. Double clicking the device will take you to the device and you will be able to interact with the device exactly like you would do on the real device
  10. Note: it does not support emulators or iOS devices

What do we need to buy to create our own device farm ?

  • a Mac Mini box
  • a couple of Android devices (ex. 5 devices)
  • a couple of USB hubs
  • a couple of USB cables

Which are the features that OpenSTF supports ?

  • Android OS support: Android 2.3 ~ 8.0 including Wear 5.1, FireOS, CyanogenMod
  • Keyboard & Mouse Input with Multi-Touch, even in Safari iOS
  • Copy & Paste to and from the device
  • Take Screenshots and resize them
  • Drag & Drop APK files
  • Install and launch apps in 1 step
  • Open URLs in any installed browser
  • Display Logs with realtime filtering
  • Run Shell Commands without leaving your browser
  • Debug Remotely with Android Studio, Chrome Debug Tools, as if it was plugged into your computer
  • Reverse Port Forwarding
  • Device Rotation
  • Play Store Automation

For a complete description of OpenSTF and the features please visit the project page: https://openstf.io/

 

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

Filed under Puppeteer, UI Testing.

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

Using Puppeteer to interact with the UI elements from web pages is quite simple. All you need to do is to open a new page with Puppeteer:

const page = await browser.newPage();

Then, all you have to do, is identifying the element you want to perform the action, click in our case:

await page.click('.clickButton');

Puppeteer comes with a set of options for the click action, all optional:
According to the documentation, you can have these options sent to the click action:

  • button left, right, or middle, defaults to left.
  • clickCount defaults to 1. See UIEvent.detail.
  • delay Time to wait between mouse down and mouse up in milliseconds. Defaults to 0.

Click method will scroll into view the selected element (if it’s found, and if the element is not in the viewport) and click in the center of the element.
If the element is not found, it will throw an error.

More details can be found on the documentation page from Puppeteer: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md

Check out more details of how to take screenshots with Puppeteer using Headless Chrome Browser.

 

LoadFocus – is a easy to use cloud testing tool, and you don’t need any setup, it runs from the browser. It helps you understand better your Website’s and WebServices’ performance and find potential bottleneck before your customers do. Run a free website speed test today.

Filed under JAVA.

Why do we need to mock APIs ?

A lot of times when you develop integration tests there is a need to mock different APIs to test for negative cases to check how the rest of the platform behaves in that case and it is not feasible to change the API to return errors. This is especially true when you are trying to run the tests against a staging environment which simulates exactly the Production environment.

In these cases we can use WireMock to mock different APIs and to make the API return exactly what response we need for our tests.
Below we are going to run WireMock as a standalone server not as a Gradle or Maven dependency.

How to install WireMock ?
1. Browse to :
http://wiremock.org/docs/running-standalone/
2. Download the standalone jar from :
http://repo1.maven.org/maven2/com/github/tomakehurst/wiremock-standalone/2.14.0/wiremock-standalone-2.14.0.jar
3. After the jar finished downloaded, open the terminal where the jar has been downloaded and run the following command:
java -jar wiremock-standalone-2.14.0.jar

4. In the terminal you should see something like below:

➜ wiremock java -jar wiremock-standalone-2.14.0.jar
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
 /$$ /$$ /$$ /$$ /$$ /$$
| $$ /$ | $$|__/ | $$$ /$$$ | $$
| $$ /$$$| $$ /$$ /$$$$$$ /$$$$$$ | $$$$ /$$$$ /$$$$$$ /$$$$$$$| $$ /$$
| $$/$$ $$ $$| $$ /$$__ $$ /$$__ $$| $$ $$/$$ $$ /$$__ $$ /$$_____/| $$ /$$/
| $$$$_ $$$$| $$| $$ \__/| $$$$$$$$| $$ $$$| $$| $$ \ $$| $$ | $$$$$$/
| $$$/ \ $$$| $$| $$ | $$_____/| $$\ $ | $$| $$ | $$| $$ | $$_ $$
| $$/ \ $$| $$| $$ | $$$$$$$| $$ \/ | $$| $$$$$$/| $$$$$$$| $$ \ $$
|__/ \__/|__/|__/ \_______/|__/ |__/ \______/ \_______/|__/ \__/

port: 8080
enable-browser-proxying: false
no-request-journal: false
verbose: false

5. By the default the server will start on port 8080
6. Next we are going to create a mapping for our API using the JSON configuration.
For this we are going to create a JSON file “myApi.json” with the following contents:

{
 "request": {
 "method": "GET",
 "url": "/myApi"
 },
 "response": {
 "status": 200,
 "body": "My API custom response\n"
 }
 }

7. We are going to place the file in the mappings folder that was created in the same place where the wiremock-standalone-2.14.0.jar is located.
8. Restart the wiremock server by running:
java -jar wiremock-standalone-2.14.0.jar
9. Now let’s browse to: http://localhost:8080/test just to test that our API mapping has been picked up.
The page should display something like the below:


10. Browsing to our mapped API should respond with the configured response (status code 200 and with the body “My API custom response”)

Filed under JAVA, JUnit.

Why structure your integration tests ?

When the number of integration tests grow is a good practice to separate the tests based on the feature/api that they are testing.

The advantages of doing this are:

  • easier to maitain as the tests for a specific feature/API are grouped in a specific category
  • easier to run the tests only for that specific category when only that feature changes to get faster feedback
  • easier to group the tests when running them in Jenkins/Travis CI or any other CI so that multiple jobs can be run in parallel and also when a job fails imediatelly the team will know which feature/API has failed

How is this done using Java, JUnit and Maven ?

  1. Create a basic Maven project
  2. Create a new package in the src/main/java with the name “tutorial.junit.categories” in which the different categories will be added.
    The structure of the project should look like below:
  3. Create a few JUnit test cases in a few test classes. In our case we are going to create:
    •  LoginApiTestIT.java
    •  SearchApiTestIT.java
  4. Create a few categories in which we want to split the tests. In our case we are going to split them as follows:
    • LoginApiCategory – tests run against the login api
    • SearchApiCategory – tests run against the search api
    • SmokeTestCategory – which includes tests from either Login or Search API tests that we want to run as smoke tests
  5. A category is just an empty interface as in the example below:
  6. Setting the category for each of the tests. For setting the category we just need to add the @Category annotation either at the class or method level as  below.
  7. Example below:
    
    package tutorial.junit;
    
    import org.junit.Test;
     import org.junit.experimental.categories.Category;
    
    import tutorial.junit.categories.LoginApiCategory;
     import tutorial.junit.categories.SmokeTestCategory;
    
    @Category(LoginApiCategory.class)
     public class LoginApiTestIT {
    
    @Category(SmokeTestCategory.class)
     @Test
     public void shouldReturn200ForValidCredentials(){
     System.out.println("Running Login API tests - Positive case");
     }
    
    @Test
     public void shouldReturn404ForInvalidCredentials(){
     System.out.println("Running Login API tests - Negative case");
     }
     }
    
    
  8. Also for running just the integration tests we are going to create a Maven profile in our Maven pom.xml for the tests as below:
    • <profiles>
      
           <!-- The Configuration of the integration-test profile -->
      
           <profile>
      
              <id>integration-test</id>
      
              <build>
      
           <plugins>
      
            <plugin>
      
              <groupId>org.apache.maven.plugins</groupId>
      
              <artifactId>maven-failsafe-plugin</artifactId>
      
              <version>2.18.1</version>
      
              <executions>
      
                <execution>
      
                  <id>integration-test</id>
      
                  <goals>
      
                    <goal>integration-test</goal>
      
                  </goals>
      
                </execution>
      
                <execution>
      
                  <id>verify</id>
      
                  <goals>
      
                    <goal>verify</goal>
      
                  </goals>
      
                </execution>
      
              </executions>
      
            </plugin>
      
           </plugins>
      
         </build>
      
           </profile>
      
      </profiles>

       

  9. Running the tests based on the category is pretty simple using the Maven commands below. Open the Maven pom.xml file location in your terminal and run the following commands:
    • running the login tests
      mvn clean install -P integration-test -Dgroups="tutorial.junit.categories.LoginApiCategory"
    • running the search tests
      mvn clean install -P integration-test -Dgroups="tutorial.junit.categories.SearchApiCategory"
    • running the smoke tests which include tests from login and search
      mvn clean install -P integration-test -Dgroups="tutorial.junit.categories.SmokeTestCategory"

Filed under JAVA.

Here is how to click a link by text with Selenium WebDriver in Java using the built in WebDriver helper methods or by XPath:

Click link by full text using Selenium WebDriver

WebElement linkByText = driver.findElement(By.linkText("My Link"));
linkByText.click();

Click link by partial text using Selenium WebDriver

WebElement linkByPartialText = driver.findElement(By.partialLinkText("First"))
linkByPartialText.click();

Click link by text using XPath in Selenium WebDriver

WebElement linkByTextUsingXPath = driver.findElement(By.xpath("//a[text()='First']"));
linkByTextUsingXPath.click();

Click link by partial text using XPath in Selenium WebDriver

WebElement linkByPartialTextUsingXPath = driver.findElement(By.xpath("//a[contains(text(),'ABC')]"));
linkByPartialTextUsingXPath.click();

More details here on how to locate elements with Selenium WebDriver.

 

LoadFocus is a cloud testing platform for:

Filed under NodeJS, Screenshot Testing.

What is Puppeteer? Puppeteer is Node library that you can use in order to control Headless Chrome with the DevTools Protocol.

The Chrome DevTools Protocol allows for tools to instrument, inspect, debug and profile for Chromium and Chrome browsers.

Puppeteer – Headless Chrome Node API works only with Chrome and uses the latest versions of Chromium.

Chromium is an open-source browser project that forms the basis for the Chrome web browser. One of the biggest differences between the two browsers is that, while Chrome is based on Chromium, Google adds some of proprietary features to Chrome, features like automatic updates and support for additional video formats. Other features like usage-tracking or “user metrics” feature can be found only in Chrome browser.

Note: Puppeteer requires at least Node v6.4.0but the examples below use async/await which is only supported in Node v7.6.0 or greater.

Node.js has a simple module loading system. In Node.js, files and modules are in one-to-one correspondence (each file is treated as a separate module).

You can use Visual Regression Testing to take website screenshots and compare the generated images and identify differences pixel by pixel, a comparison image will be shown next to the result’s screenshot that highlights the differences in red.

visual regression testing tool

 

Install Puppeteer

Here is how to install puppeteer from NPM Modules Registry (npm is the package manager for JavaScript):

 npm i puppeteer 

Below are code snippets on how to use Puppeteer – Headless Chrome Node API in order to take screenshots of your website.

Example – navigating to https://example.com and saving a screenshot as a PNG file named example.png:

Generate screenshots with Puppetteer

 const puppeteer = require('puppeteer'); (async () =&amp;amp;amp;amp;amp;gt; { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://example.com'); await page.screenshot({path: 'example.png'}); await browser.close(); })(); 

await page.goto('https://example.com'), {
timeout: 120000,
slowMo: 2000,
waitUntil: ['load', 'networkidle'],
networkIdleTimeout: 5000,
});

By default, Puppeteer take screenshots of the viewport (the area which is visible by default when you open a website).

In order to take a screenshot of the full web page, you need to add the fullPage parameter to the screenshot method:


await page.screenshot({ path: 'example.png', fullPage: true });

Here is an example on how to take a screenshot of a webpage with Puppeteer using a customer web page size.
You just need to pass the width and height of the viewport in order for the browser to resize the web page to the desired size.


export async function takeScreenshot(page, filename, width, height) {
await page.setViewport({ width, height });

await page.screenshot({ path: `${filename}-${width}-${height}.png`, fullPage: true });
}

Here is how to call the above custom screenshot method:

await takeScreenshot(page, 'example.png', 320, 480);

Puppeteer, also provides a list of Mobile Devices as a list of objects called DeviceDescriptors.
In order to emulate a web page in a mobile emulator, with specific characteristics, you can import the
pre-defined list of mobile emulators from puppeteer/DeviceDescriptors

 const devices from 'puppeteer/DeviceDescriptors'; 

Puppeteer's API is very similar to Selenium WebDriver, but works only with Google Chrome, while WebDriver work with most popular browsers.

More details on how to locate elements to use in order to interact with Puppeteer or Selenium WebDriver.

Debugging and Troubleshooting Puppeteer

1. Non Headless Mode - for debugging purposes, sometimes it's useful to see what the browser is displaying. Instead of launching in headless mode, launch a full version of Chrome using headless: false when you launch the browser using Puppeteer:


const browser = await puppeteer.launch({headless: false});

2. Slow down screenshot generation - the slowMo option slows down Puppeteer operations by the specified amount of milliseconds. It's another way to understand better what's happening with the code you've written and debug easier.


const browser = await puppeteer.launch({
headless: false,
slowMo: 250 // slow down by 250ms
});

3. Capture browser's console output


page.on('console', msg =&amp;amp;amp;amp;amp;gt; console.log('PAGE LOG:', ...msg.args));

await page.evaluate(() =&amp;amp;amp;amp;amp;gt; console.log(`url is ${location.href}`));

4. Enable verbose logging - All public API calls and internal protocol traffic will be logged via the debug module under the puppeteer namespace.

# Basic verbose logging

env DEBUG="puppeteer:*" node script.js

You can use Visual Regression Testing to take website screenshots and compare the generated images and identify differences pixel by pixel, a comparison image will be shown next to the result’s screenshot that highlights the differences in red.

LoadFocus is a cloud testing platform for: