Filed under Native Device Testing, UI Testing.

What is OpenSTF ?

OpenSTF is an open source project that gives us the ability to create our own test farm with real devices.

Using OpenSTF we can create a device farm that gives us the possibility to share devices between team members without actually physically moving the device from the device farm.

Why is OpenSTF useful ?

The are quite a few advantages for using OpenSTF:

  • it’s open source
  • we do not need to physically move the devices around
  • we can easy install the application on the devices

How do we install OpenSTF on our server ?

  1. First we need to install the dependencies for STF using home brew by running the following from the command line:
    • brew install rethinkdb graphicsmagick zeromq protobuf yasm pkg-config
  2. If we want to run STF from source we need to:
    • Clone the repository from https://github.com/openstf/stf by running:
      git clone git@github.com:openstf/stf.git
    • build the application by running:
      npm install
    • link the module so that you’ll be able to access the stf command directly from the command line
      npm link
  3. If we just want to install STF without building the latest version we can simply run:
    npm install -g stf
  4. To start openstf we first need to start rethinkdb; for that we need to run the following command in the command line:
    rethinkdb
  5. Now we are ready to start STF; for that we run the following command in the command line:
    stf local
  6. Open your favourite browser and browse to:
    http://localhost:7100
  7. Enter a username and password (you can enter any username and password)
  8. You should now be able to see in the OpenSTF UI the devices that you have connected to your machine as in the picture below

  9. Double clicking the device will take you to the device and you will be able to interact with the device exactly like you would do on the real device
  10. Note: it does not support emulators or iOS devices

What do we need to buy to create our own device farm ?

  • a Mac Mini box
  • a couple of Android devices (ex. 5 devices)
  • a couple of USB hubs
  • a couple of USB cables

Which are the features that OpenSTF supports ?

  • Android OS support: Android 2.3 ~ 8.0 including Wear 5.1, FireOS, CyanogenMod
  • Keyboard & Mouse Input with Multi-Touch, even in Safari iOS
  • Copy & Paste to and from the device
  • Take Screenshots and resize them
  • Drag & Drop APK files
  • Install and launch apps in 1 step
  • Open URLs in any installed browser
  • Display Logs with realtime filtering
  • Run Shell Commands without leaving your browser
  • Debug Remotely with Android Studio, Chrome Debug Tools, as if it was plugged into your computer
  • Reverse Port Forwarding
  • Device Rotation
  • Play Store Automation

For a complete description of OpenSTF and the features please visit the project page: https://openstf.io/

Filed under Puppeteer, UI Testing.

Using Puppeteer to interact with the UI elements from web pages is quite simple. All you need to do is to open a new page with Puppeteer:

const page = await browser.newPage();

Then, all you have to do, is identifying the element you want to perform the action, click in our case:

await page.click('.clickButton');

Puppeteer comes with a set of options for the click action, all optional:
According to the documentation, you can have these options sent to the click action:

  • button left, right, or middle, defaults to left.
  • clickCount defaults to 1. See UIEvent.detail.
  • delay Time to wait between mouse down and mouse up in milliseconds. Defaults to 0.

Click method will scroll into view the selected element (if it’s found, and if the element is not in the viewport) and click in the center of the element.
If the element is not found, it will throw an error.

More details can be found on the documentation page from Puppeteer: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md

Check out more details of how to take screenshots with Puppeteer using Headless Chrome Browser.

Filed under JAVA.

Why do we need to mock APIs ?

A lot of times when you develop integration tests there is a need to mock different APIs to test for negative cases to check how the rest of the platform behaves in that case and it is not feasible to change the API to return errors. This is especially true when you are trying to run the tests against a staging environment which simulates exactly the Production environment.

In these cases we can use WireMock to mock different APIs and to make the API return exactly what response we need for our tests.
Below we are going to run WireMock as a standalone server not as a Gradle or Maven dependency.

How to install WireMock ?
1. Browse to :
http://wiremock.org/docs/running-standalone/
2. Download the standalone jar from :
http://repo1.maven.org/maven2/com/github/tomakehurst/wiremock-standalone/2.14.0/wiremock-standalone-2.14.0.jar
3. After the jar finished downloaded, open the terminal where the jar has been downloaded and run the following command:
java -jar wiremock-standalone-2.14.0.jar

4. In the terminal you should see something like below:

➜ wiremock java -jar wiremock-standalone-2.14.0.jar
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
 /$$ /$$ /$$ /$$ /$$ /$$
| $$ /$ | $$|__/ | $$$ /$$$ | $$
| $$ /$$$| $$ /$$ /$$$$$$ /$$$$$$ | $$$$ /$$$$ /$$$$$$ /$$$$$$$| $$ /$$
| $$/$$ $$ $$| $$ /$$__ $$ /$$__ $$| $$ $$/$$ $$ /$$__ $$ /$$_____/| $$ /$$/
| $$$$_ $$$$| $$| $$ \__/| $$$$$$$$| $$ $$$| $$| $$ \ $$| $$ | $$$$$$/
| $$$/ \ $$$| $$| $$ | $$_____/| $$\ $ | $$| $$ | $$| $$ | $$_ $$
| $$/ \ $$| $$| $$ | $$$$$$$| $$ \/ | $$| $$$$$$/| $$$$$$$| $$ \ $$
|__/ \__/|__/|__/ \_______/|__/ |__/ \______/ \_______/|__/ \__/

port: 8080
enable-browser-proxying: false
no-request-journal: false
verbose: false

5. By the default the server will start on port 8080
6. Next we are going to create a mapping for our API using the JSON configuration.
For this we are going to create a JSON file “myApi.json” with the following contents:

{
 "request": {
 "method": "GET",
 "url": "/myApi"
 },
 "response": {
 "status": 200,
 "body": "My API custom response\n"
 }
 }

7. We are going to place the file in the mappings folder that was created in the same place where the wiremock-standalone-2.14.0.jar is located.
8. Restart the wiremock server by running:
java -jar wiremock-standalone-2.14.0.jar
9. Now let’s browse to: http://localhost:8080/test just to test that our API mapping has been picked up.
The page should display something like the below:


10. Browsing to our mapped API should respond with the configured response (status code 200 and with the body “My API custom response”)

Filed under JAVA, JUnit.

Why structure your integration tests ?

When the number of integration tests grow is a good practice to separate the tests based on the feature/api that they are testing.

The advantages of doing this are:

  • easier to maitain as the tests for a specific feature/API are grouped in a specific category
  • easier to run the tests only for that specific category when only that feature changes to get faster feedback
  • easier to group the tests when running them in Jenkins/Travis CI or any other CI so that multiple jobs can be run in parallel and also when a job fails imediatelly the team will know which feature/API has failed

How is this done using Java, JUnit and Maven ?

  1. Create a basic Maven project
  2. Create a new package in the src/main/java with the name “tutorial.junit.categories” in which the different categories will be added.
    The structure of the project should look like below:
  3. Create a few JUnit test cases in a few test classes. In our case we are going to create:
    •  LoginApiTestIT.java
    •  SearchApiTestIT.java
  4. Create a few categories in which we want to split the tests. In our case we are going to split them as follows:
    • LoginApiCategory – tests run against the login api
    • SearchApiCategory – tests run against the search api
    • SmokeTestCategory – which includes tests from either Login or Search API tests that we want to run as smoke tests
  5. A category is just an empty interface as in the example below:
  6. Setting the category for each of the tests. For setting the category we just need to add the @Category annotation either at the class or method level as  below.
  7. Example below:
    
    package tutorial.junit;
    
    import org.junit.Test;
     import org.junit.experimental.categories.Category;
    
    import tutorial.junit.categories.LoginApiCategory;
     import tutorial.junit.categories.SmokeTestCategory;
    
    @Category(LoginApiCategory.class)
     public class LoginApiTestIT {
    
    @Category(SmokeTestCategory.class)
     @Test
     public void shouldReturn200ForValidCredentials(){
     System.out.println("Running Login API tests - Positive case");
     }
    
    @Test
     public void shouldReturn404ForInvalidCredentials(){
     System.out.println("Running Login API tests - Negative case");
     }
     }
    
    
  8. Also for running just the integration tests we are going to create a Maven profile in our Maven pom.xml for the tests as below:
    • <profiles>
      
           <!-- The Configuration of the integration-test profile -->
      
           <profile>
      
              <id>integration-test</id>
      
              <build>
      
           <plugins>
      
            <plugin>
      
              <groupId>org.apache.maven.plugins</groupId>
      
              <artifactId>maven-failsafe-plugin</artifactId>
      
              <version>2.18.1</version>
      
              <executions>
      
                <execution>
      
                  <id>integration-test</id>
      
                  <goals>
      
                    <goal>integration-test</goal>
      
                  </goals>
      
                </execution>
      
                <execution>
      
                  <id>verify</id>
      
                  <goals>
      
                    <goal>verify</goal>
      
                  </goals>
      
                </execution>
      
              </executions>
      
            </plugin>
      
           </plugins>
      
         </build>
      
           </profile>
      
      </profiles>

       

  9. Running the tests based on the category is pretty simple using the Maven commands below. Open the Maven pom.xml file location in your terminal and run the following commands:
    • running the login tests
      mvn clean install -P integration-test -Dgroups="tutorial.junit.categories.LoginApiCategory"
    • running the search tests
      mvn clean install -P integration-test -Dgroups="tutorial.junit.categories.SearchApiCategory"
    • running the smoke tests which include tests from login and search
      mvn clean install -P integration-test -Dgroups="tutorial.junit.categories.SmokeTestCategory"

Filed under JAVA.

Here is how to click a link by text with Selenium WebDriver in Java using the built in WebDriver helper methods or by XPath:

Click link by full text using Selenium WebDriver

WebElement linkByText = driver.findElement(By.linkText("My Link"));
linkByText.click();

Click link by partial text using Selenium WebDriver

WebElement linkByPartialText = driver.findElement(By.partialLinkText("First"))
linkByPartialText.click();

Click link by text using XPath in Selenium WebDriver

WebElement linkByTextUsingXPath = driver.findElement(By.xpath("//a[text()='First']"));
linkByTextUsingXPath.click();

Click link by partial text using XPath in Selenium WebDriver

WebElement linkByPartialTextUsingXPath = driver.findElement(By.xpath("//a[contains(text(),'ABC')]"));
linkByPartialTextUsingXPath.click();

More details here on how to locate elements with Selenium WebDriver.

 

LoadFocus is a cloud testing platform for:

Filed under NodeJS, Screenshot Testing.

What is Puppeteer? Puppeteer is Node library that you can use in order to control Headless Chrome with the DevTools Protocol.

The Chrome DevTools Protocol allows for tools to instrument, inspect, debug and profile for Chromium and Chrome browsers.

Puppeteer – Headless Chrome Node API works only with Chrome and uses the latest versions of Chromium.

Chromium is an open-source browser project that forms the basis for the Chrome web browser. One of the biggest differences between the two browsers is that, while Chrome is based on Chromium, Google adds some of proprietary features to Chrome, features like automatic updates and support for additional video formats. Other features like usage-tracking or “user metrics” feature can be found only in Chrome browser.

Note: Puppeteer requires at least Node v6.4.0but the examples below use async/await which is only supported in Node v7.6.0 or greater.

Node.js has a simple module loading system. In Node.js, files and modules are in one-to-one correspondence (each file is treated as a separate module).

You can use Visual Regression Testing to take website screenshots and compare the generated images and identify differences pixel by pixel, a comparison image will be shown next to the result’s screenshot that highlights the differences in red.

visual regression testing tool

 

Install Puppeteer

Here is how to install puppeteer from NPM Modules Registry (npm is the package manager for JavaScript):

 npm i puppeteer 

Below are code snippets on how to use Puppeteer – Headless Chrome Node API in order to take screenshots of your website.

Example – navigating to https://example.com and saving a screenshot as a PNG file named example.png:

Generate screenshots with Puppetteer

 const puppeteer = require('puppeteer'); (async () =&amp;amp;amp;amp;amp;gt; { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://example.com'); await page.screenshot({path: 'example.png'}); await browser.close(); })(); 

await page.goto('https://example.com'), {
timeout: 120000,
slowMo: 2000,
waitUntil: ['load', 'networkidle'],
networkIdleTimeout: 5000,
});

By default, Puppeteer take screenshots of the viewport (the area which is visible by default when you open a website).

In order to take a screenshot of the full web page, you need to add the fullPage parameter to the screenshot method:


await page.screenshot({ path: 'example.png', fullPage: true });

Here is an example on how to take a screenshot of a webpage with Puppeteer using a customer web page size.
You just need to pass the width and height of the viewport in order for the browser to resize the web page to the desired size.


export async function takeScreenshot(page, filename, width, height) {
await page.setViewport({ width, height });

await page.screenshot({ path: `${filename}-${width}-${height}.png`, fullPage: true });
}

Here is how to call the above custom screenshot method:

await takeScreenshot(page, 'example.png', 320, 480);

Puppeteer, also provides a list of Mobile Devices as a list of objects called DeviceDescriptors.
In order to emulate a web page in a mobile emulator, with specific characteristics, you can import the
pre-defined list of mobile emulators from puppeteer/DeviceDescriptors

 const devices from 'puppeteer/DeviceDescriptors'; 

Puppeteer's API is very similar to Selenium WebDriver, but works only with Google Chrome, while WebDriver work with most popular browsers.

More details on how to locate elements to use in order to interact with Puppeteer or Selenium WebDriver.

Debugging and Troubleshooting Puppeteer

1. Non Headless Mode - for debugging purposes, sometimes it's useful to see what the browser is displaying. Instead of launching in headless mode, launch a full version of Chrome using headless: false when you launch the browser using Puppeteer:


const browser = await puppeteer.launch({headless: false});

2. Slow down screenshot generation - the slowMo option slows down Puppeteer operations by the specified amount of milliseconds. It's another way to understand better what's happening with the code you've written and debug easier.


const browser = await puppeteer.launch({
headless: false,
slowMo: 250 // slow down by 250ms
});

3. Capture browser's console output


page.on('console', msg =&amp;amp;amp;amp;amp;gt; console.log('PAGE LOG:', ...msg.args));

await page.evaluate(() =&amp;amp;amp;amp;amp;gt; console.log(`url is ${location.href}`));

4. Enable verbose logging - All public API calls and internal protocol traffic will be logged via the debug module under the puppeteer namespace.

# Basic verbose logging

env DEBUG="puppeteer:*" node script.js

You can use Visual Regression Testing to take website screenshots and compare the generated images and identify differences pixel by pixel, a comparison image will be shown next to the result’s screenshot that highlights the differences in red.

LoadFocus is a cloud testing platform for:

Filed under NodeJS.

Here are the steps you need to follow in order to debug ES6 code in the WebStorm IDE. After this, you’ll be able to take advantage of all debugging advantages, like setting breakpoints, moving away from console logs and faster understanding of the code of your application.

Prerequisites for Debugging ES6 in WebStorm IDE

 

1.Add the following NPM modules:

babel-core
babel-preset-es2015

3. Add the following devDependencies:

"devDependencies": {
    "gulp": "^3.9.1",
    "gulp-babel": "^6.1.2",
    "gulp-sourcemaps": "^2.4.1"
  }
 

2. Create a new gulp file, you can call it: gulp.babel.js, and paste the below code to this file:

import gulp from 'gulp';
import sourceMaps from 'gulp-sourcemaps';
import babel from 'gulp-babel';
import path from 'path';

const paths = {
  es6: ['./src/**/*.js', './test/**/*.js'],
  es5: './dist',
  
  sourceRoot: path.join(__dirname, 'src'),
};

gulp.task('babel', () => gulp.src(paths.es6)
    .pipe(sourceMaps.init())
    .pipe(babel({
      presets: ['es2015'],
    }))
    .pipe(sourceMaps.write('.', { sourceRoot: paths.sourceRoot }))
    .pipe(gulp.dest(paths.es5)));

gulp.task('watch', ['babel'], () => {
  gulp.watch(paths.es6, ['babel']);
});

gulp.task('default', ['watch']);

 

4. Install NPM modules
It is a good idea to install gulp globally:

 npm install gulp -g
 

Install the npm modules, by running npm install in the root of your application.

5. Run gulp from the root folder of your application, which will create the dist folder with the transpiled scripts + sourcemaps used for debugging ES6 code

Debug Mocha Unit Tests with WebStorm

  • in order to debug/add breakpoint to the Mocha tests, you need to do the following:
  • add `–compilers js:babel-core/register` the `WebStorm Mocha Configuration -> Extra Mocha options` field
  • start debugging the tests by adding breakpoints to the unit tests and while running in the Debug mode