Filed under Load Testing, Performance Testing.

Performance Testing, Load Testing and Stress Testing determine the behavior of the application under different types of usage. If you need to do a performance assessment of a website, web application, API you have to focus on a series of metrics. These are Response Time, Latency, Throughput, Bytes/second, Transactions/second, Hits/second and others.

For a performance test you need to see if the system meets the performance criteria. For example: render a website with 50 concurrent users with Response Times, CPU, Memory and other test metrics under expected limits, or login into the system 10 times per second.
For a load/stress test you need to identify areas in the architecture of your application which don’t function properly, either too slow or not functioning at all. Also it is very important to spot the moments when these happen and try get steps to reproduce the testing scenario.
Every system may have its custom performance testing, load testing and stress testing scenarios. You need to find out the best testing scenarios which suit your application’s architecture.
Load Testing and Stress Testing are often used for doing the same testing scenario, the fact is that these quality assurance terms are quite different and the goals are also different.
For a website, Load Testing would provide data for rendering or doing different actions on the website with a number of concurrent users and still have a good performance. You need to determine what good performance means for your System under Test.

Stress Testing will provide the date when the system under test will start throwing errors and maybe start crashing under a heavy load of actions/concurrent users.

It’s your duty to create the proper testing scenario for Performance Testing, Load Testing and Stress Testing for your System under Test.

LoadFocus.com – Cloud Performance, Load and Stress Testing Service provides the infrastructure and the ability to run all these tests in less than a few minutes, keep history of the results, compare different runs to inspect performance improvements or performance degradation. We also offer the possibility to simulate hits from locations all over the world. Scenarios like doing different actions while system is under heavy load are very easy to execute.

Try it now for free

Filed under Apache JMeter, Cloud Services.

Running your own custom JMeter test script in the cloud has never been easier.

To accomplish this you have to:

  • save your .jmx file (Apache JMeter file) to your disk,
  • log into https://loadfocus.com and create a new JMeter test
  • upload the your JMeter .jmx test file and execute the test

You can also run different JMeter (.jmx) test files at the same time from different injector machines. This will help you simulate different scenarios like: signing up when the website or API is under heavy load.

 

Hope this information helps, try JMeter Cloud Testing Service for free – Load Focus.

Filed under Load Testing, Page Load Time, Performance Testing, UI Testing.

Create automated website tests for free with LoadFocus.com.

Testing the UI (user interface) of web applications has been always a main concern of developers.
It is the first thing that a user/customer notices once something is wrong.

There quite a few solutions on the testing world out there. I will enumerate most of them here and
go in deep with the solution that I have found pretty attractive.

1. The first solution is the well known Selenium WeDriver which is by far the most used solution.

Pros:

  • well known by most of the automation developers 
  • offers good documentation
  • lots of information to be found on the internet
  • used successfully by lots of companies

Cons:

  • the main disadvantage is the fact that is pretty slow; running a long suite of tests can take quite some time
  • was not intended to be used for performance testing of the page load times
  • is not very accessible for developers which are not familiar with Selenium Webdriver
  • the web application that have a lot of dynamically embedded JS code are pretty hard to test (in this situation the solution would be to rely on Rhino or something similar but even in that case it is very tricky)

2. The second solution would be to use the now pretty known PhantomJS headless browser combined with CasperJS for
testing your web application.

Pros:

  • runs much faster than the Selenium based solution
  • tests are written in javascript which could be in the advantage of the frontend developers which are used with this
  • need to learn the API for creating the tests

Cons:

  • the automation engineers need to create an environment which can deploy the applications and start the tests
  • is not as out of the box as Selenium
  • can also be used for performance testing, but for measuring the page load time the developer cannot rely on the Navigation API as is notsupported by PhantomJS yet; the XHR requests cannot be measured 100% reliable
  • not extremely stable

3. The third solution I thought of is building a javascript solution which is embedded directly in the web application itself. The solution can be seen as it creates a self testable web application.

Pros:

  • runs quite fast
  • the tests are in pretty much the same place as the javascript used by the UI to render and add interactivity to the UI
  • the tests can have a built in user interface for running the tests on a test to test bases for debugging or developing the tests
  • the tests are based on javascript and any javascript library that the developer is used with (JQuery at least in my implementation of this solution) so are very appealing for the developers which do not have to get used with some other third party library or testing solution

Cons:

  • the framework must be  built from scratch as I did not find anything that does eactly what I wanted
  • the transition from one URL to another is pretty tricky as the test has to remember what test was active and which step it was at inside the test
  • can be used also for performance testing of the page load time

Developing a self testable web application using an embedded javascript framework

Objectives:

  •   a javascript framework that will be embedded directly in the web application
  •   the web application will have a specific run mode in which the test framework will be included otherwise it will not be included because of performance issues
  •   the framework should have a solution for managing the tests which will be implemented as the test manager (will retrieve the tests from different js files and make them accessible for running); also the manager will register the state of a test (active or not) and the step of the test (we need a step fragmentation of the test as in this way we can register the state when a URL transition is made)
  •   also a transition manager will be needed for managing the transitions between different URLs; this will be built based on the cookies or local storage support
  •   the framework should also have a UI interface which will be accessible when running the web application in test mode; this will help develop and debug the tests
  •   a performance module which will log all the timing information that can be retrieved from the Navigation Timing API which will be based on the JavaScript library created by Google’s Steve Souders

Implementation:

  •   implementation steps:
  •   general structure of the framework files and the test files
  •   determine the solution for creating specific run modes of your web application
  •   implementation of the test manager
  •   implementation of the transition manager
  •   implementation of the UI for the test framework
  •   implementation of the performance module
  •   implementation of very light web application which will retrieve the performance page load time information during the test run (this will be based on XHR beacons sent from the tested application to the light web application that will register the results and will be able to display the results using charts and graphs)
  •   the implementation details will be posted as the solution evolves so stay tune for the next post in the series.

Filed under Mobile Apps Testing, Web Testing Books.

Details on how to debug mobile applications, HTML5 or Native, will be available in the Web and Mobile Application Automate Testing Cookbook.

For every Software Developer or Quality Assurance Engineer interested into testing and debugging mobile HTML5 or native application, we are going to share:

  • tools to debug mobile application
  • how to inspect elements and CSS styles applied
  • how to set breakpoints and how to spot errors easily
  • how to find important issues first and how to used iOS Simulator to debug a web app

More details about tools, steps, and ways to debug mobile applications will be present in the book.

Subscribe now for a discount on the QA Cookbook, enter your email below or in the right hand form.

Filed under Web Testing Books.

Details of what means a good test plan and how to write a Test Plan for a feature will be available in the Web and Mobile Application Automate Testing Cookbook.

For every interested Quality Assurance Engineer, we are going to share:

  • essential points of structuring a test plan
  • how to group test cases is test suites
  • what is a test case and how to write a proper test case

More details about test plans, test suites, test cases will be present in the book.

Subscribe now for a discount on the QA Cookbook, enter your email below or in the right hand form.

Filed under Apache JMeter, Performance Testing.

With Load Focus you can run your own Apache JMeter test scripts (.jmx files).

All you need to do to run a performance test (to check web app or website rendering, REST / SOAP APIs behavior, or any other endpoint) is:

  • create a new Apache JMeter test after you login in the performance service
  • add an injector machine and select the type of injector machine you want to use (i.e. t1.micro) and location (i.e. North America)
  • upload the Apache JMeter test file into Load Focus
  • duplicate the injector machine or add new injector machines (you can upload different jmeter script files on different machines)
  • start the test and check the results
  • compare results with older test runs

More details about how to create a JMeter Performance Test script can be found in this blog post.

Start using Load Focus now and see how cost effective and reliable is the performance tool.

Add your email below for other QA tutorials.

Filed under CQ Testing, Performance Testing, Selenium WebDriver, Test Automation, UI Testing.

If you are using Adobe CQ for your web application, and you’d like to have an automated framework and automated tests to run the nightly, here is what we can offer:

1. Continuous integration infrastructure using Jenkins CI and custom created test framework

  • CI Jenkins/Hudson job created for all the below test types
  • CI Jenkins/Hudson job reporting for all the below test types

2. Integration test infrastructure based on JUnit or TestNG for different functionalities desired to be tested

  • test grouping based on feature
  • test grouping based on test types -smoke, release etc
  • framework for running integration tests
  • 10 demo tests + plus additional tests on demand

3. UI Testing using:

  • Selenium Webdriver & Thucydides
  • Webdriver tests
  • Javascript UI framework on demand
  • UI test framework
  • cross browser testing
  • mobile automation testing
  • 10 demo tests + plus additional tests on demand

4. Server side performance testing using:

  • JMeter scripts
  • Java and Sling performance framework
  • performance tests integrated with Jenkins for running nightly
  • 10 demo tests + plus additional tests on demand

5. Client performance testing:

  • page load time measurements using PhantomJS
  • page load time measurements using Webdriver
  • 10 demo tests + plus additional tests on demand

6. CQ server side test infrastructure for your application

7. BDD tests based on JBehave or Cucumber

8. Sling health check integration

Entire infrastructure implementation time: 1 – 2 months.

Subscribe here for free audit: click here