What is TestNG?

What is TestNG?

TestNG is an open-source Java testing framework inspired by JUnit and NUnit. You mark tests and setup code with annotations, and it adds groups, data providers, test dependencies and parallel execution on top. It is designed to cover unit, functional, end-to-end and integration tests, which is why it is common in Selenium UI suites. The name stands for Test Next Generation.

How TestNG works

A test is a method annotated with @Test inside a normal Java class. Setup and teardown use configuration annotations that run at different levels: @BeforeSuite, @BeforeTest, @BeforeClass and @BeforeMethod, each with an @After... twin. You run the tests from your IDE, from Maven or Gradle, or from a testng.xml file that lists which classes, packages or groups to include.

import org.testng.annotations.*;
import static org.testng.Assert.*;

public class CalculatorTest {
    private Calculator calc;

    @BeforeMethod
    public void setUp() { calc = new Calculator(); }

    @Test(groups = {"smoke"})
    public void addsTwoNumbers() {
        assertEquals(calc.add(2, 3), 5);
    }

    @Test(dataProvider = "divisions")
    public void divides(int a, int b, int expected) {
        assertEquals(calc.divide(a, b), expected);
    }

    @DataProvider
    public Object[][] divisions() {
        return new Object[][] { {10, 2, 5}, {20, 4, 5}, {100, 10, 10} };
    }
}

Each @BeforeMethod runs before every test, and the data provider feeds three input sets into divides, so the report shows three test results from one method.

Key TestNG features

• Groups: tag a test with @Test(groups = {"smoke"}) and choose groups in testng.xml, for example smoke tests on every commit and the full set nightly.
• Dependencies: dependsOnMethods runs a test after the methods it names. If one of them fails, the dependent test is marked SKIP, not FAIL, unless you set alwaysRun=true.
• Data providers: @DataProvider runs one test method against many inputs. Setting parallel = true on the provider runs those invocations in parallel. See how to use the DataProvider in TestNG for a worked example.
• Retry analyzers: implement IRetryAnalyzer and bind it with @Test(retryAnalyzer = ...) to re-run failed tests.
• Listeners: hook into test start, success and failure, for example to save a screenshot when a UI test fails.
• JUnit compatibility: TestNG can run JUnit 3 and JUnit 4 tests, which helps during a gradual migration.

Running TestNG tests in parallel

Parallelism is set on the suite in testng.xml. The parallel attribute picks the unit that gets its own thread, and thread-count sets how many threads are available:

• methods: each test method runs in its own thread.
• tests: each <test> block gets a thread, and its methods share it.
• classes: each class gets a thread.
• instances: methods on the same instance share a thread; different instances can run side by side.

<suite name="Regression" parallel="methods" thread-count="4">
  <test name="Smoke">
    <groups><run><include name="smoke"/></run></groups>
    <packages><package name="com.example.tests"/></packages>
  </test>
</suite>

Parallel data providers use their own pool, set with data-provider-thread-count (the default is 10). Parallel runs only help when tests are independent. For Selenium, keep one WebDriver per thread, for example in a ThreadLocal.

TestNG vs JUnit 5

Both are mature and both work with Maven, Gradle, Selenium and REST Assured. The practical differences are in what is built in:

• Suites and groups: TestNG ships testng.xml and groups. JUnit 5 uses @Tag and build-tool filters.
• Dependencies between tests: TestNG supports them. JUnit 5 does not by design, which pushes you toward independent tests.
• Parameterized tests: @DataProvider in TestNG, @ParameterizedTest in JUnit 5.
• Parallel execution: built into TestNG's suite configuration. JUnit 5 can run in parallel too, enabled through configuration.

For new unit tests either works, so follow your team's existing tooling. Mixing them in one project is possible but adds confusion.

Common TestNG mistakes

• Shared state between tests. A test that passes alone but fails in the suite usually relies on a static field another test changed.
• One WebDriver shared across threads. It works serially and breaks the day you turn on parallel.
• Long dependsOnMethods chains. One failure skips everything downstream and hides the original cause.
• Fixed sleeps. Thread.sleep(5000) slows the suite and still flakes. Wait for an explicit condition instead.
• Unlimited retries. Retrying a flaky test until it passes hides real bugs. Cap retries at one or two and investigate any test that needs them.
• A testng.xml that lists every test by hand. Point it at packages and groups and let TestNG discover the classes.

TestNG, regression testing and performance

TestNG answers "does this still behave correctly?", which is the job of regression testing. It does not tell you how the system behaves under many simultaneous users. For that you need load testing, and for Java systems the usual tool is JMeter. LoadFocus runs existing JMeter scripts unchanged from 25 cloud regions, and API monitoring checks status codes, response bodies and latency on a schedule once the code is in production. TestNG tests, load tests and monitors cover different failure modes, so none of them replaces the others.

FAQ: TestNG

Should I use TestNG or JUnit 5 for a new project?

Either is a sound choice. TestNG gives you suite files, groups, dependencies and parallel settings out of the box, which suits large Selenium suites. JUnit 5 is a good fit for plain unit tests. Pick the one your team and your build already use.

Is TestNG still maintained?

Yes, TestNG 7.x is still being released. Versions from 7.6.0 need JDK 11 or newer, while 7.5 and earlier run on JDK 8. Check the TestNG site or Maven Central for the current release.

How do I run TestNG tests in parallel?

Set parallel and thread-count on the suite in testng.xml, for example <suite parallel="methods" thread-count="4">. Valid values are methods, tests, classes and instances. Make sure the tests do not share mutable state first.

Can TestNG run JUnit tests?

It can run JUnit 3 and JUnit 4 tests, by marking the test with junit="true" in testng.xml. JUnit 5 (Jupiter) tests need the JUnit Platform instead.

Why was my TestNG test skipped instead of failed?

A test that uses dependsOnMethods is skipped when a method it depends on fails. Fix the failing dependency first, or add alwaysRun=true if the test should run regardless.