Groups

TestNG groups is one of the useful feature that allows you to group your test cases based on different category. You can add test methods of one class or different class to a different group. Also you can a test method to multiple groups as well. Grouping adds more flexibility in test execution. To create a group, add annotation @groups to the test method and add desired group names as string by comma separated inside curly braces. Let's look at an example of how to use TestNG groups.

Create a java class file GroupExample1.java

Java
Copy
package org.example;

import org.testng.annotations.Test;

public class GroupExample1 {
    @Test(groups = {"Smoke","Regression", "Feature1"})
    public void ExampleTestMethod1()
    {
        System.out.println("This is GroupExample1.ExampleTestMethod1");
    }
    @Test(groups = {"Smoke","Feature1"})
    public void ExampleTestMethod2()
    {
        System.out.println("This is GroupExample1.ExampleTestMethod2");
    }
    @Test(groups = {"Regression","Feature2"})
    public void ExampleTestMethod3()
    {
        System.out.println("This is GroupExample1.ExampleTestMethod3");
    }
    @Test(groups = {"Feature3"})
    public void ExampleTestMethod4()
    {
        System.out.println("This is GroupExample1.ExampleTestMethod4");
    }
}

Create a java class file GroupExample2.java

Java
Copy
package org.example;

import org.testng.annotations.Test;
public class GroupExample2 {
    @Test(groups = {"Regression", "Feature1"})
    public void ExampleTestMethod1()
    {
        System.out.println("This is GroupExample2.ExampleTestMethod1");
    }
    @Test(groups = {"Feature2"})
    public void ExampleTestMethod2()
    {
        System.out.println("This is GroupExample2.ExampleTestMethod2");
    }
    @Test(groups = {"Regression","Feature2"})
    public void ExampleTestMethod3()
    {
        System.out.println("This is GroupExample2.ExampleTestMethod3");
    }
    @Test(groups = {"Feature3"})
    public void ExampleTestMethod4()
    {
        System.out.println("This is GroupExample2.ExampleTestMethod4");
    }
}

Create a java class file testng.xml file

XML
Copy
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="DemoTestSuite">
    <test name="Demo Test">
        <groups>
            <run>
                <include name="Regression"/>
            </run>
        </groups>
        <classes>
            <class name="org.example.GroupExample1"/>
            <class name="org.example.GroupExample2"/>
        </classes>
    </test>
</suite>

Now let's understand the above example in more detail and will try to execute and see the results.

In the first java file GroupExample1.java we have the following:

  • Four test methods which are part of five groups ie Smoke, Regression, Feature1, Feature2 and Feature3.
  • First test method ExampleTestMethod1 is part of three groups ie Smoke, Regression and Feature1.
  • Second test method ExampleTestMethod2 is part of two groups ie Smoke and Feature1
  • Third test method ExampleTestMethod3 is part of two groups ie Regression and Feature2
  • Fourth test method ExampleTestMethod4 is part of one groups ie Feature3

In the second java file GroupExample2.java we have the following:
  • Four test methods which are part of four groups ie Regression, Feature1, Feature2 and Feature3.
  • First test method ExampleTestMethod1 is part of two groups ie Regression and Feature1.
  • Second test method ExampleTestMethod2 is part of one group ie and Feature2
  • Third test method ExampleTestMethod3 is part of two groups ie Regression and Feature2
  • Fourth test method ExampleTestMethod4 is part of one groups ie Feature3

In the tesng.xml we have the following:
  • Added both above classes which allows the TestNG to access the test methods from both the classes.
  • Created a groups element which instructs the TestNG that we are instructing to run some groups.
  • Under groups we have added sub element run, this instructs TestNG to run all the sub groups.
  • Under run element an include sub element with name attribute value of group name Regression which we added in the java class. This instructs TestNG to include this as part of run.

Now Run the above testng.xml file. You should see below results.


This is GroupExample1.ExampleTestMethod1
This is GroupExample1.ExampleTestMethod3
This is GroupExample2.ExampleTestMethod1
This is GroupExample2.ExampleTestMethod3
===============================================
DemoTestSuite
Total tests run: 4, Passes: 4, Failures: 0, Skips: 0
===============================================

Now let's make some changes in the testng.xml file and add another element under run to exclude test methods that are part of smoke group. The updated testng.xml file will look like the following.

XML
Copy
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="DemoTestSuite">
    <test name="Demo Test">
        <groups>
            <run>
                <include name="Regression"/>
                <exclude name="Smoke"/>
            </run>
        </groups>
        <classes>
            <class name="org.example.GroupExample1"/>
            <class name="org.example.GroupExample2"/>
        </classes>
    </test>
</suite>

Now run the above updated testng.xml file. You may see all the test methods that are part of smoke group will be excluded from the run even though they are also part of regression. The console log will look like the following.

This is GroupExample1.ExampleTestMethod3
This is GroupExample2.ExampleTestMethod1
This is GroupExample2.ExampleTestMethod3
===============================================
DemoTestSuite
Total tests run: 3, Passes: 3, Failures: 0, Skips: 0
===============================================

Now let's make one following changes in the testng.xml file:

  • Exclude all groups with name Smoke
  • Exclude all groups with name Regression
  • Include all the groups that are part of any feature.
Since we have three feature groups (Feature1, Feature2 and Feature3), we do not need to add each feature name in testng.xml. Rather we will use regular expression(.*) at the end of the group name to match all the group that starts with name Feature to run all the groups that are matching name Feature. Use the following updated xml file to run the groups using regular expression.

XML
Copy
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="DemoTestSuite">
    <test name="Demo Test">
        <groups>
            <run>
                <exclude name="Regression"/>
                <exclude name="Smoke"/>
                <include name="Feature.*"/>
            </run>
        </groups>
        <classes>
            <class name="org.example.GroupExample1"/>
            <class name="org.example.GroupExample2"/>
        </classes>
    </test>
</suite>

Run the above testng.xml file and you can see in the log as all test methods that are part of the groups with name starts with Feature are run excluding the groups with name Regression and Smoke. You should see below result.

This is GroupExample1.ExampleTestMethod4
This is GroupExample2.ExampleTestMethod2
This is GroupExample2.ExampleTestMethod4
===============================================
DemoTestSuite
Total tests run: 3, Passes: 3, Failures: 0, Skips: 0
===============================================