Test Reports

Test reports are essential in understanding how your tests are performing. They provide details about passed and failed tests, errors, and execution time. Playwright makes it easy to generate and analyze test reports with its built-in reporting tools.

Why Use Test Reports?

Test reports help you:

• Identify failed tests and understand why they failed.
• Measure the time taken by each test.
• Share test results with your team.
• Track test performance over time.

Types of Reports in Playwright

Playwright supports various reporting formats, such as:

1. Default Console Report: A summary displayed in the terminal.
2. HTML Report: A visually appealing, detailed test report in a browser.
3. JSON Report: A structured report for programmatic analysis.
4. JUnit XML Report: Compatible with CI/CD tools like Jenkins.

Configuring Reports in Playwright

Reports are configured in the playwright.config.ts file. Let’s explore how to set them up.

Default Reporter (Console Output)

The default Playwright test runner outputs results to the terminal.

Run the tests:

Output example:

Adding Custom Reporters

Playwright allows you to add multiple reporters. Specify them in the reporter field in your configuration file.

Example Configuration:

const config = {
  reporter: [
    ['list'], // Console reporter
    ['html', { outputFolder: 'html-report' }], // HTML report
    ['junit', { outputFile: 'results.xml' }], // JUnit XML report
  ],
};
export default config;

Generating HTML Reports

Step 1: Configure HTML Reporter

Add the HTML reporter to your playwright.config.ts:

const config = {
  reporter: [['html', { outputFolder: 'test-reports' }]],
};
export default config;

Step 2: Run the Tests

Execute your tests:

Step 3: Open the Report

After the tests finish, open the report:

What You'll See in the HTML Report

• A summary of all tests: passed, failed, skipped.
• Execution time for each test.
• Screenshots, videos, and trace logs for failed tests (if enabled).

Generating JUnit XML Reports

JUnit XML reports are used in CI/CD pipelines.

Step 1: Configure JUnit Reporter

Add the JUnit reporter in playwright.config.ts:

const config = {
  reporter: [['junit', { outputFile: 'junit-results.xml' }]],
};
export default config;

Step 2: Run the Tests

Run your tests:

Step 3: Use JUnit Report in CI/CD

Integrate the junit-results.xml file into tools like Jenkins, CircleCI, or GitHub Actions for detailed reporting.

Debugging with JSON Reports

JSON reports are useful for custom analysis.

Step 1: Configure JSON Reporter

Add the JSON reporter in playwright.config.ts:

const config = {
  reporter: [['json', { outputFile: 'report.json' }]],
};
export default config;

Step 2: Parse the JSON Report

The JSON file contains detailed information about each test run, including:

• Test names.
• Status (passed, failed).
• Error messages.
• Execution time.

You can use this data to create custom dashboards or analytics.

Adding Screenshots, Videos, and Traces to Reports

Enhance your reports with visuals for better debugging.

Screenshots

Add screenshots for failed tests:

const config = {
  use: {
    screenshot: 'only-on-failure', // Options: 'off', 'on', 'only-on-failure'
  },
};
export default config;

Videos

Record videos for test sessions:

const config = {
  use: {
    video: 'retain-on-failure', // Options: 'off', 'on', 'retain-on-failure'
  },
};
export default config;

Trace Viewer

Generate trace files for failed tests to analyze browser activity:

const config = {
  use: {
    trace: 'retain-on-failure', // Options: 'off', 'on', 'retain-on-failure'
  },
};
export default config;

Viewing Traces

Open the trace file: