test-reporter/README.md

Test Reporter

This Github Action displays test results from popular testing frameworks directly in GitHub.

• Parses test results in XML or JSON format and creates nice report (Github Check Run)
• Annotates code where it failed based on message and stack trace captured during test execution
• Sets output variable conclusion to success if all tests passed or failure if any test failed

Supported languages / frameworks:

• .NET / xUnit / NUnit / MSTest
• Dart / test
• Flutter / test
• JavaScript / JEST

For more information see Supported formats section.

Support is planned for:

• Java / JUnit 5

Do you miss support for your favorite language or framework? Please create Issue or contribute with PR.

Example

jobs:
  build-test:
    name: 'Build & Test'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2     # checkout the repo
      - run: npm ci                   # install packages
      - run: npm test                 # run tests (configured to use jest-junit reporter)

      - name: 'Test Report'
        uses: dorny/test-reporter@v1
        if: always()                  # run this step even if previous step failed
        with:
          name: 'JEST Tests'          # Name of the check run which will be created
          path: 'reports/jest-*.xml'  # Path to test report
          reporter: 'jest-junit'      # Format of test report

Usage

- uses: dorny/test-reporter@v1
  with:

    # Name of the Check Run which will be created
    name: ''

    # Coma separated list of paths to test reports
    # Supports wildcards via [fast-glob](https://github.com/mrmlnc/fast-glob)
    # All matched result files must be of same format
    path: ''

    # Format of test report. Supported options:
    #   dart-json
    #   dotnet-trx
    #   flutter-json
    #   jest-junit
    reporter: ''

    # Limits which test suites are listed:
    #   all
    #   failed
    list-suites: 'all'

    # Limits which test cases are listed:
    #   all
    #   failed
    #   none
    list-tests: 'all'

    # Limits number of created annotations with error message and stack trace captured during test execution.
    # Must be less or equal to 50.
    max-annotations: '10'

    # Set action as failed if test report contain any failed test
    fail-on-error: 'true'

    # Relative path under $GITHUB_WORKSPACE where the repository was checked out.
    working-directory: ''

    # Personal access token used to interact with Github API
    # Default: ${{ github.token }}
    token: ''

Supported formats

dart-json

Test run must be configured to use JSON reporter. You can configure it in dart_test.yaml:

file_reporters:
  json: reports/test-results.json

Or with CLI arguments:

dart test --file-reporter="json:test-results.json"

For more information see:

• test package
• test configuration

dotnet-trx

Test execution must be configured to produce Visual Studio Test Results files (TRX). To get test results in TRX format you can execute your tests with CLI arguments:

dotnet test --logger "trx;LogFileName=test-results.trx"

Or you can configure TRX test output in *.csproj or Directory.Build.props:

<PropertyGroup>
  <VSTestLogger>trx%3bLogFileName=$(MSBuildProjectName).trx</VSTestLogger>
  <VSTestResultsDirectory>$(MSBuildThisFileDirectory)/TestResults/$(TargetFramework)</VSTestResultsDirectory>
</PropertyGroup>

Supported testing frameworks:

• xUnit
• NUnit
• MSTest

For more information see dotnet test

flutter-json

Test run must be configured to use JSON reporter. You can configure it in dart_test.yaml:

file_reporters:
  json: reports/test-results.json

Or with (undocumented) CLI argument:

flutter test --machine > test-results.json

According to documentation dart_test.yaml should be at the root of the package, next to the package's pubspec. It works in dart projects but the file is ignored by flutter. With flutter it works when dart_test.yaml is placed inside your test folder.

For more information see:

• test package
• test configuration
• flutter-cli
• unit testing introduction

jest-junit

JEST testing framework support requires usage of jest-junit reporter. It will create test results in junit XML format which can be then processed by this action. You can use following example configuration in package.json:

"scripts": {
  "test": "jest --ci --reporters=default --reporters=jest-junit"
},
"devDependencies": {
  "jest": "^26.5.3",
  "jest-junit": "^12.0.0"
},
"jest-junit": {
  "outputDirectory": "reports",
  "outputName": "jest-junit.xml",
  "ancestorSeparator": " › ",
  "uniqueOutputName": "false",
  "suiteNameTemplate": "{filepath}",
  "classNameTemplate": "{classname}",
  "titleTemplate": "{title}"
}

Configuration of uniqueOutputName, suiteNameTemplate, classNameTemplate, titleTemplate is important for proper visualization of test results.

See also

• paths-filter - Conditionally run actions based on files modified by PR, feature branch or pushed commits

License

The scripts and documentation in this project are released under the MIT License