mirror of
https://github.com/dorny/test-reporter.git
synced 2025-12-15 22:07:09 +01:00
Improve README and action.yml
This commit is contained in:
Michal Dorner
parent
8fdf2aead9
commit
92cb68e21e
7 changed files with 299 additions and 80 deletions
186
README.md
186
README.md
|
|
@ -1,12 +1,180 @@
|
|||
**WIP**
|
||||
# Test Reporter
|
||||
|
||||
# Test Check Report
|
||||
This [Github Action](https://github.com/features/actions) displays test results from popular testing frameworks directly in GitHub.
|
||||
- [x] Parses test results in XML or JSON format and creates nice report (Github Check Run)
|
||||
- [x] Annotates code where it failed based on message and stack trace captured during test execution
|
||||
- [x] Sets output variable conclusion to `success` if all tests passed or `failure` if any test failed
|
||||
|
||||
Goal of this project is to create [Github Action](https://github.com/features/actions)
|
||||
that could presents test results from popular testing frameworks as Github check run with code annotation in places where test failed.
|
||||
**Supported languages / frameworks:**
|
||||
- .NET / [xUnit](https://xunit.net/) / [NUnit](https://nunit.org/) / [MSTest](https://github.com/Microsoft/testfx-docs)
|
||||
- Dart / [test](https://pub.dev/packages/test)
|
||||
- Flutter / [test](https://pub.dev/packages/test)
|
||||
- JavaScript / [JEST](https://jestjs.io/)
|
||||
|
||||
Support for following test reports are planned for initial release:
|
||||
- [ ] dart-json: [`dart test --file-reporter=\"json:test-results.json`](https://pub.dev/packages/test)
|
||||
- [ ] dotnet-trx: [`dotnet test --logger "trx;LogFileName=test-results.trx"`](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#examples)
|
||||
- [ ] flutter-machine: [`flutter test --machine > test-results.json`](https://flutter.dev/docs/cookbook/testing/unit/introduction)
|
||||
- [ ] jest-junit: [`jest --ci --reporters=jest-junit`](https://github.com/jest-community/jest-junit#readme)
|
||||
For more information see [Supported formats](#supported-formats) section.
|
||||
|
||||
**Support is planned for:**
|
||||
- Java / [JUnit 5](https://junit.org/junit5/)
|
||||
|
||||
Do you miss support for your favorite language or framework?
|
||||
Please create [Issue](https://github.com/dorny/test-reporter/issues/new) or contribute with PR.
|
||||
|
||||
## Example
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
build-test:
|
||||
name: 'Build & Test'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v2 # checkout the repo
|
||||
- run: npm ci # install packages
|
||||
- run: npm test # run tests (configured to use jest-junit reporter)
|
||||
|
||||
- name: 'Test Report'
|
||||
uses: dorny/test-reporter@v1
|
||||
if: always() # run this step even if previous step failed
|
||||
with:
|
||||
name: 'JEST Tests' # Name of the check run which will be created
|
||||
path: '__reports__/jest-*.xml' # Path to test report
|
||||
reporter: 'jest-junit' # Format of test report
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```yaml
|
||||
- uses: dorny/test-reporter@v1
|
||||
with:
|
||||
|
||||
# Name of the Check Run which will be created
|
||||
name: ''
|
||||
|
||||
# Path to test report
|
||||
# Supports wildcards via [fast-glob](https://github.com/mrmlnc/fast-glob)
|
||||
# Path may match multiple result files of same format
|
||||
path: ''
|
||||
|
||||
# Format of test report. Supported options:
|
||||
# dart-json
|
||||
# dotnet-trx
|
||||
# flutter-machine
|
||||
# jest-junit
|
||||
reporter: ''
|
||||
|
||||
# Enables code annotations with error message and stack trace
|
||||
annotations: 'true'
|
||||
|
||||
# Set action as failed if test report contain any failed test
|
||||
fail-on-error: 'true'
|
||||
|
||||
# Relative path under $GITHUB_WORKSPACE where the repository was checked out.
|
||||
working-directory: ''
|
||||
|
||||
# Personal access token used to interact with Github API
|
||||
# Default: ${{ github.token }}
|
||||
token: ''
|
||||
```
|
||||
|
||||
## Supported formats
|
||||
|
||||
<details>
|
||||
<summary>dart-json</summary>
|
||||
|
||||
Test run must be configured to use [JSON](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md#reporter) reporter.
|
||||
You can configure it in `dart_test.yaml`:
|
||||
|
||||
```yml
|
||||
file_reporters:
|
||||
json: reports/test-results.json
|
||||
```
|
||||
|
||||
Or with CLI arguments:
|
||||
|
||||
[`dart test --file-reporter="json:test-results.json"`](https://pub.dev/packages/test)
|
||||
|
||||
For more information see:
|
||||
- [test package](https://pub.dev/packages/test)
|
||||
- [test configuration](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md)
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>dotnet-trx</summary>
|
||||
|
||||
Test execution must be configured to produce *Visual Studio Test Results* files (TRX).
|
||||
To get test results in TRX format you can execute your tests with CLI arguments:
|
||||
|
||||
`dotnet test --logger "trx;LogFileName=test-results.trx"`
|
||||
|
||||
Or you can configure TRX test output in `*.csproj` or `Directory.Build.props`:
|
||||
```xml
|
||||
<PropertyGroup>
|
||||
<VSTestLogger>trx%3bLogFileName=$(MSBuildProjectName).trx</VSTestLogger>
|
||||
<VSTestResultsDirectory>$(MSBuildThisFileDirectory)/reports</VSTestResultsDirectory>
|
||||
</PropertyGroup>
|
||||
```
|
||||
|
||||
Supported testing frameworks:
|
||||
- [xUnit](https://xunit.net/)
|
||||
- [NUnit](https://nunit.org/)
|
||||
- [MSTest](https://github.com/Microsoft/testfx-docs)
|
||||
|
||||
For more information see [dotnet test](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#examples)
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>flutter-machine</summary>
|
||||
|
||||
Test run must be configured to use [JSON](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md#reporter) reporter.
|
||||
You can configure it in `dart_test.yaml`:
|
||||
```yml
|
||||
file_reporters:
|
||||
json: reports/test-results.json
|
||||
```
|
||||
|
||||
Or with (undocumented) CLI argument:
|
||||
|
||||
`flutter test --machine > test-results.json`
|
||||
|
||||
|
||||
According to documentation `dart_test.yaml` should be at the root of the package, next to the package's pubspec.
|
||||
It works in dart projects but the file is ignored by flutter. With flutter it works when `dart_test.yaml` is placed inside your `test` folder.
|
||||
|
||||
For more information see:
|
||||
- [test package](https://pub.dev/packages/test)
|
||||
- [test configuration](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md)
|
||||
- [flutter-cli](https://flutter.dev/docs/reference/flutter-cli)
|
||||
- [unit testing introduction](https://flutter.dev/docs/cookbook/testing/unit/introduction)
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>jest-junit</summary>
|
||||
|
||||
[JEST](https://jestjs.io/) testing framework support requires usage of [jest-junit](https://github.com/jest-community/jest-junit) reporter.
|
||||
It will create test results in junit XML format which can be then processed by this action.
|
||||
You can use following example configuration in `package.json`:
|
||||
```json
|
||||
"scripts": {
|
||||
"test": "jest --ci --reporters=default --reporters=jest-junit"
|
||||
},
|
||||
"devDependencies": {
|
||||
"jest": "^26.5.3",
|
||||
"jest-junit": "^12.0.0"
|
||||
},
|
||||
"jest-junit": {
|
||||
"outputDirectory": "__reports__",
|
||||
"outputName": "jest-junit.xml",
|
||||
"ancestorSeparator": " › ",
|
||||
"uniqueOutputName": "false",
|
||||
"suiteNameTemplate": "{filepath}",
|
||||
"classNameTemplate": "{classname}",
|
||||
"titleTemplate": "{title}"
|
||||
}
|
||||
```
|
||||
|
||||
Configuration of `uniqueOutputName`, `suiteNameTemplate`, `classNameTemplate`, `titleTemplate` is important for proper visualization of test results.
|
||||
</details>
|
||||
|
||||
## License
|
||||
|
||||
The scripts and documentation in this project are released under the [MIT License](https://github.com/dorny/test-reporter/blob/master/LICENSE)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue