rtems/rtems-docs
1
0
Fork 0
mirror of https://git.rtems.org/rtems-docs/ synced 2025-05-14 13:39:45 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update docs for log mode change and JSON reporting

Browse Source 
The option --report-mode has been changed to --log-mode to reduce ambiguity
and --report-format has been added along with --report-path for machine
readable test reporting.
This commit is contained in:
Kinsey Moore 2019-08-21 20:41:43 -05:00 committed by Chris Johns
parent 3605600247
commit b836c43f19
1 changed files with 21 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
user/tools/tester.rst
View File

@ -251,7 +251,7 @@ timeouts and 1 invalid test.
* ``sparc/erc32`` is the architecture and BSP names. * ``sparc/erc32`` is the architecture and BSP names.
* ``hello.exe`` is the executable name. * ``hello.exe`` is the executable name.
The test log records all the tests and results. The reporting mode by default The test log records all the tests and results. The logging mode by default
only provides the output history if a test fails, times out, or is invalid. The only provides the output history if a test fails, times out, or is invalid. The
time taken by each test is also recorded. time taken by each test is also recorded.
@ -317,16 +317,16 @@ If no start marker is seen the test is marked as invalid. If you are testing on
real target hardware things can sometimes go wrong and the target may not real target hardware things can sometimes go wrong and the target may not
initialize or respond to the debugger in an expected way. initialize or respond to the debugger in an expected way.
Reporting Logging
--------- -------
The report written to the log has the following modes: The following modes of logging are available:
* All (``all``) * All (``all``)
* Failures (``failures``) * Failures (``failures``)
* None (``none``) * None (``none``)
The mode is controlled using the command line option ``--report-mode`` using The mode is controlled using the command line option ``--log-mode`` using
the values listed above. the values listed above.
All All
@ -410,6 +410,17 @@ running on a Coldfire MCF5235 using GDB and a BDM pod:
* Line starting with ``]`` are from the target's console. * Line starting with ``]`` are from the target's console.
* The result with the test time. * The result with the test time.
Reporting
---------
The RTEMS Tester supports output in a machine parsable format. This can be
enabled using the options "--report-path" and "--report-format". Currently,
JSON output is supported using these options like so:
'--report-path="report" --report-format=json'
This will produce a file "report.json" that contains output equivalent to the
"failure" logging mode.
Running Tests in Parallel Running Tests in Parallel
------------------------- -------------------------
@ -419,8 +430,8 @@ resource contention. Simulators are an example of back-ends that can run in
parallel. A hardware debug tool like a BDM or JTAG pod can manage only a parallel. A hardware debug tool like a BDM or JTAG pod can manage only a
single test at once so the tests need to be run one at a time. single test at once so the tests need to be run one at a time.
The test framework manages the test jobs and orders the output in the report The test framework manages the test jobs and orders the output in the log
log in test order. Output is held for completed tests until the next test to be in test order. Output is held for completed tests until the next test to be
reported has finished. reported has finished.
Command Line Help Command Line Help
@ -444,7 +455,9 @@ review the available option by the ``--help`` option:
--macros file[,file] : Macro format files to load after the defaults --macros file[,file] : Macro format files to load after the defaults
--no-clean : Do not clean up the build tree --no-clean : Do not clean up the build tree
--quiet : Quiet output (not used) --quiet : Quiet output (not used)
--report-mode : Reporting modes, failures (default),all,none --report-path : Report output base path (file extension will be added)
--report-format : Formats in which to report test results: json
--log-mode : Log modes, failures (default),all,none
--rtems-bsp : The RTEMS BSP to run the test on --rtems-bsp : The RTEMS BSP to run the test on
--rtems-tools : The path to the RTEMS tools --rtems-tools : The path to the RTEMS tools
--target : Set the target triplet --target : Set the target triplet