rtems/rtems-docs
1
0
Fork 0
mirror of https://git.rtems.org/rtems-docs/ synced 2025-10-17 09:38:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

user: Add RTEMS executable and test documentation.

Browse Source
This commit is contained in:
Chris Johns
2018-05-20 08:32:42 +12:00
parent 21c1a4492a
commit 8b67c9135c
33 changed files with 1934 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

312
user/testing/configuration.rst Normal file
View File

@@ -0,0 +1,312 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
Tester Configuration
--------------------
The RTEMS Tester and RTEMS Run are controlled by configuration data and
scripts. The user specifies a BSP on the command line using the ``--rtems-bsp``
option as well as optionally specifying a user configuration file using
``--user-config``.
The Figure :ref:`fig-tester-config-1` shows the various sources of
configuration data and their format. The ``ini`` files are the standard INI
format, the ``mc`` are the internal RTEMS Toolkit's Macro format, and ``cfg``
is the RTEMS Toolkit's Configuration script format, the same format used by the
RTEMS Source Builder.
.. _fig-tester-config-1:
.. figure:: ../../images/user/test-cfg-1.png
:width: 50%
:alt: RTEMS Tester and Run Configuration Files
:figclass: align-center
RTEMS Tester and Run Configuration Files
Configuration data is held in a macro database keyed on the macro name. Macros
can be expanded in configuration scripts using the syntax ``%{name}``. The
macro database is layered using maps. The defaults and values created when a
configure script runs live the in the ``global`` map. Values read from the BSP
and User INI configuration files are loaded into maps based on the BSP
name. This lets a single User configuration file contain specialized
configuration values for a number of BSPs and the tester and run commands
select the values based on the selected BSP. Macros are expanded using the BSP
map first giving those values the highest priority. User defined values are
loaded after the BSP configuration values overwriting them letting a user
speckles a BSP's default configuration for their local needs.
Figure :ref:`fig-tester-config-2` shows the configuration loading and script
execution order.
.. _fig-tester-config-2:
.. figure:: ../../images/user/test-cfg-2.png
:width: 50%
:alt: RTEMS Tester and Run Configuration Load and Execute Sequence
:figclass: align-center
RTEMS Tester and Run Configuration Load and Execute Sequence
Defaults
^^^^^^^^
The RTEMS Tester and RTEMS Run are primed using defaults from the file
``rtems/testing/testing.mc``. All default settings can be overridden in a BSP or
User configuration file.
.. index:: BSP configuration, User configuration
BSP and User Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^
The BSP and User configuration files are INI format files. The BSP
configuration file has to have an INI section that is the name of the BSP
passed on the command line. The section has the following mandatory values:
.. index:: bsp
``bsp``
The name of the BSP. The BSP name is used to create a macro map to hold the
BSP's configuration data. Typically this is the same as the BSP name used on
the command line.
.. index:: arch
``arch``
The name of the BSP architecture. This is need for the GDB configuration
scripts where the architecture specific GDB needs to run. It is mandatory so
the *arch/bsp* standard RTEMS BSP string can be used.
.. index:: tester
``tester``
The tester or run configuration script. This is the name of the configuration
script the RTEMS Tester or RTEMS Run executes as a back end. The ``tester``
value is typically of the form ``%{_rtscripts}/<script>`` where ``<script>``
is name of the back end script to be run.
Target commands support expansion of specific tags to provide a convenient way
for users to customize a local test environment. The parameters expanded are:
.. index:: @ARCH@
``@ARCH@``
The BSP architecture.
.. index:: @BSP@
``@BSP@``
The BSP's name set by the ``bsp`` value.
.. index:: @EXE@
``@EXE@``
The executable name as an absolute path
.. index:: @FEXE@
``@FEXE@``
The filtered executable if a ``target_exe_filter`` is provided else the
executable's file name.
The following are optional and depend on the back end being used and the local
target hardware set up:
.. index:: jobs
``jobs``
The jobs value sets the number of jobs that can be run at once. This setting
only effects the RTEMS Tester. The tester can run up to the ``jobs`` value of
tests concurrently. If the tester back end is a simulator running a job on
each available core lowers the total test time. Overloading a machine with
too many simulators running in parallel can slow down each simulation and
test timeouts may be recorded.
.. index:: bsp_tty_dev
``bsp_tty_dev``
The BSP's tty device. This can be a real device on the host machine the
executable is being run from or it can be a telnet server and port defined
using the stand host format. See :ref:`tester-consoles` for details.
.. index:: target_pretest_command
``target_pretest_command``
The pre-test command is a host shell command that is called before each test
runs. It can be used to construct a suitable environment or image needed by a
simulator or target. The RTEMS executate being run is provided as an argument
and the bootloader specific format is the output.
.. index:: target_posttest_command
``target_posttest_command``
The post-test command is a host shell command that is called after each test
has finished. It can be used to destroy any environment or image created by
the pre-test command.
.. index:: target_exe_filter
``target_exe_filter``
The target executable filter transforms the executable name into a filtered
executable name. This filter lets the tester or run command track the name of
any generated file a pre-test command may generate. The syntax is a simplified
``sed`` regular expression. The first character is a delimiter and there must
be 2 sections therefore 3 delimiter. The first section is a Python regular
expression and the second section is plain text that replaces anywhere the
regular expression matches. For example ``/\.exe/.exe.img/`` will search for
``.exe`` in the executable name and replace it with ``.exe.img``. Note, there
is no need to escape the text in the second part, it is just plain test.
.. index:: test_restarts
``test_restarts``
The number of restarts before the test is considered ``invalid``. Currently
not used.
.. index:: target_reset_regex
``target_reset_regex``
The target reset regular expression. This is a `Python regular expression
<https://docs.python.org/2/library/re.html#regular-expression-syntax>`_ used
to filter the console input. If a match is made something has happened during
the boot process that requires a reset. The ``target_reset_command`` is
issued to perform the reset. Typically this field looks for boot loader error
messages that indicate the boot process as failed.
.. index:: target_start_regex
``target_start_regex``
The target start regular expression. This is a Python regular expression to
filter the console input to asynchronously detect if a target has reset. If a
board crashes running a test or at any point reset this filter detects the
restart and ends the test with a suitable result.
.. index:: target_on_command
``target_on_command``
The target on command is a host shell command that is called before the first
test. This command powers on a target. Targets should be left powered off
when not running tests or the target may request TFTP downloads that are for
another target interfering with those test results. We recommend you
implement this command as a target off command, a pause, then a target on
command.
.. index:: target_off_command
``target_off_command``
The target off command is a host shell command that is called after the last
test powering off the target.
.. index:: target_reset_command
``target_reset_command``
The target reset command is a host shell command that is called when the
target needs to be reset. This command can power cycle the target or toggle a
reset signal connected to the target. If you are power cycling a target make
sure you have a suitable pause to let the target completely power down.
.. _tester-config-scripts:
Configuration Scripts
^^^^^^^^^^^^^^^^^^^^^
Configuration scripts are provided for each supported RTEMS Tester and RTEMS
Run back end and console management. The scripts are in the standard RTEMS
Toolkit Configuration Script format. Please refer to the RTEMS Source Builder
documentation for the basic scripting syntax and usage.
The RTEMS Tester and RTEMS Run specializes the standard configuration syntax
providing a directive for the console and each supported back end. The
supported directives are:
- ``%console``
- ``%execute``
- ``%gdb``
- ``%tftp``
.. _tester-config-console:
.. index:: Console, %console
Console
~~~~~~~
The ``%console`` configures the console used to access the target's
console. The console can be a process's ``stdout``, a termios tty on Unix and
MacOS and Telnet on all hosts. The directive accepts:
``stdio``
The standard output stream from the executing processing.
``tty <dev> <settings>``
The name of the ``tty`` to open and use. The ``tty`` device or ``<dev>`` can
be a *termio* device and the ``<settings>`` are standard termios values.
The Python termios document provides details of the settings that can be
controlled. The settings are a single string where prefix the value with
``~`` negates the setting. Setting are:
- ``B115200`` (an example buadrate)
- ``BRKINT``
- ``IGNBRK``
- ``IGNCR``
- ``ICANON``
- ``ISIG``
- ``IEXTEN``
- ``ECHO``
- ``CLOCAL``
- ``CRTSCTS``
- ``VMIN=<value>``
- ``VTIME=<value``
A example in a configuration script is::
%define bsp_tty_dev /dev/ttyUSB2
%define bsp_tty_settings B115200,~BRKINT,IGNBRK,IGNCR,~ICANON,~ISIG,~IEXTEN,~ECHO,CLOCAL,~CRTSCTS,VMIN=1,VTIME=2
A example BSP or User configuration file is::
[bsp-special]
bsp = example-bsp
bsp_tty_dev = /dev/ttyUSB2
bsp_tty_settings = B115200,~BRKINT,IGNBRK,IGNCR,~ICANON,~ISIG,~IEXTEN,~ECHO,CLOCAL,~CRTSCTS,VMIN=1,VTIME=2
The console directive is managed in the ``%{_rtscripts}/console.cfg``
configuration script. If the ``%{console_stdio}`` is defined the console will
be ``stdio`` else the console will be the BSP console or ``%{bsp_tty_dev}``.
Telnet can be combined with the ``ser2net`` daemon to remotely access a
target's physical serial UART interface.
.. _tester-config-execute:
.. index:: Execute, %execute
Execute
~~~~~~~
The ``%execute`` directive executes a command for each rest. The execute forks
the command and arguments supplied to the execute directive and captures the
``stdout`` stream as the console. If the console directive is set to ``stdout``
the sub-processes ``stdout`` stream is used as the console.
The RTEMS Tester will run parallel tests as jobs.
An example is::
%execute %{run_cmd} %{run_opts} %{test_executable} %{test_executable_opts}
.. _tester-config-gdb:
.. index:: GDB, %gdb
GDB
~~~
The ``%gdb`` directive executes GDB in the machine interface mode give the
RTEMS Tester and RTEMS Run commands control. The console is taken from
GDB if it is ``stdout``.
The RTEMS Tester will run parallel tests as jobs.
An example is::
%gdb %{gdb_cmd} %{test_executable} %{gdb_script}
.. _tester-config-tftp:
.. index:: TFTP, %tftp
TFTP
~~~~
The ``%tftp`` directive starts a TFTP session on a specified port sending the
test executable to the target over a networking using the TFTP protocol.
The RTEMS Tester will run only one test at a time. There is just one physical
board running the test.
An example is::
%tftp %{test_executable} %{tftp_port}

66
user/testing/consoles.rst Normal file
View File

@@ -0,0 +1,66 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
.. _tester-consoles:
Consoles
--------
The RTEMS Tester uses the target's console output to determine the state of a
test. Console interfaces vary depending on the testing mode, the BSP, and the
target hardware.
Consoles for simulator work best if mapped to the simulator's ``stdout``
interface. The RTEMS Tester can capture and process the ``stdout`` data from a
simulator while it is running.
Target hardware console interfaces can vary. The most universal and stable
interface target hardware is a UART interface. There are a number of physical
interfaces for UART data these days. They are:
#. RS232
#. TTL
#. USB
RS232 is still present on a number of targets. The best solution is to use a
RS232 to USB pod and convert the port to USB.
TTL is common on a number of boards where cost is important. A console
interface is typically a development tool and removing the extra devices need
to convert the signal to RS232 or directly to USB is not needed on production
builds of the target. There is a standard header pin out for TTL UART consoles
and you can purchase low cost cables with the header and a built in UART to USB
converter. The cables come is different voltage levels so make sure you check
and use the correct voltage level.
The USB interface on a target is typcially a slave or OTG interface and all you
need to a standard USB cable.
We recommend a low cost and low power device to be a terminal server. A
Raspberry Pi or similar low cost computer running Linux can be set up quickly
and with a powered USB hub and can support a number of USB UART ports. A USB
hub with a high power port is recommended that can suppy the Raspberry Pi.
The open source daemon ``ser2net`` is easy to configure to map the USB UART
ports to the Telnet protocol. There is no need for security because a typical
test environment is part of a lab network that should be partitioned off from
an enginnering or corportate network and not directly connected to the
internet.
A test set up like this lets you place a terminal server close to your target
hardware providing you with the flexibility to select where you run the RTEMS
Tester. It could be your desktop or an expensive fast host machine in a server
rack. None of this equipment needs to directly interface to the target
hardware.
The RTEMS Tester directly supports the telnet protcol as a console and can
interface to the ``ser1net`` server. The telnet console will poll the server
waiting for the remote port to connect. If the terminal server ``ser2net`` does
not have a ``tty`` device it will not listen on the port assigned to that
``tty``. A USB ``tty`` can come and go depending on the power state of the
hardware and the target hardware's design and this can cause timing issues if
the target hardware is power cycled as part of a reset process.

28
user/testing/gdb-jtag.rst Normal file
View File

@@ -0,0 +1,28 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
GDB and JTAG
------------
.. index:: GDB, JTAG, Testing
GDB with JTAG provides a low level way to runs tests on hardware with limited
resources. The RTEMS Tester runs and controls an instance of GDB per test and
GDB connects via the GDB remote protocol to a GDB server that interfaces to the
JTAG port of a target.
.. _fig-tester-gdb-jtag:
.. figure:: ../../images/user/test-gdb-jtag.png
:width: 35%
:alt: RTEMS Tester using GDB and JTAG
:figclass: align-center
RTEMS Tester using GDB and JTAG
The :ref:`fig-tester-gdb-jtag` figure shows the structure of RTEMS Testing
using GDB and JTAG. The executables are built and the ``rtems-test`` command is
run from the top of the build directory. The RTEMS Tester executes the BSP
architecture's GDB and expects the user to provide a ``gdb-script`` to connect
t the JTAG GDB server.

46
user/testing/index.rst Normal file
View File

@@ -0,0 +1,46 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
Testing
*******
RTEMS developers run test executables when adding new features or testing a bug
fix. All tests are run to make sure changes do not introduce regressions. Users
can run the RTEMS tests to be certain the build of the kernel they have is
functioning.
The section describes using and configuring the RTEMS Tester and RTEMS Run
tools, the types of laboratory set ups supported and how to add your BSP to the
framework. The tools command line interfaces are detailed in
:ref:`rtems-tester-command`.
An RTEMS Test is an RTEMS executable where the application code is a
test. Tests in RTEMS print banners to the console to indicate the configuration
of the test and if it has start and finished.
The RTEMS Tools Project provides the RTEMS Tester and RTEMS Run tools. The
RTEMS Tester command is ``rtems-test`` and the RTEMS Run command is
``rtems-run``. These commands manage the complexity of running embedded
executables. The commands provide a consistent command line interface to a
testing framework that supports the various run time and testing scenarios we
encounter such as simulators, GDB and executing directly on target hardware.
The RTEMS kernel code contains an extensive set of tests to exercise and test
the RTEMS kernel. The tests check functionality, provide coverage testing and
make sure the kernel is operating as intended on your target system. The
testsuite has support to make creating a test simple and uniform.
The tests are built by adding ``--enable-tests`` to the RTEMS build
configuration command line. There are over 600 tests and building them does
extend the RTEMS kernel's build time and use more disk space but it worth
building and running them. The RTEMS test executables have the ``.exe`` file
extension.
.. include:: tests.rst
.. include:: configuration.rst
.. include:: consoles.rst
.. include:: simulation.rst
.. include:: gdb-jtag.rst
.. include:: tftp.rst

27
user/testing/simulation.rst Normal file
View File

@@ -0,0 +1,27 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
Simulation
----------
.. index:: Simulation, Testing
Simulation is a important regression and development tool for RTEMS. Developers
use simulation to work on core parts of RTEMS as it provides excellent
debugging supporting. Simulation run via the RTEMS Tester allows a test to run
on each core of your testing host machine lower the time to run all tests.
.. _fig-tester-simulation:
.. figure:: ../../images/user/test-simulation.png
:width: 30%
:alt: RTEMS Tester Simulation
:figclass: align-center
RTEMS Tester Simulation
The :ref:`fig-tester-simulation` figure shows the structure of RTEMS Testing
using simulation. The executables are built and the ``rtems-test`` command is
run from the top of the build directory. The RTEMS Tester executes the
BSP specific simulator for each test capturing the output

210
user/testing/tests.rst Normal file
View File

@@ -0,0 +1,210 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
Test Banners
------------
All test output banners or strings are embedded in each test and the test
outputs them to the BSP's console as it executes. The RTEMS Tester captures the
BSP's console and uses this information to manage the state of the executing
test. The banner strings are:
.. _test-banner-begin:
.. index:: test begin, TEST BEGIN
``*** BEGIN TEST <name> ***``
The test has loaded, RTEMS has initialized and the test specific code is
about to start executing. The ``<name>`` field is the name of the test. The
test name is internal to the test and may not match the name of the
executable. The test name is informative and not used by the RTEMS Tester.
.. _test-banner-end:
.. index:: test end, TEST END
``*** END TEST <name> ***``
The test has finished without error and has passed. The ``<name>`` field is
the name of the test. See the :ref:`Test Begin Banner <test-banner-begin>`
for details about the name.
.. index:: test banner version, TEST VERSION
``*** TEST VERSION: <version>``
The test prints the RTEMS version return by the RTEMS Version API as
``<version>``. All tests must match the first test's version or the Wrong
Version error count is incremented.
.. _test-banner-state:
.. index:: test state, TEST STATE
``*** TEST STATE: <state>``
The test is tagged in the RTEMS sources with a special ``<state>`` for this
BSP. See :ref:`Test States <test-states>` for the list of possible
states. The state banner lets the RTEMS Tester categorize and manage the
test. For example a user input test typically needing user interaction may
never complete producing an *invalid* test result. A user input test is
terminated to avoid extended delays in a long test run.
.. _test-banner-build:
.. index:: test build, TEST BUILD
``*** TEST BUILD: <build>``
The test prints the RTEMS build as a space separated series of labels as
``<build>``. The build labels are created from the configuration settings in
the Super Score header file ``rtems/score/cputops.h``. All tests must match
the first test's build or the Wrong Build error count is incremented.
.. _test-banner-tools:
.. index:: test tools, TEST TOOLS
``*** TEST TOOLS: <version>``
The test prints the RTEMS tools version returned the GGC internal macro
``_VERSION_`` as ``<version>``. All tests must match the first test's tools
version string or the Wrong Tools error count is incremented.
.. _test-states:
.. index:: Test states
Test States
-----------
The tests states are:
.. index:: test state passed
``passed``
The test start and end banners have been sent to the console.
.. index:: test state failure
``failure``
The test start banner has been sent to the console and no end banner has been
seen when a target restart is detected.
.. index:: test state expected-fail
``excepted-fail``
The test is tagged as ``expected-fail`` in the RTEMS sources for this BSP and
outputs the banner ``*** TEST STATE: EXPECTED_FAIL``. The test is known not
to pass on this BSP. The RTEMS Tester will let the test run as far as it
can and if the test passes it is recorded as a pass in the test results
otherwise it is recorded as *expected-fail*.
.. index:: test state indeterminate
``indeterminate``
The test is tagged as ``indeterminate`` in the RTEMS sources for this BSP and
outputs the banner ``*** TEST STATE: INDETERMINATE``. The test may or may not
pass so the result is not able to be determined. The RTEMS Tester will let
the test run as far as it can and record the result as indeterminate.
.. index:: test state user-input
``user-input``
The test is tagged as ``user-input`` in the RTEMS sources and outputs the
banner ``*** TEST STATE: USER_INPUT``. The RTEMS Tester will reset the target
if the target's configuration provides a target reset command.
.. index:: test state benchmark
``benchmark``
The test is tagged as ``benchmark`` in the RTEMS sources and outputs the
banner ``*** TEST STATE: BENCHMARK``. Benchmarks can take a while to run and
performance is not regression tested in RTEMS. The RTEMS Tester will reset
the target if the target's configuration provides a target reset command.
.. index:: test state timeout
``timeout``
The test start banner has been sent to the console and no end banner is seen
within the *timeout* period and the target has not restart. A default
*timeout* can be set in a target configuration, a user configuration or
provide on the RTEMS Tester's command line using the ``--timeout`` option.
.. index:: test state invalid
``invalid``
The test did not output a start banner and the RTEMS Tester has detected the
target has restarted. This means the executable did not load correctly, the
RTEMS kernel did not initialize or the RTEMS kernel configuration failed for
this BSP.
Expected Test States
^^^^^^^^^^^^^^^^^^^^
A test's expected state is set in the RTEMS kernel's testsuite. The default for
a tested is to ``pass``. If a test is known to fail it can have it's state set
to ``expected-fail``. Setting tests that are known to fail to ``expected-fail``
lets everyone know a failure is not to be countered and consider a regression.
Expected test states are list in test configuration files that end with the
file extension ``.tcfg``. The testsuite supports global test configurations in
the ``testsuite/testdata`` directory. Global test states are applied to all
BSPs. BSPs can provide a test configuration that applies to just that BSP.
The test configuration file format is::
state: test test test
where ``test test test`` is a list of tests the state applies too. The ``state`` is one
of:
``include``
The test list is the name of a test configuration file to include
``exclude``
The tests listed are not build. This can happen if a BSP cannot support a
test. For example it does not have enough memory.
``expected-fail``
The tests listed are set to expected fail. The test will fail on the BSP
being built.
``user-input``
The tests listed require user input to run and are not supported by automatic
testers.
``indeterminate``
The tests listed may pass or may not, the result is not reliable.
``benchmark``
The tests listed are benchmarks. Benchmarks are flagged and not left to
run to completion because they may take too long.
Test Builds
-----------
The test reports the build of RTEMS being tested. The build are:
.. index:: build default
``default``
The build is the default. No RTEMS configure options have been used.
.. index:: build posix
``posix``
The build includes the POSIX API. The RTEMS configure option
``--enable-posix`` has been used. The ``cpuopts.h`` define ``RTEMS_POSIX``
has defined and it true.
.. index:: build smp
``smp``
The build is an SMP kernel. The RTEMS configure option ``--enable-smp`` has
been used. The ``cpuopts.h`` define ``RTEMS_SMP`` has defined and it true.
.. index:: build mp
``mp``
The build is an MP kernel. The RTEMS configure option
``--enable-multiprocessing`` has been used. The ``cpuopts.h`` define
``RTEMS_MULTIPROCESSING`` has defined and it true.
.. index:: build paravirt
``paravirt``
The build is a paravirtualization kernel. The ``cpuopts.h`` define
``RTEMS_PARAVIRT`` has defined and it true.
.. index:: build debug
``debug``
The build includes kernel debugging support. The RTEMS configure option
``--enable-debug`` has been used. The ``cpuopts.h`` define ``RTEMS_DEBUG``
has defined and it true.
.. index:: build profiling
``profiling``
The build include profiling support. The RTEMS configure option
``--enable-profiling`` has been used. The ``cpuopts.h`` define
``RTEMS_PROFILING`` has defined and it true.

257
user/testing/tftp.rst Normal file
View File

@@ -0,0 +1,257 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
TFTP and U-Boot
---------------
.. index:: TFTP, U-Boot, Testing
TFTP and U-Boot provides a simple way to test RTEMS on a network capable
target. The RTEMS Tester starts a TFTP server for each test and the target's
boot monitor, in this case U-Boot request a file, any file, which the TFTP
server supplies. U-Boot loads the executable and boots it using a standard
U-Boot script.
.. _fig-tester-tftp-u-boot:
.. figure:: ../../images/user/test-tftp.png
:width: 35%
:alt: RTEMS Tester using TFTP and U-Boot
:figclass: align-center
RTEMS Tester using TFTP and U-Boot.
The :ref:`fig-tester-tftp-u-boot` figure shows the structure and control flow
of the RTEMS Tester using TFTP and U-boot. The executables are built and the
``rtems-test`` command is run from the top of the build directory.
This test mode can only support a single test job running at once. You cannot
add more test target hardware and run the tests in parallel.
Target Hardware
^^^^^^^^^^^^^^^
The RTEMS Tester TFTP and U-Boot method of testing requires:
#. A target with network interface.
#. U-Boot, iPXE or similar boot loader with network driver support for your
target hardware and support for the TFTP protocol.
#. Network power of IO switch.
#. Network DHCP server.
#. Console interface cable that matches your target's console UART interface.
#. Telnet terminal server. See :ref:`tester-consoles`.
The network power or IO switch is a device that can control power or an IO pin
over a network connection using a script-able protocol such as Telnet or
curl. This device can be used with the target control commands.
U-Boot Set Up
~~~~~~~~~~~~~
Obtain a working image of the U-Boot boot loader for your target. We suggest
you follow the instructions for you target.
Configure U-Boot to network boot using the TFTP protocol. This is U-Boot script
for a Zedboard::
loadaddr=0x02000000
uenvcmd=echo Booting RTEMS Zed from net; set autoload no; dhcp; set serverip 10.10.5.2; tftpboot zed/rtems.img; bootm; reset;
The load address variable ``loadaddr`` is specific to the Zedboard and can be
found in the various examples scripts on the internet. The script then sets
U-Boot environment variable ``autoload`` to ``no`` causing DHCP to only request
a DHCP lease from the DHCP server. The script sets the ``serverip`` to the host
that will be running the RTEMS Tester then issues a TFTP request. The file name
can be anything because the RTEMS Tester ignores it sending the executable
image under test. Finally the script boots the download executable and if that
fails the catch all ``reset`` resets the board and starts the boot process
over.
Test the target boots and U-Boot runs and obtains a valid DHCP lease. Manually
connect the console's telnet port.
BSP Configuration
^^^^^^^^^^^^^^^^^
The BSP's configuration file must contain the standard fields:
- ``bsp``
- ``arch``
- ``jobs`` - Must be set to ``1``.
- ``tester`` - Set to ``%{_rtscripts}/tftp.cfg``
For example the Zedboard's configuration is::
[xilinx_zynq_zedboard]
bsp = xilinx_zynq_zedboard
arch = arm
jobs = 1
tester = %{_rtscripts}/tftp.cfg
The TFTP configuration supports the following field's:
``bsp_tty_dev``
The target's tty console. For telnet this is a host and port pair written in
the standard networking format, for example ``serserver:12345``.
``test_restarts``
The number of restarts before the test is considered ``invalid``.
``target_reset_regex``
The target reset regular expression. This is a `Python regular expression
<https://docs.python.org/2/library/re.html#regular-expression-syntax>`_ used
to filter the console input. If a match is made something has happened during
the boot process that requires a reset. The ``target_reset_command``
is issued to perform the reset. This field is typically looks for boot loader
error messages that indicate the boot process as failed.
``target_start_regex``
The target start regular expression. This also a Python regular expression to
filter the console input to detect if a target has reset. If a board crashes
running a test or at any point in time and reset this filter detects this as
happened and end the test with a suitable result.
``target_on_command``
The target on command is a host shell command that is called before the first
test. This command powers on a target. Targets should be left powered off
when not running tests or the target may request TFTP downloads that are for
another target interfering with those test results. We recommend you
implement this command as a target off command, a pause, then a target on
command.
``target_off_command``
The target off command is a host shell command that is called after the last
test powering off the target.
``target_reset_command``
The target reset command is a host shell command that is called when the
target needs to be reset. This command can power cycle the target or toggle a
reset signal connected to the target. If you are power cycling a target make
sure you have a suitable pause to let the target completely power down.
``target_pretest_command``
The target pretest command is a host shell comment that is called before the
test is run
The commands in the listed fields can include parameters that are
substituted. The parameters are:
``@ARCH@``
The BSP architecture
``@BSP@``
The BSP's name
``@EXE@``
The executable name.
``@FEXE@``
The
. The
``@ARCH`` is the
substituted
Some of these field are normally provided by a user's configuration. To do this
use::
requires = bsp_tty_dev, target_on_command, target_off_command, target_reset_command
The ``requires`` value requires the user provide these settings in their
configuration file.
The Zedboard's configuration file is::
[xilinx_zynq_zedboard]
bsp = xilinx_zynq_zedboard
arch = arm
jobs = 1
tester = %{_rtscripts}/tftp.cfg
test_restarts = 3
target_reset_regex = ^No ethernet found.*|^BOOTP broadcast 6.*|^.+complete\.+ TIMEOUT.*
target_start_regex = ^U-Boot SPL .*
requires = target_on_command, target_off_command, target_reset_command, bsp_tty_dev
The ``target_start_regex`` searches for U-Boot's first console message. This
indicate the board can restarted.
The ``target_reset_regex`` checks if no ethernet interface is found. This can
happen if U-Boot cannot detect the PHY device. It also checks if too many DHCP
requests happen and finally a check is made for any timeouts reported by
U-Boot.
An example of a user configuration for the Zedboard is::
[xilinx_zynq_zedboard]
bsp_tty_dev = selserver:12345
target_pretest_command = zynq-mkimg @EXE@
target_exe_filter = /\.exe/.exe.img/
target_on_command = power-ctl toggle-on 1 4
target_off_command = power-ctl off 1
target_reset_command = power-ctl toggle-on 1 3
TFTP Sequences
^^^^^^^^^^^^^^
Running a large number of tests on real hardware exposes a range of issues and
RTEMS Tester is designed to be tolerant of failures in booting or loading that
can happen, for example a hardware design. These sequence diagrams document
some of the sequences that can occur when errors happen.
The simplest sequence is running a test. The target is powered on, the test is
loaded and executed and a pass or fail is determined:
.. _fig-tester-tftp-seq-1:
.. figure:: ../../images/user/test-tftp-seq-1.png
:width: 90%
:alt: Test Pass and Fail Sequence
:figclass: align-center
Test Pass and Fail Sequences
The target start filter triggers if a start condition is detected. This can
happen if the board crashes or resets with no output. If this happens
repeatedly the test result is invalid:
.. _fig-tester-tftp-seq-2:
.. figure:: ../../images/user/test-tftp-seq-2.png
:width: 80%
:alt: Target Start Filter Trigger
:figclass: align-center
Target Start Filter Trigger
The reset filter triggers if an error condition is found such as the bootloader
not being able to load the test executable. If the filter triggers the
``target_reset_command`` is run:
.. _fig-tester-tftp-seq-3:
.. figure:: ../../images/user/test-tftp-seq-3.png
:width: 50%
:alt: Target Reset Filter Trigger
:figclass: align-center
Target Reset Filter Trigger
If the RTEMS Tester does not detect a test has started it can restart the test
by resetting the target. The reset command can toggle an IO pin connected to
reset, request a JTAG pod issue a reset or turn the power off and on:
.. _fig-tester-tftp-seq-4:
.. figure:: ../../images/user/test-tftp-seq-4.png
:width: 60%
:alt: Target Timeout
:figclass: align-center
Target Timeout