Adding Trace Documentation

- Updates #3454
- This commit adds Tracing Framework Chapter in the RTEMS User Manual
- It comprises of subchapters on RTEMS Trace Linker, Capture Engine, Trace
  generation techniques explaining trace generation using Trace Buffering and
  Printk generators and sample demonstrations.
This commit is contained in:
Vidushi Vashishth 2018-06-14 19:47:58 +05:30 committed by Chris Johns
parent 2b7c9e7fa7
commit 8f4f80d437
6 changed files with 996 additions and 0 deletions

View File

@ -52,6 +52,8 @@ to the Community Project hosted at http://www.rtems.org/.
tools/index
tracing/index
support/index
glossary/index

View File

@ -0,0 +1,169 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2018 Vidushi Vashishth <vidushivashishth96@gmail.com>
.. comment: All rights reserved.
.. _capturengine:
Capture Engine
**************
Capture Engine is a trace tool built inside the RTEMS operating system. Capture
Engine is designed to cause the lowest load on the system when operating. Hence
it does not effect RTEMS when operating or when disabled. It binds to RTEMS at
runtime and does not require RTEMS or your application to be rebuilt in order
to use it.
The Capture Engine's sample testcase for the `sparc/erc32` is available in
build directory created when building RTEMS in the path
file: `sparc-rtems5/c/erc32/testsuites/samples`. In order to access the capture
testcase perform the following set of operations inside the RTEMS build
directory.
.. code-block:: shell
$ cd /sparc-rtems5/c/erc32/testsuites/samples
$ sparc-rtems5-run ./capture.exe
*** BEGIN OF TEST CAPTURE ENGINE ***
*** TEST VERSION: 5.0.0.de9b7d712bf5da6593386fd4fbca0d5f8b8431d8
*** TEST STATE: USER_INPUT
*** TEST BUILD: RTEMS_NETWORKING RTEMS_POSIX_API
*** TEST TOOLS: 7.3.0 20180125 (RTEMS 5, RSB a3a6c34c150a357e57769a26a460c475e188438f, Newlib 3.0.0)
Press any key to start capture engine (20s remaining)
Press any key to start capture engine (19s remaining)
Press any key to start capture engine (18s remaining)
Monitor ready, press enter to login.
1-rtems $
Capture Engine comes with a set of commands to perform various actions.
Capture Engine Commands
-----------------------
1) ``copen <buffer-size>``: Used to initialize the Capture Engine with the
trace buffer size in bytes. By default the Capture Engine is not initialized
and not running.
2) ``cwceil <priority-value>``: Capture Engine filter used to put an upper
limit on the event priority to be captured.
3) ``cwfloor <priority-value>``: Capture Engine filter used to put a lower
limit on the event priority to be captured.
4) ``cwglob <on/off>``: Enable or disable the global watch.
5) ``cenable``: Enables the Capture Engine. Capture Engine is by default
disabled after being opened.
6) ``cdisable``: Disables the Capture Engine.
7) ``ctlist``: Lists the watch and trigger configurations.
8) ``ctrace``: Dumps the recorded traces. By default this command displays 24
trace records. Repeated use of this command will display all the recorded
traces.
9) ``cwadd <task-name>``: Add watch on a particular task.
10) ``cwtctl <task-name> <on/off>``: Enable or disable watch on a particular
task.
11) ``ctset``: Used to set a trigger. The general form of the command is:
``ctset [-?] type [to name/id] [from] [from name/id]``
`type` in the above command refers to the type of trigger needed. The types of
triggers that currently exist are:
- switch : a context switch from one task to another task
- create : the executing task creates a task
- start : the executing task starts a task
- restart : the executing task restarts a task
- delete : the executing task deletes a task
- begin : a task is beginning
- exitted : a task is exitting
Example
-------
The following is a sample run of the capture testsuite. The `test1` command on
the Capture Engine Command Line Interface (CLI) makes the `RMON` task invoke a
call to the `capture_test_1()` command. This function (in the `test1.c` source
code) creates and starts three tasks : `CT1a`, `CT1b` and `CT1c`. These tasks
are passed the object id of a semaphore as a task argument. This run through
traces the context switches between these tasks. ``cwceil`` and ``cwfloor`` are
set to a narrow range of task priorities to avoid creating noise from a large
number of context switches between tasks we are not interested in.
.. code:: shell
*** BEGIN OF TEST CAPTURE ENGINE ***
*** TEST VERSION: 5.0.0.de9b7d712bf5da6593386fd4fbca0d5f8b8431d8
*** TEST STATE: USER_INPUT
*** TEST BUILD: RTEMS_NETWORKING RTEMS_POSIX_API
*** TEST TOOLS: 7.3.0 20180125 (RTEMS 5, RSB a3a6c34c150a357e57769a26a460c475e188438f, Newlib 3.0.0)
Press any key to start capture engine (20s remaining)
Press any key to start capture engine (19s remaining)
Press any key to start capture engine (18s remaining)
Press any key to start capture engine (17s remaining)
Monitor ready, press enter to login.
1-rtems $ copen 50000
capture engine opened.
1-rtems $ cwceil 100
watch ceiling is 100.
1-rtems $ cwfloor 102
watch floor is 102.
1-rtems $ cwglob on
global watch enabled.
1-rtems $ ctset RMON
trigger set.
1-rtems $ cenable
capture engine enabled.
1-rtems $ test1
1-rtems $ cdisable
capture engine disabled.
1-rtems $ ctrace
0 0:18:17.462314124 0a010003 CT1a 102 102 102 4096 TASK_RECORD
0 0:18:17.462398963 0 0a010003 CT1a 102 102 CREATED
0 0:18:17.462647987 249024 0a010003 CT1a 102 102 STARTED
0 0:18:17.462904334 256347 0a010003 CT1a 102 102 SWITCHED_IN
0 0:18:17.463069129 164795 0a010003 CT1a 102 102 BEGIN
0 0:18:17.463335853 266724 0a010003 CT1a 102 102 SWITCHED_OUT
0 0:18:18.461348547 0a010004 CT1b 101 101 101 4096 TASK_RECORD
0 0:18:18.461433997 998098144 0a010004 CT1b 101 101 CREATED
0 0:18:18.461683631 249634 0a010004 CT1b 101 101 STARTED
0 0:18:18.461934485 250854 0a010004 CT1b 101 101 SWITCHED_IN
0 0:18:18.462099891 165406 0a010004 CT1b 101 101 BEGIN
0 0:18:19.460935339 998835448 0a010004 CT1b 101 101 SWITCHED_OUT
0 0:18:19.461431555 0a010005 CT1c 100 100 100 4096 TASK_RECORD
0 0:18:19.461516394 581055 0a010005 CT1c 100 100 CREATED
0 0:18:19.461765418 249024 0a010005 CT1c 100 100 STARTED
0 0:18:19.462019324 253906 0a010005 CT1c 100 100 SWITCHED_IN
0 0:18:19.462184119 164795 0a010005 CT1c 100 100 BEGIN
0 0:18:19.462475257 291138 0a010005 CT1c 100 100 SWITCHED_OUT
0 0:18:19.462551551 76294 0a010004 CT1b 101 101 SWITCHED_IN
0 0:18:19.960935645 498384094 0a010004 CT1b 101 101 SWITCHED_OUT
0 0:18:19.961012549 76904 0a010003 CT1a 102 100 SWITCHED_IN
0 0:18:19.961341528 328979 0a010003 CT1a 102 102 SWITCHED_OUT
1-rtems $ ctrace
0 0:18:19.961418433 0 0a010005 CT1c 100 100 SWITCHED_IN
0 0:18:19.961672339 253906 0a010005 CT1c 100 100 SWITCHED_OUT
0 0:18:19.961749854 77515 0a010004 CT1b 101 101 SWITCHED_IN
0 0:18:20.460967077 499217223 0a010004 CT1b 101 101 SWITCHED_OUT
0 0:18:20.461219763 252686 0a010005 CT1c 100 100 SWITCHED_IN
0 0:18:20.461424231 204468 0a010005 CT1c 100 100 TERMINATED
0 0:18:20.461747107 322876 0a010005 CT1c 100 100 SWITCHED_OUT
0 0:18:20.461824011 76904 0a010004 CT1b 101 101 SWITCHED_IN
0 0:18:20.462015052 191041 0a010004 CT1b 101 101 TERMINATED
0 0:18:20.462336707 321655 0a010004 CT1b 101 101 SWITCHED_OUT
0 0:18:20.462414222 77515 0a010003 CT1a 102 102 SWITCHED_IN
0 0:18:20.462608924 194702 0a010003 CT1a 102 102 TERMINATED
0 0:18:20.462933021 324097 0a010003 CT1a 102 102 SWITCHED_OUT
1-rtems $ ctrace
1-rtems $

199
user/tracing/examples.rst Normal file
View File

@ -0,0 +1,199 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2018 Vidushi Vashishth <vidushivashishth96@gmail.com>
.. comment: All rights reserved.
.. _examples:
Tracing Examples
****************
The following example executes RTEMS trace using trace buffering for the
`fileio` sample testcase.
Features
--------
Tracing using trace buffering consists of the following sets of features:
- Individual entry and exit records.
- Task details such as CPU, current priority, real priority, task state and
interrupt state.
- Nano-second timestamp.
- Interrupt safe buffer management.
- Function argument capture.
- Return value capture.
- Shell command support to report to the console, save a buffer, assess status
of tracing, or view buffers between specified index ranges.
Prerequisites
-------------
1. Setup RTEMS for the `sparc/erc32` architecture-bsp pair to run the
following example.
2. Download the fileio `configuration file <https://devel.rtems.org/attachment
/wiki/Developer/Tracing/Trace_Buffering/fileio-trace.ini>`_ and store it on
the top of the installed BSP's directory.
3. Change the value of the keys: `rtems-path` and `prefix` according to your
rtems installation. The `rtems-path` is the path to the bsp installation
and `prefix` is the path to the tools used to build rtems. Also set the
value of the `rtems-bsp` key to `sparc/erc32`.
Demonstration
-------------
Inside the RTEMS build directory (the directory where the fileio configuration
has been stored) run the following commands to generate traces:
BSP is configured with the following command -
.. code:: shell
../rtems/configure --target=sparc-rtems5 --prefix=/development/rtems/5 \
--enable-networking --enable-tests --enable-rtemsbsp=erc32 --enable-cxx
The next two commands are used to link the fileio executable.The `-B` option
signifies the use of the complete path to the required directory or file. Write
the full path instead of the path file: `sparc-rtems5/erc32/lib/` in the
following commands according to your installation. Also confirm the path of the
fileios executable and object files in the last line of the command according
to your installation.
.. code:: shell
sparc-rtems5-gcc -Bsparc-rtems5/erc32/lib/ \
-specs bsp_specs -qrtems -mcpu=cypress -O2 -g -ffunction-sections \
-fdata-sections -Wall -Wmissing-prototypes -Wimplicit-function-declaration \
-Wstrict-prototypes -Wnested-externs -Wl,--gc-sections -mcpu=cypress \
-o sparc-rtems5/c/erc32/testsuites/samples/fileio.exe sparc-rtems5/c/erc32/\
testsuites/samples/fileio/fileio-init.o
This is the trace linker command to generate and compile the wrapper c file for
the application. The link command follows the escape sequence "--". "-C" option
denotes the name of the user configuration file and "-W" specifies the name of
the wrapper c file.
.. code:: shell
rtems-tld -C fileio-trace.ini -W fileio-wrapper -- -Bsparc-rtems5/erc32/lib/ \
-specs bsp_specs -qrtems -mcpu=cypress -O2 -g -ffunction-sections \
-fdata-sections -Wall -Wmissing-prototypes -Wimplicit-function-declaration \
-Wstrict-prototypes -Wnested-externs -Wl,--gc-sections -mcpu=cypress \
-o sparc-rtems5/c/erc32/testsuites/samples/fileio.exe sparc-rtems5/c/erc32/\
testsuites/samples/fileio/fileio-init.o
The following command is used to run the application. Hit enter key quickly and
type "s" and "root" and "pwd" to run the rtems shell. Use the `rtrace status`,
`rtrace trace` and `rtrace save` commands to know the status of the tracing,
display the contents of the trace buffer and save the buffer to disk in the form
of binary files. Use `rtrace -l` to list the availalble options for commands
with `rtrace`.
.. code:: shell
sparc-rtems5-run sparc-rtems5/c/erc32/testsuites/samples/fileio.exe
The output from the above commands will be as follows:
.. code:: shell
*** BEGIN OF TEST FILE I/O ***
*** TEST VERSION: 5.0.0.de9b7d712bf5da6593386fd4fbca0d5f8b8431d8
*** TEST STATE: USER_INPUT
*** TEST BUILD: RTEMS_NETWORKING RTEMS_POSIX_API
*** TEST TOOLS: 7.3.0 20180125 (RTEMS 5, RSB a3a6c34c150a357e57769a26a460c475e188438f, Newlib 3.0.0)
Press any key to start file I/O sample (20s remaining)
Press any key to start file I/O sample (19s remaining)
Press any key to start file I/O sample (18s remaining)
Press any key to start file I/O sample (17s remaining)
Press any key to start file I/O sample (16s remaining)
Press any key to start file I/O sample (15s remaining)
Press any key to start file I/O sample (14s remaining)
=========================
RTEMS FILE I/O Test Menu
=========================
p -> part_table_initialize
f -> mount all disks in fs_table
l -> list file
r -> read file
w -> write file
s -> start shell
Enter your selection ==>s
Creating /etc/passwd and group with four useable accounts:
root/pwd
test/pwd
rtems/NO PASSWORD
chroot/NO PASSWORD
Only the root user has access to all available commands.
=========================
starting shell
=========================
Welcome to rtems-5.0.0 (SPARC/w/FPU/erc32)
COPYRIGHT (c) 1989-2008.
On-Line Applications Research Corporation (OAR).
Login into RTEMS
/dev/foobar login: root
Password:
RTEMS Shell on /dev/foobar. Use 'help' to list commands.
SHLL [/] # rtrace status
RTEMS Trace Bufferring: status
Running: yes
Triggered: yes
Level: 0%
Traces: 25
SHLL [/] # rtrace stop
RTEMS Trace Bufferring: stop
SHLL [/] # rtrace trace
RTEMS Trace Bufferring: trace
Trace buffer: 0x20921d8
Words traced: 1487
Traces: 25
0:00:40.983197010 2081910 0a010002 [ 2/ 2] > malloc((size_t) 00000130)
0:00:40.983333119 136109 0a010002 [ 2/ 2] < malloc => (void*) 0x219bb88
0:00:40.983471669 138550 0a010002 [ 2/ 2] > malloc((size_t) 00000006)
0:00:40.983606557 134888 0a010002 [ 2/ 2] < malloc => (void*) 0x219bcc0
0:00:40.983684682 78125 0a010002 [ 2/ 2] > malloc((size_t) 00000007)
0:00:40.983819569 134887 0a010002 [ 2/ 2] < malloc => (void*) 0x219bcd0
0:00:40.983909901 90332 0a010002 [ 2/ 2] > malloc((size_t) 000003fc)
0:00:40.984046620 136719 0a010002 [ 2/ 2] < malloc => (void*) 0x219bce0
0:00:40.986624137 2577517 0a010003 [200/200] > malloc((size_t) 00000080)
0:00:40.986767569 143432 0a010003 [200/200] < malloc => (void*) 0x219bce0
0:00:40.987531119 763550 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 0000005d)
0:00:40.987603751 72632 0a010003 [200/200] > malloc((size_t) 0000005d)
0:00:40.987744743 140992 0a010003 [200/200] < malloc => (void*) 0x219bce0
0:00:40.987824699 79956 0a010003 [200/200] < calloc => (void*) 0x219bce0
0:00:40.988302604 477905 0a010003 [200/200] > malloc((size_t) 00000080)
0:00:40.988446647 144043 0a010003 [200/200] < malloc => (void*) 0x219bd48
0:00:40.988667595 220948 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000080)
0:00:40.988740837 73242 0a010003 [200/200] > malloc((size_t) 00000080)
0:00:40.988884880 144043 0a010003 [200/200] < malloc => (void*) 0x219bdd0
0:00:40.988964836 79956 0a010003 [200/200] < calloc => (void*) 0x219bdd0
0:00:40.989042961 78125 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000080)
0:00:40.989110100 67139 0a010003 [200/200] > malloc((size_t) 00000080)
0:00:40.989254143 144043 0a010003 [200/200] < malloc => (void*) 0x219be58
0:00:40.989334099 79956 0a010003 [200/200] < calloc => (void*) 0x219be58
0:00:40.990118401 784302 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000061)
0:00:40.990176995 58594 0a010003 [200/200] > malloc((size_t) 00000061)
0:00:40.990309441 132446 0a010003 [200/200] < malloc => (void*) 0x219bd48
0:00:40.990384515 75074 0a010003 [200/200] < calloc => (void*) 0x219bd48
0:00:40.990870355 485840 0a010003 [200/200] > malloc((size_t) 00000080)
0:00:40.991011346 140991 0a010003 [200/200] < malloc => (void*) 0x219bee0
0:00:40.991227411 216065 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000080)
0:00:40.991296380 68969 0a010003 [200/200] > malloc((size_t) 00000080)
0:00:40.991438593 142213 0a010003 [200/200] < malloc => (void*) 0x219bf68
0:00:40.991514276 75683 0a010003 [200/200] < calloc => (void*) 0x219bf68
0:00:40.991589349 75073 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000080)
0:00:40.991653437 64088 0a010003 [200/200] > malloc((size_t) 00000080)
0:00:40.991794428 140991 0a010003 [200/200] < malloc => (void*) 0x219bff0
0:00:40.991871332 76904 0a010003 [200/200] < calloc => (void*) 0x219bff0
0:00:40.992283320 411988 0a010003 [200/200] > malloc((size_t) 00000008)
SHLL [/] # rtrace save fileio-trace.bin
RTEMS Trace Bufferring: trace
Trace File: fileio-trace.bin
Trace buffer: 0x20921d8
Words traced: 1487
Traces: 25
SHLL [/] #

29
user/tracing/index.rst Normal file
View File

@ -0,0 +1,29 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2016 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
.. _tracing-framework:
RTEMS Tracing Framework
***********************
.. index:: Tracing Framework
RTEMS Tracing Framework is an on-target software based system which helps track
the ongoings inside the operation of applications, 3rd party packages, and the
kernel in real time.
Software based tracing is a complex process which requires components on both
the target and the host to work together. However its portability across all
architectures and board support packages makes it a useful asset. A key
requirement in RTEMS trace process is to take existing code in compiled format
(ELF) and instrument it in order to log various events and records in real time.
However instrumenting of the code for tracing should happen without rebuilding
the code from the source and without annotating the source with trace code.
.. toctree::
introduction
examples
captureengine
tracelinker

View File

@ -0,0 +1,103 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2016 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
.. _introduction:
Introduction to Tracing
***********************
Tracing is an important function which has several applications including
identification of complex threading, detection of deadlocks, tracing
functions along with their argument values, and return values through
progression of several function calls and audit the performance of an
application according to required specifications.
RTEMS tracing framework is under development and welcomes contribution by users.
RTEMS has the following trace components:
- RTEMS :ref:`tracelinker`
- RTEMS :ref:`capturengine`
- Common Trace Format Integration
RTEMS trace framework can currently function using the following methods. Both
of the methods make use of the :ref:`tracelinker` :
.. _tracebuffering:
RTEMS Trace Using Trace Buffering
=================================
This scheme of tracing goes through the flow of events described in a
subsequent flowchart:
Step 1: The user creates an application and user configuration file. The
configuration file specifies the use of the trace buffer generator and other
standard initializations. The user then configures their BSP and invokes the
trace linker using a command to link the application executable. The trace
linker uses the application files in compiled format (ELF) and the libraries
used to build the application for performing this link.
Step 2: The RTEMS Trace Linker reads the users configuration file and that
results in it reading the standard Trace Buffering Configuration files
installed with the RTEMS Trace Linker. The trace linker uses the target
compiler and linker to create the trace enabled application executable. It
wraps the functions defined in the users configuration with code that captures
trace records into the statically allocated buffer. The trace wrapper code is
compiled with the target compiler and the resulting ELF object file is added to
the standard link command line used to link the application and the application
is re-linked using the wrapping option of the GNU linker.
Step 3: The trace linker creates an executable which is capable of running on
the target hardware or simulator.
Step 4: RTEMS shell provides the “rtrace” command to display and save trace
buffers.
.. comment: taken from https://devel.rtems.org/wiki/Developer/Tracing
.. figure:: ../../images/user/rtems-trace-buffering.png
:align: center
:width: 75%
.. _printk:
RTEMS Trace Using Printk
========================
This scheme of tracing goes through the flow of events described in a subsequent
flowchart:
Step 1: The user creates an RTEMS application in the normal manner as well as a
Trace Linker configuration file. The configuration file specifies using the
Printk trace mode and the functions to trace. The user invokes the Trace Linker
with the configuration and the normal link command line used to the link the
application executable. The application ELF object files and libraries,
including the RTEMS libraries are standard and do not need to be built
specially.
Step 2: The RTEMS Trace Linker reads the user's configuration file and that
results in it reading the standard Printk Trace Configuration files installed
with the RTEMS Trace Linker. The trace linker uses the target compiler and
linker to create the trace enabled application executable. It wraps the
functions defined in the user's configuration with code that prints the entry
with arguments and exit and return value if any. The trace wrapper code is
compiled with the target compiler and the resulting ELF object file is added to
the standard link command line used to link the application and the application
is relinked using the wrapping option of the GNU linker.
Step 3: The trace linker creates and RTEMS ELF executable that can be run on the
target hardware or simulator.
Step 4: The application is run in the hardware directly or using a debugger. The
printk() output appears on the target console and the user can save that to a
file.
.. comment: taken from https://devel.rtems.org/wiki/Developer/Tracing
.. figure:: ../../images/user/rtems-trace-printk.png
:align: center
:width: 75%
The :ref:`examples` section describes generation of traces using Trace Buffering
technique for the `fileio` testsuite available with RTEMS installation.

View File

@ -0,0 +1,494 @@
.. comment SPDX-License-Identifier: CC-BY-SA-4.0
.. comment: Copyright (c) 2016 Chris Johns <chrisj@rtems.org>
.. comment: All rights reserved.
.. _tracelinker:
Trace Linker
************
RTEMS trace linker is a post link tool central to the RTEMS trace framework. It
is installed as a part of the RTEMS Tool Project. The RTEMS Trace Linker is a
post link tool that performs a re-link of your application to produce a trace
executable. A trace executable has been instrumented by the RTEMS Trace Linker
with additional code that implements software tracing. A key requirement of the
trace process in RTEMS is to take existing code in a compiled format (ELF) and
instrument it without rebuilding that code from source and without annotating
that source with trace code.
Command Line
============
A typical command to invoke the trace linker consists of two parts separated by
``--``. The first part controls the trace linker and provides the various
options it needs and the second part is a standard linker command line you would
use to link an RTEMS application. The current command line for trace linker
consists of:
.. code-block:: shell
$ rtems-tld -h
rtems-trace-ld [options] objects
Options and arguments:
-h : help (also --help)
-V : print linker version number and exit (also --version)
-v : verbose (trace import parts), can supply multiple times
to increase verbosity (also --verbose)
-w : generate warnings (also --warn)
-k : keep temporary files (also --keep)
-c compiler : target compiler is not standard (also --compiler)
-l linker : target linker is not standard (also --linker)
-E prefix : the RTEMS tool prefix (also --exec-prefix)
-f cflags : C compiler flags (also --cflags)
-r path : RTEMS path (also --rtems)
-B bsp : RTEMS arch/bsp (also --rtems-bsp)
-W wrapper : wrapper file name without ext (also --wrapper)
-C ini : user configuration INI file (also --config)
-P path : user configuration INI file search path (also --path)
The trace linker generates code that needs to be compiled and linked to the
application executable so it needs to know the target compiler and `CFLAGS`.
There are a couple of ways to do this. The simplest is to provide the path to
RTEMS using the `-r` option and the architecture and BSP name in the standard
RTEMS format of arch/bsp. The trace linker will extract the compiler and flags
used to build RTEMS and will use them. If you require specific options you can
use the `-f`, `-c`, `-l`, and `-E` options to provide them. If the functions you
are tracing use types from your code then add the include path to the `CFLAGS`.
The trace linker requires you to provide a user configuration file using the
`-C` or ``--config`` option. This is an INI format file detailed in the
Configuration section. You can also provide an INI file search path using the
`-P` option.
If you are working with new configuration files and you want to view the files
the trace linker generates, add the `-k` option to keep the temporary files, and
`-W` to specify an explicit wrapper C file name. If you set the
``dump-on-error`` option in the configuration options section you will get a
dump of the configuration on an error.
Configuration (INI) files
=========================
The Trace Linker is controlled using configuration files. Configuration files
are categorized into 3 types:
- User Configuration: These are specific to the user application to be traced.
This file initializes the values of the trace generator, triggers, enables,
and traces.
- Tracer Configuration: These are like a library of common or base trace
functions that can be referenced by an application. These files tend to hold
the details needed to wrap a specific set of functions. Examples provided with
the RTEMS Linker are the RTEMS API and Libc.
- Generator Configuration: This is used to encapsulate a specific method of
tracing. RTEMS currently provides generators for trace buffering, printk, and
printf.
The configuration files are in the *INI file format* which is composed of
`sections`. Each section has a section name and set of *keys* which consist of
*names* and *values*. A typical key is of the form ``name=value``. Keys can be
used to include other INI files using the include key name. This is shown in the
following example where the values indicate rtems and rtld-base configuration
files:
.. code-block:: shell
include = rtems.ini, rtld-base.ini
The trace linker also uses values in keys to specify other sections. In this
example the functions name lists `test-trace-funcs` and that section contains a
headers key that further references a section called `test-headers`:
.. code-block:: shell
functions = test-trace-funcs, rtems-api
[test-trace-funcs]
; Parsed via the 'function-set', not parse as a 'trace'.
headers = test-headers
[test-headers]
header = '#include "test-trace-1.h"'
The format of a configuration file is explained next. Snippets of the file:
`test-trace.ini` have been used for explicit understanding. This file can
be found in the rtems-tools directory of the rtems installation.
Tracer Section
--------------
The topmost level section is the ``tracer`` section. It can contains the
following keys:
- ``name``: The name of trace being linked.
- ``options``: A list of option sections.
- ``defines``: A list of sections containing defines or define record.
- ``define``: A list of define string that are single or double quoted.
- ``enables``: The list of sections containing enabled functions to trace.
- ``triggers``: The list of sections containing enabled functions to trigger
trace on.
- ``traces``: The list of sections containing function lists to trace.
- ``functions``: The list of sections containing function details.
- ``include``: The list of files to include.
The tracer section of the file:`test-trace.ini` is shown below with explanatory
comments.
.. code-block:: shell
;
; RTEMS Trace Linker Test Configuration.
;
; We must provide a top level trace section.
;
[tracer]
;
; Name of the trace.
;
name = RTEMS Trace Linker Test
;
; The BSP.
;
bsp = sparc/sis
;
; Functions to trace.
;
traces = test-trace, test-trace-funcs, rtems-api-task
;
; Specify the options.
;
options = test-options
;
; Define the function sets. These are the function's that can be
; added to the trace lists.
;
functions = test-trace-funcs, rtems-api
;
; Include RTEMS Trace support.
;
include = rtems.ini, rtld-base.ini
Options section
---------------
The options section in the fileio-trace.ini is called the `fileio-options`. A
general options section can contain following sets of keys:
- ``dump-on-error``: Dump the parsed configuration data on error. The value can
be true or false.
- ``verbose``: Set the verbose level. The value can be true or a number value.
- ``prefix``: The prefix for the tools and an install RTEMS if rtems-path is not
set.
- ``cc``: The compiler used to compile the generated wrapper code. Overrides the
BSP configuration value if a BSP is specified.
- ``ld``: The linker used to link the application. The default is the cc value
as read from the BSP configuration if specified. If your application contains
C++ code use this setting to the change the linker to g++.
- ``cflags``: Set the CFLAGS used to compiler the wrapper. These flags are
pre-pended to the BSP read flags if a BSP is specified. This option is used
to provide extra include paths to header files in your application that
contain types referenced by functions being traced.
- ``rtems-path``: The path to an install RTEMS if not installed under the
prefix.
- ``rtems-bsp``: The BSP we are building the trace executable for. The is an
arch and bsp pair. For example sparc/erc32.
The options section of the file: `test-trace.ini` uses two of the aforementioned
keys as shown below:
.. code-block:: shell
;
; Options can be defined here or on the command line.
;
[test-options]
prefix = /development/rtems/5
verbose = true
Trace Section
-------------
A trace section defines how trace wrapper functions are built. To build a trace
function that wraps an existing function in an ELF object file or library
archive we need to have the function's signature. A signature is the function's
declaration with any types used. The signature has specific types and we need
access to those types which means the wrapper code needs to include header files
that define those types. There may also be specific defines needed to access
those types. A trace section can contain the following keys:
- ``generator``: The generator defines the type of tracing being used.
- ``headers``: List of sections that contain header file's keys.
- ``header``: A header key. Typically the include code.
- ``defines``: List of sections that contain defines.
- ``define``: A define key. Typically the define code.
- ``signatures``: List of function signature sections.
- ``trace``: Functions that are instrumented with trace code.
The trace section of the file: `test-trace.ini` is shown below. A trace section
can reference other trace sections of a specific type. This allows a trace
sections to build on other trace sections.
.. code:: shell
; User application trace example.
;
[test-trace]
generator = printf-generator
; Just here for testing.
trace = test_trace_3
[test-trace-funcs]
; Parsed via the 'function-set', not parse as a 'trace'.
headers = test-headers
header = '#include "test-trace-2.h"'
defines = test-defines
define = "#define TEST_TRACE_2 2"
signatures = test-signatures
; Parsed via the 'trace', not parsed as a function-set
trace = test_trace_1, test_trace_2
[test-headers]
header = '#include "test-trace-1.h"'
[test-defines]
define = "#define TEST_TRACE_1 1"
[test-signatures]
test_trace_1 = void, int
test_trace_2 = test_type_2, test_type_1
test_trace_3 = float, float*
Function Section
----------------
Function sections define functions that can be traced. Defining a function so it
can be traced does not mean it is traced. The function must be added to a trace
list to be traced. Function sections provide any required defines, header files,
and the function signatures.
A function signature is the function's declaration. It is the name of the
function, the return value, and the arguments. Tracing using function wrappers
requires that we have accurate function signatures and ideally we would like to
determine the function signature from the data held in ELF files. ELF files can
contain DWARF data, the ELF debugging data format. In time the trace project
would like to support libdwarf so the DWARF data can be accessed and used to
determine a function's signature. This work is planned but not scheduled to be
done and so in the meantime we explicitly define the function signatures in the
configuration files.
A function section can consist of the following keys:
- ``headers``: A list of sections containing headers or header records.
- ``header``: A list of include string that are single or double quoted.
- ``defines``: A list of sections containing defines or define record.
- ``defines``: A list of define string that are single or double quoted.
- ``signatures``: A list of section names of function signatures.
- ``includes``: A list of files to include.
Function signatures are specified with the function name being the key's name
and the key's value being the return value and a list of function arguments. You
need to provide void if the function uses void. Variable argument list are
currently not supported. There is no way to determine statically a variable
argument list. The function section in the file: `test-trace.ini` has been
labeled as `test-trace-funcs`. This can be seen in the file snippet of the
previous section.
Generators
----------
The trace linker's major role is to wrap functions in the existing executable
with trace code. The directions on how to wrap application functions is provided
by the generator configuration. The wrapping function uses a GNU linker option
called --wrap=symbol. The GNU Ld manual states:
"Use a wrapper function for symbol. Any undefined reference to symbol will be
resolved to __wrap_symbol. Any undefined reference to __real_symbol will be
resolved to symbol."
Generator sections specify how to generate trace wrapping code. The trace
linker and generator section must match to work. The trace linker expects a some
things to be present when wrapping functions. The section's name specifies the
generator and can be listed in a generator key in a tracer or trace section. If
the generator is not interested in a specific phase it does not need to define
it. Nothing will be generated in regard to this phase. For example code to
profile specific functions may only provide the entry-trace and exit-trace code
where a nano-second time stamp is taken.
The generate code will create an entry and exit call and the generator code
block can be used to allocate buffer space for each with the lock held. The
entry call and argument copy is performed with the lock released. The buffer
space having been allocated will cause the trace events to be in order. The same
goes for the exit call. Space is allocated in separate buffer allocate calls so
the blocking calls will have the exit event appear in the correct location in
the buffer.
The following keys can be a part of the generator configuration:
- ``headers``: A list of sections containing headers or header records.
- ``header``: A list of include string that are single or double quoted.
- ``defines``: A list of sections containing defines or define record.
- ``define``: A list of define string that are single or double quoted.
- ``entry-trace``: The wrapper call made on a function's entry. Returns bool
where true is the function is being traced. This call is made without the lock
being held if a lock is defined.
- ``arg-trace``: The wrapper call made for each argument to the trace function
if the function is being traced. This call is made without the lock being held
if a lock is defined.
- ``exit-trace``: The wrapper call made after a function's exit. Returns bool
where true is the function is being traced. This call is made without the lock
being held if a lock is defined.
- ``ret-trace``: The wrapper call made to log the return value if the function
is being traced. This call is made without the lock being held if a lock is
defined.
- ``lock-local``: The wrapper code to declare a local lock variable.
- ``lock-acquire``: The wrapper code to acquire the lock.
- ``lock-release``: The wrapper code to release the lock.
- ``buffer-local``: The wrapper code to declare a buffer index local variable.
- ``buffer-alloc``: The wrapper call made with a lock held if defined to
allocate buffer space to hold the trace data. A suitable 32bit buffer index is
returned. If there is no space an invalid index is returned. The generator
must handle any overhead space needed. The generator needs to make sure the
space is available before making the alloc all.
- ``code-blocks``: A list of code block section names.
- ``code``: A code block in <<CODE --- CODE (without the single quote).
- ``includes``: A list of files to include.
The following macros can be used in wrapper calls:
- ``@FUNC_NAME@``: The trace function name as a quote C string.
- ``@FUNC_INDEX@``: The trace function index as a held in the sorted list of
trace functions by the trace linker. It can be used to index the names,
enables, and triggers data.
- ``@FUNC_LABEL@``: The trace function name as a C label that can be referenced.
You can take the address of the label.
- ``@FUNC_DATA_SIZE@``: The size of the data in bytes.
- ``@FUNC_DATA_ENTRY_SIZE@``: The size of the entry data in bytes.
- ``@FUNC_DATA_RET_SIZE@``: The size of the return data in bytes.
- ``@ARG_NUM@``: The argument number to the trace function.
- ``@ARG_TYPE@``: The type of the argument as a C string.
- ``@ARG_SIZE@``: The size of the type of the argument in bytes.
- ``@ARG_LABEL@``: The argument as a C label that can be referenced.
- ``@RET_TYPE@``: The type of the return value as a C string.
- ``@RET_SIZE@``: The size of the type of the return value in bytes.
- ``@RET_LABEL@``: The return value as a C label that can be referenced.
The `buffer-alloc`, `entry-trace`, and `exit-trace` can be transformed using the
following macros:
- ``@FUNC_NAME@``
- ``@FUNC_INDEX@``
- ``@FUNC_LABEL@``
- ``@FUNC_DATA_SZIE@``
- ``@FUNC_DATA_ENTRY_SZIE@``
- ``@FUNC_DATA_EXIT_SZIE@``
The `arg-trace` can be transformed using the following macros:
- ``@ARG_NUM@``
- ``@ARG_TYPE@``
- ``@ARG_SIZE@``
- ``@ARG_LABEL@``
The `ret-trace` can be transformed using the following macros:
- ``@RET_TYPE@``
- ``@RET_SIZE@``
- ``@RET_LABEL@``
The file: `test-trace.ini` specifies ``printf-generator`` as its generator. This
section can be found in the file: `rtld-print.ini` in the rtems-tools directory
and is shown below:
.. code:: shell
;
; A printf generator prints to stdout the trace functions.
;
[printf-generator]
headers = printf-generator-headers
entry-trace = "rtld_pg_printf_entry(@FUNC_NAME@, (void*) &@FUNC_LABEL@);"
arg-trace = "rtld_pg_printf_arg(@ARG_NUM@, @ARG_TYPE@, @ARG_SIZE@, (void*) &@ARG_LABEL@);"
exit-trace = "rtld_pg_printf_exit(@FUNC_NAME@, (void*) &@FUNC_LABEL@);"
ret-trace = "rtld_pg_printf_ret(@RET_TYPE@, @RET_SIZE@, (void*) &@RET_LABEL@);"
code = <<<CODE
static inline void rtld_pg_printf_entry(const char* func_name,
void* func_addr)
{
printf (">>> %s (0x%08x)\n", func_name, func_addr);
}
static inline void rtld_pg_printf_arg(int arg_num,
const char* arg_type,
int arg_size,
void* arg)
{
const unsigned char* p = arg;
int i;
printf (" %2d] %s(%d) = ", arg_num, arg_type, arg_size);
for (i = 0; i < arg_size; ++i, ++p) printf ("%02x", (unsigned int) *p);
printf ("\n");
}
static inline void rtld_pg_printf_exit(const char* func_name,
void* func_addr)
{
printf ("<<< %s (0x%08x)\n", func_name, func_addr);
}
static inline void rtld_pg_printf_ret(const char* ret_type,
int ret_size,
void* ret)
{
const unsigned char* p = ret;
int i;
printf (" rt] %s(%d) = ", ret_type, ret_size);
for (i = 0; i < ret_size; ++i, ++p) printf ("%02x", (unsigned int) *p);
printf ("\n");
}
CODE
[printf-generator-headers]
header = "#include <stdio.h>"
The trace linker generates C code with a wrapper for each function to be
instrumented. The trace code generated is driven by the configuration INI files.
Development
===========
The Trace Linker is part of the RTEMS tools git repository available at :
https://git.rtems.org/rtems-tools
The RTEMS tools project utilizes the waf build system. Use the following
commands in the topmost build directory to build the tools project:
First we configure using:
.. code-block:: shell
$./waf configure --prefix=$HOME/development/rtems/5
Then we build and install using:
.. code-block:: shell
$./waf build install