If the options added by
* commit 022f20f663 (ctest: add command line option to run the tests listed
in a given file, 2023-11-29, v3.29.0-rc1~66^2~2)
* commit dbacc1d5a8 (ctest: add command line option to exclude tests listed
in a given file, 2023-11-30, v3.29.0-rc1~66^2~1)
* commit 701029726f (ctest_test: add options INCLUDE_FROM_FILE and
EXCLUDE_FROM_FILE, 2023-12-03, v3.29.0-rc1~66^2)
are given a missing file, fail instead of ignoring it.
Fixes: #25740
Add `--tests-from-file <filename>` to run only the tests listed in the
given file. The test names must match exactly, no regexps or something.
The listed tests can still be filtered with a regexp using -R.
Issue: #25455
`include-what-you-use` diagnostics, in practice, are specific to
the environment's compiler and standard library. Update includes
to satisfy IWYU for our CI job under Debian 12.
CTest normally terminates test processes on timeout using `SIGKILL`.
Offer tests a chance to exit gracefully, on platforms supporting POSIX
signals, by setting `TIMEOUT_SIGNAL_{NAME,GRACE_PERIOD}` properties.
Fixes: #17288
Remove a comment that has not been relevant since commit 177edc5ed1
(Fixed ctest -N segfault issue. Further refactored ctest. Enabled
failover for ctest, 2009-08-27, v2.8.0~250).
Add `--test-output-truncation` to `ctest`. This option can be used to
customize which part of the test output is being truncated. Currently
supported values are `tail`, `middle` and `head`.
Also add equivalent `CTEST_CUSTOM_TEST_OUTPUT_TRUNCATION` variable.
Fixes: #23206
This property allows projects to modify environment variables at test
time rather than trying to guess what the state should be based on what
is present at configure time. Of particular interest is the ability to
use a `PATH` present at test time while adding entries known to be
necessary for the test itself.
There are multiple operations provided to modify variables, including:
- setting and unsetting
- appending and prepending as:
- strings
- path lists
- CMake lists
Additionally, a `reset` action is provided to cancel any prior
modifications to that particular variable in the case of incremental
additions to the test property.
Teach CTest to parse output for <CTestMeasurement> in addition to
<DartMeasurement> for measurements defined at runtime.
Use a new class (cmCTestTestMeasurementXMLParser) derived from cmXMLParser
to parse the data and attributes these XML elements. This is an improvement
over our previous approach of using a series of regular expressions.
As part of this commit we also rename some member variables and methods
to make their purpose more clear.
DartStuff -> AllTestMeasurementsRegex
DartStuff1 -> SingleTestMeasurementRegex
DartString -> TestMeasurementsOutput
GenerateDartOutput() -> GenerateCTestXML()
GenerateRegressionImages() -> RecordCustomTestMeasurements()
cmCTestRunTest::DartProcessing() -> ParseOutputForMeasurements()
Allow tests to specify files to upload at runtime. Previously this was
only possible to specify at configure time with the ATTACHED_FILES
test properties.
This commit also fixes a bug in how our test data tarballs were generated
by CTest. Previously, if you tried to attach a file outside of the binary
directory, CTest would generate a tar file with a relative path, and tar
would not allow you to extract it. We resolve this problem by creating
tar files with a flat directory structure instead.
Fixes: #22284
#pragma once is a widely supported compiler pragma, even though it is
not part of the C++ standard. Many of the issues keeping #pragma once
from being standardized (distributed filesystems, build farms, hard
links, etc.) do not apply to CMake - it is easy to build CMake on a
single machine. CMake also does not install any header files which can
be consumed by other projects (though cmCPluginAPI.h has been
deliberately omitted from this conversion in case anyone is still using
it.) Finally, #pragma once has been required to build CMake since at
least August 2017 (7f29bbe6 enabled server mode unconditionally, which
had been using #pragma once since September 2016 (b13d3e0d)). The fact
that we now require C++11 filters out old compilers, and it is unlikely
that there is a compiler which supports C++11 but does not support
#pragma once.
The `PROCESSES` test property name added for CMake 3.16 is too close to
the existing `PROCESSORS` test property. Furthermore, the property in
principle specifies groups of resources organized in a way that is
meaningful to a particular test. The groups may often correspond to
processes but they could have other meanings. Since the property name
`PROCESSES` has not been in a final 3.16 release yet, simply rename it
to `RESOURCE_GROUPS`.
Fixes: #19914
Exit code constants on Windows, such as `STATUS_NO_MEMORY` do not fit in
a 32-bit signed integer type. They do fit in an unsigned 32-bit type,
but for compatibility with UNIX semantics we treat exit codes as signed.
Use a 64-bit signed integer to handle both.
- Number of passed/failed tests is colored according to the whole outcome
- Individual listed tested are colored according to their completion status:
* Disabled: blue
* Failed: red
* Not Run: yellow
In commit v2.8.0~170 (ENH: Added ctest test options PROCESSORS and
RUN_SERIAL, 2009-09-07) CTest learned to track the number of processors
allocated to running tests in order to balance it against the desired
level of parallelism. Extend this idea by introducing a new
`PROCESSOR_AFFINITY` test property to ask that CTest run a test
with the CPU affinity mask set. This will allow a set of tests
that are running concurrently to use disjoint CPU resources.
This commit introduces cmDuration as a typedef for
std::chrono::duration<double, std::ratio<1>>. It is less verbose and
provides for a point to put future common functionality for durations.
No functional change intended.
This commit continues the refactoring of CTest to adopt std::chrono.
After the last sets of changes that introduced std::chrono::steady_clock
and std::chrono::system_clock respectively, it makes sense to have all
the timeouts be stored as std::chrono::duration.
No functional change intended.
After the refactor to make CTest use std::chrono::steady_clock for the
keeping of time for test duration, there are still references to
cmSystemTools::GetTime() left.
To further adopt std::chrono for time related activities, this commit
changes those remaining references to std::chrono::system_clock::now()
calls and alters the storage from either unsigned int or double to
std::chrono::system_clock::time_point.
For ease of conversion, a converter method is added to cmXMLWriter that
converts from a std::chrono::system_clock::time_point to the number of
seconds since the UN*X epoch as that is expected behaviour. This means
no more casts as required.
Functionally should be no difference as the system_clock is implemented
in the same terms.
