add_test: Optionally use a launcher for tests running in-project targets

Add a `CMAKE_TEST_LAUNCHER` variable and corresponding `TEST_LAUNCHER`
target property.

Issue: #23672

Help/command/add_test.rst

@@ -27,9 +27,31 @@ directory the test is created in.
 ``add_test`` options are:
 
 ``COMMAND``
-  Specify the test command-line.  If ``<command>`` specifies an executable
+  Specify the test command-line.
-  target created by :command:`add_executable`, it will automatically be
+
-  replaced by the location of the executable created at build time.
+  If ``<command>`` specifies an executable target created by
+  :command:`add_executable`:
+
+  * It will automatically be replaced by the location of the executable
+    created at build time.
+
+  * .. versionadded:: 3.3
+
+      The target's :prop_tgt:`CROSSCOMPILING_EMULATOR`, if set, will be
+      used to run the command on the host::
+
+        <emulator> <command>
+
+  * .. versionadded:: 3.29
+
+      The target's :prop_tgt:`TEST_LAUNCHER`, if set, will be
+      used to launch the command::
+
+        <launcher> <command>
+
+      If the :prop_tgt:`CROSSCOMPILING_EMULATOR` is also set, both are used::
+
+        <launcher> <emulator> <command>
 
   The command may be specified using
   :manual:`generator expressions <cmake-generator-expressions(7)>`.

Help/envvar/CMAKE_TEST_LAUNCHER.rst

@@ -0,0 +1,11 @@
+CMAKE_TEST_LAUNCHER
+-------------------
+
+.. versionadded:: 3.29
+
+.. include:: ENV_VAR.txt
+
+The default value for the :variable:`CMAKE_TEST_LAUNCHER` variable when there
+is no explicit configuration given on the first run while creating a new
+build tree.  On later runs in an existing build tree the value persists in
+the cache as :variable:`CMAKE_TEST_LAUNCHER`.

Help/manual/cmake-env-variables.7.rst

@@ -56,6 +56,7 @@ Environment Variables that Control the Build
    /envvar/CMAKE_MSVCIDE_RUN_PATH
    /envvar/CMAKE_NO_VERBOSE
    /envvar/CMAKE_OSX_ARCHITECTURES
+   /envvar/CMAKE_TEST_LAUNCHER
    /envvar/CMAKE_TOOLCHAIN_FILE
    /envvar/DESTDIR
    /envvar/LDFLAGS

Help/manual/cmake-properties.7.rst

@@ -398,6 +398,7 @@ Properties on Targets
    /prop_tgt/Swift_MODULE_DIRECTORY
    /prop_tgt/Swift_MODULE_NAME
    /prop_tgt/SYSTEM
+   /prop_tgt/TEST_LAUNCHER
    /prop_tgt/TYPE
    /prop_tgt/UNITY_BUILD
    /prop_tgt/UNITY_BUILD_BATCH_SIZE

Help/manual/cmake-variables.7.rst

@@ -117,6 +117,7 @@ Variables that Provide Information
    /variable/CMAKE_Swift_COMPILATION_MODE
    /variable/CMAKE_Swift_MODULE_DIRECTORY
    /variable/CMAKE_Swift_NUM_THREADS
+   /variable/CMAKE_TEST_LAUNCHER
    /variable/CMAKE_TOOLCHAIN_FILE
    /variable/CMAKE_TWEAK_VERSION
    /variable/CMAKE_VERBOSE_MAKEFILE

Help/prop_test/PASS_REGULAR_EXPRESSION.rst

@@ -23,7 +23,8 @@ Example:
 To run a test that may have a system-level failure, but still pass if
 ``PASS_REGULAR_EXPRESSION`` matches, use a CMake command to wrap the
 executable run. Note that this will prevent automatic handling of the
-:prop_tgt:`CROSSCOMPILING_EMULATOR` target property.
+:prop_tgt:`CROSSCOMPILING_EMULATOR` and :prop_tgt:`TEST_LAUNCHER`
+target property.
 
 .. code-block:: cmake
 

Help/prop_test/SKIP_REGULAR_EXPRESSION.rst

@@ -25,7 +25,8 @@ Example:
 To run a test that may have a system-level failure, but still skip if
 ``SKIP_REGULAR_EXPRESSION`` matches, use a CMake command to wrap the
 executable run. Note that this will prevent automatic handling of the
-:prop_tgt:`CROSSCOMPILING_EMULATOR` target property.
+:prop_tgt:`CROSSCOMPILING_EMULATOR` and :prop_tgt:`TEST_LAUNCHER`
+target property.
 
 .. code-block:: cmake
 

Help/prop_test/SKIP_RETURN_CODE.rst

@@ -25,7 +25,8 @@ signal abort, or heap errors may fail the test even if the return code matches.
 To run a test that may have a system-level failure, but still skip if
 ``SKIP_RETURN_CODE`` matches, use a CMake command to wrap the executable run.
 Note that this will prevent automatic handling of the
-:prop_tgt:`CROSSCOMPILING_EMULATOR` target property.
+:prop_tgt:`CROSSCOMPILING_EMULATOR` and :prop_tgt:`TEST_LAUNCHER` target
+property.
 
 .. code-block:: cmake
 

Help/prop_test/WILL_FAIL.rst

@@ -19,7 +19,8 @@ is ``true``:
 To run a test that may have a system-level failure, but still pass if
 ``WILL_FAIL`` is set, use a CMake command to wrap the executable run.
 Note that this will prevent automatic handling of the
-:prop_tgt:`CROSSCOMPILING_EMULATOR` target property.
+:prop_tgt:`CROSSCOMPILING_EMULATOR` and :prop_tgt:`TEST_LAUNCHER`
+target property.
 
 .. code-block:: cmake
 

Help/prop_tgt/TEST_LAUNCHER.rst

@@ -0,0 +1,20 @@
+TEST_LAUNCHER
+-------------
+
+.. versionadded:: 3.29
+
+Use the given launcher to run executables.
+This command will be added as a prefix to :command:`add_test` commands
+for build target system executables and is meant to be run on the host
+machine.
+
+It effectively acts as a run script for tests in a similar way
+to how :variable:`CMAKE_<LANG>_COMPILER_LAUNCHER` works for compilation.
+
+If this property contains a :ref:`semicolon-separated list <CMake Language
+Lists>`, then the first value is the command and remaining values are its
+arguments.
+
+This property is initialized by the value of the
+:variable:`CMAKE_TEST_LAUNCHER` variable if it is set when a target
+is created.

Help/release/dev/cmake-test-launcher.rst

@@ -0,0 +1,7 @@
+cmake-test-launcher
+-------------------
+
+* A :variable:`CMAKE_TEST_LAUNCHER` variable and corresponding
+  :prop_tgt:`TEST_LAUNCHER` target property were added to specify
+  a launcher to be used by executable targets when invoked by
+  tests added by the :command:`add_test` command.

Help/variable/CMAKE_TEST_LAUNCHER.rst

@@ -0,0 +1,16 @@
+CMAKE_TEST_LAUNCHER
+-------------------
+
+.. versionadded:: 3.29
+
+This variable is used to specify a launcher for running tests, added
+by the :command:`add_test` command, that run an executable target.
+If this variable contains a :ref:`semicolon-separated list <CMake Language
+Lists>`, then the first value is the command and remaining values are its
+arguments.
+
+This variable can be initialized via an
+:envvar:`CMAKE_TEST_LAUNCHER` environment variable.
+
+It is also used as the default value for the
+:prop_tgt:`TEST_LAUNCHER` target property of executables.

Source/cmTarget.cxx

@@ -594,6 +594,7 @@ TargetProperty const StaticTargetProperties[] = {
   { "CROSSCOMPILING_EMULATOR"_s, IC::ExecutableTarget },
   { "EXPORT_COMPILE_COMMANDS"_s, IC::CanCompileSources },
   { "FOLDER"_s },
+  { "TEST_LAUNCHER"_s, IC::ExecutableTarget },
 
   // Xcode properties
   { "XCODE_GENERATE_SCHEME"_s, IC::NeedsXcode },

Source/cmTestGenerator.cxx

@@ -168,6 +168,18 @@ void cmTestGenerator::GenerateScriptForConfig(std::ostream& os,
     // Use the target file on disk.
     exe = target->GetFullPath(config);
 
+    // Prepend with the test launcher if specified.
+    cmValue launcher = target->GetProperty("TEST_LAUNCHER");
+    if (cmNonempty(launcher)) {
+      cmList launcherWithArgs{ *launcher };
+      std::string launcherExe(launcherWithArgs[0]);
+      cmSystemTools::ConvertToUnixSlashes(launcherExe);
+      os << cmOutputConverter::EscapeForCMake(launcherExe) << " ";
+      for (std::string const& arg : cmMakeRange(launcherWithArgs).advance(1)) {
+        os << cmOutputConverter::EscapeForCMake(arg) << " ";
+      }
+    }
+
     // Prepend with the emulator when cross compiling if required.
     cmValue emulator = target->GetProperty("CROSSCOMPILING_EMULATOR");
     if (cmNonempty(emulator)) {

Source/cmake.cxx

@@ -2517,6 +2517,16 @@ int cmake::ActualConfigure()
                         "Name of generator toolset.", cmStateEnums::INTERNAL);
   }
 
+  if (!this->State->GetInitializedCacheValue("CMAKE_TEST_LAUNCHER")) {
+    cm::optional<std::string> testLauncher =
+      cmSystemTools::GetEnvVar("CMAKE_TEST_LAUNCHER");
+    if (testLauncher && !testLauncher->empty()) {
+      std::string message = "Test launcher to run tests executable.";
+      this->AddCacheEntry("CMAKE_TEST_LAUNCHER", *testLauncher, message,
+                          cmStateEnums::STRING);
+    }
+  }
+
   if (!this->State->GetInitializedCacheValue(
         "CMAKE_CROSSCOMPILING_EMULATOR")) {
     cm::optional<std::string> emulator =

Tests/RunCMake/CrosscompilingEmulator/AddTest-check.cmake

@@ -26,3 +26,7 @@ endif()
 if(testfile_contents MATCHES "add_test[(]DoesNotUseEmulatorWithExecTargetFromSubdirAddedWithGenex [^\n]+pseudo_emulator[^\n]+\n")
   message(SEND_ERROR "Used emulator when it should not be used. ${error_details}")
 endif()
+
+if(NOT testfile_contents MATCHES "add_test[(]UsesTestLauncherAndEmulator[^\n]+pseudo_test_launcher.*pseudo_emulator[^\n]+\n")
+  message(SEND_ERROR "Did not use test launcher and emulator when they should be used. ${error_details}")
+endif()

Tests/RunCMake/CrosscompilingEmulator/AddTest.cmake

@@ -18,3 +18,9 @@ add_test(NAME UsesEmulatorWithExecTargetFromSubdirAddedWithoutGenex
 
 add_test(NAME DoesNotUseEmulatorWithExecTargetFromSubdirAddedWithGenex
   COMMAND $<TARGET_FILE:generated_exe_in_subdir_added_to_test_with_genex>)
+
+add_executable(generated_exe_test_launcher simple_src_exiterror.cxx)
+set_property(TARGET generated_exe_test_launcher PROPERTY TEST_LAUNCHER "pseudo_test_launcher")
+
+add_test(NAME UsesTestLauncherAndEmulator
+  COMMAND generated_exe_test_launcher)

Tests/RunCMake/add_test/RunCMakeTest.cmake

@@ -41,3 +41,9 @@ block()
   set(RunCMake_TEST_NO_CLEAN 1)
   run_cmake_command(EmptyArgument-ctest ${CMAKE_CTEST_COMMAND} -C Debug)
 endblock()
+
+set(RunCMake_TEST_OPTIONS
+    "-DCMAKE_TEST_LAUNCHER=/path/to/pseudo_test_launcher")
+
+run_cmake(TestLauncherProperty)
+run_cmake(TestLauncher)

Tests/RunCMake/add_test/TestLauncher-check.cmake

@@ -0,0 +1,28 @@
+set(testfile "${RunCMake_TEST_BINARY_DIR}/CTestTestfile.cmake")
+if(EXISTS "${testfile}")
+  file(READ "${testfile}" testfile_contents)
+else()
+  message(FATAL_ERROR "Could not find expected CTestTestfile.cmake.")
+endif()
+
+set(error_details "There is a problem with generated test file: ${testfile}")
+
+if(testfile_contents MATCHES "add_test[(]DoesNotUseTestLauncher [^\n]+pseudo_test_launcher[^\n]+\n")
+  message(SEND_ERROR "Used test launcher when it should not be used. ${error_details}")
+endif()
+
+if(NOT testfile_contents MATCHES "add_test[(]UsesTestLauncher [^\n]+pseudo_test_launcher[^\n]+\n")
+  message(SEND_ERROR "Did not use test launcher when it should be used. ${error_details}")
+endif()
+
+if(testfile_contents MATCHES "add_test[(]DoesNotUseTestLauncherWithGenex [^\n]+pseudo_test_launcher[^\n]+\n")
+  message(SEND_ERROR "Used test launcher when it should not be used. ${error_details}")
+endif()
+
+if(NOT testfile_contents MATCHES "add_test[(]UsesTestLauncherWithExecTargetFromSubdirAddedWithoutGenex [^\n]+pseudo_test_launcher[^\n]+\n")
+  message(SEND_ERROR "Did not use test launcher when it should be used. ${error_details}")
+endif()
+
+if(testfile_contents MATCHES "add_test[(]DoesNotUseTestLauncherWithExecTargetFromSubdirAddedWithGenex [^\n]+pseudo_test_launcher[^\n]+\n")
+  message(SEND_ERROR "Used test launcher when it should not be used. ${error_details}")
+endif()

Tests/RunCMake/add_test/TestLauncher.cmake

@@ -0,0 +1,22 @@
+project(test_launcher LANGUAGES C)
+
+enable_testing()
+add_test(NAME DoesNotUseLauncher
+  COMMAND ${CMAKE_COMMAND} -E echo "Hi")
+
+add_executable(generated_exe simple_src_exiterror.cxx)
+set_target_properties(generated_exe PROPERTIES LINKER_LANGUAGE C)
+
+add_test(NAME UsesTestLauncher
+  COMMAND generated_exe)
+
+add_test(NAME DoesNotUseTestLauncherWithGenex
+  COMMAND $<TARGET_FILE:generated_exe>)
+
+add_subdirectory(TestLauncher)
+
+add_test(NAME UsesTestLauncherWithExecTargetFromSubdirAddedWithoutGenex
+  COMMAND generated_exe_in_subdir_added_to_test_without_genex)
+
+add_test(NAME DoesNotUseTestLauncherWithExecTargetFromSubdirAddedWithGenex
+  COMMAND $<TARGET_FILE:generated_exe_in_subdir_added_to_test_with_genex>)

Tests/RunCMake/add_test/TestLauncher/CMakeLists.txt

@@ -0,0 +1,9 @@
+add_executable(generated_exe_in_subdir_added_to_test_without_genex
+  ${CMAKE_CURRENT_SOURCE_DIR}/../simple_src_exiterror.cxx)
+set_target_properties(generated_exe_in_subdir_added_to_test_without_genex
+  PROPERTIES LINKER_LANGUAGE C)
+
+add_executable(generated_exe_in_subdir_added_to_test_with_genex
+  ${CMAKE_CURRENT_SOURCE_DIR}/../simple_src_exiterror.cxx)
+set_target_properties(generated_exe_in_subdir_added_to_test_with_genex
+  PROPERTIES LINKER_LANGUAGE C)

Tests/RunCMake/add_test/TestLauncherProperty.cmake

@@ -0,0 +1,41 @@
+
+# This tests setting the TEST_LAUNCHER target property from the
+# CMAKE_TEST_LAUNCHER variable.
+
+# -DCMAKE_TEST_LAUNCHER=/path/to/pseudo_test_launcher is passed to this
+# test
+
+project(test_launcher LANGUAGES C)
+
+add_executable(target_with_test_launcher simple_src_exiterror.cxx)
+set_target_properties(target_with_test_launcher PROPERTIES LINKER_LANGUAGE C)
+get_property(launcher TARGET target_with_test_launcher
+             PROPERTY TEST_LAUNCHER)
+if(NOT "${launcher}" MATCHES "pseudo_test_launcher")
+  message(SEND_ERROR "Default TEST_LAUNCHER property not set")
+endif()
+
+set_property(TARGET target_with_test_launcher
+             PROPERTY TEST_LAUNCHER "another_test_launcher")
+get_property(launcher TARGET target_with_test_launcher
+             PROPERTY TEST_LAUNCHER)
+if(NOT "${launcher}" MATCHES "another_test_launcher")
+  message(SEND_ERROR
+    "set_property/get_property TEST_LAUNCHER is not consistent")
+endif()
+
+unset(CMAKE_TEST_LAUNCHER CACHE)
+add_executable(target_without_test_launcher simple_src_exiterror.cxx)
+set_target_properties(target_without_test_launcher PROPERTIES LINKER_LANGUAGE C)
+get_property(launcher TARGET target_without_test_launcher
+             PROPERTY TEST_LAUNCHER)
+if(NOT "${launcher}" STREQUAL "")
+  message(SEND_ERROR "Default TEST_LAUNCHER property not set to null")
+endif()
+
+add_executable(target_with_empty_test_launcher simple_src_exiterror.cxx)
+set_target_properties(target_with_empty_test_launcher PROPERTIES LINKER_LANGUAGE C)
+set_property(TARGET target_with_empty_test_launcher PROPERTY TEST_LAUNCHER "")
+
+enable_testing()
+add_test(NAME test_target_with_empty_test_launcher COMMAND target_with_empty_test_launcher)

Tests/RunCMake/add_test/simple_src_exiterror.cxx

@@ -0,0 +1,4 @@
+int main(int, char**)
+{
+  return 13;
+}