Revert commit e5ec0e52f4 (AUTOUIC: Fix generating of dependency rules
for UI header files, 2021-07-22, v3.21.1~8^2) because it caused
regressions. For example, changing one C++ source can now cause many
others to rebuild. Revert the change pending further investigation.
Fixes: #22531
Issue: #16776
We could not rely on .ui files when generating the ninja rules
for the generated UI header files. .ui files might be added to the
target sources but never processed by AUTOUIC afterward, since UI
header files are never included in a source code. Instead of adding
dependency rules based on the .ui files, this approach scans
non-generated source files for includes of the UI header files,
as AUTOUIC does. This gives the consistent set of UI header files
at configure time, that could be used to generate byproducts rules
for the AUTOUIC. Also, the path to the generated UI header file depends
not on the .ui file location but on the include line is used in source
files.
Fixes: #16776
#pragma once is a widely supported compiler pragma, even though it is
not part of the C++ standard. Many of the issues keeping #pragma once
from being standardized (distributed filesystems, build farms, hard
links, etc.) do not apply to CMake - it is easy to build CMake on a
single machine. CMake also does not install any header files which can
be consumed by other projects (though cmCPluginAPI.h has been
deliberately omitted from this conversion in case anyone is still using
it.) Finally, #pragma once has been required to build CMake since at
least August 2017 (7f29bbe6 enabled server mode unconditionally, which
had been using #pragma once since September 2016 (b13d3e0d)). The fact
that we now require C++11 filters out old compilers, and it is unlikely
that there is a compiler which supports C++11 but does not support
#pragma once.
We used to store information for the _autogen target in a CMake script
file AutogenInfo.cmake, which was imported by a temporary cmake instance in
the _autogen target. This introduced the overhead of creating a temporary
cmake instance and inherited the limitations of the CMake language which
only supports lists.
This patch introduces JSON files to pass information to AUTORCC and
autogen_ targets. JSON files are more flexible for passing data, e.g. they
support nested lists.
The patch has the side effects that
- AutogenInfo.cmake is renamed to AutogenInfo.json
- AutogenOldSettings.txt is renamed to AutogenUsed.txt
- RCC<qrcBaseName><checksum>Info.cmake is renamed to
AutoRcc_<qrcBaseName>_<checksum>_Info.json
- RCC<qrcBaseName><checksum>.lock is renamed to
AutoRcc_<qrcBaseName>_<checksum>_Lock.lock
- RCC<qrcBaseName><checksum>Settings.txt is renamed to
AutoRcc_<qrcBaseName>_<checksum>_Used.txt
Provide a standardized way to handle the C++ "standard" headers
customized to be used with current CMake C++ standard constraints.
Offer under directory `cm` headers which can be used as direct
replacements of the standard ones. For example:
#include <cm/string_view>
can be used safely for CMake development in place of the `<string_view>`
standard header.
Fixes: #19491
This introduces the `cmQtAutoGenerator::MessagePath` method, that abbreviates
paths by placing a
- "SRC:" prefix in place of the project source directory
- "BIN:" prefix in place of the project binary directory
The method is used in `AUTO{MOC,UIC,RCC}` when paths are displayed in
messages. This makes the messages generated by `AUTO{MOC,UIC,RCC}` shorter
and improves their readability.
New features
------------
CMake's `AUTOMOC` and `AUTOUIC` now cache information extracted when parsing
source files in `CMakeFiles/<ORIGIN>_autogen.dir/ParseCache.txt`.
This leads to faster `<ORIGIN>_autogen` target rebuilds, because source files
will be parsed again only if they're newer than the `ParseCache.txt` file.
The parse cache will be recomputed if it is older than the CMake executable.
`AUTOMOC` and `AUTOUIC` now check if `moc` or `uic` output files are older
than the `moc` or `uic` executable. If an output file is older than the
compiler, it will be regenerated. Therefore if a new `moc` or `uic` version
is installed, all output files will be regenerated.
`AUTOMOC` and `AUTOUIC` error and warning messages are more detailed.
Internal changes
----------------
`moc` and `uic` output file names are not computed in the `_autogen`
target anymore but in `cmQtAutoGenInitializer`. This makes the available at
the configuration stage for improved dependency computations (to be done).
In `AutogenInfo.cmake`, equally sized lists for "source file names",
"source file flags" and "compiler output file names" are passed to the
`_autogen` target. This replaces the separate file lists for
`AUTOMOC` and `AUTOUIC`.
Files times are read from the file system only once by using `cmFileTime`
instances instead of `cmQtAutoGenerator::FileSystem::FileIsOlderThan` calls.
All calls to not thread safe file system functions are moved to non concurrent
fence jobs (see `cmWorkerPool::JobT::IsFence()`). This renders the
`cmQtAutoGenerator::FileSystem` wrapper class obsolete and it is removed.
Instead of composing a single large settings string that is fed to the
`cmCryptoHash`, now all setting sub strings are fed one by one to the
`cmCryptoHash` and the finalized result is stored.
The `std::mutex` in `cmQtAutoGenerator::Logger` is tagged `mutable` and most
`cmQtAutoGenerator::Logger` methods become `const`.
Outlook
-------
This patch provides the framework required to
- extract dependencies from `.ui` files in `AUTOUIC`.
These will help to address issue
#15420 "AUTOUIC: Track uic external inputs".
- generate adaptive `make` and `ninja` files in the `_autogen` target.
These will help to address issue
#16776 "AUTOUIC: Ninja needs two passes to correctly build Qt project".
- generate (possibly empty) `moc` and `uic` files for all headers instead of a
`mocs_compilation.cpp` file.
This will help to address issue
#17277 "AUTOMOC: Provide a option to allow AUTOMOC to compile individual "
"moc_x.cxx instead of including all in mocs_compilation.cxx"
This factors out the concurrency framework in `cmQtAutoGeneratorMocUic` to a
dedicated class `cmWorkerPool` which might be reused in other places.
`cmWorkerPool` supports fence jobs that require that
- all other jobs before in the queue have been processed before the fence
job processing gets started,
- no jobs later in the queue will be processed before the fence job processing
has been completed.
Fence jobs are needed where the completion of all previous jobs in the queue
is a requirement for further processing. E.g. in `cmQtAutoGeneratorMocUic`
the generation of `mocs_compilation.cpp` requires that all previous
source file parse jobs have been completed.
`cmQtAutoGenerator` automatically started a libuv loop in the constructor.
The loop is needed in `cmQtAutoGeneratorMocUic`, but not in
`cmQtAutoGeneratorRcc` (anymore). To avoid starting the loop in
`cmQtAutoGeneratorRcc`, this patch moves the loop variables and startup code
from `cmQtAutoGenerator` to `cmQtAutoGeneratorMocUic`.
`cmQtAutoGenerator::FileSystem` used to have a reference to a
`cmQtAutoGenerator::Logger` instances. This was used for utility methods
that automatically generated an error message on demand. Unfortunately this
resulted in double error messages in some places. To fix these and with
the additional purpose of removing the dependency of
`cmQtAutoGenerator::FileSystem` to `cmQtAutoGenerator::Logger`, this patch
removes the `Logger` reference in `cmQtAutoGenerator::FileSystem`.
In the process some silent error conditions gained error messages and
some existing error messages were extended.
`cmQtAutoGenerator` automatically added `cmQtAutoGenerator::Logger` and
`cmQtAutoGenerator::FileSystem` member variables to all inherited classes.
This patch moves these members variable declarations to the inherited classes,
where needed.
`cmQtAutoGenerator::FileSystem` is only required for concurrent file system
access, but `cmQtAutoGeneratorRcc` isn't concurrent. Therefore this patch
replaces all `cmQtAutoGenerator::FileSystem` uses in `cmQtAutoGeneratorRcc`.
`CollapseCombinedPath` was introduced by commit 551d3343cd (cmDependsC:
Collapse relative include paths, 2013-06-19, v2.8.12~237^2) where the
existing `CollapseFullPath` should have been used instead. Then its use
proliferated slightly. Since `CollapseCombinedPath` is less widely used
and less robust (see issue #19049), use `CollapseFullPath` everywhere
instead.
Issue: #19050
This adds a dedicated mutex for file reading and writing to
cmQtAutoGenerator::FileSystem. The purpose of the change is
to avoid that long files reads block cmsys based path computations,
which are protected by an other mutex.
This introduces concurrent thread processing in the `_autogen`
target wich processes AUTOMOC and AUTOUIC.
Source file parsing is distributed among the threads by
using a job queue from which the threads pull new parse jobs.
Each thread might start an independent ``moc`` or ``uic`` process.
Altogether this roughly speeds up the AUTOMOC and AUTOUIC build
process by the number of physical CPUs on the host system.
The exact number of threads to start in the `_autogen` target
is controlled by the new AUTOGEN_PARALLEL target property which
is initialized by the new CMAKE_AUTOGEN_PARALLEL variable.
If AUTOGEN_PARALLEL is empty or unset (which is the default)
the thread count is set to the number of physical CPUs on
the host system.
The AUTOMOC/AUTOUIC generator and the AUTORCC generator are
refactored to use a libuv loop internally.
Closes#17422.
