Additionally, I have attempted to clean up some CMake issues to make the
package's build interface cleaner, in particular, avoiding polluting the
parent directory's include path with our config.h file (if PCRE2 is being
included as a subdirectory).
This re-adds changes from Theodore's commit:
def175f4a9
and partially reverts changes from Carlo's commit:
92d56a1f7c
---------
Co-authored-by: Theodore Tsirpanis <teo@tsirpanis.gr>
* The primary purpose of pcre2_next_match() is to make it much easier for
PCRE2 clients to iterate over matches, without needing an advanced knowledge
of regular expressions.
* Secondly, we can simplify our own code by merging the three duplicate
implementations of the /g global match behaviour: pcre2demo, pcre2_substitute,
and pcre2test.
* Thirdly, as I look closely at the issue, I can improve the documentation.
* Fourthly, I would like to actually simplify the logic, removing a complex loop
which makes several match attempts, swallows duplicate matches, and more.
We can have identical behaviour with a simple retry using
PCRE2_NOTEMPTY_ATSTART.
Both the Autoconf and CMake build systems are updated to detect linker support for symbol versioning.
Currently, Linux, Solaris, and FreeBSD are tested and working. Windows (COFF) and macOS (Mach-O) have no symbol versioning.
There is an Autoconf/CMake flag to opt out of the versioning behaviour.
We have four files which have .c extensions, but which are actually #included rather than treated as their own compilation unit.
This goes against conventions - Autotools, CMake, and Bazel all assume that the .h/.c distinction indicates which files are compilation units.
pcre2_jit_match.c -> _inc.h
pcre2_jit_misc.c -> _inc.h
pcre2_printint.c -> _inc.h
pcre2_ucptables.c -> _inc.h
Fixes to enable the code to build with a simple `CC=xlc ./configure --enable-ebcdic --disable-unicode`.
Fixes to the tests, so that `make check` passes on EBCDIC platforms.
Add a CI job to do z/OS testing.
An additional testing argument, `-malloc` is added to pcre2test and to RunTest.
The ManyConfig tests run this now in CI.
We exercise each malloc failure in the core code by counting how many mallocs are done, then repeating compilation and matching with a failure on each successive malloc.
Add a simple hash code for group names to improve search speed.
Ignore duplicates when group names are searched.
Improve finding of duplicates (they have the same name pointer).
Improve creating name table (duplicates are handled in one step).
Create a new file for name management.
The pcre2test utility needs quite a few changes to accommodate this.
It is simpler to add a new mode to it, than to make it fully
EBCDIC-native. On an ASCII system, pcre2test performs ASCII I/O, but
tranlates the input when passing it to the fully-EBCDIC-supporting
library.
We want to ensure that the symbols included in shared object files (.so, .dylib, .dll) are as expected.
This verifies that our Autoconf and CMake builds are using the `-fvisibility` flags and attributes correctly.
As it turns out, the CMake build on Solaris is not working, and was exposing internal symbols due to a CMake issue.
Debian's "lintian" picked this up - line 950 in the man page starts
with a ' which is how you start a roff request. You can reproduce the
warning thus:
```
LC_ALL=C.UTF-8 MANROFFSEQ='' MANWIDTH=80 \
man --warnings -E UTF-8 -l -Tutf8 -Z doc/pcre2grep.1 >/dev/null
```
The fix is to add a zero-width space (`\&`) to the start of the
relevant line (indeed `groff_man(7)` suggests exactly this use for \&).
---------
Co-authored-by: Matthew Vernon <matthew@debian.org>
__GNUC__ is defined by any compiler that claims compliance to GNU
but that doesn't include the cmdline interface, so avoid passing
GCC specific warning flags to clang.
* autotools: retire conditional for debug build
likely added by mistake, the functionality works through `--enable-debug`
instead.
* maint: allow selecting compiler for ManyConfigTests
Instead of hardcoding the compiler as `cc`, let a CC environment
variable dictate which compiler to use.
For example, in macOS/arm64 where the GNU compiler is provided by
brew the following will allow using it instead of the system compiler
(which ALSO answers to `gcc` even though is `clang`)
% CC=gcc-13 maint/ManyConfigTests
Part of #651
* Use much stricter windows warnings (`/W3` rather than `/W1`). This requires quite a few fixes for all the sloppy places where we do implicit assignment of 64-bit values to 32-bit storage.
* Use and test CMake build & install on FreeBSD and Solaris
* Add 64 bit Solaris build (`cc -m64`) and fix existing Solaris warnings
* Make compile flags used in CI consistent across platforms. Previously Mac & Linux were building with different warning flags.
* Add `--enable-Werror` to `configure.ac`. This means that you can build with `-Werror` in a clean way. Previously, you had to hackily override the CPPFLAGS when calling `make` since you can't pass `-Werror` as a CFLAG into `./configure` (it messes with compiler feature detection).
The workflow shall be:
* When the release number is bumped, all references to that release number need to be bumped immediately. (For example, when the source code moves from 10.45 → 10.46, the man pages must do so as well.)
* When documentation is updated, there's no need to update the "last modified" dates by hand. We can sweep those all up during the release process. Or update them immediately - there's no harm in it; we simply aren't obliged to.
I believe this is the correct procedure, based on Philip's documentation.
The libpcre2-posix interface is completely unchanged, but the source code has been updated.
The libpcre2-NN interface has been extended in backwards-compatible ways, with new enum values and API functions, so callers compiled and linked against the old version may use the newer as a drop-in replacement, but callers compiled against the 10.45 headers will fail when used against the old version (if they require any newly-added functions).
The new CI job ensures that `make distcheck` passes.
It also bundles up the tarball and includes in the GitHub artifacts, along with a GitHub-provided attestation that the tarball is derived from the given build steps.
* Add details on new maintainership
* Remove checked-in autoconf outputs
* Sync & cleanup files with Detrail
* Add CI job for ensuring PrepareRelease is run
* Add Ubuntu-20.04 autoconf runner
* Make CMake installed files match autoconf
* Update acknowledgements
New flag: PCRE2_EXTRA_TURKISH_CASING, and pre-pattern flag
(*TURKISH_CASING).
Also added a pre-pattern flag (*CASELESS_RESTRICT) for this existing
flag.
UCD 16 makes a lot of changes to scripts, so make sure that we have
sufficient coverage by keeping the original autogenerated tests in
addition.
Complete the code updates for changes to ScriptExtensions.txt which
is no longer sorted by script and allow for multiple unicode property
test files, depending on Unicode version.
The original asserts weren't very useful in debug mode as they
were lacking information on where they were being triggered and
were also unreliable and dangerous as they could result in
important code being removed and trigger crashes (even in non
debug mode).
Instead of implementing one generic assert for both modes, build
a more useful one for each one, so PCRE2_UNREACHABLE() could be
also used in non debug builds to help with optimization.
Reinstate all original assertions to use the new versions, which
will have the sideeffect of fixing indentation issues introduced
in the original, and include additional asserts that were provided
as the original ones were being audited for safety. Note that during
such audit the use of the original asserts might had been refactored
so it also includes all those relevant code changes.
While at it, update cmake and CI to help with testing as well as
other documentation.
Co-authored-by: Alex Dowad <alexinbeijing@gmail.com>
Updates to Unicode files to Unicode 16, adjusts tests, and the
scripts used to parse UCD, to adapt to minor formatting differences
in UCD 16.
The `GenerateTest26.py` and `GenerateCommon.py` had a regexp to
extract properties from the `ScriptExtensions.txt` file. Previously,
all property lines had one space after space-separated list of scripts.
In UCD-16, this list is adjusted with right-padding, which throws off
the parser.
This commit adjusts the regexps to ignore padding spaces.
JIT has several good uses of unsigned integer wraparounds, that
clang's UBSAN doesn't like (which is controversial, because it is
clearly not undefined behaviour), but since it is usually good to
know when they happen by accident it makes sense to make sure the
rest of PCRE2 codebase benefits from checking it.
While at it, upgrade the version of the base image to use a newer
OS as a canary from when the rest of the jobs upgrade themselves
and be a little more strict to catch other constructs that are not
welcomed in our codebase.
Adds more sanitizers and including "fixes" to the code to make them
mostly clean, while also using link-size 3.
While at it, make sure the original "basic" job (now renamed) error
on failure for better visibility, cleanup some 32bit signed shifts
and update documentation.
Starting with Unicode 15, the provided DerivedBidiClass data file
reports different default values to use for unassigned characters
in different groups.
Process the additional hints for that specific file, and allow
overriding the values later if more specific.
Since that was previously forbidden, change get_other_case() to
report when no valid value could be provided and allow skipping
conflicting rule lines that required that restriction.
While at it, Allow using the long identifiers in `ucptest` with
the `find bidi` command (underscores also allowed).
* ucptest: regenerate testoutput
Last sync with 1a5fcd (Remove unused variables in ucptest.c and update test data
for added properties, 2022-04-25), and showing significant differences.
* fix `findprop +` with UTF-8 characters and duplicated other case
