Additionally, I have attempted to clean up some CMake issues to make the
package's build interface cleaner, in particular, avoiding polluting the
parent directory's include path with our config.h file (if PCRE2 is being
included as a subdirectory).
This re-adds changes from Theodore's commit:
def175f4a9
and partially reverts changes from Carlo's commit:
92d56a1f7c
---------
Co-authored-by: Theodore Tsirpanis <teo@tsirpanis.gr>
* The primary purpose of pcre2_next_match() is to make it much easier for
PCRE2 clients to iterate over matches, without needing an advanced knowledge
of regular expressions.
* Secondly, we can simplify our own code by merging the three duplicate
implementations of the /g global match behaviour: pcre2demo, pcre2_substitute,
and pcre2test.
* Thirdly, as I look closely at the issue, I can improve the documentation.
* Fourthly, I would like to actually simplify the logic, removing a complex loop
which makes several match attempts, swallows duplicate matches, and more.
We can have identical behaviour with a simple retry using
PCRE2_NOTEMPTY_ATSTART.
We won't implement more advanced/alternative global replacement strategies, but we can at least write a few sentences explaining how to do it in application code.
Both the Autoconf and CMake build systems are updated to detect linker support for symbol versioning.
Currently, Linux, Solaris, and FreeBSD are tested and working. Windows (COFF) and macOS (Mach-O) have no symbol versioning.
There is an Autoconf/CMake flag to opt out of the versioning behaviour.
We have four files which have .c extensions, but which are actually #included rather than treated as their own compilation unit.
This goes against conventions - Autotools, CMake, and Bazel all assume that the .h/.c distinction indicates which files are compilation units.
pcre2_jit_match.c -> _inc.h
pcre2_jit_misc.c -> _inc.h
pcre2_printint.c -> _inc.h
pcre2_ucptables.c -> _inc.h
An additional testing argument, `-malloc` is added to pcre2test and to RunTest.
The ManyConfig tests run this now in CI.
We exercise each malloc failure in the core code by counting how many mallocs are done, then repeating compilation and matching with a failure on each successive malloc.
Add a simple hash code for group names to improve search speed.
Ignore duplicates when group names are searched.
Improve finding of duplicates (they have the same name pointer).
Improve creating name table (duplicates are handled in one step).
Create a new file for name management.
The pcre2test utility needs quite a few changes to accommodate this.
It is simpler to add a new mode to it, than to make it fully
EBCDIC-native. On an ASCII system, pcre2test performs ASCII I/O, but
tranlates the input when passing it to the fully-EBCDIC-supporting
library.
Fixes#564
The previous API was not extensible to handle multi-character case rules. It required a fair bit of reworking in order to accommodate this. I had to delay the casing transformations to be done later, by buffering up the string to transform, and then allowing the callback to do an in-place transformation on the entire input to be transformed.
I reckon that callers are assuming that when you use the PCRE2_SUBSTITUTE_OVERFLOW_LENGTH option, it will calculate the entire memory requirement in one go. Just two calls should be sufficient (rather than needing to loop with a gradually-increasing buffer size).
However, with a substitution callout this is not true. If you call once with PCRE2_SUBSTITUTE_OVERFLOW_LENGTH, the buffer length returned might still not be sufficient for the second call to succeed.
This is because the callout might not be called the first time, but the second time it will be called and can affect control flow, by requiring even more buffer to be used. This occurs even if the callout is completely stateless, idempotent and well-behaved.
This fix ensures that when we skip a callout (due to overflow), we still request enough buffer size for either option that the callout might return.
* Add details on new maintainership
* Remove checked-in autoconf outputs
* Sync & cleanup files with Detrail
* Add CI job for ensuring PrepareRelease is run
* Add Ubuntu-20.04 autoconf runner
* Make CMake installed files match autoconf
* Update acknowledgements
Avoid one crash introduced with recent changes to substitute code as well as clarify what the expected offset value should be when overflowing the provided buffer.
---------
Co-authored-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
I haven't tackled any controversial steps in this PR - simply tidying the formatting.
I have used the `gersemi` tool, which simply "does its thing". I have additionally renamed a few variables to match standard casing conventions (but I am aware that some lowercased variables are used, for example in package-config files, and have left those alone).
* Move some existing character class code into pcre2_compile_class.c
* Add a new flag PCRE2_ALT_EXTENDED_CLASS to change the behaviour of
parsing [...] character classes, to emit new META codes, and new
OP_ECLASS codes for nested character classes with operators
* Document the behaviour relative to the UTS#18 standard
* No JIT support; it falls back to the interpreter. DFA is supported.
New flag: PCRE2_EXTRA_TURKISH_CASING, and pre-pattern flag
(*TURKISH_CASING).
Also added a pre-pattern flag (*CASELESS_RESTRICT) for this existing
flag.
Perl does not use $0 anymore to refer to the text of the matched subject
and `pcre2_substitute()` was recently updated to also provide that value
using the variable Perl prefers: `$&`.
In a similar context, either as part of the formatted output from a match
or during the processing of a callback, teach pcre2grep to also populate
$&.
While at it, update the ChangeLog with recent changes.
It is anticipated that over time, more and more optimizations will be
added to PCRE2, and we want to be able to switch optimizations off/on,
both for testing purposes and to be able to work around bugs in a
released library version.
The number of free bits left in the compile options word is very small.
Hence, we will start putting all optimization enable/disable flags in
a separate word. To switch these off/on, the new API function
pcre2_set_optimization() will be used.
The values which can be passed to pcre2_set_optimization() are
different from the internal flag bit values. The values accepted by
pcre2_set_optimization() are contiguous integers, so there is no
danger of ever running out of them. This means in the future, the
internal representation can be changed at any time without breaking
backwards compatibility. Further, the 'directives' passed to
pcre2_set_optimization() are not restricted to control a single,
specific optimization. As an example, passing PCRE2_OPTIMIZATION_FULL
will turn on all optimizations supported by whatever version of
PCRE2 the client program happens to be linked with.
Co-authored-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Co-authored-by: Zoltan Herczeg <hzmester@freemail.hu>
