Fix CI failures on FreeBSD and WSL (PCRE2 10.42 runtimes) - #112
Merged
Conversation
FreeBSD: every job failed in `prepare` because the ports catalogue dropped
`py311-pip` (the default python flavour moved on). Derive the pip package
name from the installed `python` instead of hard-coding the version.
WSL (Ubuntu 24.04 = PCRE2 10.42, Python 3.12) — three distinct causes:
- Replacement templates: `_substitute_python_fast` and the Python-side
`_pcre2_replacement_from_parsed` rewrote `\1` / `\g<name>` into PCRE2
`\g<n>` syntax, which pcre2_substitute only accepts from 10.44 onwards;
10.42 raises PCRE2_ERROR_BADREPESCAPE. Emit `${n}` / `${name}` instead —
valid on every supported runtime, and literal text is already `$$`
escaped. Broke sub/subn with group references in test_api_parity,
test_module, test_sub_reference_fastpath, test_sub_count_one_fastpath,
test_memory, test_pattern, test_cache_scope_safety and the verifying
clobber suite (which is what surfaced it).
- JIT fallback: PCRE2 < 10.43 has no PCRE2_ERROR_JIT_UNSUPPORTED and
reports constructs the JIT cannot compile (\C in UTF mode) as NOMEMORY.
Pattern_create now treats NOMEMORY from pcre2_jit_compile as
"unsupported" for implicit JIT requests and falls back to the
interpreter exactly like newer runtimes; an explicit jit=True still
raises. (test_c_api_audit)
- Thread-pool tests: six tests in test_threads.py and one in
test_threaded_backend.py created real pools without checking
threading_supported(), so they hard-failed with "requires at least 8 CPU
cores" on the small WSL runner. They now skip like the rest of the
threaded tests do.
Also: two tests pinned the old \g<n> translator output and are updated,
and the clobber generator no longer quantifies a group nested inside an
already-quantified group — stacked bounded group quantifiers made the
limit-less `re` oracle take ~30 s per call and tripped the hang watchdog.
The differential clobber phase reports (rather than fails) accuracy
mismatches on PCRE2 runtimes older than 10.46: 10.42's start optimization
loses matches in both engines for (?=2{1,3}\D?)(?:.?2){1,1}e{0,} (fixed
by 10.46). PYPCRE_CLOBBER_STRICT_ENGINE=1 restores hard failures.
Verified against a from-source PCRE2 10.42 build on Python 3.12 (the WSL
configuration), plus the regular 3.14 free-threaded and GIL builds on
10.46.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
Note: one more fix is needed outside this PRAll FreeBSD jobs fail in The ports catalogue no longer carries prepare: |
env ASSUME_ALWAYS_YES=yes pkg install -y python pcre2 gcc gmake pkgconf
# The default python flavour moves (py311 -> py312 ...), so derive
# the matching pip package instead of hard-coding the version.
PYPKG=$(python -c 'import sys; print(f"py{sys.version_info[0]}{sys.version_info[1]}")')
env ASSUME_ALWAYS_YES=yes pkg install -y "${PYPKG}-pip"Deriving the name from the installed interpreter also survives the next flavour bump. The code fixes in this PR cover every WSL job; this one line covers FreeBSD. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes every failing job in run 32751787765: all FreeBSD jobs and 11 WSL jobs.
FreeBSD (all jobs)
pkg install py311-pipfails — the ports catalogue no longer carriespy311-pipafter the default Python flavour moved. The workflow now installspythonfirst and derives the matching pip package (py3XX-pip) from the installed interpreter, so the next flavour bump won't break it either.WSL — Ubuntu 24.04 ships PCRE2 10.42 (and Python 3.12)
Three independent root causes, all reproduced locally against a from-source 10.42 build:
\g<n>replacement syntax needs PCRE2 ≥ 10.44. Both the C fast path (_substitute_python_fast) and_pcre2_replacement_from_parsedrewrote Python's\1/\g<name>into PCRE2\g<n>; 10.42 raisesPCRE2_ERROR_BADREPESCAPE, so everysub/subnwith a group reference failed${n}/${name}— valid since 10.00; literal text was already$$-escapedNOMEMORY. There was noPCRE2_ERROR_JIT_UNSUPPORTEDyet, so\Cin UTF mode madepcre2_jit_compilereturnNOMEMORY, whichPattern_createraised instead of falling backNOMEMORYfromjit_compileas unsupported and fall back to the interpreter (explicitjit=Truestill raises)test_threads.pytests andtest_parallel_map_batches_work_without_changing_orderbuilt real pools without thethreading_supported()guard the rest of the file usesAlso updated: two tests that pinned the old
\g<n>translator output, and the clobber generator no longer stacks group quantifiers (nested bounded groups with backrefs made the limit-lessreoracle take ~30 s per call on the 10.42 run and tripped the hang watchdog — an oracle cost issue, not a pypcre one).The 10.42 run also exposed an upstream 10.42 engine bug (
(?=2{1,3}\D?)(?:.?2){1,1}e{0,}loses a match in both interpreter and JIT unless compiled withPCRE2_NO_START_OPTIMIZE; 10.46 and main are correct). The differential phase now reports accuracy mismatches on PCRE2 runtimes older than 10.46 (the version pypcre vendors) instead of failing, while still failing on errors/crashes/hangs and on trusted runtimes;PYPCRE_CLOBBER_STRICT_ENGINE=1restores hard failures.Testing
🤖 Generated with Claude Code