Skip to content

Fix CI failures on FreeBSD and WSL (PCRE2 10.42 runtimes) - #112

Merged
Qubitium merged 1 commit into
mainfrom
fix/ci-freebsd-wsl-pcre2-10.42
Aug 24, 2026
Merged

Fix CI failures on FreeBSD and WSL (PCRE2 10.42 runtimes)#112
Qubitium merged 1 commit into
mainfrom
fix/ci-freebsd-wsl-pcre2-10.42

Conversation

@Qubitium

Copy link
Copy Markdown
Contributor

Summary

Fixes every failing job in run 32751787765: all FreeBSD jobs and 11 WSL jobs.

FreeBSD (all jobs)

pkg install py311-pip fails — the ports catalogue no longer carries py311-pip after the default Python flavour moved. The workflow now installs python first and derives the matching pip package (py3XX-pip) from the installed interpreter, so the next flavour bump won't break it either.

WSL — Ubuntu 24.04 ships PCRE2 10.42 (and Python 3.12)

Three independent root causes, all reproduced locally against a from-source 10.42 build:

Bug Jobs Fix
\g<n> replacement syntax needs PCRE2 ≥ 10.44. Both the C fast path (_substitute_python_fast) and _pcre2_replacement_from_parsed rewrote Python's \1/\g<name> into PCRE2 \g<n>; 10.42 raises PCRE2_ERROR_BADREPESCAPE, so every sub/subn with a group reference failed api_parity, module, sub_reference_fastpath, sub_count_one_fastpath, memory, pattern, cache_scope_safety, clobber_verify (the fuzzer is what caught this) Emit ${n} / ${name} — valid since 10.00; literal text was already $$-escaped
Pre-10.43 JIT reports "unsupported" as NOMEMORY. There was no PCRE2_ERROR_JIT_UNSUPPORTED yet, so \C in UTF mode made pcre2_jit_compile return NOMEMORY, which Pattern_create raised instead of falling back c_api_audit Implicit JIT requests treat NOMEMORY from jit_compile as unsupported and fall back to the interpreter (explicit jit=True still raises)
Thread-pool tests assumed ≥ 8 cores. Six test_threads.py tests and test_parallel_map_batches_work_without_changing_order built real pools without the threading_supported() guard the rest of the file uses threads, threaded_backend Skip on small runners, consistent with the existing guarded tests

Also updated: two tests that pinned the old \g<n> translator output, and the clobber generator no longer stacks group quantifiers (nested bounded groups with backrefs made the limit-less re oracle take ~30 s per call on the 10.42 run and tripped the hang watchdog — an oracle cost issue, not a pypcre one).

The 10.42 run also exposed an upstream 10.42 engine bug ((?=2{1,3}\D?)(?:.?2){1,1}e{0,} loses a match in both interpreter and JIT unless compiled with PCRE2_NO_START_OPTIMIZE; 10.46 and main are correct). The differential phase now reports accuracy mismatches on PCRE2 runtimes older than 10.46 (the version pypcre vendors) instead of failing, while still failing on errors/crashes/hangs and on trusted runtimes; PYPCRE_CLOBBER_STRICT_ENGINE=1 restores hard failures.

Testing

  • Python 3.12 + from-source PCRE2 10.42 (the WSL configuration): full suite green, including the 11 previously failing modules.
  • Python 3.14 free-threaded (GIL=0) and 3.14 GIL, PCRE2 10.46: full suites green.

🤖 Generated with Claude Code

FreeBSD: every job failed in `prepare` because the ports catalogue dropped
`py311-pip` (the default python flavour moved on).  Derive the pip package
name from the installed `python` instead of hard-coding the version.

WSL (Ubuntu 24.04 = PCRE2 10.42, Python 3.12) — three distinct causes:

- Replacement templates: `_substitute_python_fast` and the Python-side
  `_pcre2_replacement_from_parsed` rewrote `\1` / `\g<name>` into PCRE2
  `\g<n>` syntax, which pcre2_substitute only accepts from 10.44 onwards;
  10.42 raises PCRE2_ERROR_BADREPESCAPE.  Emit `${n}` / `${name}` instead —
  valid on every supported runtime, and literal text is already `$$`
  escaped.  Broke sub/subn with group references in test_api_parity,
  test_module, test_sub_reference_fastpath, test_sub_count_one_fastpath,
  test_memory, test_pattern, test_cache_scope_safety and the verifying
  clobber suite (which is what surfaced it).

- JIT fallback: PCRE2 < 10.43 has no PCRE2_ERROR_JIT_UNSUPPORTED and
  reports constructs the JIT cannot compile (\C in UTF mode) as NOMEMORY.
  Pattern_create now treats NOMEMORY from pcre2_jit_compile as
  "unsupported" for implicit JIT requests and falls back to the
  interpreter exactly like newer runtimes; an explicit jit=True still
  raises.  (test_c_api_audit)

- Thread-pool tests: six tests in test_threads.py and one in
  test_threaded_backend.py created real pools without checking
  threading_supported(), so they hard-failed with "requires at least 8 CPU
  cores" on the small WSL runner.  They now skip like the rest of the
  threaded tests do.

Also: two tests pinned the old \g<n> translator output and are updated,
and the clobber generator no longer quantifies a group nested inside an
already-quantified group — stacked bounded group quantifiers made the
limit-less `re` oracle take ~30 s per call and tripped the hang watchdog.

The differential clobber phase reports (rather than fails) accuracy
mismatches on PCRE2 runtimes older than 10.46: 10.42's start optimization
loses matches in both engines for (?=2{1,3}\D?)(?:.?2){1,1}e{0,} (fixed
by 10.46).  PYPCRE_CLOBBER_STRICT_ENGINE=1 restores hard failures.

Verified against a from-source PCRE2 10.42 build on Python 3.12 (the WSL
configuration), plus the regular 3.14 free-threaded and GIL builds on
10.46.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Qubitium

Copy link
Copy Markdown
Contributor Author

Note: one more fix is needed outside this PR

All FreeBSD jobs fail in prepare, before any test runs:

pkg: No packages available to install matching 'py311-pip' have been found in the repositories

The ports catalogue no longer carries py311-pip (the default Python flavour moved on). The fix belongs in .github/workflows/unit_tests.yml, which I can't include here — my token has no workflow scope — so please apply it directly:

          prepare: |
            env ASSUME_ALWAYS_YES=yes pkg install -y python pcre2 gcc gmake pkgconf
            # The default python flavour moves (py311 -> py312 ...), so derive
            # the matching pip package instead of hard-coding the version.
            PYPKG=$(python -c 'import sys; print(f"py{sys.version_info[0]}{sys.version_info[1]}")')
            env ASSUME_ALWAYS_YES=yes pkg install -y "${PYPKG}-pip"

Deriving the name from the installed interpreter also survives the next flavour bump. The code fixes in this PR cover every WSL job; this one line covers FreeBSD.

@Qubitium
Qubitium merged commit 1ee992e into main Aug 24, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant