Build Linux language extensions with GCC 13 on the RHEL 9 baseline - #101
Draft
SicongLiu2000 wants to merge 3 commits into
Draft
Build Linux language extensions with GCC 13 on the RHEL 9 baseline#101SicongLiu2000 wants to merge 3 commits into
SicongLiu2000 wants to merge 3 commits into
Conversation
SicongLiu2000
force-pushed
the
dev/sicongliu/exthost-v3-gcc13-all-languages
branch
9 times, most recently
from
July 28, 2026 01:38
c2ff604 to
a69f918
Compare
The Linux extensions are built on Ubuntu 24.04 with GCC 13 and must load on RHEL 9, which ships glibc 2.34 and libstdc++ 3.4.29. Rather than change where they are built, this makes the produced binaries meet that floor. GCC 13 emits a reference to std::ios_base_library_init()@GLIBCXX_3.4.32 into every translation unit that includes <iostream>, and that single symbol was the only libstdc++ requirement above the floor. The shipped R sources therefore no longer include it: the includes in RExtension.cpp and RTypeUtils.cpp were unused, and Logger.cpp writes through stdio. The Rcpp and RInside headers also include <iostream> and cannot be changed here, so the symbol is additionally defined locally. R must not link libstdc++ statically: libR.so already links libstdc++.so.6, and a second static copy puts two C++ runtimes in one process, which aborts partway through script execution. language-extensions/common/linux/AbiFloorCompat_linux.cpp collects the local definitions needed to stay within the floor: std::ios_base_library_init 3.4.32 <iostream>, via Rcpp/RInside _dl_find_object 2.35 libgcc.a unwinder arc4random 2.36 libstdc++.a __isoc23_strtoul 2.38 libstdc++.a __isoc23_strtoll/strtoull 2.38 <stdlib.h> C23 redirect arc4random is backed by getrandom(2) with a /dev/urandom fallback and aborts rather than returning predictable bytes. _dl_find_object reports "no information", which makes the libgcc unwinder use its dl_iterate_phdr path - the same path taken on every glibc older than 2.35. All definitions are hidden, so the extensions never interpose on glibc for the rest of the host process. Also in this change: - restore-packages.sh asserts the GCC major version so the toolchain cannot silently vary between languages. - The Java extension links libstdc++/libgcc statically (it does not load libR, so there is no duplicate-runtime hazard) and uses the modern JDK lib/server layout instead of the Java 8 jre/lib/amd64/server path. - Test fixes: a dangling c_str() in the R result-column tests, and UTF-16 literals and raw byte comparisons in the Python output-parameter tests, which were relying on std::wstring under -fshort-wchar.
SicongLiu2000
force-pushed
the
dev/sicongliu/exthost-v3-gcc13-all-languages
branch
from
July 28, 2026 02:00
a69f918 to
ac09800
Compare
… shim The ABI-floor shim defined std::ios_base_library_init() as an empty function. That was wrong, and it crashed libPythonExtension on Ubuntu 22.04 and RHEL 9. GCC 13 changed how the standard streams are constructed. Before 13, every translation unit including <iostream> emitted its own static std::ios_base::Init object. GCC 13 emits a call to std::ios_base_library_init() instead and performs the construction inside libstdc++. Stubbing that call out therefore means the streams are never constructed at all, and the first write to std::cout or std::cerr faults. It only shows up on hosts whose libstdc++ predates the change: Ubuntu 22.04 libstdc++.so.6.0.30 no self-init -> crash RHEL 9 GCC 11/12 era no self-init -> crash Ubuntu 24.04 GCC 13+ library self-initialises -> fine RHEL 10 GCC 13+ library self-initialises -> fine AzureLinux 3 GCC 13+ library self-initialises -> fine which matches the PVS board exactly: the Python satellite core-dumped on Ubuntu 22.04 and RHEL 9 immediately after receiving its first data chunk, while all three newer distros passed. libPythonExtension.so does reference _ZSt4cout@GLIBCXX_3.4 and _ZSt4cerr@GLIBCXX_3.4, so the comment claiming no shipped source used the standard streams was simply incorrect. R was unaffected because libR.so is built by the distro toolchain and its own translation units still emit std::ios_base::Init; libpython3.12 is C and initialises nothing, so the Python extension had no other initialiser. Construct a function-local static std::ios_base::Init instead. The constructor is refcounted and idempotent, the local static guard runs it exactly once no matter how many translation units call in, and std::ios_base::Init::Init is GLIBCXX_3.4 - present in every libstdc++ we target - so the ABI floor is unchanged.
The Python satellite segfaulted on Ubuntu 22.04 and RHEL 9 while passing on Ubuntu 24.04, RHEL 10 and AzureLinux 3. Two core dumps place the fault inside the host's own libstdc++, not in our code: round 1 Ubuntu 22.04 libstdc++.so.6.0.30 SIGSEGV at +0xa1f8a round 2 RHEL 9 libstdc++.so.6.0.29 SIGSEGV at +0xa16ea Same function, two different builds of the library. The extension is compiled with GCC 13, so the stream insertion operators are inlined from the GCC 13 headers into this library and then execute against stream objects owned by an older runtime. Ubuntu 24.04, RHEL 10 and AzureLinux 3 ship a GCC 13-era libstdc++ and are unaffected. The satellite log pins it exactly: a passing session's first line after the handshake is the 'Init' trace emitted by Logger::Log, and on the failing distros the log stops immediately before it - the process dies on its very first log call. The symbol-version ABI gate cannot catch this. Every symbol referenced is within GLIBCXX_3.4.29 and the gate passes; the dependency is inlined code relying on newer libstdc++ internals, which no version check can see. Route the three remaining stream users through stdio, which has no such coupling: Logger::LogError/LogException/Log and the relay of the user script's captured stdout/stderr in PythonSession. EKM and the ONNX extension were fixed the same way earlier. R was never affected because it does not reference the standard streams, and Java is safe because it links libstdc++ statically. Also hardens the ZIP normalization added for external libraries, which had two real defects even though it is not the cause of this crash (an uncaught Python exception would raise SIGABRT, and both cores are SIGSEGV): entry data was read after mutating entry.filename, which makes ZipFile.open compare the directory name against the local header and raise BadZipFile on exactly the archives the code exists to fix; and bp::exec was unguarded, so any Python exception would escape into a process where that is fatal. It now reads before renaming, only rewrites archives that actually contain backslashes, and falls back to the original package on any failure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Build all Linux language extensions with GCC 13 against a RHEL 9 / glibc 2.34 baseline, and fix the runtime/test issues exposed by end-to-end qualification.
This change:
-fshort-wchar;Why
The Linux package must run on Ubuntu 22.04/24.04, RHEL 9/10, and Azure Linux 3 while using one compiler major version for all native extensions. Building on newer distro baselines introduced GLIBC/GLIBCXX requirements unavailable on RHEL 9. GCC 11 also cannot consume the fixed-underlying-type enum declarations exposed by the R 4.5 toolchain.
The selected baseline is UBI 9.6 with GCC 13. This preserves the required compiler while targeting glibc 2.34.
Validation
Validated through the private
Data-SQL-Language-Extensionsbranch pinned to this source:173434936: all required Linux language builds, tests, ELF gates, packaging, and automatic publication succeeded.data-sql-language-extensions-linuxversion1.0.0-PR-exthost-v3-gcc13-real-e2e-20260723.20.The PR is intentionally draft while downstream productization and final broad-board signoff are reviewed.