native: 64-bit multi-ABI + latest engine + OpenSSL 3, and bootstrap CI#5
Merged
Conversation
Ports the JNI native build to 64-bit and the current engine. Validated locally
with NDK r26d (Clang 17): libiconv.so, libhttrack.so (OpenSSL 3.0.15 statically
linked), libhtslibjni.so all build for arm64-v8a AND armeabi-v7a.
Android.mk:
- per-ABI prebuilt OpenSSL (../prebuild/$(TARGET_ARCH_ABI)/lib{ssl,crypto}.a) +
per-ABI openssl include dir
- drop hardcoded TARGET_ARCH := arm
- gate armv7-only -Wl,--no-merge-exidx-entries behind armeabi-v7a (it breaks the
arm64/x86_64 link)
- libiconv-1.15 -> libiconv-1.17
- add new engine library sources (htssniff.c, hts*_selftest.c) — required by the
3.49.12 engine; drop the vestigial libhtsjava module (engine removed htsjava.c)
Application.mk: APP_ABI = arm64-v8a armeabi-v7a x86_64; APP_PLATFORM android-24
(bionic introduces minizip's fseeko/ftello at API 24 — min-API decision noted).
config.h: SIZEOF_LONG/SIZEOF_LONG_LONG compiler-derived (was hardcoded 4 — wrong
on 64-bit ABIs).
HTTrackLib.java: drop 'htsjava' from the native load list.
Submodule httrack -> 627ae2b (3.49.12), which already selects TLS_client_method()
for OpenSSL 3.x. Requires 'git submodule update --init --recursive' (coucal is
nested). x86_64 is deferred pending a coffeecatch x86_64 ucontext fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Greenfield CI (repo had no .github). Matches the httrack repo conventions (contents: read, concurrency cancel, checkout@v6 + submodules recursive, CodeQL security-extended). - docker/: reproducible toolchain image (JDK17 + SDK35 + NDK r26d + OpenSSL 3.x statics for all ABIs). wget (curl's threaded resolver fails on some Docker hosts); build-essential for libiconv ./configure. - tools/ci/vendor-libiconv.sh: fetch + configure libiconv 1.17, patch the host-configured HAVE_LANGINFO_CODESET (bionic nl_langinfo is API 26+). - tools/ci/fetch-openssl-statics.sh: stage per-ABI OpenSSL into prebuild/. - .github/workflows/docker-image.yml: build+push the image to GHCR (+ statics artifact). - .github/workflows/android.yml: 'native' ndk-build job (arm64-v8a + armeabi-v7a; x86_64 pending coffeecatch fix). assemble/emulator jobs documented for later phases. - .github/workflows/codeql.yml + codeql-config.yml: java-kotlin + c-cpp, c-cpp scoped to htslibjni*.c (engine scanned by its own repo). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The container-based jobs failed with 'manifest unknown' — the GHCR toolchain image can't be bootstrapped from a PR branch (workflow_dispatch needs the workflow on the default branch). Rework native + codeql to set up the NDK, cross-compile OpenSSL (cached), vendor libiconv, and ndk-build inline — the flow validated locally. java-kotlin CodeQL uses build-mode: none (the app doesn't build under modern Gradle until the AGP 8 migration). docker/ image + docker-image.yml stay for the later Gradle/assemble phase. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
…sis)
codeql-config.yml's `paths` applies to every language, so restricting it to the
htslibjni.* C files left the java-kotlin analysis with zero source ('no source
code seen'). Add app/src/main/java to paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The first run downloaded a clean NDK and built; the next run restored an incomplete NDK from local-cache and failed with 'clang: No such file or directory'. Download the NDK fresh each run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 12, 2026
xroche
added a commit
that referenced
this pull request
Jul 16, 2026
…ed05faa, drop armeabi-v7a) (#7) Finishes the native foundation that Phase A (#5) left half-done. The coffeecatch bump to 9a0bdce carries the x86_64 ucontext fix (xroche/coffeecatch#46) and the i386 redefinition fix against the modern NDK sysroot. Without it the x86_64 ABI does not compile, which is why x86_64 was declared in `Application.mk` but built by neither the committed pin nor CI. The engine moves from 627ae2b to engine master (ed05faa). That range fixes a 32-bit chunked-transfer overflow (#593), files of 2GB and over (#568), an `hts_free_opt` leak on every crawl (#585), an unbounded filter-matcher recursion (#590/#575), and `--update` destroying a good mirrored copy (#584/#560), and it adds scheme-aware proxy parsing with SOCKS5 (#566/#570), all in sources Android compiles. Two new modules come with it, `htscodec.c` and `htsproxy.c`, now in the build; brotli and zstd stay compile-off, so `htscodec` falls back to zlib and pulls no new dependencies. The JNI surface is untouched. The ABI set is now arm64-v8a plus x86_64. Dropping 32-bit armeabi-v7a is safe: Play allows 64-bit-only builds, and minSdk 24 already excludes almost all 32-bit-only hardware. That leaves the armv7 exidx-merge gating dead, so it goes, and `Application.mk`, the CI ABI matrix and the Gradle `abiFilters` now agree on the same two ABIs. Part of #4. Signed-off-by: Xavier Roche <roche@httrack.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bootstraps a 64-bit, modern-NDK native build for httrack-android on the latest httrack engine (3.49.12) with OpenSSL 3.x, and adds GitHub Actions CI (the repo had none). This is the first slice of a larger modernization and touches only the native build and CI, not the Java/Gradle app, which comes in later phases.
The app shipped a single 32-bit ABI (armeabi-v7a), a 2017 engine snapshot, and a hand-hacked OpenSSL 1.0.2k build (setenv-android.sh, GCC, FIPS). 64-bit is a Play requirement, OpenSSL 1.0.2 is long past end-of-life, and the current engine already picks
TLS_client_method()for OpenSSL 3.x, so bumping it drops the whole hack in favor of a clean OpenSSL 3 build.Native changes: Android.mk moves to per-ABI prebuilt OpenSSL and a per-ABI openssl include, drops the hardcoded
TARGET_ARCH := arm, gates the armv7-only-Wl,--no-merge-exidx-entries(it breaks the arm64/x86_64 link), moves libiconv 1.15 to 1.17, adds the engine sources the current tree needs (htssniff.c, hts*_selftest.c) and drops the vestigial libhtsjava module since the engine removed htsjava.c. Application.mk setsAPP_ABIto arm64-v8a, armeabi-v7a, x86_64 at android-24. config.h derivesSIZEOF_LONGfrom the compiler (it was hardcoded to 4, wrong on 64-bit). HTTrackLib drops htsjava from the load list. The httrack submodule moves to 3.49.12, sogit submodule update --init --recursiveis required (coucal is nested).CI: a reproducible toolchain (docker/ image plus tools/ci vendor and fetch scripts) and three workflows: a self-contained native ndk-build job, a GHCR image publish, and dual-language CodeQL scoped to the JNI glue.
Built locally with NDK r26d: libiconv.so, libhttrack.so (with OpenSSL 3.0.15 statically linked), and libhtslibjni.so all build for arm64-v8a and armeabi-v7a.
Follow-ups: minSdk is 24 because minizip's fseeko/ftello are bionic API 24+; x86_64 is deferred until the coffeecatch x86_64 fix lands (xroche/coffeecatch#46).