Android: bspatch integration - #40
Conversation
18576fe to
6e1695c
Compare
6e1695c to
a53c022
Compare
There was a problem hiding this comment.
Pull request overview
Introduces Android-native BSDIFF40 patch application using vendored HDiffPatch and bzip2.
Changes:
- Adds Kotlin/JNI/C patching APIs and native build configuration.
- Vendors trimmed HDiffPatch and bzip2 sources.
- Adds instrumented tests, fixtures, packaging exclusions, and CI execution.
Reviewed changes
Copilot reviewed 38 out of 45 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/ci-test.yml |
Runs Android instrumented tests. |
.gitignore |
Ignores native build cache. |
.npmignore |
Excludes tests and build artifacts. |
android/app/build.gradle |
Configures CMake and instrumentation. |
android/app/src/main/java/com/microsoft/codepush/react/diffpatch/DiffPatch.kt |
Exposes the Kotlin patch API. |
android/app/src/main/cpp/CMakeLists.txt |
Builds the native shared library. |
android/app/src/main/cpp/bspatch_bridge.c |
Implements file-based patching. |
android/app/src/main/cpp/bspatch_bridge.h |
Declares bridge results and API. |
android/app/src/main/cpp/bzip2_error_stub.c |
Supplies bzip2 error handling. |
android/app/src/main/cpp/hdiffpatch_jni.c |
Connects JNI to the C bridge. |
android/app/src/main/cpp/third_party/README.md |
Documents vendored dependencies. |
android/app/src/main/cpp/third_party/hdiffpatch/LICENSE |
Adds HDiffPatch licensing. |
android/app/src/main/cpp/third_party/hdiffpatch/decompress_plugin_demo.h |
Provides bzip2 decompression integration. |
android/app/src/main/cpp/third_party/hdiffpatch/file_for_patch.c |
Provides file stream operations. |
android/app/src/main/cpp/third_party/hdiffpatch/file_for_patch.h |
Declares file stream operations. |
android/app/src/main/cpp/third_party/hdiffpatch/bsdiff_wrapper/bspatch_wrapper.c |
Implements BSDIFF patch decoding. |
android/app/src/main/cpp/third_party/hdiffpatch/bsdiff_wrapper/bspatch_wrapper.h |
Declares BSDIFF patch APIs. |
android/app/src/main/cpp/third_party/hdiffpatch/dirDiffPatch/dir_patch/dir_patch_types.h |
Supplies shared path definitions. |
android/app/src/main/cpp/third_party/hdiffpatch/libHDiffPatch/HPatch/checksum_plugin.h |
Defines checksum plugin types. |
android/app/src/main/cpp/third_party/hdiffpatch/libHDiffPatch/HPatch/hpatch_mt/hpatch_mt.h |
Defines HDiffPatch threading interfaces. |
android/app/src/main/cpp/third_party/hdiffpatch/libHDiffPatch/HPatch/patch.c |
Provides HDiffPatch internals. |
android/app/src/main/cpp/third_party/hdiffpatch/libHDiffPatch/HPatch/patch.h |
Declares patch operations. |
android/app/src/main/cpp/third_party/hdiffpatch/libHDiffPatch/HPatch/patch_private.h |
Declares internal patch helpers. |
android/app/src/main/cpp/third_party/hdiffpatch/libHDiffPatch/HPatch/patch_types.h |
Defines patch stream types. |
android/app/src/main/cpp/third_party/bzip2/LICENSE |
Adds bzip2 licensing. |
android/app/src/main/cpp/third_party/bzip2/bzlib.c |
Provides trimmed bzip2 runtime. |
android/app/src/main/cpp/third_party/bzip2/bzlib.h |
Declares bzip2 APIs. |
android/app/src/main/cpp/third_party/bzip2/bzlib_private.h |
Defines bzip2 internals. |
android/app/src/main/cpp/third_party/bzip2/crctable.c |
Supplies bzip2 CRC data. |
android/app/src/main/cpp/third_party/bzip2/decompress.c |
Implements bzip2 decompression. |
android/app/src/main/cpp/third_party/bzip2/huffman.c |
Implements Huffman decoding helpers. |
android/app/src/main/cpp/third_party/bzip2/randtable.c |
Supplies bzip2 randomization data. |
android/app/src/androidTest/java/com/microsoft/codepush/react/diffpatch/DiffPatchInstrumentedTest.kt |
Tests patching through JNI. |
android/app/src/androidTest/assets/basic/old.dat |
Basic base-file fixture. |
android/app/src/androidTest/assets/basic/new.dat |
Basic expected-output fixture. |
android/app/src/androidTest/assets/basic/patch.bsdiff |
Basic patch fixture. |
android/app/src/androidTest/assets/identical/old.dat |
Identical-file base fixture. |
android/app/src/androidTest/assets/identical/new.dat |
Identical expected output. |
android/app/src/androidTest/assets/identical/patch.bsdiff |
Identical-file patch fixture. |
android/app/src/androidTest/assets/empty_old/old.dat |
Empty base fixture. |
android/app/src/androidTest/assets/empty_old/new.dat |
Empty-base expected output. |
android/app/src/androidTest/assets/empty_old/patch.bsdiff |
Empty-base patch fixture. |
android/app/src/androidTest/assets/bad_header/old.dat |
Invalid-header base fixture. |
android/app/src/androidTest/assets/bad_header/patch.bsdiff |
Invalid-header patch fixture. |
android/app/src/androidTest/assets/wrong_old/old.dat |
Mismatched-base fixture. |
Suppressed comments (1)
android/app/src/main/cpp/bspatch_bridge.c:84
- This removal also runs for
OPEN_OLD,OPEN_DIFF, andBAD_DIFF_HEADER, although those paths return before this invocation opens the output. Consequently, an unrelated input error deletes any pre-existing destination (andremove()can even delete an empty directory). Track whether this call successfully opened/created the output and only remove a file owned by this patch attempt; alternatively stage into a temporary file and replace the destination only on success.
if (result != CODEPUSH_BSPATCH_OK) {
// Don't leave a corrupt, partially-written "new bundle" on disk.
// It's a no-op when outNewFilePath was never created (e.g. ERR_OPEN_OLD)
hpatch_removeFile(outNewFilePath);
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| // Why open the output only at this point: maxLength isn't known until the diff header above has been parsed. | ||
| if (!hpatch_TFileStreamOutput_open(&outStream, outNewFilePath, diffInfo.newDataSize)) | ||
| return CODEPUSH_BSPATCH_ERR_OPEN_OUT; |
There was a problem hiding this comment.
I think this is overstated. It's more like a "user error" to call applyPatch() with identical input and output file paths from the perspective of this bridge code. And even if it's the case, nothing irreversible happens, CodePushUpdateUtils does a full copy of the bundle dir before patching is done, so the only thing to get corrupted is this copy of the previous bundle. We also have the rollback mechanism (if the bundle is not loadable for any reason).
a53c022 to
6e38001
Compare
6e38001 to
dc6d2b3
Compare
Why
Foundations and first steps of the binary diffing feature's SDK-side implementation.
When binary diffing is enabled server-side, the delta update ZIP contains per-file binary diffs, which are applied on top of the base files (from prev. release).
Our choice of binary diffing is the BSDIFF40 format, but decoded using HDiffPatch instead of the original
bsdiff/bspatchtools.What
The goal of this PR is to integrate
HDiffPatchinto the project and expose the following entrypoint for native Android code:This involves the following:
HDiffPatchandbzip2(BSDIFF40 format uses bzip2-compressed content blocks)HDiffPatchand itsbspatch_with_cache()entrypoint ergonomic to use (see decisions section and code comments for more details)Out of scope
applyPatch()function -> next few PRsDecisions
Raw JNI vs. fbjni: this is a React Native module, so fbjni wouldn't be an unreasonable thing, but the JNI interface at this moment is a single function with 3 parameters and without any exception throwing code, so I don't think the extra complexity of
fbjniis justified in this case. We can revisit this decision any time though.Instrumented tests vs more focused unit tests: there are too many brittle layers (JNI, bridge code), so I think it's worth testing the Java/Kotlin side that exercises all those layers. The specific business logic that's going to call
applyPatch()can be a JVM unit test later, but the code in this PR is worth testing independently IMHO.Vendoring vs git submodules: vendoring is simpler, and we don't expect these libs to be updated often. The process is documented at third_party/README.md so it can be repeated when we want to update the two libraries.
Exclude bzip2's compress-side code: in order to minimize library size, we make some minor tweaks to the vendored
bzip2source. Even though we only need the decompress side API (BZ2_bzDecompress()), the decompression code is interwined with compression code and originally forced us to build and link files such ascompress.candblocksort.c. See third_party/README.md for details, but the TL;DR is this library size reduction:arm64-v8a: 118,528 B → 79,792 BFile paths vs streams: HDiffPatch has its own stream types (
hpatch_TStreamInput/hpatch_TStreamOutput) for the input and output params, but utilizing them would have meant exposing them through JNI and writing the bridge between those types and Java streams. I also don't think it's a worthwhile optimization (at this point): by the time this new codepath is called, both the base file and the patch are already written to disk, there is no in-memory data streaming.HDiffPatch bridge code written in C, not C++: My initial attempt was actually C++ with nice RAAI wrapper classes for HDiffPatch types, but this bloated the shared library to unacceptable levels (119 KB → 356 KB). Most of the increase came from
libc++abi's exception unwinding mechanism. We could compile with-fno-exceptions -fno-rtti, but this felt a bit hacky (I might be wrong!). The JNI glue can be written in C, and the bridge code rewritten in C doesn't look that horrible either. Again, we can revisit this decision any time in the future, this is not part of the public API.