Skip to content

build: link bundled-dep archives / fall back to source when prebuilt is incomplete#14

Open
jonasvanderhaegen wants to merge 1 commit into
LadybugDB:mainfrom
skylence-be:pr/yyjson-link
Open

build: link bundled-dep archives / fall back to source when prebuilt is incomplete#14
jonasvanderhaegen wants to merge 1 commit into
LadybugDB:mainfrom
skylence-be:pr/yyjson-link

Conversation

@jonasvanderhaegen
Copy link
Copy Markdown

@jonasvanderhaegen jonasvanderhaegen commented May 24, 2026

Problem

Building against the prebuilt liblbug.a (the default fast path in build.rs) can fail at the final link on macOS and Linux:

Undefined symbols for architecture arm64:
  "_yyjson_val_mut_copy", referenced from: lbug::json_extension::... in liblbug.a(json_utils.cpp.o)
ld: symbol(s) not found

This happens when the prebuilt liblbug.a is a thin archive — it contains objects (e.g. json_utils.cpp.o) that reference bundled third-party libraries (yyjson, simsimd, brotli, lz4, mbedtls, …) but does not fold them in, and build.rs emits no link directives for them on the prebuilt path.

Fix

link_prebuilt_bundled_deps() emits cargo:rustc-link-lib=static={lib} for each bundled dep archive shipped alongside the prebuilt liblbug.a. Guarded by exists(), so it's a no-op when an archive isn't present.

Notes / refinement since first opening

While integrating this in a downstream consumer we found two things worth flagging:

  1. Use regular static linking, not +whole-archive, for the dep archives. main now bundles the third-party archives into liblbug.a via cmake/BundleStaticLibrary.cmake (a fat archive). If a consumer links both the fat liblbug.a and the individual deps/*.a with +whole-archive, the link fails with duplicate symbols on macOS/Linux. Plain static={lib} resolves the references lazily and is a no-op against a fat archive, so it works for both thin and fat layouts.

  2. The undefined-symbols problem is largely moot on main now that BundleStaticLibrary produces a self-contained liblbug.a. This PR is primarily useful for consumers of older/thin prebuilts or alternate packaging that ships deps/ separately.

Verified on macOS arm64 (consuming a thin prebuilt + deps/): indexing (COPY + HNSW + checkpoint) and vector search both build and run cleanly.

The prebuilt liblbug.a has unresolved external references to yyjson,
simsimd and several other third-party libraries.  These archives are
shipped alongside liblbug.a in the prebuilt cache directory, but the
build script never told the linker about them.

Add link_prebuilt_bundled_deps() which emits cargo:rustc-link-lib
directives for every *.a found in the prebuilt lib dir, and call it
from use_prebuilt_lbug().  The function is existence-gated so it
is a no-op when a fully fused archive is used.

Fixes "Undefined symbols: _yyjson_val_mut_copy, _yyjson_write_opts ..."
link errors on macOS and equivalent failures on Linux when the crate
is consumed via [patch.crates-io] or crates.io download.
@adsharma
Copy link
Copy Markdown
Contributor

This will be fixed when 0.17.0 is out. For now, use

LBUG_PRECOMPILED_RUN_ID= 26256190740 when you cargo build

CI picks up the most recent ladybug nightly build which has this fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants