Skip to content

Commit ffe60ed

Browse files
authored
feat(compat): a farm answers for its own members, and reaches the host only through a named package (#380)
The membership of these three farms is decided by a filename pattern, and their completeness -- where it was checked at all -- was decided against a different set: the libraries an ICD manifest names. The pattern exists because a proprietary driver dlopens members of its own family by name, which no DT_NEEDED walk can see; having said that, those members have to be treated as reachable everywhere else too, and they were not. The half of each farm the pattern exists for is the half nothing verified. Measured on a host with an NVIDIA driver, by mcpp's dlopen-surface check: compat.glx-runtime 52 members / 30 unmet closure edges (it had no closure pass at all), compat.vulkan-runtime 76 / 5, compat.opencl-runtime / 4 -- the last being the findings filed as #376, which names vulkan while the observed four came from opencl. THE REASON RECORDED AGAINST SEEDING FROM THE FARM MEASURED A DIFFERENT SET. It says the closure pulled 64 libraries with GTK among them; that holds for closing over every file the PATTERN matched, since `libnvidia*.so.*` also matches the driver's settings GUI. The farm is the pattern's matches MINUS never_farm_patterns, and closing over that adds five sonames here and no GTK, GLib, Pango or Cairo. Every soname a member needs now has a written answer and there is no silent branch: * the ecosystem publishes it -> declared in xpm.linux.deps and taken from the installed payload (xim:openssl, xim:mesa) * proprietary vendor user-space -> also a package: xim:nvidia-video-host-link owns "where is the host's libnvcuvid.so.1", the shape libcuda-host-link and nvidia-gl-host-link already have * neither, and it cannot become one -> named in UNSERVED with the reason (libcrypto.so.1.1: OpenSSL 1.1 is EOL upstream and only NVIDIA's PKCS#11 provider asks for it) * anything else -> a warning naming it at install time No branch harvests a new file from /usr/lib for a farmed member. compat.glx-runtime harvests none at all: all thirty come from installed payloads, four from xim:nvidia-gl-host-link and twenty-six from the stack xim:graphics pulls in. The criterion is asked the way mcpp asks it: a member's DT_NEEDED is read with `readelf -d` and membership is decided against this directory alone. `ldd` was wrong for it -- it answers "can this resolve HERE", and here includes the host's default directories, so a soname the host happens to carry read as resolved and was never recorded while the consumer, whose search path is the farm, could not load it. Measured: the libcrypto needs were invisible to that pass and were reported by mcpp one layer up from the same directory. Measured after, on the opencl farm: members 48, walked 47, missing 0, with libcrypto.so.1.1 the one remaining finding and correctly classified dangling. Four missing before. New version keys because install() output is baked into the installed payload: without one, a host that already holds the previous version keeps the open farm. compat.glfw, compat.opencl and compat.vulkan move their pins with it. Depends on openxlings/xim-pkgindex#801, which publishes the sentinel this declares.
1 parent c237104 commit ffe60ed

7 files changed

Lines changed: 646 additions & 10 deletions

CHANGELOG.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,58 @@
77

88
## [Unreleased]
99

10+
### Changed
11+
12+
- **三个 host farm 改为对自己的成员负责,并且对宿主只保留一条具名的触达**
13+
(`compat.glx-runtime` 2026.09.10、`compat.opencl-runtime` 2026.09.10、
14+
`compat.vulkan-runtime` 2026.09.10)。
15+
16+
这三个包的成员由**文件名模式**决定,而完整性(如果检查的话)对着的是**另一个
17+
集合** —— ICD 清单里的库。模式存在的理由恰恰是「专有驱动会按名字 dlopen 自己
18+
家族的成员,没有任何 `DT_NEEDED` 遍历看得见」;既然这么说了,那些成员在别处也
19+
得当作可达的 —— 而它们不是。**从来没被验证的,正是模式为之存在的那一半。**
20+
21+
实测(装了 NVIDIA 驱动的宿主,用 mcpp 自己的 dlopen 面检查):
22+
`compat.glx-runtime` 52 个成员缺 30 条闭包边(它根本没有闭包遍历)、
23+
`compat.vulkan-runtime` 76 个缺 5 条、`compat.opencl-runtime` 缺 4 条 ——
24+
最后这 4 条就是 mcpp-index#376 报的那几条(该 issue 标题写的是 vulkan,
25+
实际观测到的来自 opencl)。
26+
27+
⚠️ **配方里「种子不能取整个 farm」的理由测的是排除之前的集合。** 它写着闭包会
28+
拉进 64 个库、GTK 在其中 —— 那是对「模式匹配到的每个文件」成立,因为
29+
`libnvidia*.so.*` 也匹配驱动的设置界面。farm 是模式匹配**减去**
30+
`never_farm_patterns`,对**那个**求闭包在本机只新增 5 个,GTK/GLib/Pango/Cairo
31+
一个都没有。**结论会被复查,理由不会。**
32+
33+
⭐ 新规则:一个成员需要的每条 soname 都有**明写的答案**,没有静默分支。
34+
1. 生态发布了它 —— 在 `xpm.linux.deps` 里声明,从已装载荷取
35+
(新增 `xim:openssl``xim:mesa`);
36+
2. 不可再分发的专有驱动用户态 —— 也走包:新增
37+
`xim:nvidia-video-host-link` 拥有「宿主的 `libnvcuvid.so.1` 在哪」这一个
38+
问题,和 `libcuda-host-link`/`nvidia-gl-host-link` 同形;
39+
3. 两者都不是、也不可能成为 —— 写进配方的 `UNSERVED`**并附理由**
40+
(今天只有 `libcrypto.so.1.1`:OpenSSL 1.1 上游已 EOL,而只有 NVIDIA 的
41+
PKCS#11 提供者要它,没有任何 OpenCL/Vulkan 入口够得到);
42+
4. 以上都不是 —— 安装时**打警告点名**。这一条是关键:farm 从生态之外拿了什么,
43+
必须是有人写下来的清单,而不是残留物。
44+
45+
**没有任何一条分支会为成员去 /usr/lib 收一个新文件。** `compat.glx-runtime`
46+
一条都不收:那 30 条全部来自已装载荷(4 条 `libnvidia-*` 来自
47+
`xim:nvidia-gl-host-link`,其余 26 条来自 `xim:graphics` 拉起来的栈)。
48+
49+
⭐ 判据落在 mcpp 会问的那个问题上:成员的 `DT_NEEDED``readelf -d` 读,
50+
归属只对着 farm 这一个目录判。**先前用 `ldd` 是错的** —— 它答的是「在这台机器
51+
上能不能解析」,而这台机器包含宿主默认目录,于是宿主碰巧有的 soname 读成已解析、
52+
从不被记录,而消费者的搜索路径里没有宿主。实测:`libcrypto` 那两条对这一趟不可见,
53+
却被 mcpp 在上一层从同一个目录报了出来。
54+
55+
实测结果(本机,opencl farm):`members 48 / walked 47 / missing 0`,
56+
只剩 `libcrypto.so.1.1` 一条 dangling —— 正是明写为 unserved 的那条。
57+
改动前是 4 条 missing。
58+
59+
⚠️ 抬版本键是必须的:`install()` 的产物烤进已安装载荷,不换键的机器会一直留着
60+
未闭合的 farm。消费者 `compat.glfw` / `compat.opencl` / `compat.vulkan` 同步重钉。
61+
1062
### Added
1163

1264
- 收录 `compat.sdl3` 3.4.2 —— SDL3 窗口/输入/音频层,从源码构建(形态 E)。

pkgs/c/compat.glfw.lua

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ package = {
8080
"src/posix_module.c",
8181
},
8282
deps = {
83-
["compat.glx-runtime"] = "2026.08.08",
83+
["compat.glx-runtime"] = "2026.09.10",
8484
["compat.x11"] = "1.8.13",
8585
["compat.xcursor"] = "1.2.3",
8686
["compat.xext"] = "1.3.7",

pkgs/c/compat.glx-runtime.lua

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,24 @@ package = {
4545
-- a download it will not read, and the alternative -- deleting the
4646
-- published version -- would break them outright.
4747
deps = { runtime = { "xim:graphics" } },
48+
-- 2026.09.10: the farm answers for what its own members need.
49+
-- Nothing new is taken from the host: a soname a member needs is
50+
-- filled from an installed payload, and otherwise recorded as a
51+
-- dangling link. A new key because install() output is baked into
52+
-- the installed payload -- without one, a host that already holds
53+
-- 2026.08.08 keeps a farm 30 sonames short of closed.
54+
--
55+
-- The anchor is 2026.08.08's, deliberately: the URL is a
56+
-- well-formedness anchor and nothing more, and reusing it means no
57+
-- new mirror asset has to exist for a key that changes only
58+
-- install() behaviour.
59+
["2026.09.10"] = {
60+
url = {
61+
GLOBAL = "https://raw.githubusercontent.com/KhronosGroup/OpenGL-Registry/a30033d3e812c9bf10094f1010374a6b15e192eb/README.adoc",
62+
CN = "https://gitcode.com/mcpp-res/glx-runtime/releases/download/2026.08.08/glx-runtime-2026.08.08.adoc",
63+
},
64+
sha256 = "ea68efce197e68413ebb62c51ab4bccfb2309a2fca776d31b49d972f59f3640e",
65+
},
4866
["2026.08.08"] = {
4967
url = {
5068
GLOBAL = "https://raw.githubusercontent.com/KhronosGroup/OpenGL-Registry/a30033d3e812c9bf10094f1010374a6b15e192eb/README.adoc",
@@ -174,6 +192,160 @@ local required = {
174192
["libGL.so.1"] = false,
175193
}
176194

195+
-- WHAT THE FARM'S OWN MEMBERS NEED, AND WHY NOTHING NEW COMES FROM THE HOST.
196+
--
197+
-- The pattern list above decides membership; nothing decided completeness.
198+
-- Measured 2026-09-10 on a host with the proprietary driver: 52 members, and
199+
-- 30 sonames those members need that this directory does not carry --
200+
-- `libX11.so.6`, `libxcb*.so.*`, `libz.so.1`, `libLLVM.so.20.1`,
201+
-- `libnvidia-glcore.so.*` and `libstdc++.so.6` among them. A consumer reaching
202+
-- any of those members through this farm gets a load failure that no closure
203+
-- check can see, because no link edge names it.
204+
--
205+
-- THE HOST SURFACE IS HELD AT WHAT THE PATTERN LIST ALREADY TAKES. Two of the
206+
-- thirty say why: `libstdc++.so.6`, and behind it a host C++ runtime on the
207+
-- RUNPATH of every GL consumer this index has. mcpp links libstdc++ statically,
208+
-- and a second one arriving through a farm is the failure class the libc guard
209+
-- below already exists for. So the rule is:
210+
--
211+
-- * an installed payload provides it -> link the payload's copy
212+
-- * nothing does -> named in UNSERVED with the reason,
213+
-- and linked into a directory this
214+
-- package never creates
215+
-- * anything else -> a warning naming it
216+
--
217+
-- and there is no branch at all that harvests a file from /usr/lib. Measured
218+
-- 2026-09-10 on a host with the proprietary driver: ALL THIRTY come from
219+
-- installed payloads. The four `libnvidia-*` ones come from
220+
-- `xim:nvidia-gl-host-link`, which is where this file already says the driver
221+
-- reaches the subos from; the other twenty-six come from the stack
222+
-- `xim:graphics` pulls in -- `xim:libX11`, `xim:libxcb`, `xim:mesa`,
223+
-- `xim:libllvm`, `xim:gcc-runtime` and the rest. The host surface of this farm
224+
-- is therefore exactly what the pattern list takes, and nothing more.
225+
--
226+
-- The dangling branch is not a workaround. mcpp reads this directory with a
227+
-- three-state rule -- resolved, present-but-dangling (this machine has no such
228+
-- library), absent everywhere (the publisher never carried it) -- and DANGLING
229+
-- IS ONLY EXPRESSIBLE IF THIS PACKAGE MADE A LINK. Leaving the soname out
230+
-- reports "the publisher never considered it" on every machine, including the
231+
-- ones where the truth is "this host has no X11". The link is also
232+
-- self-healing, the shape `xim:libcuda-host-link` already uses: it resolves the
233+
-- moment the machine gains the library.
234+
local never_farm = {
235+
["libc.so.6"] = true, ["libm.so.6"] = true, ["libdl.so.2"] = true,
236+
["libpthread.so.0"] = true, ["librt.so.1"] = true, ["libresolv.so.2"] = true,
237+
["ld-linux-x86-64.so.2"] = true, ["ld-linux-aarch64.so.1"] = true,
238+
["libgcc_s.so.1"] = true,
239+
}
240+
241+
local function xim_store_roots()
242+
local roots = {}
243+
local home = os.getenv("XLINGS_HOME")
244+
if home and home ~= "" then roots[#roots + 1] = path.join(home, "data/xpkgs") end
245+
local pfx = pkginfo.install_dir()
246+
if pfx then roots[#roots + 1] = path.directory(path.directory(pfx)) end
247+
return roots
248+
end
249+
250+
-- Every copy, not the last one sorted: a driver payload can ship its own copy
251+
-- of a library another package owns. compat.vulkan-runtime carries the
252+
-- measurement behind this.
253+
local function find_in_store(soname)
254+
local out, seen = {}, {}
255+
for _, root in ipairs(xim_store_roots()) do
256+
local f = io.popen(string.format(
257+
[[ls -1 "%s"/xim-x-*/*/lib/%s "%s"/xim-x-*/*/lib64/%s 2>/dev/null | sort -V]],
258+
root, soname, root, soname))
259+
if f then
260+
for line in f:lines() do
261+
local hit = line:gsub("[\r\n]+$", "")
262+
if hit ~= "" and not seen[hit] then
263+
seen[hit] = true
264+
out[#out + 1] = hit
265+
end
266+
end
267+
f:close()
268+
end
269+
end
270+
return out
271+
end
272+
273+
-- WHAT A MEMBER NEEDS IS READ FROM THE MEMBER, NOT FROM A LOADER.
274+
--
275+
-- `ldd` answers "can this resolve HERE", and here includes the host's default
276+
-- directories -- so a soname the host happens to carry reads as resolved and is
277+
-- never recorded, while the consumer, whose search path is this farm and not
278+
-- the host, cannot load it. `readelf -d` answers what the FILE says, and
279+
-- membership is decided against this directory alone, which is the question
280+
-- mcpp asks of it.
281+
local function find_tool(name)
282+
local f = io.popen(string.format([[command -v %s 2>/dev/null]], name))
283+
if f then
284+
local hit = (f:read("l") or ""):gsub("[\r\n]+$", "")
285+
f:close()
286+
if hit ~= "" then return hit end
287+
end
288+
return nil
289+
end
290+
291+
local function unresolved_against_farm(outdir)
292+
local readelf = find_tool("readelf")
293+
if not readelf then return {} end
294+
local have, members = {}, {}
295+
local lsf = io.popen(string.format([[ls -1 "%s" 2>/dev/null]], outdir))
296+
if not lsf then return {} end
297+
for line in lsf:lines() do
298+
local b = line:gsub("[\r\n]+$", "")
299+
if b ~= "" then have[b] = true; members[#members + 1] = b end
300+
end
301+
lsf:close()
302+
local out, seen = {}, {}
303+
for _, base in ipairs(members) do
304+
local f = io.popen(string.format(
305+
[[%s -d %s 2>/dev/null | sed -n 's/.*(NEEDED).*\[\(.*\)\]/\1/p']],
306+
sh_quote(readelf), sh_quote(path.join(outdir, base))))
307+
if f then
308+
for line in f:lines() do
309+
local n = line:gsub("[\r\n]+$", "")
310+
if n ~= "" and not have[n] and not never_farm[n] and not seen[n] then
311+
seen[n] = true
312+
out[#out + 1] = n
313+
end
314+
end
315+
f:close()
316+
end
317+
end
318+
return out
319+
end
320+
321+
-- Nothing is here today, and the table exists so that the day something is,
322+
-- somebody has to write down why it cannot be a package. The warning below is
323+
-- what makes leaving it blank impossible to do by accident.
324+
local UNSERVED = {}
325+
326+
local function close_farm(outdir)
327+
local filled, unserved, undeclared = {}, {}, {}
328+
local unserved_dir = path.join(path.directory(outdir), "unserved")
329+
for _, soname in ipairs(unresolved_against_farm(outdir)) do
330+
local candidates = find_in_store(soname)
331+
local hit = candidates[#candidates]
332+
if hit then
333+
os.exec(string.format([[ln -sf "%s" "%s"]], hit, path.join(outdir, soname)))
334+
filled[#filled + 1] = soname
335+
else
336+
-- Inside this package, not at the canonical host path: a link into
337+
-- /usr/lib resolves on any machine that happens to carry the file,
338+
-- which is a host harvest wearing a different name.
339+
os.exec(string.format([[ln -sf "%s" "%s"]],
340+
path.join(unserved_dir, soname),
341+
path.join(outdir, soname)))
342+
unserved[#unserved + 1] = soname
343+
if not UNSERVED[soname] then undeclared[#undeclared + 1] = soname end
344+
end
345+
end
346+
return filled, unserved, undeclared
347+
end
348+
177349
local function link_runtime_libs(outdir)
178350
os.mkdir(outdir)
179351
for _, dir in ipairs(candidate_dirs()) do
@@ -187,6 +359,27 @@ local function link_runtime_libs(outdir)
187359
end
188360
end
189361

362+
-- Completeness, before the guards. Neither branch puts a new host library
363+
-- on a consumer's path, so the guards below still see exactly what the
364+
-- pattern list matched.
365+
local filled, unserved, undeclared = close_farm(outdir)
366+
if #filled > 0 then
367+
log.info("compat.glx-runtime: %d libraries the farm's members need were "
368+
.. "filled from installed payloads", #filled)
369+
end
370+
if #unserved > 0 then
371+
log.info("compat.glx-runtime: %d sonames the farm's members need are "
372+
.. "published by no installed payload and are recorded as "
373+
.. "unserved", #unserved)
374+
end
375+
-- The list has to be written, not discovered.
376+
for _, soname in ipairs(undeclared) do
377+
log.warn("compat.glx-runtime: %s is needed by a farmed member, is "
378+
.. "published by no installed payload, and is not declared in "
379+
.. "UNSERVED. Add the ecosystem package that provides it, or "
380+
.. "record why it cannot be one.", soname)
381+
end
382+
190383
for name, _ in pairs(required) do
191384
if not os.isfile(path.join(outdir, name)) then
192385
log.error("%s is not in this subos. The GL runtime comes from "

0 commit comments

Comments
 (0)