Audit log: agent provenance — ventisrv content store + veltro trajectory emitters (INFR-355) - #508
Open
pdfinn wants to merge 6 commits into
Open
Audit log: agent provenance — ventisrv content store + veltro trajectory emitters (INFR-355)#508pdfinn wants to merge 6 commits into
pdfinn wants to merge 6 commits into
Conversation
The tree has shipped the full vac client stack (vacput/vacget/vacfs,
module/{vac,venti}.m, appl/lib/{vac,venti}.b) since the fork, and
man/4/vacfs referenced ventisrv(8) — but the server itself was never
in-tree. Salvage it from Mechiel Lukkien's ventisrv (GSoC 2007
ventivac project, public domain): a venti server with an in-memory
index over an append-only data file + regenerable index file.
Compiles unmodified against the in-tree venti.m (only Score/Scoresize/
Vmsg are used). Verified in emu: ventisrv -q config check, and a full
vacput round-trip against a live server (data file carries correct
block headers; index syncs).
This is the content-store substrate for the audit log's agent-
provenance extension (audit-log-design.md §8): bulky prompt/completion
payloads become write-once content-addressed blocks, referenced from
chain records by score.
Refs: INFR-355
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nance records The agent-provenance layer from audit-log-design.md §8, composed from what the tree already had: payloads go to ventisrv as vac hash trees (Datatype blocks under a packed entry sealed as a Dirtype block); the audit record carries 'content=<score> sha256=<hex> size=<n>'. The venti score is the locator — mechanism only; the SHA-256 is the integrity pin, sealed into auditfs's SHA-256 chain, so venti's SHA-1 addressing cannot be used to equivocate a payload (chosen-prefix collisions buy nothing against a pinned record). module/auditprov.m + appl/lib/auditprov.b — thin lib in the audit(2) style: attach() dials $auditventi (default tcp!127.0.0.1!17034); dialraw()/attachfd() split the dial from the handshake so a parent can hand the connection fd to a namespace-restricted child (the fd survives pctl(NEWNS), the child handshakes). put()/get() round-trip payloads; log() seals a record, degrading to content=unstored (rc -2) when the store is unreachable — the event still seals, the trail shows the gap. Session RPC is semaphore-serialized (venti Session.rpc is a bare write-then-read and veltro runs tool calls concurrently). appl/cmd/auditget.b — auditor-side fetch: score in, payload out, -s <sha256> pins it to what the chain sealed. tests/auditprov_test.b — live integration: real ventisrv + real auditfs (pipe/spawn/mount, no sh namespace split), 8 tests: sha256 vector, plain + payload records, unstored degradation, dedup, multi-block and empty payloads, fetch-back by record score. All pass. Refs: INFR-355 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wire the agent-provenance emitters (audit-log-design.md §8, INFR-355) through the whole agent stack: veltro.b (parent): agentstart/nscaps/sysprompt/task/plan at session setup, per-step prompt/llm/toolcall/toolres in the loop (tool results captured pre-truncation, full payloads), agentdone on every exit path. Fail-closed under Audit->ONFILE: an install that requires auditing refuses to run an agent whose actions cannot be sealed (the secstored authok posture). The content-store session is dialed before the parent's own restrictns, so auditing adds no network grant. tools/spawn.b: when the install audits, every child's caps auto-grant the append-only /mnt/audit/log (auditcontrolpath already keeps root/ chain/ctl ungrantable — placement does the rest); the parent seals spawn (task payload) + subcaps (serialized grant) per child, dials a store connection, and arms the child via the new SubAgent setprov() (mirrors setmcp: after restrictns, before runloop). subagent.b: completes the venti handshake inside the restricted namespace on the inherited fd, then seals substart (system prompt, tool list) / subtask / per-step subllm / subtool / subtoolres / subdone — the same funnel as the existing trajectory log, now tamper-evident. nsconstruct.b: restrictns now drives the previously-uncalled emitauditlog hook with the granted caps summary, placed just before step 10 hides /tmp/.veltro-ns; emitauditlog builds the manifest, writes it to AUDIT_DIR as before, and seals an nsrestrict record carrying its SHA-256 so the manifest cannot be quietly edited. lib/sh/profile: when auditing is enabled, boot also starts the provenance ventisrv on a persistent store under /usr/inferno/audit/ venti (localhost, auditprov default address; touch never truncates). All veltro/spawn test suites pass (16+33+12+12+33+12) plus the live auditprov integration test (8). Refs: INFR-355 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
audit-log-design.md §8 designed-in -> BUILT, recording the two design decisions the ticket left open: (1) the SHA-256 pin — venti scores are SHA-1 (collision-broken), so the score is locator only and the chain-sealed sha256= field pins the payload; (2) the confidentiality model — plaintext blocks by namespace placement (possession-of-score is the read capability, subjects never see the chain), with the dedup presence-oracle noted as the honest residual and encryption-at-rest left to deployment. §10 lists the shipped v2 pieces. SP800-92 AU-12 row + residual-gap table, SP800-53 AU-12, and the NIST control mappings updated: veltro provenance wired; CDS emitters follow the CDS guard itself. Roadmap Tier-1 item 2 updated likewise. New man pages: auditprov(2), auditget(1). Refs: INFR-355 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mnt/ ships .gitkeep placeholders but n/ was never tracked, so a fresh
checkout has no /n and the profile's 'mount -ac {mntgen} /n' fails
(silently — stderr is discarded). Surfaced by the INFR-355 worktree
smoke, which is exactly a fresh checkout.
Refs: INFR-355
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Member
Author
|
Live end-to-end evidence (headless emu, llmsrv over local Ollama Offline verification over that chain: |
…ventisrv The 64-bit port left all three casel consumers walking the historical 32-bit table shape. limbo's binary emitter (dis.c Tcasel) lays the table out as: count in a 2*IBY2WD slot, then per case [lo][hi][pc+pad] with IBY2LG bounds and the pc padded to 2*IBY2WD, then the default pc. With IBY2WD == IBY2LG == 8 that is a 4-WORD stride with the pc at l[2]; the code still assumed the 32-bit shape — 6 four-byte words per case, pc at l[4]: - xec.c OP(casel): read garbage bounds/destinations — the interpreter looped forever or jumped wild on the first `case big` executed. - comp-arm64.c / comp-amd64.c comcasel: patched the wrong slots at compile time, corrupting module data and indexing patch[] with garbage — an immediate SEGV in comp when loading any module containing `case big` under -c1. Nothing in the entire dis tree used `case big` until INFR-355 salvaged ventisrv(8), whose Dhdrmagic/Fhdrmagic dispatch is a case on big — so the instruction had been broken since the 64-bit migration with no symptom. It presented as ventisrv wedging (interp) or crashing emu (JIT) the moment it restarted on a populated store. Fix all three to the stride-4 layout. tests/casel_test.b keeps the instruction exercised (single labels, boundary probes, a 10k-iteration dispatch loop); 3/3 pass under both -c0 and -c1, and ventisrv now restarts cleanly on an existing data/index pair. The amd64 comcasel had the identical bug — worth noting for the x86-64 emu-corruption flake investigation (GoDis de-flake): any module with a casel would have corrupted its own module data at JIT load on those hosts too. Refs: INFR-355 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Member
Author
|
Bonus find while validating store persistence: restarting ventisrv on a populated index deterministically wedged the interpreter and crashed the JIT. Root cause was not ventisrv — it was the Dis With the fix in, the full persistence loop verifies end-to-end on a freshly built emu: ventisrv restarts on the existing data/index pair (index replay), and Note for the x86-64 emu-corruption investigation: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes INFR-355 (Audit log: AI-agent provenance extension — vac content-store layer).
What this is
The v2 "agent provenance" extension the audit log was designed for (
docs/compliance/audit-log-design.md§8): record what an agent was prompted with and what it did — system prompt, task, granted caps, every tool call/result, spawns, completion — in the tamper-evident/mnt/auditchain, with bulky payloads stored by reference in a write-once content store.The pieces
ventisrv(8)salvaged (appl/cmd/ventisrv.b, public domain, Mechiel Lukkien's GSoC-2007 ventivac work). The tree has shipped the entire vac client stack since the fork (vacput/vacget/vacfs,module/{vac,venti}.m) andman/4/vacfsalready referencedventisrv(8)— this adds the server. Compiles unmodified against the in-treeventi.m. Append-only data file + regenerable index; localhost.auditprov(2)(module/auditprov.m+appl/lib/auditprov.b): stores a payload as a vac hash tree and sealscontent=<score> sha256=<hex> size=<n>onto the record viaaudit(2). The venti score is the locator only — venti addressing is SHA-1, which is collision-broken, so the chain-sealed SHA-256 is what pins the payload (a chosen-prefix collision buys nothing). Store-unreachable degrades tocontent=unstored— the event still seals.dialraw/attachfdsplit the dial from the handshake so a connection crossespctl(NEWNS)into a restricted child. Session RPC is semaphore-serialized (ventiSession.rpcis a bare write-then-read; veltro runs tools concurrently).auditget(1): auditor-side fetch by score,-sverifies against the chain-sealed SHA-256.veltro.b) sealsagentstart/nscaps/sysprompt/task/plan, per-stepprompt/llm/toolcall/toolres(full tool output, pre-truncation),agentdoneon every exit path; fail-closed underAudit->ONFILE(thesecstoredauthok posture).spawnauto-grants the append-only/mnt/audit/loginto every child's caps (auditcontrolpathalready keeps root/chain/ctlungrantable), sealsspawn+subcapsper child, and arms the child via a newSubAgent.setprov()(mirrorssetmcp). The child (subagent.b) completes the venti handshake inside the restricted namespace and sealssubstart/subtask/subllm/subtool/subtoolres/subdone.nsconstruct.b's previously-uncalledemitauditloghook is now driven byrestrictnsand seals annsrestrictrecord carrying the manifest's SHA-256./usr/inferno/audit/venti.Evidence
tests/auditprov_test.b— live integration (real ventisrv + real auditfs in-emu): SHA-256 vector, plain and payload-bearing records,content=unstoreddegradation, dedup, multi-block + empty payloads, fetch-back by record score. 8/8.veltro_test16,veltro_security_test33,veltro_tools_test12,spawn_test12,spawn_helpers_test33,spawn_schedule_test12.gpt-oss:20b): a real agent run seals the full trajectory — see the chain excerpt in the PR discussion.SP800-92-audit-log.md(AU-12 row + residual table),SP800-53-controls.md,nist-control-mappings.md, roadmap Tier-1 item 2,audit-log-design.md§8/§10. New man pages:ventisrv(8),auditprov(2),auditget(1).Also fixes a fresh-clone paper cut surfaced by the worktree smoke:
n/was never tracked, somount {mntgen} /nfailed silently on fresh checkouts (n/.gitkeep, matching themnt/convention).Refs: INFR-355
🤖 Generated with Claude Code