feat(sdk): graduate the chunked writer to stable API

[dmihalcik-virtru]

Part 20 of 20 in the DSPX-2604 re-cut. Base branch: dspx-2604-19-drop-encrypt-spool.

This stack replaces #3782 / #3865 / #3921, which stay open and untouched
until it lands. Nothing here is a rebase of those branches — the work was
re-cut from the ticket so each PR stands on its own.

Proposed Changes

The chunked writer has been the sole implementation of TDF construction since
CreateTDF was rewritten on top of it. Every TDF the SDK produces already goes
through this code; the "Experimental: not part of the stable SDK API; may
change or be removed" markers on it stopped being true at that point. Drop
them, and give the writer the constructor it was missing.

SDK.NewChunkedWriter resolves key access against the platform the SDK is
connected to, running attributes through the same autoconfigure path
CreateTDF uses. Until now a chunked caller who wanted multi-KAS attribute
grants had to implement a KeySplitter to get them, reimplementing
autoconfigure by hand; DefaultKeySplitter is single-KAS and attribute-blind.
The TDFOptions that shape key access -- WithDataAttributes,
WithKasInformation, WithWrappingKeyAlg -- go in through
WithChunkedTDFOptions.

Resolution happens at Finalize, not at construction. That is the difference
from CreateTDF and the reason this is a distinct resolver rather than a shared
one: a chunked caller may still be adding attributes while segments are in
flight, so the attribute set is not known until Finalize is called. The cost
is that an unreachable KAS surfaces at Finalize, after segments have already
been handed back. Passing WithChunkedKeySplitter opts out entirely and the
given splitter is used as-is -- the config tracks whether the option was
supplied so that "left at the default" and "deliberately overridden" are
distinguishable.

The test seams -- archiveFactory, cipherFactory, clock -- stay unexported.
archiveFactory returns a zipstream.SegmentWriter, and zipstream lives under
internal/, so there is no form of these that could be exported and supported.
Graduating the package does not graduate them.

sdk/experimental/tdf is deprecated at the package level rather than deleted.
It has forwarded to this implementation since the Writer delegation commit and
its manifest and assertion types are already aliases of the sdk types, so
values move across the boundary without conversion and callers can migrate a
call site at a time. The package doc carries the full option-by-option
mapping.

The experimental-writer benchmark example moves with it:
benchmark-experimental-writer becomes benchmark-chunked-writer and builds on
the package-level NewChunkedWriter, since it talks to one KAS whose key it has
already fetched and has no reason to pay for platform resolution inside the
timed section. Two bugs in it are fixed in passing: the segment goroutines
read the loop variable to compute their own offsets, and shared a single err.

Testing: the chunked suite and the classic TDFSuite round trips both pass
unchanged. Two new cases cover the constructor -- one asserting that
attributes resolve through the platform to the right KAS URL and KID and that
the result round-trips through LoadTDF, one asserting that an explicit
splitter is honored, using a KAS URL that would fail loudly if platform
resolution ran anyway. The fake KAS gained a PublicKey handler so the first
exercises the real key fetch rather than being handed a PEM. Also verified
against cross-module builds of examples, otdfctl, service, and tests-bdd.

Graduating also settles what a nil default KAS means, which the experimental
options never did. WithChunkedDefaultKAS and WithChunkedDefaultKASForFinalize
now reject nil, like every other option in the file that takes a pointer or an
interface. Storing a nil is not recoverable later -- the field is
indistinguishable from "not set" -- and for this particular field the failure
is silent rather than a panic: resolution falls through to the platform base
key, and the caller finds out their data went somewhere they never named only
when a reader cannot unwrap it. That is the shape of the base-key contention
that broke the streaming e2e file. SDK.NewChunkedWriter's doc comment now
spells the fallback out, including that WithKasInformation does not suppress
it, since it fills kasInfoList without clearing autoconfigure.

sdk/experimental/tdf keeps taking nil as "unset" -- WithDefaultKAS and
WithDefaultKASForWriter ship on main and are not this stack's to tighten -- so
the delegation appends the stable option only when a KAS is actually set. A
nil there still surfaces as the splitter's ErrNoDefaultKAS at Finalize, which
is what callers already handle, and a test pins that so a future change cannot
leak the option error through.

TestChunkedOptionsRejectNil covers both new guards and
TestSDKChunkedWriterFallsBackToBaseKey pins the fallback itself, so the
behavior the doc comment describes is asserted rather than just written down.
Both were mutation-checked: removing either guard fails them.

Checklist

• I have added or updated unit tests
• I have added or updated integration tests (if appropriate)
• I have added or updated documentation

Testing Instructions

cd sdk && go test ./... -race
cd examples && go build ./...

TestSDKChunkedWriterResolvesKeyAccess and
TestSDKChunkedWriterKeepsAnExplicitSplitter are the new coverage for the
constructor. The first goes through the fake KAS's new PublicKey handler, so
it exercises the real key fetch rather than being handed a PEM.

For the nil-KAS half, the cases to look at are
TestChunkedOptionsRejectNil/default_KAS{,_for_finalize},
TestSDKChunkedWriterFallsBackToBaseKey, and
TestWriterEndToEnd/ErrorConditions/ExplicitNilKASStaysUnset. Both directions
were mutation-checked: dropping either guard in chunked_options.go fails the
first two, and forwarding the KAS unconditionally from
experimental/tdf/writer.go again makes plain NewWriter(ctx) fail with
chunked: default KAS must not be nil — which is the coupling between the two
changes.

The full DSPX-2604 stack — 20 PRs

#	PR	Based on
01	#3930 chore: bump go.work toolchain to go1.25.12 and simplify an rt_test condition	main
02	#3931 feat(sdk): make the zipstream clock injectable for deterministic ZIP output	main
03	#3932 fix(sdk): reject a zipstream write set that omits segment 0	#3931
04	#3933 fix(sdk): map ReadAt plaintext offsets from cumulative segment sizes	main
05	#3934 chore(sdk): extract integrityAlgorithmString, createPolicyBinding, signAssertions	main
06	#3935 chore(sdk): add direct tests for createKeyAccess, encryptMetadata and tdfSalt	main
07	#3936 fix(sdk): fill each segment with io.ReadFull and size the buffer to the input	main
08	#3937 chore(cli): move streaming IO helpers into pkg	main
09	#3938 fix(cli): stream encrypt instead of buffering the whole payload	#3937
10	#3939 fix(cli): stream decrypt and inspect instead of buffering	#3938
11	#3940 feat(sdk): add a chunked segment writer (experimental)	dspx-2604-base-11 = #3932 + #3934 + #3935
12	#3941 fix(sdk): stop GetManifest from splitting the key under the lock	#3940
13	#3942 fix(sdk): reject a chunked split naming a KAS with no resolved public key	#3941
14	#3943 chore(sdk): alias experimental/tdf manifest and assertion types	#3942
15	#3944 fix(sdk): emit spec-compliant key access in experimental/tdf and delegate Writer	#3943
16	#3945 feat(sdk): accept io.Reader in CreateTDF and drop the 64 GB payload cap	#3936
17	#3946 chore(sdk): rewrite CreateTDF on top of the chunked writer	dspx-2604-base-17 = #3944 + #3945
18	#3947 chore(sdk): drop dead TDFConfig fields and deprecate the TDFFormat enum	#3946
19	#3948 fix(cli): drop the encrypt-side stdin spool	dspx-2604-base-19 = #3947 + #3939
20	#3949 feat(sdk): graduate the chunked writer to stable API	#3948

Reviewable in parallel right now, since they sit directly on main and depend on
nothing else: 01, 02, 04, 05, 06, 07, 08.

Why three PRs have a dspx-2604-base-* base. A GitHub PR takes one base branch,
but 11, 17 and 19 each build on more than one parent. The base-* branches are empty
merge commits that exist only to join those parents so the PR diff shows exactly its
own change and nothing else. They contain no code, have no PR of their own, and go
away once their parents land — retarget the child onto main at that point.

Wants a cross-SDK xtest run before merge: 15, 17 (and therefore 20). They touch
the KAS wire format.

Red checks you may see are network flakes, not this stack. Four distinct ones hit
this batch and all clear on re-run: golangci-lint config verify timing out on
https://golangci-lint.run/.../golangci.v2.8.jsonschema.json (fails the whole go (<module>) job and fail-fast cancels its siblings), the bats installer getting a 403,
Docker Hub timing out on keycloak/keycloak:26.4, and buf reporting "the server
hosted at that remote is unavailable" while the Java SDK generates sources. The
govulncheck step also emits ##[error] annotations against the go1.25.11 stdlib, but
it is continue-on-error: true and never fails a job — 01 bumps the toolchain and
clears those annotations.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Team

Run ID: 2b693341-996e-4211-9396-c17f411458b9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	186.881136ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	94.207624ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	345.156513ms
Throughput	289.72 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	35.134085364s
Average Latency	350.739721ms
Throughput	142.31 requests/second

[github-actions]

X-Test Results

✅ go@pull-3949-main
✅ go@pull-3949-pull-3949
✅ go@main-main
✅ go@main-pull-3949
bats-test-results
✅ go@pull-3949-v0.9.0
✅ java@main-main
✅ java@main-pull-3949
✅ go@main-v0.9.0
✅ js@main-pull-3949
cukes-report
✅ js@main-main
✅ java@main-v0.9.0
✅ js@main-v0.9.0
opentdfplatformQ4ZHEQ.dockerbuild
govulncheck-failure-8
govulncheck-failure-3
govulncheck-failure-4
govulncheck-failure-1
govulncheck-failure-0
govulncheck-failure-2
govulncheck-failure-5

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	191.211585ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	101.954742ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	328.070574ms
Throughput	304.81 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	36.291102987s
Average Latency	361.68107ms
Throughput	137.77 requests/second

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	244.259739ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	133.144717ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	428.758457ms
Throughput	233.23 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	43.840791722s
Average Latency	437.582977ms
Throughput	114.05 requests/second

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	254.268867ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	151.692848ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	426.923905ms
Throughput	234.23 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	45.775324299s
Average Latency	456.847758ms
Throughput	109.23 requests/second

[github-actions]

X-Test Failure Report

opentdfplatformN5QOND.dockerbuild
govulncheck-failure-8
govulncheck-failure-3
govulncheck-failure-0
govulncheck-failure-1
govulncheck-failure-2
govulncheck-failure-5
govulncheck-failure-4

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	207.430662ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	107.629188ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	403.514422ms
Throughput	247.82 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	41.045847765s
Average Latency	409.537599ms
Throughput	121.82 requests/second

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	241.334724ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	120.350844ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	438.894964ms
Throughput	227.84 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	43.190566999s
Average Latency	431.058232ms
Throughput	115.77 requests/second

[github-actions]

X-Test Results

✅ go@main-main
bats-test-results
✅ go@pull-3949-main
✅ go@main-pull-3949
✅ go@main-v0.9.0
✅ go@pull-3949-v0.9.0
✅ java@main-pull-3949
✅ go@pull-3949-pull-3949
✅ java@main-main
✅ js@main-main
✅ java@main-v0.9.0
cukes-report
✅ js@main-v0.9.0
✅ js@main-pull-3949
opentdfplatform2UKFGH.dockerbuild
govulncheck-failure-1
govulncheck-failure-0
govulncheck-failure-4
govulncheck-failure-5

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	256.504445ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	138.661153ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	453.907296ms
Throughput	220.31 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	44.889534837s
Average Latency	448.037585ms
Throughput	111.38 requests/second

[github-actions]

X-Test Results

bats-test-results
✅ go@pull-3949-main
✅ go@main-main
✅ go@pull-3949-pull-3949
cukes-report
✅ go@pull-3949-v0.9.0
✅ java@main-pull-3949
✅ go@main-pull-3949
✅ java@main-main
✅ js@main-main
✅ go@main-v0.9.0
✅ java@main-v0.9.0
✅ js@main-v0.9.0
✅ js@main-pull-3949
opentdfplatformJ77QB1.dockerbuild
govulncheck-failure-3
govulncheck-failure-8
govulncheck-failure-5
govulncheck-failure-2
govulncheck-failure-4
govulncheck-failure-1
govulncheck-failure-0

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	250.347593ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	134.014981ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	424.496679ms
Throughput	235.57 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	44.931642032s
Average Latency	448.470594ms
Throughput	111.28 requests/second

[github-actions]

X-Test Results

✅ go@pull-3949-main
✅ go@main-main
✅ go@pull-3949-pull-3949
✅ java@main-main
cukes-report
✅ go@main-pull-3949
✅ go@main-v0.9.0
✅ go@pull-3949-v0.9.0
✅ js@main-main
✅ js@main-v0.9.0
✅ java@main-pull-3949
✅ java@main-v0.9.0
✅ js@main-pull-3949
opentdfplatformBPDQBL.dockerbuild
govulncheck-failure-5
govulncheck-failure-4
govulncheck-failure-1
govulncheck-failure-0
govulncheck-failure-2

[github-actions]

X-Test Failure Report

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	248.67514ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	134.966701ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	429.871421ms
Throughput	232.63 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	1m1.357984846s
Average Latency	612.230612ms
Throughput	81.49 requests/second

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	269.158188ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	137.726497ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	436.014763ms
Throughput	229.35 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	1m0.750042642s
Average Latency	606.266972ms
Throughput	82.30 requests/second

[github-actions]

X-Test Results

✅ go@pull-3949-pull-3949
✅ go@main-pull-3949
✅ go@main-main
✅ go@pull-3949-main
bats-test-results
✅ go@main-v0.9.0
✅ go@pull-3949-v0.9.0
✅ java@main-pull-3949
✅ java@main-main
cukes-report
✅ js@main-main
✅ js@main-pull-3949
✅ java@main-v0.9.0
✅ js@main-v0.9.0
opentdfplatformLRGXKU.dockerbuild
govulncheck-failure-3
govulncheck-failure-8
govulncheck-failure-1

[github-actions]

X-Test Failure Report

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	177.253057ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	100.204865ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	350.081612ms
Throughput	285.65 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	46.017967188s
Average Latency	459.207407ms
Throughput	108.65 requests/second

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	235.881957ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	137.269322ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	444.705922ms
Throughput	224.87 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	58.261487227s
Average Latency	580.801404ms
Throughput	85.82 requests/second

[github-actions]

⚠️ Govulncheck found vulnerabilities ⚠️

The following modules have known vulnerabilities:

• otdfctl
• service
• tests-bdd

See the workflow run for details.

[github-actions]

⚠️ Govulncheck found vulnerabilities ⚠️

The following modules have known vulnerabilities:

• otdfctl
• service
• tests-bdd

See the workflow run for details.

[github-actions]

X-Test Results

✅ go@main-main
bats-test-results
✅ go@main-pull-3949
✅ go@main-v0.9.0
✅ go@pull-3949-pull-3949
✅ java@main-main
✅ go@pull-3949-main
✅ java@main-pull-3949
cukes-report
✅ js@main-pull-3949
✅ go@pull-3949-v0.9.0
✅ java@main-v0.9.0
✅ js@main-v0.9.0
✅ js@main-main
opentdfplatformR5YXSL.dockerbuild
govulncheck-failure-8
govulncheck-failure-3
govulncheck-failure-1