fix(sdk): emit spec-compliant key access in experimental/tdf and delegate Writer - #3944
Open
dmihalcik-virtru wants to merge 1 commit into
Open
fix(sdk): emit spec-compliant key access in experimental/tdf and delegate Writer#3944dmihalcik-virtru wants to merge 1 commit into
dmihalcik-virtru wants to merge 1 commit into
Conversation
|
Warning Review limit reachedNext included review available in 6 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository UI Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Sep 1, 2026
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
dmihalcik-virtru
force-pushed
the
dspx-2604-15-delegate-writer
branch
from
September 1, 2026 03:30
7971d6c to
daa07d2
Compare
Contributor
X-Test Failure Report |
Contributor
X-Test Failure Report |
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
dmihalcik-virtru
force-pushed
the
dspx-2604-15-delegate-writer
branch
from
September 3, 2026 14:13
95ae918 to
903f660
Compare
Contributor
X-Test Failure Report |
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
dmihalcik-virtru
force-pushed
the
dspx-2604-15-delegate-writer
branch
from
September 3, 2026 14:39
903f660 to
1273c11
Compare
Contributor
X-Test Failure Report |
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
dmihalcik-virtru
force-pushed
the
dspx-2604-15-delegate-writer
branch
from
September 8, 2026 17:15
1273c11 to
2f5a2d8
Compare
Contributor
X-Test Failure Report |
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
…gate Writer
The experimental writer built its own key access objects, and for EC KAS
keys it built them wrong in three ways at once. It set keyType
`"eccWrapped"`, but `service/kas/access/rewrap.go` dispatches on the exact
string `"ec-wrapped"` and has no case for the other spelling. It derived
the wrapping key with HKDF and then XORed the DEK, where the spec and every
KAS expect AES-GCM under that derived key. And it omitted `schemaVersion`
from the KAO entirely. Any TDF this package produced against an EC KAS was
undecryptable, and nothing in the repo caught it because the package tested
its own output against its own expectations.
The fix is not a patch to that code but its deletion. `key_access.go` (-266)
goes away and `Writer` delegates to `sdk.NewChunkedWriter`, so key access
objects come from `sdk.createKeyAccess` -- the same code path
`SDK.CreateTDF` has always used and that the cross-SDK tests exercise. RSA,
EC, ML-KEM and hybrid wrapping now have exactly one implementation.
`key_access_test.go` (-652) goes with it; equivalent coverage against the
sdk functions landed earlier in this stack, so nothing is lost.
`writer.go` drops from 680 lines to ~292: `Writer` becomes its config plus
an inner `sdk.ChunkedWriter` and a `finalized` flag. Manifest assembly,
segment encryption, integrity hashing and assertion signing all move to the
one implementation. `manifest.go` sheds the `calculateSignature` copy and
the three constants that only its callers needed.
`keysplit_adapter.go` (+60) is why this is a delegation rather than a
rename. `sdk.DefaultKeySplitter` is single-KAS and ignores attributes;
`keysplit.XORSplitter` evaluates the full ABAC boolean expression and
XOR-splits the DEK across every KAS the resulting clauses require. The two
result shapes are field-identical, so the adapter is a straight copy. The
one structural mismatch is where the default KAS enters -- sdk passes it per
`Split` call, keysplit takes it at construction -- so the splitter is built
inside `Split`.
API changes callers will notice
`Finalize` now returns a single `*FinalizeResult` instead of
`(finalBytes, manifest, error)`. Error values are aliases of their sdk
counterparts rather than copies, so `errors.Is` matches under either name.
`WithSegments` no longer requires a contiguous prefix starting at 0.
Indices may be sparse -- a caller mapping fixed index blocks onto S3
multipart uploads writes gaps by construction -- but must still name written
segments in ascending order and may only drop from the end, because that is
the order the payload is laid out in.
`WithExcludeVersionFromManifest` is deprecated. It was always a no-op: the
manifest builder never read the flag. Omitting `schemaVersion` is how a
reader is told the TDF predates 4.3.0, and such a reader then expects
hex-then-base64 signatures, which are decided per segment at write time,
long before Finalize sees the option. `WithTargetMode` sets both together
and is the replacement.
This package's `"application/octet-stream"` MIME default is preserved
independently of the sdk default. `examples/cmd/benchmark_experimental.go`,
the only non-test consumer in the repo, compiles unchanged.
Because this changes the KAS wire format for EC keys, it wants a cross-SDK
run before merge:
gh workflow run xtest.yml --repo opentdf/tests --ref main \
-f platform-ref=<branch> -f otdfctl-ref=main -f java-ref=main -f js-ref=main
Signed-off-by: Dave Mihalcik <dmihalcik@virtru.com>
Contributor
dmihalcik-virtru
force-pushed
the
dspx-2604-15-delegate-writer
branch
from
September 8, 2026 17:36
2f5a2d8 to
853c55a
Compare
Contributor
X-Test Failure Report |
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Contributor
|
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Changes
The experimental writer built its own key access objects, and for EC KAS
keys it built them wrong in three ways at once. It set keyType
"eccWrapped", butservice/kas/access/rewrap.godispatches on the exactstring
"ec-wrapped"and has no case for the other spelling. It derivedthe wrapping key with HKDF and then XORed the DEK, where the spec and every
KAS expect AES-GCM under that derived key. And it omitted
schemaVersionfrom the KAO entirely. Any TDF this package produced against an EC KAS was
undecryptable, and nothing in the repo caught it because the package tested
its own output against its own expectations.
The fix is not a patch to that code but its deletion.
key_access.go(-266)goes away and
Writerdelegates tosdk.NewChunkedWriter, so key accessobjects come from
sdk.createKeyAccess-- the same code pathSDK.CreateTDFhas always used and that the cross-SDK tests exercise. RSA,EC, ML-KEM and hybrid wrapping now have exactly one implementation.
key_access_test.go(-652) goes with it; equivalent coverage against thesdk functions landed earlier in this stack, so nothing is lost.
writer.godrops from 680 lines to ~292:Writerbecomes its config plusan inner
sdk.ChunkedWriterand afinalizedflag. Manifest assembly,segment encryption, integrity hashing and assertion signing all move to the
one implementation.
manifest.gosheds thecalculateSignaturecopy andthe three constants that only its callers needed.
keysplit_adapter.go(+60) is why this is a delegation rather than arename.
sdk.DefaultKeySplitteris single-KAS and ignores attributes;keysplit.XORSplitterevaluates the full ABAC boolean expression andXOR-splits the DEK across every KAS the resulting clauses require. The two
result shapes are field-identical, so the adapter is a straight copy. The
one structural mismatch is where the default KAS enters -- sdk passes it per
Splitcall, keysplit takes it at construction -- so the splitter is builtinside
Split.API changes callers will notice
Finalizenow returns a single*FinalizeResultinstead of(finalBytes, manifest, error). Error values are aliases of their sdkcounterparts rather than copies, so
errors.Ismatches under either name.WithSegmentsno longer requires a contiguous prefix starting at 0.Indices may be sparse -- a caller mapping fixed index blocks onto S3
multipart uploads writes gaps by construction -- but must still name written
segments in ascending order and may only drop from the end, because that is
the order the payload is laid out in.
WithExcludeVersionFromManifestis deprecated. It was always a no-op: themanifest builder never read the flag. Omitting
schemaVersionis how areader is told the TDF predates 4.3.0, and such a reader then expects
hex-then-base64 signatures, which are decided per segment at write time,
long before Finalize sees the option.
WithTargetModesets both togetherand is the replacement.
This package's
"application/octet-stream"MIME default is preservedindependently of the sdk default.
examples/cmd/benchmark_experimental.go,the only non-test consumer in the repo, compiles unchanged.
Because this changes the KAS wire format for EC keys, it wants a cross-SDK
run before merge:
gh workflow run xtest.yml --repo opentdf/tests --ref main
-f platform-ref= -f otdfctl-ref=main -f java-ref=main -f js-ref=main
Checklist
Testing Instructions
This changes the KAS wire format for EC keys, so it wants a cross-SDK run
before merge:
The full DSPX-2604 stack — 20 PRs
mainmainmainmainmainmainmaindspx-2604-base-11= #3932 + #3934 + #3935dspx-2604-base-17= #3944 + #3945dspx-2604-base-19= #3947 + #3939Reviewable in parallel right now, since they sit directly on
mainand depend onnothing else: 01, 02, 04, 05, 06, 07, 08.
Why three PRs have a
dspx-2604-base-*base. A GitHub PR takes one base branch,but 11, 17 and 19 each build on more than one parent. The
base-*branches are emptymerge commits that exist only to join those parents so the PR diff shows exactly its
own change and nothing else. They contain no code, have no PR of their own, and go
away once their parents land — retarget the child onto
mainat that point.Wants a cross-SDK xtest run before merge: 15, 17 (and therefore 20). They touch
the KAS wire format.
Red checks you may see are network flakes, not this stack. Four distinct ones hit
this batch and all clear on re-run:
golangci-lint config verifytiming out onhttps://golangci-lint.run/.../golangci.v2.8.jsonschema.json(fails the wholego (<module>)job and fail-fast cancels its siblings), the bats installer getting a 403,Docker Hub timing out on
keycloak/keycloak:26.4, andbufreporting "the serverhosted at that remote is unavailable" while the Java SDK generates sources. The
govulncheckstep also emits##[error]annotations against the go1.25.11 stdlib, butit is
continue-on-error: trueand never fails a job — 01 bumps the toolchain andclears those annotations.