policy: Deduplicate SecretWatchers and TLS contexts - #2006
Merged
Conversation
jrajahalme
force-pushed
the
policy-secret-sharing
branch
3 times, most recently
from
August 20, 2026 13:05
5782f11 to
b96455e
Compare
jrajahalme
force-pushed
the
policy-secret-sharing
branch
from
August 20, 2026 13:48
b96455e to
e0b9f37
Compare
Member
Author
|
added a missing include for tidy |
Member
Author
|
restarted integration test due to a flake fixed by #2007 |
nezdolik
reviewed
Aug 21, 2026
nezdolik
reviewed
Aug 21, 2026
Network policies may contain hundreds or thousands of HeaderMatch or TLS
rules referring to a small set of Secrets. Each reference previously
constructed a separate SecretWatcher or Envoy TLS ContextConfigImpl,
including its SDS provider callbacks and other context state.
Add separate weak caches for SecretWatchers and upstream and downstream
TLS contexts, keyed by the SDS name or Cilium TLSContext protobuf. Reuse
a live context when an equivalent policy configuration is encountered.
Reset the caches together with the policy maps after an NPDS stream restart,
preserving the existing agent-restart behavior. Prune expired cache entries
asynchronously after policy updates and worker quiescence, keeping pruning
off the policy-update hot path.
Benchmark an NPDS update containing 1,000 policies that reference the same
SDS-derived CA validation context. The CA is delivered through a real dynamic
SDS provider rather than embedded in the policy TLS contexts.
Optimized benchmark results (3 repetitions):
CPU time Retained heap Heap/policy
TLS context cache enabled 1.47 ms 1.581 MiB 1.619 KiB
TLS context cache disabled 3.95 ms 4.990 MiB 5.109 KiB
This reduces CPU time by approximately 63% and retained heap by 68%, saving
about 3.41 MiB per 1,000-policy update and increasing throughput by 2.68x.
The benchmark uses Envoy's small, single-certificate test CA. Production SDS
resources commonly contain larger CA bundles comparable to those shipped
with operating systems, so the benchmark likely understates the memory impact
of retaining duplicate certificate-related context state.
Benchmark results for deduplication of SecretWatchers on HeaderMatches
are more modest (duplicating header sized secrets carries less overhead),
but are still positive around 20-25% for CPU and memory.
Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
jrajahalme
force-pushed
the
policy-secret-sharing
branch
from
August 21, 2026 13:47
e0b9f37 to
a3faf4d
Compare
nezdolik
approved these changes
Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Network policies may contain hundreds or thousands of HeaderMatch or TLS rules referring to a small set of Secrets. Each reference previously constructed a separate SecretWatcher or Envoy TLS ContextConfigImpl, including its SDS provider callbacks and other context state.
Add separate weak caches for SecretWatchers and upstream and downstream TLS contexts, keyed by the SDS name or Cilium TLSContext protobuf, respectively. Reuse a live context when an equivalent policy configuration is encountered.
Reset the caches together with the policy maps after an NPDS stream restart, preserving the existing agent-restart behavior. Prune expired cache entries asynchronously after policy updates and worker quiescence, keeping pruning off the policy-update hot path.
Benchmark an NPDS update containing 1,000 policies that reference the same SDS-derived CA validation context. The CA is delivered through a real dynamic SDS provider rather than embedded in the policy TLS contexts.
Optimized benchmark results (3 repetitions):
This reduces CPU time by approximately 63% and retained heap by 68%, saving about 3.41 MiB per 1,000-policy update and increasing throughput by 2.68x.
The benchmark uses Envoy's small, single-certificate test CA. Production SDS resources commonly contain larger CA bundles comparable to those shipped with operating systems, so the benchmark likely understates the memory impact of retaining duplicate certificate-related context state.
Benchmark results for deduplication of SecretWatchers on HeaderMatches are more modest (duplicating header sized secrets carries less overhead), but are still positive around 20-25% for CPU and memory.