Skip to content

Implement the core cache API replace family - #692

Open
zshenker wants to merge 1 commit into
fastly:mainfrom
zshenker:cache-replace
Open

zshenker wants to merge 1 commit into
fastly:mainfrom
zshenker:cache-replace

Conversation

@zshenker

@zshenker zshenker commented Sep 3, 2026

Copy link
Copy Markdown

This addresses #495, and closes it.

fastly::cache::core::replace() was the last unimplemented piece of the core cache API. Every replace* hostcall returned NotAvailable("Cache API primitives") (witx) or Unsupported (component), so ReplaceBuilder::begin() failed immediately and read-modify-write patterns such as a cached counter could not be tested locally at all.

This implements the whole family in both host paths. No .witx, .wit, or adapter changes were needed — that surface was already declared and adapted.

Semantics

A replace reads the current object for the request's variant and then writes a new one, which maps onto the existing CacheValue/Obligation machinery. The strategy decides how it interacts with the obligation slot:

Strategy At replace() At replace_insert()
Immediate (default) Read the matching variant; don't wait, don't take the obligation. The existing object stays visible to lookups. Plain insert — last writer wins, like non-transactional insert.
ImmediateForceMiss Same read, then clear present for the matching variants under the write lock and take the obligation, so concurrent lookups wait for the replacement instead of being served the old object. Fulfill the obligation, waking waiters.
Wait Block until no matching variant is obligated, then take the slot. This serialization is what makes the counter pattern correct. Fulfill the obligation.

The existing object is returned regardless of freshness (the docs note it "may be stale"); freshness only shows up in the lookup state, where MUST_INSERT_OR_UPDATE is always set. Obligation's existing Drop impl already gives the right "replace abandoned" behavior, so closing or dropping a replace handle needs nothing new.

Two SDK behaviors worth calling out, since they turn host-side "not supported" into guest panics rather than errors:

  • ReplaceBuilder::begin() calls replace_get_state unconditionally and panics on any status other than OK, including NONE — so replace_get_state always succeeds, returning empty flags when there is no existing object.
  • Found::hits() and Found::stale_while_revalidate() panic when the host reports unsupported, and Replace::existing_object() returns exactly that Found. Both were unimplemented for plain lookups too, so this closes those gaps as well. Hits are counted on delivery through a counter shared between the cached object and the Found handed to the guest; a replace reading the object it is about to replace does not count as a hit.

Known deviations

Both are commented at their definitions:

  • With ImmediateForceMiss, if another party already holds the obligation we clear present but cannot take a second one (the store's invariant is one Obligation per obligated CacheValue), so we fall back to a racing plain insert.
  • A guest that starts a Wait replace while itself holding an unfulfilled obligation blocks forever, where Compute would eventually time out. No timeout was added.

Tests

Rust unit tests in src/cache.rs cover each strategy, an abandoned replace releasing its obligation, a replacement inserted under a different vary rule, and the hits/stale_while_revalidate round-trip. Four concurrent Wait replaces starting from "1" produce "5", which is the property the counter pattern depends on.

Eight guest tests were added to test-fixtures/src/bin/cache.rs, so they run against both ABI paths via the existing viceroy_test! configurations. Since this fixture is also meant to pass on real Compute, assertions stick to documented behavior — e.g. hit counts are asserted monotonic rather than exact, and the force-miss test uses a transactional lookup, because "requests wait for the replacement" is the documented behavior and is well-defined on both.

cargo test --all passes (176 integration + 132 lib tests). cargo clippy --all-targets --all-features reports only pre-existing warnings in http_downstream.rs/backend.rs. The only Unsupported returns left in the cache hostcalls are the deliberate on_behalf_of/service-ID ones, matching the existing lookup and insert paths.

🤖 Generated with Claude Code

`fastly::cache::core::replace()` was the last unimplemented piece of the
core cache API: every `replace*` hostcall returned `NotAvailable`/
`Unsupported`, so `ReplaceBuilder::begin()` failed immediately and
read-modify-write patterns such as a cached counter could not be tested
locally at all.

Implement the whole family in both the witx and component host paths. A
replace reads the current object for the request's variant and then writes
a new one, which maps onto the existing `CacheValue`/`Obligation`
machinery; the strategy decides how it interacts with the obligation slot:

  * `Immediate` (the default) neither waits nor takes the obligation, so
    the existing object stays visible to lookups until the replacement is
    provided, and the insert is last-writer-wins.
  * `ImmediateForceMiss` clears the existing object under the write lock
    and takes the obligation, so concurrent lookups wait for the
    replacement rather than being served the old object.
  * `Wait` blocks until no matching variant is obligated before taking the
    slot, which serializes concurrent replaces and is what makes the
    counter pattern correct.

The existing object is returned regardless of freshness, as documented;
freshness only shows up in the lookup state, where `MUST_INSERT_OR_UPDATE`
is always set. Note that `replace_get_state` must always succeed, because
the SDK calls it unconditionally from `begin()` and panics on any status
other than OK, including NONE.

Also implement `Found::hits()` and `Found::stale_while_revalidate()`,
which `Replace::existing_object()` exposes. These were unimplemented for
plain lookups too, and the SDK panics rather than returning an error when
the host reports `unsupported`. Hits are counted on delivery via a
counter shared between the cached object and the `Found` handed to the
guest; a replace reading the object it is about to replace does not count
as a hit.

No `.witx`, `.wit`, or adapter changes: the ABI surface was already
declared and adapted.

Closes fastly#495

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant