fix(epp): bound prefill reservations for v1.3.0 Telus backport - #1
Closed
sarma-rafay wants to merge 17 commits into
Conversation
|
👋 Hi sarma-rafay! Thank you for contributing to RafaySystems/dynamo. Just a reminder: The 🚀 |
…orer (ai-dynamo#11991) Signed-off-by: Avinash Varma <avinashvarmap@gmail.com> Co-authored-by: Avinash Varma <avinashvarmap@gmail.com> (cherry picked from commit ffae596)
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com>
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com>
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com>
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com>
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com> (cherry picked from commit 2ead078)
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com> (cherry picked from commit cab8cac)
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com> (cherry picked from commit c32ce78)
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com> (cherry picked from commit fa3e7bc)
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com>
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com> (cherry picked from commit 3343242)
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com> (cherry picked from commit 7b90075)
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com>
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com> (cherry picked from commit ece553d)
Adds a self-contained way to build a drop-in dynamo-frontend image carrying the EPP KV-router tool-call fix: - hack/Dockerfile.epp — compiles the EPP binary from a single build context (repo root), so a plain `docker build` works (no buildx / named contexts). - hack/Dockerfile.frontend-overlay — overlays the built /epp onto the upstream monolithic frontend image, leaving every other role byte-identical. - hack/Makefile — orchestrates epp-image -> frontend-image -> push, with overridable VERSION / REGISTRY / BASE_IMAGE. - hack/README.md — usage. Build: `make -C hack push` Signed-off-by: Avinash Varma <avinashvarmap@gmail.com> (cherry picked from commit dcc1cfd) Signed-off-by: Sarma Ayyadevara <sarma@rafay.co>
deploy/inference-gateway/epp/go.mod declares 'go 1.26.3', but the builder stage pinned golang:1.25. The build only succeeded because Go's automatic toolchain switching downloaded go1.26.3 mid-build, which needs network access in the builder stage and silently floats the compiler version. Pin the builder to the version go.mod already requires so the image build is reproducible and does no toolchain download. Signed-off-by: Avinash Varma <avinashvarmap@gmail.com> Signed-off-by: Sarma Ayyadevara <sarma@rafay.co>
hack/Dockerfile.epp copied only .cargo/, the root manifests and lib/ into the Rust builder stage. deploy/inference-gateway/ext-proc is a workspace member in the root Cargo.toml, so cargo refuses to resolve the workspace without its manifest on disk and the stage fails before compiling anything: error: failed to load manifest for workspace member `/dynamo/deploy/inference-gateway/ext-proc` referenced by workspace at `/dynamo/Cargo.toml` The canonical deploy/inference-gateway/epp/Dockerfile already copies it for exactly this reason. Mirror that here so `make -C hack push` works. Signed-off-by: Avinash Varma <avinashvarmap@gmail.com> Signed-off-by: Sarma Ayyadevara <sarma@rafay.co>
sarma-rafay
force-pushed
the
backport/v1.3.0/epp-pass-full-request-body-telus
branch
from
August 14, 2026 21:12
5422753 to
54d8577
Compare
github-actions
Bot
added
the
documentation
sarma-rafay
changed the title
sarma-rafay
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebuilds
backport/v1.3.0/epp-pass-full-request-body-telusfrom NVIDIAtmonty12/dyn-12864-prefill-reservations-release-1.3.0(tip94901a158) and re-applies Rafayhack/EPP build commits on top.Brings the full prefill-reservation stack and the newer hardening
fixes (C-handle scoping, booking cleanup executor, ownership/reaping,
late release, raised admission defaults) onto the Rafay full-body
backport line.
BuildOpenAIRequestJSONremains the request-build path(NVIDIA
#11991).Validation
hack/commits (5e8a84a0b,246af95f2,5eea05f6b)onto NVIDIA tip with DCO sign-off; no conflicts.
BuildOpenAIRequestJSONindisagg/shared.goanddynamo_kv_scorer/plugin.go.booking_executor.go,prefill_scorer.go,lib/llm/.../reservations.rs.hack/(expectedwith NVIDIA-as-baseline).
Source:
ai-dynamo/dynamobranchtmonty12/dyn-12864-prefill-reservations-release-1.3.0.