feat: Add optional startupProbe that gates on relay /status health - #128
Closed
DrFaust92 wants to merge 2 commits into
Closed
feat: Add optional startupProbe that gates on relay /status health#128DrFaust92 wants to merge 2 commits into
DrFaust92 wants to merge 2 commits into
Conversation
The relay's /status endpoint always returns HTTP 200, even while the relay
is still initializing or when a data source connection is degraded. The
health verdict lives only in the JSON body ("status": "healthy" | "degraded").
A plain httpGet probe therefore cannot distinguish "ready to serve flags" from
"listener is up but still initializing".
This adds an optional, off-by-default startupProbe. Enabling the documented
exec probe holds off liveness/readiness until /status first reports "healthy",
so a slow LaunchDarkly connection on boot will not get the pod killed.
Unlike the approach in launchdarkly#48 (which relies on curl and
its companion image change in launchdarkly/ld-relay#259), this uses BusyBox
`wget`, which already ships in the Alpine-based launchdarkly/ld-relay image, so
no image change is required. It is also scoped to a startupProbe rather than
replacing the default readiness probe, avoiding the correlated-NotReady risk of
a low failureThreshold on readiness when a single environment reports degraded.
Adds test coverage for the default-disabled state and for setting an exec
startup probe.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Closing this. Upstream 3.9.0 (#125/#126) landed the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe the solution you've provided
Adds an optional, off-by-default
relay.startupProbe.The relay's
/statusendpoint always returns HTTP 200 — even while the relay is still initializing or when a data source connection is degraded (confirmed inrelay/endpoints_status.go: the handler never callsw.WriteHeader). The health verdict lives only in the JSON body: the top-levelstatusfield is"healthy"only once the relay is fully configured and every environment is connected, and"degraded"otherwise. A plainhttpGetprobe therefore can't distinguish "ready to serve flags" from "listener is up but still initializing" — it passes the instant the port binds.Enabling the documented exec probe holds off the liveness/readiness probes until
/statusfirst reports"healthy", so a slow LaunchDarkly connection on boot won't get the pod killed.Relationship to #48 / launchdarkly/ld-relay#259
This is intended to supersede the stalled pair #48 and launchdarkly/ld-relay#259, which reached the same conclusion (parse the body for
healthy) but:curl, which is not in the image. The officiallaunchdarkly/ld-relayimage is Alpine-based and ships BusyBoxwget, not curl — which is exactly why Enhance default readiness probe to check health status by parsing the payload (requires curl) #48 needed the companion image change in Add curl to relay Docker image ld-relay#259 and both have been parked. Usingwgetrequires no image change and unblocks the feature unilaterally.failureThreshold: 1. Because/statusreports"degraded"if any single environment is disconnected, that makes every pod goNotReadyat once during a transient LaunchDarkly-side blip. Scoping this to an opt-instartupProbe(rather than changing the default readiness probe) avoids that correlated-failure risk. The liveness probe intentionally stays onhttpGet /status— its always-200 behavior is desirable there, since a degraded upstream shouldn't trigger restart loops.Caveats (documented in
values.yaml)127.0.0.1:8030; update the URL ifservice.ports[].targetPortchanges.wget --no-check-certificate -qO- https://....readinessProbeif desired, with a sanefailureThreshold.Requirements
TestStartupProbeDisabledByDefault,TestCanSetExecStartupProbe)helm lint+go test ./testpass; default output unchanged, so golden files are untouched)🤖 Generated with Claude Code
Note
Low Risk
Default installs are unchanged; risk is limited to clusters that enable the probe, where mis-tuned thresholds or wrong port/TLS in the exec command could delay or fail pod startup.
Overview
Adds an opt-in
relay.startupProbeso deployments can delay liveness/readiness checks until the relay is actually healthy, without changing default chart behavior.The deployment template now renders a
startupProbewhenrelay.startupProbeis non-empty. Defaults stay{}, so existing releases are unchanged.values.yamldocuments why plainhttpGeton/statusis insufficient (always HTTP 200) and shows an exec probe that uses BusyBoxwgetto require"status":"healthy"in the JSON body, plus notes on port/TLS.README lists the new value, and helm tests assert no startup probe by default and that a custom exec probe (command,
periodSeconds,failureThreshold) renders correctly.Reviewed by Cursor Bugbot for commit db84253. Bugbot is set up for automated code reviews on this repo. Configure here.