Skip to content

feat: Add optional startupProbe that gates on relay /status health - #128

Closed
DrFaust92 wants to merge 2 commits into
launchdarkly:mainfrom
DrFaust92:feat/startup-probe-health-parse
Closed

feat: Add optional startupProbe that gates on relay /status health#128
DrFaust92 wants to merge 2 commits into
launchdarkly:mainfrom
DrFaust92:feat/startup-probe-health-parse

Conversation

@DrFaust92

@DrFaust92 DrFaust92 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Describe the solution you've provided

Adds an optional, off-by-default relay.startupProbe.

The relay's /status endpoint always returns HTTP 200 — even while the relay is still initializing or when a data source connection is degraded (confirmed in relay/endpoints_status.go: the handler never calls w.WriteHeader). The health verdict lives only in the JSON body: the top-level status field is "healthy" only once the relay is fully configured and every environment is connected, and "degraded" otherwise. A plain httpGet probe therefore can't distinguish "ready to serve flags" from "listener is up but still initializing" — it passes the instant the port binds.

Enabling the documented exec probe holds off the liveness/readiness probes until /status first reports "healthy", so a slow LaunchDarkly connection on boot won't get the pod killed.

relay:
  startupProbe:
    exec:
      command:
        - /bin/sh
        - -c
        - wget -qO- http://127.0.0.1:8030/status | grep -q '"status":"healthy"'
    periodSeconds: 3
    timeoutSeconds: 2
    failureThreshold: 20

Relationship to #48 / launchdarkly/ld-relay#259

This is intended to supersede the stalled pair #48 and launchdarkly/ld-relay#259, which reached the same conclusion (parse the body for healthy) but:

  • Used curl, which is not in the image. The official launchdarkly/ld-relay image is Alpine-based and ships BusyBox wget, not curl — which is exactly why Enhance default readiness probe to check health status by parsing the payload (requires curl) #48 needed the companion image change in Add curl to relay Docker image ld-relay#259 and both have been parked. Using wget requires no image change and unblocks the feature unilaterally.
  • Replaced the default readiness probe with failureThreshold: 1. Because /status reports "degraded" if any single environment is disconnected, that makes every pod go NotReady at once during a transient LaunchDarkly-side blip. Scoping this to an opt-in startupProbe (rather than changing the default readiness probe) avoids that correlated-failure risk. The liveness probe intentionally stays on httpGet /status — its always-200 behavior is desirable there, since a degraded upstream shouldn't trigger restart loops.

Caveats (documented in values.yaml)

  • The command hard-codes 127.0.0.1:8030; update the URL if service.ports[].targetPort changes.
  • Under TLS, switch to wget --no-check-certificate -qO- https://....
  • The same exec block can be applied to readinessProbe if desired, with a sane failureThreshold.

Requirements

  • I have added test coverage for new or changed functionality (TestStartupProbeDisabledByDefault, TestCanSetExecStartupProbe)
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions (helm lint + go test ./test pass; default output unchanged, so golden files are untouched)

🤖 Generated with Claude Code


Note

Low Risk
Default installs are unchanged; risk is limited to clusters that enable the probe, where mis-tuned thresholds or wrong port/TLS in the exec command could delay or fail pod startup.

Overview
Adds an opt-in relay.startupProbe so deployments can delay liveness/readiness checks until the relay is actually healthy, without changing default chart behavior.

The deployment template now renders a startupProbe when relay.startupProbe is non-empty. Defaults stay {}, so existing releases are unchanged. values.yaml documents why plain httpGet on /status is insufficient (always HTTP 200) and shows an exec probe that uses BusyBox wget to require "status":"healthy" in the JSON body, plus notes on port/TLS.

README lists the new value, and helm tests assert no startup probe by default and that a custom exec probe (command, periodSeconds, failureThreshold) renders correctly.

Reviewed by Cursor Bugbot for commit db84253. Bugbot is set up for automated code reviews on this repo. Configure here.

The relay's /status endpoint always returns HTTP 200, even while the relay
is still initializing or when a data source connection is degraded. The
health verdict lives only in the JSON body ("status": "healthy" | "degraded").
A plain httpGet probe therefore cannot distinguish "ready to serve flags" from
"listener is up but still initializing".

This adds an optional, off-by-default startupProbe. Enabling the documented
exec probe holds off liveness/readiness until /status first reports "healthy",
so a slow LaunchDarkly connection on boot will not get the pod killed.

Unlike the approach in launchdarkly#48 (which relies on curl and
its companion image change in launchdarkly/ld-relay#259), this uses BusyBox
`wget`, which already ships in the Alpine-based launchdarkly/ld-relay image, so
no image change is required. It is also scoped to a startupProbe rather than
replacing the default readiness probe, avoiding the correlated-NotReady risk of
a low failureThreshold on readiness when a single environment reports degraded.

Adds test coverage for the default-disabled state and for setting an exec
startup probe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@DrFaust92
DrFaust92 requested a review from a team as a code owner July 3, 2026 03:31
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@DrFaust92

Copy link
Copy Markdown
Contributor Author

Closing this. Upstream 3.9.0 (#125/#126) landed the relay.startupProbe mechanism independently — template block, values passthrough, and tests — so this PR's implementation is now redundant. The only remaining delta was documentation of the /status always-returns-200 gotcha (why an httpGet startup probe on /status is ineffective and an exec probe parsing the body for "status":"healthy" is needed), which isn't worth a standalone PR.

@DrFaust92 DrFaust92 closed this Jul 3, 2026
@DrFaust92
DrFaust92 deleted the feat/startup-probe-health-parse branch July 3, 2026 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant