Skip to content

Stop reporting success when the SDK is exporting nothing — TS, Go, Python, .NET - #87

Merged
brentrager merged 2 commits into
mainfrom
bootstrap-honest-status-polyglot
Aug 15, 2026
Merged

Stop reporting success when the SDK is exporting nothing — TS, Go, Python, .NET#87
brentrager merged 2 commits into
mainfrom
bootstrap-honest-status-polyglot

Conversation

@brentrager

Copy link
Copy Markdown
Contributor

Problem

The Rust SDK told a production service it was fine while it emitted nothing for months (#84). The other four SDKs all had the same bug. bootstrap() reported installed: true whenever it was not explicitly disabled — including when no OTLP endpoint was configured, so telemetry had nowhere to go. The no-endpoint branch was completely silent: it warned about missing AUTH but not about the far more consequential missing DESTINATION.

Per-language verdict

I verified each language independently rather than assuming the bug ported.

SDK Verdict Evidence
TypeScript 🔴 Bug present — worse than Rust bootstrapped = { installed: true, otel, stopRefresh } set unconditionally; no endpoint check anywhere. Worse: setupOtelSdk falls back to new OTLPTraceExporter({ headers }) with no url, which OTel defaults to http://localhost:4318 — so a container doesn't no-op, it retries into the void forever.
Go 🔴 Bug present result = BootstrapResult{Installed: true, Otel: otelHandle} regardless of endpoint. SetupOtelSDK correctly skips exporters when the endpoint is empty, leaving every provider nil — so the handle already knew the truth and nothing asked it.
Python 🔴 Bug present — worse than Rust BootstrapResult(installed=True, ...) unconditionally. Same localhost fallback as TS (OTLPSpanExporter(headers=...) with no endpoint). The bug had already left a note about itself: python/tests/conftest.py exists to silence the resulting "connection refused to localhost:4318" spam.
.NET 🔴 Bug present return Cache(new BootstrapResult { Installed = true, Otel = otel }) unconditionally. Like Go, ObservabilitySdk.Build gates each exporter on a non-empty endpoint, so providers are null and nobody checked.
Rust ✅ Already fixed #84, merged as d40f623 — this PR mirrors its semantics and wording.

What changed in each

  1. An honest status flagexporting (TS, Python), Exporting (Go, .NET), matching Rust's exporting. installed/Installed keeps its old meaning and now says so honestly in its doc comment instead of implying more.

    Two deliberate per-language differences:

    • Go derives it from the handle, not the endpoint strings — an endpoint whose exporter failed to construct leaves every provider nil, and that is just as much "not exporting" as having no endpoint.
    • .NET counts traces + metrics only. Setup builds exporters for exactly those two; a LogsEndpoint alone is consumed by the ILoggingBuilder extension, so claiming Exporting on it would be the same lie in a new place.
  2. A loud warning when no endpoint is configured, wording matched to Rust's so the five SDKs read alike. It names the variable to set AND offers SMOOAI_OBSERVABILITY_DISABLED=true, so an intentional no-op can be declared rather than inferred from absence.

  3. Both halves tested in all four. With only one asserted, an implementation that hard-codes either value passes.

Three tests enshrined the misleading shape and now assert the honest one (same as the Rust one #84 fixed): Go's TestBootstrapInstallsClientAndCapture, Python's test_never_raises_on_bad_config, .NET's Run_NeverThrows_OnBadConfig — each asserted installed while nothing had a destination. Each now also pins the warning and clears the OTEL_EXPORTER_OTLP_* env vars, so "no endpoint" means no endpoint from any source rather than whatever the CI runner happens to export.

Mutation table

Every new assertion was mutation-checked: apply the mutation, confirm the specific test fails, restore, verify byte-identical via sha256 on raw bytes.

# Mutation Test that failed Exit Residue
TS-M1 exporting hard-coded true reports exporting=false and warns loudly…expected true to be false 1 none
TS-M2 exporting hard-coded false reports exporting=true and stays quiet…expected false to be true 1 none
TS-M3 no-endpoint warning removed reports exporting=false…expected '' to contain 'NO OTLP ENDPOINT CONFIGURED' 1 none
GO-M1 Exporting hard-coded true TestBootstrapInstallsClientAndCapture 1 none
GO-M2 Exporting hard-coded false TestBootstrapExportingWhenEndpointConfigured 1 none
GO-M3 no-endpoint warning removed TestBootstrapInstallsClientAndCapture 1 none
PY-M1 exporting hard-coded True test_never_raises_on_bad_configassert True is False 1 none
PY-M2 exporting hard-coded False test_exporting_true_when_endpoint_configuredassert False is True 1 none
PY-M3 no-endpoint warning removed test_never_raises_on_bad_config'NO OTLP ENDPOINT CONFIGURED' not in stderr 1 none
NET-M1 Exporting hard-coded true Run_NeverThrows_OnBadConfigAssert.False() Failure 1 none
NET-M2 Exporting hard-coded false Run_WithEndpoint_ReportsExportingAssert.True() Failure 1 none
NET-M3 no-endpoint warning removed Run_NeverThrows_OnBadConfigAssert.Contains() Failure 1 none

12/12 mutations killed the intended test and only that test. Residue was verified with python3 over raw bytes (not grep) across all touched files: 0 hits.

Two pre-existing .NET CI failures (second commit)

Both reproduce on a stashed clean origin/main tree and are unrelated to this change — but the dotnet lane only runs when dotnet/** changes, so this is the PR that has to face them.

  1. dotnet format --verify-no-changes exits 2 on CrashChild.cs — 14 WHITESPACE errors, a braced switch-case body indented one level short. Identical on the clean tree. Fixed by running dotnet format on that one file; pure indentation.

  2. OtelSetupTests.Setup_IsIdempotent is a flake — 3 of 8 full-suite runs failed on the clean tree. ObservabilitySdk._installed is a process-wide static and three classes call ResetForTests() on it, but only BootstrapTests was in a collection. xUnit parallelizes across collections, so the other two ran concurrently and a foreign reset landed between that test's two Setup() calls — wiping the very install guard the test exists to assert. It hid because it passes 6/6 when the suite is filtered down.

    Fixed by putting all three classes in one non-parallel collection (OtelGlobalStateCollection, renamed from "Bootstrap" since it guards the OTel singleton, not bootstrap). 10/10 full-suite runs green afterwards.

Verification

Every gate run as CI runs it, judged by exit code captured immediately after:

TS   typecheck 0 · lint 0 · test 0 (263 passed) · build 0 · format:check 0   (oxfmt, not prettier)
Go   gofmt -l → empty · vet 0 (., fiber, gin) · test 0 (., fiber, gin)
Py   ruff check 0 · ruff format --check 0 · pytest 0 (78 passed)
.NET restore 0 · build 0 · test 0 (80 passed) · format --verify-no-changes 0

Changeset added (minor).

🤖 Generated with Claude Code

brentrager and others added 2 commits August 15, 2026 15:32
…thon, .NET

Ports the Rust fix from #84 to the other four SDKs. Every one of them had the
same bug: `bootstrap()` reported `installed: true` whenever it was not explicitly
disabled — INCLUDING when no OTLP endpoint was configured, in which case
telemetry has nowhere to go. The no-endpoint branch was completely silent; it
warned about missing AUTH but not about the far more consequential missing
DESTINATION.

TypeScript and Python are worse than Rust was. Rust at least skipped building the
SDK with no endpoint. TS and Python construct an OTLP exporter with NO url, which
the OTel default sends to `http://localhost:4318` — so a container with no
endpoint configured doesn't no-op, it retries into the void forever. The Python
test conftest already silences the resulting "connection refused to
localhost:4318" spam, which is the bug leaving a note about itself.

Per language:

1. An honest status flag alongside the existing one — `exporting` (TS, Python),
   `Exporting` (Go, .NET), matching Rust's `exporting`. `installed`/`Installed`
   keeps its old meaning and now says so honestly in its doc comment.

   Go derives it from the handle rather than the endpoint strings: an endpoint
   whose exporter failed to construct leaves every provider nil, and that is just
   as much "not exporting" as having no endpoint. .NET counts traces + metrics
   only — `Setup` builds exporters for exactly those two, and a LogsEndpoint
   alone is consumed by the ILoggingBuilder extension, so claiming Exporting on
   it would be the same lie in a new place.

2. A loud warning when no endpoint is configured, wording matched to Rust's: it
   names the variable to set AND offers SMOOAI_OBSERVABILITY_DISABLED=true so an
   intentional no-op can be declared rather than inferred from absence.

3. Both halves tested in all four — no endpoint ⇒ flag false, endpoint set ⇒ flag
   true. With only one asserted, an implementation that hard-codes either value
   passes; each assertion was mutation-checked to confirm it fails on its own.

Three tests enshrined the misleading shape and now assert the honest one:
Go's TestBootstrapInstallsClientAndCapture, Python's
test_never_raises_on_bad_config, and .NET's Run_NeverThrows_OnBadConfig all
asserted `installed` while nothing had a destination. Each also pins the warning
now, and each clears the OTEL_EXPORTER_OTLP_* env vars so "no endpoint" means no
endpoint from any source rather than whatever the CI runner happens to export.

Also renames the .NET xUnit collection to OtelGlobalStateCollection — see the
next commit for why it had to grow members.

Gates, by exit code: TS typecheck/lint/test/build/format:check 0 (263 tests);
Go gofmt/vet/test 0 across all three modules; ruff check + format --check 0,
pytest 0 (78 tests); dotnet build/test/format 0 (80 tests).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both are pre-existing on origin/main and unrelated to the bootstrap change — but
the dotnet lane only runs when dotnet/** changes, so this is the PR that has to
face them.

1. `dotnet format --verify-no-changes` exits 2 on CrashChild.cs: 14 WHITESPACE
   errors, a braced switch-case body indented one level short. Verified identical
   on a stashed clean tree. Fixed by running `dotnet format` on that one file —
   pure indentation, no behavior.

2. OtelSetupTests.Setup_IsIdempotent is a flake, and a nasty one: it failed 3 of
   8 full-suite runs on a clean tree (it passes 6 of 6 when the suite is filtered
   down, which is why it hid). ObservabilitySdk._installed is a process-wide
   static and three classes call ResetForTests() on it, but only BootstrapTests
   was in a collection. xUnit parallelizes ACROSS collections, so the other two
   ran concurrently with it and a foreign reset landed between that test's two
   Setup() calls, wiping the install guard the test exists to assert.

   Fixed by putting all three classes in one non-parallel collection
   (OtelGlobalStateCollection, renamed from "Bootstrap" since it guards the OTel
   singleton, not bootstrap). 10 of 10 full-suite runs green afterwards, verified
   by exit code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Aug 15, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: fafaba9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@smooai/observability Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@brentrager
brentrager merged commit 7aecf45 into main Aug 15, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant