Skip to content

otel: make Go/Python/TS/.NET tool spans survive ingest, and stop the zero cost - #490

Merged
brentrager merged 1 commit into
mainfrom
polyglot-telemetry-parity
Aug 18, 2026
Merged

otel: make Go/Python/TS/.NET tool spans survive ingest, and stop the zero cost#490
brentrager merged 1 commit into
mainfrom
polyglot-telemetry-parity

Conversation

@brentrager

Copy link
Copy Markdown
Contributor

Epic th-9e792d parity work, pearl th-73c8b5. Ports the Rust fixes (#471, #474, #488) to the other four engines. Prod is Rust-only, so this is correctness-for-dogfooding — nothing customer-facing was affected.

Tool spans were being discarded at ingest, in all four

The api-prime OTLP ingest builds a span's attribute set from the resource attrs plus that span's own, with no inheritance from the parent, and its LLM-event gate keys on gen_ai.system. None of the four set it on the tool span, so those spans never became rows — exactly what happened to Rust's for their entire existence (zero rows with operation_name = 'tool', all time).

Tool spans now carry their own gen_ai.system, gen_ai.operation.name, gen_ai.conversation.id, and smooai.org_id where the engine has one.

gen_ai.operation.name is spelled exactly chat / tool. The ingest takes the attribute verbatim when present and only derives it from the span name as a fallback, so execute_tool would land in the column and match nothing.

Cost reaches the span for the first time in all four

gen_ai.usage.cost_usd when positive, smooai.gen_ai.cost_unavailable = unpriced otherwise. Never a bare 0 — the gateway answers 0 for a model it has no price for, so a zero means "not measured", never "free".

Two engine-specific bugs found while wiring, both fixed

  • .NET published input_tokens = 0. It guarded only on sawUsage, which a usage chunk carrying null counts sets while both totals resolve to 0 via ?? 0. The other three guard on the counts themselves.
  • .NET was the one engine carrying a literal new TurnUsage(0, ...) on the cost fallback path. Its own XML doc admitted "0 means 'nothing priced it', not 'free'" while nothing downstream could tell the difference. That zero no longer reaches the span.

That second one is a fifth instance of the fabricate-a-plausible-value habit this workstream keeps turning up, after the seed org, the fresh UUID, content.len()/4 tokens, and the unpriced 0.

The fabrication was deliberately not ported for parity's sake. Parity means the same honest contract — absent where the value is unknown — not the same lies.

Verification

Each engine's telemetry test asserts the new contract, and each fails without the change (tool-span identifiers reverted in all four, run, restored):

engine failure
Go telemetry_test.go:121: span missing attribute "gen_ai.system" (want "smooth-operator")
Python assert chat.attributes[telemetry.GEN_AI_SYSTEM] == telemetry.SYSTEM_NAME
TypeScript PASS (2) FAIL (1)
.NET StreamingTurnEmitsGenAiSpansWithModelAndToolArgs [FAIL]

Full suites, all green:

engine result
Go 2 packages ok, go vet clean
Python 364 passed
TypeScript 357 passed / 38 files, tsc --noEmit clean
.NET 575 passed, 0 failed, across 5 assemblies

Cost assertions are written to the branch each engine actually takes rather than an assumed one — Go's mock turn is priced by local ModelPricing, so there it asserts the cost lands and the marker does not; .NET's scripted turn is unpriced, so it asserts the inverse. My first Go assertion assumed unpriced and failed; I fixed the assertion, not the code.

Incidental

python/server/uv.lock gets a 1-line correction: it recorded the local package at 1.52.3 while pyproject.toml says 1.54.1 — stale on main, re-stamped deterministically by uv sync.

Not in this PR

Usage/cost provenance (usage_estimated / cost_estimated) and gen_ai.response.id need the same engine support the Rust core got in 1.10.0; the other four cores don't have it. Until then those engines cannot distinguish a measured token count from an estimated one, and have no join key to LiteLLM_SpendLogs. Recorded in PARITY-STATUS.md as the one remaining Rust-only row, with footnotes explaining both the ingest rule and the provenance gap, and tracked in th-73c8b5.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LkCz96UfUxcai5RU4LwPnG

…zero cost

All four non-Rust engines had every defect Rust had, unchanged. Prod is
Rust-only so nothing customer-facing was affected, but anyone dogfooding
`th operator serve --lang go` got the same silent data loss.

**Tool spans were being discarded at ingest.** The OTLP ingest builds a
span's attribute set from the resource attrs plus THAT span's own, with no
inheritance from the parent, and its LLM-event gate keys on `gen_ai.system`.
None of the four set it on the tool span — so those spans never became rows,
exactly as Rust's didn't for their entire existence. They now carry their own
`gen_ai.system`, `gen_ai.operation.name`, `gen_ai.conversation.id` and (where
the engine has one) `smooai.org_id`.

`gen_ai.operation.name` is spelled exactly `chat` / `tool`. The ingest takes
the attribute verbatim when present and only derives it from the span name as
a fallback, so `execute_tool` would land in the column and match nothing.

**Cost reaches the span for the first time in all four**, as
`gen_ai.usage.cost_usd` when positive and `smooai.gen_ai.cost_unavailable` =
`unpriced` otherwise. Never a bare 0: the gateway answers 0 for a model it has
no price for, so a zero means "not measured", never "free".

Two engine-specific bugs found while wiring this, both fixed:

- **.NET published `input_tokens = 0`.** It guarded only on `sawUsage`, which
  a usage chunk carrying null counts sets while both totals resolve to 0 via
  `?? 0`. The other three guard on the counts. Now so does .NET.
- **.NET was the one engine with a literal `new TurnUsage(0, ...)`** on the
  cost fallback path — its own XML doc admitted "0 means 'nothing priced it',
  not 'free'" while nothing downstream could tell. The span no longer carries
  that zero.

Per instruction, the fabrication was NOT ported for parity's sake: parity
means the same honest contract — absent where the value is unknown — not the
same lies.

Each engine's telemetry test asserts the new contract, and each fails without
the change: Go `span missing attribute "gen_ai.system"`, Python on the same
attribute, TS `FAIL (1)`, .NET
`StreamingTurnEmitsGenAiSpansWithModelAndToolArgs [FAIL]`.

Go 2 packages ok · Python 364 passed · TypeScript 357 passed / 38 files ·
.NET 575 passed across 5 assemblies. go vet and tsc --noEmit clean.

`python/server/uv.lock` picks up a 1-line correction: the lock recorded the
local package at 1.52.3 while pyproject says 1.54.1, i.e. it was stale on main.

Not in this PR: usage/cost provenance and `gen_ai.response.id`, which need the
same engine support the Rust core got in 1.10.0. Recorded in PARITY-STATUS.md
as the one remaining Rust-only row, and in th-73c8b5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LkCz96UfUxcai5RU4LwPnG
@changeset-bot

changeset-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 075d41a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@brentrager
brentrager merged commit 89fc03b into main Aug 18, 2026
7 checks passed
brentrager added a commit that referenced this pull request Aug 18, 2026
Fifth in a row on this ticket to merge without one (#470, #471, #474, #488,
#490). Adding it so the parity work actually publishes.


Claude-Session: https://claude.ai/code/session_01LkCz96UfUxcai5RU4LwPnG

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant