otel: make Go/Python/TS/.NET tool spans survive ingest, and stop the zero cost - #490
Merged
Conversation
…zero cost All four non-Rust engines had every defect Rust had, unchanged. Prod is Rust-only so nothing customer-facing was affected, but anyone dogfooding `th operator serve --lang go` got the same silent data loss. **Tool spans were being discarded at ingest.** The OTLP ingest builds a span's attribute set from the resource attrs plus THAT span's own, with no inheritance from the parent, and its LLM-event gate keys on `gen_ai.system`. None of the four set it on the tool span — so those spans never became rows, exactly as Rust's didn't for their entire existence. They now carry their own `gen_ai.system`, `gen_ai.operation.name`, `gen_ai.conversation.id` and (where the engine has one) `smooai.org_id`. `gen_ai.operation.name` is spelled exactly `chat` / `tool`. The ingest takes the attribute verbatim when present and only derives it from the span name as a fallback, so `execute_tool` would land in the column and match nothing. **Cost reaches the span for the first time in all four**, as `gen_ai.usage.cost_usd` when positive and `smooai.gen_ai.cost_unavailable` = `unpriced` otherwise. Never a bare 0: the gateway answers 0 for a model it has no price for, so a zero means "not measured", never "free". Two engine-specific bugs found while wiring this, both fixed: - **.NET published `input_tokens = 0`.** It guarded only on `sawUsage`, which a usage chunk carrying null counts sets while both totals resolve to 0 via `?? 0`. The other three guard on the counts. Now so does .NET. - **.NET was the one engine with a literal `new TurnUsage(0, ...)`** on the cost fallback path — its own XML doc admitted "0 means 'nothing priced it', not 'free'" while nothing downstream could tell. The span no longer carries that zero. Per instruction, the fabrication was NOT ported for parity's sake: parity means the same honest contract — absent where the value is unknown — not the same lies. Each engine's telemetry test asserts the new contract, and each fails without the change: Go `span missing attribute "gen_ai.system"`, Python on the same attribute, TS `FAIL (1)`, .NET `StreamingTurnEmitsGenAiSpansWithModelAndToolArgs [FAIL]`. Go 2 packages ok · Python 364 passed · TypeScript 357 passed / 38 files · .NET 575 passed across 5 assemblies. go vet and tsc --noEmit clean. `python/server/uv.lock` picks up a 1-line correction: the lock recorded the local package at 1.52.3 while pyproject says 1.54.1, i.e. it was stale on main. Not in this PR: usage/cost provenance and `gen_ai.response.id`, which need the same engine support the Rust core got in 1.10.0. Recorded in PARITY-STATUS.md as the one remaining Rust-only row, and in th-73c8b5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LkCz96UfUxcai5RU4LwPnG
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Epic th-9e792d parity work, pearl th-73c8b5. Ports the Rust fixes (#471, #474, #488) to the other four engines. Prod is Rust-only, so this is correctness-for-dogfooding — nothing customer-facing was affected.
Tool spans were being discarded at ingest, in all four
The api-prime OTLP ingest builds a span's attribute set from the resource attrs plus that span's own, with no inheritance from the parent, and its LLM-event gate keys on
gen_ai.system. None of the four set it on the tool span, so those spans never became rows — exactly what happened to Rust's for their entire existence (zero rows withoperation_name = 'tool', all time).Tool spans now carry their own
gen_ai.system,gen_ai.operation.name,gen_ai.conversation.id, andsmooai.org_idwhere the engine has one.gen_ai.operation.nameis spelled exactlychat/tool. The ingest takes the attribute verbatim when present and only derives it from the span name as a fallback, soexecute_toolwould land in the column and match nothing.Cost reaches the span for the first time in all four
gen_ai.usage.cost_usdwhen positive,smooai.gen_ai.cost_unavailable=unpricedotherwise. Never a bare0— the gateway answers 0 for a model it has no price for, so a zero means "not measured", never "free".Two engine-specific bugs found while wiring, both fixed
input_tokens = 0. It guarded only onsawUsage, which a usage chunk carrying null counts sets while both totals resolve to0via?? 0. The other three guard on the counts themselves.new TurnUsage(0, ...)on the cost fallback path. Its own XML doc admitted "0 means 'nothing priced it', not 'free'" while nothing downstream could tell the difference. That zero no longer reaches the span.That second one is a fifth instance of the fabricate-a-plausible-value habit this workstream keeps turning up, after the seed org, the fresh UUID,
content.len()/4tokens, and the unpriced0.The fabrication was deliberately not ported for parity's sake. Parity means the same honest contract — absent where the value is unknown — not the same lies.
Verification
Each engine's telemetry test asserts the new contract, and each fails without the change (tool-span identifiers reverted in all four, run, restored):
telemetry_test.go:121: span missing attribute "gen_ai.system" (want "smooth-operator")assert chat.attributes[telemetry.GEN_AI_SYSTEM] == telemetry.SYSTEM_NAMEPASS (2) FAIL (1)StreamingTurnEmitsGenAiSpansWithModelAndToolArgs [FAIL]Full suites, all green:
ok,go vetcleantsc --noEmitcleanCost assertions are written to the branch each engine actually takes rather than an assumed one — Go's mock turn is priced by local
ModelPricing, so there it asserts the cost lands and the marker does not; .NET's scripted turn is unpriced, so it asserts the inverse. My first Go assertion assumed unpriced and failed; I fixed the assertion, not the code.Incidental
python/server/uv.lockgets a 1-line correction: it recorded the local package at 1.52.3 whilepyproject.tomlsays 1.54.1 — stale on main, re-stamped deterministically byuv sync.Not in this PR
Usage/cost provenance (
usage_estimated/cost_estimated) andgen_ai.response.idneed the same engine support the Rust core got in 1.10.0; the other four cores don't have it. Until then those engines cannot distinguish a measured token count from an estimated one, and have no join key toLiteLLM_SpendLogs. Recorded inPARITY-STATUS.mdas the one remaining Rust-only row, with footnotes explaining both the ingest rule and the provenance gap, and tracked in th-73c8b5.🤖 Generated with Claude Code
https://claude.ai/code/session_01LkCz96UfUxcai5RU4LwPnG