fix: stop reporting an AI-trace stats failure as 0ms latency - #320
Open
FrameAutomata wants to merge 1 commit into
Open
fix: stop reporting an AI-trace stats failure as 0ms latency#320FrameAutomata wants to merge 1 commit into
FrameAutomata wants to merge 1 commit into
Conversation
`GetTraceNameStats` on the SQLite telemetry backend swallowed the error from its percentile query with `return stats, nil`. On any failure -- context deadline on a large trace_name, a locked telemetry DB, a scan type error -- the caller got a struct with a real Count, AvgDuration and token totals next to MedianDuration and P95Duration of exactly zero, and nothing was logged or captured. Zero is a plausible latency, so this reads as a fast trace rather than a failed read. The DuckDB and ClickHouse backends compute the percentiles inside the aggregate query and propagate the error, and the SQLite endpoint and task equivalents already `return nil, err`, so this was the one path that fabricated a value. The route treats stats as best-effort -- it nils them and still answers 200 with the trace list -- so propagating the error does not turn this into a 500. It swaps a misleading zero for an absent stats block, and the discard in the controller now reports through CaptureException per the non-stopping-error convention rather than vanishing. No regression test: both queries in this function read the same table, so there is no way to fail only the percentile read from a test without injecting a fake executor, which the repositories do not support. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while reviewing #319. Pre-existing, unrelated to that PR's percentile work, so it's here on its own.
The bug
GetTraceNameStatson the SQLite telemetry backend swallowed the error from its percentile query:On any failure — context deadline on a large
trace_name, a locked telemetry DB, a scan type error — the caller got a struct with a realCount,AvgDurationand token totals sitting next toMedianDurationandP95Durationof exactly zero. Nothing logged, noCaptureException.Zero is a plausible latency. An operator reads "median 0ms, p95 0ms" as a fast trace, not a failed read.
It's also a backend divergence. DuckDB (
duckdb/ai_trace.repository.go:386) and ClickHouse compute the percentiles inside the aggregate query and propagate the error withreturn nil, err. The SQLite endpoint and task equivalents alreadyreturn nil, errtoo. This was the single path that fabricated a value.The fix
return nil, err.Worth noting what this does not do: the route already treats stats as best-effort — it nils them and still answers
200with the trace list — so propagating the error is not a new 500 path. It swaps a misleading zero for an absent stats block. The caller's discard now goes throughtraceway.CaptureExceptionper CLAUDE.md's non-stopping-error convention instead of vanishing.Trade-off
On failure the caller now loses the fields that did read successfully (count, avg, tokens, cost) rather than getting them alongside fake zeros. That's deliberate — those values are already visible in the trace list rows the same response carries, and a partially-true stats header is worse than none.
No regression test, and why
Both queries in the function read
ai_traces, so there is no way to fail only the percentile read from a test without injecting a fake executor — the repositories takedb.TelemetryDBdirectly and have no seam for it. Dropping the table fails the first aggregate query too, which already returnednil, errbefore this change, so such a test would pass identically pre- and post-fix and assert nothing. Flagging that plainly rather than shipping a test that looks like coverage.Verification
go build ./...,go vet ./app/...,go test -race -count=1 ./app/..., and the DuckDB-tagged repository suite all pass.gofmtclean.🤖 Generated with Claude Code