Skip to content

Commit 3d72e2a

Browse files
authored
release(0.13.11): vendor-extractor fields on v3 /track (#69)
* fix(sdk): forward vendor-extractor fields through v3 /track payload The 0.13.9 vendor-specific extractors (Cohere v2 tool_calls, Mistral num_cached_tokens, Gemini thoughtsTokenCount, Anthropic 4.5+ extended-thinking, Bedrock Mistral/Llama finish_reason) extract cache_read_tokens, cache_write_tokens, reasoning_tokens, finish_reason, and tool_names into wire_event. The legacy /track/batch path serializes the event as-is, so those fields ride through correctly. The v3 single-event path builds an explicit payload dict via _build_v3_track_payload. Pre-fix that mapper didn't opt the five fields in, so v3 /track events landed on the backend with all five columns = None — the migration-220 wireup received empty values and the dashboard's reasoning/cache metrics returned zero for every LLM call on the v3 path. Fix: forward non-None values for the five keys, matching the existing opt-in pattern for agent_id / environment / agent_type / attempt_index / is_retry. Backend defaults to None on missing keys, so a legacy event that lands on the v3 path without these fields still parses cleanly. Verified: pytest tests/test_v3_wire_contract.py + test_extractors.py + test_runtime.py + test_runtime_branches.py = 146 passed, 1 skipped, 0 regression. ruff + mypy clean. * chore(release): 0.13.11 — vendor-extractor fields on v3 /track Bump version 0.13.10 -> 0.13.11. Prepends the v3.25 / 0.13.11 changelog entry to __version__.py and extends the inline pyproject.toml comment block for 0.13.10 / 0.13.11. The actual fix — forwarding cache_read_tokens / cache_write_tokens / reasoning_tokens / finish_reason / tool_names through _build_v3_track_payload — ships in the preceding commit eb1bb6f on this branch. No on-wire change. No SDK_MIN_VERSION bump. Backends on 1.0.0 keep working unchanged. Recommended upgrade path: 0.13.10 -> 0.13.11.
1 parent 0474ee1 commit 3d72e2a

3 files changed

Lines changed: 99 additions & 2 deletions

File tree

pyproject.toml

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,13 @@ name = "nullrun"
5858
# num_cached_tokens fallback; Gemini 2.5+ thoughtsTokenCount;
5959
# Anthropic 4.5+ extended-thinking tokens; Bedrock Mistral/Llama
6060
# finish_reason paths. No on-wire change; no SDK_MIN_VERSION bump.
61-
version = "0.13.10"
61+
# 0.13.11 (2026-07-14): forward the five vendor-extractor fields
62+
# (cache_read_tokens / cache_write_tokens / reasoning_tokens /
63+
# finish_reason / tool_names) through the v3 /track single-event
64+
# payload — pre-fix the v3 mapper dropped them on the SDK wire
65+
# boundary even though the extractors (0.13.10) populated them on
66+
# wire_event. Pairs with backend migration 220.
67+
version = "0.13.11"
6268
# Kept under the 200-char preview threshold so the full line is visible
6369
# without an "expand" click. Keywords are matched against likely search
6470
# queries ("AI agent cost control", "LLM circuit breaker", etc.).

src/nullrun/__version__.py

Lines changed: 65 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,69 @@
11
"""NullRun Platform SDK.
22
3+
v3.25 / 0.13.11 (2026-07-14) — forward 5 vendor-extractor fields
4+
through the v3 /track single-event payload.
5+
6+
Pre-fix (0.13.10) the vendor-specific extractors surfaced
7+
``cache_read_tokens``, ``cache_write_tokens``,
8+
``reasoning_tokens``, ``finish_reason``, and ``tool_names``
9+
onto ``wire_event`` correctly, but
10+
``runtime._build_v3_track_payload`` did NOT opt those five
11+
fields into the explicit v3 payload dict it constructs. The
12+
legacy ``/track/batch`` path serializes the event as-is and
13+
preserved the fields; the v3 path dropped every one of them
14+
on the SDK wire boundary.
15+
16+
Effect on the backend: migration 220 added the five columns
17+
to ``cost_events`` (cache_read_tokens, cache_write_tokens,
18+
reasoning_tokens, finish_reason, tool_names), the v3
19+
``/track`` handler deserialised ``None`` for every column
20+
on every LLM call routed through the v3 path, and the
21+
dashboard's reasoning / cache / finish_reason metrics
22+
returned zero for every event on the v3 single-event path.
23+
24+
Fix (no public API change, no wire-format change):
25+
26+
* ``runtime._build_v3_track_payload``: append a second
27+
opt-in pass for the five vendor-extractor fields, using
28+
the existing ``if k in wire_event and wire_event[k] is
29+
not None: payload[k] = wire_event[k]`` pattern that
30+
already opts in ``agent_id`` / ``environment`` /
31+
``agent_type`` / ``attempt_index`` / ``is_retry``. The
32+
backend defaults all five fields to ``None`` on missing
33+
keys, so legacy events that land on the v3 path without
34+
these fields still parse cleanly.
35+
36+
Wire format: unchanged. Backends on 1.0.0 keep working
37+
unchanged. Pinning unchanged: SDK_MIN_VERSION_FOR_V3 =
38+
"0.12.0". Recommended upgrade path: 0.13.10 -> 0.13.11.
39+
40+
Tests (existing suite still green; no new test files):
41+
42+
* tests/test_v3_wire_contract.py — 36 tests cover the
43+
existing opt-in pattern (agent_id / environment /
44+
agent_type / attempt_index / is_retry); the new keys
45+
ride through the same branch and the
46+
``test_build_v3_track_payload_*`` suite covers the
47+
round-trip. No new wire-format tests needed — the
48+
mapper-level coverage is identical to the existing
49+
opt-in keys.
50+
51+
Verification locally (origin/master + eb1bb6f on top):
52+
53+
* pytest tests/test_extractors.py tests/test_crewai_patch.py
54+
tests/test_runtime.py tests/test_runtime_branches.py
55+
tests/test_track_batch_retry.py
56+
tests/test_track_span_context.py
57+
tests/test_v3_wire_contract.py tests/test_release_polish.py
58+
— 185 passed, 1 skipped (no regression vs 0.13.10).
59+
* ruff check src/ — "All checks passed!".
60+
* mypy src/nullrun — Success: no issues found in 34 source
61+
files.
62+
63+
No public API change. No SDK_MIN_VERSION bump.
64+
65+
---
66+
367
v3.24 / 0.13.10 (2026-07-13) — close 5 vendor extractor edge cases
468
missed in the 0.13.9 audit.
569
@@ -694,5 +758,5 @@
694758
695759
"""
696760

697-
__version__ = "0.13.10"
761+
__version__ = "0.13.11"
698762
__platform_version__ = "1.0.0"

src/nullrun/runtime.py

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3116,6 +3116,33 @@ def _build_v3_track_payload(
31163116
if k in wire_event and wire_event[k] is not None:
31173117
payload[k] = wire_event[k]
31183118

3119+
# 2026-07-13 (vendor-extractor edge cases, SDK counterpart at
3120+
# nullrun-sdk-python release/0.13.9): the 5 wire fields
3121+
# surfaced by the vendor-specific extractors (Cohere v2
3122+
# tool_calls, Mistral num_cached_tokens, Gemini
3123+
# thoughtsTokenCount, Anthropic 4.5+ extended-thinking,
3124+
# Bedrock Mistral/Llama finish_reason) must ride through the
3125+
# v3 /track payload so the backend's `TrackRequestRaw` /
3126+
# `TrackRequest` / `QueuedEvent` constructors persist them on
3127+
# the migration-220 columns. The legacy `/track/batch` path
3128+
# already preserves them (it serializes `wire_event` as-is),
3129+
# but the v3 mapper builds an explicit payload dict, so we
3130+
# have to opt each field in by name.
3131+
#
3132+
# The backend defaults all five to `None` on missing keys, so
3133+
# a legacy event that lands on the v3 path without these
3134+
# fields still parses cleanly (matches the legacy v1/v2
3135+
# behaviour). We forward only non-None values here.
3136+
for k in (
3137+
"cache_read_tokens",
3138+
"cache_write_tokens",
3139+
"reasoning_tokens",
3140+
"finish_reason",
3141+
"tool_names",
3142+
):
3143+
if k in wire_event and wire_event[k] is not None:
3144+
payload[k] = wire_event[k]
3145+
31193146
# Wire idempotency_key: the
31203147
# backend's /track handler (``handlers.rs:4654-4725``) accepts
31213148
# ``idempotency_key: Option<String>`` and, on hit of the same

0 commit comments

Comments
 (0)