Skip to content

feat(libsy): export outcome evidence as JSON - #660

Closed
nachiketb-nvidia wants to merge 3 commits into
mainfrom
nachiketb/feat-json-outcome-evidence
Closed

feat(libsy): export outcome evidence as JSON#660
nachiketb-nvidia wants to merge 3 commits into
mainfrom
nachiketb/feat-json-outcome-evidence

Conversation

@nachiketb-nvidia

@nachiketb-nvidia nachiketb-nvidia commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

What

Export optional outcome evidence as one JSON string attribute, evidence, on the existing libsy.run span.

Why

The current seven-field mapping drops custom keys and nested data. Algorithms should be able to define their evidence without changing the telemetry implementation.

How

  • Replace the evidence.* fields and mapping loops with one JSON display value.
  • Leave absent evidence unset.
  • Keep outcome IDs, selected model IDs, metrics, routing behavior, and Rust/Python outcome values unchanged.
  • Update existing telemetry tests and concise Rust docstrings. No guide changes, new dependencies, or new tests.

Consumer example

The span attribute is a string containing JSON:

{"source":"llm-classifier","score":0.9,"custom":{"accepted":true}}

Consumers decode it:

evidence = json.loads(span.attributes["evidence"])

What to review

This replaces the previous evidence.source, evidence.score, and other individual fields. Queries must parse evidence instead.

All custom evidence is now exported without filtering or redaction. Callers must keep it small and free of private data. Host/backend string limits may truncate the JSON. Existing request-metadata and error handling are unchanged.

This is separate from the docs-only PR #659 and based on main. Its new reference page will need matching wording when both changes land.

Validation

  • Existing successful-outcome test passed on main before the change.
  • All 14 observability integration tests passed, including JSON round-tripping, nested fields, escaped text, absence of old attributes, and missing evidence.
  • cargo clippy --workspace --all-targets -- -D warnings passed.
  • cargo fmt --all --check and git diff --check passed.
  • No Python tests or live provider calls.

Scope: 3 Rust files, +38 / -50 lines.

Signed-off-by: nachiketb <nachiketb@nvidia.com>
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

🚀 View preview at
https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-660/

Built to branch gh-pages at 2026-09-09 23:46 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

Signed-off-by: nachiketb <nachiketb@nvidia.com>
Signed-off-by: nachiketb <nachiketb@nvidia.com>
@nachiketb-nvidia
nachiketb-nvidia deleted the nachiketb/feat-json-outcome-evidence branch September 9, 2026 23:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant