Emit physical provider and classifier attempt lifecycles through Relay

[afourniernv]

Problem

The NeMo Relay plugin exposes one logical LLM call plus high-level Switchyard marks and metrics. Those records show that routing happened, which model calls ran, how long routing took, and whether the selected model fell back. They do not create nested timed spans for the work Switchyard performs inside that call.

A routed request can include:

Relay LLM call
  Switchyard algorithm
    classifier or judge call
      candidate model call
        HTTP attempt and retry
    answer candidate
      fallback candidate

Today Relay exports the outer LLM span and the Switchyard summary marks. In a trace viewer, the internal routing work appears as flat records rather than a timed hierarchy. This makes it hard to see where routing time was spent or which retry or fallback caused a delay.

ATIF should continue to describe the normal agent trajectory without these internal calls. This issue is about the OpenTelemetry trace.

Existing instrumentation

Switchyard already emits these spans while running inside the plugin:

• libsy.run for the algorithm run
• libsy.llm_call for classifier and judge requests
• libsy.client_call for model candidates and fallbacks
• libsy.upstream_attempt for HTTP attempts and retries

Together they cover most of the internal timeline, although they are not yet one complete hierarchy. Relay does not receive these spans through its plugin runtime, so its OpenTelemetry exporter cannot include them. The plugin currently reports summaries through Relay marks and metrics instead.

#604 and #612 improve those summaries but do not solve the missing span hierarchy.

Boundary to settle

There are two reasonable approaches:

1. Switchyard exports its existing spans directly to the same collector, using the trace context Relay provides.
2. Relay adds a native plugin API for explicitly parented child spans and remains responsible for exporting them.

Relay currently provides marks, metrics, and stack-based scopes. It does not provide an explicit child-span surface suited to overlapping work or spans that remain open while a response stream is consumed.

The second approach keeps one observability pipeline, but the boundary should be agreed with Relay maintainers before adding another lifecycle implementation to the plugin.

Desired outcome

• Keep the outer Relay LLM span.
• Show Switchyard routing work beneath it in the same trace.
• Distinguish classifier or judge calls from answer calls.
• Show model candidates, fallbacks, HTTP attempts, and retries with their own timing and outcome.
• Close streaming spans exactly once on completion, error, or cancellation.
• Preserve the existing routing marks and metrics.
• Avoid duplicate span export.
• Keep request content and unsafe provider error text out of span fields.
• Add tests for parent-child relationships, ordering, and balanced span lifecycles.

[grahamking]

@afourniernv @bbednarski9 Do you want to propose something for this? I agree we need to do it.

Ideally Switchyard would have a clean way to expose all it's metrics and logs to integrations. We use Prometheus / otel for metrics and tracing / otel for logs. Maybe that's sufficient and Relay can capture that?

[afourniernv]

I dug into this a little more. Switchyard already traces most of this work through libsy.run, libsy.llm_call, libsy.client_call, and libsy.upstream_attempt. Those cover the algorithm run, classifier or judge calls, model candidates and fallbacks, and individual HTTP attempts and retries.

There are two gaps. These spans are not one complete hierarchy today, and the Relay plugin does not forward them into Relay’s trace. The plugin only sends summary marks and metrics. Those marks attach to the surrounding Relay scope, so they do not appear as timed child operations beneath the outer LLM call.

Switchyard could use the traceparent Relay puts on the request and export its spans directly to the same collector. The other option is for Relay’s native plugin API to support explicitly parented child spans. Relay’s current stack-based scopes do not look sufficient for overlapping calls and streams.

I don’t want to add another lifecycle system to the plugin before we settle that boundary. Whichever direction we choose, I think this issue should reuse the tracing Switchyard already has, connect the hierarchy, label routing and answer work clearly, and cover streaming, cancellation, and safe error fields.

#604 and #612 are still useful for the high-level routing marks, but they do not solve the trace hierarchy.

@grahamking @bbednarski9, does that match your read? Would you rather have Switchyard export these spans directly, or should I take the child-span API question to Relay?