Skip to content

feat(agents): add the UiPath Delegate SDK agent as a built-in harness [PILOT-7267] - #153

Draft
Mihaiii wants to merge 1 commit into
mainfrom
chore/move-delegate-sdk
Draft

feat(agents): add the UiPath Delegate SDK agent as a built-in harness [PILOT-7267]#153
Mihaiii wants to merge 1 commit into
mainfrom
chore/move-delegate-sdk

Conversation

@Mihaiii

@Mihaiii Mihaiii commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Issue

PILOT-7267

Summary

  • Add DelegateSdkAgent (agent.type: delegate-sdk) as a built-in harness, moved from the internal coder_eval_uipath plugin: it drives UiPath Autopilot's Delegate agent through the @uipath/delegate-stdio Node host (JSON-Lines over stdio).
  • AgentKind.DELEGATE_SDK + DelegateSdkAgentConfig (sdk_options, project_id, session_id), registered by register_builtins; empty delegate-sdk extra (the host is a Node package, like opencode).
  • Layer-5 sdk_options override guard is now registry-driven: any kind whose config declares sdk_options qualifies (claude-code, delegate-sdk) — replaces the plugin's apply_overrides monkeypatch.
  • Delegate-routed pricing rows (hyphenated ids: virtuoso-1-5, gpt-5-6-terra, kimi-k2-7-code, …) added to pricing.py and mirrored in evalboard/lib/pricing.ts, rates unchanged from the plugin.
  • New docs/agents/DELEGATE_SDK.md; HARNESS_PARITY.md gains a delegate-sdk column; tasks/delegate_sdk_smoke_test.yaml.

Changes

Agent (src/coder_eval/agents/)

  • delegate_sdk_agent.py + _delegate_s2s_token_file.py ported; get_environment_info spreads the base (CE046), system_prompt_semantics = "unknown".
  • The plugin's logging bridge is gone: when DELEGATE_STDIO_VERBOSE forces tracing on a quiet run, host stderr is forwarded at INFO (DEBUG records would be dropped by the app handlers).
  • Auth hint updated from the removed delegate-sdk login bin to npx @uipath/delegate-cli login.

Pricing caveat

  • Rows carried over verbatim so the move is behavior-preserving. Several differ from the dotted rows for the same physical model (cache-write treatment); documented in pricing.py, deliberately not reconciled here.

Testing

  • ruff format/check, make lint (389 passed), pyright (only the pre-existing missing-extra errors), ported suites + override/registry/pricing tests (343 passed); full suite green apart from pre-existing Windows/optional-extra failures.
  • Pricing parity verified with a node mirror of pricing-parity.test.ts (no pnpm locally for vitest).
  • Live host tests (-m live) not run.

Related PRs

Sequencing: this releases first (a feat → 0.12.0). The skills nightly pin and both PRs above depend on that version, and the nightly also needs @uipath/delegate-stdio published to npmjs (manual, not yet done).

🤖 Generated with Claude Code

… [PILOT-7267]

Move `DelegateSdkAgent` (agent.type `delegate-sdk`) out of the internal
coder_eval_uipath plugin into core: it drives UiPath Autopilot's Delegate agent
through the `@uipath/delegate-stdio` Node host (JSON-Lines over stdio), with the
S2S token-file refresher, per-generation transcript reconstruction, stall
recovery and failure classification carried over intact.

- `AgentKind.DELEGATE_SDK`, `DelegateSdkAgentConfig` (sdk_options / project_id /
  session_id), registered via `register_builtins`; empty `delegate-sdk` extra.
- The layer-5 `sdk_options` override guard is now registry-driven: any kind whose
  config declares `sdk_options` qualifies (claude-code + delegate-sdk today).
- Delegate-routed pricing rows (hyphenated ids) added to pricing.py and mirrored
  in evalboard/lib/pricing.ts, verbatim from the plugin.
- Host stderr tracing forwards at INFO when DELEGATE_STDIO_VERBOSE forces it on a
  quiet run (replaces the plugin's logging bridge).
- docs/agents/DELEGATE_SDK.md, parity table column, smoke task, ported test suites.

Related PRs: UiPath/coder_eval_uipath (drops the plugin agent, nightly on public
npm) and UiPath/Autopilot (delegate-stdio 0.1.3 public-npm prep).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant