Agentic Extensions for Unity CLI (UAX)

Extends the Unity Pipeline package (com.unity.pipeline) with the agent-oriented editor commands it lacks. Everything here is invoked the same way as the built-in Pipeline commands, through the unity CLI against a running editor:

unity cmd get_frame_timing --project-path "<project root>" --format json --quiet --no-banner

Alongside the commands, the package ships three things an agent needs that Pipeline doesn't provide: an agent skill that teaches the CLI's grammar and its gotchas, a coordination server (UAC-MCP) for when several agents share one editor, and an Agent Dashboard window for watching all of it from inside Unity.

Requirements

• Unity 6000.x
• com.unity.pipeline (0.3.x)
• com.unity.nuget.newtonsoft-json
• com.unity.inputsystem, optional; enables simulate_input (compiled in via version define when the package is present)

Installation

Via Unity Package Manager (Git URL)

1. Open Window > Package Manager
2. Click the + button in the top-left
3. Select Add package from git URL...
4. Enter:

https://github.com/TomMoore515/UnityCLI.AgenticExtensions.git"

Manual Installation

Copy the UnityCLI.AgenticExtensions folder into your project's Assets directory.

Setup

On the first domain load the package installs its agent skill (currently this works for Claude Code and Codex), and starts the UAC-MCP server and writes the client configs. There is nothing else to set up.

Agent skill

An agent that has never driven the unity CLI gets the argument grammar wrong: the positional-vs-flag rule, bare boolean flags, JSON in flag values, PowerShell's --% token, and the fact that unity status under-reports running editors. The packaged skill documents all of it.

UaxSkillInstaller runs at [InitializeOnLoad] and copies Skills~/unity-cli/SKILL.md into the consumer project:

• .claude/skills/unity-cli/SKILL.md (Claude Code)
• .codex/skills/unity-cli/SKILL.md (Codex)
• a marker-delimited <!-- UAX:BEGIN --> block in AGENTS.md, for harnesses that don't load skills automatically

The packaged file is the single source of truth. Destinations get rewritten only when the content differs, and everything outside the AGENTS.md markers survives byte for byte. To force a refresh, use Pipeline ▸ UAX ▸ Reinstall Agent Skill.

Commit the generated files. They belong to the project, not to the package.

UAC: multi-agent coordination

Two agents driving the same editor collide. One enters Play Mode while the other is halfway through a test run, both kick off a compile, a macro fires during a profiler capture. UAC is a second, small MCP server hosted inside the editor at http://127.0.0.1:<port>/mcp where agents can see each other and keep out of each other's way. It exposes coordination tools only; editor actions still go through the unity CLI.

MCP tool	Purpose
whos_here	Sessions connected to this editor: callsigns, idle time, held resources, last status.
post_status	Post an intent line to the shared board that every agent on this editor sees.
read_board	Page the board: status posts plus system events (claims, sessions).
hold	Advisory single-holder claim on a named resource (play_mode, test_runner, macro_runner, compile, or any name your team agrees on). Leases out after 300s by default, refreshes on every tool call, and can queue with waitSeconds.
release	Release a resource you hold.

Claims are advisory. A conflicting hold fails and names the holder instead of blocking the work. Recent board activity rides along in every tool response as a _coordination digest, so agents hear about each other without polling. Sessions, claims, and the board all survive domain reloads, idle sessions are reaped after 15 minutes by default, and ending a session releases its claims.

The port comes from hashing the project path into the range 47100 to 47899, then gets pinned in ProjectSettings/UacSettings.json, so a project keeps the same port and several editors can run side by side. MPPM virtual-player clones never bind. With writeConfigs on, UAX maintains a uac entry in the project's .mcp.json and .codex/config.toml and leaves unrelated entries alone.

The editor menu items live under Pipeline ▸ UAC: Start Server, Stop Server, Copy Server URL, Write Client Configs. An agent can do the same over the CLI and bootstrap coordination without touching the UI:

Command	Purpose
uac_status	Running, port, URL, autostart, and whether the configs are written.
uac_start	Start the server and write client configs. Idempotent.
uac_stop	Stop the server. Claims and the board persist.
uac_write_configs	Write or refresh the uac entries in .mcp.json and .codex/config.toml.

Agent Dashboard

Pipeline ▸ Agent Dashboard opens a UI Toolkit window that refreshes about once a second while it is visible. The Servers tab shows Pipeline and UAC-MCP status with start/stop and config controls, Commands lists every discovered [CliCommand] and registered macro, and Coordination shows the live sessions, claims, and board.

Command families

Profiling: is it slow, what is slow, where is it slow, what leaks?

Command	Purpose
list_profiler_counters	Enumerate available ProfilerRecorder counters (filterable). More appear once subsystems run or in Play Mode.
sample_profiler_counters	Sample up to 16 counters over N frames; min/avg/max/last per counter, time counters in milliseconds.
get_frame_timing	CPU total / main / render thread / GPU frame times over N frames.
get_top_profiler_markers	Capture frames with the profiler and return the top main-thread markers by self-time, with GC alloc per marker.
get_profiler_call_tree	Capture frames and return the call tree for one thread (main by default), preserving hierarchy, with self/total ms, calls, and GC alloc per node.
capture_memory_snapshot	Live memory summary, native-object histogram, and a .snap written to <project>/MemoryCaptures for the Memory Profiler window. Call twice and compare.

Sampling commands are async: the HTTP response is held open while frames are captured. Pass a larger --timeout to the CLI for big frame counts.

UI Toolkit: see and drive runtime UITK panels

Command	Purpose
get_ui_tree	Runtime UI Toolkit visual trees (types, names, classes, text, rects, visibility). UITK elements are invisible to get_scene_hierarchy.
click_ui	Interact with UITK elements by name/class/text query: click (synthesized pointer events), submit, focus, set_value.

Input: real game input in Play Mode

Command	Purpose
simulate_input	Virtual Input System devices: key press/hold/tap, mouse move/buttons/scroll, sequenced steps. Routes input to the game view while the editor is unfocused.

Scene camera: aim before you screenshot

Command	Purpose
set_scene_camera	frame a GameObject (renderer bounds) or look (pivot/euler/size), then capture with the built-in capture_scene_view.

Macros: game-registered actions

The package ships no macros of its own. A project implements IUaxMacro and registers it with UaxMacroRegistry. The macro host lives in the runtime assembly, so macros work against a development PLAYER as well as the editor: register from a development-build-gated runtime assembly's [RuntimeInitializeOnLoadMethod] (players + play mode), plus [InitializeOnLoadMethod] under #if UNITY_EDITOR if list_macros should answer in Edit Mode too. The registry resets on each domain reload and overwrites on re-register, so registration has to be re-runnable (and doubled registration is harmless).

Command	Purpose
list_macros	List the game-registered macros (name, description, params schema).
run_macro	Start a macro and return a jobId immediately. One macro active at a time.
macro_status	Poll a macro job, or the most recent one. waitSeconds blocks server-side for up to 25s.
stop_macro	Request cancellation; the macro observes it at its next frame yield.

Macros use trigger-then-poll, like the built-in bake_lighting and run_tests, because the Pipeline server handles one request at a time. A long-running synchronous command would block its own status polls.

Plugins: who contributed which commands

unity list shows every command as built-in, with no way to tell which package added what or how it expects to be used. UaxPluginRegistry.RegisterPlugin, called from [InitializeOnLoad], records a package's id, version, command names, and model-facing usage guidance. Registration is metadata only; Pipeline discovers the commands either way.

Command	Purpose
list_plugins	List registered UAX plugins with their commands and usage instructions. Run once per project; UAX itself is the first entry.