REST/CLI/MCP alignment: coverage gaps and cross-surface inconsistencies

[JArmandoAnaya]

Scope

REST (visionset.server) is the most complete surface. This reviews how far CLI
(visionset.cli) and MCP (visionset.mcp) have drifted from it, and whether the
kernel-purity / sibling-independence boundaries still hold. Findings below are
ranked by severity; each has a file:line pointer and a repro.

Architecture boundary: holding. uv run lint-imports passes all 4 contracts
(kernel purity, sibling independence, jobs below surfaces, inference below
surfaces) across 243 files / 1600 dependencies, uv run pytest tests/architecture
passes (59 tests), and no delivery package imports another — CLI starts the server
by import string (cli/server.py:50) and MCP as a subprocess module
(cli/mcp.py:51,107), never by importing its internals. openapi.json and
docs/content/mcp-tools.md are both current against the live routes/tools.

Findings

1. Dataset change-log actor is inconsistent per surface. REST's
   promote(batch_id) (server/routes/batches.py:727) and its dataset-asset
   removal (routes/datasets.py:162) write actor: null; CLI writes "cli"
   (cli/batches.py:105,417,443); MCP writes "mcp"
   (mcp/batches.py:80,661,779). REST has an authenticated identity available
   and still writes null. Repro: promote one batch through each surface, then
   GET /datasets/{id}/changes — three different actor shapes for the same
   kind of event. Direction: pick the actor value in one place (kernel default,
   or REST supplying the token identity) so the log is comparable across
   surfaces.

2. An MCP tool description points at a tool that doesn't exist.
   create_inference_connection's provider_id field says "as list_providers
   names them" (mcp/inference.py:81), but list_providers is only a REST
   handler's Python function name (server/routes/providers.py:30) — neither
   CLI nor MCP can list installed providers. Direction: add a providers listing
   to CLI/MCP, or reword the description.

3. No way for MCP to list a project's assets — a removed asset becomes
   unreachable. create_batch/add_batch_assets take asset ids, but the only
   discovery paths are through a batch or job. Repro: remove_batch_assets on a
   draft batch, then no MCP tool lists that asset again (GET /projects/{id}/assets still shows it on REST). Re-ingesting the same source
   path is the only recovery, and only while that path still exists. Direction:
   add a project-level asset-listing tool, or document the gap in
   docs/content/mcp.md's exclusion list.

4. Refusals carry materially less information on CLI/MCP than on REST. The
   same schema-orphan refusal: REST returns 409 with
   code: SCHEMA_CHANGE_WOULD_ORPHAN and a blockers list (class, annotation
   count, asset count); MCP returns {message, retry_with: null, hint: null, index: null} with none of that; CLI --json exits 1 with empty stdout
   and the message only on stderr. Root cause: the code table and detail
   builder live in the server layer (server/errors.py:217,614), which
   sibling surfaces cannot import, so each surface keeps its own partial copy
   (mcp/_errors.py:64,79, cli/_errors.py:62). Direction: move the error-code
   / detail mapping into the kernel so CLI/MCP can publish the same code and
   detail REST does. (CLI's single exit code and MCP's four-key envelope are
   deliberate — only the missing code/detail is the gap.)

5. The orphan-remedy path is REST-only. The refusal's suggested fix —
   "delete or relabel those annotations first" — requires knowing which
   assets block it, which is POST /projects/{id}/schema/blocking-assets
   (server/routes/schemas.py:167). Not exposed on CLI or MCP, and not listed
   as a deliberate exclusion. preview_schema_change only gives per-class
   counts, not asset ids. Direction: expose blocking-assets on MCP/CLI, or add
   ids to the preview.

6. allowed_actions advertises actions a surface can't perform. MCP's
   get_job on a pending job returns allowed_actions: ["start", "pre_label"],
   but there is no start_job MCP tool (jobs auto-start on first write
   instead) — calling it returns "Unknown tool". Similarly, CLI batch list --json on a draft batch returns edit_membership among
   allowed_actions, but there is no CLI command for batch membership edits.
   tests/architecture/test_capability_reachability.py:28-37 checks this only
   for REST/MCP and only for BatchAction, with the job-start case already
   named as an exemption. Direction: extend the reachability gate to CLI, or
   document per-surface which declared actions have no matching command.

7. Race in MCP's auto-start: a legal write gets refused.
   mcp/_autostart.py:57-61 reads the job's state and calls start in two
   separate units of work. Repro: start the job through REST between those two
   steps, then an MCP add_annotations call fails with "job … is
   'in_progress' and cannot become 'in_progress'", and the annotation is never
   written. Direction: guard the start with the job's still-pending state in
   one transaction, or treat "already started" as success for this path.

8. MCP can't set provider_id on update_inference_connection. REST's
   ConnectionUpdate and CLI's inference update --provider
   (cli/inference.py:178,213) both have it; the MCP tool signature
   (mcp/inference.py:121) doesn't. Direction: add the parameter or document
   why it's intentionally absent.

9. REST-only endpoints not marked as deliberate exclusions anywhere:
   PUT /jobs/{id}/assignee (server/routes/jobs.py:233),
   GET /projects/{id}/stats (routes/projects.py:62),
   GET /assets/{id}/batches (routes/assets.py:155),
   dataset asset-annotations (routes/datasets.py:123), preprocessing preview,
   and the providers listing (finding 2). CLI additionally lacks batch-get,
   dataset stats, schema compare/preview, repin, correction batches, and the
   pre-label plan (all present on MCP). Direction: rule on each — implement, or
   add to the exclusion lists in docs/content/mcp.md / docs/content/cli.md.

10. Shape/name drift between surfaces (cosmetic but worth tracking):
    MCP's create_project returns {project, dataset} while REST/CLI return a
    flat project (mcp/projects.py:54); the same query parameter is spelled
    n / job / connection_id on REST and count / job_id / connection
    on MCP.

11. CLI silently drops --seed without --split.
    _split_of (cli/releases.py:52) returns before it looks at seed when
    no split is requested, so --seed is accepted and ignored with no
    warning. Direction: reject --seed when --split is absent.

12. Duplicated cross-surface logic, and thin spots in the import
    contracts. _promoted is copied five times (cli/batches.py:118,
    mcp/batches.py:84, routes/assets.py:62, routes/batches.py:86,
    routes/video_imports.py:79); name-or-id resolution is written twice
    (cli/_resolve.py, mcp/_resolve.py); the project pre-label loop exists
    in three places (cli/projects.py:103, mcp/batches.py:515,
    jobs/prelabel.py). Separately, visionset.wire, visionset.formats and
    visionset.preprocessing have no import-linter contract forbidding a
    framework import, and the jobs/inference contracts don't forbid importing
    the external mcp package — current imports are clean, but nothing
    enforces it going forward. Direction: move the shared reads into the
    kernel, and extend the import-linter contracts to cover these packages.

Reverse gaps (CLI/MCP capability REST lacks)

Machine-local operations (init, token create/list/revoke, ingest,
export, backfill-thumbnails) are CLI/MCP-only by design — REST has no
filesystem to point at. MCP's get_project also surfaces
JobService.project_progress, which no REST route exposes. Worth a decision:
either add the matching REST route, or record these explicitly as
surface-specific by design.