feat: Broad-Side — batch reconnaissance over the OpenRouter Batch API (#103)

.codecarto/.gitignore

@@ -17,6 +17,13 @@ findings/config-model/config-model.md
 scratch/*
 !scratch/.gitkeep
 
+# Broad-Side machine-local state and generated results (batch ids, run
+# directories, costs). The SKILL.md guidance and the config template are
+# tracked; the API key inside config.yaml is your own risk to commit.
+broadside/*
+!broadside/SKILL.md
+!broadside/config.yaml
+
 # Orchestrator session pointer (machine-local, written by /codecarto-init
 # when run from the Pi extension; the MCP path doesn't write it). Contains
 # absolute paths into the user's Pi session storage, so it must never be

.codecarto/GUIDE.md

@@ -57,6 +57,7 @@ Read these files in order before doing any analysis:
 4. `scratch/checkpoints/<phase>.md`, if present, to resume durable in-phase progress after compaction or interruption.
 5. The current phase's `SKILL.md` for detailed instructions on what to analyze and produce.
 6. The output template from `templates/` for the current phase (if starting a new output).
+7. `broadside/synthesis.md`, if a Broad-Side batch reconnaissance run has completed — it carries unverified scouting leads (see `broadside/SKILL.md`) that tell you where the interactive phases should spend attention.
 
 All paths in this guide are relative to `.codecarto/` unless stated otherwise.
 
@@ -71,7 +72,7 @@ Some files in this workspace are **read-only instructions** and must not be modi
 | Category | Files | Access |
 |---|---|---|
 | Orchestration (read-only) | `GUIDE.md`, `CONTRIBUTING.md`, `LICENSE` | Read only. Never modify. |
-| Skills (read-only) | `findings/*/SKILL.md`, `findings/defect-scan/passes/*.md`, `skills/*/SKILL.md` | Read only. Never modify. |
+| Skills (read-only) | `findings/*/SKILL.md`, `findings/defect-scan/passes/*.md`, `skills/*/SKILL.md`, `broadside/SKILL.md` | Read only. Never modify. |
 | Templates (read-only) | `templates/*.md` | Read only. Never modify. |
 | Pipeline definitions (read-only) | `workflow/pipeline*.yaml`, `workflow/VALIDATE.md` | Read only. Never modify. |
 | Source code (read-only) | `../` (everything outside `.codecarto/`) | Read only. Analyze but never modify. |
@@ -313,6 +314,10 @@ your-repo/
       VALIDATE.md              # Validation protocol. Run after every phase.
     closeouts/                 # Per-session closeout files (replaces monolithic THREAD_LOG body).
       <YYYY-MM-DD>-<phase-or-module>.md
+    broadside/                 # Batch reconnaissance state and results (see broadside/SKILL.md).
+      SKILL.md                 # How to read Broad-Side scouting leads (unverified, not evidence).
+      config.yaml              # Broad-Side model/key/lens configuration.
+      <run-id>/                # Per-run JSON + markdown findings, run-meta.json, synthesis report.
     CONVENTIONS.md             # (Optional, project-grown) Cross-cutting invariants. Orchestrator-maintained.
     DECISIONS.md               # (Optional, project-grown) Numbered decisions log. Orchestrator-maintained.
     BACKLOG.md                 # (Optional) Deferred items with rationale.

.codecarto/broadside/SKILL.md

@@ -0,0 +1,114 @@
+---
+name: broadside
+description: Interpret a Broad-Side batch reconnaissance run. Use after codecarto_broadside collect has produced .codecarto/broadside/<run>/ results, to triage scouting signals before or during an interactive CodeCartographer pipeline run.
+---
+
+# Broad-Side
+
+Broad-Side is CodeCartographer's batch reconnaissance pass. It fires six
+analysis lenses — architecture, API surface, security, mechanical defect scan,
+convention extraction, and porting — at the repository as single-turn prompts
+over the OpenRouter Batch API (~50% of sync pricing, asynchronous, unattended),
+then synthesizes one cross-lens report. Results live under
+`.codecarto/broadside/<run>/` alongside this file.
+
+## What Broad-Side findings are — and are not
+
+Broad-Side findings are **unverified scouting signals**, not validated claims.
+Every lens is one shot: no cross-file traversal, no runtime verification, no
+builds, no tests, no follow-up questions. The batch model is cheap, not strong.
+Treat every finding as a lead with a file:line pointer that the interactive
+pipeline — or you — must confirm before it is a fact.
+
+This is the division of labor: Broad-Side is cheap enough to run on any repo to
+decide where the expensive interactive run should spend its attention. It does
+not replace any phase; it tells phases where to look.
+
+## Reading a Broad-Side run
+
+1. Read `synthesis.md` first. It carries the executive summary, severity counts,
+   the top cross-lens findings, and per-module risk levels.
+2. Read `triage.md` for the work order: each lead scored by impact ×
+   difficulty with a P0–P3 priority and an effort estimate. It is a starting
+   point for re-verification, not a commitment — every item still needs
+   confirmation against the source before work begins.
+3. Read the per-lens files behind anything that matters to your current phase:
+   - `architecture-*.json` → the architecture phase's seed of prior knowledge
+   - `api-*.json` → endpoints and data types (contracts/protocols phases)
+   - `security-*.json` → auth, trust boundaries (defect-scan-semantic pass 5)
+   - `defect-*.json` → mechanical defect leads (defect-scan-mechanical)
+   - `conventions-*.json` → naming/idiom candidates for CONVENTIONS.md
+   - `porting-*.json` → platform coupling (porting phase)
+4. `run-meta.json` records scope: which lenses ran, at what cost, with what
+   coverage caps.
+
+## How to use the leads
+
+- **A finding that matches your phase's scope is a starting point, not an answer.**
+  Re-derive it from the source yourself; cite the source, not the Broad-Side
+  report. Broad-Side output is not evidence.
+- **Route, don't believe.** A Broad-Side "high" that your phase can neither
+  confirm nor dismiss becomes an open question with `needs-runtime-test` or
+  `needs-maintainer-decision` — never a finding.
+- **Promotable conventions are candidates only.** CONVENTIONS.md promotions
+  still require the orchestrator's review against the code, per the usual
+  promotion rules.
+- **Coverage caps are real.** Directory-sliced lenses cap each slice's input;
+  `run-meta.json` and the synthesis `coverage` field say what was scanned.
+  Everything outside that is unscouted, not clean.
+
+## Running Broad-Side
+
+Broad-Side is an executable-surface feature (MCP today):
+
+```
+codecarto_broadside {cwd, action: "submit", lenses: [...]}   # fire the batches
+codecarto_broadside {cwd, action: "collect"}                  # poll, save, synthesize
+codecarto_broadside {cwd, action: "status"}                   # show recorded runs
+codecarto_broadside {cwd, action: "models"}                   # compare batch models
+```
+
+The `models` action lists every `:batch` variant on OpenRouter — pricing per
+million tokens, context window, output ceiling, structured-output support, and
+(optionally) Artificial Analysis coding indices — cheapest first, with the
+configured model marked. Use it before switching models in `config.yaml`.
+Submits pre-flight the chosen model: pricing comes from the live catalog
+(cached 24h), requests clamp to the provider's completion ceiling, and a
+model that does not advertise structured-output support is refused outright,
+because every lens depends on `json_schema` response_format.
+
+Collect runs two cross-lens post-passes by default: **synthesis** (the
+executive report) and **triage** (the prioritized work order). Pass
+`include_synthesis: false` or `include_triage: false` on collect to skip one.
+
+It works on any git repository — no initialized workspace required — and needs
+an OpenRouter API key via the `api_key` parameter, the `OPENROUTER_API_KEY`
+environment variable, or `api_key` in this directory's `config.yaml`.
+
+Submits are priced before they fire: Broad-Side estimates the run from the
+collected file sizes against the model's live per-token pricing and refuses
+when the estimate exceeds `max_cost` (`config.yaml` or the tool parameter)
+unless `force` is passed. See `config.yaml` for the model, limit, and manual
+pricing-override keys.
+
+The `max_cost` guardrail is an **estimate-based pre-flight limit**, distinct
+from OpenRouter's runtime cost tracking: it predicts from file sizes before
+spend, it does not stop a batch mid-flight. Actual spend appears in
+`run-meta.json` after collect.
+
+## Resilience notes
+
+- **Truncation is spoken.** A lens output whose JSON does not parse — even
+  after code-fence stripping — is saved verbatim but marked `truncated`:
+  the collect summary counts it, `run-meta.json` records it, and the
+  synthesis prompt is told its module is unrepresented, not clean.
+- **Resubmission is always safe.** Batch requests are pure (no tools, no
+  filesystem, no side effects), so a failed or truncated slice can be
+  resubmitted freely. This is the same retry-safety rule OpenRouter's
+  headless-agent scaffold enforces for tool-using agents ("retry only
+  before tool calls"); Broad-Side satisfies it by construction. If
+  Broad-Side ever gains server tools, this invariant becomes load-bearing.
+- **Field shapes** for the model catalog and benchmarks endpoints follow
+  the official OpenRouter skills (`OpenRouterTeam/skills`:
+  `openrouter-models`, `openrouter-benchmarks`) — consult them when
+  extending catalog parsing.

.codecarto/broadside/config.yaml

@@ -0,0 +1,54 @@
+# Broad-Side batch reconnaissance configuration. Optional.
+# Missing keys fall back to the defaults listed below.
+
+# OpenRouter model to use for batch requests. The default is Google Gemini
+# 3.7 Flash (batch) — the cheapest batch model with tool-calling support and
+# a 1M-token context window. Change this to another OpenRouter batch model
+# if you need a different cost/capability trade-off. To compare what's
+# available, run codecarto_broadside with action "models" — it lists every
+# :batch variant with pricing, context, output caps, structured-output
+# support, and optional coding benchmarks. Beware the expensive end of that
+# list — some batch models exceed $80 per million output tokens — and note
+# that every lens requires structured-output (json_schema) support, which
+# submit refuses to proceed without.
+#
+# model: google/gemini-3.7-flash:batch
+
+# OpenRouter API key. Prefer the OPENROUTER_API_KEY environment variable —
+# keys in this file are committed if you track .codecarto/ after init.
+# The codecarto_broadside tool also accepts api_key as a parameter.
+#
+# api_key: ""
+
+# Default lens set for codecarto_broadside submit when no lenses are
+# specified. All six lenses are on by default. Remove a lens id to skip it
+# globally, or pass an explicit lenses array on the submit call to override.
+#
+# default_lenses:
+#   - architecture
+#   - api
+#   - security
+#   - defect
+#   - conventions
+#   - porting
+
+# Approximate run expense limit in USD (0 = no limit). Before submitting,
+# Broad-Side estimates the run cost from the collected file sizes and the
+# model's per-token pricing — fetched live from OpenRouter's model catalog
+# and cached for 24h. If the estimate exceeds max_cost, submit refuses and
+# prints the per-lens breakdown; pass force: true to override, or set a
+# value here so every run is guarded by default.
+#
+# This is a pre-flight estimate guardrail, not a runtime stop: OpenRouter
+# bills actual usage, which may differ from the estimate either direction.
+# Actual cost lands in each run's run-meta.json after collect.
+#
+# max_cost: 1.00
+
+# Manual pricing overrides in USD per MILLION tokens. Normally Broad-Side
+# looks the model's pricing up automatically; set both fields only when the
+# lookup fails (offline, private model) or you want to assert a ceiling.
+#
+# pricing:
+#   input_per_m: 0.1875
+#   output_per_m: 0.9375

CHANGELOG.md

@@ -4,6 +4,14 @@ All notable changes to this project are documented here. The format is based on
 
 ## [Unreleased]
 
+### Added
+
+- **Broad-Side: batch reconnaissance over the OpenRouter Batch API** (#103). New `codecarto_broadside` MCP tool (actions: `submit`, `collect`, `status`) fires six single-turn analysis lenses — architecture, API surface, security, mechanical defect scan, convention extraction, porting — at any git repository as asynchronous batch jobs on a cheap batch model (~50% of sync pricing, unattended, 24h window), slices large modules by top-level directory, saves JSON plus rendered markdown to `.codecarto/broadside/<run>/`, and optionally synthesizes a cross-lens executive report. Works without an initialized workspace; needs an OpenRouter key via the `api_key` parameter, `OPENROUTER_API_KEY`, or `.codecarto/broadside/config.yaml`. Broad-Side findings are explicitly unverified scouting signals — file:line leads for the interactive pipeline to confirm, never evidence themselves. `codecarto_init` tolerates a `.codecarto/` that holds only `broadside/` (no force/backup needed), and scaffold refresh never touches broadside state, config, or results.
+- **Broad-Side: expense guardrails and live per-model pricing** (#103). `config.yaml` now accepts `model`, `max_cost`, and `pricing.input_per_m`/`output_per_m` overrides, and the MCP tool accepts `max_cost` and `force` parameters. Before submitting, Broad-Side estimates the run cost from collected file sizes (≈4 chars/token) against the configured model's per-token pricing — looked up live from OpenRouter's model catalog (cached 24h), so models like `openai/gpt-5.2-pro:batch` at ~$84/M output are priced correctly, not at the default model's rates. A submit whose estimate exceeds `max_cost` refuses with a per-lens breakdown and creates no run entry unless `force: true`. The submit response now reports the pricing used and its source (built-in/config/live/cache).
+- **Broad-Side: model catalog action and capability pre-flight** (#103). New `models` action lists every `:batch` model on OpenRouter — pricing per million tokens, context window, completion ceiling, structured-output support, and optional Artificial Analysis coding indices (via `GET /api/v1/benchmarks`, attribution preserved) — cheapest first with the configured model marked. Submits now pre-flight the chosen model against that catalog: lens `max_tokens` clamps to the provider's completion ceiling, deprecated models are flagged, and models that do not advertise structured-output support are refused outright, since every lens depends on `json_schema` response_format. The catalog cache is shared between the `models` action and submit-time pricing resolution.
+- **Broad-Side: truncation detection with fence-tolerant parsing** (#103, #133). Lens output is now parsed tolerantly — markdown code fences are stripped before JSON parsing, mirroring the tolerance in OpenRouter's headless-agent scaffold. Parseable output is saved as clean JSON; output that still does not parse (the signature of a `max_tokens` cutoff) is saved verbatim but marked `truncated`. The collect summary and `run-meta.json` report truncation counts, and the synthesis prompt is told which modules are unrepresented rather than clean. Also documented the retry-safety invariant (batch requests are pure, resubmission always safe) and the distinction between Broad-Side's pre-flight `max_cost` estimate and OpenRouter's runtime cost accounting.
+- **Broad-Side: triage pass** (#103, #135). Collect now runs a second cross-lens post-pass alongside synthesis (skip with `include_triage: false`): every finding is scored by impact × fix difficulty and turned into a prioritized work order — P0–P3 priority, effort estimate, per-module grouping, deduplicated leads, and explicit `omitted` notes for dropped items — saved as `triage.json`/`triage.md` and surfaced in the collect summary. The triage prompt frames the queue as a starting point for re-verification, never a commitment. Both post-passes submit as separate batches together and poll independently, and the state file tracks each so a resumed collect can finish whichever is still pending.
+
 ## [0.16.0] — 2026-08-17
 
 The field-test round. Immediately after 0.15.0 shipped, the same 7-phase deepseek-harness analysis was re-run on a fresh worktree through the published binary — this time with the driving chat as orchestrator — and the run's own gaps became this release (#111–#114): the very first completion appended decision rows without their promised heading, both full runs ended with no dashboard ever rendered, the analysis→publish→synthesis library loop was unreachable from any served text, and the terminal completion message named nothing actionable while skills, amendments, a publishable spec, and the usage log all sat unused.

README.md

@@ -357,6 +357,7 @@ Implements MCP spec revision [`2025-11-25`](https://modelcontextprotocol.io/spec
 | `codecarto_publish` | MCP-only library publish |
 | `codecarto_library_list` | MCP-only library listing |
 | `codecarto_library_reindex` | MCP-only library reindex |
+| `codecarto_broadside` | MCP-only batch reconnaissance (Broad-Side) |
 
 Each workflow tool accepts an absolute `cwd` for the target repository. `codecarto_init` requires `force: true` to overwrite an existing `.codecarto/` (instead of Pi's interactive confirmation). The library tools accept an explicit absolute `library_path` or resolve `library.path` from `.codecarto/workflow/config.yaml` / `~/.codecarto/config.yaml`. The library schema is experimental and may break before v2.