Skip to content

docs: simplify and harden OpenShell quickstart#1113

Open
shaun-agent wants to merge 17 commits into
mainfrom
docs/openshell-runtime-install-contract
Open

docs: simplify and harden OpenShell quickstart#1113
shaun-agent wants to merge 17 commits into
mainfrom
docs/openshell-runtime-install-contract

Conversation

@shaun-agent

@shaun-agent shaun-agent commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: the previous OpenShell quick start mixed Day 1 setup with broader all-agent policy, automation, and E2E concerns.
  • What changed: PR docs: simplify and harden OpenShell quickstart #1113 is now intentionally scoped to one Day 1 path: OpenShell sandbox + OpenAB Discord bot + default Kiro CLI ACP agent.
  • What did NOT change: existing coding CLI images and the existing native sandbox image are not changed. Codex, Claude, Gemini, AGY, Cursor, Copilot, OpenCode, arbitrary web, package installs, and broader tool access remain Day 2 policy work.

Before / After

BEFORE:
OpenShell docs tried to describe a generic local setup path.
The happy path mixed Discord, openab-agent auth, broad policy discussion,
and automation/E2E concerns in one page.

AFTER:
docs/openshell.md documents one narrow Day 1 route:

  OpenShell sandbox
    -> provider-injected DISCORD_BOT_TOKEN
    -> OpenAB Discord REST + Discord Gateway WebSocket policy
    -> Kiro CLI default ACP agent policy
    -> human Discord mention/reply proof

Deeper policy, macOS Docker Desktop caveats, and E2E rules live in the ADR.

Review Decisions

  • The success path must use OpenShell: openshell sandbox connect, openshell sandbox exec, or an OpenShell-generated SSH route.
  • Raw docker exec may be used only for debugging; it is not proof of the OpenShell sandbox/proxy/policy path.
  • openshell/Dockerfile.kiro is an OpenShell-specific wrapper around the existing default OpenAB/Kiro image.
  • openshell/Dockerfile is no longer changed in this PR. The PR file list now has no changes to the existing native sandbox image.
  • macOS + Docker Desktop is accepted only if Discord Gateway WebSocket works through the OpenShell-managed route.
  • The Day 1 policy should not be expanded to weather/search/browser/package endpoints just to make a coding-agent prompt succeed.

Changes

File Change
docs/openshell.md Rewrites the quick start around Discord + Kiro only; uses provider-injected Discord token; documents host paths, Day 1 policy, and pass/fail rules.
docs/adr/openshell-openab-preset-module.md Adds the Day 1 decision record and moves Day 2 policy/tool/agent complexity out of the quick start.
openshell/Dockerfile.kiro Adds an OpenShell-specific Kiro wrapper with HOME=/sandbox and writable /sandbox runtime paths.
.github/workflows/build-operator.yml Adds publication wiring for ghcr.io/openabdev/openab-kiro-sandbox.
.github/workflows/docker-smoke-test.yml Adds smoke-test coverage for the new OpenShell Kiro wrapper and triggers on openshell/Dockerfile*.

Prior Research

Testing

  • git diff --check
  • git diff --exit-code origin/main...HEAD -- openshell/Dockerfile
    • Confirms this PR no longer changes the existing native sandbox Dockerfile.
  • docker build -t openab-kiro-sandbox-pr1113 -f openshell/Dockerfile.kiro .
    • Passed locally on Docker Desktop.
  • docker run --rm --entrypoint sh openab-kiro-sandbox-pr1113 -lc 'printf "user=%s home=%s pwd=%s\n" "$(id -un)" "$HOME" "$PWD"; command -v openab; command -v kiro-cli; test -d /sandbox/.local/share/kiro-cli; test -d /sandbox/.kiro; test -w /sandbox/tmp'
    • Output confirmed user=sandbox, home=/sandbox, pwd=/sandbox, /usr/local/bin/openab, and /usr/local/bin/kiro-cli.
  • Prior disposable Linux OpenShell runner validation, before this cleanup commit:
    • Installed official OpenShell package: openshell 0.0.70.
    • Created real OpenShell sandbox oab.
    • Applied Day 1 policy with openshell policy set ... --wait.
    • Verified through openshell sandbox exec, not raw docker exec, that the sandbox user, /sandbox, token injection, openab, and kiro-cli were present.
    • Verified Kiro device-flow auth reached the browser approval step.

Still Pending

This PR is not claiming final OpenShell/macOS success yet. The next E2E run needs human Kiro device-flow approval, then must prove:

  • kiro-cli whoami succeeds inside the OpenShell sandbox.
  • openab run -c /sandbox/config.toml is launched through OpenShell.
  • Logs show discord bot connected.
  • A human Discord mention receives a Kiro-backed reply through ACP.
  • A negative weather/search prompt is sent through Discord -> OpenAB -> Kiro ACP.
  • If weather/search fails due to OpenShell policy, record the denied host and binary.
  • If Kiro answers via already-allowed Kiro service traffic without observable sandbox egress denial, record that separately; it proves the ACP path can answer, but it does not prove broad sandbox web access.

Risks / Notes

  • macOS + Docker Desktop may still expose a Discord WebSocket/proxy compatibility issue. The guide treats that as a blocker, not as a pass.
  • The Kiro endpoint list is based on official docs plus observed device-flow traffic; it may need maintenance if Kiro changes service endpoints.
  • Other agents need separate profiles; this PR intentionally avoids solving all agent auth/network/tool permutations.

@chaodu-agent

This comment has been minimized.

@howie

howie commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Mob Review — PR #1113 docs: simplify and harden OpenShell quickstart

Mode: 2-voice mob (Claude 4-subagent voice + Codex). agy/Gemini unavailable this round (OAuth onboarding expired — NOT_ONBOARDED).
Verdict: 🔴 NEEDS_CHANGES — author shaun-agent, base main, +625/-86, 3 files (docs only, no app code).


🔴 Critical (must fix before merge)

C1. Documented Day 1 network-policy command does not exist in OpenShell 0.0.16

File: docs/openshell.md — "Apply The Day 1 Network Policy" (both for loops)
The quickstart's required pre-auth step runs:

openshell policy update oab --add-endpoint "<ep>" --binary /usr/local/bin/openab --wait

Empirically verified against the installed openshell 0.0.16 (the exact version the doc's installer pulls): openshell policy exposes only set / get / list / delete. There is no policy update subcommand, and no --add-endpoint or --binary flags. policy set takes --policy <YAML file> (--wait does exist). So every endpoint command in this section fails immediately, and the "Day 1 success path" the guide promises cannot be completed as written.

Cross-voice: flagged by Codex (Critical, claims it ran 0.0.16) and independently reproduced by the lead on a local openshell 0.0.16.
Note: the old doc also used policy update --add-endpoint, so this is a pre-existing defect — but this PR rewrites the entire section (new format, --binary, --wait, for-loops) and therefore owns it now.

Fix: Replace with a tested policy YAML + openshell policy set oab --policy <file> --wait, encoding endpoints in the YAML schema the installed CLI actually parses. Verify against the pinned OpenShell version before merge. (The ADR §2.1 itself says this policy "needs compatibility validation against the OpenShell CLI" — so it should not ship as the required Day 1 step until validated.)

C2. ADR E2E automation contract uses a non-existent sandbox exec command

File: docs/adr/openshell-openab-preset-module.md §2.3
The automation contract instructs agents/harnesses:

openshell sandbox exec -n oab --no-tty -- ...

Verified against 0.0.16: openshell sandbox exposes create/get/list/delete/connect/upload/download/ssh-configno exec. Any harness following this contract fails. Since the contract is meant to make "a passing run prove the real OpenShell path", a wrong command defeats its purpose.

Cross-voice: Codex (confirmed by lead repro).

Fix: Replace with a supported transport (e.g. connect, or upload + run inside connect), or pin the OpenShell version where sandbox exec exists and document installing it.


🟡 Important (should fix; defer only with a documented reason)

I1. ${DISCORD_BOT_TOKEN} silently expands to empty string if the provider credential isn't injected

File: docs/openshell.md (TL;DR + Configure blocks) — verified behavior in src/config.rs expand_env_vars (std::env::var(..).unwrap_or_default(), covered by test expand_env_vars_unknown_becomes_empty).
The mechanism is real (OpenAB does expand ${VAR} in config.toml — verified), but if OpenShell does not inject DISCORD_BOT_TOKEN into the sandbox env (provider misconfigured, var name mismatch, --provider omitted), the token expands to "" with no error. OpenAB then connects to Discord with a blank token → opaque auth failure, not "token missing". The quickstart's in-sandbox verify step checks whoami/writable but never checks the var is present. The ADR itself warns provider values "may appear as resolver handles rather than raw env values" — directly contradicting the quickstart's assumption.

Cross-voice: Codex (R1 "provider behavior contradictory") + Claude code-reviewer + silent-failure-hunter (DUPLICATE-confirmed in R2).

Fix: Add an in-sandbox pre-run check (test -n "$DISCORD_BOT_TOKEN" || echo "TOKEN MISSING" without printing it), and a Troubleshooting row mapping empty token → Discord auth error.

I2. audit enforcement overstates lockdown — matched endpoints under audit are NOT enforced

File: docs/openshell.md Day 1 policy (openab-agent endpoints use :full:rest:audit) vs "Day 2 Boundary".
audit mode logs and allows matched traffic. The "Day 2 Boundary" prose only states that unmatched host+port+binary is blocked, and frames audit as "helps inspect matched endpoints" — it never states the more important converse: every endpoint the agent matches under audit is effectively open. Combined with wildcard hosts (*.openai.com, *.oaiusercontent.com), the agent's egress there is unrestricted while the reader believes it is allowlisted.

Cross-voice: Claude silent-failure-hunter (Critical) + comment-analyzer (Important) + Codex AGREE.

Fix: Either switch the openab-agent endpoints to an enforcing mode, or add an explicit warning: "audit = allowed + logged, NOT enforced; the Discord binary is enforced, the model/auth binary is in audit (open) for Day 1."

I3. Quickstart makes Day 1 policy mandatory; its own ADR says it must not be required for first run

File: docs/openshell.md ("Before authenticating or starting OpenAB, apply the Day 1 network policy") vs docs/adr/openshell-openab-preset-module.md §2.1 ("Keep broad policy files out of the first-run happy path"; "do not require policy edits for the first successful run").
Direct doc-vs-ADR contradiction on whether policy is a required Day 1 step.

Cross-voice: comment-analyzer + Codex AGREE.

Fix: Reconcile — either confirm via testing that policy IS required and update the ADR, or demote the policy block to "apply if Discord/model traffic is blocked."

I4. Cleanup no longer deletes the credential provider (regression vs old doc)

File: docs/openshell.md Cleanup. Old doc had openshell provider delete discord; the new Cleanup only runs openshell sandbox delete oab. The openab-discord provider (holding the Discord token) is left behind; users think credentials were removed.

Cross-voice: Codex (diff-verifiable regression).

Fix: Add openshell provider delete openab-discord to Cleanup.

I5. "Test local source changes" build command does not build local source

File: docs/openshell.md (docker build -t oab-native-sandbox -f openshell/Dockerfile .).
The text says use this to "test local source changes", but openshell/Dockerfile is FROM ghcr.io/openabdev/openab-native:beta with no COPY/compile of the local checkout — it only re-layers env/dirs onto the published base. So a local OpenAB source change is NOT reflected. (Build context . is also irrelevant since there's no COPY — harmless but misleading.)

Cross-voice: Codex + Claude code-reviewer (context-irrelevant).

Fix: Reword to "test local Dockerfile changes", or make the Dockerfile build OpenAB/openab-agent from the local checkout if source testing is the goal.

I6. Broken ADR cross-reference

File: docs/adr/openshell-openab-preset-module.md:12 — Related list links ./openshell-websocket-auth.md, which does not exist in docs/adr/. (Sibling ./custom-gateway.md does resolve.)

Cross-voice: Codex + comment-analyzer (both verified absent). No markdown link-check CI exists to catch it.

Fix: Remove the link, repoint to the intended existing ADR, or add the file.

I7. CI does not exercise this Dockerfile change (ships to release build untested)

File: .github/workflows/docker-smoke-test.yml pull_request.paths.
paths is Dockerfile* / src/** / Cargo.*. GitHub path globs don't cross / without **, so Dockerfile* does not match openshell/Dockerfile. This PR touches only docs/* + openshell/Dockerfile → the smoke job is skipped, and the Dockerfile change (HOME=/sandbox, -d /sandbox, new dirs, PATH/TMPDIR) flows to build-operator.yml (tag/release) with zero PR-time validation. Even when the smoke test does run, it only asserts the CMD/agent-CLI/ACP handshake — it never asserts HOME=/sandbox or writable /sandbox, so reverting the home change would stay green while breaking the documented quickstart.

Cross-voice: pr-test-analyzer (both legs verified) + Codex AGREE. (Arguably a repo-CI follow-up rather than strictly in this PR's scope — flag for maintainer.)

Fix: Add openshell/** (or **/Dockerfile*) to paths, and add an assertion: docker run --rm <img> sh -c 'test "$HOME" = /sandbox && test -w /sandbox && command -v openab'.


🟢 Actionable NITs

  • N1. The two config.toml blocks (TL;DR vs "Configure And Run") differ only by a [reactions] block that restates code defaults (enabled=true, remove_after_reply=false — verified). Either drop it or include it in both so copy-paste targets match.
  • N2. Dockerfile creates /sandbox/work and /sandbox/.cache, but the architecture diagram/config list only bin, .local/bin, tmp; runtime mkdir -p recreates only bin/.local/bin/tmp (not .cache/work). Align the documented filesystem shape with what the image actually creates, or drop unused dirs.
  • N3. auth0.openai.comauth.openai.com + wildcards (*.openai.com) is plausibly correct for the codex-oauth flow but unverified in-repo; if *.openai.com already covers auth, the explicit auth.openai.com line is redundant. Confirm the real auth host.
  • N4. TL;DR leads with inline export DISCORD_BOT_TOKEN="your-discord-bot-token", which trains users to paste real tokens into shell history. Prefer read -rs DISCORD_BOT_TOKEN or sourcing a gitignored .env. (Not a leak — config-file hygiene is genuinely improved — just residual history/env exposure.)
  • N5. message_processing_mode is shown in the doc but absent from config.toml.example (it exists in code/tests). Add it to the example for parity so readers have a schema artifact to diff against.

✅ Verified non-issues (checked, no action)

  • ${VAR} expansion works: src/config.rs expand_env_vars matches bare ${VAR} via std::env::var, called in load_config_raw before parse, unit-tested. The mechanism is correct (only the empty/injection case in I1 is the risk).
  • Dockerfile user/home setup is correct: useradd -m -d /sandbox + mkdir + chown -R + USER/ENV/WORKDIR ordering is sound; uid 1000660000 is the deliberate OpenShift-style high uid.
  • Config fields all exist in src/config.rs and are unit-tested (allow_all_users, message_processing_mode, remove_after_reply, [agent.env]).
  • openshell/Dockerfile IS built in CI (build-operator.yml + docker-smoke-test.yml), so the prebuilt openab-native-sandbox image is real (the I7 gap is about this PR not triggering it).
  • --wait flag is real on openshell policy set.

Relationship to prior OpenShell PRs / issues (per request)

@chaodu-agent

This comment has been minimized.

@github-actions github-actions Bot added pending-contributor closing-soon PR missing Discord Discussion URL — will auto-close in 24 hours. and removed pending-maintainer labels Jun 21, 2026
@thepagent thepagent removed the closing-soon PR missing Discord Discussion URL — will auto-close in 24 hours. label Jun 24, 2026
@github-actions github-actions Bot added the closing-soon PR missing Discord Discussion URL — will auto-close in 24 hours. label Jun 24, 2026
@openab-app openab-app Bot removed the closing-soon PR missing Discord Discussion URL — will auto-close in 24 hours. label Jun 25, 2026
@github-actions github-actions Bot added the closing-soon PR missing Discord Discussion URL — will auto-close in 24 hours. label Jun 25, 2026
@openab-app openab-app Bot removed the closing-soon PR missing Discord Discussion URL — will auto-close in 24 hours. label Jun 25, 2026
@chaodu-agent

This comment has been minimized.

@github-actions github-actions Bot added closing-soon PR missing Discord Discussion URL — will auto-close in 24 hours. needs-rebase pending-contributor and removed needs-rebase pending-contributor labels Jun 25, 2026
@openab-app openab-app Bot removed the closing-soon PR missing Discord Discussion URL — will auto-close in 24 hours. label Jun 26, 2026
@chaodu-agent

Copy link
Copy Markdown
Collaborator

CHANGES REQUESTED ⚠️ — Most mob-review findings are addressed, but openshell policy update and sandbox exec references remain and are not valid in current OpenShell CLI.

What This PR Does

Rewrites docs/openshell.md from a generic OpenShell guide into a narrow, testable Day 1 quickstart for one path: OpenShell sandbox → OpenAB Discord bot → Kiro CLI ACP agent. Adds a supporting ADR, a Dockerfile.kiro wrapper, a checked-in Day 1 policy YAML, and CI wiring.

How It Works

  • openshell/Dockerfile.kiro wraps the existing openab:beta image with a non-root sandbox user and writable /sandbox runtime root.
  • openshell/samples/kiro-discord-day1-policy.yaml defines enforced egress for openab (Discord) and kiro-cli* (Kiro service endpoints).
  • The quickstart uses openshell policy set oab --policy <file> --wait for Day 1 policy application.
  • CI: build-operator.yml adds -kiro-sandbox to the matrix; docker-smoke-test.yml triggers on openshell/Dockerfile* and tests the new image.

Findings

# Severity Finding Location
1 🟡 Day 2 section uses openshell policy update --add-endpoint --binary which does not exist in OpenShell 0.0.16+ docs/openshell.md:220
2 🟡 openshell sandbox exec is referenced but does not exist in current OpenShell CLI docs/openshell.md:233, docs/adr/...:105
3 🟢 C1 fixed: Day 1 policy now uses openshell policy set --policy <YAML> --wait docs/openshell.md TL;DR
4 🟢 I1 fixed: in-sandbox test -n guard + Troubleshooting row for missing token docs/openshell.md
5 🟢 I2 fixed: all policy endpoints now use enforcement: enforce kiro-discord-day1-policy.yaml
6 🟢 I4 fixed: Cleanup includes openshell provider delete openab-discord docs/openshell.md
7 🟢 I6 fixed: broken openshell-websocket-auth.md cross-reference removed ADR Related section
8 🟢 I7 fixed: smoke-test paths now include openshell/Dockerfile* and test the Kiro sandbox docker-smoke-test.yml
9 🟢 N4 fixed: TL;DR uses read -rsp instead of inline token export docs/openshell.md
Finding Details

🟡 F1: Day 2 section uses non-existent openshell policy update command

docs/openshell.md line 220 shows:

openshell policy update oab \
  --add-endpoint api.example.com:443:read-only:rest:enforce \
  --binary /usr/local/bin/kiro-cli

The mob review (C1) confirmed openshell policy only has set / get / list / delete subcommands. While C1 is fixed for the Day 1 path (now uses policy set --policy <YAML>), the Day 2 section still documents the non-existent policy update --add-endpoint --binary workflow. Users following the Day 2 expansion instructions will fail.

Suggested fix: Replace with a workflow that edits the YAML file and re-applies via openshell policy set, or document the actual policy update syntax if a newer OpenShell version adds it (and pin that version requirement).

🟡 F2: openshell sandbox exec does not exist in current OpenShell CLI

Both docs/openshell.md (E2E Rules, line 233) and the ADR (E2E Contract, line 105) list openshell sandbox exec as a valid transport alongside connect and SSH config. The mob review (C2) confirmed sandbox only exposes create/get/list/delete/connect/upload/download/ssh-config — no exec.

This is less critical now because the TL;DR uses openshell sandbox connect oab (which works), but the E2E rules still reference a non-existent command as an option.

Suggested fix: Remove openshell sandbox exec from both locations, or replace with openshell sandbox connect and SSH config as the only two valid transports.

🟢 F3–F9: Previous mob review findings addressed

The PR has addressed the bulk of the mob review feedback:

  • Day 1 policy uses real CLI commands with a checked-in YAML artifact.
  • Token injection is validated in-sandbox before openab run.
  • All policy entries use enforcement: enforce (no misleading audit).
  • Cleanup is complete (sandbox + provider deletion).
  • CI coverage includes the new Dockerfile.
  • Cross-references are valid.
  • TL;DR avoids token in shell history.
Baseline Check
  • PR opened: 2026-06-15, labels: pending-contributor, needs-rebase
  • Main already has: docs/openshell.md (old generic guide), openshell/Dockerfile (native sandbox), CI wiring for native sandbox
  • Net-new value: Kiro-specific OpenShell wrapper (Dockerfile.kiro), enforced Day 1 policy YAML, narrowed quickstart for a single proven path, ADR documenting scope/decisions, smoke-test coverage for the new image
Addressing External Reviewer Feedback

@howie (Mob Review — Round 1, 2026-06-20)

C1: openshell policy update --add-endpoint does not exist

Addressed for Day 1 path: quickstart now uses openshell policy set oab --policy <YAML> --wait. The Day 2 section still uses policy update (see F1 above).

C2: openshell sandbox exec does not exist

⚠️ Partially addressed: TL;DR uses sandbox connect (valid). E2E rules still reference sandbox exec as an option (see F2 above).

I1: ${DISCORD_BOT_TOKEN} silently expands to empty

Addressed: in-sandbox test -n guard added + Troubleshooting row.

I2: audit enforcement overstates lockdown

Addressed: all policy entries now use enforcement: enforce.

I3: ADR says policy is not required but quickstart makes it mandatory

Addressed: ADR now states "Use enforcement: enforce for the documented happy path" — both documents agree policy is required (OpenShell is default-deny).

I4: Cleanup doesn't delete provider

Addressed: openshell provider delete openab-discord added to Cleanup.

I5: "Test local source changes" misleading

Addressed: that section no longer exists; only the OpenShell wrapper Dockerfile is referenced.

I6: Broken ADR cross-reference

Addressed: openshell-websocket-auth.md link removed from ADR Related section.

I7: CI doesn't exercise the Dockerfile change

Addressed: docker-smoke-test.yml paths include openshell/Dockerfile* and the matrix has a Kiro sandbox entry.

What's Good (🟢)
  • The narrowing to a single Day 1 path is the right call — avoids the "universal OpenShell guide" trap.
  • Checked-in policy YAML (kiro-discord-day1-policy.yaml) with enforced endpoints is excellent — testable, reviewable, diffable.
  • Dockerfile.kiro is minimal and correctly layers on top of the existing base without modifying it.
  • The Day 2 log-driven expansion workflow is well-documented conceptually (just needs correct CLI commands).
  • ADR provides clear scope boundaries for what is and isn't Day 1.
  • read -rsp for token input and in-sandbox validation guard are good security hygiene.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants