A local-first AI agent for your terminal. Runs on a single consumer GPU, talks to a real coding LLM via llama.cpp, keeps everything on your machine. No cloud account, no per-token billing, no telemetry.
If you're going to dive right in, recommended models to get started with are:
coder-qwen-3-30: Qwen Coder MoE 30B / 3B active, non-thinking. Quick, good option for coding.general-qwen-3.6-35: Qwen 3.6 MoE 35B / 3B active, thinking. Good general reasoning - has an issue going in loops when working on complex thinking tasks, though.
GLM 4.7 is a good alternative thinking coder. Uses tools effectively but not quite as quick to problem-solve as the Qwen Coder model.
Barry is two binaries:
barry-server— a daemon that owns a llama.cpp container, the agent loop, the conversation store, and the audit log.barry— a ratatui-based terminal client. Owns the tool registry, MCP fleet, and approval gating.
They speak HTTP+SSE over localhost or LAN. The CLI has zero GPU/podman dependencies, so a Mac or laptop client can drive a WSL2 / Linux server over the network if you want to keep the GPU and the keyboard on different machines.
Server tested on WSL2, client tested on WSL & Mac.
Three model profiles ship out of the box, all sized for a 32 GB GPU (RTX 5090 / A6000 / 4090 with optimised KV quant):
| Profile | Model | Ctx | Notes |
|---|---|---|---|
general-qwen-3.6-35 |
Qwen3.6 35B-A3B (Q5_K_M) | 256K | Thinking-capable, general reasoning. Default boot profile — what make setup downloads. |
coder-qwen-3-30 |
Qwen3 Coder 30B-A3B Instruct (Q5_K_M) | 128K | Non-thinking; fastest decode. |
coder-glm-4.7-30 |
GLM 4.7 Flash 30B-A3B (Q5_K_M) | 200K | Hybrid thinking, native MLA. |
general-mistral-3.2-24 |
Mistral Small 3.2 24B Instruct (Q5_K_M) | 128K | Dense 24B; comfortable on a 24 GiB card. |
general-phi-4-14 |
Phi 4 14B (Q5_K_M) | 16K | Microsoft dense 14B — gated on HF (--hf-token); short ctx. |
coder-devstral-1.1-24 |
Devstral Small 1.1 24B (Q5_K_M) | 128K | Mistral coder fine-tune; fits 24 GiB. |
The picker (barry-server download) annotates each (model, quant) combination with a recommendation icon: ✓ (recommended — fits with ctx ≥ 100K and quant ≥ q5), · (fits but suboptimal), ✗ (won't fit on the detected GPU).
Profiles are config-driven (defaults/server.toml), so retargeting to a 24 GB card is a copy-and-edit job — drop to Q4_K_M, tighten the context, optionally pick a smaller model. Hot-swap between profiles at runtime via /model <name> in the REPL.
The agent has a curated built-in tool catalogue: filesystem read/write, ripgrep-style search, bash execution, HTTP fetching, web search (via SearXNG), and conversation memory ops. Every tool is classified Safe or Dangerous and assigned an approval tier; see docs/ToolCatalogue.md.
Three permission modes, cycled with Shift+Tab in the REPL:
- Safe — read-only. Dangerous tools refused outright. The model can think and explore but can't act.
- Default — read-only tools fire silently; mutating tools raise an approval prompt. "Always" responses cache for the conversation.
- Bypass — everything fires unprompted. Confirm-once on first entry per session; never persisted across restarts.
Every tool invocation — success, denial, or error — lands in a SQLite audit log. Inspect with barry audit ls.
run_bash is double-gated:
- Args-level cwd check — out-of-scope working directories raise a scope-approval prompt.
- OS-level FS jail — the spawned shell child is wrapped in a Landlock LSM ruleset. R+W on project directories, R+X on system roots and curated dev tooling (
~/.cargo,~/.rustup,~/.cache,/tmp, etc.),EACCESon everything else. The agent can runcargo buildbut it can't read your~/.ssh.
A configurable safelist (unsandboxed_commands) bypasses Landlock for trusted binaries when the whole command line composes only of safelisted entries — a perf shortcut for the common cargo / npm / git cases that won't accept arbitrary shells like bash -c.
Model Context Protocol servers attach as additional tool surfaces. Configure them in config.toml:
[[mcp.servers]]
name = "playwright"
transport = "stdio"
command = ["npx", "-y", "@playwright/mcp@latest", "--headless"]
auto_safe = ["browser_snapshot", "browser_take_screenshot"]Subprocesses spawn at REPL startup but their tool schemas are default-off per session — they're not advertised to the model until you run /mcp <name> on inside the REPL. Keeps the per-turn context cheap (Playwright alone is ~4 KB of tool schemas). HTTP transport supports custom headers for bearer-token auth against remote MCP endpoints.
Markdown prompt-recipes invoked as /<name>. Drop *.md files in .barry/skills/ (project-local) or ~/.config/barry/skills/ (user-global), or reuse Claude Code's ~/.claude/skills/ files as-is. Same frontmatter format (name:, description:), same discovery convention. A skill isn't a tool — it's a pre-canned prompt template that fires as the user message; the model still chooses which tools to call.
For models that support it, three thinking modes (cycled with Ctrl+T):
off— model thinking disabled.on— model thinks; the transcript shows a one-line summary indicator (✦ Thought for N tokens · Ms). Open the Alt+T sidebar to view the live reasoning while a turn streams.retention(GLM only) — reasoning blocks preserved across turns for KV-cache hits on long agentic chains.
/effort low|medium|high caps the reasoning budget at 1024 / 4096 / 16384 tokens per turn — a hard ceiling on runaway deliberation.
A SQLite-backed memories table with two scopes:
user— global, available in every conversation.project:<absolute working dir>— visible only when the conversation is rooted at that path.
The model can save_memory / recall_memory / forget_memory / list_memories mid-conversation. At conversation creation, the relevant memory rows are rendered into a frozen system-prompt block — meaning the KV cache stays warm across turns. Inspect manually with /memory list; forget with /memory forget <key>.
Conversations are first-class objects, persisted to SQLite (~/.local/share/barry/conversations.db):
barry resume— interactive picker over your last 10 sessions, orbarry resume <id>straight to a specific one. Working directory, AGENTS.md, and memory snapshot are restored exactly as they were, so the KV cache stays aligned with the original session.barry fork <conv> <turn>— branch off any historic turn. Useful for "what if we'd taken the other path?"barry ls "<query>"— FTS5 full-text search across titles and content.barry show <id>— full transcript dump.barry export <id> --as md|json— round-trip a conversation out of Barry.
On conversation creation the daemon walks up from your working directory looking for an AGENTS.md file (32 KiB cap). If found, its content is spliced into the system prompt — alongside the baked identity and the memory snapshot — and frozen onto the conversation row. Ship project-specific conventions to every Barry session in that tree.
Three layers, ordered cheapest-first: a system-prompt nudge against re-deliberation, the /effort token cap, and a char-level loop detector that watches the reasoning stream and re-rolls the iteration with a fresh seed if it trips. Detector is off by default — only enable it for models with the known repeat-thinking pathology (mostly older Qwen variants).
- Type-ahead queue — submit prompts while a turn is streaming; they auto-submit FIFO once the agent returns to idle.
- Prompt history — per-conversation, walked with ↑/↓.
- Live status bar — context use, mode, thinking state, effort cap, decode rate (
@ 185 t/s), prompt/completion token counts. - Cancel cleanly — Esc unwinds your input stack one layer at a time (clear draft → recall queued prompt → cancel turn).
- Web search — opt-in bundled SearXNG container (
barry-server setup --with-search) for theweb_searchtool, all localhost-bound.
- GPU: 32 GB VRAM is the sweet spot — RTX 5090, A6000, 4090 (with care). All three shipped profiles fit comfortably with Q5_K_M weights and Q8_0 KV-cache quantisation.
- 24 GB cards (4090, 3090): pick a smaller profile (
general-mistral-3.2-24,general-phi-4-14,coder-devstral-1.1-24) viabarry-server download, or override one of the 32-GB profiles' quants. Note the quoted profile header — TOML treats.as a key separator, so versioned names need quoting:[profile."coder-glm-4.7-30"] ctx = 65536 kv_quant = "q4_0" model_file = "zai-org_GLM-4.7-Flash-Q4_K_M.gguf"
- 16 GB cards or smaller: no shipped registry entry currently lands on this tier comfortably.
general-phi-4-14fits 12 GiB+ cards at Q5_K_M but it's HuggingFace-gated (needs--hf-token) and only carries 16K ctx. For a clean small-card option, define a custom profile pointing at a smaller GGUF (any 7B-12B at Q5_K_M) — seedocs/Configuration.md. The download picker shows a ✓/·/✗ recommendation icon per (model, quant) so you can see what fits the detected GPU at a glance. - OS: Linux native or WSL2. macOS and Windows can run the client standalone for cross-LAN use (server stays on the Linux box).
See docs/Configuration.md for the full profile schema.
Three steps on a fresh WSL2 / Linux box with an NVIDIA GPU:
make setup # detect host, pull llama.cpp image, fetch the GGUF, stage config + token
make up # build + start the daemon; the boot profile loads in the background
make repl # open the REPL on a new conversationThat's it. The first make setup run downloads ~22 GiB of model weights and pulls a CUDA-enabled llama.cpp image, so set aside a coffee break. Once the daemon is up, make repl (or just barry from a release build) drops you into the agent.
make setup fetches the Qwen3.6 GGUF, which is what the default general-qwen-3.6-35 profile expects, so the three steps above work end-to-end on a fresh machine. To boot a different profile, see Pre-caching other profiles.
See
defaults/for sample config files (seedocs/Configuration.mdfor details).
cargo build --release
./target/release/barry-server setup # interactive — prompts at each step (incl. download picker)
./target/release/barry-server setup --quick # non-interactive, accept defaults
./target/release/barry-server setup --with-search # also install SearXNG (~150 MB) for web_search
./target/release/barry-server download # standalone interactive picker — fetch extra GGUFs
./target/release/barry-server doctor # read-only re-check of every setup stepSetup detects the host (WSL2 / native Linux + GPU + podman + CDI), pulls the pinned llama.cpp image, downloads a GGUF, writes ~/.config/barry/server.toml + ~/.config/barry/config.toml + the bearer-token file, and optionally installs a systemd --user unit. Idempotent — re-run any time.
make setup (barry-server setup --quick) fetches the Qwen3.6 GGUF — the family used by the default general-qwen-3.6-35 profile. To download a different model use the interactive picker:
./target/release/barry-server downloadArrow-key navigation across the model list, ←/→ to cycle quant, Space to multi-select, Enter to confirm. The picker shows approximate sizes and a VRAM-fit annotation against your detected GPU; downloads stream with a progress bar and ETA, and resume from <file>.partial if interrupted.
Standalone interactive setup (barry-server setup without --quick) opens the same picker mid-checklist, pre-ticking the bundled boot profile so a hands-off Enter does the same thing as --quick while letting power users tick extras.
Filenames live in defaults/server.toml — edit a profile's model_file if you want a different quant or HuggingFace repo than the registry's defaults.
Resolution order: --profile flag → BARRY_PROFILE env → [default_profile] in server.toml → bundled default (general-qwen-3.6-35) → first profile in the catalogue.
BARRY_PROFILE=coder-glm-4.7-30 make up # boot a specific profile for this runOr set it persistently in ~/.config/barry/server.toml:
default_profile = "coder-glm-4.7-30"Hot-swap once the REPL is open: /model <name>. The daemon stops the current container, starts the new one, and re-aligns the agent's sampler. The first turn after a swap is a slow full-prefill (different model, different tokeniser).
make up is a convenience wrapper. The underlying invocations are:
./target/release/barry-server # foreground daemon
./target/release/barry server info # introspect a running daemon
./target/release/barry server status # health probe (auth-exempt)
./target/release/barry # the REPL — default verb is "open a new conversation"Run them on different machines if you want — the client has zero GPU/podman dependencies. Set BARRY_LISTEN=0.0.0.0:7777 on the daemon (already the default) and point the client at the right host:
BARRY_SERVER=http://wsl-box:7777 ./target/release/barrybarry once "summarise the rust ownership model in two sentences" # one-shot, non-interactive
barry resume # interactive picker, last 10 conversations
barry resume <id> # resume a specific conversation
barry ls "deploy" # FTS5 search across title + content
barry show <id> # full transcript
barry fork <conv> <turn> # branch off a turn
barry export <id> --as md # or --as json
barry audit ls --since 1h # what tools fired (incl. denials)
barry mcp ls # configured MCP servers (no daemon round-trip)| Target | What it does |
|---|---|
make up |
Build + start the daemon (auto-loads the boot profile) |
make down |
Stop the daemon (lifecycle stops the model container too) |
make repl |
Open the REPL |
make once PROMPT="..." |
One-shot non-streaming smoke turn |
make model-status |
Show the current model state + profile catalogue |
make model-logs |
Tail the llama container logs |
make server-logs |
Tail the daemon logs (~/.local/state/barry/server.log) |
make server-restart |
Rebuild + bounce the daemon (use after code changes) |
make doctor |
Re-run setup checks read-only |
make audit |
Show audit log entries from the last hour |
make nuke-state |
Wipe Barry metadata for a clean re-test (preserves model weights) |
Barry reads two TOML files, split along the client/server boundary:
| File | Loaded by | Owns |
|---|---|---|
server.toml |
barry-server |
default_profile, [agent], [model], [profile.<name>] |
config.toml |
barry (CLI) |
[search], [shell], [[mcp.servers]] |
Both files are optional — bundled defaults from defaults/server.toml and defaults/config.toml are compiled into the binaries and merged underneath at runtime, so new profiles or knobs added in a future release apply automatically.
Discovery chain (first match wins):
--config <path>(server only) orBARRY_CONFIG/BARRY_CLIENT_CONFIGenv vars.<ancestor>/.barry/<filename>— walked up from cwd. Lets a project ship its own MCP fleet, sandbox tweaks, or profile overrides alongside its.barry/skills/directory.~/.config/barry/<filename>— user-global.barry-server setupwrites the bundled defaults here on first run as a starting point.
# ~/.config/barry/server.toml
# Profile names contain dots, so quote the header — TOML treats
# unquoted dots as table separators.
[profile."coder-glm-4.7-30"]
ctx = 65536 # tighter than the bundled 200000
temperature = 0.5# ~/.config/barry/config.toml
[[mcp.servers]]
name = "filesystem"
transport = "stdio"
command = ["npx", "-y", "@modelcontextprotocol/server-filesystem", "/home/me/projects"]
auto_safe = ["read_file", "list_directory"] # promote to Safe so Safe mode allows them
auto_read = ["read_file", "list_directory"] # fire silently in Default modeFor the full schema — every section, every field, layering rules, migration guidance — see docs/Configuration.md.
| Key / command | Action |
|---|---|
| Enter | Submit prompt (queues if a turn is in flight) |
\ + Enter |
Insert a literal newline |
| Esc | Unwind input stack: clear draft → recall queued prompt → cancel turn |
| ↑ / ↓ | Walk per-conversation prompt history |
| Shift+Tab | Cycle permission mode (Safe → Default → Bypass) |
| Ctrl+T | Cycle thinking mode (off / on / retention) |
| Alt+T | Toggle the live reasoning sidebar |
| PgUp / PgDn | Scroll transcript |
| Ctrl+C ×2 (within ~2 s) | Exit REPL |
/help |
Full slash-command list |
/model <name> |
Hot-swap profile (no conversation reset) |
/think on | off | retention |
Set thinking mode |
/mode safe | default | bypass |
Set permission mode |
/effort low | medium | high |
Cap reasoning tokens |
/mcp <name> on | off |
Toggle an MCP server's tools for this session |
/memory list | show <key> | forget <key> |
Inspect / prune long-term memory |
/skills [list | reload] |
Skill registry ops |
/clear |
Reset the conversation (turns + memory snapshot) |
barry/
├── crates/
│ ├── barry-protocol/ # serde wire types, versioned (v1::*)
│ ├── barry-server/ # axum daemon
│ └── barry-cli/ # `barry` binary (crate name ≠ binary name)
├── defaults/ # bundled config (server.toml + config.toml)
├── docs/ # design docs
├── prompts/ # baked system prompt
└── scripts/ # smoke tests, helper scripts
docs/Architecture.md— runtime topology, request paths, storagedocs/Configuration.md— config files, profiles, schema referencedocs/Setup.md—barry-server setuprunbookdocs/ProtocolV1.md— wire protocoldocs/ThreatModel.md— security modeldocs/ToolCatalogue.md— built-in tools and approval defaults
Dual-licensed under MIT or Apache-2.0.