From 478667735963778d336ab6ba4b2f6a1f6f19b20d Mon Sep 17 00:00:00 2001 From: Daniel Holanda Date: Mon, 24 Aug 2026 14:45:57 -0700 Subject: [PATCH 1/3] Do not assume lemonade port --- skills/local-ai-use/SKILL.md | 57 +++- skills/local-ai-use/reference.md | 38 +-- skills/local-ai-use/scripts/setup_local_ai.py | 254 +++++++++++++++--- 3 files changed, 289 insertions(+), 60 deletions(-) diff --git a/skills/local-ai-use/SKILL.md b/skills/local-ai-use/SKILL.md index 38e0a27..682ce58 100644 --- a/skills/local-ai-use/SKILL.md +++ b/skills/local-ai-use/SKILL.md @@ -33,7 +33,10 @@ The skill does three things: by the OS — so the setup script waits for the service and, if it stays down, prints the exact OS-specific command to start it (e.g. `sudo systemctl start lemond` on Linux). -2. **Verifies that local Lemonade is reachable.** +2. **Verifies that local Lemonade is reachable**, at the endpoint the service + actually bound. The port comes from `lemonade status`, so a server that + already runs on a non-default port is used as-is instead of being reported + missing; `http://localhost:13305` is only the fallback default. 3. **Drops a `Local AI Use` block into the workspace `AGENTS.md`** so the agent reads the routing rule on every later turn, in Cursor, Claude Code, Codex, Gemini CLI, and any other agent that respects `AGENTS.md`. @@ -58,7 +61,8 @@ Use this skill when **all** of the following are true: - The user wants local Lemonade. If it is not yet installed, the setup script installs the latest version for them automatically. -- The user accepts the default Lemonade endpoint `http://localhost:13305`. +- The user accepts the Lemonade endpoint on this machine, whichever port the + service reports (`http://localhost:13305` if it has never been changed). - The user wants the change to be **persistent** across future turns and agent restarts (the rule is written to disk). @@ -162,22 +166,46 @@ After a Windows install the CLI lands in `%LOCALAPPDATA%\lemonade_server` and is added to the *user* PATH (new shells only); the setup script probes that directory so it works in the same run. -**1b. Is the service running?** Check `lemonade status --json`. The `lemond` -service auto-starts on install — there is **no** `lemonade serve` in modern -Lemonade. +**1b. Is the service running, and where?** Check `lemonade status --json`, +which answers both questions at once: it prints the bound port +(`{"port": 13305}`). The `lemond` service auto-starts on install — there is +**no** `lemonade serve` in modern Lemonade. | `lemonade status` says | Action | |---|---| -| `Server is running on port 13305` | Continue to Step 2. | -| `Server is not running` | Wait a few seconds for the auto-started service (the script polls `/api/v1/health`). If it stays down, start it via the OS service manager: `sudo systemctl start lemond` (Linux system install) or `systemctl --user start lemond` (per-user install); `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS); the Lemonade tray app or `Start-Service lemond` (Windows). | +| `Server is running on port ` | Use port `` for every later request and for the rule, even when it is not 13305. Continue to Step 2. | +| `Server is not running` | Wait a few seconds for the auto-started service (the script polls `/api/v1/health`, re-asking `status` in case the service comes up on a different port). If it stays down, start it via the OS service manager: `sudo systemctl start lemond` (Linux system install) or `systemctl --user start lemond` (per-user install); `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS); the Lemonade tray app or `Start-Service lemond` (Windows). | + +Never treat 13305 as the definition of "running": the port is a config value +(`lemonade config set port`), and an existing config, a different install +channel, or a port conflict all move it. Probing only the default reports a +perfectly healthy server as missing. + +`status` is the right way to ask, because the CLI already does the discovery +for you: a running service broadcasts a UDP beacon announcing its API URL, and +the CLI listens for that beacon (enabled by default, `--no-discovery` to opt +out) before falling back to the default port. Do not run `lemonade scan` to +pick the local endpoint — it reports beacons from Lemonade servers on other +machines on the LAN too, and routing this workspace at a colleague's server is +exactly the failure this skill exists to prevent. Use `scan` only when the user +deliberately wants a remote server (see +[reference.md](reference.md#re-pointing-the-rule-at-a-remote-host)). + +The setup script resolves the endpoint in +this order — `--host` / `--port`, then `LEMONADE_HOST` / `LEMONADE_PORT`, then +`lemonade status`, then the install default — and bakes whatever it settles on +into the rule. Pass `--port` explicitly only to override discovery, and pass it +together with `--host` when pointing at another machine, since the local CLI +cannot report a remote service's port. Only if the automatic install genuinely fails (no `apt-get`, no `sudo`, download blocked) should you stop and point the user at . -The rest of this skill assumes the endpoint is `http://localhost:13305/api/v1` -and no API key is required (the system-wide server defaults to no auth on -loopback). If the user has set `LEMONADE_API_KEY`, the routing rule template +The rest of this skill writes the endpoint as `http://localhost:13305/api/v1`, +the default; substitute the port `lemonade status` reported if it differs. It +also assumes no API key is required (the system-wide server defaults to no auth +on loopback). If the user has set `LEMONADE_API_KEY`, the routing rule template in `templates/local-ai-rule.md` shows where to add the `Authorization` header. **1c. Are the backends ready per modality?** Backend health is **per @@ -283,6 +311,7 @@ machine. |---|---|---| | `lemonade: command not found` | CLI not installed | Re-run `python scripts/setup_local_ai.py` (auto-installs the latest version). If it just installed on Windows, open a new shell so the user PATH refreshes, or the script will find it under `%LOCALAPPDATA%\lemonade_server`. | | `status` gives an "invalid choice" / usage error | An old `lemonade` (pre-v10.1.0) is shadowing the modern CLI | Uninstall it (see the Step 1a table: `winget uninstall -e --id AMD.LemonadeServer` / `sudo apt remove lemonade-server` / `brew uninstall --cask lemonade-server`), then re-run the setup script. | +| Requests to `http://localhost:13305` are refused, but `lemonade status` says the server is running | The service is bound to a non-default port (existing config, another install channel, or a port conflict) | Use the port from `lemonade status --json` and re-run `python scripts/setup_local_ai.py` so the rule is rewritten with it. Do not start a second server; the running one is fine. | | `Server is not running` | `lemond` service stopped | Start it via the OS service manager — `sudo systemctl start lemond` / `systemctl --user start lemond` (Linux), `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS), or the tray app / `Start-Service lemond` (Windows). There is no `lemonade serve`. | | `POST /v1/images/generations` returns 404 model not found | Image model not downloaded | `lemonade pull SD-Turbo` and retry. | | `lemonade pull` keeps printing `Progress: NN%` but never finishes | Download target is a bad path (out of space, no write permission, quota, read-only mount). The write error may surface only in the server log while the console keeps showing progress | Check the target and free space first: `GET /api/v1/system-info` reports `models_dir` and `model_storage.free_bytes`. If a pull stalls, read the recent lines of the server log (typically `lemonade-server.log` in the OS temp dir) for the real error (e.g. a download/write failure like `CURL code 23`, or an out-of-space message), then point the download at a writable disk with room. | @@ -296,14 +325,16 @@ machine. Mark this skill complete only when **all** of the following are true: -- [ ] `lemonade status --json` reports the server running on port 13305. +- [ ] `lemonade status --json` reports the server running, and the port it + reports is the port in the installed rule (13305 unless the service was + configured otherwise). - [ ] The workspace `AGENTS.md` contains the `amd-skills:local-ai-use` block. This is required even when Lemonade was already installed and running — generating an image alone does not complete the skill. - [ ] On a follow-up turn, asking the agent to "generate an image of X" - causes it to POST to `http://localhost:13305/api/v1/images/generations` - (pulling the model on first use) rather than calling a cloud tool. + causes it to POST to `/api/v1/images/generations` on the endpoint in the + rule (pulling the model on first use) rather than calling a cloud tool. - [ ] `lemonade backends --all` shows `installed` for every backend variant this workspace's routing depends on (see Step 1c). Do not treat a working image or chat path as proof that transcription will work. diff --git a/skills/local-ai-use/reference.md b/skills/local-ai-use/reference.md index 824813d..da8ccd3 100644 --- a/skills/local-ai-use/reference.md +++ b/skills/local-ai-use/reference.md @@ -21,11 +21,6 @@ The default trio (`SD-Turbo`, `kokoro-v1`, `Whisper-Tiny`) is sized for "keeps cost savings real on a typical laptop". Override only if the user asks for higher quality or has explicit hardware to spare. -Model catalogs move between releases, so treat the IDs below as a starting -point, not a fixed list. Confirm any ID with `GET /api/v1/models` (add -`?show_all=true` for the full catalog) before writing it into the rule. Do -not rely on a stale `server_models.json` snapshot. - ### Image generation (`recipe: sd-cpp`) | Model | Approx size | When to use | Trade-off | @@ -69,10 +64,6 @@ Whisper requires 16 kHz mono PCM WAV input. Convert anything else first: ffmpeg -i input.mp3 -ar 16000 -ac 1 input.wav ``` -The HTTP endpoint adds no meaningful overhead over calling `whisper-cli` -directly on the same engine/backend/model. Throughput is equivalent; use the -endpoint unless you have a specific reason not to. - For full live coverage, run `lemonade list` after starting the server, or browse . @@ -107,8 +98,7 @@ Notable per-endpoint quirks: - **`/v1/audio/transcriptions`**: only `wav` input is supported; re-encode anything else with `ffmpeg`. `response_format` accepts `json`, `verbose_json`, `text`, `srt`, and `vtt` — the first two return JSON, the - rest return a raw text body (as of Lemonade 11.7.0). Any other value is a - 400. + rest return a raw text body (as of Lemonade 11.7.0). - **`/v1/audio/speech`**: `mp3`, `wav`, `opus`, and `pcm` outputs supported. Streaming requires `stream_format: "audio"`, which only emits `pcm`. @@ -235,15 +225,33 @@ read `examples/lemonade_tools.py` in the upstream lemonade-sdk repo. Lemonade can run on another machine (a workstation with a Ryzen AI NPU, say) while the agent runs on the laptop. To point this skill at it: -1. Set `LEMONADE_HOST` and `LEMONADE_PORT` (or pass `--host` / `--port` to - `setup_local_ai.py`). -2. Re-run `python scripts/setup_local_ai.py` so the rule block is rewritten +1. Find the remote endpoint with `lemonade scan --duration 5`, which listens + for the UDP beacons Lemonade servers broadcast on the local network and + prints each one as ` at http://:/api/v1/`: + +``` +Scan complete. Found 2 beacon(s): + - workstation at http://192.168.0.233:13305/api/v1/ + - my-laptop at http://127.0.0.1:13305/api/v1/ +``` + +2. Set `LEMONADE_HOST` and `LEMONADE_PORT` to the host and port from the + beacon you want (or pass `--host` / `--port` to `setup_local_ai.py`). Set + **both**: the script otherwise discovers the port via `lemonade status`, + which speaks only for the service on this machine, so a remote host given + without a port falls back to 13305. +3. Re-run `python scripts/setup_local_ai.py` so the rule block is rewritten with the new endpoint baked in. -3. Make sure the remote server is bound to a non-loopback interface +4. Make sure the remote server is bound to a non-loopback interface (`lemonade config set host 0.0.0.0`) and that firewall rules allow inbound 13305. Setting `host` to `0.0.0.0` exposes the server; pair it with `LEMONADE_API_KEY` so it isn't open to the LAN. +A server that `scan` cannot see is not necessarily down: beacons are UDP, are +only broadcast on RFC1918 (private) networks, and can be switched off with +`--no-broadcast` or `lemonade config set no_broadcast=true`. Fall back to +asking the user for the host and port in that case. + --- ## Removing the rule diff --git a/skills/local-ai-use/scripts/setup_local_ai.py b/skills/local-ai-use/scripts/setup_local_ai.py index 3184f2a..4c175a1 100644 --- a/skills/local-ai-use/scripts/setup_local_ai.py +++ b/skills/local-ai-use/scripts/setup_local_ai.py @@ -12,13 +12,15 @@ Performs the setup steps from SKILL.md: 1. Ensures modern Lemonade is installed and its background service (the - `lemond` daemon) is reachable on http://localhost:13305 (override with - --host / --port or LEMONADE_HOST / LEMONADE_PORT). If no modern - `lemonade` CLI is found, the latest version is installed on the user's - behalf. The daemon auto-starts on install and is managed by the OS - service manager, so this script never runs a `serve` command; it waits - for the service to come up and, if it does not, prints the OS-specific - start command and exits non-zero. + `lemond` daemon) is reachable. The endpoint is whatever the user asks + for (--host / --port or LEMONADE_HOST / LEMONADE_PORT), else whatever + `lemonade status` reports the service actually bound, else the install + default http://localhost:13305. If no modern `lemonade` CLI is found, + the latest version is installed on the user's behalf. The daemon + auto-starts on install and is managed by the OS service manager, so this + script never runs a `serve` command; it waits for the service to come up + and, if it does not, prints the OS-specific start command and exits + non-zero. 2. Writes the routing rule from `templates/local-ai-rule.md` into /AGENTS.md, between stable BEGIN/END markers so re-runs replace the block in place rather than appending. @@ -59,10 +61,30 @@ # Defaults match the system-wide Lemonade Server install. Both the CLI # (LEMONADE_HOST / LEMONADE_PORT) and the OpenAI-compatible HTTP endpoints -# bind to these by default. +# bind to these by default. They are only a last resort: the service binds +# whatever its own config says, so `lemonade status` is asked first (see +# resolve_endpoint). DEFAULT_HOST = "127.0.0.1" DEFAULT_PORT = 13305 +# Hosts that mean "the Lemonade service on this machine". Only for those is +# the local CLI authoritative about the port; a user-supplied remote host has +# its own service and its own config, which `lemonade status` here cannot see. +LOCAL_HOSTS = {"127.0.0.1", "localhost", "::1", "0.0.0.0", "::"} + +# A service bound to a wildcard address is reached over loopback, not by +# connecting to the wildcard itself. +WILDCARD_HOSTS = {"0.0.0.0", "::"} + +# Human-readable `lemonade status` says "Server is running on port 13305"; +# `status --json` says {"port": 13305}. Anchoring on "running on" keeps this +# off the other ports the same output lists (e.g. "WebSocket Port 9001"), and +# the optional URL form covers builds that print "running on http://host:port". +STATUS_TEXT_ENDPOINT_RE = re.compile( + r"running\s+on\s+(?:https?://(?P[^\s:/]+):)?(?:port\s+)?(?P\d{2,5})", + re.IGNORECASE, +) + # Picked because each default fits in under ~5 GB and runs on commodity CPU # hardware, so the savings vs. cloud calls are real on a typical developer # laptop. See SKILL.md for upgrade paths. @@ -172,6 +194,22 @@ def _candidate_clis() -> list[str]: return candidates +def _run_status(cli: str, *, as_json: bool = False) -> tuple[str, str] | None: + """Run ` status` and return (stdout, stderr), or None if it did not run. + + `status` is read-only and cheap, which is why it doubles as the modern-CLI + probe and the endpoint lookup. + """ + cmd = [cli, "status"] + if as_json: + cmd.append("--json") + try: + result = subprocess.run(cmd, capture_output=True, text=True, timeout=20) + except (OSError, subprocess.SubprocessError): + return None + return result.stdout, result.stderr + + def is_modern_cli(cli: str) -> bool: """True if `cli` is the modern Lemonade CLI (drives the `lemond` service). @@ -186,16 +224,10 @@ def is_modern_cli(cli: str) -> bool: (exiting 0 or 1 accordingly). We key off that phrasing rather than the exit code alone. """ - try: - result = subprocess.run( - [cli, "status"], - capture_output=True, - text=True, - timeout=20, - ) - except (OSError, subprocess.SubprocessError): + streams = _run_status(cli) + if streams is None: return False - blob = f"{result.stdout}\n{result.stderr}".lower() + blob = "\n".join(streams).lower() # Modern `lemonade status` always reports on the server, whether or not it # is running. An old/incompatible CLI never prints this phrasing (it errors # with "invalid choice: 'status'"), so the presence of the phrase is a @@ -203,6 +235,109 @@ def is_modern_cli(cli: str) -> bool: return "server is running" in blob or "server is not running" in blob +def _valid_port(value: object) -> int | None: + """Coerce a reported port to an int, or None if it is not a usable port.""" + try: + port = int(value) # type: ignore[arg-type] + except (TypeError, ValueError): + return None + return port if 1 <= port <= 65535 else None + + +def _connectable_host(host: str | None) -> str | None: + """Normalise a reported bind address into one we can actually connect to.""" + if not host: + return None + return DEFAULT_HOST if host.lower() in WILDCARD_HOSTS else host + + +def _parse_status_json(stdout: str) -> tuple[str | None, int | None]: + """Pull host/port out of `status --json` output (today: `{"port": 13305}`).""" + try: + data = json.loads(stdout.strip() or "{}") + except ValueError: + return None, None + if not isinstance(data, dict): + return None, None + host = data.get("host") + return ( + _connectable_host(host if isinstance(host, str) else None), + _valid_port(data.get("port")), + ) + + +def discover_endpoint(cli: str) -> tuple[str | None, int | None]: + """Ask the CLI where the service is actually bound. + + The port is a config value (`lemonade config set port`), and an existing + install, a different install channel, or a port conflict can all move it. + Treating DEFAULT_PORT as the definition of "reachable" makes a healthy + server on any other port look absent, so ask instead of assuming. + + Asking the CLI *is* the beacon route: the running service broadcasts a UDP + beacon announcing its API URL, and the CLI listens for it (discovery is on + by default) before falling back to the default port. So one `status` call + gets us the beacon's answer, already resolved, with no scan window. Do not + replace this with `lemonade scan` or a UDP listener of our own: beacons + also arrive from Lemonade servers on other machines on the LAN, and this + skill must route to the user's own machine. + + `status --json` is authoritative; the human-readable output is the + fallback for builds where `--json` is missing or reshaped. Returns + ``(host, port)`` with either element None when the CLI does not report it + -- a stopped service prints "Server is not running" and no port. + """ + host: str | None = None + port: int | None = None + + streams = _run_status(cli, as_json=True) + if streams is not None: + host, port = _parse_status_json(streams[0]) + + if port is None: + streams = _run_status(cli) + if streams is not None: + match = STATUS_TEXT_ENDPOINT_RE.search("\n".join(streams)) + if match is not None: + host = host or _connectable_host(match.group("host")) + port = _valid_port(match.group("port")) + + return host, port + + +def is_local_host(host: str) -> bool: + """True if `host` names the Lemonade service on this machine.""" + return host.lower() in LOCAL_HOSTS + + +def resolve_endpoint( + cli: str | None, + host_arg: str | None, + port_arg: int | None, +) -> tuple[str, int, str]: + """Pick the endpoint to health-check, and say where the answer came from. + + Precedence: what the user asked for (--host / --port, or the matching env + vars), then what `lemonade status` reports, then the install default. The + middle step is what stops a server on a non-default port from reading as + "not running". + """ + if port_arg is None and cli is not None and is_local_host(host_arg or DEFAULT_HOST): + discovered_host, discovered_port = discover_endpoint(cli) + if discovered_port is not None: + return ( + host_arg or discovered_host or DEFAULT_HOST, + discovered_port, + f"`{CLI_NAME} status`", + ) + requested = host_arg is not None or port_arg is not None + return ( + host_arg or DEFAULT_HOST, + port_arg if port_arg is not None else DEFAULT_PORT, + "the requested endpoint" if requested else "the install default", + ) + + def find_cli() -> tuple[str | None, str | None]: """Locate a Lemonade CLI and classify it. @@ -380,14 +515,32 @@ def uninstall_hint() -> str: return f"remove it with your platform's package manager; see {INSTALL_DOCS_URL}" -def wait_for_server(host: str, port: int, timeout_s: float = 90.0) -> bool: - """Poll /api/v1/health until it answers 200 or we hit the timeout.""" +def wait_for_server( + host: str, + port: int, + *, + cli: str | None = None, + timeout_s: float = 90.0, +) -> tuple[bool, str, int]: + """Poll /api/v1/health until it answers 200 or we hit the timeout. + + Returns ``(reachable, host, port)``. When `cli` is given the bound port is + re-checked on every pass, because a service that is still starting has no + port to report yet and may come up on one other than our first guess. + """ deadline = time.monotonic() + timeout_s while time.monotonic() < deadline: if check_server_reachable(host, port): - return True + return True, host, port + if cli is not None: + discovered_host, discovered_port = discover_endpoint(cli) + host = discovered_host or host + if discovered_port is not None and discovered_port != port: + _print(f"`{CLI_NAME} status` reports port {discovered_port}; checking there.") + port = discovered_port + continue time.sleep(2.0) - return False + return False, host, port def check_server_reachable(host: str, port: int) -> bool: @@ -498,6 +651,25 @@ def upsert_agents_md( return target +def _env_port() -> int | None: + """LEMONADE_PORT as an int, dropping an unusable value from the environment. + + The `lemonade` CLI reads the same variable and refuses to start on a value + it cannot parse ("Could not convert: --port = ..."). Leaving such a value + in the environment our subprocesses inherit would turn a typo into a + misdiagnosed stale CLI, so an unusable value is removed here, once, before + anything shells out. + """ + raw = os.environ.get("LEMONADE_PORT") + if not raw: + return None + port = _valid_port(raw) + if port is None: + _print(f"ignoring LEMONADE_PORT={raw!r}: not a port number.") + os.environ.pop("LEMONADE_PORT", None) + return port + + def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument( @@ -508,14 +680,17 @@ def main(argv: list[str] | None = None) -> int: ) parser.add_argument( "--host", - default=os.environ.get("LEMONADE_HOST", DEFAULT_HOST), - help="Lemonade Server host (default: 127.0.0.1 / $LEMONADE_HOST).", + default=None, + help="Lemonade Server host (default: $LEMONADE_HOST, else 127.0.0.1).", ) parser.add_argument( "--port", type=int, - default=int(os.environ.get("LEMONADE_PORT", str(DEFAULT_PORT))), - help="Lemonade Server port (default: 13305 / $LEMONADE_PORT).", + default=None, + help=( + "Lemonade Server port (default: $LEMONADE_PORT, else the port " + f"`{CLI_NAME} status` reports, else {DEFAULT_PORT})." + ), ) parser.add_argument( "--image-model", @@ -539,6 +714,11 @@ def main(argv: list[str] | None = None) -> int: ) args = parser.parse_args(argv) + # Read the endpoint request before shelling out: the CLI inherits these + # env vars, so an unusable one has to be dealt with first. + host_arg = args.host or os.environ.get("LEMONADE_HOST") or None + port_arg = args.port if args.port is not None else _env_port() + cli, stale = find_cli() # An old `lemonade` on PATH shadows the modern CLI. We never drive or @@ -578,34 +758,44 @@ def main(argv: list[str] | None = None) -> int: return 2 _print(f"using Lemonade CLI: {cli}") + # Ask the service where it bound before assuming the default port; an + # existing config or a port conflict moves it, and a "not running" verdict + # from the wrong port would be baked into the rule as well. + host, port, source = resolve_endpoint(cli, host_arg, port_arg) + _print(f"endpoint http://{host}:{port} (from {source})") + + # Only a local CLI can correct a port we did not pin ourselves. + rediscover_with = cli if port_arg is None and is_local_host(host) else None + # Modern Lemonade auto-starts the `lemond` service on install; there is no # `lemonade serve`. If it is not up yet (e.g. still starting right after a # fresh install), poll briefly, then guide the user to start the OS # service rather than trying to spawn it ourselves. - if not check_server_reachable(args.host, args.port): + if not check_server_reachable(host, port): if args.no_install: _print( f"FAIL: Lemonade Server is not responding at " - f"http://{args.host}:{args.port}/api/v1/health (--no-install set)." + f"http://{host}:{port}/api/v1/health (--no-install set)." ) _print(f"Start the service: {service_start_hint()}") return 3 _print("Lemonade service not reachable yet; waiting for it to come up.") - if not wait_for_server(args.host, args.port): + reachable, host, port = wait_for_server(host, port, cli=rediscover_with) + if not reachable: _print( f"FAIL: the Lemonade service did not become reachable at " - f"http://{args.host}:{args.port}/api/v1/health." + f"http://{host}:{port}/api/v1/health." ) _print(f"Start it manually, then re-run: {service_start_hint()}") _print(f"If it is not installed, see {INSTALL_DOCS_URL}") return 3 - _print(f"server reachable at http://{args.host}:{args.port}") + _print(f"server reachable at http://{host}:{port}") upsert_agents_md( args.workspace.resolve(), - host=args.host, - port=args.port, + host=host, + port=port, image_model=args.image_model, tts_model=args.tts_model, stt_model=args.stt_model, From 0f558a6fb3cb32c6421e3e718f5992b5afb3a9f3 Mon Sep 17 00:00:00 2001 From: Daniel Holanda Date: Mon, 24 Aug 2026 14:53:44 -0700 Subject: [PATCH 2/3] simplify --- skills/local-ai-use/SKILL.md | 48 ++-- skills/local-ai-use/reference.md | 29 +- skills/local-ai-use/scripts/setup_local_ai.py | 255 +++++------------- 3 files changed, 99 insertions(+), 233 deletions(-) diff --git a/skills/local-ai-use/SKILL.md b/skills/local-ai-use/SKILL.md index 682ce58..1f6b061 100644 --- a/skills/local-ai-use/SKILL.md +++ b/skills/local-ai-use/SKILL.md @@ -33,10 +33,9 @@ The skill does three things: by the OS — so the setup script waits for the service and, if it stays down, prints the exact OS-specific command to start it (e.g. `sudo systemctl start lemond` on Linux). -2. **Verifies that local Lemonade is reachable**, at the endpoint the service - actually bound. The port comes from `lemonade status`, so a server that - already runs on a non-default port is used as-is instead of being reported - missing; `http://localhost:13305` is only the fallback default. +2. **Verifies that local Lemonade is reachable** on the port `lemonade status` + reports, so a server already running on a non-default port is used as-is + rather than reported missing (`13305` is only the fallback). 3. **Drops a `Local AI Use` block into the workspace `AGENTS.md`** so the agent reads the routing rule on every later turn, in Cursor, Claude Code, Codex, Gemini CLI, and any other agent that respects `AGENTS.md`. @@ -167,9 +166,9 @@ is added to the *user* PATH (new shells only); the setup script probes that directory so it works in the same run. **1b. Is the service running, and where?** Check `lemonade status --json`, -which answers both questions at once: it prints the bound port -(`{"port": 13305}`). The `lemond` service auto-starts on install — there is -**no** `lemonade serve` in modern Lemonade. +which answers both at once by printing the bound port (`{"port": 13305}`). The +`lemond` service auto-starts on install — there is **no** `lemonade serve` in +modern Lemonade. | `lemonade status` says | Action | |---|---| @@ -177,35 +176,24 @@ which answers both questions at once: it prints the bound port | `Server is not running` | Wait a few seconds for the auto-started service (the script polls `/api/v1/health`, re-asking `status` in case the service comes up on a different port). If it stays down, start it via the OS service manager: `sudo systemctl start lemond` (Linux system install) or `systemctl --user start lemond` (per-user install); `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS); the Lemonade tray app or `Start-Service lemond` (Windows). | Never treat 13305 as the definition of "running": the port is a config value -(`lemonade config set port`), and an existing config, a different install -channel, or a port conflict all move it. Probing only the default reports a -perfectly healthy server as missing. - -`status` is the right way to ask, because the CLI already does the discovery -for you: a running service broadcasts a UDP beacon announcing its API URL, and -the CLI listens for that beacon (enabled by default, `--no-discovery` to opt -out) before falling back to the default port. Do not run `lemonade scan` to -pick the local endpoint — it reports beacons from Lemonade servers on other -machines on the LAN too, and routing this workspace at a colleague's server is -exactly the failure this skill exists to prevent. Use `scan` only when the user -deliberately wants a remote server (see -[reference.md](reference.md#re-pointing-the-rule-at-a-remote-host)). - -The setup script resolves the endpoint in -this order — `--host` / `--port`, then `LEMONADE_HOST` / `LEMONADE_PORT`, then -`lemonade status`, then the install default — and bakes whatever it settles on -into the rule. Pass `--port` explicitly only to override discovery, and pass it -together with `--host` when pointing at another machine, since the local CLI -cannot report a remote service's port. +(`lemonade config set port`) that an existing config, another install channel, +or a port conflict all move, so probing only the default reports a healthy +server as missing. Ask `status`, which resolves the port from the service's UDP +beacon for you — `lemonade scan` is for finding servers on *other* machines +(see [reference.md](reference.md#re-pointing-the-rule-at-a-remote-host)), never +for picking the local endpoint. The setup script prefers `--host` / `--port` or +`LEMONADE_HOST` / `LEMONADE_PORT` over discovery, and bakes whatever it settles +on into the rule; pass both when pointing at another machine, since the CLI +here cannot report a remote service's port. Only if the automatic install genuinely fails (no `apt-get`, no `sudo`, download blocked) should you stop and point the user at . The rest of this skill writes the endpoint as `http://localhost:13305/api/v1`, -the default; substitute the port `lemonade status` reported if it differs. It -also assumes no API key is required (the system-wide server defaults to no auth -on loopback). If the user has set `LEMONADE_API_KEY`, the routing rule template +the default; substitute the port `status` reported if it differs. It also +assumes no API key is required (the system-wide server defaults to no auth on +loopback). If the user has set `LEMONADE_API_KEY`, the routing rule template in `templates/local-ai-rule.md` shows where to add the `Authorization` header. **1c. Are the backends ready per modality?** Backend health is **per diff --git a/skills/local-ai-use/reference.md b/skills/local-ai-use/reference.md index da8ccd3..c3b7ee4 100644 --- a/skills/local-ai-use/reference.md +++ b/skills/local-ai-use/reference.md @@ -225,21 +225,13 @@ read `examples/lemonade_tools.py` in the upstream lemonade-sdk repo. Lemonade can run on another machine (a workstation with a Ryzen AI NPU, say) while the agent runs on the laptop. To point this skill at it: -1. Find the remote endpoint with `lemonade scan --duration 5`, which listens - for the UDP beacons Lemonade servers broadcast on the local network and - prints each one as ` at http://:/api/v1/`: - -``` -Scan complete. Found 2 beacon(s): - - workstation at http://192.168.0.233:13305/api/v1/ - - my-laptop at http://127.0.0.1:13305/api/v1/ -``` - -2. Set `LEMONADE_HOST` and `LEMONADE_PORT` to the host and port from the - beacon you want (or pass `--host` / `--port` to `setup_local_ai.py`). Set - **both**: the script otherwise discovers the port via `lemonade status`, - which speaks only for the service on this machine, so a remote host given - without a port falls back to 13305. +1. Find it with `lemonade scan --duration 5`, which lists the UDP beacons + Lemonade servers broadcast on the local network as + ` at http://:/api/v1/`. +2. Set `LEMONADE_HOST` and `LEMONADE_PORT` from that beacon (or pass + `--host` / `--port` to `setup_local_ai.py`). Set **both**: the script + otherwise discovers the port via `lemonade status`, which speaks only for + the service on this machine, so a remote host alone keeps port 13305. 3. Re-run `python scripts/setup_local_ai.py` so the rule block is rewritten with the new endpoint baked in. 4. Make sure the remote server is bound to a non-loopback interface @@ -247,10 +239,9 @@ Scan complete. Found 2 beacon(s): inbound 13305. Setting `host` to `0.0.0.0` exposes the server; pair it with `LEMONADE_API_KEY` so it isn't open to the LAN. -A server that `scan` cannot see is not necessarily down: beacons are UDP, are -only broadcast on RFC1918 (private) networks, and can be switched off with -`--no-broadcast` or `lemonade config set no_broadcast=true`. Fall back to -asking the user for the host and port in that case. +A server `scan` cannot see is not necessarily down: beacons are UDP, only go +out on RFC1918 networks, and can be disabled (`--no-broadcast`). Ask the user +for the host and port in that case. --- diff --git a/skills/local-ai-use/scripts/setup_local_ai.py b/skills/local-ai-use/scripts/setup_local_ai.py index 4c175a1..a5ebcbd 100644 --- a/skills/local-ai-use/scripts/setup_local_ai.py +++ b/skills/local-ai-use/scripts/setup_local_ai.py @@ -12,15 +12,14 @@ Performs the setup steps from SKILL.md: 1. Ensures modern Lemonade is installed and its background service (the - `lemond` daemon) is reachable. The endpoint is whatever the user asks - for (--host / --port or LEMONADE_HOST / LEMONADE_PORT), else whatever - `lemonade status` reports the service actually bound, else the install - default http://localhost:13305. If no modern `lemonade` CLI is found, - the latest version is installed on the user's behalf. The daemon - auto-starts on install and is managed by the OS service manager, so this - script never runs a `serve` command; it waits for the service to come up - and, if it does not, prints the OS-specific start command and exits - non-zero. + `lemond` daemon) is reachable. The port is the one `lemonade status` + reports, unless --host / --port or LEMONADE_HOST / LEMONADE_PORT say + otherwise; http://localhost:13305 is only the fallback. If no modern + `lemonade` CLI is found, the latest version is installed on the user's + behalf. The daemon auto-starts on install and is managed by the OS + service manager, so this script never runs a `serve` command; it waits + for the service to come up and, if it does not, prints the OS-specific + start command and exits non-zero. 2. Writes the routing rule from `templates/local-ai-rule.md` into /AGENTS.md, between stable BEGIN/END markers so re-runs replace the block in place rather than appending. @@ -61,29 +60,15 @@ # Defaults match the system-wide Lemonade Server install. Both the CLI # (LEMONADE_HOST / LEMONADE_PORT) and the OpenAI-compatible HTTP endpoints -# bind to these by default. They are only a last resort: the service binds -# whatever its own config says, so `lemonade status` is asked first (see -# resolve_endpoint). +# bind to these by default -- but only by default, so the port is a fallback +# for when `lemonade status` cannot tell us the real one (see resolve_port). DEFAULT_HOST = "127.0.0.1" DEFAULT_PORT = 13305 -# Hosts that mean "the Lemonade service on this machine". Only for those is -# the local CLI authoritative about the port; a user-supplied remote host has -# its own service and its own config, which `lemonade status` here cannot see. -LOCAL_HOSTS = {"127.0.0.1", "localhost", "::1", "0.0.0.0", "::"} - -# A service bound to a wildcard address is reached over loopback, not by -# connecting to the wildcard itself. -WILDCARD_HOSTS = {"0.0.0.0", "::"} - -# Human-readable `lemonade status` says "Server is running on port 13305"; -# `status --json` says {"port": 13305}. Anchoring on "running on" keeps this -# off the other ports the same output lists (e.g. "WebSocket Port 9001"), and -# the optional URL form covers builds that print "running on http://host:port". -STATUS_TEXT_ENDPOINT_RE = re.compile( - r"running\s+on\s+(?:https?://(?P[^\s:/]+):)?(?:port\s+)?(?P\d{2,5})", - re.IGNORECASE, -) +# `lemonade status --json` reports `{"port": 13305}`; the human-readable form +# says "Server is running on port 13305". Anchoring on "running on" keeps this +# off the other ports the same output lists (e.g. "WebSocket Port 9001"). +STATUS_PORT_RE = re.compile(r"running on port (\d+)", re.IGNORECASE) # Picked because each default fits in under ~5 GB and runs on commodity CPU # hardware, so the savings vs. cloud calls are real on a typical developer @@ -194,20 +179,18 @@ def _candidate_clis() -> list[str]: return candidates -def _run_status(cli: str, *, as_json: bool = False) -> tuple[str, str] | None: - """Run ` status` and return (stdout, stderr), or None if it did not run. +def _run_status(cli: str, *, as_json: bool = False) -> str | None: + """Combined output of ` status`, or None if it did not run. `status` is read-only and cheap, which is why it doubles as the modern-CLI - probe and the endpoint lookup. + probe and the port lookup. """ - cmd = [cli, "status"] - if as_json: - cmd.append("--json") + cmd = [cli, "status"] + (["--json"] if as_json else []) try: result = subprocess.run(cmd, capture_output=True, text=True, timeout=20) except (OSError, subprocess.SubprocessError): return None - return result.stdout, result.stderr + return f"{result.stdout}\n{result.stderr}" def is_modern_cli(cli: str) -> bool: @@ -224,10 +207,7 @@ def is_modern_cli(cli: str) -> bool: (exiting 0 or 1 accordingly). We key off that phrasing rather than the exit code alone. """ - streams = _run_status(cli) - if streams is None: - return False - blob = "\n".join(streams).lower() + blob = (_run_status(cli) or "").lower() # Modern `lemonade status` always reports on the server, whether or not it # is running. An old/incompatible CLI never prints this phrasing (it errors # with "invalid choice: 'status'"), so the presence of the phrase is a @@ -235,107 +215,39 @@ def is_modern_cli(cli: str) -> bool: return "server is running" in blob or "server is not running" in blob -def _valid_port(value: object) -> int | None: - """Coerce a reported port to an int, or None if it is not a usable port.""" - try: - port = int(value) # type: ignore[arg-type] - except (TypeError, ValueError): - return None - return port if 1 <= port <= 65535 else None - +def discover_port(cli: str) -> int | None: + """The port the service actually bound, or None if it is not running. -def _connectable_host(host: str | None) -> str | None: - """Normalise a reported bind address into one we can actually connect to.""" - if not host: - return None - return DEFAULT_HOST if host.lower() in WILDCARD_HOSTS else host - - -def _parse_status_json(stdout: str) -> tuple[str | None, int | None]: - """Pull host/port out of `status --json` output (today: `{"port": 13305}`).""" - try: - data = json.loads(stdout.strip() or "{}") - except ValueError: - return None, None - if not isinstance(data, dict): - return None, None - host = data.get("host") - return ( - _connectable_host(host if isinstance(host, str) else None), - _valid_port(data.get("port")), - ) - - -def discover_endpoint(cli: str) -> tuple[str | None, int | None]: - """Ask the CLI where the service is actually bound. - - The port is a config value (`lemonade config set port`), and an existing - install, a different install channel, or a port conflict can all move it. - Treating DEFAULT_PORT as the definition of "reachable" makes a healthy - server on any other port look absent, so ask instead of assuming. - - Asking the CLI *is* the beacon route: the running service broadcasts a UDP - beacon announcing its API URL, and the CLI listens for it (discovery is on - by default) before falling back to the default port. So one `status` call - gets us the beacon's answer, already resolved, with no scan window. Do not - replace this with `lemonade scan` or a UDP listener of our own: beacons - also arrive from Lemonade servers on other machines on the LAN, and this - skill must route to the user's own machine. - - `status --json` is authoritative; the human-readable output is the - fallback for builds where `--json` is missing or reshaped. Returns - ``(host, port)`` with either element None when the CLI does not report it - -- a stopped service prints "Server is not running" and no port. + The port is a config value (`lemonade config set port`) that an existing + config, another install channel, or a port conflict all move, so treating + DEFAULT_PORT as the definition of "reachable" hides a healthy server. The + CLI resolves this from the service's UDP beacon for us; do not scan for + beacons here, because those also arrive from servers on other machines. """ - host: str | None = None - port: int | None = None - - streams = _run_status(cli, as_json=True) - if streams is not None: - host, port = _parse_status_json(streams[0]) - - if port is None: - streams = _run_status(cli) - if streams is not None: - match = STATUS_TEXT_ENDPOINT_RE.search("\n".join(streams)) - if match is not None: - host = host or _connectable_host(match.group("host")) - port = _valid_port(match.group("port")) - - return host, port - - -def is_local_host(host: str) -> bool: - """True if `host` names the Lemonade service on this machine.""" - return host.lower() in LOCAL_HOSTS + blob = _run_status(cli, as_json=True) + if blob: + try: + port = json.loads(blob.strip()).get("port") + except (AttributeError, ValueError): + port = None + if isinstance(port, int): + return port + blob = _run_status(cli) # older builds may not support --json + match = STATUS_PORT_RE.search(blob) if blob else None + return int(match.group(1)) if match else None -def resolve_endpoint( - cli: str | None, - host_arg: str | None, - port_arg: int | None, -) -> tuple[str, int, str]: - """Pick the endpoint to health-check, and say where the answer came from. +def resolve_port(cli: str | None, host_arg: str | None, port_arg: int | None) -> int: + """Port to health-check: what was asked for, else what `status` reports. - Precedence: what the user asked for (--host / --port, or the matching env - vars), then what `lemonade status` reports, then the install default. The - middle step is what stops a server on a non-default port from reading as - "not running". + An explicit host is taken at face value: the CLI here knows nothing about + another machine's config, so a remote host keeps the default port. """ - if port_arg is None and cli is not None and is_local_host(host_arg or DEFAULT_HOST): - discovered_host, discovered_port = discover_endpoint(cli) - if discovered_port is not None: - return ( - host_arg or discovered_host or DEFAULT_HOST, - discovered_port, - f"`{CLI_NAME} status`", - ) - requested = host_arg is not None or port_arg is not None - return ( - host_arg or DEFAULT_HOST, - port_arg if port_arg is not None else DEFAULT_PORT, - "the requested endpoint" if requested else "the install default", - ) + if port_arg is not None: + return port_arg + if cli is not None and host_arg is None: + return discover_port(cli) or DEFAULT_PORT + return DEFAULT_PORT def find_cli() -> tuple[str | None, str | None]: @@ -516,31 +428,21 @@ def uninstall_hint() -> str: def wait_for_server( - host: str, - port: int, - *, - cli: str | None = None, - timeout_s: float = 90.0, -) -> tuple[bool, str, int]: + host: str, port: int, *, cli: str | None = None, timeout_s: float = 90.0 +) -> tuple[bool, int]: """Poll /api/v1/health until it answers 200 or we hit the timeout. - Returns ``(reachable, host, port)``. When `cli` is given the bound port is - re-checked on every pass, because a service that is still starting has no - port to report yet and may come up on one other than our first guess. + Returns ``(reachable, port)``: when `cli` is given the port is re-checked + each pass, since a service that is still starting has none to report yet. """ deadline = time.monotonic() + timeout_s while time.monotonic() < deadline: if check_server_reachable(host, port): - return True, host, port + return True, port if cli is not None: - discovered_host, discovered_port = discover_endpoint(cli) - host = discovered_host or host - if discovered_port is not None and discovered_port != port: - _print(f"`{CLI_NAME} status` reports port {discovered_port}; checking there.") - port = discovered_port - continue + port = discover_port(cli) or port time.sleep(2.0) - return False, host, port + return False, port def check_server_reachable(host: str, port: int) -> bool: @@ -652,22 +554,18 @@ def upsert_agents_md( def _env_port() -> int | None: - """LEMONADE_PORT as an int, dropping an unusable value from the environment. - - The `lemonade` CLI reads the same variable and refuses to start on a value - it cannot parse ("Could not convert: --port = ..."). Leaving such a value - in the environment our subprocesses inherit would turn a typo into a - misdiagnosed stale CLI, so an unusable value is removed here, once, before - anything shells out. - """ + """LEMONADE_PORT as an int, ignoring an unusable value.""" raw = os.environ.get("LEMONADE_PORT") if not raw: return None - port = _valid_port(raw) - if port is None: + try: + return int(raw) + except ValueError: + # The CLI reads this variable too and refuses to start on a value it + # cannot parse, which would then look like a stale CLI. + os.environ.pop("LEMONADE_PORT") _print(f"ignoring LEMONADE_PORT={raw!r}: not a port number.") - os.environ.pop("LEMONADE_PORT", None) - return port + return None def main(argv: list[str] | None = None) -> int: @@ -680,17 +578,14 @@ def main(argv: list[str] | None = None) -> int: ) parser.add_argument( "--host", - default=None, + default=os.environ.get("LEMONADE_HOST"), help="Lemonade Server host (default: $LEMONADE_HOST, else 127.0.0.1).", ) parser.add_argument( "--port", type=int, - default=None, - help=( - "Lemonade Server port (default: $LEMONADE_PORT, else the port " - f"`{CLI_NAME} status` reports, else {DEFAULT_PORT})." - ), + default=_env_port(), + help=f"Lemonade Server port (default: $LEMONADE_PORT, else the port `{CLI_NAME} status` reports, else {DEFAULT_PORT}).", ) parser.add_argument( "--image-model", @@ -714,11 +609,6 @@ def main(argv: list[str] | None = None) -> int: ) args = parser.parse_args(argv) - # Read the endpoint request before shelling out: the CLI inherits these - # env vars, so an unusable one has to be dealt with first. - host_arg = args.host or os.environ.get("LEMONADE_HOST") or None - port_arg = args.port if args.port is not None else _env_port() - cli, stale = find_cli() # An old `lemonade` on PATH shadows the modern CLI. We never drive or @@ -758,14 +648,11 @@ def main(argv: list[str] | None = None) -> int: return 2 _print(f"using Lemonade CLI: {cli}") - # Ask the service where it bound before assuming the default port; an - # existing config or a port conflict moves it, and a "not running" verdict - # from the wrong port would be baked into the rule as well. - host, port, source = resolve_endpoint(cli, host_arg, port_arg) - _print(f"endpoint http://{host}:{port} (from {source})") - - # Only a local CLI can correct a port we did not pin ourselves. - rediscover_with = cli if port_arg is None and is_local_host(host) else None + # Ask the service where it bound rather than assuming the default port: a + # "not running" verdict from the wrong port would be baked into the rule. + host = args.host or DEFAULT_HOST + port = resolve_port(cli, args.host, args.port) + rediscover_with = cli if args.host is None and args.port is None else None # Modern Lemonade auto-starts the `lemond` service on install; there is no # `lemonade serve`. If it is not up yet (e.g. still starting right after a @@ -780,7 +667,7 @@ def main(argv: list[str] | None = None) -> int: _print(f"Start the service: {service_start_hint()}") return 3 _print("Lemonade service not reachable yet; waiting for it to come up.") - reachable, host, port = wait_for_server(host, port, cli=rediscover_with) + reachable, port = wait_for_server(host, port, cli=rediscover_with) if not reachable: _print( f"FAIL: the Lemonade service did not become reachable at " From d4859f04cf5fabca4dae0e219c195e9f797a7362 Mon Sep 17 00:00:00 2001 From: Daniel Holanda Date: Fri, 28 Aug 2026 15:25:21 -0700 Subject: [PATCH 3/3] Suggested changes --- skills/local-ai-use/reference.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/skills/local-ai-use/reference.md b/skills/local-ai-use/reference.md index c3b7ee4..85c19a0 100644 --- a/skills/local-ai-use/reference.md +++ b/skills/local-ai-use/reference.md @@ -236,8 +236,9 @@ say) while the agent runs on the laptop. To point this skill at it: with the new endpoint baked in. 4. Make sure the remote server is bound to a non-loopback interface (`lemonade config set host 0.0.0.0`) and that firewall rules allow - inbound 13305. Setting `host` to `0.0.0.0` exposes the server; pair it - with `LEMONADE_API_KEY` so it isn't open to the LAN. + inbound connections on the port it listens on. Setting `host` to + `0.0.0.0` exposes the server; pair it with `LEMONADE_API_KEY` so it + isn't open to the LAN. A server `scan` cannot see is not necessarily down: beacons are UDP, only go out on RFC1918 networks, and can be disabled (`--no-broadcast`). Ask the user