diff --git a/skills/local-ai-use/SKILL.md b/skills/local-ai-use/SKILL.md index 38e0a27..1f6b061 100644 --- a/skills/local-ai-use/SKILL.md +++ b/skills/local-ai-use/SKILL.md @@ -33,7 +33,9 @@ The skill does three things: by the OS — so the setup script waits for the service and, if it stays down, prints the exact OS-specific command to start it (e.g. `sudo systemctl start lemond` on Linux). -2. **Verifies that local Lemonade is reachable.** +2. **Verifies that local Lemonade is reachable** on the port `lemonade status` + reports, so a server already running on a non-default port is used as-is + rather than reported missing (`13305` is only the fallback). 3. **Drops a `Local AI Use` block into the workspace `AGENTS.md`** so the agent reads the routing rule on every later turn, in Cursor, Claude Code, Codex, Gemini CLI, and any other agent that respects `AGENTS.md`. @@ -58,7 +60,8 @@ Use this skill when **all** of the following are true: - The user wants local Lemonade. If it is not yet installed, the setup script installs the latest version for them automatically. -- The user accepts the default Lemonade endpoint `http://localhost:13305`. +- The user accepts the Lemonade endpoint on this machine, whichever port the + service reports (`http://localhost:13305` if it has never been changed). - The user wants the change to be **persistent** across future turns and agent restarts (the rule is written to disk). @@ -162,21 +165,34 @@ After a Windows install the CLI lands in `%LOCALAPPDATA%\lemonade_server` and is added to the *user* PATH (new shells only); the setup script probes that directory so it works in the same run. -**1b. Is the service running?** Check `lemonade status --json`. The `lemond` -service auto-starts on install — there is **no** `lemonade serve` in modern -Lemonade. +**1b. Is the service running, and where?** Check `lemonade status --json`, +which answers both at once by printing the bound port (`{"port": 13305}`). The +`lemond` service auto-starts on install — there is **no** `lemonade serve` in +modern Lemonade. | `lemonade status` says | Action | |---|---| -| `Server is running on port 13305` | Continue to Step 2. | -| `Server is not running` | Wait a few seconds for the auto-started service (the script polls `/api/v1/health`). If it stays down, start it via the OS service manager: `sudo systemctl start lemond` (Linux system install) or `systemctl --user start lemond` (per-user install); `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS); the Lemonade tray app or `Start-Service lemond` (Windows). | +| `Server is running on port ` | Use port `` for every later request and for the rule, even when it is not 13305. Continue to Step 2. | +| `Server is not running` | Wait a few seconds for the auto-started service (the script polls `/api/v1/health`, re-asking `status` in case the service comes up on a different port). If it stays down, start it via the OS service manager: `sudo systemctl start lemond` (Linux system install) or `systemctl --user start lemond` (per-user install); `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS); the Lemonade tray app or `Start-Service lemond` (Windows). | + +Never treat 13305 as the definition of "running": the port is a config value +(`lemonade config set port`) that an existing config, another install channel, +or a port conflict all move, so probing only the default reports a healthy +server as missing. Ask `status`, which resolves the port from the service's UDP +beacon for you — `lemonade scan` is for finding servers on *other* machines +(see [reference.md](reference.md#re-pointing-the-rule-at-a-remote-host)), never +for picking the local endpoint. The setup script prefers `--host` / `--port` or +`LEMONADE_HOST` / `LEMONADE_PORT` over discovery, and bakes whatever it settles +on into the rule; pass both when pointing at another machine, since the CLI +here cannot report a remote service's port. Only if the automatic install genuinely fails (no `apt-get`, no `sudo`, download blocked) should you stop and point the user at . -The rest of this skill assumes the endpoint is `http://localhost:13305/api/v1` -and no API key is required (the system-wide server defaults to no auth on +The rest of this skill writes the endpoint as `http://localhost:13305/api/v1`, +the default; substitute the port `status` reported if it differs. It also +assumes no API key is required (the system-wide server defaults to no auth on loopback). If the user has set `LEMONADE_API_KEY`, the routing rule template in `templates/local-ai-rule.md` shows where to add the `Authorization` header. @@ -283,6 +299,7 @@ machine. |---|---|---| | `lemonade: command not found` | CLI not installed | Re-run `python scripts/setup_local_ai.py` (auto-installs the latest version). If it just installed on Windows, open a new shell so the user PATH refreshes, or the script will find it under `%LOCALAPPDATA%\lemonade_server`. | | `status` gives an "invalid choice" / usage error | An old `lemonade` (pre-v10.1.0) is shadowing the modern CLI | Uninstall it (see the Step 1a table: `winget uninstall -e --id AMD.LemonadeServer` / `sudo apt remove lemonade-server` / `brew uninstall --cask lemonade-server`), then re-run the setup script. | +| Requests to `http://localhost:13305` are refused, but `lemonade status` says the server is running | The service is bound to a non-default port (existing config, another install channel, or a port conflict) | Use the port from `lemonade status --json` and re-run `python scripts/setup_local_ai.py` so the rule is rewritten with it. Do not start a second server; the running one is fine. | | `Server is not running` | `lemond` service stopped | Start it via the OS service manager — `sudo systemctl start lemond` / `systemctl --user start lemond` (Linux), `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS), or the tray app / `Start-Service lemond` (Windows). There is no `lemonade serve`. | | `POST /v1/images/generations` returns 404 model not found | Image model not downloaded | `lemonade pull SD-Turbo` and retry. | | `lemonade pull` keeps printing `Progress: NN%` but never finishes | Download target is a bad path (out of space, no write permission, quota, read-only mount). The write error may surface only in the server log while the console keeps showing progress | Check the target and free space first: `GET /api/v1/system-info` reports `models_dir` and `model_storage.free_bytes`. If a pull stalls, read the recent lines of the server log (typically `lemonade-server.log` in the OS temp dir) for the real error (e.g. a download/write failure like `CURL code 23`, or an out-of-space message), then point the download at a writable disk with room. | @@ -296,14 +313,16 @@ machine. Mark this skill complete only when **all** of the following are true: -- [ ] `lemonade status --json` reports the server running on port 13305. +- [ ] `lemonade status --json` reports the server running, and the port it + reports is the port in the installed rule (13305 unless the service was + configured otherwise). - [ ] The workspace `AGENTS.md` contains the `amd-skills:local-ai-use` block. This is required even when Lemonade was already installed and running — generating an image alone does not complete the skill. - [ ] On a follow-up turn, asking the agent to "generate an image of X" - causes it to POST to `http://localhost:13305/api/v1/images/generations` - (pulling the model on first use) rather than calling a cloud tool. + causes it to POST to `/api/v1/images/generations` on the endpoint in the + rule (pulling the model on first use) rather than calling a cloud tool. - [ ] `lemonade backends --all` shows `installed` for every backend variant this workspace's routing depends on (see Step 1c). Do not treat a working image or chat path as proof that transcription will work. diff --git a/skills/local-ai-use/reference.md b/skills/local-ai-use/reference.md index 824813d..85c19a0 100644 --- a/skills/local-ai-use/reference.md +++ b/skills/local-ai-use/reference.md @@ -21,11 +21,6 @@ The default trio (`SD-Turbo`, `kokoro-v1`, `Whisper-Tiny`) is sized for "keeps cost savings real on a typical laptop". Override only if the user asks for higher quality or has explicit hardware to spare. -Model catalogs move between releases, so treat the IDs below as a starting -point, not a fixed list. Confirm any ID with `GET /api/v1/models` (add -`?show_all=true` for the full catalog) before writing it into the rule. Do -not rely on a stale `server_models.json` snapshot. - ### Image generation (`recipe: sd-cpp`) | Model | Approx size | When to use | Trade-off | @@ -69,10 +64,6 @@ Whisper requires 16 kHz mono PCM WAV input. Convert anything else first: ffmpeg -i input.mp3 -ar 16000 -ac 1 input.wav ``` -The HTTP endpoint adds no meaningful overhead over calling `whisper-cli` -directly on the same engine/backend/model. Throughput is equivalent; use the -endpoint unless you have a specific reason not to. - For full live coverage, run `lemonade list` after starting the server, or browse . @@ -107,8 +98,7 @@ Notable per-endpoint quirks: - **`/v1/audio/transcriptions`**: only `wav` input is supported; re-encode anything else with `ffmpeg`. `response_format` accepts `json`, `verbose_json`, `text`, `srt`, and `vtt` — the first two return JSON, the - rest return a raw text body (as of Lemonade 11.7.0). Any other value is a - 400. + rest return a raw text body (as of Lemonade 11.7.0). - **`/v1/audio/speech`**: `mp3`, `wav`, `opus`, and `pcm` outputs supported. Streaming requires `stream_format: "audio"`, which only emits `pcm`. @@ -235,14 +225,24 @@ read `examples/lemonade_tools.py` in the upstream lemonade-sdk repo. Lemonade can run on another machine (a workstation with a Ryzen AI NPU, say) while the agent runs on the laptop. To point this skill at it: -1. Set `LEMONADE_HOST` and `LEMONADE_PORT` (or pass `--host` / `--port` to - `setup_local_ai.py`). -2. Re-run `python scripts/setup_local_ai.py` so the rule block is rewritten +1. Find it with `lemonade scan --duration 5`, which lists the UDP beacons + Lemonade servers broadcast on the local network as + ` at http://:/api/v1/`. +2. Set `LEMONADE_HOST` and `LEMONADE_PORT` from that beacon (or pass + `--host` / `--port` to `setup_local_ai.py`). Set **both**: the script + otherwise discovers the port via `lemonade status`, which speaks only for + the service on this machine, so a remote host alone keeps port 13305. +3. Re-run `python scripts/setup_local_ai.py` so the rule block is rewritten with the new endpoint baked in. -3. Make sure the remote server is bound to a non-loopback interface +4. Make sure the remote server is bound to a non-loopback interface (`lemonade config set host 0.0.0.0`) and that firewall rules allow - inbound 13305. Setting `host` to `0.0.0.0` exposes the server; pair it - with `LEMONADE_API_KEY` so it isn't open to the LAN. + inbound connections on the port it listens on. Setting `host` to + `0.0.0.0` exposes the server; pair it with `LEMONADE_API_KEY` so it + isn't open to the LAN. + +A server `scan` cannot see is not necessarily down: beacons are UDP, only go +out on RFC1918 networks, and can be disabled (`--no-broadcast`). Ask the user +for the host and port in that case. --- diff --git a/skills/local-ai-use/scripts/setup_local_ai.py b/skills/local-ai-use/scripts/setup_local_ai.py index 3184f2a..a5ebcbd 100644 --- a/skills/local-ai-use/scripts/setup_local_ai.py +++ b/skills/local-ai-use/scripts/setup_local_ai.py @@ -12,8 +12,9 @@ Performs the setup steps from SKILL.md: 1. Ensures modern Lemonade is installed and its background service (the - `lemond` daemon) is reachable on http://localhost:13305 (override with - --host / --port or LEMONADE_HOST / LEMONADE_PORT). If no modern + `lemond` daemon) is reachable. The port is the one `lemonade status` + reports, unless --host / --port or LEMONADE_HOST / LEMONADE_PORT say + otherwise; http://localhost:13305 is only the fallback. If no modern `lemonade` CLI is found, the latest version is installed on the user's behalf. The daemon auto-starts on install and is managed by the OS service manager, so this script never runs a `serve` command; it waits @@ -59,10 +60,16 @@ # Defaults match the system-wide Lemonade Server install. Both the CLI # (LEMONADE_HOST / LEMONADE_PORT) and the OpenAI-compatible HTTP endpoints -# bind to these by default. +# bind to these by default -- but only by default, so the port is a fallback +# for when `lemonade status` cannot tell us the real one (see resolve_port). DEFAULT_HOST = "127.0.0.1" DEFAULT_PORT = 13305 +# `lemonade status --json` reports `{"port": 13305}`; the human-readable form +# says "Server is running on port 13305". Anchoring on "running on" keeps this +# off the other ports the same output lists (e.g. "WebSocket Port 9001"). +STATUS_PORT_RE = re.compile(r"running on port (\d+)", re.IGNORECASE) + # Picked because each default fits in under ~5 GB and runs on commodity CPU # hardware, so the savings vs. cloud calls are real on a typical developer # laptop. See SKILL.md for upgrade paths. @@ -172,6 +179,20 @@ def _candidate_clis() -> list[str]: return candidates +def _run_status(cli: str, *, as_json: bool = False) -> str | None: + """Combined output of ` status`, or None if it did not run. + + `status` is read-only and cheap, which is why it doubles as the modern-CLI + probe and the port lookup. + """ + cmd = [cli, "status"] + (["--json"] if as_json else []) + try: + result = subprocess.run(cmd, capture_output=True, text=True, timeout=20) + except (OSError, subprocess.SubprocessError): + return None + return f"{result.stdout}\n{result.stderr}" + + def is_modern_cli(cli: str) -> bool: """True if `cli` is the modern Lemonade CLI (drives the `lemond` service). @@ -186,16 +207,7 @@ def is_modern_cli(cli: str) -> bool: (exiting 0 or 1 accordingly). We key off that phrasing rather than the exit code alone. """ - try: - result = subprocess.run( - [cli, "status"], - capture_output=True, - text=True, - timeout=20, - ) - except (OSError, subprocess.SubprocessError): - return False - blob = f"{result.stdout}\n{result.stderr}".lower() + blob = (_run_status(cli) or "").lower() # Modern `lemonade status` always reports on the server, whether or not it # is running. An old/incompatible CLI never prints this phrasing (it errors # with "invalid choice: 'status'"), so the presence of the phrase is a @@ -203,6 +215,41 @@ def is_modern_cli(cli: str) -> bool: return "server is running" in blob or "server is not running" in blob +def discover_port(cli: str) -> int | None: + """The port the service actually bound, or None if it is not running. + + The port is a config value (`lemonade config set port`) that an existing + config, another install channel, or a port conflict all move, so treating + DEFAULT_PORT as the definition of "reachable" hides a healthy server. The + CLI resolves this from the service's UDP beacon for us; do not scan for + beacons here, because those also arrive from servers on other machines. + """ + blob = _run_status(cli, as_json=True) + if blob: + try: + port = json.loads(blob.strip()).get("port") + except (AttributeError, ValueError): + port = None + if isinstance(port, int): + return port + blob = _run_status(cli) # older builds may not support --json + match = STATUS_PORT_RE.search(blob) if blob else None + return int(match.group(1)) if match else None + + +def resolve_port(cli: str | None, host_arg: str | None, port_arg: int | None) -> int: + """Port to health-check: what was asked for, else what `status` reports. + + An explicit host is taken at face value: the CLI here knows nothing about + another machine's config, so a remote host keeps the default port. + """ + if port_arg is not None: + return port_arg + if cli is not None and host_arg is None: + return discover_port(cli) or DEFAULT_PORT + return DEFAULT_PORT + + def find_cli() -> tuple[str | None, str | None]: """Locate a Lemonade CLI and classify it. @@ -380,14 +427,22 @@ def uninstall_hint() -> str: return f"remove it with your platform's package manager; see {INSTALL_DOCS_URL}" -def wait_for_server(host: str, port: int, timeout_s: float = 90.0) -> bool: - """Poll /api/v1/health until it answers 200 or we hit the timeout.""" +def wait_for_server( + host: str, port: int, *, cli: str | None = None, timeout_s: float = 90.0 +) -> tuple[bool, int]: + """Poll /api/v1/health until it answers 200 or we hit the timeout. + + Returns ``(reachable, port)``: when `cli` is given the port is re-checked + each pass, since a service that is still starting has none to report yet. + """ deadline = time.monotonic() + timeout_s while time.monotonic() < deadline: if check_server_reachable(host, port): - return True + return True, port + if cli is not None: + port = discover_port(cli) or port time.sleep(2.0) - return False + return False, port def check_server_reachable(host: str, port: int) -> bool: @@ -498,6 +553,21 @@ def upsert_agents_md( return target +def _env_port() -> int | None: + """LEMONADE_PORT as an int, ignoring an unusable value.""" + raw = os.environ.get("LEMONADE_PORT") + if not raw: + return None + try: + return int(raw) + except ValueError: + # The CLI reads this variable too and refuses to start on a value it + # cannot parse, which would then look like a stale CLI. + os.environ.pop("LEMONADE_PORT") + _print(f"ignoring LEMONADE_PORT={raw!r}: not a port number.") + return None + + def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument( @@ -508,14 +578,14 @@ def main(argv: list[str] | None = None) -> int: ) parser.add_argument( "--host", - default=os.environ.get("LEMONADE_HOST", DEFAULT_HOST), - help="Lemonade Server host (default: 127.0.0.1 / $LEMONADE_HOST).", + default=os.environ.get("LEMONADE_HOST"), + help="Lemonade Server host (default: $LEMONADE_HOST, else 127.0.0.1).", ) parser.add_argument( "--port", type=int, - default=int(os.environ.get("LEMONADE_PORT", str(DEFAULT_PORT))), - help="Lemonade Server port (default: 13305 / $LEMONADE_PORT).", + default=_env_port(), + help=f"Lemonade Server port (default: $LEMONADE_PORT, else the port `{CLI_NAME} status` reports, else {DEFAULT_PORT}).", ) parser.add_argument( "--image-model", @@ -578,34 +648,41 @@ def main(argv: list[str] | None = None) -> int: return 2 _print(f"using Lemonade CLI: {cli}") + # Ask the service where it bound rather than assuming the default port: a + # "not running" verdict from the wrong port would be baked into the rule. + host = args.host or DEFAULT_HOST + port = resolve_port(cli, args.host, args.port) + rediscover_with = cli if args.host is None and args.port is None else None + # Modern Lemonade auto-starts the `lemond` service on install; there is no # `lemonade serve`. If it is not up yet (e.g. still starting right after a # fresh install), poll briefly, then guide the user to start the OS # service rather than trying to spawn it ourselves. - if not check_server_reachable(args.host, args.port): + if not check_server_reachable(host, port): if args.no_install: _print( f"FAIL: Lemonade Server is not responding at " - f"http://{args.host}:{args.port}/api/v1/health (--no-install set)." + f"http://{host}:{port}/api/v1/health (--no-install set)." ) _print(f"Start the service: {service_start_hint()}") return 3 _print("Lemonade service not reachable yet; waiting for it to come up.") - if not wait_for_server(args.host, args.port): + reachable, port = wait_for_server(host, port, cli=rediscover_with) + if not reachable: _print( f"FAIL: the Lemonade service did not become reachable at " - f"http://{args.host}:{args.port}/api/v1/health." + f"http://{host}:{port}/api/v1/health." ) _print(f"Start it manually, then re-run: {service_start_hint()}") _print(f"If it is not installed, see {INSTALL_DOCS_URL}") return 3 - _print(f"server reachable at http://{args.host}:{args.port}") + _print(f"server reachable at http://{host}:{port}") upsert_agents_md( args.workspace.resolve(), - host=args.host, - port=args.port, + host=host, + port=port, image_model=args.image_model, tts_model=args.tts_model, stt_model=args.stt_model,