Feature-gated Wasm host tools for the CLI

.github/workflows/host-checks.yml

@@ -60,6 +60,9 @@ jobs:
       - name: Cargo clippy (warnings are errors)
         run: cargo clippy --all-targets --locked -- -D warnings
 
+      - name: Cargo clippy with Wasm host tools
+        run: cargo clippy --bin hyperlight-unikraft --tests --locked --features wasm-host-fns -- -D warnings
+
       - name: Enable KVM permissions
         working-directory: .
         run: |
@@ -75,6 +78,11 @@ jobs:
           RUST_BACKTRACE: '1'
         run: cargo test --all-targets --locked
 
+      - name: Cargo test with Wasm host tools
+        env:
+          RUST_BACKTRACE: '1'
+        run: cargo test --bin hyperlight-unikraft --locked --features wasm-host-fns wasm_host_fns::tests
+
   host-checks-passed:
     if: always()
     needs: [checks]

README.md

@@ -49,7 +49,7 @@ This project enables running Linux applications (Python, Node.js, Go, Rust, C/C+
 
 ### Key Features
 
-- **Thin, opt-in host surface** — by default, the guest has no access to the host filesystem, network, or any host functions. When you enable features like `--mount`, `--net`, or `--enable-tools`, a single `__dispatch` JSON-RPC bridge is registered as the only guest→host channel. See [HOST_FUNCTIONS.md](HOST_FUNCTIONS.md) for the full list of dispatchable operations
+- **Thin, opt-in host surface** — by default, the guest has no access to the host filesystem, network, or any host functions. When you enable features like `--mount`, `--net`, `--enable-tools`, or `--tool`, a single `__dispatch` JSON-RPC bridge is registered as the only guest→host channel. See [host_functions.md](docs/host_functions.md) for the full list of dispatchable operations
 - **Identity-mapped memory** - Simplified memory layout (vaddr == paddr)
 - **Generic cmdline mechanism** - Pass arguments to any application via `-- arg1 arg2 ...`
 - **Fast cold start** - Hyperlight's lightweight design enables millisecond startup times
@@ -281,7 +281,19 @@ Options:
   -m, --memory <MEMORY>        Memory allocation (e.g., 256Mi, 512Mi, 1Gi) [default: 512Mi]
       --stack <STACK>          Stack size (e.g., 8Mi) [default: 8Mi]
   -q, --quiet                  Quiet mode — suppress host-side status messages
-      --enable-tools           Enable tool dispatch via __dispatch host function
+      --enable-tools           Register the built-in echo tool
+      --tool <NAME=WASM>        Register a WASIp1 module as a host tool (requires wasm-host-fns)
+      --tool-wasi-dir <HOST[:GUEST]>
+                               Preopen a read-write host directory for Wasm tools
+      --tool-wasi-dir-ro <HOST[:GUEST]>
+                               Preopen a read-only host directory for Wasm tools
+      --tool-wasi-env <KEY=VALUE>
+                               Set an environment variable for Wasm tools
+      --tool-wasi-env-inherit <KEY>
+                               Inherit one host environment variable into Wasm tools
+      --tool-wasi-fuel <FUEL>  Fuel units available to each Wasm tool call [default: 100000000]
+      --tool-wasi-output-limit <SIZE>
+                               Maximum stdout or stderr captured from one Wasm tool call [default: 1Mi]
       --mount <HOST[:GUEST]>   Preopen a host directory for the guest's sandboxed filesystem
                                (repeatable; default guest path: /host)
       --net                    Enable guest networking (off by default)
@@ -297,6 +309,17 @@ Options:
   -V, --version                Print version
 ```
 
+### Wasm host tools
+
+Build the CLI with the optional `wasm-host-fns` feature to register host-side custom tools from WASIp1 modules:
+
+```bash
+cargo build --manifest-path host/Cargo.toml --release --features wasm-host-fns --bin hyperlight-unikraft
+hyperlight-unikraft kernel --initrd app.cpio --tool greet=./greet.wasm
+```
+
+The guest still calls the existing `__dispatch` envelope, for example `{"name":"greet","args":{"who":"Ada"}}`. The Wasm handler runs on the host in Wasmtime, receives that request JSON on stdin, and writes either a raw JSON result or `{"result": ...}` / `{"error": "..."}` to stdout. WASI filesystem and environment access are off unless granted with `--tool-wasi-dir`, `--tool-wasi-dir-ro`, `--tool-wasi-env`, or `--tool-wasi-env-inherit`.
+
 ## Project Structure
 
 ```

docs/host_functions.md

@@ -15,7 +15,7 @@ Implementation lives in [`host/src/lib.rs`](../host/src/lib.rs). Guest-side call
 | `fs_*` tools | **Off** | `--mount HOST[:GUEST]` (repeatable) |
 | `net_*` tools | **Off** | `--net`, `--net-allow`, or `--net-block` |
 | Inbound listen | **Off** | `--port PORT` (requires network enabled) |
-| Custom tools | **Off** | `--enable-tools` + `SandboxBuilder::tool()` |
+| Custom tools | **Off** | `--tool NAME=WASM` with `wasm-host-fns`, `SandboxBuilder::tool()`, or legacy/demo `--enable-tools` echo |
 
 With **no flags**, the guest cannot reach the host filesystem or network through dispatch. Only internal plumbing (`__hl_exit`, `__hl_sleep`) is wired.
 
@@ -85,7 +85,18 @@ hyperlight-unikraft KERNEL [--initrd CPIO] [options] [-- APP_ARGS...]
 | `--net-allow HOST_OR_IP` | Allow-list outbound destinations (implies `--net`). Repeatable. |
 | `--net-block HOST_OR_IP` | Block-list; all other destinations allowed (implies `--net`). Mutually exclusive with `--net-allow`. |
 | `--port PORT` | Allow `net_bind` / listen on `PORT` (implies `--net`). Without `--port`, outbound-only: bind is rejected. |
-| `--enable-tools` | Enables custom tool registration. Registers a built-in `echo` tool (used by the `python-tools` example). Library users add their own tools via `SandboxBuilder::tool()`. |
+| `--enable-tools` | Registers only the built-in `echo` demo tool. It does not load user code; prefer `--tool NAME=WASM` for CLI custom tools or `SandboxBuilder::tool()` for library users. |
+| `--tool NAME=WASM` | With the Cargo feature `wasm-host-fns`, registers `WASM` as a host-side WASIp1 custom tool named `NAME`. Repeatable. |
+| `--tool-wasi-dir HOST[:GUEST]` | Preopens a read-write host directory for every CLI Wasm tool. Default guest path is `/host`. Repeatable. |
+| `--tool-wasi-dir-ro HOST[:GUEST]` | Preopens a read-only host directory for every CLI Wasm tool. Default guest path is `/host`. Repeatable. |
+| `--tool-wasi-env KEY=VALUE` | Sets an environment variable for every CLI Wasm tool. Repeatable. |
+| `--tool-wasi-env-inherit KEY` | Copies one host environment variable into every CLI Wasm tool. Repeatable. |
+| `--tool-wasi-fuel FUEL` | Sets the instruction-fuel budget for each call to every CLI Wasm tool. Default `100000000`. |
+| `--tool-wasi-output-limit SIZE` | Caps captured stdout and stderr for each call to every CLI Wasm tool. Default `1Mi`. |
+
+`--tool-wasi-*` flags configure the Wasmtime/WASI sandbox for Wasm custom tools only. They do not expose the guest `--mount` filesystem, and they do not change the `fs_*` handlers used by `lib/hostfs`.
+
+The CLI currently applies the same Wasm filesystem, environment, fuel, and output settings to every `--tool` registered in one invocation. If tools need different permissions or limits, do not grant the union to all handlers; that requires a narrower per-tool configuration surface or a separate host integration.
 
 **Mount rules (host-enforced before boot):**
 
@@ -189,7 +200,38 @@ Sockets are host-side (`socket2`); the guest sees opaque numeric **`fd`** handle
 
 ## Custom tools
 
-**CLI:** `--enable-tools` registers a built-in `echo` tool (returns `args` unchanged) used by the [`python-tools` example](../examples/python-tools). The primary purpose of `--enable-tools` is to demonstrate custom host function registration via the API.
+**CLI demo tool:** `--enable-tools` registers only a built-in `echo` tool that returns `args` unchanged. It is useful as a smoke test and compatibility path, but it is not the CLI extension mechanism for user-provided host functions. CLI examples should prefer a Wasm `echo.wasm` registered with `--tool echo=...`; library examples should register an echo handler with `SandboxBuilder::tool()`.
+
+**CLI Wasm tools:** build with the optional feature and pass one or more `--tool` flags:
+
+```bash
+cargo build --manifest-path host/Cargo.toml --features wasm-host-fns --bin hyperlight-unikraft
+hyperlight-unikraft kernel --initrd app.cpio --tool greet=./greet.wasm
+```
+
+Each `--tool NAME=WASM` module is compiled and linked before VM boot, then invoked as a fresh WASIp1 command for every matching guest `__dispatch` call. The handler receives the existing dispatch request on stdin:
+
+```json
+{"name":"NAME","args":<json_value>}
+```
+
+The handler writes JSON to stdout. It may write either a raw JSON result value or the normal dispatch envelope:
+
+```json
+{"result":<json_value>}
+```
+
+```json
+{"error":"message"}
+```
+
+A raw value is treated as the tool result. A single-key `result` envelope is unwrapped. A single-key `error` envelope becomes the outer `__dispatch` error response. Empty stdout returns JSON null.
+
+Wasm tools are separate from the built-in `fs_*` and `net_*` dispatch handlers. `--mount` controls what the guest can access through `lib/hostfs`; `--tool-wasi-dir*` controls what the host-side Wasm handler can access through its own WASI filesystem view.
+
+WASI capabilities are denied by default except stdio used for the protocol, clocks, and random. Use `--tool-wasi-dir`, `--tool-wasi-dir-ro`, `--tool-wasi-env`, and `--tool-wasi-env-inherit` to grant explicit filesystem and environment access to handlers. These grants and the `--tool-wasi-fuel` / `--tool-wasi-output-limit` settings apply to every CLI Wasm tool registered by the process. Tool names beginning with `__`, `fs_`, or `net_` are reserved.
+
+**Why WASIp1 command modules today?** The current CLI maps one `--tool NAME=WASM` flag to one tool name and one fresh handler invocation. WASIp1 keeps that ABI small: JSON request on stdin, JSON response on stdout, no long-lived reactor state, and broad language/toolchain support. Component-model or reactor-style handlers could support a future `--tools component.wasm` shape with multiple exported tools and auto-registration, but that would need a separate registration and lifecycle model; it is not the current ABI.
 
 **Library:**
 
@@ -199,7 +241,7 @@ Sandbox::builder("kernel")
     .build()?;
 ```
 
-Custom handlers run with the same JSON request/response envelope as built-in tools.
+`SandboxBuilder::tool()` handlers receive the inner `args` JSON value from the dispatch request; the registry has already matched the outer `name`. Handler return values become the `result` field in the outer `__dispatch` response, and handler errors become `{"error": "..."}`.
 
 ---
 
@@ -214,6 +256,8 @@ Custom handlers run with the same JSON request/response envelope as built-in too
 | `fs_list` entries | 100 000 |
 | `net_send` / `net_sendto` | 1 MiB decoded bytes |
 | `__hl_sleep` | 60 s |
+| Wasm tool fuel | 100 000 000 instructions per call by default; configurable with `--tool-wasi-fuel`; same value applies to every CLI Wasm tool |
+| Wasm tool stdout / stderr | 1 MiB each per call by default; configurable with `--tool-wasi-output-limit`; same value applies to every CLI Wasm tool |
 | Open host sockets | 1024 per sandbox |
 | AllowList learned DNS IPs | 256 |
 
@@ -242,6 +286,14 @@ Custom handlers run with the same JSON request/response envelope as built-in too
 - A compromised guest can invoke any **registered** tool name; do not register powerful custom tools unless needed.
 - Payload size is capped; malformed JSON fails closed with an error response.
 
+**When `--tool` is used with `wasm-host-fns`:**
+
+- Handler code runs on the host inside Wasmtime, not inside the Unikraft VM.
+- WASI filesystem and environment access are capability-based and off unless explicitly granted with `--tool-wasi-*` flags.
+- CLI Wasm capability and limit flags apply to every registered Wasm tool; avoid combining handlers with different privilege needs in one invocation.
+- Fuel limits bound Wasm instruction execution, but do not turn filesystem operations into a full wall-clock timeout.
+- Handlers are untrusted code from the host operator's filesystem; only load modules you intend to grant these capabilities to.
+
 **Not exposed via dispatch:** Host shell, arbitrary process spawn, unrestricted host `exec`, or kernel modules — only the tools listed above.
 
 **Operators should:** Use minimal flags, allow-lists over `--net` where possible, mount least-privilege directories, and run guests with the smallest initrd/runtime required.