Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,6 @@ jobs:

- name: Packed installation smoke tests
run: pnpm test:pack

- name: Container image browser-gateway contract
run: pnpm test:image
5 changes: 4 additions & 1 deletion .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ jobs:
- name: Packed installation smoke tests
run: pnpm test:pack

- name: Container image browser-gateway contract
run: pnpm test:image

- name: Upload built package artifacts
uses: actions/upload-artifact@v4
with:
Expand Down Expand Up @@ -90,7 +93,7 @@ jobs:
echo '| Packages | `@guionai/web, @guionai/pi-web, @guionai/dsh-web` |'
printf '| Synchronized version | `%s` |\n' "$version"
printf '| npm dist-tag | `%s` |\n' "$dist_tag"
echo "| Checks | format, typecheck, build, tests, release version invariants, packed smoke |"
echo "| Checks | format, typecheck, build, tests, release version invariants, packed smoke, image contract |"
} >> "$GITHUB_STEP_SUMMARY"

publish:
Expand Down
90 changes: 90 additions & 0 deletions .scratch/browser-gateway-renderer/implementation-report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Browser gateway renderer implementation report

## Result

The whole `browser-gateway-renderer` spec and its only ticket are implemented
on the `browser-gateway-renderer` branch. The containerized HTTP Service now
delegates explicit browser rendering to the server-local Browser Rendering
Gateway (`POST /api/render`), while Web Core continues to own HTML extraction,
navigation, links, and content limits. CLI, MCP, Pi, and DSH keep the existing
direct `agent-browser` renderer. Review fixes make the boundary image-only:
`GUIONAI_HTTP_IMAGE=1` is set by the Docker runtime, while a normal npm/local
`web serve` preserves its supplied direct operations.

## Acceptance criteria

- [x] Configured gateway Fetch and Links requests send `{ url, waitMs }`, use
returned raw DOM and final URL, preserve navigation/link response contracts,
and accept `waitMs` from 0 through 30,000.
- [x] Gateway delegation is limited to the explicitly marked image path;
normal npm/local servers with direct operations retain direct browser
rendering.
- [x] Missing, overloaded, unreachable, timed-out, malformed, oversized, or
failed gateway work is translated to explicit `render_*` capability errors;
cancellation propagates and HTTP rendering remains independent.
- [x] The production image uses the slim Node runtime without Chromium or
`agent-browser`; direct browser users outside the HTTP image retain their
existing capability.
- [x] README, HTTP-service/operator documentation, glossary, Dockerfile
comments, release documentation, and ADRs describe the gateway boundary and
`BROWSER_GATEWAY_URL` configuration.
- [x] Tests cover the fetch transport seam, HTTP routes, failure/cancellation
behavior, and the image contract without a live browser, cluster,
credentials, or production service.

## Verification

All checks completed successfully from the final implementation:

- `pnpm typecheck`
- `pnpm build`
- `pnpm test` — 20 files, 153 tests passed
- `pnpm test:release`
- `pnpm test:pack` — web, Pi, and DSH package smoke checks passed
- `pnpm test:image` — builds a disposable image, runs a fake `/api/render`,
verifies browser Fetch, and probes browser binaries are absent
- `pnpm format:check`
- `git diff --check`
- `docker build --tag guionai-web:browser-gateway-test .`
- Test-owned gateway plus image smoke request — HTTP Fetch returned rendered
content and the gateway log confirmed `/api/render` with `{ url, waitMs }`.
- Runtime binary probe — `no-browser-binaries` for `agent-browser`, Chromium,
and Google Chrome.

## Fixed-point LOC accounting

The fixed point is `ed74871`. Generated files and lockfiles are excluded.
Actual additions and deletions are:

| Category | Additions | Deletions |
| --- | ---: | ---: |
| Product code | 343 | 8 |
| Tests | 590 | 0 |
| Configuration and docs | 118 | 37 |
| **Total** | **1,051** | **45** |

The total (1,096 changed lines) exceeds the original 570–960 estimate because
the review required a repeatable 193-line Docker contract harness, explicit
image-mode selection, CI/release invocation, and corresponding documentation.
The added paths remain test-owned and bounded; no compatibility layer or
unfinished infrastructure was added.

## Commits and scope

- `9c9a088 feat(http): delegate container browser rendering to gateway`
- `aec662e fix(http): keep gateway rendering image-only`
- Changed implementation paths: `packages/web-core/src/`,
`packages/web-core/test/`, `packages/web/src/http.ts`,
`packages/web/src/program.ts`, `packages/web/test/http.test.ts`,
`scripts/test-image-contract.mjs`, `Dockerfile`, `README.md`, `CONTEXT.md`,
`docs/http-service.md`, `docs/adr/`, `package.json`, and CI/release workflows.

The required report is intentionally kept under `.scratch` and is excluded
from the implementation LOC table above.

## Remaining operational boundary

The separately deployed Browser Rendering Gateway must be reachable and
configured through `BROWSER_GATEWAY_URL`; its deployment, capacity, proxy,
authentication, and rollout remain outside this repository. Code review and
deployment are excluded from this implementation task.
13 changes: 9 additions & 4 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,10 @@ _Avoid_: Bridge URL parameter
**Page Rendering**:
Fetch and Links use `render: "http"` by default or explicit `render: "browser"`.
Browser rendering requires `waitMs` from 0 through 30,000 and is never selected
automatically. The operator-installed `agent-browser` runtime is an implementation
and setup detail, not an adapter-facing request value.
automatically. The containerized HTTP Service delegates browser requests to the
server-local Browser Rendering Gateway; CLI, MCP, Pi, and DSH retain the
operator-installed `agent-browser` capability. Both are implementation and setup
details, not adapter-facing request values.
_Avoid_: Backend-specific renderer labels, automatic fallback

**Page Navigation**:
Expand All @@ -57,8 +59,11 @@ The Hono-based `/api/v1/web` JSON API shipped by `web serve` and the GHCR image.
uses server-local credentials, Bridge Route, and optional DeepSeek provider
configuration; clients do not select providers or submit a generic Bridge
command. Its page-reading routes use
the same `render: "http" | "browser"`, `mode`, and `section_id` contract; the
browser executable name appears only in operator setup.
the same `render: "http" | "browser"`, `mode`, and `section_id` contract. The
GHCR image sets `GUIONAI_HTTP_IMAGE=1` and delegates browser rendering to the
server-local Browser Rendering Gateway configured by the operator; a local/npm
server with direct operations retains its direct browser capability. The
browser executable name appears only in gateway-side setup for the image.
_Avoid_: Remote MCP, public service

**Gateway Web API prefix**:
Expand Down
21 changes: 8 additions & 13 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Guion Web's supported HTTP-service image. The build stage bundles the web
# executable; the runtime stage adds the optional rendered-fetch capability.
# executable; browser rendering is delegated to the configured in-cluster
# Browser Rendering Gateway rather than installed in this image.
FROM node:24-bookworm-slim AS build

WORKDIR /workspace
Expand All @@ -13,22 +14,16 @@ RUN pnpm install --frozen-lockfile
COPY . .
RUN pnpm --filter @guionai/web run build

FROM node:24-bookworm
FROM node:24-bookworm-slim

ENV NODE_ENV=production
# Chrome for Testing has no Linux ARM64 distribution. Debian's Chromium works
# on both released container architectures, and agent-browser documents this
# executable override for containers.
ENV AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium
# The image entrypoint opts into the gateway-only browser path. Local npm
# installs leave this marker unset and keep their supplied direct operations.
ENV GUIONAI_HTTP_IMAGE=1
WORKDIR /app

# Rendering is deliberately explicit at the HTTP contract. The executable and
# browser runtime are image capabilities, while credentials remain env-only.
ARG AGENT_BROWSER_VERSION=0.36.0
RUN apt-get update \
&& apt-get install --yes --no-install-recommends chromium \
&& rm -rf /var/lib/apt/lists/* \
&& npm install --global agent-browser@${AGENT_BROWSER_VERSION}
# `BROWSER_GATEWAY_URL` is optional at startup; image browser requests fail
# explicitly until the operator points the service at the internal gateway.

COPY --from=build /workspace/packages/web/dist ./dist

Expand Down
37 changes: 25 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ export DEEPSEEK_API_KEY="..."
export CONTEXT7_API_KEY="..."
# optional complete Bridge route for `web serve`
export KEPOS_BRIDGE_ENDPOINT="http://127.0.0.1:8787/codex/web-search"
# optional Browser Rendering Gateway origin for `web serve` browser requests
export BROWSER_GATEWAY_URL="http://browser-gateway"
# HTTP/Pi: select DeepSeek server-side (HTTP clients still send {"query":"..."})
export WEB_SEARCH_PROVIDER="deepseek"
```
Expand All @@ -70,6 +72,7 @@ web serve --host 0.0.0.0 --port 8787
docker run --rm -p 8787:8787 \
-e EXA_API_KEY="$EXA_API_KEY" \
-e KEPOS_BRIDGE_ENDPOINT="http://host.docker.internal:17480/codex/web-search" \
-e BROWSER_GATEWAY_URL="http://host.docker.internal:8788" \
ghcr.io/guionai/web:v0.1.0
```

Expand Down Expand Up @@ -101,10 +104,12 @@ cancellation is reported as 499.
Fetch and Links use HTTP rendering when `render` is omitted (or set to
`"http"`). Browser rendering is explicit and requires both `render: "browser"`
and an integer `waitMs` from 0 through 30,000; HTTP rendering never silently
switches backends. The container installs the `agent-browser` executable with
Debian Chromium (including Linux ARM64, where Chrome for Testing has no build),
while credentials and Bridge configuration remain server-local environment
variables.
switches backends. A local/npm `web serve` keeps its supplied direct operations.
The GHCR image sets `GUIONAI_HTTP_IMAGE=1` and sends browser requests to the
server-local Browser Rendering Gateway configured by `BROWSER_GATEWAY_URL`.
The GHCR image contains no Chromium or `agent-browser`; an absent, unreachable,
overloaded, or failed gateway returns an explicit browser-render failure while
ordinary HTTP rendering remains available.

This is a Personal Web Service: a single-trust-boundary deployment for its
operator and agents. It is not hardened for public or multi-tenant exposure;
Expand Down Expand Up @@ -226,9 +231,11 @@ from the original page DOM.

`web fetch` has two renderers. `http` (the default) uses Node `fetch`, `linkedom`,
and Defuddle for HTML-to-Markdown extraction from static, SSR, and pre-rendered
pages. `browser` renders client-side pages through the separately installed
host browser capability. HTTP rendering is used by default; choose browser
explicitly when needed. The implementation never falls back automatically:
pages. `browser` renders client-side pages through the host capability: `web
serve` delegates to its configured Browser Rendering Gateway, while CLI, MCP,
Pi, and DSH use the separately installed `agent-browser` capability. HTTP
rendering is used by default; choose browser explicitly when needed. The
implementation never falls back automatically:

```bash
web fetch https://example.com/app --render=browser --wait=2000
Expand All @@ -249,7 +256,8 @@ or `links` requests must not provide `--wait`. The same `render: "browser"` and
`javascript_rendering_may_be_required` hint with the 2,000 ms suggestion; the
agent decides whether to retry with a longer wait or abandon the page.

Rendering is an optional host capability. If you choose to use it, install
Direct rendering is an optional host capability for CLI, MCP, Pi, and DSH. If
you choose to use it, install
[agent-browser](https://github.com/vercel-labs/agent-browser) separately on the
host:

Expand Down Expand Up @@ -305,26 +313,31 @@ pnpm build
pnpm test
pnpm test:release
pnpm test:pack
pnpm test:image
```

`test:release` uses disposable manifests to exercise tag-version
synchronization. `test:pack` runs each public package's packed installation or
host-loading contract in test-owned temporary directories.
host-loading contract in test-owned temporary directories. `test:image` builds a
test-owned disposable Docker image, runs it against a fake `/api/render` gateway,
and verifies the image has no browser executable.

## Releases

A `v<semver>` tag is the release source of truth for all three public packages:
`@guionai/web`, `@guionai/pi-web`, and `@guionai/dsh-web`. The release preflight
synchronizes its checkout manifests from that tag, then completes formatting,
typechecking, build, tests, release-version checks, and packed smoke tests
typechecking, build, tests, release-version checks, packed smoke tests, and the
Docker image contract
before any publication begins.

Three independent, non-fail-fast protected `npm` Environment matrix cells then
publish one package each through npm Trusted Publishing with provenance. The
synchronized version selects npm's `latest` tag for stable SemVer and `beta` for
a prerelease. A matching immutable-tagged image is published to
`ghcr.io/guionai/web:<tag>` with the `web serve` entrypoint and the
`agent-browser` runtime. After all three npm cells and the image job succeed,
`ghcr.io/guionai/web:<tag>` with the `web serve` entrypoint. The image delegates
explicit browser rendering to the configured internal Browser Rendering
Gateway and contains no browser executable. After all three npm cells and the image job succeed,
the workflow creates the GitHub release with generated notes, source archives,
and the build-generated `openapi.yaml` asset. The asset is generated from the
same Hono route schemas as the image and package; it is not checked in or
Expand Down
6 changes: 6 additions & 0 deletions docs/adr/0001-containerized-rest-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,10 @@ Guion Web will add a self-hosted, single-user HTTP service in a portable contain

## Consequences

The HTTP Service's explicit browser mode is implemented by the server-local
Browser Rendering Gateway; the image does not install or launch a browser.
The shared Core still gives CLI, MCP, Pi, and DSH adapters their direct host
browser capability. See [ADR 0003](0003-http-browser-rendering-gateway.md) for
the boundary and failure behavior.

The service's provider credentials, Bridge Route, and optional `WEB_SEARCH_PROVIDER=deepseek` selection are server-local configuration; clients cannot choose a provider or supply a Bridge Route per request. With no provider selection, search tries Kepos Bridge and uses Exa only when the Bridge is operationally unavailable; it does not fall back for cancellation, malformed client input, or an empty result set. A failed Bridge attempt is retried through Exa exactly once, and a successful empty Bridge response is returned unchanged. When DeepSeek is selected, the service calls only DeepSeek and never falls back. DeepSeek performs one auxiliary model call internally and returns the same normalized result contract; its Messages/tool protocol is not exposed to HTTP callers. Typed Bridge operations are intentionally not exposed: Exa has no equivalent official weather, sports, or time API, and its premium finance integration is not contract-compatible. Fetch and Links use `render: "http"` by default or explicit `render: "browser"` with a required `waitMs`; the operator-installed executable remains an implementation detail. Fetch navigation input uses `mode: "auto"` by default, with explicit `"full"` or `"tree"` modes; a non-empty `section_id` with omitted or `"auto"` mode retrieves a section, while full/tree reject it. Results report `"auto"`, `"full"`, `"tree"`, or `"section"` as appropriate and include a `truncated` flag for content cut by the Core limit. `openapi.yaml` is generated from the release build rather than manually maintained or independently versioned; the standalone [HTTP service reference](../http-service.md) is its human-readable companion. Public or multi-tenant deployment hardening is deliberately deferred in `.scratch/defered/public-http-service-security.md`.
44 changes: 44 additions & 0 deletions docs/adr/0003-http-browser-rendering-gateway.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Delegate HTTP browser rendering to the Browser Rendering Gateway

## Status

Accepted

## Context

The GHCR image previously installed Chromium and `agent-browser` so each
explicit `render: "browser"` request could start a private browser process.
The apps-dev deployment already operates a persistent Browser Rendering
Gateway with the proxy, anti-bot, and capacity controls needed for browser
work. Maintaining a second browser runtime in the HTTP-service image adds
startup, memory, and lifecycle cost.

## Decision

Only the containerized HTTP Service, explicitly marked with
`GUIONAI_HTTP_IMAGE=1`, delegates explicit browser rendering to the server-local
Browser Rendering Gateway. It sends `POST /api/render` with
`{ "url", "waitMs" }` and receives raw rendered `{ "html", "url" }`. The
existing Web Core then performs target validation, extraction, navigation, and
link handling, so Fetch and Links keep their public contracts. `waitMs` remains
caller-visible and is required from 0 through 30,000; browser rendering never
falls back to HTTP.

`BROWSER_GATEWAY_URL` is image-only server-local configuration. Missing
configuration, gateway overload, transport failure, timeout, or malformed output
becomes an explicit `render_*` capability failure. HTTP rendering remains
independent. Local/npm `web serve` instances leave the image marker unset and
retain their supplied direct operations.

CLI, MCP, Pi, and DSH continue to use the shared Web Core's direct
`agent-browser` capability. The GHCR image therefore contains neither
Chromium nor `agent-browser`; the gateway owns that runtime and its deployment
boundary.

## Consequences

The HTTP-service image is smaller and does not launch a local browser. Image
browser requests require the in-cluster gateway to be reachable, and operators
must configure `BROWSER_GATEWAY_URL` before using them. Gateway deployment,
capacity, authentication, and rollout remain outside this repository and are
owned by the separate browser-gateway service.
25 changes: 18 additions & 7 deletions docs/http-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,16 @@ or `"tree"`. A non-empty `section_id` may be supplied with omitted mode or
`"tree"`.

HTTP rendering fetches the page with Node HTTP, linkedom, and Defuddle.
Browser rendering invokes the operator-installed `agent-browser` executable
through the isolated renderer implementation; the executable name is not a
public request value. Browser rendering is never selected automatically, and
the service does not fall back between renderers.
Browser rendering delegates the raw DOM request to the internal Browser
Rendering Gateway; the gateway's browser executable is not a public request
value. Configure its origin with the server-local `BROWSER_GATEWAY_URL`
environment variable when running the GHCR image. A local/npm `web serve` with
supplied direct operations keeps its direct browser capability; the image sets
`GUIONAI_HTTP_IMAGE=1` to select the gateway-only path. Browser rendering is
never selected automatically, and the service does not fall back between
renderers. If gateway configuration is missing or the gateway rejects, times
out, or returns an invalid response, the image operation fails explicitly with
a `render_*` capability error.

The shared module owns the 5,000-character automatic-tree policy. An `"auto"`
request for an unsectioned document longer than that threshold with navigable
Expand Down Expand Up @@ -155,9 +161,14 @@ response bodies.

## Configuration and OpenAPI

Credentials, the optional `KEPOS_BRIDGE_ENDPOINT`, and the optional
server-local `WEB_SEARCH_PROVIDER=deepseek` selection are environment
variables. They are not accepted in request bodies. DeepSeek performs one
Credentials, the optional `KEPOS_BRIDGE_ENDPOINT`, the optional image-only
`BROWSER_GATEWAY_URL`, and the optional server-local
`WEB_SEARCH_PROVIDER=deepseek` selection are environment variables. The
image sets `GUIONAI_HTTP_IMAGE=1`; local/npm servers leave that marker unset and
retain their direct operations. The gateway URL is a base URL; image mode calls
its `POST /api/render` raw-render operation with `{ "url", "waitMs" }`. They
are not accepted in request bodies.
DeepSeek performs one
auxiliary model call per search; callers receive only normalized results and do
not need to know the Messages/tool wire protocol. The route
schemas in [`packages/web/src/http.ts`](../packages/web/src/http.ts) are the
Expand Down
Loading
Loading