admin: models explorer dashboard UI + local dev server

.gitignore

@@ -53,3 +53,4 @@ tmp/
 # Built Go binaries (build artifacts, never commit)
 /duckgres-controlplane
 /duckgres-worker
+/devserver

CLAUDE.md

@@ -46,7 +46,7 @@ In topologies 2 and 3, the control plane also exposes an Arrow Flight SQL ingres
   - Flight SQL ingress adapter: `flight_ingress.go`
   - Runtime loops: `janitor.go`, `leader_loop.go`, `memory_rebalancer.go`, `runtime_tracker.go`
   - K8s / multitenant under build tag `kubernetes` (including: `multitenant.go`, `k8s_pool.go`, `k8s_pool_acquire.go`, `k8s_pool_spawn.go`, `k8s_pool_lifecycle.go`, `k8s_pool_reconcile.go`, `k8s_pool_helpers.go`, `k8s_factory.go`, `org_router.go`, `org_reserved_pool.go`, `sts_broker.go`, `shared_worker_activator.go`, `worker_rpc_security.go`, `janitor_leader_k8s.go`)
-  - Subpackages: `admin/` (HTTP admin API, `kubernetes` tag), `provisioner/` (k8s controller, `kubernetes` tag), `provisioning/` (HTTP API), `configstore/` (Postgres-backed config)
+  - Subpackages: `admin/` (HTTP admin API + dashboard, `kubernetes` tag; includes the models explorer UI `static/models.html` + `models_api.go`, and `devserver/` for local UI dev against a port-forwarded CP — see `admin/README.md`), `provisioner/` (k8s controller, `kubernetes` tag), `provisioning/` (HTTP API), `configstore/` (Postgres-backed config)
 - **duckdbservice/** — DuckDB Arrow Flight SQL service
   - Core: `service.go`, `flight_handler.go`, `arrow_helpers.go`, `auth.go`, `config.go`
   - Lifecycle, caching, profiling, metrics: `activation.go`, `transient.go`, `cache_proxy.go`, `profiling.go`, `progress.go`, `metrics.go`

controlplane/admin/README.md

@@ -0,0 +1,74 @@
+# Control-plane admin dashboard
+
+The control plane (multi-tenant / `kubernetes` build tag) serves a small admin
+dashboard on `:8080`. It is gated behind the internal secret — there is no
+public exposure; reach it via `kubectl port-forward`.
+
+## Pages
+
+| Route | Page | Purpose |
+|-------|------|---------|
+| `/`, `/models` | `static/models.html` | **Models explorer** — the primary surface. Sidebar of every config-store model grouped Tenants / Config / Runtime, a table of the selected model's rows, and a click-through detail panel. Nested warehouse sub-configs render as expandable sections. |
+| `/orgs` | `static/orgs.html` | Org list + create/delete. |
+| `/workers` | `static/workers.html` | Live worker/duckling status. |
+| `/sessions` | `static/sessions.html` | Active sessions. |
+| `/settings` | `static/settings.html` | Singleton config editing. |
+| `POST /login` | `static/login.html` | Token login (sets the `duckgres_admin_token` cookie). |
+
+The pages are plain HTML + vanilla JS served via `//go:embed static/*` and
+`html/template`. The models explorer and login page follow the dark "mission
+control" design language (Chakra Petch + IBM Plex Mono, cyan/amber accents).
+Because the templates are parsed with `html/template`, **do not write `{{` in
+their inline JS** — only `login.html` uses template directives (`{{.Next}}` /
+`{{.Error}}`).
+
+## Auth
+
+- Service-to-service: `X-Duckgres-Internal-Secret` header.
+- Dashboard UI: the `POST /login` form mints an `HttpOnly` cookie.
+- The internal secret plus rotation fallbacks are accepted (see `TokenSet`).
+- `?token=` URL auth is deliberately rejected (#721).
+
+## Models explorer API (read-only)
+
+Backed by `models_api.go`. The registry in `modelDescriptors()` is the single
+source of truth — add a config-store model there and it appears in the sidebar,
+listing, and column derivation.
+
+- `GET /api/v1/models` → `{ "models": [ { key, label, group, count } ] }` — sidebar with live row counts.
+- `GET /api/v1/models/:model` → `{ key, label, group, table, columns, count, truncated, rows }` — one model's rows.
+
+Rows are scanned into the typed model and marshaled via its `json` tags, so
+columns tagged `json:"-"` (`OrgUser.Password`, `OrgUserSecret.Ciphertext`,
+`DuckLakeConfig.S3SecretKey`, the singleton IDs) are dropped from the response —
+**never swap the typed scan for a raw `map` scan**, that would leak them.
+Runtime-schema tables (`worker_records`, `cp_instances`, …) are schema-qualified
+with `ConfigStore.RuntimeSchema()`. Listings are capped at `modelsRowLimit`
+(`truncated: true` when hit).
+
+Covered by `models_api_test.go` (redaction + real query path) and the
+`models_explorer_api` assertion in `tests/e2e-mw-dev/harness.sh`.
+
+## Local UI development (no redeploy)
+
+Iterate on the UI against a *deployed* control plane without rebuilding the
+image. A tiny dev proxy (`devserver/`) serves `static/` off disk and proxies
+`/api`, `/login`, `/health` to the port-forwarded CP, injecting the internal
+secret server-side. The browser sees one origin → no CORS, and the secret never
+reaches page JS. The embedded UI uses relative `/api/v1` paths, so the same HTML
+runs byte-for-byte embedded or under the dev proxy.
+
+```sh
+# 1. port-forward the deployed control plane's admin API (own terminal):
+just ui-port-forward                       # defaults: posthog-mw-dev / duckgres / duckgres-control-plane
+#   or: kubectl --context <ctx> -n <ns> port-forward deploy/<cp> 8080:8080
+
+# 2. serve the UI locally, pointed at the port-forward:
+DUCKGRES_INTERNAL_SECRET=<cp-secret> just ui-dev
+#   → http://127.0.0.1:5173
+
+# 3. edit controlplane/admin/static/*.html and refresh the browser. No rebuild.
+```
+
+Once the markup is right, it is already what `//go:embed` bakes into the CP — no
+separate build step.

controlplane/admin/api.go

@@ -72,6 +72,10 @@ type OrgStackInfo interface {
 // RegisterAPI registers all admin REST endpoints on the given router group.
 func RegisterAPI(r *gin.RouterGroup, store *configstore.ConfigStore, info OrgStackInfo) {
 	registerAPIWithStore(r, newGormAPIStore(store), info)
+	// Generic read-only models explorer (sidebar + table + detail UI). Reads
+	// the concrete store directly because it needs the runtime schema name and
+	// raw DB for tables the typed apiStore interface doesn't surface.
+	registerModelsAPI(r, store)
 }
 
 func registerAPIWithStore(r *gin.RouterGroup, store apiStore, info OrgStackInfo) {

controlplane/admin/dashboard.go

@@ -85,7 +85,11 @@ func APIAuthMiddleware(tokens TokenSet) gin.HandlerFunc {
 
 // RegisterDashboard serves the admin dashboard on the Gin engine.
 func RegisterDashboard(r *gin.Engine, tokens TokenSet) {
-	r.GET("/", dashboardPageHandler("index.html", tokens))
+	// The models explorer is the primary dashboard surface: a sidebar of every
+	// config-store model, a table of its rows, and a detail view per row. "/"
+	// and "/models" both land here. The legacy per-entity pages stay reachable.
+	r.GET("/", dashboardPageHandler("models.html", tokens))
+	r.GET("/models", dashboardPageHandler("models.html", tokens))
 	r.GET("/orgs", dashboardPageHandler("orgs.html", tokens))
 	r.GET("/workers", dashboardPageHandler("workers.html", tokens))
 	r.GET("/sessions", dashboardPageHandler("sessions.html", tokens))

controlplane/admin/devserver/main.go

@@ -0,0 +1,131 @@
+// Command devserver serves the admin dashboard's static assets off disk and
+// reverse-proxies the API to a port-forwarded control plane, so the UI can be
+// iterated on locally without rebuilding/redeploying the control-plane image.
+//
+// It mirrors the embedded serving contract exactly: the browser talks to a
+// single origin (this server), static files are served from ./static, and any
+// /api/, /login, or /health request is proxied to the target CP with the
+// internal-secret header injected server-side. Because it's same-origin from
+// the browser's point of view there is no CORS to configure and the secret
+// never reaches the page's JavaScript — the UI uses relative /api/v1 paths and
+// works byte-for-byte the same whether embedded or served here.
+//
+// Usage:
+//
+//	# 1. port-forward the control plane's admin API (separate terminal):
+//	kubectl --context posthog-mw-dev -n <ns> port-forward deploy/duckgres-control-plane 8080:8080
+//
+//	# 2. run the dev server pointing at it:
+//	DUCKGRES_INTERNAL_SECRET=<secret> go run ./controlplane/admin/devserver
+//	# then open http://127.0.0.1:5173
+//
+// Edit controlplane/admin/static/*.html and just refresh the browser — no
+// rebuild. Once happy, the same files are what //go:embed bakes into the CP.
+package main
+
+import (
+	"flag"
+	"log"
+	"net/http"
+	"net/http/httputil"
+	"net/url"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+func main() {
+	listen := flag.String("listen", "127.0.0.1:5173", "address to serve the UI on")
+	target := flag.String("target", "http://127.0.0.1:8080", "base URL of the port-forwarded control plane admin API")
+	staticDir := flag.String("static", defaultStaticDir(), "directory of static UI assets to serve off disk")
+	secret := flag.String("secret", os.Getenv("DUCKGRES_INTERNAL_SECRET"), "internal secret injected as X-Duckgres-Internal-Secret on proxied requests (defaults to $DUCKGRES_INTERNAL_SECRET)")
+	flag.Parse()
+
+	if *secret == "" {
+		log.Println("WARNING: no internal secret set — proxied API requests will be rejected with 401. Set --secret or $DUCKGRES_INTERNAL_SECRET.")
+	}
+
+	targetURL, err := url.Parse(*target)
+	if err != nil {
+		log.Fatalf("invalid --target %q: %v", *target, err)
+	}
+
+	absStatic, err := filepath.Abs(*staticDir)
+	if err != nil {
+		log.Fatalf("resolve --static %q: %v", *staticDir, err)
+	}
+	if _, err := os.Stat(filepath.Join(absStatic, "models.html")); err != nil {
+		log.Fatalf("static dir %q has no models.html (run from repo root, or pass --static): %v", absStatic, err)
+	}
+
+	proxy := httputil.NewSingleHostReverseProxy(targetURL)
+	baseDirector := proxy.Director
+	proxy.Director = func(req *http.Request) {
+		baseDirector(req)
+		req.Host = targetURL.Host
+		// Inject the service-to-service secret server-side; never trust or
+		// forward a browser cookie/credential through the dev proxy.
+		req.Header.Del("Cookie")
+		if *secret != "" {
+			req.Header.Set("X-Duckgres-Internal-Secret", *secret)
+		}
+	}
+
+	mux := http.NewServeMux()
+	mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
+		if isProxiedPath(r.URL.Path) {
+			proxy.ServeHTTP(w, r)
+			return
+		}
+		serveStatic(w, r, absStatic)
+	})
+
+	log.Printf("duckgres admin UI dev server: http://%s  →  proxying /api,/login,/health to %s", *listen, targetURL)
+	log.Printf("serving static assets from %s (edit + refresh, no rebuild)", absStatic)
+	if err := http.ListenAndServe(*listen, mux); err != nil {
+		log.Fatal(err)
+	}
+}
+
+// isProxiedPath reports whether a request should be forwarded to the control
+// plane rather than served from the local static dir.
+func isProxiedPath(p string) bool {
+	return strings.HasPrefix(p, "/api/") || p == "/login" || p == "/health"
+}
+
+// dashboardPages maps a request path (route or bare filename) to the static
+// asset that serves it. The values are literals, so the filename handed to
+// http.ServeFile never derives from the request — no path traversal is possible
+// regardless of what the client sends. Routes mirror the CP's
+// RegisterDashboard table; "/" and "/models" both land on the models explorer.
+var dashboardPages = map[string]string{
+	"":              "models.html",
+	"models":        "models.html",
+	"orgs":          "orgs.html",
+	"workers":       "workers.html",
+	"sessions":      "sessions.html",
+	"settings":      "settings.html",
+	"login":         "login.html",
+	"models.html":   "models.html",
+	"orgs.html":     "orgs.html",
+	"workers.html":  "workers.html",
+	"sessions.html": "sessions.html",
+	"settings.html": "settings.html",
+	"login.html":    "login.html",
+}
+
+// serveStatic serves the dashboard's static assets. Any unrecognized path falls
+// back to the models explorer (single-page-style entry).
+func serveStatic(w http.ResponseWriter, r *http.Request, dir string) {
+	file, ok := dashboardPages[strings.TrimPrefix(r.URL.Path, "/")]
+	if !ok {
+		file = "models.html"
+	}
+	http.ServeFile(w, r, filepath.Join(dir, file))
+}
+
+// defaultStaticDir resolves the static dir relative to the repo layout so the
+// server can be run from the repo root with no flags.
+func defaultStaticDir() string {
+	return filepath.Join("controlplane", "admin", "static")
+}