admin: models explorer dashboard UI + local dev server#821
Merged
Conversation
Add a generic, read-only models explorer to the control-plane admin dashboard: a sidebar of every config-store model (grouped Tenants / Config / Runtime), a table of the selected model's rows, and a click-through detail panel where nested managed-warehouse sub-configs render as expandable sections. "/" and "/models" now serve this as the primary dashboard surface; the legacy per-entity pages stay reachable. Backend (controlplane/admin/models_api.go, kubernetes tag): - GET /api/v1/models — sidebar entries with live row counts. - GET /api/v1/models/:model — one model's rows, derived columns, count, truncation flag. - modelDescriptors() is the single source of truth (13 persisted models across the config + runtime schemas). Rows are scanned into the typed model and marshaled via json tags, so secret-bearing columns tagged json:"-" (OrgUser.Password, OrgUserSecret.Ciphertext, DuckLakeConfig.S3SecretKey, the singleton IDs) are dropped from the response — never swap the typed scan for a raw map scan. Runtime-schema tables are schema-qualified via RuntimeSchema(); listings are capped at modelsRowLimit. UI (static/models.html, static/login.html): - New mission-control design language (Chakra Petch + IBM Plex Mono, dark HUD, cyan/amber accents). Login page restyled to match. Uses relative /api/v1 paths and cookie auth, with a session-expired gate on 401. Local UI development without redeploy (controlplane/admin/devserver): - A stdlib dev proxy serves static/ off disk and proxies /api,/login,/health to a port-forwarded control plane, injecting the internal secret server-side. Same-origin from the browser → no CORS, secret never reaches page JS, and the embedded UI runs byte-for-byte the same under the proxy. just recipes ui-port-forward + ui-dev wire it up. Auth is unchanged: the existing internal-secret token (header / login cookie) gates the whole /api/v1 group and the dashboard pages. Tests: - models_api_test.go: a no-DB guard pinning the registry + redaction invariant (sensitive columns must stay json:"-"), plus a Postgres-backed test exercising the real query path (sidebar counts, config + runtime listings, end-to-end "password hash never appears in the response bytes"). - tests/e2e-mw-dev/harness.sh: models_explorer_api assertion against the live cluster — sidebar enumeration, counts, runtime-schema listing, 404 on unknown model, and the load-bearing check that org_users listings never expose a password column/field. Docs: controlplane/admin/README.md (pages, API, redaction invariant, local dev loop) and the CLAUDE.md admin bullet.
Test Impact PlanDeterministic summary of how this PR changes tests, CI runners, and coverage-risk signals. Summary
Signals
Coverage risk: needs review Warnings
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Replace the join-then-validate path handling in the dev server's static handler with a constant lookup table keyed by request path. The filename passed to http.ServeFile now always comes from literal map values, so it can never derive from client input — closes CodeQL alerts 568/569 and removes the need for the manual traversal guard.
benben
added a commit
that referenced
this pull request
Jun 29, 2026
#821 (models explorer UI) merged to main after this branch was cut. Its models_api.go registered the four config-singleton models this branch deletes, so the merged tree failed to build (undefined: configstore.GlobalConfig etc). Merge main in and reconcile: - Remove the global/ducklake/ratelimit/querylog descriptors from models_api.go (and the now-unused modelGroupConfig group). - Drop them from models_api_test.go (wantKeys + the global-config redaction entry) and the harness models_explorer_api sidebar key list. - Drop the dead Settings nav link + the empty Config sidebar group from models.html, and the /settings row from admin/README.md (the /settings page and its config API were removed on this branch). Build (plain + kubernetes), all kubernetes test binaries, and the admin redaction test pass.
benben
added a commit
that referenced
this pull request
Jun 29, 2026
* configstore: remove dead config singletons + unused model fields
Audit-driven cleanup of config-store surface that is persisted/served but never
read to drive runtime behavior. No user-visible behavior change.
Removed (confirmed dead by a full cross-repo audit incl. the Crossplane
compositions in posthog/charts):
- The four cluster-wide singleton config tables — GlobalConfig, DuckLakeConfig,
RateLimitConfig, QueryLogConfig — and their Snapshot fields, FirstOrCreate
seeds, snapshot loads, the admin GET/PUT /config/* API (handlers, routes,
apiStore methods, gormAPIStore impls), and the /settings dashboard page.
Nothing reads Snapshot.Global/DuckLake/RateLimit/QueryLog; effective config
comes from CLI flags/env (server.Config) and the per-org ManagedWarehouse
contract. (DuckLakeConfig was already marked legacy.)
- ManagedWarehouse dead columns: warehouse_database_{region,database_name,
username}, metadata_store_{engine,region}, worker_identity_service_account_name
(the worker SA name is computed, not read from config), warehouse_database_state
(warehouse_database has no provisioning sub-state), and the six per-component
*_status_message columns (the provisioner only writes the top-level
status_message). Kept warehouse_database endpoint/port (curated provision
response) and the per-component *_state fields that drive readiness.
- Runtime coordination dead columns: cp_instances.pod_uid/boot_id (already
encoded into the cp instance id), worker_records.pod_uid (never written),
org_connection_queue.canceled_at (cancel is a hard DELETE, never set) — plus
the ControlPlaneRuntimeTracker podUID/bootID fields and constructor params.
Migration & rollout safety:
- New goose migration 000004 drops the four config tables and the 13 dead
managed_warehouses columns (with a best-effort Down that recreates them).
- Runtime tables are AutoMigrate'd in a per-deployment schema, so the dead
runtime columns are dropped in autoMigrateRuntimeTables (schema-qualified,
DROP COLUMN IF EXISTS, idempotent). This runs before the first heartbeat —
critical because cp_instances.pod_uid/boot_id were NOT NULL, so a rollout that
stopped supplying them would otherwise fail every UpsertControlPlaneInstance.
Also updated: seed SQL (k8s + test fixtures), all affected unit/postgres tests
(including the migrations metadata-parity test, now asserting the config tables
are absent at version 4), and the README note about the former DuckLakeConfig
singleton.
Verified: `go build ./...`, `go build -tags kubernetes ./controlplane/... ./cmd/...`,
all kubernetes-tagged test binaries compile, and the configstore + admin
no-docker unit tests pass.
* Reconcile with #821: drop config-singleton models from the explorer
#821 (models explorer UI) merged to main after this branch was cut. Its
models_api.go registered the four config-singleton models this branch deletes,
so the merged tree failed to build (undefined: configstore.GlobalConfig etc).
Merge main in and reconcile:
- Remove the global/ducklake/ratelimit/querylog descriptors from
models_api.go (and the now-unused modelGroupConfig group).
- Drop them from models_api_test.go (wantKeys + the global-config redaction
entry) and the harness models_explorer_api sidebar key list.
- Drop the dead Settings nav link + the empty Config sidebar group from
models.html, and the /settings row from admin/README.md (the /settings page
and its config API were removed on this branch).
Build (plain + kubernetes), all kubernetes test binaries, and the admin
redaction test pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a generic, read-only models explorer to the control-plane admin dashboard, and a local dev workflow for iterating on the UI without redeploying.
s3,iceberg,metadata_store, theSecretRefs, …) render as expandable sections./and/modelsnow serve this as the primary dashboard surface; the legacy per-entity pages stay reachable.Backend (
controlplane/admin/models_api.go,kubernetestag)GET /api/v1/models→ sidebar entries + counts.GET /api/v1/models/:model→ rows, columns, count, truncation flag.modelDescriptors()is the single source of truth — add a model there and it appears in the sidebar, listing, and column derivation.jsontags, so columns taggedjson:"-"(OrgUser.Password,OrgUserSecret.Ciphertext,DuckLakeConfig.S3SecretKey, the singleton IDs) are dropped. Runtime-schema tables are schema-qualified viaRuntimeSchema(); listings capped atmodelsRowLimit.UI (
static/models.html, restyledstatic/login.html)Mission-control design language (Chakra Petch + IBM Plex Mono, dark HUD, cyan/amber accents). Uses relative
/api/v1paths + cookie auth, with a session-expired gate on 401.Local UI dev without redeploy (
controlplane/admin/devserver/)A stdlib dev proxy serves
static/off disk and proxies/api,/login,/healthto a port-forwarded control plane, injecting the internal secret server-side. Same-origin from the browser → no CORS, secret never reaches page JS, and the embedded UI runs byte-for-byte the same under the proxy.Auth
Unchanged — the existing internal-secret token (header / login cookie) gates the whole
/api/v1group and the dashboard pages. Token login is enough "for now" as requested; can be hardened later.Tests
models_api_test.go: no-DB guard pinning the registry + the redaction invariant (sensitive columns must stayjson:"-"), plus a Postgres-backed test exercising the real query path including "password hash never appears in the response bytes".tests/e2e-mw-dev/harness.sh::models_explorer_api: live-cluster assertion — sidebar enumeration, counts, runtime-schema listing, 404 on unknown model, and the load-bearing check thatorg_userslistings never expose a password column/field.Docs
controlplane/admin/README.md(pages, API, redaction invariant, local dev loop) + theCLAUDE.mdadmin bullet.🤖 Generated with Claude Code