feat(cryptify): Grafana usage dashboard, pinned to the /metrics exporter

CLAUDE.md

@@ -19,6 +19,7 @@ Migrated from the dobby memory repo (`encryption4all/dobby`). This file is the h
 - The oasdiff gate's settings are **not** self-evident and `--fail-on ERR` alone fails open. `fail-on: WARN` is deliberate: oasdiff rates removing or renaming an *optional* response property, and removing a request parameter, as WARN, and this spec marks only `status` as `required`, so at ERR the gate silently passed a removed `key` (the IBE user secret key `/v2/request/key` exists to return), a renamed `proofStatus` and a dropped `timestamp` parameter. Two more, a changed non-success status (401 to 403) and a dropped response enum value, rate ERR but are **opt-in**, so they only run when named in `include-checks`. Of the 31 WARN checks, `response-property-enum-value-added` is the only one that fires on something `COMPATIBILITY.md` does not already forbid, which is why that document now names a new response enum value as non-additive too. Reproduce a verdict with the exact flags the action's entrypoint builds (`oasdiff/oasdiff-action/breaking@v0.1.10` is `FROM tufin/oasdiff:v1.26.1`, so the pinned tag is what makes a local run authoritative): `git show origin/main:pg-pkg/api-description.yaml > /tmp/base.yaml && oasdiff breaking /tmp/base.yaml pg-pkg/api-description.yaml --allow-external-refs=false --composed=false --fail-on WARN --include-checks response-non-success-status-removed,response-property-enum-value-removed`. Two traps: `oasdiff --version` prints `oasdiff version main` after a `go install` of a tag because the version comes from release ldflags (the code is still the tag), and `--fail-on` takes `ERR`/`WARN` while `oasdiff checks --severity` takes `error`/`warn`/`info`. `--severity ERR` is a usage error, so `oasdiff checks --severity ERR | wc -l` counts the help text instead and badly undercounts the tier (it is 213 error checks, 31 warn, 265 info). The spec has no external `$ref`s, so `allow-external-refs` stays at its safe (SSRF-guarding) default. The gate only sees paths the spec documents, and the spec documents canonical paths only, so dropping the `/v2/irma/...` alias handlers (#257) passes it.
 - **Importing another repo's history here silently closes issues here. Two vectors fire it, each sufficient on its own, so rule out neither.** Imported commits carry their original messages verbatim, closing keywords included, and GitHub resolves those against the **destination** repo's numbering. **Vector 1, the squash body.** This repo's `squash_merge_commit_message` is `COMMIT_MESSAGES` (the same setting the release-plz bullet above turns on its head — there it is why a `BREAKING CHANGE:` footer in a PR body never reaches the commit), so squash-merging the import PR concatenates every imported commit message into the merge commit's body. `ba380a1`'s body is 1678 lines and carries all 17 refs (`Closes #38 #45 #52 #54 #123 #125 #134 #142 #146 #153 #155 #157 #159 #167 #186 #191 #194`); 16 pointed at numbers already closed here, and one — #146, a live unimplemented feature request — was closed four seconds after it landed, with nothing warning, and stood four days before being found and reopened. Note what this means: a plain squash merge is enough by itself, *because* squashing concatenates the messages even as it throws the history away. **Vector 2, the history itself.** Here GitHub never reads the merge commit's message — it attributes the closes to the **imported commits**, once they become reachable from the default branch. This vector went untested in this repo rather than disproven: `9887e1a` carries no keywords of its own, and by the time it landed thirteen minutes later all 17 targets were already closed, and GitHub does not re-close a closed issue. It is measured in encryption4all/postguard-js#139, which is also the guard: import PR #137 merged with a real two-parent merge commit (`b1bb2ee`, a three-line message, no keywords, no squash body anywhere), and js#128/#129 closed six seconds later attributed to imported commits `a0ce27f`/`2acf42f` — single-parent, committed 2026-06-04 — then reopened fifteen minutes later. **So the audit is the only step that covers both.** Before merging *and* after, from the import branch: `git log origin/main..HEAD --pretty=%B | grep -oiE '(close[sd]?|fix(e[sd])?|resolve[sd]?) +#[0-9]+' | sort -u`, then check each number here. Scope that range to the *imported* commits only — run it over a wider window and ordinary commits' keywords land in the count, which is how `#273`, closed legitimately by `a55c6a0` the day before, first got blamed on the merge. Editing the squash body in the merge dialog defuses vector 1 and only vector 1; on vector 2 there is no body to edit, and rewriting the keywords out of the imported commits costs every imported SHA (`git filter-repo`). Do **not** reach for `commit_message` on `PUT /pulls/N/merge` as the scripted version of that: the REST reference words it "Extra detail to append to automatic commit message" and documents nothing about `merge_method=squash`, so whether it replaces the concatenated body or appends to it is unverified — and if it appends, the defusal silently no-ops and every keyword still fires. Confirm it on a throwaway repo and record the answer here before relying on it. Reading close *state* cannot tell you which happened, so read the close **event** — and read it correctly, because the obvious rule is wrong. A keyword in a *commit message* is attributed to the commit and carries a `commit_id`; a keyword in the *PR body* is attributed to the PR and carries `commit_id: null`. #146 shows `ba380a1464…` because PR #277's body said only `Closes #255`, while #273 shows `null` because PR #274's body said `Closes #273`. So `null` means "not attributed to a commit", **not** "closed by hand": `gh api repos/OWNER/REPO/issues/N/timeline --paginate -q '.[] | select(.event=="closed") | .created_at+" "+(.commit_id//"-")'`. Same failure class as the unapplied-workflow half below (#272).
 - The `dobby-coder` GitHub App lacks `workflows: write` on this repo; any push touching `.github/workflows/*.yml` is rejected at the remote. Before treating a fix as blocked, check whether the same effect can be achieved in a pushable file (crate manifest, source, committed script); if a fix genuinely can only live in a workflow file, ship the pushable half and hand the maintainer ready-to-paste YAML in the PR body. The block covers *merge* commits too, which is easy to miss: once a branch carries its own `build.yml` change (typically a maintainer applying such a patch onto it), a later `git merge origin/main` that has to touch `build.yml` produces a commit updating a workflow file, and the push is rejected even when the resolution is only "keep both new jobs". Nothing can be split out of a merge commit, so that merge has to be landed by a maintainer, or the App needs `workflows: write`. Measured exception, worth trying before handing the sync over: the App pushed `ce0fc59` on this branch, a merge whose diff against its first parent added main's 64 new `build.yml` lines. That merge needed no resolution inside `build.yml` — it took main's side whole, so the blob it committed already existed in the repo. Try the merge and read the remote's answer; only escalate on an actual rejection.
+- `gh pr edit` does not work on this repo and **fails in the direction that looks like success**: a classic project is attached, so the mutation `gh` sends requests `repository.pullRequest.projectCards` and the whole call dies with `GraphQL: Projects (classic) is being deprecated ... (repository.pullRequest.projectCards)`. That message reads as a deprecation warning, `gh` still exits 0, and the edit silently does not land — so a `--body-file` update looks applied and isn't. Use REST instead, which touches no project fields: `gh api -X PATCH repos/encryption4all/postguard/pulls/N -f body="$(cat body.md)"`. Verify by reading the body back (`gh pr view N --json body`), because the failure is invisible otherwise. Same trap for `--title`/`--add-label` through `gh pr edit`.
 - **Consolidating a repo into a monorepo means transfer its open issues first, then archive — not archive-then-orphan.** `postguard-website`, `postguard-outlook-addon`, `postguard-tb-addon` and `postguard-examples` were archived (read-only) after folding into `postguard-js`, and the "open issues transfer here" step was silently skipped on all four: 62 open issues sat stranded, unworkable (`HTTP 403: Repository was archived so is read-only` on label/assign/comment/close). GitHub will not transfer an issue out of an already-archived repo, so recovering from the skip costs an unarchive → transfer → re-archive round trip per repo instead of a single transfer before archiving (decided in postguard#282). Do the transfer as part of the same change that archives the repo, and leave a "development moved to `<new repo>`, see `<new path>`" banner at the top of the archived repo's README before re-archiving, so an old link still finds the new home.
 
 ## Dependencies

cryptify/CLAUDE.md

@@ -203,8 +203,9 @@ works fine. The regex is anchored (`^...$`), so there's no subdomain/wildcard
 bypass.
 
 ## Metrics
-- `GET /metrics`: Prometheus text format, unauthenticated by design. Lock down at
-  the firewall, not the endpoint.
+- `GET /metrics`: Prometheus text format. Gated by a Bearer token when
+  `metrics_token` is set; with the key unset the endpoint is open and startup
+  logs a warning. Lock it down at the firewall either way.
 - Channel label derived in priority: `X-Cryptify-Source`, then
   `Authorization: Bearer` / `X-Api-Key` (-> `api`), then `Origin` (-> `website` /
   `staging-website`), then `User-Agent` (-> `outlook` / `thunderbird`), then
@@ -213,6 +214,18 @@ bypass.
   `metrics_scan_interval_secs`).
 - `FileState.source_channel` is populated at `upload_init` from request headers;
   populate it in any new test fixtures too.
+- **No metric here knows which deployment it runs in.** There is no `env` label
+  in the exporter and adding one would be wrong: staging and Procolix production
+  are separate scrape targets, so `env` belongs in the Prometheus job's static
+  `labels:`. `docs/grafana/README.md` has the scrape config.
+- The reference dashboard is `docs/grafana/cryptify-usage.json`, pinned to the
+  exporter by `mod dashboard_tests` in `src/metrics.rs`. It reads the `# TYPE`
+  lines out of a real `Metrics::render()` and compares them against the metric
+  names in the committed panel queries, both directions, so renaming a metric
+  fails `cargo test` rather than silently emptying a graph. It also requires
+  every panel query to carry `env=~"$env"`. Adding a metric therefore means
+  adding a panel in the same PR. That coupling is deliberate; don't loosen the
+  test to avoid it.
 
 ## Integration test harness
 - `build_rocket(figment, vk)` is the injection point. `#[launch] rocket()` wraps it

cryptify/docs/grafana/README.md

@@ -0,0 +1,85 @@
+# Grafana dashboard for cryptify usage
+
+`cryptify-usage.json` is the reference dashboard behind
+[postguard#305](https://github.com/encryption4all/postguard/issues/305): messages
+sent per channel, and cryptify storage in use per environment.
+
+It reads only metrics that `GET /metrics` already exports (see
+`cryptify/src/metrics.rs`). No exporter change is needed to import it.
+
+## Metrics it uses
+
+| Metric | Type | Labels | Panel |
+| --- | --- | --- | --- |
+| `cryptify_uploads_total` | counter | `channel` | Messages sent per channel |
+| `cryptify_upload_bytes_total` | counter | `channel` | Bytes uploaded per channel |
+| `cryptify_uploads_by_app_total` | counter | `app` | Messages per client app |
+| `cryptify_storage_bytes` | gauge | none | Storage in use |
+| `cryptify_active_files` | gauge | none | Files on disk |
+| `cryptify_expired_files_total` | counter | none | Uploads expired before finalize |
+
+`channel` is `website`, `staging-website`, `outlook`, `thunderbird`, `api` or
+`unknown`; `app` is `pg-js`, `pg-dotnet`, `pg4ol`, `pg4tb` or `unknown`. Both
+label sets are seeded at 0 on startup, so a channel with no traffic still shows
+as a zero line instead of vanishing from the legend.
+
+The counters are per process. A cryptify restart resets them to 0, which is why
+every panel goes through `increase()` rather than reading the raw counter.
+
+## The `env` label is supplied by Prometheus, not by cryptify
+
+Nothing in the exporter knows which deployment it is running in. Splitting
+staging from Procolix production is the scrape config's job: attach a static
+`env` label per job, and the dashboard's `Environment` variable picks it up.
+
+```yaml
+scrape_configs:
+  - job_name: cryptify
+    metrics_path: /metrics
+    scheme: https
+    authorization:
+      type: Bearer
+      credentials_file: /etc/prometheus/cryptify-metrics-token
+    static_configs:
+      - targets: ["cryptify.staging.postguard.eu"]
+        labels:
+          env: staging
+      - targets: ["cryptify.postguard.eu"]
+        labels:
+          env: production
+```
+
+Set the same token as `metrics_token` in each deployment's `conf/config.toml`
+(or `ROCKET_METRICS_TOKEN` in its environment). With no token configured the
+endpoint answers unauthenticated and logs a warning at startup, so keep it
+restricted to the Prometheus segment at the firewall as well.
+
+Adjust the target hostnames to whatever the two deployments actually resolve to.
+The dashboard does not care about the hostnames, only that `env` is present.
+
+The two gauges carry no labels of their own, so a raw select distinguishes series
+only by `instance` and `job`. The storage and file-count panels therefore go
+through `max by (env)`: with one target per environment it changes nothing, and
+if an environment ever gets a second target sharing the volume it reports the
+volume once instead of twice. Switch those three panels to `sum by (env)` if the
+targets get separate volumes.
+
+## Importing
+
+Grafana, Dashboards, New, Import, upload `cryptify-usage.json`, pick the
+Prometheus data source. The datasource is a dashboard variable rather than a
+baked-in UID, so the same file imports into any Grafana instance.
+
+Storage panels are sampled from `data_dir` on a background task every
+`metrics_scan_interval_secs` (60 by default), so they trail a burst of uploads
+by up to one interval. The dashboard refreshes every 5 minutes and opens on a
+30-day window, which suits monthly usage reporting; shorten both if you are
+watching a deploy.
+
+## Keeping it honest
+
+`mod dashboard_tests` in `cryptify/src/metrics.rs` reads this JSON and checks it
+against `Metrics::render()`: every metric the exporter emits appears on the
+dashboard, every `cryptify_*` name the dashboard queries is one the exporter
+actually emits, and every panel filters on `env=~"$env"`. Renaming a metric or
+adding a panel that ignores the environment filter fails `cargo test`.