From 50e36a928d9ab2f32faa2d4280ebebbcd6746a2a Mon Sep 17 00:00:00 2001 From: "dobby-yivi-agent[bot]" <275734547+dobby-yivi-agent[bot]@users.noreply.github.com> Date: Sat, 8 Aug 2026 13:10:24 +0000 Subject: [PATCH 1/4] feat(cryptify): commit the Grafana usage dashboard and pin it to the exporter The /metrics exporter has been in place since cryptify#102, and metrics.rs line 9 already points readers at `docs/grafana/` for the dashboard that goes with it. That directory never existed, so the reference dangled and the dashboard side of postguard#305 stayed unbuilt. Adds cryptify-usage.json: messages sent per channel and per client app, bytes uploaded, storage in use, files on disk, and uploads that expired before finalize. Staging and Procolix production split on an `env` label, which the Prometheus job supplies via static labels rather than the exporter, since one process has no way to know which deployment it is. `mod dashboard_tests` keeps the two honest. It reads the `# TYPE` lines out of a real Metrics::render() and compares them against the metric names in the committed panel queries, in both directions, so a rename fails cargo test instead of silently emptying a graph. It also requires every panel query to carry the env matcher, so no panel can quietly sum staging into production. Refs encryption4all/postguard#305 Co-Authored-By: Claude Opus 5 --- cryptify/CLAUDE.md | 17 +- cryptify/docs/grafana/README.md | 78 ++++++ cryptify/docs/grafana/cryptify-usage.json | 299 ++++++++++++++++++++++ cryptify/src/metrics.rs | 164 ++++++++++++ 4 files changed, 556 insertions(+), 2 deletions(-) create mode 100644 cryptify/docs/grafana/README.md create mode 100644 cryptify/docs/grafana/cryptify-usage.json diff --git a/cryptify/CLAUDE.md b/cryptify/CLAUDE.md index 073a7878..dca983ac 100644 --- a/cryptify/CLAUDE.md +++ b/cryptify/CLAUDE.md @@ -203,8 +203,9 @@ works fine. The regex is anchored (`^...$`), so there's no subdomain/wildcard bypass. ## Metrics -- `GET /metrics`: Prometheus text format, unauthenticated by design. Lock down at - the firewall, not the endpoint. +- `GET /metrics`: Prometheus text format. Gated by a Bearer token when + `metrics_token` is set; with the key unset the endpoint is open and startup + logs a warning. Lock it down at the firewall either way. - Channel label derived in priority: `X-Cryptify-Source`, then `Authorization: Bearer` / `X-Api-Key` (-> `api`), then `Origin` (-> `website` / `staging-website`), then `User-Agent` (-> `outlook` / `thunderbird`), then @@ -213,6 +214,18 @@ bypass. `metrics_scan_interval_secs`). - `FileState.source_channel` is populated at `upload_init` from request headers; populate it in any new test fixtures too. +- **No metric here knows which deployment it runs in.** There is no `env` label + in the exporter and adding one would be wrong: staging and Procolix production + are separate scrape targets, so `env` belongs in the Prometheus job's static + `labels:`. `docs/grafana/README.md` has the scrape config. +- The reference dashboard is `docs/grafana/cryptify-usage.json`, pinned to the + exporter by `mod dashboard_tests` in `src/metrics.rs`. It reads the `# TYPE` + lines out of a real `Metrics::render()` and compares them against the metric + names in the committed panel queries, both directions, so renaming a metric + fails `cargo test` rather than silently emptying a graph. It also requires + every panel query to carry `env=~"$env"`. Adding a metric therefore means + adding a panel in the same PR. That coupling is deliberate; don't loosen the + test to avoid it. ## Integration test harness - `build_rocket(figment, vk)` is the injection point. `#[launch] rocket()` wraps it diff --git a/cryptify/docs/grafana/README.md b/cryptify/docs/grafana/README.md new file mode 100644 index 00000000..4460eb45 --- /dev/null +++ b/cryptify/docs/grafana/README.md @@ -0,0 +1,78 @@ +# Grafana dashboard for cryptify usage + +`cryptify-usage.json` is the reference dashboard behind +[postguard#305](https://github.com/encryption4all/postguard/issues/305): messages +sent per channel, and cryptify storage in use per environment. + +It reads only metrics that `GET /metrics` already exports (see +`cryptify/src/metrics.rs`). No exporter change is needed to import it. + +## Metrics it uses + +| Metric | Type | Labels | Panel | +| --- | --- | --- | --- | +| `cryptify_uploads_total` | counter | `channel` | Messages sent per channel | +| `cryptify_upload_bytes_total` | counter | `channel` | Bytes uploaded per channel | +| `cryptify_uploads_by_app_total` | counter | `app` | Messages per client app | +| `cryptify_storage_bytes` | gauge | none | Storage in use | +| `cryptify_active_files` | gauge | none | Files on disk | +| `cryptify_expired_files_total` | counter | none | Uploads expired before finalize | + +`channel` is `website`, `staging-website`, `outlook`, `thunderbird`, `api` or +`unknown`; `app` is `pg-js`, `pg-dotnet`, `pg4ol`, `pg4tb` or `unknown`. Both +label sets are seeded at 0 on startup, so a channel with no traffic still shows +as a zero line instead of vanishing from the legend. + +The counters are per process. A cryptify restart resets them to 0, which is why +every panel goes through `increase()` rather than reading the raw counter. + +## The `env` label is supplied by Prometheus, not by cryptify + +Nothing in the exporter knows which deployment it is running in. Splitting +staging from Procolix production is the scrape config's job: attach a static +`env` label per job, and the dashboard's `Environment` variable picks it up. + +```yaml +scrape_configs: + - job_name: cryptify + metrics_path: /metrics + scheme: https + authorization: + type: Bearer + credentials_file: /etc/prometheus/cryptify-metrics-token + static_configs: + - targets: ["cryptify.staging.postguard.eu"] + labels: + env: staging + - targets: ["cryptify.postguard.eu"] + labels: + env: production +``` + +Set the same token as `metrics_token` in each deployment's `conf/config.toml` +(or `ROCKET_METRICS_TOKEN` in its environment). With no token configured the +endpoint answers unauthenticated and logs a warning at startup, so keep it +restricted to the Prometheus segment at the firewall as well. + +Adjust the target hostnames to whatever the two deployments actually resolve to. +The dashboard does not care about the hostnames, only that `env` is present. + +## Importing + +Grafana, Dashboards, New, Import, upload `cryptify-usage.json`, pick the +Prometheus data source. The datasource is a dashboard variable rather than a +baked-in UID, so the same file imports into any Grafana instance. + +Storage panels are sampled from `data_dir` on a background task every +`metrics_scan_interval_secs` (60 by default), so they trail a burst of uploads +by up to one interval. The dashboard refreshes every 5 minutes and opens on a +30-day window, which suits monthly usage reporting; shorten both if you are +watching a deploy. + +## Keeping it honest + +`mod dashboard_tests` in `cryptify/src/metrics.rs` reads this JSON and checks it +against `Metrics::render()`: every metric the exporter emits appears on the +dashboard, every `cryptify_*` name the dashboard queries is one the exporter +actually emits, and every panel filters on `env=~"$env"`. Renaming a metric or +adding a panel that ignores the environment filter fails `cargo test`. diff --git a/cryptify/docs/grafana/cryptify-usage.json b/cryptify/docs/grafana/cryptify-usage.json new file mode 100644 index 00000000..874c8dec --- /dev/null +++ b/cryptify/docs/grafana/cryptify-usage.json @@ -0,0 +1,299 @@ +{ + "title": "PostGuard usage (cryptify)", + "uid": "postguard-cryptify-usage", + "description": "Messages sent per channel and cryptify storage in use, per environment. Scrapes the cryptify /metrics endpoint; see docs/grafana/README.md for the Prometheus job.", + "tags": ["postguard", "cryptify", "usage"], + "timezone": "browser", + "editable": true, + "schemaVersion": 39, + "version": 1, + "refresh": "5m", + "time": { + "from": "now-30d", + "to": "now" + }, + "templating": { + "list": [ + { + "name": "datasource", + "label": "Data source", + "type": "datasource", + "query": "prometheus", + "current": {}, + "hide": 0 + }, + { + "name": "env", + "label": "Environment", + "type": "query", + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "query": "label_values(cryptify_storage_bytes, env)", + "refresh": 1, + "includeAll": true, + "multi": true, + "current": {}, + "hide": 0 + } + ] + }, + "panels": [ + { + "id": 1, + "type": "row", + "title": "Messages sent", + "collapsed": false, + "gridPos": { "h": 1, "w": 24, "x": 0, "y": 0 }, + "panels": [] + }, + { + "id": 2, + "type": "stat", + "title": "Messages sent (selected range)", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "gridPos": { "h": 8, "w": 6, "x": 0, "y": 1 }, + "fieldConfig": { + "defaults": { "unit": "short", "decimals": 0 }, + "overrides": [] + }, + "options": { + "reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }, + "colorMode": "none", + "graphMode": "none", + "textMode": "value" + }, + "targets": [ + { + "refId": "A", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "expr": "sum(increase(cryptify_uploads_total{env=~\"$env\"}[$__range]))", + "instant": true, + "legendFormat": "messages" + } + ] + }, + { + "id": 3, + "type": "timeseries", + "title": "Messages sent per channel", + "description": "Channel comes from X-Cryptify-Source, then API auth, then Origin, then User-Agent. A channel with no traffic reports 0 rather than dropping out.", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "gridPos": { "h": 8, "w": 18, "x": 6, "y": 1 }, + "fieldConfig": { + "defaults": { + "unit": "short", + "custom": { "drawStyle": "line", "lineWidth": 1, "fillOpacity": 10, "showPoints": "auto" } + }, + "overrides": [] + }, + "options": { + "legend": { "displayMode": "table", "placement": "right", "calcs": ["sum"] }, + "tooltip": { "mode": "multi", "sort": "desc" } + }, + "targets": [ + { + "refId": "A", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "expr": "sum by (channel) (increase(cryptify_uploads_total{env=~\"$env\"}[$__rate_interval]))", + "legendFormat": "{{channel}}" + } + ] + }, + { + "id": 4, + "type": "bargauge", + "title": "Messages per channel (selected range)", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "gridPos": { "h": 8, "w": 12, "x": 0, "y": 9 }, + "fieldConfig": { + "defaults": { "unit": "short", "decimals": 0 }, + "overrides": [] + }, + "options": { + "reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }, + "displayMode": "basic", + "orientation": "horizontal", + "showUnfilled": true + }, + "targets": [ + { + "refId": "A", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "expr": "sum by (channel) (increase(cryptify_uploads_total{env=~\"$env\"}[$__range]))", + "instant": true, + "legendFormat": "{{channel}}" + } + ] + }, + { + "id": 5, + "type": "timeseries", + "title": "Bytes uploaded per channel", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "gridPos": { "h": 8, "w": 12, "x": 12, "y": 9 }, + "fieldConfig": { + "defaults": { + "unit": "bytes", + "custom": { "drawStyle": "line", "lineWidth": 1, "fillOpacity": 10, "showPoints": "auto" } + }, + "overrides": [] + }, + "options": { + "legend": { "displayMode": "table", "placement": "right", "calcs": ["sum"] }, + "tooltip": { "mode": "multi", "sort": "desc" } + }, + "targets": [ + { + "refId": "A", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "expr": "sum by (channel) (increase(cryptify_upload_bytes_total{env=~\"$env\"}[$__rate_interval]))", + "legendFormat": "{{channel}}" + } + ] + }, + { + "id": 6, + "type": "bargauge", + "title": "Messages per client app (selected range)", + "description": "From the app field of X-POSTGUARD-CLIENT-VERSION: pg4ol is the Outlook add-in, pg4tb the Thunderbird one. Clients that do not send the header land in unknown.", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "gridPos": { "h": 8, "w": 24, "x": 0, "y": 17 }, + "fieldConfig": { + "defaults": { "unit": "short", "decimals": 0 }, + "overrides": [] + }, + "options": { + "reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }, + "displayMode": "basic", + "orientation": "horizontal", + "showUnfilled": true + }, + "targets": [ + { + "refId": "A", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "expr": "sum by (app) (increase(cryptify_uploads_by_app_total{env=~\"$env\"}[$__range]))", + "instant": true, + "legendFormat": "{{app}}" + } + ] + }, + { + "id": 7, + "type": "row", + "title": "Storage", + "collapsed": false, + "gridPos": { "h": 1, "w": 24, "x": 0, "y": 25 }, + "panels": [] + }, + { + "id": 8, + "type": "stat", + "title": "Storage in use", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "gridPos": { "h": 8, "w": 6, "x": 0, "y": 26 }, + "fieldConfig": { + "defaults": { "unit": "bytes" }, + "overrides": [] + }, + "options": { + "reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }, + "colorMode": "none", + "graphMode": "area", + "textMode": "value_and_name" + }, + "targets": [ + { + "refId": "A", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "expr": "cryptify_storage_bytes{env=~\"$env\"}", + "instant": true, + "legendFormat": "{{env}}" + } + ] + }, + { + "id": 9, + "type": "timeseries", + "title": "Storage in use over time", + "description": "Sampled from data_dir by a background task every metrics_scan_interval_secs (60s by default), so it lags a burst of uploads by up to that interval.", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "gridPos": { "h": 8, "w": 18, "x": 6, "y": 26 }, + "fieldConfig": { + "defaults": { + "unit": "bytes", + "custom": { "drawStyle": "line", "lineWidth": 1, "fillOpacity": 10, "showPoints": "auto" } + }, + "overrides": [] + }, + "options": { + "legend": { "displayMode": "table", "placement": "right", "calcs": ["max"] }, + "tooltip": { "mode": "multi", "sort": "desc" } + }, + "targets": [ + { + "refId": "A", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "expr": "cryptify_storage_bytes{env=~\"$env\"}", + "legendFormat": "{{env}}" + } + ] + }, + { + "id": 10, + "type": "timeseries", + "title": "Files on disk", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "gridPos": { "h": 8, "w": 12, "x": 0, "y": 34 }, + "fieldConfig": { + "defaults": { + "unit": "short", + "decimals": 0, + "custom": { "drawStyle": "line", "lineWidth": 1, "fillOpacity": 10, "showPoints": "auto" } + }, + "overrides": [] + }, + "options": { + "legend": { "displayMode": "table", "placement": "right", "calcs": ["max"] }, + "tooltip": { "mode": "multi", "sort": "desc" } + }, + "targets": [ + { + "refId": "A", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "expr": "cryptify_active_files{env=~\"$env\"}", + "legendFormat": "{{env}}" + } + ] + }, + { + "id": 11, + "type": "timeseries", + "title": "Uploads expired before finalize", + "description": "Sessions purged after the idle timeout without ever being finalized. A rise here means senders are starting uploads they do not complete.", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "gridPos": { "h": 8, "w": 12, "x": 12, "y": 34 }, + "fieldConfig": { + "defaults": { + "unit": "short", + "custom": { "drawStyle": "bars", "lineWidth": 1, "fillOpacity": 60, "showPoints": "never" } + }, + "overrides": [] + }, + "options": { + "legend": { "displayMode": "table", "placement": "right", "calcs": ["sum"] }, + "tooltip": { "mode": "multi", "sort": "desc" } + }, + "targets": [ + { + "refId": "A", + "datasource": { "type": "prometheus", "uid": "${datasource}" }, + "expr": "sum by (env) (increase(cryptify_expired_files_total{env=~\"$env\"}[$__rate_interval]))", + "legendFormat": "{{env}}" + } + ] + } + ] +} diff --git a/cryptify/src/metrics.rs b/cryptify/src/metrics.rs index 6d44c81d..2c02aac6 100644 --- a/cryptify/src/metrics.rs +++ b/cryptify/src/metrics.rs @@ -366,6 +366,170 @@ pub async fn storage_sampler( } } +/// Pins the committed Grafana dashboard to what this module actually exports. +/// +/// The dashboard is a separate artefact from the exporter, so nothing but a +/// test stops the two drifting: renaming a metric here leaves panels querying +/// a name that no longer exists, and Grafana renders that as an empty graph +/// rather than an error. These tests fail instead. +#[cfg(test)] +mod dashboard_tests { + use super::*; + use std::collections::BTreeSet; + + use serde_json::Value; + + const DASHBOARD: &str = include_str!("../docs/grafana/cryptify-usage.json"); + + /// Label matcher every panel must carry so staging and production numbers + /// never end up summed into one series. + const ENV_SELECTOR: &str = "env=~\"$env\""; + + fn dashboard() -> Value { + serde_json::from_str(DASHBOARD).expect("cryptify-usage.json must be valid JSON") + } + + /// The metric names the exporter emits, read off the `# TYPE` lines of a + /// real render rather than from a hand-kept list. + fn exported_metrics() -> BTreeSet { + Metrics::new() + .render() + .lines() + .filter_map(|l| l.strip_prefix("# TYPE ")) + .filter_map(|rest| rest.split_whitespace().next()) + .map(str::to_string) + .collect() + } + + /// Every `expr` in the dashboard. Rows nest their panels one level deeper, + /// so this walks the whole tree instead of looping over `panels`. + fn panel_exprs(v: &Value, out: &mut Vec) { + match v { + Value::Object(map) => { + if let Some(Value::String(expr)) = map.get("expr") { + out.push(expr.clone()); + } + for child in map.values() { + panel_exprs(child, out); + } + } + Value::Array(items) => { + for child in items { + panel_exprs(child, out); + } + } + _ => {} + } + } + + fn all_exprs() -> Vec { + let mut out = Vec::new(); + panel_exprs(&dashboard(), &mut out); + assert!(!out.is_empty(), "dashboard has no queries at all"); + out + } + + /// The `query` of each dashboard variable, where it is a plain string. + /// The datasource variable's query is the string `prometheus`, which + /// carries no metric name and so contributes nothing. + fn variable_queries() -> Vec { + dashboard() + .get("templating") + .and_then(|t| t.get("list")) + .and_then(Value::as_array) + .map(|list| { + list.iter() + .filter_map(|v| v.get("query").and_then(Value::as_str)) + .map(str::to_string) + .collect() + }) + .unwrap_or_default() + } + + /// Pull `cryptify_*` identifiers out of a PromQL string. + fn cryptify_idents(promql: &str) -> BTreeSet { + let bytes = promql.as_bytes(); + let mut found = BTreeSet::new(); + let mut i = 0; + while let Some(rel) = promql[i..].find("cryptify_") { + let start = i + rel; + let mut end = start; + while end < bytes.len() { + let c = bytes[end] as char; + if c.is_ascii_alphanumeric() || c == '_' { + end += 1; + } else { + break; + } + } + found.insert(promql[start..end].to_string()); + i = end; + } + found + } + + fn referenced_metrics() -> BTreeSet { + all_exprs() + .iter() + .chain(variable_queries().iter()) + .flat_map(|q| cryptify_idents(q)) + .collect() + } + + #[test] + fn every_exported_metric_appears_on_the_dashboard() { + let referenced = referenced_metrics(); + for metric in exported_metrics() { + assert!( + referenced.contains(&metric), + "{metric} is exported but no dashboard panel queries it — \ + add a panel to cryptify/docs/grafana/cryptify-usage.json" + ); + } + } + + #[test] + fn every_metric_the_dashboard_queries_is_exported() { + let exported = exported_metrics(); + for metric in referenced_metrics() { + assert!( + exported.contains(&metric), + "the dashboard queries {metric}, which this module does not emit — \ + the panel would render empty" + ); + } + } + + #[test] + fn every_panel_filters_on_the_environment_variable() { + for expr in all_exprs() { + assert!( + expr.contains(ENV_SELECTOR), + "query is missing the {ENV_SELECTOR} matcher, so it mixes staging \ + and production into one number:\n{expr}" + ); + } + } + + #[test] + fn the_environment_variable_is_declared() { + let declared = dashboard() + .get("templating") + .and_then(|t| t.get("list")) + .and_then(Value::as_array) + .map(|list| { + list.iter() + .any(|v| v.get("name").and_then(Value::as_str) == Some("env")) + }) + .unwrap_or(false); + assert!( + declared, + "panels filter on $env but no `env` variable is declared, so every \ + panel resolves to an empty selector" + ); + } +} + #[cfg(test)] mod tests { use super::*; From 3cc508fdfde6a2f34b19ffa32197cadebfa6353a Mon Sep 17 00:00:00 2001 From: "dobby-yivi-agent[bot]" <275734547+dobby-yivi-agent[bot]@users.noreply.github.com> Date: Sat, 8 Aug 2026 13:37:58 +0000 Subject: [PATCH 2/4] fix(cryptify): make the dashboard guard rail cover the storage metrics The exported->dashboard test folded template-variable queries into the set it checked against, and the `env` variable is `label_values(cryptify_storage_bytes, env)`. So `cryptify_storage_bytes` certified itself: deleting both panels that display storage left all four tests green. That is the one metric #305 names explicitly, so the guard rail was missing exactly what it was built for. Split the panel-only set out and use it for that direction. The other direction keeps the combined set, since a variable querying a metric the exporter dropped is broken too. Also from review: - Panel 8 asked for a `graphMode: area` sparkline but ran an instant query, which returns one sample and draws nothing. - Panels 8, 9 and 10 selected the two unlabelled gauges raw while labelling the result `{{env}}`, so two targets sharing an env rendered as two identically-named series. They now aggregate with `max by (env)`, like panel 11 already did. - Reworded the env-matcher comment and assertion: a matcher filters, it does not stop panels 2, 4 and 6 aggregating `env` away. Refs #305 --- cryptify/docs/grafana/README.md | 7 +++++ cryptify/docs/grafana/cryptify-usage.json | 8 +++--- cryptify/src/metrics.rs | 31 +++++++++++++++++------ 3 files changed, 34 insertions(+), 12 deletions(-) diff --git a/cryptify/docs/grafana/README.md b/cryptify/docs/grafana/README.md index 4460eb45..00601fe8 100644 --- a/cryptify/docs/grafana/README.md +++ b/cryptify/docs/grafana/README.md @@ -57,6 +57,13 @@ restricted to the Prometheus segment at the firewall as well. Adjust the target hostnames to whatever the two deployments actually resolve to. The dashboard does not care about the hostnames, only that `env` is present. +The two gauges carry no labels of their own, so a raw select distinguishes series +only by `instance` and `job`. The storage and file-count panels therefore go +through `max by (env)`: with one target per environment it changes nothing, and +if an environment ever gets a second target sharing the volume it reports the +volume once instead of twice. Switch those three panels to `sum by (env)` if the +targets get separate volumes. + ## Importing Grafana, Dashboards, New, Import, upload `cryptify-usage.json`, pick the diff --git a/cryptify/docs/grafana/cryptify-usage.json b/cryptify/docs/grafana/cryptify-usage.json index 874c8dec..4c010ae2 100644 --- a/cryptify/docs/grafana/cryptify-usage.json +++ b/cryptify/docs/grafana/cryptify-usage.json @@ -208,8 +208,8 @@ { "refId": "A", "datasource": { "type": "prometheus", "uid": "${datasource}" }, - "expr": "cryptify_storage_bytes{env=~\"$env\"}", - "instant": true, + "expr": "max by (env) (cryptify_storage_bytes{env=~\"$env\"})", + "instant": false, "legendFormat": "{{env}}" } ] @@ -236,7 +236,7 @@ { "refId": "A", "datasource": { "type": "prometheus", "uid": "${datasource}" }, - "expr": "cryptify_storage_bytes{env=~\"$env\"}", + "expr": "max by (env) (cryptify_storage_bytes{env=~\"$env\"})", "legendFormat": "{{env}}" } ] @@ -263,7 +263,7 @@ { "refId": "A", "datasource": { "type": "prometheus", "uid": "${datasource}" }, - "expr": "cryptify_active_files{env=~\"$env\"}", + "expr": "max by (env) (cryptify_active_files{env=~\"$env\"})", "legendFormat": "{{env}}" } ] diff --git a/cryptify/src/metrics.rs b/cryptify/src/metrics.rs index 2c02aac6..172899ce 100644 --- a/cryptify/src/metrics.rs +++ b/cryptify/src/metrics.rs @@ -381,8 +381,10 @@ mod dashboard_tests { const DASHBOARD: &str = include_str!("../docs/grafana/cryptify-usage.json"); - /// Label matcher every panel must carry so staging and production numbers - /// never end up summed into one series. + /// Label matcher every panel must carry, so that every panel is scoped to + /// the environments selected in the `env` variable. Note that this only + /// filters: panels 2, 4 and 6 still aggregate `env` away, so with All + /// selected they report staging and production as one total. const ENV_SELECTOR: &str = "env=~\"$env\""; fn dashboard() -> Value { @@ -468,20 +470,33 @@ mod dashboard_tests { found } - fn referenced_metrics() -> BTreeSet { + /// Metrics named by an actual panel query. The "everything exported is on + /// the dashboard" direction must use this rather than `referenced_metrics`: + /// a metric mentioned only in a template variable's `query` is on no graph, + /// so it must not count as covered. + fn panel_metrics() -> BTreeSet { all_exprs() .iter() - .chain(variable_queries().iter()) .flat_map(|q| cryptify_idents(q)) .collect() } + /// Every `cryptify_*` name the dashboard mentions anywhere, panels and + /// template variables alike. Used for the other direction, where a variable + /// querying a metric the exporter dropped is just as broken. + fn referenced_metrics() -> BTreeSet { + panel_metrics() + .into_iter() + .chain(variable_queries().iter().flat_map(|q| cryptify_idents(q))) + .collect() + } + #[test] fn every_exported_metric_appears_on_the_dashboard() { - let referenced = referenced_metrics(); + let on_a_panel = panel_metrics(); for metric in exported_metrics() { assert!( - referenced.contains(&metric), + on_a_panel.contains(&metric), "{metric} is exported but no dashboard panel queries it — \ add a panel to cryptify/docs/grafana/cryptify-usage.json" ); @@ -505,8 +520,8 @@ mod dashboard_tests { for expr in all_exprs() { assert!( expr.contains(ENV_SELECTOR), - "query is missing the {ENV_SELECTOR} matcher, so it mixes staging \ - and production into one number:\n{expr}" + "query is missing the {ENV_SELECTOR} matcher, so it ignores the \ + environment the dashboard is scoped to:\n{expr}" ); } } From 53493e5551a313fe1709660af63c4ea7a5e1cc5d Mon Sep 17 00:00:00 2001 From: "dobby-yivi-agent[bot]" <275734547+dobby-yivi-agent[bot]@users.noreply.github.com> Date: Sat, 8 Aug 2026 13:40:46 +0000 Subject: [PATCH 3/4] docs: record that gh pr edit fails silently on this repo Updating the PR body with `gh pr edit --body-file` exits 0, prints what looks like a deprecation warning about projects classic, and does not apply the edit. Cost a cycle here; REST works. --- CLAUDE.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CLAUDE.md b/CLAUDE.md index 8005b8b1..62679e4f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -19,6 +19,7 @@ Migrated from the dobby memory repo (`encryption4all/dobby`). This file is the h - The oasdiff gate's settings are **not** self-evident and `--fail-on ERR` alone fails open. `fail-on: WARN` is deliberate: oasdiff rates removing or renaming an *optional* response property, and removing a request parameter, as WARN, and this spec marks only `status` as `required`, so at ERR the gate silently passed a removed `key` (the IBE user secret key `/v2/request/key` exists to return), a renamed `proofStatus` and a dropped `timestamp` parameter. Two more, a changed non-success status (401 to 403) and a dropped response enum value, rate ERR but are **opt-in**, so they only run when named in `include-checks`. Of the 31 WARN checks, `response-property-enum-value-added` is the only one that fires on something `COMPATIBILITY.md` does not already forbid, which is why that document now names a new response enum value as non-additive too. Reproduce a verdict with the exact flags the action's entrypoint builds (`oasdiff/oasdiff-action/breaking@v0.1.10` is `FROM tufin/oasdiff:v1.26.1`, so the pinned tag is what makes a local run authoritative): `git show origin/main:pg-pkg/api-description.yaml > /tmp/base.yaml && oasdiff breaking /tmp/base.yaml pg-pkg/api-description.yaml --allow-external-refs=false --composed=false --fail-on WARN --include-checks response-non-success-status-removed,response-property-enum-value-removed`. Two traps: `oasdiff --version` prints `oasdiff version main` after a `go install` of a tag because the version comes from release ldflags (the code is still the tag), and `--fail-on` takes `ERR`/`WARN` while `oasdiff checks --severity` takes `error`/`warn`/`info`. `--severity ERR` is a usage error, so `oasdiff checks --severity ERR | wc -l` counts the help text instead and badly undercounts the tier (it is 213 error checks, 31 warn, 265 info). The spec has no external `$ref`s, so `allow-external-refs` stays at its safe (SSRF-guarding) default. The gate only sees paths the spec documents, and the spec documents canonical paths only, so dropping the `/v2/irma/...` alias handlers (#257) passes it. - **Importing another repo's history here silently closes issues here. Two vectors fire it, each sufficient on its own, so rule out neither.** Imported commits carry their original messages verbatim, closing keywords included, and GitHub resolves those against the **destination** repo's numbering. **Vector 1, the squash body.** This repo's `squash_merge_commit_message` is `COMMIT_MESSAGES` (the same setting the release-plz bullet above turns on its head — there it is why a `BREAKING CHANGE:` footer in a PR body never reaches the commit), so squash-merging the import PR concatenates every imported commit message into the merge commit's body. `ba380a1`'s body is 1678 lines and carries all 17 refs (`Closes #38 #45 #52 #54 #123 #125 #134 #142 #146 #153 #155 #157 #159 #167 #186 #191 #194`); 16 pointed at numbers already closed here, and one — #146, a live unimplemented feature request — was closed four seconds after it landed, with nothing warning, and stood four days before being found and reopened. Note what this means: a plain squash merge is enough by itself, *because* squashing concatenates the messages even as it throws the history away. **Vector 2, the history itself.** Here GitHub never reads the merge commit's message — it attributes the closes to the **imported commits**, once they become reachable from the default branch. This vector went untested in this repo rather than disproven: `9887e1a` carries no keywords of its own, and by the time it landed thirteen minutes later all 17 targets were already closed, and GitHub does not re-close a closed issue. It is measured in encryption4all/postguard-js#139, which is also the guard: import PR #137 merged with a real two-parent merge commit (`b1bb2ee`, a three-line message, no keywords, no squash body anywhere), and js#128/#129 closed six seconds later attributed to imported commits `a0ce27f`/`2acf42f` — single-parent, committed 2026-06-04 — then reopened fifteen minutes later. **So the audit is the only step that covers both.** Before merging *and* after, from the import branch: `git log origin/main..HEAD --pretty=%B | grep -oiE '(close[sd]?|fix(e[sd])?|resolve[sd]?) +#[0-9]+' | sort -u`, then check each number here. Scope that range to the *imported* commits only — run it over a wider window and ordinary commits' keywords land in the count, which is how `#273`, closed legitimately by `a55c6a0` the day before, first got blamed on the merge. Editing the squash body in the merge dialog defuses vector 1 and only vector 1; on vector 2 there is no body to edit, and rewriting the keywords out of the imported commits costs every imported SHA (`git filter-repo`). Do **not** reach for `commit_message` on `PUT /pulls/N/merge` as the scripted version of that: the REST reference words it "Extra detail to append to automatic commit message" and documents nothing about `merge_method=squash`, so whether it replaces the concatenated body or appends to it is unverified — and if it appends, the defusal silently no-ops and every keyword still fires. Confirm it on a throwaway repo and record the answer here before relying on it. Reading close *state* cannot tell you which happened, so read the close **event** — and read it correctly, because the obvious rule is wrong. A keyword in a *commit message* is attributed to the commit and carries a `commit_id`; a keyword in the *PR body* is attributed to the PR and carries `commit_id: null`. #146 shows `ba380a1464…` because PR #277's body said only `Closes #255`, while #273 shows `null` because PR #274's body said `Closes #273`. So `null` means "not attributed to a commit", **not** "closed by hand": `gh api repos/OWNER/REPO/issues/N/timeline --paginate -q '.[] | select(.event=="closed") | .created_at+" "+(.commit_id//"-")'`. Same failure class as the unapplied-workflow half below (#272). - The `dobby-coder` GitHub App lacks `workflows: write` on this repo; any push touching `.github/workflows/*.yml` is rejected at the remote. Before treating a fix as blocked, check whether the same effect can be achieved in a pushable file (crate manifest, source, committed script); if a fix genuinely can only live in a workflow file, ship the pushable half and hand the maintainer ready-to-paste YAML in the PR body. The block covers *merge* commits too, which is easy to miss: once a branch carries its own `build.yml` change (typically a maintainer applying such a patch onto it), a later `git merge origin/main` that has to touch `build.yml` produces a commit updating a workflow file, and the push is rejected even when the resolution is only "keep both new jobs". Nothing can be split out of a merge commit, so that merge has to be landed by a maintainer, or the App needs `workflows: write`. Measured exception, worth trying before handing the sync over: the App pushed `ce0fc59` on this branch, a merge whose diff against its first parent added main's 64 new `build.yml` lines. That merge needed no resolution inside `build.yml` — it took main's side whole, so the blob it committed already existed in the repo. Try the merge and read the remote's answer; only escalate on an actual rejection. +- `gh pr edit` does not work on this repo and **fails in the direction that looks like success**: a classic project is attached, so the mutation `gh` sends requests `repository.pullRequest.projectCards` and the whole call dies with `GraphQL: Projects (classic) is being deprecated ... (repository.pullRequest.projectCards)`. That message reads as a deprecation warning, `gh` still exits 0, and the edit silently does not land — so a `--body-file` update looks applied and isn't. Use REST instead, which touches no project fields: `gh api -X PATCH repos/encryption4all/postguard/pulls/N -f body="$(cat body.md)"`. Verify by reading the body back (`gh pr view N --json body`), because the failure is invisible otherwise. Same trap for `--title`/`--add-label` through `gh pr edit`. - **Consolidating a repo into a monorepo means transfer its open issues first, then archive — not archive-then-orphan.** `postguard-website`, `postguard-outlook-addon`, `postguard-tb-addon` and `postguard-examples` were archived (read-only) after folding into `postguard-js`, and the "open issues transfer here" step was silently skipped on all four: 62 open issues sat stranded, unworkable (`HTTP 403: Repository was archived so is read-only` on label/assign/comment/close). GitHub will not transfer an issue out of an already-archived repo, so recovering from the skip costs an unarchive → transfer → re-archive round trip per repo instead of a single transfer before archiving (decided in postguard#282). Do the transfer as part of the same change that archives the repo, and leave a "development moved to ``, see ``" banner at the top of the archived repo's README before re-archiving, so an old link still finds the new home. ## Dependencies From 3a88f5450ff6e43a625053448ae448526e041923 Mon Sep 17 00:00:00 2001 From: "dobby-yivi-agent[bot]" <275734547+dobby-yivi-agent[bot]@users.noreply.github.com> Date: Sat, 8 Aug 2026 14:03:22 +0000 Subject: [PATCH 4/4] docs(cryptify): name every env-aggregating panel in the ENV_SELECTOR caveat Panels 3 and 5 are `sum by (channel) (...)`, which drops `env` just as surely as panel 2's bare `sum(...)` or panel 6's `by (app)`. The caveat listed only 2, 4 and 6, implying 3 and 5 were safe. Panels 2 through 6 are the whole "Messages sent" row, so the range says it exactly. --- cryptify/src/metrics.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cryptify/src/metrics.rs b/cryptify/src/metrics.rs index 172899ce..2c320aff 100644 --- a/cryptify/src/metrics.rs +++ b/cryptify/src/metrics.rs @@ -383,7 +383,7 @@ mod dashboard_tests { /// Label matcher every panel must carry, so that every panel is scoped to /// the environments selected in the `env` variable. Note that this only - /// filters: panels 2, 4 and 6 still aggregate `env` away, so with All + /// filters: panels 2 through 6 all aggregate `env` away, so with All /// selected they report staging and production as one total. const ENV_SELECTOR: &str = "env=~\"$env\"";