Hash PII instead of leaking it (Rust SDK)

.changeset/rust-pii-hashing.md

@@ -0,0 +1,25 @@
+---
+'@smooai/observability': minor
+---
+
+Rust: PII is now hashed rather than passed through. `pii::scrub_string` handled
+credentials only — `Bearer`, `password=`, `token`/`api_key`/`secret=`, `sk-…` —
+while the module doc claimed PII scrubbing, so an email or phone in a message,
+breadcrumb or GenAI tool argument reached the wire intact.
+
+Emails, phone numbers and street addresses are now detected and replaced with a
+keyed token: `a@b.com` → `[email:9f2a41c8]`. HMAC-SHA256, not a bare digest —
+those values are a small enumerable space a rainbow table reverses in seconds —
+and the org id is mixed into the message so identical PII hashes differently in
+different orgs. The type prefix stays visible, which keeps "are these two spans
+the same person?" answerable while storing nothing reversible.
+
+Credentials are still **dropped**, never hashed: a hash of a live token is a
+token oracle. With no key configured (`SMOOAI_OBSERVABILITY_PII_HASH_KEY`, or
+`pii::set_pii_hash_key`), personal identifiers are fully redacted rather than
+hashed under a guessable key.
+
+New: `pii::scrub_string_for_org`, `pii::scrub_headers_for_org`, `pii::pii_token`,
+`pii::PiiKind`, `pii::set_pii_hash_key`, `BootstrapEnv::pii_hash_key`.
+`scrub_string` / `scrub_headers` keep their signatures and now scrub personal
+identifiers too, under the empty org salt.

rust/observability/Cargo.toml

@@ -31,6 +31,10 @@ uuid = { version = "1", features = ["v4"] }
 backtrace = "0.3"
 regex = "1"
 once_cell = "1"
+# Keyed PII hashing (`pii.rs`). Single-purpose, zero-dependency HMAC-SHA256 —
+# chosen over the `hmac` + `sha2` + `digest` trait stack because this crate needs
+# exactly one primitive and nothing generic over it.
+hmac-sha256 = "1"
 async-trait = "0.1"
 http = "1"
 bytes = "1"

rust/observability/README.md

@@ -18,7 +18,8 @@ error-safe and degrades to a no-op (plus one stderr line) rather than panicking.
 | Stack capture (`backtrace`)        | `stack-parser.ts` (string parse)   | ✅   |
 | Scope / context (per-task)         | `scope.ts`                         | ✅   |
 | Breadcrumb buffer (max 100)        | `scope.ts`                         | ✅   |
-| PII scrubbing                      | `pii.ts`                           | ✅   |
+| PII scrubbing (credentials)        | `pii.ts`                           | ✅   |
+| PII **hashing** (email/phone/addr) | — (Rust only)                      | ✅   |
 | Batched webhook transport + retry  | `transport.ts`                     | ✅   |
 | OTLP traces + metrics export       | `otel/setup-otel-sdk.ts`           | ✅   |
 | Per-request M2M auth (no staleness)| `otel/auth-injecting-exporter.ts`  | ✅   |
@@ -63,6 +64,37 @@ walkthrough (scope, error capture with cause chains, `with_scope`, metrics).
   re-mints on 401, so a rotated token is picked up on the next export with no
   exporter restart (the Rust analogue of the TS SMOODEV-1206 fix).
 
+## PII scrubbing
+
+Two classes, handled differently:
+
+- **Credentials** (`Bearer …`, `password=`, `token`/`api_key`/`secret=`, `sk-…`)
+  are **dropped**. A hash of a live token is still a token oracle.
+- **Personal identifiers** (email, phone, street address) are **hashed**:
+  `a@b.com` → `[email:9f2a41c8]`. The type prefix stays visible, so you can see
+  *what kind* of value was there and that two spans carry the *same* one —
+  without ever seeing it.
+
+The hash is **HMAC-SHA256**, not a bare digest (emails and phones are a small
+enumerable space a rainbow table reverses in seconds), and the org id is mixed
+into the message so the same value hashes **differently in different orgs**.
+
+```rust
+use smooai_observability::pii::{scrub_string_for_org, pii_token, PiiKind};
+
+let scrubbed = scrub_string_for_org("mail a@b.com", org_id);   // "mail [email:9f2a41c8]"
+// Search: hash the typed term the same way and match the stored token.
+let needle = pii_token(PiiKind::Email, "A@B.com", org_id);
+```
+
+Set the key with `SMOOAI_OBSERVABILITY_PII_HASH_KEY` (read by `bootstrap()`) or
+`pii::set_pii_hash_key`. **With no key, personal identifiers are fully redacted**
+(`[email:redacted]`) rather than hashed under a guessable one.
+
+⚠️ **The key and the org id are load-bearing.** Rotating either silently breaks
+correlation with every hash already stored — treat the key as permanent, and do
+not reuse a secret that rotates on a schedule.
+
 ## GenAI spans
 
 ```rust

rust/observability/src/bootstrap.rs

@@ -20,6 +20,10 @@
 //! - `SMOOAI_OBSERVABILITY_ENVIRONMENT` — default `STAGE` / `unknown`.
 //! - `SMOOAI_OBSERVABILITY_RELEASE` — default `GIT_SHA` / `dev`.
 //! - `SMOOAI_OBSERVABILITY_DISABLED` — `1`/`true` skips bootstrap entirely.
+//! - `SMOOAI_OBSERVABILITY_PII_HASH_KEY` — HMAC key used to hash emails /
+//!   phones / addresses in scrubbed strings (see [`crate::pii`]). Unset means
+//!   those values are fully redacted instead of hashed. **Rotating it breaks
+//!   correlation with every hash already stored** — treat it as permanent.
 //!
 //! Never panics: missing config / init errors are logged to stderr and the SDK
 //! degrades gracefully. Idempotent: a second call returns the same handle.
@@ -46,6 +50,7 @@ pub struct BootstrapEnv {
     pub environment: Option<String>,
     pub release: Option<String>,
     pub disabled: bool,
+    pub pii_hash_key: Option<String>,
 }
 
 impl BootstrapEnv {
@@ -69,6 +74,7 @@ impl BootstrapEnv {
                 .ok()
                 .or_else(|| env::var("GIT_SHA").ok()),
             disabled: truthy(env::var("SMOOAI_OBSERVABILITY_DISABLED").ok().as_deref()),
+            pii_hash_key: env::var("SMOOAI_OBSERVABILITY_PII_HASH_KEY").ok(),
         }
     }
 }
@@ -105,6 +111,13 @@ pub async fn bootstrap_with(env: BootstrapEnv) -> BootstrapResult {
 }
 
 async fn build(env: BootstrapEnv) -> BootstrapResult {
+    // Before anything can emit: a scrubbed string written without this key
+    // redacts PII outright, so installing it late would silently produce a
+    // window of uncorrelatable spans rather than an error.
+    if let Some(key) = &env.pii_hash_key {
+        crate::pii::set_pii_hash_key(key.as_bytes());
+    }
+
     let service_name = env
         .service_name
         .clone()

rust/observability/src/otel_capture.rs

@@ -212,7 +212,17 @@ mod tests {
         client.capture_message("just fyi", Level::Info);
 
         provider.force_flush().ok();
-        let spans = exporter.get_finished_spans().expect("exporter readable");
+        // Filter to the spans THIS test produced: the provider is process-wide,
+        // so every other test in the binary that opens a span exports into the
+        // same in-memory exporter. Counting everything made this assertion fail
+        // as the suite grew (8 spans, not 2) while still catching what it is
+        // for — a double-report shows up as three `capture_` spans, not two.
+        let spans: Vec<_> = exporter
+            .get_finished_spans()
+            .expect("exporter readable")
+            .into_iter()
+            .filter(|s| s.name.starts_with("observability.capture_"))
+            .collect();
         assert_eq!(spans.len(), 2, "one synthetic span per capture");
 
         let exception_span = spans