Hash PII instead of leaking it (Rust SDK)

[brentrager]

Problem

rust/observability/src/pii.rs's scrub_string handled credentials only — Bearer, password=, token/api_key/secret=, sk-… — while the module doc claimed PII scrubbing. An email, phone or address in a message, breadcrumb, header value, or GenAI tool argument reached the wire intact. The sibling repo's rust/chat-ws/src/traced_tool.rs has a test whose tool returns "did the thing; email=a@b.com" and asserts it passes straight through.

That is latent only because chat-ws telemetry never exports today. The export fix is the next phase, so the scrubber goes first — the first rows that ever land are already clean, and there is nothing to backfill or purge later.

Solution — hash, don't drop

[redacted] destroys the one question worth asking of this data: are these two spans the same person? Emails, phones and street addresses now become a keyed token instead:

a@b.com   ->  [email:9f2a41c8]
555-0142  ->  [phone:3b7e0d92]

• HMAC-SHA256, not a bare digest. Emails and phone numbers are a small enumerable space; an unkeyed digest is rainbow-tabled in seconds.
• Per-org salt — the org id is mixed into the HMAC message, so identical PII hashes differently in different orgs. No cross-tenant correlation, matching the ClickHouse org-isolation posture.
• Type prefix stays visible. You can see what kind of value was there, and that two spans carry the same one, without ever seeing it.
• 8 hex chars — readable in a span attribute, collisions acceptable for correlation.
• No key configured → fully redacted ([email:redacted]), never hashed under a guessable one. Fail safe, not open.
• Credentials stay DROPPED, never hashed. A hash of a live token is still a token oracle. Credential patterns run FIRST, so a personal identifier inside a secret (token=a@b.com) goes with the secret rather than surviving as a hash.

Phones normalize to digits and emails to lowercase before hashing, so (415) 555-0142 and 415-555-0142 correlate. The phone pattern requires a separator, so ids, versions, ISO timestamps and amounts are not eaten — there is a test pinning that.

API

scrub_string / scrub_headers	unchanged signatures. Now scrub personal identifiers too, under the empty org salt — strictly more redaction, no caller breakage. The four client.rs call sites have no org in hand.
scrub_string_for_org(&str, org_id)	new — the org-salted form.
scrub_headers_for_org(&map, org_id)	new.
pii_token(PiiKind, raw, org_id)	new — the searchability seam. Hash a typed query term the same way and match the stored token.
set_pii_hash_key(key)	new — direct injection. Set-once.
BootstrapEnv::pii_hash_key	new — SMOOAI_OBSERVABILITY_PII_HASH_KEY.

The key arrives the way every other setting in this crate does: an env var read by bootstrap(), which is this crate's config seam (it deliberately has no Smoo-internal coupling). It is set-once — a mid-process rotation would silently fork every correlation — and the docs say rotate never, for the same reason.

One new dependency: hmac-sha256 (zero-dep, single-purpose) rather than the hmac + sha2 + digest trait stack, since this crate needs exactly one primitive.

Verification

87 tests pass, clippy -D warnings clean, cargo fmt --check clean, pnpm format:check (oxfmt) clean.

Every guard is mutation-checked — removed, confirmed a specific named test fails, restored:

Mutation	Test that caught it
email pattern removed	hashes_emails_keeping_the_type_prefix, tool_result_shape_from_traced_tool_is_scrubbed (+5)
phone pattern removed	hashes_phone_numbers (+2)
address pattern removed	hashes_street_addresses
per-org salt removed from HMAC message	same_value_different_org_hashes_differently
credential drop removed	credentials_are_dropped_not_hashed, scrubs_bearer_tokens, scrubs_passwords, scrubs_sk_keys (+4)
keying removed (constant key)	different_key_hashes_differently
no-key fallback hashes instead of redacting	no_key_redacts_rather_than_hashing

Tests drive the private key-taking core rather than installing the process-wide key: OnceCell is set-once and cargo runs the suite in one process, so a global write would make it racy.

Also in here

Fixes a pre-existing failure on main, unrelated to this change but in the way of a green Rust lane: otel_capture's test counted every span in the in-memory exporter, but the tracer provider is process-wide, so it collected the whole suite's spans (8, not 2). Confirmed failing on a clean origin/main tree with this work stashed; passes in isolation. It now counts only the observability.capture_* spans it produces — a double-report still fails it, which is what the assertion was for.

Follow-ups (not in this PR)

• traced_tool.rs (smooai monorepo, rust/chat-ws/) should call scrub_string_for_org with the conversation's org id, and its leaky test assertion should be inverted. Today it gets the empty-salt behaviour by default — redacted/hashed, but not org-isolated.
• The other four SDKs (TS/Go/Python/.NET) still scrub credentials only. The parity corpus does not cover PII, so nothing gates this; the README parity table now says so explicitly.
• The operator's second redactor — smooth-operator/rust/smooth-operator/src/telemetry.rs::redact_tool_arguments, key-name-based JSON walking with a 2048-byte cap — is a different algorithm on the same data class. Converge or document.

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: 111161b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@smooai/observability	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR