diff --git a/.env.example b/.env.example index 433af92..d6b8c52 100644 --- a/.env.example +++ b/.env.example @@ -63,3 +63,9 @@ LIFECYCLE_SWEEP_INTERVAL_MS=0 SMS_PROVIDER=mock RATE_LIMIT_OTP_WINDOW_MS=900000 RATE_LIMIT_OTP_MAX=5 + +# Data Lifecycle / Audit Configuration — see docs/DATA_LIFECYCLE.md +# HMAC key for the source-IP hash on immutable audit events. Rotating it makes +# older hashes uncorrelatable with newer ones. If unset in production, audit +# events omit the IP hash rather than storing an unkeyed (reversible) digest. +# AUDIT_IP_HASH_SECRET=change-me-in-production diff --git a/docs/DATA_LIFECYCLE.md b/docs/DATA_LIFECYCLE.md new file mode 100644 index 0000000..9298217 --- /dev/null +++ b/docs/DATA_LIFECYCLE.md @@ -0,0 +1,392 @@ +# Data Lifecycle and Archive Policy + +How every persisted record in Learnault is classified, how long it is kept, what +happens to it when a subject asks to be erased, and which changes must leave an +audit trail. + +The machine-readable source of truth is +[`src/audit/classification.ts`](../src/audit/classification.ts). This document +explains it. `tests/audit/classification.test.ts` fails if a model exists in +`prisma/schema.prisma` without a rule, so the two cannot drift apart. + +--- + +## 1. Record classes + +Every record is exactly one of four classes. + +| Class | Meaning | Deletion | +| --- | --- | --- | +| **MUTABLE** | Updated in place. Where the history matters, it lives in audit events, not in the row. | Purged when retention expires | +| **ARCHIVABLE** | Withdrawn by stamping `archivedAt` rather than deleted, because something else still depends on it. Excluded from reads by default. | Purged some time after being archived | +| **DELETABLE** | Safe to hard-delete. Nothing else depends on it. | Deleted on expiry or erasure | +| **IMMUTABLE** | Append-only. Never updated. Deleted only by a retention purge, if at all. | Purge only | + +The distinction that matters most in practice is **archivable vs deletable**. A +record is archivable when deleting it would strand another record: a `Completion` +needs the `Module` a learner actually took, and a `Referral` needs the +`ReferralCode` that created it. Deleting the parent would either cascade away +real history or leave a dangling reference. Archiving keeps the referent and +hides the row. + +--- + +## 2. The lifecycle matrix + +`Retention` is counted from the anchor column. "Indefinite" means nothing purges +it. `Audited` means mutations must go through +[`auditedMutation`](../src/audit/audited-mutation.ts). + +### Identity + +| Model | Class | Retention (anchor) | On erasure | Audited | +| --- | --- | --- | --- | --- | +| `User` | MUTABLE | Indefinite | **Anonymize** | Yes | +| `LearnerPreference` | MUTABLE | Indefinite | Cascade | Yes | +| `LearnerProfile` | ARCHIVABLE | 365d (`archivedAt`) | Cascade | Yes | +| `OnboardingProgress` | MUTABLE | Indefinite | Cascade | No | +| `NotificationPreference` | MUTABLE | Indefinite | Cascade | No | +| `DataExportRequest` | DELETABLE | **7d** (`completedAt`) | Delete | Yes | +| `AccountDeletionRequest` | MUTABLE | 7y (`createdAt`) | **Retain** | Yes | + +`User` is anonymized rather than deleted. Money and credential rows outlive the +account (see below), and they need a valid referent — so the row survives as a +tombstone with every identifying column overwritten. + +`AccountDeletionRequest` is retained *past the erasure it triggers*: it is the +evidence the request was honoured. That is only acceptable because it holds no +personal data beyond the user id. + +`DataExportRequest` has the shortest retention of anything in the schema, because +its `artifact` column is a full-fidelity dump of one person's data — the +highest-value single row in the database. + +### Money + +| Model | Class | Retention (anchor) | On erasure | Audited | +| --- | --- | --- | --- | --- | +| `Transaction` | IMMUTABLE | 7y (`createdAt`) | Retain | Yes | +| `Wallet` | MUTABLE | Indefinite | Retain | Yes | +| `StellarFunding` | MUTABLE | 7y (`createdAt`) | Retain | Yes | +| `Referral` | IMMUTABLE | 7y (`createdAt`) | Retain | Yes | +| `ReferralCode` | ARCHIVABLE | 365d (`archivedAt`) | Cascade | Yes | + +A ledger a subject can erase is not a ledger. Money rows survive erasure, which +is only defensible because they carry no personal data in-row — they reference a +user id whose row has been anonymized. `StellarFunding` is keyed by Stellar +public key rather than by user, so erasure severs the link by nulling +`User.walletAddress` without touching the funding row. + +Corrections to `Transaction` are booked as new reversing entries. Nothing edits a +ledger row. + +### Credentials + +| Model | Class | Retention | On erasure | Audited | +| --- | --- | --- | --- | --- | +| `Credential` | IMMUTABLE | Indefinite | Retain | Yes | +| `Completion` | IMMUTABLE | Indefinite | **Delete** | Yes | + +`Credential` is verifiable by third parties against the chain, so issuance is +permanent; revocation appends a revocation record rather than editing the row. + +`Completion` is the one place these two diverge. It is immutable while the +account lives, but **deleted** on erasure, because a completion reveals what a +named learner studied — and that is personal data in a way an issued credential +identifier is not. + +### Security + +| Model | Class | Retention (anchor) | On erasure | Audited | +| --- | --- | --- | --- | --- | +| `AuditEvent` | IMMUTABLE | 7y (`occurredAt`) | Retain | n/a | +| `AuditLog` *(legacy)* | IMMUTABLE | 2y (`createdAt`) | Anonymize | n/a | +| `Session` | MUTABLE | 90d (`updatedAt`) | Delete | Yes | +| `RefreshToken` | MUTABLE | 90d (`updatedAt`) | Cascade | Yes | +| `VerificationToken` | MUTABLE | 30d (`createdAt`) | Delete | Yes | +| `OtpChallenge` | MUTABLE | 30d (`createdAt`) | Delete | Yes | +| `ManagedKeyReference` | IMMUTABLE | Indefinite | Retain | Yes | + +Indefinite retention of security data is the failure mode this category exists to +prevent, so everything here is bounded — with one exception. +`ManagedKeyReference` is retained forever because it is an opaque KMS handle +(never key material), and destroying the handle orphans any funds held under that +key. + +Session revocation is a status change, not an archive: a revoked session must +stay visible to the learner in device management until it is purged. Consumed +`RefreshToken` rows are kept for the same window so that replaying an +already-rotated token is still detectable as theft. + +### Consent + +| Model | Class | Retention (anchor) | On erasure | Audited | +| --- | --- | --- | --- | --- | +| `ConsentRecord` | IMMUTABLE | 7y (`createdAt`) | Retain | Yes | +| `PreferenceAuditLog` | IMMUTABLE | 7y (`createdAt`) | Retain | n/a | + +Proof of consent must outlive the account it describes — otherwise a withdrawal +cannot be demonstrated after the fact. Withdrawal appends a new row; it never +edits the granting one. + +### Content + +| Model | Class | Retention (anchor) | On erasure | Audited | +| --- | --- | --- | --- | --- | +| `Module` | ARCHIVABLE | Indefinite (`archivedAt`) | Retain | Yes | +| `Avatar` | ARCHIVABLE | 365d (`archivedAt`) | Cascade | Yes | +| `AvatarVariant` | IMMUTABLE | 365d (`createdAt`) | Cascade | No | + +`Module` is archived and never purged: completions and credentials reference the +module a learner actually took, so withdrawing content archives it permanently +rather than removing it. + +### Operational + +| Model | Class | Retention (anchor) | On erasure | Audited | +| --- | --- | --- | --- | --- | +| `WebhookEndpoint` | ARCHIVABLE | 365d (`archivedAt`) | Retain | Yes | +| `WebhookDelivery` | MUTABLE | 30d (`createdAt`) | Retain | No | +| `EmailDelivery` | MUTABLE | 30d (`createdAt`) | Delete | No | +| `NotificationLog` | MUTABLE | 30d (`createdAt`) | Delete | No | +| `DeviceToken` | DELETABLE | 90d (`updatedAt`) | Delete | No | +| `SyncEvent` | IMMUTABLE | 90d (`createdAt`) | Delete | No | +| `OutboxEvent` | MUTABLE | 30d (`createdAt`) | Retain | No | +| `JobAttempt` | MUTABLE | 30d (`createdAt`) | Cascade | No | +| `RolledBackRecord` | IMMUTABLE | 30d (`createdAt`) | Retain | No | +| `WalletProvisioningJob` | MUTABLE | 90d (`updatedAt`) | Cascade | No | + +`EmailDelivery` and `NotificationLog` hold rendered message bodies, which is +personal data — hence the short window and hard deletion on erasure. +`DeviceToken` is deleted rather than archived: an archived push token would still +be a live address. + +--- + +## 3. Immutable audit events + +`audit_events` is the audit spine. One row records **who** did **what** to +**which record**, **why**, and under **which request**. + +| Column | Purpose | +| --- | --- | +| `actorType`, `actorId`, `actorRole` | Who acted. `USER`, `ADMIN`, `SYSTEM`, `WORKER`, `ANONYMOUS`. Role as held *at the time*. | +| `action` | Dotted name, e.g. `account.deactivated` | +| `targetType`, `targetId` | Which record changed | +| `recordClass` | Lifecycle class of the target, from the matrix | +| `reason` | Justification. Required for `ADMIN` actors | +| `requestId`, `correlationId`, `source` | Correlation to request logs, outbox events, and the code path | +| `metadata` | Redacted JSON context | +| `actorIpHash` | Keyed HMAC of the request IP — never the address | +| `userAgentFamily` | Coarse family (`Chrome`, `Android`) — never the raw UA | +| `occurredAt` | When | + +Separating **actor** from **target** is the point: "an admin deactivated a +learner" is a materially different event from "a learner deactivated themselves", +and a single `userId` column cannot express the difference. + +### Immutability is enforced by the database + +Triggers in +[the migration](../prisma/migrations/20260824090000_auditable_data_lifecycle/migration.sql), +not by convention: + +- **`UPDATE` is rejected unconditionally.** There is no escape hatch. +- **`DELETE` is rejected** unless the transaction has set + `learnault.audit_purge`, which only `AuditEventService.purgeExpired()` does — + and it deletes by timestamp, so the hatch cannot target one inconvenient event. +- **`TRUNCATE` is rejected** by a separate statement-level trigger, because + `TRUNCATE` bypasses row-level triggers entirely. + +`SET LOCAL` (not `SET`) scopes the purge permission to a single transaction, so +it cannot leak to a later request that picks up the same pooled connection. + +### No foreign key to `User` — deliberately + +A relation would need either `onDelete: Cascade`, letting erasure destroy the +trail, or `onDelete: SetNull`, mutating a row that must never change. `actorId` +and `targetId` are soft references instead. + +### Erasure-safe by construction + +Nothing in `audit_events` needs scrubbing when a subject is erased, because +nothing identifying is written in the first place. This is the resolution of the +central tension in the policy: **immutability and the right to erasure are only +compatible if the immutable store holds no personal data.** + +The legacy `audit_logs` table predates this and does hold a raw IP and +User-Agent, so it is scrubbed on erasure and retained for two years rather than +seven. New code writes `audit_events`. + +--- + +## 4. Redaction + +Because audit rows cannot be scrubbed later, metadata is filtered on the way +*in*, by [`src/audit/redaction.ts`](../src/audit/redaction.ts). Two independent +passes run over every value: + +1. **Key matching** — a field named `password`, `refreshToken`, `email`, … is + replaced regardless of content. +2. **Value matching** — a value shaped like a Stellar seed, a Stellar public key, + a JWT, a bearer credential, an email address, an E.164 number, an IPv4 + address, a 40+ character hex blob, or a PEM header is replaced *even under an + innocuous key*, because callers nest secrets in unexpected places. + +Structural caps then bound the whole object: depth 4, 20 array entries, 32 keys, +256 characters per string, 4 KB serialized. + +Over-redaction is treated as its own failure. `statusCode`, `failureCode`, +`referralCode`, `amountStroops` and `requestId` are all allowed, because an audit +trail nobody can read is not reviewable. Notably `to` is *allowed*: it is the +obvious name for an email recipient, but also the standard name for the +destination of a status transition — and the value-level email pattern catches an +actual recipient anyway. + +When anything is replaced, the row records a `_redacted` array of the paths. +A reader can see that redaction happened rather than guessing. + +### IP hashing + +`actorIpHash` is an **HMAC**, not a bare digest: the IPv4 space is small enough to +enumerate, so an unkeyed hash is reversible in seconds. The key is +`AUDIT_IP_HASH_SECRET`. Rotating it makes older hashes uncorrelatable with newer +ones, which is the intended trade-off. **If it is unset in production, the IP hash +is omitted entirely** rather than falling back to a value published in this +repository. + +--- + +## 5. The audited mutation helper + +```ts +import { auditedMutation, actorFromRequest } from '../audit' + +const context = actorFromRequest(req) + +const wallet = await auditedMutation({ + action: 'wallet.status_changed', + actor: context.actor, + target: { type: 'Wallet', id: walletId }, + reason: 'provisioning completed', + requestId: context.requestId, + ipAddress: context.ipAddress, + userAgent: context.userAgent, + source: 'worker.wallet-provisioning', + metadata: { from: 'RESERVED', to: 'ACTIVE' }, + mutate: (tx) => + tx.wallet.update({ where: { id: walletId }, data: { status: 'ACTIVE' } }), +}) +``` + +The mutation and its audit event **commit in one transaction**. An audit written +after a successful mutation leaves holes when the process dies in between; an +audit written before records changes that never happened. + +Every write must go through the `tx` client the helper provides. A write through +the global client commits outside the transaction and escapes the guarantee. + +Two consequences worth knowing: + +- **A failed audit write rolls the mutation back.** A sensitive change that + cannot be attributed is a change that should not land. This is the opposite of + `auditEventService.record()`, which swallows failures and is for standalone + events (a failed login, a rate-limit trip) where there is no state change to + protect. +- **Policy is checked before anything runs.** An `ADMIN` actor without a reason, + or a `USER`/`ADMIN` actor without an id, throws `AuditPolicyError` before the + transaction opens. + +`auditedArchive` and `auditedRestore` wrap the archive patch and its audit event +the same way, and refuse models the matrix does not classify as `ARCHIVABLE`. + +--- + +## 6. Archived records are excluded by default + +The cost of soft deletion is that every read must remember to filter, and one +forgotten filter leaks withdrawn content. So the filter is not left to callers: +`archiveExclusionExtension` in +[`src/audit/archive.ts`](../src/audit/archive.ts) is applied to the Prisma client +in [`src/config/database.ts`](../src/config/database.ts) and injects +`archivedAt: null` into reads on archivable models. + +**Covered:** `findFirst`, `findFirstOrThrow`, `findMany`, `count`, `aggregate`, +`groupBy`. + +**Not covered, deliberately:** + +- **`findUnique` / `findUniqueOrThrow`** — a point lookup by primary key. A + silent filter would turn a found row into `null`, which reads as "deleted" to + code holding the id. Use `assertActive(record)` or `isArchived(record)` to + decide explicitly. +- **Writes** — an archive or restore must be able to see the row it is changing. + +To opt out for one query, use `includeArchived(where)`. It sets `archivedAt` to +`undefined`: Prisma ignores an undefined filter, while the extension sees the key +and stands down. The opt-out is therefore visible at the call site, which is the +point — an unfiltered read of archivable data should show up in review. + +An archived row must always state a reason. This is enforced by a database +`CHECK` constraint per archivable table, not only by the helper, because +"archive" is a write that lands from several call sites and a row archived +without a reason is indistinguishable from an accident months later. + +--- + +## 7. Erasure + +`AccountLifecycleService.finalizeDeletion` applies the `On erasure` column of the +matrix after the cooling-off window (`DELETION_COOLING_OFF_DAYS`, default 30): + +- **Delete** — sessions, tokens, OTP challenges, device tokens, sync events, + notification and email deliveries, completions, export requests. +- **Anonymize** — the `User` row becomes a tombstone; legacy `audit_logs` rows + have their IP, User-Agent and metadata scrubbed. +- **Retain** — money, credentials, consent proof, `audit_events`. +- **Cascade** — dependents removed by database cascade with their parent. + +--- + +## 8. Adding a model + +1. Add it to `prisma/schema.prisma`. +2. Add a rule to `src/audit/classification.ts` — class, category, retention, + anchor, erasure behaviour, whether it is audited, and *why*. +3. If it is `ARCHIVABLE`, add `archivedAt`, `archivedById` and `archivedReason`, + plus the `CHECK` constraint in the migration. +4. Add it to the matrix in this document. + +Step 2 is not optional: `tests/audit/classification.test.ts` fails on an +unclassified model. An unclassified record has no retention, no erasure +behaviour and no audit requirement, which is exactly the state this policy exists +to prevent. + +--- + +## 9. Configuration + +| Variable | Default | Purpose | +| --- | --- | --- | +| `AUDIT_IP_HASH_SECRET` | *(unset)* | HMAC key for `actorIpHash`. Unset in production omits the hash. | +| `DELETION_COOLING_OFF_DAYS` | `30` | Window before an erasure request is finalized | +| `EXPORT_TTL_DAYS` | `7` | Lifetime of an export artifact | +| `LIFECYCLE_SWEEP_INTERVAL_MS` | `0` (disabled) | Background sweep interval | + +--- + +## 10. Tests + +| File | Covers | +| --- | --- | +| `tests/audit/classification.test.ts` | Every schema model classified; retention, erasure and audit invariants per category | +| `tests/audit/redaction.test.ts` | Key and value deny-lists, structural caps, IP hashing, UA coarsening, over-redaction | +| `tests/audit/audit-event.service.test.ts` | Attribution, redaction on write, no mutating API, purge uses the session variable | +| `tests/audit/audited-mutation.test.ts` | Transaction atomicity, rollback on audit failure, policy enforcement, archive/restore | +| `tests/audit/archive.test.ts` | Default exclusion, the `findUnique` and write carve-outs, opt-out, patches | +| `tests/integration/audit-immutability.test.ts` | Database-level `UPDATE`/`DELETE`/`TRUNCATE` rejection and the archive `CHECK` constraints | + +The integration test is skipped when no test database is reachable. Because +`tests/globalSetup.ts` prepares schemas with `prisma db push` — which never +executes migration SQL — that test applies the immutability DDL from the shipped +migration file itself, so it verifies the artifact that actually ships. diff --git a/prisma/migrations/20260824090000_auditable_data_lifecycle/migration.sql b/prisma/migrations/20260824090000_auditable_data_lifecycle/migration.sql new file mode 100644 index 0000000..6c8de38 --- /dev/null +++ b/prisma/migrations/20260824090000_auditable_data_lifecycle/migration.sql @@ -0,0 +1,187 @@ +-- ───────────────────────────────────────────────────────────────────────────── +-- AUDITABLE DATA LIFECYCLE AND ARCHIVE POLICY +-- ───────────────────────────────────────────────────────────────────────────── +-- Adds the two structures the lifecycle policy needs at the database level: +-- +-- 1. "audit_events" — the immutable audit spine. Append-only, enforced by a +-- trigger rather than by convention, because an audit trail an application +-- bug can quietly rewrite is not an audit trail. +-- +-- 2. Archive columns on archivable models, so withdrawing a record that other +-- records depend on is a soft delete instead of a cascade. +-- +-- See docs/DATA_LIFECYCLE.md for the full lifecycle matrix. + +-- CreateTable "audit_events" +-- Deliberately has no foreign key to "users": a cascade would let erasure +-- destroy the trail, and SET NULL would mutate a row that must never change. +-- "actorId" and "targetId" are soft references. +CREATE TABLE "audit_events" ( + "id" TEXT NOT NULL, + -- Actor: who caused the change + "actorType" TEXT NOT NULL, + "actorId" TEXT, + "actorRole" TEXT, + -- Action and target: what changed + "action" TEXT NOT NULL, + "recordClass" TEXT NOT NULL, + "targetType" TEXT NOT NULL, + "targetId" TEXT, + -- Justification and correlation + "reason" TEXT, + "requestId" TEXT, + "correlationId" TEXT, + "source" TEXT, + -- Safe context: redacted JSON, keyed IP hash, coarse User-Agent family + "metadata" TEXT, + "actorIpHash" TEXT, + "userAgentFamily" TEXT, + "occurredAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "audit_events_pkey" PRIMARY KEY ("id") +); + +-- CreateIndex +CREATE INDEX "audit_events_targetType_targetId_occurredAt_idx" ON "audit_events"("targetType", "targetId", "occurredAt"); + +-- CreateIndex +CREATE INDEX "audit_events_actorType_actorId_occurredAt_idx" ON "audit_events"("actorType", "actorId", "occurredAt"); + +-- CreateIndex +CREATE INDEX "audit_events_action_occurredAt_idx" ON "audit_events"("action", "occurredAt"); + +-- CreateIndex +CREATE INDEX "audit_events_requestId_idx" ON "audit_events"("requestId"); + +-- CreateIndex +CREATE INDEX "audit_events_occurredAt_idx" ON "audit_events"("occurredAt"); + + +-- ───────────────────────────────────────────────────────────────────────────── +-- IMMUTABILITY ENFORCEMENT +-- ───────────────────────────────────────────────────────────────────────────── +-- UPDATE is rejected unconditionally. DELETE is rejected unless the caller has +-- set "learnault.audit_purge" for the current transaction, which only +-- AuditEventService.purgeExpired() does — and it deletes by timestamp, so the +-- escape hatch cannot be used to remove one specific inconvenient event. +-- +-- SECURITY DEFINER is not used: the check must apply to every role, including +-- the migration owner. +CREATE OR REPLACE FUNCTION "audit_events_reject_mutation"() +RETURNS TRIGGER AS $$ +BEGIN + IF TG_OP = 'UPDATE' THEN + RAISE EXCEPTION + 'audit_events rows are immutable: UPDATE is not permitted (row %)', OLD."id" + USING ERRCODE = 'restrict_violation'; + END IF; + + -- TG_OP = 'DELETE'. current_setting(..., true) returns NULL rather than + -- raising when the setting has never been assigned in this session. + IF coalesce(current_setting('learnault.audit_purge', true), 'off') <> 'on' THEN + RAISE EXCEPTION + 'audit_events rows may only be deleted by the retention purge (row %)', OLD."id" + USING ERRCODE = 'restrict_violation'; + END IF; + + RETURN OLD; +END; +$$ LANGUAGE plpgsql; + +-- DROP ... IF EXISTS before each CREATE so this block can be re-applied. It is +-- also how `prisma db push` environments (which skip migration SQL entirely, +-- and so never get these triggers) can bootstrap them — see +-- tests/integration/audit-immutability.test.ts. +DROP TRIGGER IF EXISTS "audit_events_no_update" ON "audit_events"; +CREATE TRIGGER "audit_events_no_update" + BEFORE UPDATE ON "audit_events" + FOR EACH ROW EXECUTE FUNCTION "audit_events_reject_mutation"(); + +DROP TRIGGER IF EXISTS "audit_events_no_delete" ON "audit_events"; +CREATE TRIGGER "audit_events_no_delete" + BEFORE DELETE ON "audit_events" + FOR EACH ROW EXECUTE FUNCTION "audit_events_reject_mutation"(); + +-- TRUNCATE bypasses row-level triggers entirely, so it needs a statement-level +-- one of its own. Without this, `TRUNCATE audit_events` would erase the whole +-- trail despite the row triggers above. +CREATE OR REPLACE FUNCTION "audit_events_reject_truncate"() +RETURNS TRIGGER AS $$ +BEGIN + IF coalesce(current_setting('learnault.audit_purge', true), 'off') <> 'on' THEN + RAISE EXCEPTION 'audit_events may not be truncated' + USING ERRCODE = 'restrict_violation'; + END IF; + + RETURN NULL; +END; +$$ LANGUAGE plpgsql; + +DROP TRIGGER IF EXISTS "audit_events_no_truncate" ON "audit_events"; +CREATE TRIGGER "audit_events_no_truncate" + BEFORE TRUNCATE ON "audit_events" + FOR EACH STATEMENT EXECUTE FUNCTION "audit_events_reject_truncate"(); + + +-- ───────────────────────────────────────────────────────────────────────────── +-- ARCHIVE COLUMNS +-- ───────────────────────────────────────────────────────────────────────────── +-- ARCHIVABLE models only. A row is withdrawn by stamping "archivedAt"; reads +-- exclude archived rows by default via the Prisma extension in +-- src/audit/archive.ts. "archivedById" is a soft reference to users.id, not a +-- foreign key, so archive attribution survives the actor's own erasure. + +-- AlterTable: learner_profiles +ALTER TABLE "learner_profiles" + ADD COLUMN "archivedAt" TIMESTAMP(3), + ADD COLUMN "archivedById" TEXT, + ADD COLUMN "archivedReason" TEXT; + +-- AlterTable: Module +ALTER TABLE "Module" + ADD COLUMN "archivedAt" TIMESTAMP(3), + ADD COLUMN "archivedById" TEXT, + ADD COLUMN "archivedReason" TEXT; + +-- AlterTable: avatars +ALTER TABLE "avatars" + ADD COLUMN "archivedAt" TIMESTAMP(3), + ADD COLUMN "archivedById" TEXT, + ADD COLUMN "archivedReason" TEXT; + +-- AlterTable: referral_codes +ALTER TABLE "referral_codes" + ADD COLUMN "archivedAt" TIMESTAMP(3), + ADD COLUMN "archivedById" TEXT, + ADD COLUMN "archivedReason" TEXT; + +-- AlterTable: WebhookEndpoint +ALTER TABLE "WebhookEndpoint" + ADD COLUMN "archivedAt" TIMESTAMP(3), + ADD COLUMN "archivedById" TEXT, + ADD COLUMN "archivedReason" TEXT; + +-- CreateIndex +-- Serves both sides of the archive filter: the "archivedAt" IS NULL predicate +-- every default read carries, and the range scan the retention purge runs over +-- archived rows. Plain rather than partial so it matches the @@index in +-- schema.prisma and no drift is reported. +CREATE INDEX "learner_profiles_archivedAt_idx" ON "learner_profiles"("archivedAt"); +CREATE INDEX "Module_archivedAt_idx" ON "Module"("archivedAt"); +CREATE INDEX "avatars_archivedAt_idx" ON "avatars"("archivedAt"); +CREATE INDEX "referral_codes_archivedAt_idx" ON "referral_codes"("archivedAt"); +CREATE INDEX "WebhookEndpoint_archivedAt_idx" ON "WebhookEndpoint"("archivedAt"); + +-- An archived row must always say why and when. Enforced in the database +-- because "archive" is a write that lands from several call sites, and a row +-- archived without a reason is indistinguishable from an accident months later. +ALTER TABLE "learner_profiles" ADD CONSTRAINT "learner_profiles_archive_reason_check" + CHECK ("archivedAt" IS NULL OR "archivedReason" IS NOT NULL); +ALTER TABLE "Module" ADD CONSTRAINT "Module_archive_reason_check" + CHECK ("archivedAt" IS NULL OR "archivedReason" IS NOT NULL); +ALTER TABLE "avatars" ADD CONSTRAINT "avatars_archive_reason_check" + CHECK ("archivedAt" IS NULL OR "archivedReason" IS NOT NULL); +ALTER TABLE "referral_codes" ADD CONSTRAINT "referral_codes_archive_reason_check" + CHECK ("archivedAt" IS NULL OR "archivedReason" IS NOT NULL); +ALTER TABLE "WebhookEndpoint" ADD CONSTRAINT "WebhookEndpoint_archive_reason_check" + CHECK ("archivedAt" IS NULL OR "archivedReason" IS NOT NULL); diff --git a/prisma/schema.prisma b/prisma/schema.prisma index 647380d..2477b20 100644 --- a/prisma/schema.prisma +++ b/prisma/schema.prisma @@ -99,12 +99,19 @@ model LearnerProfile { visibility String @default("private") // private, employer, public + // Archive (soft delete). Reads exclude archived rows by default — see + // src/audit/archive.ts. Purged 365 days after archivedAt. + archivedAt DateTime? + archivedById String? + archivedReason String? + createdAt DateTime @default(now()) updatedAt DateTime @updatedAt @@index([country]) @@index([level]) @@index([visibility]) + @@index([archivedAt]) @@map("learner_profiles") } @@ -210,6 +217,60 @@ model RefreshToken { @@map("refresh_tokens") } +/// Immutable audit spine for security-, money-, credential-, and +/// content-sensitive mutations. See docs/DATA_LIFECYCLE.md. +/// +/// Append-only: a database trigger rejects every UPDATE, and rejects DELETE +/// unless the retention purge has set the `learnault.audit_purge` session +/// variable. There is no code path that edits a row here. +/// +/// Erasure-safe by construction. Nothing in this table needs scrubbing when a +/// subject is erased, because nothing identifying is stored in the first place: +/// actor and target are opaque ids, the request IP is kept only as a keyed +/// hash, the User-Agent only as a coarse family, and `metadata` passes through +/// the redaction filter in src/audit/redaction.ts on the way in. +/// +/// Deliberately has no relation to User. A foreign key would need either +/// `onDelete: Cascade` (erasure destroys the trail) or `onDelete: SetNull` +/// (erasure mutates an immutable row). `actorId` is a soft reference instead. +model AuditEvent { + id String @id @default(uuid()) + + // ── Actor: who caused the change ── + actorType String // USER, ADMIN, SYSTEM, WORKER, ANONYMOUS + actorId String? // opaque id; null for SYSTEM and ANONYMOUS actors + actorRole String? // role held at the time of the action, not now + + // ── Action and target: what changed ── + action String // e.g. "account.deactivated", "wallet.provisioned" + recordClass String // MUTABLE, ARCHIVABLE, DELETABLE, IMMUTABLE + targetType String // Prisma model name, e.g. "User", "Wallet" + targetId String? // primary key of the affected row + + // ── Justification and correlation ── + reason String? // caller-supplied; required for ADMIN actors + requestId String? // x-request-id, correlates to request logs + correlationId String? // outbox event or job id for async changes + source String? // e.g. "api.account.deactivate", "worker.sweep" + + // ── Safe context: redacted, hashed, coarsened ── + metadata String? // redacted JSON; never secrets, never raw PII + actorIpHash String? // keyed SHA-256 of the request IP, never the raw IP + userAgentFamily String? // e.g. "Chrome", "Android"; never the raw UA + + occurredAt DateTime @default(now()) + + @@index([targetType, targetId, occurredAt]) + @@index([actorType, actorId, occurredAt]) + @@index([action, occurredAt]) + @@index([requestId]) + @@index([occurredAt]) + @@map("audit_events") +} + +/// Superseded by AuditEvent, and retained only for the trail written before the +/// data-lifecycle policy landed. Unlike AuditEvent it holds a raw IP and +/// User-Agent, so erasure scrubs those columns. model AuditLog { id String @id @default(uuid()) userId String? @@ -276,10 +337,18 @@ model Module { assetIssuer String? /// Stellar network: "testnet" or "mainnet". assetNetwork String @default("testnet") + /// Archive (soft delete). Withdrawn content is archived, never deleted: + /// completions and credentials reference the module a learner actually took. + /// Reads exclude archived rows by default — see src/audit/archive.ts. + archivedAt DateTime? + archivedById String? + archivedReason String? createdAt DateTime @default(now()) updatedAt DateTime @updatedAt completions Completion[] credentials Credential[] + + @@index([archivedAt]) } model Completion { @@ -339,9 +408,16 @@ model ReferralCode { code String @unique userId String @unique user User @relation(fields: [userId], references: [id], onDelete: Cascade) + /// Archive (soft delete). Retiring a code archives it so the Referral rows + /// pointing at it keep a valid referent. Reads exclude archived rows by + /// default — see src/audit/archive.ts. + archivedAt DateTime? + archivedById String? + archivedReason String? createdAt DateTime @default(now()) referrals Referral[] + @@index([archivedAt]) @@map("referral_codes") } @@ -390,9 +466,17 @@ model WebhookEndpoint { description String? isActive Boolean @default(true) events String // Comma-separated list of events: "module.completed,reward.issued" + /// Archive (soft delete), distinct from isActive: isActive pauses delivery, + /// archiving retires the endpoint while its delivery history keeps a valid + /// referent. Reads exclude archived rows by default — see src/audit/archive.ts. + archivedAt DateTime? + archivedById String? + archivedReason String? createdAt DateTime @default(now()) updatedAt DateTime @updatedAt deliveries WebhookDelivery[] + + @@index([archivedAt]) } model WebhookDelivery { @@ -616,8 +700,16 @@ model Avatar { replacedBy Avatar? @relation("AvatarReplacement", fields: [replacedById], references: [id]) replacements Avatar[] @relation("AvatarReplacement") variants AvatarVariant[] + /// Archive (soft delete), distinct from replacedAt: replacedAt records that a + /// newer avatar superseded this one, archivedAt records that it was withdrawn + /// (by the learner or by moderation). Reads exclude archived rows by default — + /// see src/audit/archive.ts. + archivedAt DateTime? + archivedById String? + archivedReason String? @@index([userId, status]) + @@index([archivedAt]) @@map("avatars") } diff --git a/src/audit/archive.ts b/src/audit/archive.ts new file mode 100644 index 0000000..606e284 --- /dev/null +++ b/src/audit/archive.ts @@ -0,0 +1,226 @@ +/** + * Archive (soft-delete) semantics and the default-exclusion rule. + * + * An archivable record is withdrawn by stamping `archivedAt` instead of being + * deleted, because something else still depends on it — a Completion needs the + * Module a learner actually took, a Referral needs the code that created it. + * + * The cost of soft deletion is that every read has to remember to filter, and + * one forgotten filter leaks withdrawn content. So the filter is not left to + * callers: {@link archiveExclusionExtension} injects it into read queries, and + * a caller that wants archived rows has to say so explicitly. + */ + +import { modelsInClass } from './classification.js' +import { RecordClass } from './types.js' + +/** Models carrying archive columns, derived from the lifecycle matrix. */ +export const ARCHIVABLE_MODELS: ReadonlySet = new Set( + modelsInClass(RecordClass.ARCHIVABLE) +) + +/** + * Read operations that get the `archivedAt: null` filter injected. + * + * `findUnique` is deliberately absent. It resolves a single row by primary key, + * where a silent filter turns a found row into `null` and reads as "deleted" to + * calling code that has the id in hand. Point lookups therefore return archived + * rows, and callers that care use {@link isArchived} or {@link assertActive}. + * + * Writes are absent for the same reason: an archive or a restore is a write + * that must be able to see the row it is changing. + */ +const FILTERED_OPERATIONS: ReadonlySet = new Set([ + 'findFirst', + 'findFirstOrThrow', + 'findMany', + 'count', + 'aggregate', + 'groupBy', +]) + +/** Columns stamped when a record is archived. */ +export interface ArchiveColumns { + archivedAt: Date | null + archivedById: string | null + archivedReason: string | null +} + +/** A record that may or may not be archived. */ +type MaybeArchived = { archivedAt?: Date | null } + +/** + * Whether a `where` clause already talks about `archivedAt`, at the top level or + * inside a boolean combinator. If it does, the caller has an opinion and the + * extension leaves it alone. + * + * `hasOwnProperty` rather than a truthiness check, so `{ archivedAt: undefined }` + * — which is how {@link includeArchived} opts out — counts as an opinion even + * though Prisma itself ignores the undefined value. + */ +export function mentionsArchivedAt(where: unknown): boolean { + if (!where || typeof where !== 'object') { + return false + } + + if (Array.isArray(where)) { + return where.some(mentionsArchivedAt) + } + + const clause = where as Record + + if (Object.prototype.hasOwnProperty.call(clause, 'archivedAt')) { + return true + } + + return (['AND', 'OR', 'NOT'] as const).some( + (combinator) => + Object.prototype.hasOwnProperty.call(clause, combinator) && + mentionsArchivedAt(clause[combinator]) + ) +} + +/** One intercepted Prisma operation, as the client extension sees it. */ +export interface QueryInterception { + model?: string + operation: string + args: A + query: (args: A) => Promise +} + +/** + * The interceptor behind {@link archiveExclusionExtension}: forwards the query + * with `archivedAt: null` added when the model is archivable, the operation is a + * read, and the caller has not already filtered on `archivedAt`. + * + * Exported as a plain function because `Prisma.defineExtension` returns an + * opaque closure — the decision this makes is the whole soft-delete guarantee, + * so it needs to be testable without a database. + */ +export async function excludeArchivedFromReads({ + model, + operation, + args, + query, +}: QueryInterception): Promise { + if (!model || !ARCHIVABLE_MODELS.has(model) || !FILTERED_OPERATIONS.has(operation)) { + return query(args) + } + + const typed = (args ?? {}) as { where?: Record } + + if (mentionsArchivedAt(typed.where)) { + return query(args) + } + + return query({ + ...typed, + where: { ...(typed.where ?? {}), archivedAt: null }, + }) +} + +/** + * Prisma client extension that hides archived rows from list and aggregate + * queries on archivable models. + * + * Applied once, in src/config/database.ts, so "exclude archived records by + * default" holds for code that has never heard of this module. + * + * A plain object rather than `Prisma.defineExtension(...)`. `defineExtension` + * only adds type inference this does not need, and importing the `Prisma` + * namespace here would drag it into every module that reaches the Prisma client + * — breaking any test that mocks `@prisma/client` without re-exporting it. + */ +export const archiveExclusionExtension = { + name: 'archiveExclusion', + query: { + $allModels: { + $allOperations: excludeArchivedFromReads, + }, + }, +} as const + +/** + * Restrict a `where` clause to live rows. Redundant for the operations the + * extension already covers; useful for `findUnique` and for raw queries. + */ +export function activeOnly(where?: W): W & { archivedAt: null } { + return { ...((where ?? {}) as W), archivedAt: null } +} + +/** Restrict a `where` clause to archived rows only. */ +export function archivedOnly( + where?: W +): W & { archivedAt: { not: null } } { + return { ...((where ?? {}) as W), archivedAt: { not: null } } +} + +/** + * Opt out of default exclusion for one query. + * + * Sets `archivedAt` to `undefined`: Prisma ignores an undefined filter, while + * the extension sees the key and stands down. Explicit at the call site, which + * is the point — an unfiltered read of archivable data should be visible in + * review. + */ +export function includeArchived( + where?: W +): W & { archivedAt: undefined } { + return { ...((where ?? {}) as W), archivedAt: undefined } +} + +/** + * The `data` patch that archives a record. `reason` is required: an archive + * with no stated reason is indistinguishable from an accident six months later. + */ +export function archivePatch( + reason: string, + archivedById?: string | null, + now: Date = new Date() +): ArchiveColumns { + return { + archivedAt: now, + archivedById: archivedById ?? null, + archivedReason: reason, + } +} + +/** The `data` patch that restores an archived record. */ +export function restorePatch(): ArchiveColumns { + return { + archivedAt: null, + archivedById: null, + archivedReason: null, + } +} + +/** Whether a loaded record is archived. */ +export function isArchived(record: MaybeArchived | null | undefined): boolean { + return Boolean(record?.archivedAt) +} + +/** + * Narrow a point lookup to a live record, returning `null` for an archived one. + * The counterpart to `findUnique` being exempt from the extension. + */ +export function assertActive(record: T | null): T | null { + return record && !isArchived(record) ? record : null +} + +/** + * Cut-off before which archived rows of a model may be purged, or `null` when + * archived rows of that model are kept indefinitely. + * + * Re-exported through the module's retention helper so callers do not need to + * know that the anchor column differs per model. + */ +export function archivedPurgeCutoff( + retentionDays: number | null, + now: Date = new Date() +): Date | null { + if (retentionDays === null) { + return null + } + + return new Date(now.getTime() - retentionDays * 24 * 60 * 60_000) +} diff --git a/src/audit/audit-event.service.ts b/src/audit/audit-event.service.ts new file mode 100644 index 0000000..c49a9aa --- /dev/null +++ b/src/audit/audit-event.service.ts @@ -0,0 +1,254 @@ +/** + * Immutable audit event writer and reader. + * + * Every row records who did what to which record, why, and under which request. + * Nothing here can update or delete an event: the only mutating methods are the + * append path and the retention purge, and the database rejects anything else + * (see the trigger in the auditable_data_lifecycle migration). + */ + +import prisma from '../config/database' +import logger from '../utils/logger' +import { env } from '../config/env' +import { recordClassFor, retentionCutoff } from './classification.js' +import { hashIpAddress, serializeMetadata, userAgentFamily } from './redaction.js' +import { + AuditEventInput, + AuditEventQuery, + AuditEventRecord, + RecordClassValue, +} from './types.js' + +/** + * Postgres session variable that lets the retention purge — and only the + * retention purge — delete audit rows. The trigger checks it by name. + */ +export const AUDIT_PURGE_SETTING = 'learnault.audit_purge' + +/** + * Fallback IP-hash secret for development and test, where a stable value across + * restarts is more useful than a strong one. Never reached in production: an + * unset secret there disables IP hashing outright rather than falling back to a + * value that is public in this repository. + */ +const DEV_IP_HASH_SECRET = 'learnault-dev-audit-ip-hash' + +/** Rows returned by a single {@link AuditEventService.list} call, by default. */ +const DEFAULT_PAGE_SIZE = 50 + +/** Hard ceiling on a page, so a caller cannot pull the whole trail at once. */ +const MAX_PAGE_SIZE = 200 + +/** Minimal shape this service needs from a Prisma client or transaction. */ +type AuditEventWriter = { + auditEvent: { + create: (args: { data: AuditEventRow }) => unknown + } +} + +/** The row as it is written. Mirrors the AuditEvent model in the schema. */ +export interface AuditEventRow { + actorType: string + actorId: string | null + actorRole: string | null + action: string + recordClass: string + targetType: string + targetId: string | null + reason: string | null + requestId: string | null + correlationId: string | null + source: string | null + metadata: string | null + actorIpHash: string | null + userAgentFamily: string | null +} + +export class AuditEventService { + /** + * Append an audit event outside a transaction. + * + * Never throws — an observability failure must not take down the flow it is + * observing. Use this only for events that stand alone (a failed login, a + * rate-limit trip). Anything that accompanies a state change belongs in + * `auditedMutation`, where the audit row and the change commit together. + */ + async record(input: AuditEventInput): Promise { + try { + await prisma.auditEvent.create({ data: this.toRow(input) }) + } catch (error) { + // Log the action, never the metadata: the metadata is the part that may + // have been rejected for being oversized or malformed. + logger.error('[AuditEventService] Failed to append audit event', { + action: input.action, + targetType: input.target.type, + error: error instanceof Error ? error.message : String(error), + }) + } + } + + /** + * Build an `auditEvent.create` operation for a `prisma.$transaction([...])` + * array, so the event commits atomically with the change it describes. + */ + op(input: AuditEventInput) { + return prisma.auditEvent.create({ data: this.toRow(input) }) + } + + /** + * Append an audit event on a specific transaction client. Unlike + * {@link record} this propagates failures, because inside a transaction a + * rejected audit write must roll the mutation back rather than let an + * unaudited change through. + */ + async recordWithin(tx: AuditEventWriter, input: AuditEventInput): Promise { + await tx.auditEvent.create({ data: this.toRow(input) }) + } + + /** + * Read the audit trail. Returns rows exactly as stored: the redaction that + * matters already happened at write time, so there is nothing left to filter + * on the way out. + */ + async list(query: AuditEventQuery = {}): Promise { + const take = Math.min(Math.max(query.take ?? DEFAULT_PAGE_SIZE, 1), MAX_PAGE_SIZE) + + const occurredAt = + query.from || query.to + ? { ...(query.from ? { gte: query.from } : {}), ...(query.to ? { lte: query.to } : {}) } + : undefined + + const rows = await prisma.auditEvent.findMany({ + where: { + ...(query.actorId ? { actorId: query.actorId } : {}), + ...(query.actorType ? { actorType: query.actorType } : {}), + ...(query.action ? { action: query.action } : {}), + ...(query.targetType ? { targetType: query.targetType } : {}), + ...(query.targetId ? { targetId: query.targetId } : {}), + ...(query.requestId ? { requestId: query.requestId } : {}), + ...(occurredAt ? { occurredAt } : {}), + }, + orderBy: { occurredAt: 'desc' }, + take, + skip: query.skip ?? 0, + }) + + return rows as AuditEventRecord[] + } + + /** Every event touching one record, oldest first — the record's history. */ + async historyFor( + targetType: string, + targetId: string, + take = DEFAULT_PAGE_SIZE + ): Promise { + const rows = await prisma.auditEvent.findMany({ + where: { targetType, targetId }, + orderBy: { occurredAt: 'asc' }, + take: Math.min(Math.max(take, 1), MAX_PAGE_SIZE), + }) + + return rows as AuditEventRecord[] + } + + /** + * Delete audit events past their retention window. + * + * This is the only sanctioned deletion path. It opens a transaction, sets the + * purge session variable the immutability trigger checks, and deletes by + * timestamp — so a purge can never be used to remove a *specific* + * inconvenient event, only everything older than the cut-off. + */ + async purgeExpired(now: Date = new Date()): Promise { + const cutoff = retentionCutoff('AuditEvent', now) + + if (!cutoff) { + return 0 + } + + try { + return await prisma.$transaction(async (tx) => { + // SET LOCAL takes no bind parameters, hence the raw statement. The + // setting name is a module constant and never caller-supplied. + await tx.$executeRawUnsafe(`SET LOCAL "${AUDIT_PURGE_SETTING}" = 'on'`) + + const deleted = await tx.$executeRaw` + DELETE FROM "audit_events" WHERE "occurredAt" < ${cutoff} + ` + + return deleted + }) + } catch (error) { + logger.error('[AuditEventService] Retention purge failed', { + cutoff: cutoff.toISOString(), + error: error instanceof Error ? error.message : String(error), + }) + + return 0 + } + } + + /** + * Build the row for an input: resolves the lifecycle class, redacts metadata, + * hashes the IP and coarsens the User-Agent. + * + * Exposed so tests and callers can assert on exactly what would be persisted + * without touching a database. + */ + toRow(input: AuditEventInput): AuditEventRow { + const recordClass: RecordClassValue = + input.recordClass ?? recordClassFor(input.target.type) + + // No secret means no hash. An HMAC under an empty key is a plain digest, + // and the IPv4 space is small enough that a plain digest is reversible. + const ipHashSecret = this.ipHashSecret() + + return { + actorType: input.actor.type, + actorId: input.actor.id ?? null, + actorRole: input.actor.role ?? null, + action: input.action, + recordClass, + targetType: input.target.type, + targetId: input.target.id ?? null, + reason: input.reason ?? null, + requestId: input.requestId ?? null, + correlationId: input.correlationId ?? null, + source: input.source ?? null, + metadata: serializeMetadata(input.metadata), + actorIpHash: ipHashSecret ? hashIpAddress(input.ipAddress, ipHashSecret) : null, + userAgentFamily: userAgentFamily(input.userAgent), + } + } + + /** + * Resolve the HMAC secret for IP hashing. + * + * Returns an empty string in production when unconfigured, which + * {@link toRow} treats as "do not hash at all". Correlating events by source + * is a convenience; storing a reversible IP hash is not an acceptable price + * for it. + */ + private ipHashSecret(): string { + if (env.AUDIT_IP_HASH_SECRET) { + return env.AUDIT_IP_HASH_SECRET + } + + if (env.NODE_ENV === 'production') { + if (!this.warnedAboutSecret) { + this.warnedAboutSecret = true + logger.warn( + '[AuditEventService] AUDIT_IP_HASH_SECRET is not set; audit events will omit the source IP hash.' + ) + } + + return '' + } + + return DEV_IP_HASH_SECRET + } + + private warnedAboutSecret = false +} + +export const auditEventService = new AuditEventService() diff --git a/src/audit/audited-mutation.ts b/src/audit/audited-mutation.ts new file mode 100644 index 0000000..b380b76 --- /dev/null +++ b/src/audit/audited-mutation.ts @@ -0,0 +1,293 @@ +/** + * The reusable audited mutation. + * + * Wraps a state change and its audit event in one transaction, so the two + * commit together or not at all. That is the whole point: an audit trail written + * *after* a successful mutation is a trail with holes in it, because the process + * can die in between, and a trail written *before* records changes that never + * happened. + * + * Note the difference from `auditEventService.record`, which swallows failures. + * Here a failed audit write rolls the mutation back. A sensitive change that + * cannot be attributed is a change that should not land. + */ + +import prisma from '../config/database' +import { auditEventService } from './audit-event.service.js' +import { lifecycleRuleFor } from './classification.js' +import { archivePatch, restorePatch } from './archive.js' +import { ActorType, AuditActor, AuditEventInput, AuditTarget } from './types.js' + +/** + * The client handed to a mutation body: the full Prisma client minus the + * operations that cannot run inside an interactive transaction. + * + * Derived from the configured client rather than from `Prisma.TransactionClient` + * so it stays correct as extensions are added in src/config/database.ts. + */ +export type AuditedTransactionClient = Omit< + typeof prisma, + '$connect' | '$disconnect' | '$on' | '$transaction' | '$extends' | '$use' +> + +/** Request-scoped context, as produced by {@link actorFromRequest}. */ +export interface AuditContext { + actor: AuditActor + requestId?: string | null + ipAddress?: string | null + userAgent?: string | null +} + +export interface AuditedMutationSpec { + /** Dotted action name, e.g. `"account.deactivated"`. */ + action: string + /** Who is making the change. */ + actor: AuditActor + /** + * What is being changed. `id` may be omitted when it is only known after the + * mutation runs — use {@link resolveTargetId} for that case. + */ + target: AuditTarget + /** + * Why. Mandatory for ADMIN actors: a staff member changing another person's + * record without a stated reason is the exact case an audit trail exists for. + */ + reason?: string | null + requestId?: string | null + correlationId?: string | null + /** Code path producing the change, e.g. `"api.account.deactivate"`. */ + source?: string | null + /** Context for the event. Redacted before storage — never pass secrets. */ + metadata?: Record | null + ipAddress?: string | null + userAgent?: string | null + /** + * The state change. Receives the transaction client and must perform every + * write through it, or the write will not be covered by the audit's atomicity. + */ + mutate: (tx: AuditedTransactionClient) => Promise + /** Derive the target id from the mutation result (e.g. for a create). */ + resolveTargetId?: (result: T) => string | null | undefined + /** Derive extra metadata from the result (e.g. the status actually reached). */ + resolveMetadata?: (result: T) => Record | null | undefined +} + +/** Thrown when a spec violates the audit policy, before anything is written. */ +export class AuditPolicyError extends Error { + constructor(message: string) { + super(message) + this.name = 'AuditPolicyError' + } +} + +/** + * Run a mutation and append its audit event atomically, returning the mutation's + * result. + * + * ```ts + * const wallet = await auditedMutation({ + * action: 'wallet.status_changed', + * actor: { type: ActorType.WORKER, id: 'wallet-provisioning' }, + * target: { type: 'Wallet', id: walletId }, + * source: 'worker.wallet-provisioning', + * metadata: { from: 'RESERVED', to: 'ACTIVE' }, + * mutate: (tx) => + * tx.wallet.update({ where: { id: walletId }, data: { status: 'ACTIVE' } }), + * }) + * ``` + */ +export async function auditedMutation(spec: AuditedMutationSpec): Promise { + assertPolicy(spec) + + return prisma.$transaction(async (tx) => { + const result = await spec.mutate(tx as AuditedTransactionClient) + + const resolvedId = spec.resolveTargetId?.(result) + const extraMetadata = spec.resolveMetadata?.(result) + + const event: AuditEventInput = { + action: spec.action, + actor: spec.actor, + target: { + type: spec.target.type, + id: spec.target.id ?? resolvedId ?? null, + }, + reason: spec.reason, + requestId: spec.requestId, + correlationId: spec.correlationId, + source: spec.source, + metadata: + spec.metadata || extraMetadata + ? { ...(spec.metadata ?? {}), ...(extraMetadata ?? {}) } + : null, + ipAddress: spec.ipAddress, + userAgent: spec.userAgent, + } + + await auditEventService.recordWithin(tx, event) + + return result + }) +} + +/** + * Archive a record and audit it, in one transaction. + * + * `model` is the Prisma model name; it is checked against the lifecycle matrix, + * so archiving something the policy does not classify as ARCHIVABLE fails loudly + * instead of writing an `archivedAt` to a column that may not exist. + */ +export async function auditedArchive(input: { + model: string + id: string + reason: string + actor: AuditActor + context?: Omit + correlationId?: string | null + source?: string | null + metadata?: Record | null + archive: (tx: AuditedTransactionClient, patch: ReturnType) => Promise +}): Promise { + assertArchivable(input.model, 'archive') + + if (!input.reason?.trim()) { + throw new AuditPolicyError( + `Archiving ${input.model} requires a reason: an archive with no stated reason cannot be reviewed later.` + ) + } + + const patch = archivePatch(input.reason, input.actor.id ?? null) + + return auditedMutation({ + action: `${camelToSnake(input.model)}.archived`, + actor: input.actor, + target: { type: input.model, id: input.id }, + reason: input.reason, + requestId: input.context?.requestId, + ipAddress: input.context?.ipAddress, + userAgent: input.context?.userAgent, + correlationId: input.correlationId, + source: input.source, + metadata: { ...(input.metadata ?? {}), archivedAt: patch.archivedAt }, + mutate: (tx) => input.archive(tx, patch), + }) +} + +/** Restore an archived record and audit it, in one transaction. */ +export async function auditedRestore(input: { + model: string + id: string + reason: string + actor: AuditActor + context?: Omit + correlationId?: string | null + source?: string | null + metadata?: Record | null + restore: (tx: AuditedTransactionClient, patch: ReturnType) => Promise +}): Promise { + assertArchivable(input.model, 'restore') + + return auditedMutation({ + action: `${camelToSnake(input.model)}.restored`, + actor: input.actor, + target: { type: input.model, id: input.id }, + reason: input.reason, + requestId: input.context?.requestId, + ipAddress: input.context?.ipAddress, + userAgent: input.context?.userAgent, + correlationId: input.correlationId, + source: input.source, + metadata: input.metadata ?? null, + mutate: (tx) => input.restore(tx, restorePatch()), + }) +} + +/** + * Build audit context from an Express request. + * + * Reads the actor from `req.actor` (set by the request-context middleware after + * authentication) and falls back to an ANONYMOUS actor, so an unauthenticated + * action is still attributable to a request rather than to nobody. + * + * The raw IP and User-Agent are carried through, but the audit writer hashes and + * coarsens them before they reach the database. + */ +export function actorFromRequest(req: { + actor?: { id: string; role: string } | undefined + requestId?: string | undefined + ip?: string | undefined + headers?: Record | undefined +}): AuditContext { + const userAgent = req.headers?.['user-agent'] + + return { + actor: req.actor + ? { + // A staff role acting through the API is an ADMIN actor: it is the + // actor's authority, not the endpoint, that decides how much scrutiny + // the event deserves. + type: req.actor.role === 'ADMIN' ? ActorType.ADMIN : ActorType.USER, + id: req.actor.id, + role: req.actor.role, + } + : { type: ActorType.ANONYMOUS }, + requestId: req.requestId ?? null, + ipAddress: req.ip ?? null, + userAgent: typeof userAgent === 'string' ? userAgent : null, + } +} + +/** A system actor, for sweeps and migrations with no human trigger. */ +export function systemActor(component: string): AuditActor { + return { type: ActorType.SYSTEM, id: component } +} + +/** A worker actor, for queue-draining background work. */ +export function workerActor(worker: string): AuditActor { + return { type: ActorType.WORKER, id: worker } +} + +function assertPolicy(spec: AuditedMutationSpec): void { + if (!spec.action.trim()) { + throw new AuditPolicyError('An audited mutation requires an action name.') + } + + if (!spec.target.type.trim()) { + throw new AuditPolicyError( + `Audited mutation "${spec.action}" requires a target type (the Prisma model name).` + ) + } + + if (spec.actor.type === ActorType.ADMIN && !spec.reason?.trim()) { + throw new AuditPolicyError( + `Audited mutation "${spec.action}" is performed by an ADMIN actor and therefore requires a reason.` + ) + } + + if ((spec.actor.type === ActorType.USER || spec.actor.type === ActorType.ADMIN) && !spec.actor.id) { + throw new AuditPolicyError( + `Audited mutation "${spec.action}" has a ${spec.actor.type} actor with no id; the event would be unattributable.` + ) + } +} + +function assertArchivable(model: string, verb: string): void { + const rule = lifecycleRuleFor(model) + + if (!rule) { + throw new AuditPolicyError( + `Cannot ${verb} ${model}: it has no rule in the lifecycle matrix (src/audit/classification.ts).` + ) + } + + if (rule.recordClass !== 'ARCHIVABLE') { + throw new AuditPolicyError( + `Cannot ${verb} ${model}: the lifecycle matrix classifies it as ${rule.recordClass}, not ARCHIVABLE.` + ) + } +} + +/** `LearnerProfile` → `learner_profile`, for building default action names. */ +function camelToSnake(model: string): string { + return model.replace(/([a-z0-9])([A-Z])/g, '$1_$2').toLowerCase() +} diff --git a/src/audit/classification.ts b/src/audit/classification.ts new file mode 100644 index 0000000..628c37d --- /dev/null +++ b/src/audit/classification.ts @@ -0,0 +1,530 @@ +/** + * The lifecycle matrix: one rule per persisted model. + * + * This file is the machine-readable source of truth behind + * docs/DATA_LIFECYCLE.md. Adding a model to prisma/schema.prisma without adding + * it here fails tests/audit/classification.test.ts, which is deliberate — an + * unclassified record has no retention, no erasure behaviour and no audit + * requirement, and that is exactly the state this policy exists to prevent. + */ + +import { + DataCategory, + ErasureAction, + LifecycleRule, + RecordClass, + RecordClassValue, +} from './types.js' + +/** Retention windows, in days. Named so the intent survives the number. */ +export const Retention = { + /** Financial and consent records: statutory bookkeeping horizon. */ + SEVEN_YEARS: 2555, + /** Security events: long enough for forensics on a late-discovered breach. */ + TWO_YEARS: 730, + /** Archived content: reversible for a full release cycle before purge. */ + ONE_YEAR: 365, + /** Operational journals worth keeping for trend analysis. */ + NINETY_DAYS: 90, + /** Delivery queues and short-lived auth material. */ + THIRTY_DAYS: 30, + /** Export artifacts containing full-fidelity personal data. */ + SEVEN_DAYS: 7, + /** Retain indefinitely — nothing purges it. */ + INDEFINITE: null, +} as const + +const RULES: readonly LifecycleRule[] = [ + // ── Identity ────────────────────────────────────────────────────────────── + { + model: 'User', + table: 'users', + recordClass: RecordClass.MUTABLE, + category: DataCategory.IDENTITY, + retentionDays: Retention.INDEFINITE, + retentionAnchor: null, + onErasure: ErasureAction.ANONYMIZE, + audited: true, + notes: + 'Anonymized in place rather than deleted, so retained money and credential rows keep a valid referent. The tombstone carries no personal data.', + }, + { + model: 'LearnerPreference', + table: 'learner_preferences', + recordClass: RecordClass.MUTABLE, + category: DataCategory.IDENTITY, + retentionDays: Retention.INDEFINITE, + retentionAnchor: null, + onErasure: ErasureAction.CASCADE, + audited: true, + notes: + 'Authoritative source for privacy-impacting preferences, so every change is audited. Prior values live in PreferenceAuditLog.', + }, + { + model: 'LearnerProfile', + table: 'learner_profiles', + recordClass: RecordClass.ARCHIVABLE, + category: DataCategory.IDENTITY, + retentionDays: Retention.ONE_YEAR, + retentionAnchor: 'archivedAt', + onErasure: ErasureAction.CASCADE, + audited: true, + notes: + 'Learner-authored profile. Archived on deactivation so employer-visible listings drop it immediately while the learner can still come back.', + }, + { + model: 'OnboardingProgress', + table: 'onboarding_progress', + recordClass: RecordClass.MUTABLE, + category: DataCategory.IDENTITY, + retentionDays: Retention.INDEFINITE, + retentionAnchor: null, + onErasure: ErasureAction.CASCADE, + audited: false, + notes: 'Step tracking only. Holds no personal data beyond the step names.', + }, + { + model: 'Avatar', + table: 'avatars', + recordClass: RecordClass.ARCHIVABLE, + category: DataCategory.CONTENT, + retentionDays: Retention.ONE_YEAR, + retentionAnchor: 'archivedAt', + onErasure: ErasureAction.CASCADE, + audited: true, + notes: + 'Learner-supplied image. Archived rather than deleted so a replaced avatar can be restored and so moderation decisions stay reviewable.', + }, + { + model: 'AvatarVariant', + table: 'avatar_variants', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.CONTENT, + retentionDays: Retention.ONE_YEAR, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.CASCADE, + audited: false, + notes: + 'Derived renditions of an Avatar. Never edited — a new variant set replaces the old one. Purged with its parent avatar.', + }, + + // ── Consent ─────────────────────────────────────────────────────────────── + { + model: 'ConsentRecord', + table: 'consent_records', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.CONSENT, + retentionDays: Retention.SEVEN_YEARS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'Proof of consent must outlive the account it describes, otherwise a withdrawal cannot be demonstrated. Withdrawal appends a new row; it never edits the old one.', + }, + { + model: 'PreferenceAuditLog', + table: 'preference_audit_logs', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.CONSENT, + retentionDays: Retention.SEVEN_YEARS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.RETAIN, + audited: false, + notes: + 'Field-level history for privacy preferences. Append-only and self-auditing, so it needs no audit event of its own.', + }, + + // ── Security ────────────────────────────────────────────────────────────── + { + model: 'AuditEvent', + table: 'audit_events', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.SECURITY, + retentionDays: Retention.SEVEN_YEARS, + retentionAnchor: 'occurredAt', + onErasure: ErasureAction.RETAIN, + audited: false, + notes: + 'The audit spine. UPDATE is rejected outright by a database trigger; DELETE is permitted only to the retention purge. Retained through erasure because it stores no raw personal data.', + }, + { + model: 'AuditLog', + table: 'audit_logs', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.SECURITY, + retentionDays: Retention.TWO_YEARS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.ANONYMIZE, + audited: false, + notes: + 'Superseded by AuditEvent. Kept for the trail written before this policy landed; it holds raw IP and User-Agent, so erasure scrubs those columns rather than retaining the row untouched.', + }, + { + model: 'Session', + table: 'sessions', + recordClass: RecordClass.MUTABLE, + category: DataCategory.SECURITY, + retentionDays: Retention.NINETY_DAYS, + retentionAnchor: 'updatedAt', + onErasure: ErasureAction.DELETE, + audited: true, + notes: + 'Revocation is a status change, not an archive: a revoked session must stay visible to the learner in device management until it is purged.', + }, + { + model: 'RefreshToken', + table: 'refresh_tokens', + recordClass: RecordClass.MUTABLE, + category: DataCategory.SECURITY, + retentionDays: Retention.NINETY_DAYS, + retentionAnchor: 'updatedAt', + onErasure: ErasureAction.CASCADE, + audited: true, + notes: + 'Rotation family. Consumed rows are kept until purge so replay of an already-rotated token is still detectable as theft.', + }, + { + model: 'VerificationToken', + table: 'verification_tokens', + recordClass: RecordClass.MUTABLE, + category: DataCategory.SECURITY, + retentionDays: Retention.THIRTY_DAYS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.DELETE, + audited: true, + notes: + 'Single-use credential. Hard-deleted on erasure; the audit event records that it was issued and consumed.', + }, + { + model: 'OtpChallenge', + table: 'otp_challenges', + recordClass: RecordClass.MUTABLE, + category: DataCategory.SECURITY, + retentionDays: Retention.THIRTY_DAYS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.DELETE, + audited: true, + notes: + 'Holds a phone number and a code hash. Short retention because the lockout and rate-limit decisions it supports are themselves short-lived.', + }, + { + model: 'ManagedKeyReference', + table: 'managed_key_references', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.SECURITY, + retentionDays: Retention.INDEFINITE, + retentionAnchor: null, + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'Opaque KMS handle — never key material. Retained because destroying the reference orphans any funds held under that key.', + }, + + // ── Money ───────────────────────────────────────────────────────────────── + { + model: 'Transaction', + table: 'Transaction', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.MONEY, + retentionDays: Retention.SEVEN_YEARS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'Ledger entry. Corrections are booked as new reversing entries, never as edits. Carries no in-row personal data, so erasure keeps it.', + }, + { + model: 'Wallet', + table: 'wallets', + recordClass: RecordClass.MUTABLE, + category: DataCategory.MONEY, + retentionDays: Retention.INDEFINITE, + retentionAnchor: null, + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'Custody state machine. Every status transition is audited because it changes who controls the funds.', + }, + { + model: 'WalletProvisioningJob', + table: 'wallet_provisioning_jobs', + recordClass: RecordClass.MUTABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.NINETY_DAYS, + retentionAnchor: 'updatedAt', + onErasure: ErasureAction.CASCADE, + audited: false, + notes: + 'Lease and retry bookkeeping for provisioning. The wallet it provisions carries the audit trail.', + }, + { + model: 'StellarFunding', + table: 'stellar_fundings', + recordClass: RecordClass.MUTABLE, + category: DataCategory.MONEY, + retentionDays: Retention.SEVEN_YEARS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'Keyed by Stellar public key rather than by user, so erasure severs the link without touching the row.', + }, + { + model: 'ReferralCode', + table: 'referral_codes', + recordClass: RecordClass.ARCHIVABLE, + category: DataCategory.MONEY, + retentionDays: Retention.ONE_YEAR, + retentionAnchor: 'archivedAt', + onErasure: ErasureAction.CASCADE, + audited: true, + notes: + 'Retiring a code must not break the Referral rows pointing at it, so retirement archives the code instead of deleting it.', + }, + { + model: 'Referral', + table: 'referrals', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.MONEY, + retentionDays: Retention.SEVEN_YEARS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'Records a bonus obligation between two accounts. Only the payout columns advance, and each advance is audited.', + }, + + // ── Credentials ─────────────────────────────────────────────────────────── + { + model: 'Credential', + table: 'Credential', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.CREDENTIAL, + retentionDays: Retention.INDEFINITE, + retentionAnchor: null, + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'A credential a third party may verify against the chain. Revocation appends a revocation record; the issuance row itself is permanent.', + }, + { + model: 'Completion', + table: 'Completion', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.CREDENTIAL, + retentionDays: Retention.INDEFINITE, + retentionAnchor: null, + onErasure: ErasureAction.DELETE, + audited: true, + notes: + 'Evidence behind a credential. Immutable while the account lives, but deleted on erasure because it reveals what a named learner studied.', + }, + + // ── Content ─────────────────────────────────────────────────────────────── + { + model: 'Module', + table: 'Module', + recordClass: RecordClass.ARCHIVABLE, + category: DataCategory.CONTENT, + retentionDays: Retention.INDEFINITE, + retentionAnchor: 'archivedAt', + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'Authored content, never purged: completions and credentials reference the module a learner actually took, so withdrawing it archives it.', + }, + + // ── Operational ─────────────────────────────────────────────────────────── + { + model: 'SyncEvent', + table: 'sync_events', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.NINETY_DAYS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.DELETE, + audited: false, + notes: + 'Idempotency journal for offline clients. Append-only by construction; the unique key is what makes replay safe.', + }, + { + model: 'EmailDelivery', + table: 'email_deliveries', + recordClass: RecordClass.MUTABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.THIRTY_DAYS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.DELETE, + audited: false, + notes: + 'Outbox row holding a rendered message body, and therefore personal data. Deleted on erasure and purged aggressively.', + }, + { + model: 'NotificationLog', + table: 'NotificationLog', + recordClass: RecordClass.MUTABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.THIRTY_DAYS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.DELETE, + audited: false, + notes: 'Rendered push payload. Same reasoning as EmailDelivery.', + }, + { + model: 'NotificationPreference', + table: 'NotificationPreference', + recordClass: RecordClass.MUTABLE, + category: DataCategory.IDENTITY, + retentionDays: Retention.INDEFINITE, + retentionAnchor: null, + onErasure: ErasureAction.CASCADE, + audited: false, + notes: + 'Channel opt-ins. Privacy-impacting consent lives in ConsentRecord, not here.', + }, + { + model: 'DeviceToken', + table: 'DeviceToken', + recordClass: RecordClass.DELETABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.NINETY_DAYS, + retentionAnchor: 'updatedAt', + onErasure: ErasureAction.DELETE, + audited: false, + notes: + 'Push handle for one device. Stale handles are deleted, never archived — an archived token would still be a live address.', + }, + { + model: 'WebhookEndpoint', + table: 'WebhookEndpoint', + recordClass: RecordClass.ARCHIVABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.ONE_YEAR, + retentionAnchor: 'archivedAt', + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'Partner configuration holding a signing secret. Archived so the delivery history keeps its endpoint; audited because changing the URL redirects learner data.', + }, + { + model: 'WebhookDelivery', + table: 'WebhookDelivery', + recordClass: RecordClass.MUTABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.THIRTY_DAYS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.RETAIN, + audited: false, + notes: + 'Attempt log with request and response bodies. Short retention keeps that payload exposure bounded.', + }, + { + model: 'DataExportRequest', + table: 'data_export_requests', + recordClass: RecordClass.DELETABLE, + category: DataCategory.IDENTITY, + retentionDays: Retention.SEVEN_DAYS, + retentionAnchor: 'completedAt', + onErasure: ErasureAction.DELETE, + audited: true, + notes: + 'The artifact is a full personal-data dump — the highest-value row in the schema. Shortest retention of anything here, and every state change is audited.', + }, + { + model: 'AccountDeletionRequest', + table: 'account_deletion_requests', + recordClass: RecordClass.MUTABLE, + category: DataCategory.IDENTITY, + retentionDays: Retention.SEVEN_YEARS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.RETAIN, + audited: true, + notes: + 'Evidence that an erasure request was honoured. Retained past the erasure it triggers, which is why it must hold no personal data beyond the user id.', + }, + { + model: 'OutboxEvent', + table: 'outbox_events', + recordClass: RecordClass.MUTABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.THIRTY_DAYS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.RETAIN, + audited: false, + notes: + 'Delivery status advances until PUBLISHED or DEAD_LETTER. The payload is domain data, not an audit record — audit events are written separately.', + }, + { + model: 'JobAttempt', + table: 'job_attempts', + recordClass: RecordClass.MUTABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.THIRTY_DAYS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.CASCADE, + audited: false, + notes: 'Lease and retry state. Purged with its outbox event.', + }, + { + model: 'RolledBackRecord', + table: 'rolled_back_records', + recordClass: RecordClass.IMMUTABLE, + category: DataCategory.OPERATIONAL, + retentionDays: Retention.THIRTY_DAYS, + retentionAnchor: 'createdAt', + onErasure: ErasureAction.RETAIN, + audited: false, + notes: 'Tombstone marking an event as unprocessable. Written once, then only read.', + }, +] + +const BY_MODEL: ReadonlyMap = new Map( + RULES.map((rule) => [rule.model, rule]) +) + +/** Every rule in the matrix, in declaration order. */ +export function lifecycleRules(): readonly LifecycleRule[] { + return RULES +} + +/** The rule for a model, or `undefined` if the model is unclassified. */ +export function lifecycleRuleFor(model: string): LifecycleRule | undefined { + return BY_MODEL.get(model) +} + +/** + * Lifecycle class of a model. Unclassified models fall back to MUTABLE so a + * missing rule degrades to the least surprising behaviour instead of throwing + * from inside an audit write. + */ +export function recordClassFor(model: string): RecordClassValue { + return BY_MODEL.get(model)?.recordClass ?? RecordClass.MUTABLE +} + +/** Models whose mutations must go through an audited mutation. */ +export function auditedModels(): readonly string[] { + return RULES.filter((rule) => rule.audited).map((rule) => rule.model) +} + +/** Whether mutations of a model must be audited. */ +export function requiresAudit(model: string): boolean { + return BY_MODEL.get(model)?.audited ?? false +} + +/** Models in a given lifecycle class. */ +export function modelsInClass(recordClass: RecordClassValue): readonly string[] { + return RULES.filter((rule) => rule.recordClass === recordClass).map((rule) => rule.model) +} + +/** + * Cut-off before which a model's rows are eligible for a retention purge, or + * `null` when the model is retained indefinitely. + */ +export function retentionCutoff(model: string, now: Date = new Date()): Date | null { + const rule = BY_MODEL.get(model) + if (!rule || rule.retentionDays === null) { + return null + } + + return new Date(now.getTime() - rule.retentionDays * 24 * 60 * 60_000) +} diff --git a/src/audit/index.ts b/src/audit/index.ts new file mode 100644 index 0000000..639a41b --- /dev/null +++ b/src/audit/index.ts @@ -0,0 +1,24 @@ +/** + * Auditable Data Lifecycle module. + * + * Four pieces, each answering one requirement of the policy: + * + * - classification.ts — every model classified as mutable, archivable, + * deletable or immutable, with a retention window and an erasure behaviour. + * - audit-event.service.ts — the append-only audit trail. + * - audited-mutation.ts — the helper that makes a change and its audit event + * commit together. + * - archive.ts — soft deletion, and the rule that archived rows are excluded + * from reads by default. + * - redaction.ts — the filter that keeps secrets and unnecessary PII out of + * the trail, applied on the way in because immutable rows cannot be scrubbed. + * + * See docs/DATA_LIFECYCLE.md for the matrix and the reasoning behind it. + */ + +export * from './types.js' +export * from './classification.js' +export * from './redaction.js' +export * from './archive.js' +export * from './audit-event.service.js' +export * from './audited-mutation.js' diff --git a/src/audit/redaction.ts b/src/audit/redaction.ts new file mode 100644 index 0000000..d9f0502 --- /dev/null +++ b/src/audit/redaction.ts @@ -0,0 +1,419 @@ +/** + * Redaction filter for audit metadata. + * + * Audit rows are immutable, so anything written into them cannot be scrubbed + * later. That inverts the usual defence: metadata is filtered on the way *in*, + * and the filter is deny-by-default for anything that looks like a secret or an + * identifier of a natural person. + * + * Two independent passes run over every value: + * + * 1. Key matching — a field named `password`, `refreshToken`, `email`, … is + * replaced regardless of what it holds. + * 2. Value matching — a *value* that looks like a Stellar seed, a JWT, an + * email address or a long opaque blob is replaced even under an innocuous + * key, because the caller may have nested a secret somewhere unexpected. + * + * Structural caps (depth, breadth, string length, serialized size) then bound + * how much a caller can push into the audit trail at all. + */ + +import { createHmac } from 'crypto' + +/** Marker written in place of a redacted value. */ +export const REDACTED = '[REDACTED]' + +/** Marker written where a structural cap truncated the input. */ +export const TRUNCATED = '[TRUNCATED]' + +/** Structural caps applied to every metadata object. */ +export const RedactionLimits = { + /** Nesting levels kept; deeper values collapse to TRUNCATED. */ + maxDepth: 4, + /** Array entries kept per array. */ + maxArrayLength: 20, + /** Object keys kept per object. */ + maxKeys: 32, + /** Characters kept per string value. */ + maxStringLength: 256, + /** Bytes of serialized JSON kept for the whole object. */ + maxSerializedBytes: 4096, +} as const + +/** + * Field names that are always replaced. Compared after normalizing the key to + * lowercase alphanumerics, so `user_agent`, `userAgent` and `USERAGENT` all + * match the same entry. + */ +const DENIED_KEYS: ReadonlySet = new Set([ + // Authentication material + 'password', + 'passwordhash', + 'newpassword', + 'currentpassword', + 'confirmpassword', + 'pin', + 'otp', + 'otpcode', + 'codehash', + 'verificationcode', + 'resetcode', + 'salt', + 'signature', + 'sig', + 'nonce', + // Direct identifiers of a natural person. + // + // Note what is *not* here: `to`. It is the obvious name for an email + // recipient, but it is also the standard name for the destination of a status + // transition — the single most common thing audit metadata records. The + // value-level email pattern catches an actual recipient either way. + 'email', + 'emailaddress', + 'recipient', + 'phone', + 'phonenumber', + 'msisdn', + 'username', + 'fullname', + 'firstname', + 'lastname', + 'middlename', + 'displayname', + 'dob', + 'dateofbirth', + 'birthdate', + 'address', + 'street', + 'postcode', + 'zip', + 'ssn', + 'nin', + 'bvn', + 'taxid', + 'passportnumber', + // Payment instruments + 'cvv', + 'pan', + 'cardnumber', + 'iban', + 'accountnumber', + 'routingnumber', + // Request fingerprinting + 'ip', + 'ipaddress', + 'useragent', + 'fingerprint', + 'latitude', + 'longitude', + 'geo', + // Message bodies, which carry whatever the template rendered + 'body', + 'html', + 'text', + 'payload', + 'artifact', +]) + +/** + * Substrings that deny a key wherever they appear in it. Kept deliberately + * narrow: `secret`, `token` and friends have no legitimate use in audit + * metadata, whereas a broad pattern like `code` would eat `statusCode`, + * `failureCode` and `referralCode`, which reviewers genuinely need. + */ +const DENIED_KEY_PATTERNS: readonly string[] = [ + 'password', + 'passphrase', + 'secret', + 'token', + 'apikey', + 'authorization', + 'credential', + 'privatekey', + 'publickeyseed', + 'seedphrase', + 'mnemonic', + 'cookie', + 'bearer', + 'jwt', +] + +/** + * Value shapes that are replaced under any key. Ordered cheapest-first; a value + * matching any of them is a secret or a direct identifier regardless of where + * the caller put it. + */ +const DENIED_VALUE_PATTERNS: readonly RegExp[] = [ + // Stellar secret seed — the single most damaging string in this system. + /\bS[A-Z2-7]{55}\b/, + // Stellar public key. Not a secret, but it links a person to on-chain history. + /\bG[A-Z2-7]{55}\b/, + // JWT / compact JWS. + /\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]*/, + // `Bearer ` and friends. + /\b(?:bearer|basic)\s+[A-Za-z0-9._~+/=-]{12,}/i, + // Email address. + /[\w.+-]+@[\w-]+\.[\w.-]+/, + // E.164 phone number. + /(?:^|\s)\+[1-9]\d{7,14}(?=$|\s)/, + // IPv4 address. + /\b(?:\d{1,3}\.){3}\d{1,3}\b/, + // Opaque high-entropy blob: 40+ hex chars is a hash, a key or a raw token. + /\b[0-9a-f]{40,}\b/i, + // PEM block header. + /-----BEGIN [A-Z ]*PRIVATE KEY-----/, +] + +/** Normalize a key to lowercase alphanumerics for deny-list comparison. */ +function normalizeKey(key: string): string { + return key.toLowerCase().replace(/[^a-z0-9]/g, '') +} + +/** Whether a field name is denied by the key deny-list. */ +export function isDeniedKey(key: string): boolean { + const normalized = normalizeKey(key) + + if (DENIED_KEYS.has(normalized)) { + return true + } + + return DENIED_KEY_PATTERNS.some((pattern) => normalized.includes(pattern)) +} + +/** Whether a string value looks like a secret or a direct identifier. */ +export function isDeniedValue(value: string): boolean { + return DENIED_VALUE_PATTERNS.some((pattern) => pattern.test(value)) +} + +/** Outcome of redacting a metadata object. */ +export interface RedactionResult { + /** + * The safe object, or `null` when the input was empty. Carries a `_redacted` + * key listing the paths that were replaced, so a reviewer reading the audit + * trail can see that redaction happened rather than guessing. + */ + value: Record | null + /** Dotted paths that were replaced, in traversal order. */ + redactedPaths: string[] + /** Whether a structural cap discarded part of the input. */ + truncated: boolean +} + +/** + * Redact a metadata object for storage in an audit event. + * + * Never throws: a value that cannot be serialized (a circular reference, a + * BigInt, a function) is replaced rather than propagated, because a failure here + * would take down the mutation being audited. + */ +export function redactMetadata( + input: Record | null | undefined +): RedactionResult { + if (input === null || input === undefined) { + return { value: null, redactedPaths: [], truncated: false } + } + + const redactedPaths: string[] = [] + const state = { truncated: false } + const seen = new WeakSet() + + const walk = (value: unknown, path: string, depth: number): unknown => { + if (value === null || value === undefined) { + return null + } + + if (depth > RedactionLimits.maxDepth) { + state.truncated = true + + return TRUNCATED + } + + switch (typeof value) { + case 'string': { + if (isDeniedValue(value)) { + redactedPaths.push(path) + + return REDACTED + } + if (value.length > RedactionLimits.maxStringLength) { + state.truncated = true + + return `${value.slice(0, RedactionLimits.maxStringLength)}${TRUNCATED}` + } + + return value + } + + case 'number': + return Number.isFinite(value) ? value : null + + case 'boolean': + return value + + // A BigInt is not JSON-serializable, so render it as a decimal string. + // Stroop amounts arrive this way and are safe to keep. + case 'bigint': + return value.toString() + + // A function or symbol in audit metadata is always a caller mistake. + case 'function': + case 'symbol': + redactedPaths.push(path) + + return REDACTED + + default: + break + } + + if (value instanceof Date) { + return value.toISOString() + } + + if (value instanceof Error) { + // Keep the class and message; a stack trace can embed request payloads. + return { name: value.name, message: walk(value.message, `${path}.message`, depth + 1) } + } + + if (seen.has(value as object)) { + state.truncated = true + + return TRUNCATED + } + seen.add(value as object) + + if (Array.isArray(value)) { + const kept = value.slice(0, RedactionLimits.maxArrayLength) + if (value.length > kept.length) { + state.truncated = true + } + + return kept.map((entry, index) => walk(entry, `${path}[${index}]`, depth + 1)) + } + + const entries = Object.entries(value as Record) + const kept = entries.slice(0, RedactionLimits.maxKeys) + if (entries.length > kept.length) { + state.truncated = true + } + + const output: Record = {} + for (const [key, entry] of kept) { + const childPath = path ? `${path}.${key}` : key + + if (isDeniedKey(key)) { + redactedPaths.push(childPath) + output[key] = REDACTED + continue + } + + output[key] = walk(entry, childPath, depth + 1) + } + + return output + } + + const safe = walk(input, '', 0) as Record + + if (redactedPaths.length > 0) { + safe._redacted = redactedPaths + } + + return { value: safe, redactedPaths, truncated: state.truncated } +} + +/** + * Serialize redacted metadata to the JSON string stored on the audit row, or + * `null` when there is nothing to store. Enforces the serialized-size cap by + * dropping the payload rather than storing a truncated, unparseable prefix. + */ +export function serializeMetadata( + input: Record | null | undefined +): string | null { + const { value, truncated } = redactMetadata(input) + + if (value === null || Object.keys(value).length === 0) { + return null + } + + let serialized: string + try { + serialized = JSON.stringify(value) + } catch { + return JSON.stringify({ _redacted: ['*'], _reason: 'unserializable' }) + } + + if (Buffer.byteLength(serialized, 'utf8') > RedactionLimits.maxSerializedBytes) { + return JSON.stringify({ + _truncated: true, + _reason: 'metadata exceeded size limit', + _keys: Object.keys(value).slice(0, RedactionLimits.maxKeys), + }) + } + + if (truncated) { + // Surface that a cap fired so a reviewer does not read the row as complete. + return JSON.stringify({ ...value, _truncated: true }) + } + + return serialized +} + +/** + * Keyed hash of a request IP, for correlating events from one source without + * storing the address. + * + * HMAC rather than a bare digest: the IPv4 space is small enough to enumerate, + * so an unkeyed hash is reversible in seconds. Rotating the secret makes older + * hashes uncorrelatable with newer ones, which is the intended trade-off. + */ +export function hashIpAddress( + ipAddress: string | null | undefined, + secret: string +): string | null { + if (!ipAddress) { + return null + } + + return createHmac('sha256', secret).update(ipAddress.trim()).digest('hex').slice(0, 32) +} + +/** + * Reduce a User-Agent to a coarse family label. Enough to tell "the admin + * console" from "the mobile app" in an investigation, not enough to fingerprint + * a device. + */ +export function userAgentFamily(userAgent: string | null | undefined): string | null { + if (!userAgent) { + return null + } + + const ua = userAgent.toLowerCase() + + // Order matters: Edge and Opera both advertise Chrome, Chrome advertises + // Safari, and every mobile web view advertises something else entirely. + const families: readonly [string, string][] = [ + ['edg/', 'Edge'], + ['opr/', 'Opera'], + ['firefox/', 'Firefox'], + ['chrome/', 'Chrome'], + ['safari/', 'Safari'], + ['okhttp', 'Android'], + ['cfnetwork', 'iOS'], + ['dart:io', 'Flutter'], + ['curl/', 'curl'], + ['postman', 'Postman'], + ['node', 'Node'], + ['axios', 'Node'], + ['bot', 'Bot'], + ['spider', 'Bot'], + ] + + for (const [needle, family] of families) { + if (ua.includes(needle)) { + return family + } + } + + return 'Other' +} diff --git a/src/audit/types.ts b/src/audit/types.ts new file mode 100644 index 0000000..afff5cc --- /dev/null +++ b/src/audit/types.ts @@ -0,0 +1,186 @@ +/** + * Auditable Data Lifecycle — type definitions. + * + * The lifecycle policy answers four questions for every persisted record: + * 1. What class is it? (mutable / archivable / deletable / immutable) + * 2. How long do we keep it? (retention) + * 3. What happens on a subject's erasure request? + * 4. Must mutations of it be audited? + * + * See docs/DATA_LIFECYCLE.md for the authoritative matrix and rationale. + */ + +/** + * Lifecycle class of a record. + * + * - MUTABLE: Updated in place. History, where it matters, lives in audit events. + * - ARCHIVABLE: Soft-deleted by stamping `archivedAt`. Rows survive so dependent + * records keep their referent; reads exclude them by default. + * - DELETABLE: Safe to hard-delete once expired or on erasure. Holds no record + * that anything else depends on. + * - IMMUTABLE: Append-only. Never updated, and deleted only by a retention purge. + */ +export const RecordClass = { + MUTABLE: 'MUTABLE', + ARCHIVABLE: 'ARCHIVABLE', + DELETABLE: 'DELETABLE', + IMMUTABLE: 'IMMUTABLE', +} as const + +export type RecordClassValue = (typeof RecordClass)[keyof typeof RecordClass] + +/** + * Data category, which is what actually drives the retention period. Records in + * the same category are kept for the same reason (regulatory, contractual, or + * operational) and therefore for the same length of time. + */ +export const DataCategory = { + /** Account identity and profile data for a natural person. */ + IDENTITY: 'IDENTITY', + /** Money movement: ledger entries, payouts, bonuses, funding. */ + MONEY: 'MONEY', + /** Earned credentials and the completions that back them. */ + CREDENTIAL: 'CREDENTIAL', + /** Authentication material and security-relevant events. */ + SECURITY: 'SECURITY', + /** Proof of consent and privacy-preference history. */ + CONSENT: 'CONSENT', + /** Authored learning content and learner-supplied media. */ + CONTENT: 'CONTENT', + /** Queues, delivery logs, sync journals — infrastructure bookkeeping. */ + OPERATIONAL: 'OPERATIONAL', +} as const + +export type DataCategoryValue = (typeof DataCategory)[keyof typeof DataCategory] + +/** + * What happens to a record when the subject's erasure request is finalized. + * + * - DELETE: Row is hard-deleted. + * - ANONYMIZE: Row survives with identifying columns overwritten (tombstone), + * so retained foreign keys stay valid. + * - RETAIN: Row is kept as-is because it carries no in-row PII and there is + * an overriding obligation to keep it (money, credentials). + * - CASCADE: Row disappears with its parent via a database cascade. + */ +export const ErasureAction = { + DELETE: 'DELETE', + ANONYMIZE: 'ANONYMIZE', + RETAIN: 'RETAIN', + CASCADE: 'CASCADE', +} as const + +export type ErasureActionValue = (typeof ErasureAction)[keyof typeof ErasureAction] + +/** One row of the lifecycle matrix. */ +export interface LifecycleRule { + /** Prisma model name. */ + model: string + /** Physical table name, as the migrations create it. */ + table: string + recordClass: RecordClassValue + category: DataCategoryValue + /** + * Days a row is kept before a retention purge may remove it, counted from the + * anchor column. `null` means "retain indefinitely" — nothing purges it. + */ + retentionDays: number | null + /** Column the retention clock runs from (e.g. `createdAt`, `archivedAt`). */ + retentionAnchor: string | null + onErasure: ErasureActionValue + /** Whether mutations of this record must go through an audited mutation. */ + audited: boolean + /** Why this classification — read by reviewers, not by code. */ + notes: string +} + +/** Who caused an audited change. */ +export const ActorType = { + /** An end user acting on their own data. */ + USER: 'USER', + /** A staff operator acting on someone else's data. */ + ADMIN: 'ADMIN', + /** An automated in-process action with no human trigger (sweeps, migrations). */ + SYSTEM: 'SYSTEM', + /** A background worker draining a queue. */ + WORKER: 'WORKER', + /** An unauthenticated caller (e.g. a failed login attempt). */ + ANONYMOUS: 'ANONYMOUS', +} as const + +export type ActorTypeValue = (typeof ActorType)[keyof typeof ActorType] + +export interface AuditActor { + type: ActorTypeValue + /** Opaque identifier. Omitted for SYSTEM and ANONYMOUS actors. */ + id?: string | null + /** Role held at the time of the action, not the role held now. */ + role?: string | null +} + +export interface AuditTarget { + /** Prisma model name of the affected record, e.g. `"User"`. */ + type: string + /** Primary key of the affected row, when there is a single one. */ + id?: string | null +} + +/** + * An audit event to be appended. Everything here is either non-identifying or + * passed through the redaction filter before it reaches the database. + */ +export interface AuditEventInput { + action: string + actor: AuditActor + target: AuditTarget + /** Caller-supplied justification. Required for ADMIN actors by convention. */ + reason?: string | null + /** `x-request-id`, correlating the event to request logs. */ + requestId?: string | null + /** Outbox event or job id when the change was applied asynchronously. */ + correlationId?: string | null + /** Code path that produced the event, e.g. `"api.account.deactivate"`. */ + source?: string | null + /** Free-form context. Redacted before it is written — never pass secrets. */ + metadata?: Record | null + /** Raw request IP. Stored only as a keyed hash, never in the clear. */ + ipAddress?: string | null + /** Raw User-Agent. Stored only as a coarse family label. */ + userAgent?: string | null + /** Lifecycle class of the target, defaulted from the matrix when omitted. */ + recordClass?: RecordClassValue +} + +/** The persisted shape of an audit event row. */ +export interface AuditEventRecord { + id: string + actorType: string + actorId: string | null + actorRole: string | null + action: string + recordClass: string + targetType: string + targetId: string | null + reason: string | null + requestId: string | null + correlationId: string | null + source: string | null + metadata: string | null + actorIpHash: string | null + userAgentFamily: string | null + occurredAt: Date +} + +/** Filter for reading the audit trail. */ +export interface AuditEventQuery { + actorId?: string + actorType?: ActorTypeValue + action?: string + targetType?: string + targetId?: string + requestId?: string + from?: Date + to?: Date + take?: number + skip?: number +} diff --git a/src/config/database.ts b/src/config/database.ts index 9ec2033..e4fbec2 100644 --- a/src/config/database.ts +++ b/src/config/database.ts @@ -1,6 +1,7 @@ import { PrismaClient } from '@prisma/client' import { PrismaPg } from '@prisma/adapter-pg' import { Pool } from 'pg' +import { archiveExclusionExtension } from '../audit/archive.js' const connectionString = process.env.DATABASE_URL ?? @@ -25,7 +26,19 @@ function createPrismaClient(): PrismaClient { const adapter = new PrismaPg(pool) - return new PrismaClient({ adapter }) + // Archived (soft-deleted) rows are excluded from list and aggregate reads + // here, at the client, rather than at each call site — one forgotten filter + // would otherwise leak withdrawn content. See src/audit/archive.ts for the + // operations covered and how to opt out. + // + // The cast keeps the exported type as PrismaClient. $extends() narrows the + // nominal type (it drops $on and $use), but this extension only rewrites the + // `where` of a read: it adds no methods, removes none, and changes no result + // shape. Widening every injection site to a union of both client types is a + // worse trade — a union of overloaded $transaction signatures stops resolving. + return new PrismaClient({ adapter }).$extends( + archiveExclusionExtension + ) as unknown as PrismaClient } const prisma = globalForPrisma.prisma ?? createPrismaClient() diff --git a/src/config/env.ts b/src/config/env.ts index c2c1c2d..b789b9a 100644 --- a/src/config/env.ts +++ b/src/config/env.ts @@ -36,4 +36,10 @@ export const env = { DELETION_COOLING_OFF_DAYS: parseInt(process.env.DELETION_COOLING_OFF_DAYS || '30', 10), EXPORT_TTL_DAYS: parseInt(process.env.EXPORT_TTL_DAYS || '7', 10), LIFECYCLE_SWEEP_INTERVAL_MS: parseInt(process.env.LIFECYCLE_SWEEP_INTERVAL_MS || '0', 10), // 0 = disabled + + // Data lifecycle / audit configurations — see docs/DATA_LIFECYCLE.md + // HMAC key for the source-IP hash on audit events. Unset in production means + // audit events omit the IP hash entirely, rather than storing an unkeyed + // digest of a search space small enough to enumerate. + AUDIT_IP_HASH_SECRET: process.env.AUDIT_IP_HASH_SECRET || '', } \ No newline at end of file diff --git a/tests/audit/archive.test.ts b/tests/audit/archive.test.ts new file mode 100644 index 0000000..fccf868 --- /dev/null +++ b/tests/audit/archive.test.ts @@ -0,0 +1,261 @@ +import { describe, it, expect, vi } from 'vitest' +import { + ARCHIVABLE_MODELS, + activeOnly, + archivePatch, + archivedOnly, + archivedPurgeCutoff, + archiveExclusionExtension, + assertActive, + excludeArchivedFromReads, + includeArchived, + isArchived, + mentionsArchivedAt, + restorePatch, +} from '../../src/audit/archive' + +/** Run the interceptor and return the args it forwarded to the actual query. */ +async function forwardedArgs( + model: string | undefined, + operation: string, + args: unknown +): Promise> { + const query = vi.fn().mockResolvedValue(null) + await excludeArchivedFromReads({ model, operation, args, query }) + + return query.mock.calls[0][0] as Record +} + +describe('archive semantics', () => { + describe('ARCHIVABLE_MODELS', () => { + it('is derived from the lifecycle matrix, not a second hand-kept list', () => { + expect([...ARCHIVABLE_MODELS].sort()).toEqual([ + 'Avatar', + 'LearnerProfile', + 'Module', + 'ReferralCode', + 'WebhookEndpoint', + ]) + }) + }) + + describe('extension wiring', () => { + it('registers the tested interceptor for every model and operation', () => { + // Without this, the extension object could drift from the function the + // rest of this file exercises and nothing would fail. + expect(archiveExclusionExtension).toMatchObject({ + name: 'archiveExclusion', + query: { $allModels: { $allOperations: excludeArchivedFromReads } }, + }) + }) + }) + + describe('default exclusion', () => { + it('hides archived rows from findMany on an archivable model', async () => { + expect(await forwardedArgs('Module', 'findMany', { where: { category: 'stellar' } })).toEqual( + { where: { category: 'stellar', archivedAt: null } } + ) + }) + + it('adds the filter when there is no where clause at all', async () => { + expect(await forwardedArgs('Module', 'findMany', {})).toEqual({ + where: { archivedAt: null }, + }) + }) + + it('adds the filter when args are absent entirely', async () => { + expect(await forwardedArgs('Module', 'findMany', undefined)).toEqual({ + where: { archivedAt: null }, + }) + }) + + it.each(['findFirst', 'findFirstOrThrow', 'findMany', 'count', 'aggregate', 'groupBy'])( + 'filters %s', + async (operation) => { + const args = await forwardedArgs('Module', operation, {}) + + expect(args.where).toEqual({ archivedAt: null }) + } + ) + + it('preserves other arguments while injecting the filter', async () => { + const args = await forwardedArgs('Module', 'findMany', { + orderBy: { createdAt: 'desc' }, + take: 10, + select: { id: true }, + }) + + expect(args).toEqual({ + orderBy: { createdAt: 'desc' }, + take: 10, + select: { id: true }, + where: { archivedAt: null }, + }) + }) + }) + + describe('exemptions', () => { + it('leaves non-archivable models alone', async () => { + // Injecting archivedAt here would reference a column that does not exist. + expect(await forwardedArgs('User', 'findMany', { where: { status: 'ACTIVE' } })).toEqual({ + where: { status: 'ACTIVE' }, + }) + }) + + it('leaves findUnique alone, so a point lookup by id still resolves', async () => { + // A silent filter here would turn a found row into null and read as + // "deleted" to code that has the id in hand. + expect(await forwardedArgs('Module', 'findUnique', { where: { id: 'm-1' } })).toEqual({ + where: { id: 'm-1' }, + }) + }) + + it('leaves writes alone, so archive and restore can see their own row', async () => { + for (const operation of ['update', 'updateMany', 'delete', 'deleteMany', 'upsert']) { + expect(await forwardedArgs('Module', operation, { where: { id: 'm-1' } })).toEqual({ + where: { id: 'm-1' }, + }) + } + }) + + it('leaves raw and model-less operations alone', async () => { + expect(await forwardedArgs(undefined, 'findMany', { where: { id: 'x' } })).toEqual({ + where: { id: 'x' }, + }) + }) + + it('stands down when the caller filters on archivedAt explicitly', async () => { + expect( + await forwardedArgs('Module', 'findMany', { where: { archivedAt: { not: null } } }) + ).toEqual({ where: { archivedAt: { not: null } } }) + }) + + it('stands down for includeArchived, which opts out by naming the key', async () => { + const args = await forwardedArgs('Module', 'findMany', { + where: includeArchived({ category: 'stellar' }), + }) + + // Prisma ignores an undefined filter, so this reads both live and + // archived rows — and the opt-out is visible at the call site. + expect(args.where).toEqual({ category: 'stellar', archivedAt: undefined }) + }) + + it('stands down when archivedAt appears inside a combinator', async () => { + const where = { OR: [{ archivedAt: null }, { archivedAt: { gt: new Date(0) } }] } + + expect(await forwardedArgs('Module', 'findMany', { where })).toEqual({ where }) + }) + }) + + describe('mentionsArchivedAt', () => { + it('detects the key at the top level', () => { + expect(mentionsArchivedAt({ archivedAt: null })).toBe(true) + }) + + it('detects the key set to undefined, which is how opting out works', () => { + expect(mentionsArchivedAt({ archivedAt: undefined })).toBe(true) + }) + + it.each(['AND', 'OR', 'NOT'])('detects the key nested under %s', (combinator) => { + expect(mentionsArchivedAt({ [combinator]: [{ archivedAt: null }] })).toBe(true) + expect(mentionsArchivedAt({ [combinator]: { archivedAt: null } })).toBe(true) + }) + + it('returns false for an unrelated clause', () => { + expect(mentionsArchivedAt({ status: 'ACTIVE', AND: [{ title: 'x' }] })).toBe(false) + }) + + it('returns false for empty and non-object input', () => { + expect(mentionsArchivedAt(undefined)).toBe(false) + expect(mentionsArchivedAt(null)).toBe(false) + expect(mentionsArchivedAt('archivedAt')).toBe(false) + }) + }) + + describe('scope helpers', () => { + it('activeOnly restricts to live rows', () => { + expect(activeOnly({ category: 'stellar' })).toEqual({ + category: 'stellar', + archivedAt: null, + }) + }) + + it('archivedOnly restricts to archived rows', () => { + expect(archivedOnly({ category: 'stellar' })).toEqual({ + category: 'stellar', + archivedAt: { not: null }, + }) + }) + + it('works with no base clause', () => { + expect(activeOnly()).toEqual({ archivedAt: null }) + expect(archivedOnly()).toEqual({ archivedAt: { not: null } }) + expect(includeArchived()).toEqual({ archivedAt: undefined }) + }) + + it('does not mutate the clause it was given', () => { + const where = { category: 'stellar' } + activeOnly(where) + + expect(where).toEqual({ category: 'stellar' }) + }) + }) + + describe('patches', () => { + it('archivePatch stamps time, actor and reason', () => { + const now = new Date('2026-08-24T12:00:00.000Z') + + expect(archivePatch('withdrawn by author', 'admin-1', now)).toEqual({ + archivedAt: now, + archivedById: 'admin-1', + archivedReason: 'withdrawn by author', + }) + }) + + it('archivePatch tolerates a missing actor for system archives', () => { + expect(archivePatch('retention sweep').archivedById).toBeNull() + }) + + it('restorePatch clears all three columns', () => { + // Leaving a stale reason behind would make a live row look archived to + // anyone reading the columns rather than the timestamp. + expect(restorePatch()).toEqual({ + archivedAt: null, + archivedById: null, + archivedReason: null, + }) + }) + }) + + describe('record inspection', () => { + it('isArchived reads the timestamp', () => { + expect(isArchived({ archivedAt: new Date() })).toBe(true) + expect(isArchived({ archivedAt: null })).toBe(false) + expect(isArchived({})).toBe(false) + expect(isArchived(null)).toBe(false) + }) + + it('assertActive passes a live record through', () => { + const live = { id: 'm-1', archivedAt: null } + + expect(assertActive(live)).toBe(live) + }) + + it('assertActive nulls an archived record', () => { + expect(assertActive({ id: 'm-1', archivedAt: new Date() })).toBeNull() + expect(assertActive(null)).toBeNull() + }) + }) + + describe('archivedPurgeCutoff', () => { + it('subtracts the retention window', () => { + expect( + archivedPurgeCutoff(365, new Date('2026-08-24T00:00:00.000Z'))?.toISOString() + ).toBe('2025-08-24T00:00:00.000Z') + }) + + it('returns null for indefinite retention', () => { + expect(archivedPurgeCutoff(null)).toBeNull() + }) + }) +}) diff --git a/tests/audit/audit-event.service.test.ts b/tests/audit/audit-event.service.test.ts new file mode 100644 index 0000000..8b77397 --- /dev/null +++ b/tests/audit/audit-event.service.test.ts @@ -0,0 +1,406 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest' + +vi.mock('../../src/config/database', () => ({ + default: { + auditEvent: { + create: vi.fn(), + findMany: vi.fn(), + }, + $transaction: vi.fn(), + }, +})) + +vi.mock('../../src/utils/logger', () => ({ + default: { info: vi.fn(), warn: vi.fn(), error: vi.fn() }, +})) + +import prisma from '../../src/config/database' +import logger from '../../src/utils/logger' +import { + AUDIT_PURGE_SETTING, + AuditEventService, +} from '../../src/audit/audit-event.service' +import { REDACTED } from '../../src/audit/redaction' +import { ActorType, RecordClass } from '../../src/audit/types' + +describe('AuditEventService', () => { + let service: AuditEventService + + beforeEach(() => { + vi.resetAllMocks() + service = new AuditEventService() + }) + + describe('attribution', () => { + it('records actor, action, target, reason and request id', () => { + const row = service.toRow({ + action: 'account.deactivated', + actor: { type: ActorType.ADMIN, id: 'admin-1', role: 'ADMIN' }, + target: { type: 'User', id: 'user-9' }, + reason: 'support request #4821', + requestId: 'req-abc', + source: 'api.account.deactivate', + }) + + expect(row).toMatchObject({ + actorType: 'ADMIN', + actorId: 'admin-1', + actorRole: 'ADMIN', + action: 'account.deactivated', + targetType: 'User', + targetId: 'user-9', + reason: 'support request #4821', + requestId: 'req-abc', + source: 'api.account.deactivate', + }) + }) + + it('distinguishes the actor from the target, so acting-on-behalf-of is visible', () => { + const row = service.toRow({ + action: 'account.deactivated', + actor: { type: ActorType.ADMIN, id: 'admin-1', role: 'ADMIN' }, + target: { type: 'User', id: 'learner-2' }, + reason: 'abuse report', + }) + + // The whole point of separate columns: "an admin deactivated a learner" + // is a different event from "a learner deactivated themselves". + expect(row.actorId).toBe('admin-1') + expect(row.targetId).toBe('learner-2') + expect(row.actorId).not.toBe(row.targetId) + }) + + it('accepts a SYSTEM actor with no id', () => { + const row = service.toRow({ + action: 'export.purged', + actor: { type: ActorType.SYSTEM, id: 'lifecycle-sweep' }, + target: { type: 'DataExportRequest', id: 'exp-1' }, + }) + + expect(row.actorType).toBe('SYSTEM') + expect(row.actorRole).toBeNull() + }) + + it('stamps the lifecycle class of the target from the matrix', () => { + expect( + service.toRow({ + action: 'transaction.created', + actor: { type: ActorType.WORKER, id: 'reward' }, + target: { type: 'Transaction', id: 't-1' }, + }).recordClass + ).toBe(RecordClass.IMMUTABLE) + + expect( + service.toRow({ + action: 'module.archived', + actor: { type: ActorType.ADMIN, id: 'a-1' }, + target: { type: 'Module', id: 'm-1' }, + }).recordClass + ).toBe(RecordClass.ARCHIVABLE) + }) + + it('lets an explicit record class override the matrix', () => { + expect( + service.toRow({ + action: 'thing.changed', + actor: { type: ActorType.SYSTEM }, + target: { type: 'Unknown' }, + recordClass: RecordClass.DELETABLE, + }).recordClass + ).toBe(RecordClass.DELETABLE) + }) + + it('nulls every optional field rather than leaving it undefined', () => { + const row = service.toRow({ + action: 'login.failed', + actor: { type: ActorType.ANONYMOUS }, + target: { type: 'User' }, + }) + + expect(row).toEqual({ + actorType: 'ANONYMOUS', + actorId: null, + actorRole: null, + action: 'login.failed', + recordClass: RecordClass.MUTABLE, + targetType: 'User', + targetId: null, + reason: null, + requestId: null, + correlationId: null, + source: null, + metadata: null, + actorIpHash: null, + userAgentFamily: null, + }) + }) + }) + + describe('redaction on write', () => { + it('never stores a raw IP address', () => { + const row = service.toRow({ + action: 'login.succeeded', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'Session', id: 's-1' }, + ipAddress: '203.0.113.42', + }) + + expect(row.actorIpHash).not.toBeNull() + expect(row.actorIpHash).not.toContain('203') + expect(JSON.stringify(row)).not.toContain('203.0.113.42') + }) + + it('never stores a raw User-Agent', () => { + const row = service.toRow({ + action: 'login.succeeded', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'Session', id: 's-1' }, + userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120.0.6099.109', + }) + + expect(row.userAgentFamily).toBe('Chrome') + expect(JSON.stringify(row)).not.toContain('Windows NT') + expect(JSON.stringify(row)).not.toContain('6099') + }) + + it('redacts metadata secrets before they reach the row', () => { + const row = service.toRow({ + action: 'wallet.exported', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'Wallet', id: 'w-1' }, + metadata: { + walletId: 'w-1', + secretSeed: 'SBQWY3DNPFWGSZTFNV4WQZLBOJ7SFQNDBQFXHTOYIY5QYVCSFRCFUKPP', + email: 'learner@example.com', + }, + }) + + expect(row.metadata).not.toContain('SBQWY3DN') + expect(row.metadata).not.toContain('learner@example.com') + expect(row.metadata).toContain(REDACTED) + expect(JSON.parse(row.metadata!).walletId).toBe('w-1') + }) + + it('keeps the metadata parseable as JSON', () => { + const row = service.toRow({ + action: 'wallet.status_changed', + actor: { type: ActorType.WORKER, id: 'provisioning' }, + target: { type: 'Wallet', id: 'w-1' }, + metadata: { from: 'RESERVED', to: 'ACTIVE', attempt: 2 }, + }) + + expect(JSON.parse(row.metadata!)).toEqual({ from: 'RESERVED', to: 'ACTIVE', attempt: 2 }) + }) + }) + + describe('record', () => { + it('appends the event', async () => { + vi.mocked(prisma.auditEvent.create).mockResolvedValue({} as never) + + await service.record({ + action: 'account.deactivated', + actor: { type: ActorType.USER, id: 'u-1', role: 'LEARNER' }, + target: { type: 'User', id: 'u-1' }, + }) + + expect(prisma.auditEvent.create).toHaveBeenCalledWith({ + data: expect.objectContaining({ + action: 'account.deactivated', + actorId: 'u-1', + targetType: 'User', + }), + }) + }) + + it('swallows a write failure — standalone auditing must not break the caller', async () => { + vi.mocked(prisma.auditEvent.create).mockRejectedValue(new Error('db down')) + + await expect( + service.record({ + action: 'login.failed', + actor: { type: ActorType.ANONYMOUS }, + target: { type: 'User' }, + }) + ).resolves.toBeUndefined() + + expect(logger.error).toHaveBeenCalled() + }) + + it('does not log the metadata when a write fails', async () => { + vi.mocked(prisma.auditEvent.create).mockRejectedValue(new Error('db down')) + + await service.record({ + action: 'login.failed', + actor: { type: ActorType.ANONYMOUS }, + target: { type: 'User' }, + metadata: { attemptedEmail: 'learner@example.com' }, + }) + + // The rejected metadata is the least safe thing to route into logs. + const logged = JSON.stringify(vi.mocked(logger.error).mock.calls) + expect(logged).not.toContain('learner@example.com') + expect(logged).toContain('login.failed') + }) + }) + + describe('recordWithin', () => { + it('writes through the given transaction client', async () => { + const create = vi.fn().mockResolvedValue({}) + const tx = { auditEvent: { create } } + + await service.recordWithin(tx, { + action: 'user.anonymized', + actor: { type: ActorType.SYSTEM, id: 'deletion-sweep' }, + target: { type: 'User', id: 'u-1' }, + }) + + expect(create).toHaveBeenCalledOnce() + // Not the module-level client: the event must land in the caller's + // transaction or its atomicity guarantee is worthless. + expect(prisma.auditEvent.create).not.toHaveBeenCalled() + }) + + it('propagates a failure so the surrounding transaction rolls back', async () => { + const tx = { auditEvent: { create: vi.fn().mockRejectedValue(new Error('constraint')) } } + + await expect( + service.recordWithin(tx, { + action: 'user.anonymized', + actor: { type: ActorType.SYSTEM, id: 'sweep' }, + target: { type: 'User', id: 'u-1' }, + }) + ).rejects.toThrow('constraint') + }) + }) + + describe('immutability', () => { + it('exposes no way to update or delete a single event', () => { + const surface = [ + ...Object.getOwnPropertyNames(AuditEventService.prototype), + ...Object.keys(service), + ] + + // The database trigger is the real enforcement; this asserts the service + // offers no API that would tempt a caller to try. + expect(surface).not.toContain('update') + expect(surface).not.toContain('delete') + expect(surface).not.toContain('deleteMany') + expect(surface).not.toContain('redact') + expect(surface).not.toContain('scrub') + }) + + it('never calls a mutating Prisma operation on audit events', async () => { + const auditEvent = prisma.auditEvent as unknown as Record + + expect(auditEvent.update).toBeUndefined() + expect(auditEvent.delete).toBeUndefined() + expect(auditEvent.updateMany).toBeUndefined() + }) + }) + + describe('purgeExpired', () => { + it('sets the purge session variable the trigger checks, then deletes by cutoff', async () => { + const executeRawUnsafe = vi.fn().mockResolvedValue(0) + const executeRaw = vi.fn().mockResolvedValue(12) + + vi.mocked(prisma.$transaction).mockImplementation( + (async (callback: (tx: unknown) => Promise) => + callback({ $executeRawUnsafe: executeRawUnsafe, $executeRaw: executeRaw })) as never + ) + + const deleted = await service.purgeExpired(new Date('2026-08-24T00:00:00.000Z')) + + expect(deleted).toBe(12) + expect(executeRawUnsafe).toHaveBeenCalledWith(`SET LOCAL "${AUDIT_PURGE_SETTING}" = 'on'`) + expect(executeRaw).toHaveBeenCalledOnce() + }) + + it('deletes by timestamp only, so no single event can be targeted', async () => { + const executeRaw = vi.fn().mockResolvedValue(0) + + vi.mocked(prisma.$transaction).mockImplementation( + (async (callback: (tx: unknown) => Promise) => + callback({ $executeRawUnsafe: vi.fn(), $executeRaw: executeRaw })) as never + ) + + await service.purgeExpired(new Date('2026-08-24T00:00:00.000Z')) + + // The tagged-template call carries the SQL fragments as its first + // argument; assert the predicate is the retention cutoff and nothing else. + const fragments = (executeRaw.mock.calls[0][0] as string[]).join('?') + + expect(fragments).toContain('DELETE FROM "audit_events"') + expect(fragments).toContain('"occurredAt" <') + expect(fragments).not.toContain('"id"') + }) + + it('returns 0 and logs instead of throwing when the purge fails', async () => { + vi.mocked(prisma.$transaction).mockRejectedValue(new Error('deadlock')) + + await expect(service.purgeExpired()).resolves.toBe(0) + expect(logger.error).toHaveBeenCalled() + }) + }) + + describe('list', () => { + it('filters by actor, target and window, newest first', async () => { + vi.mocked(prisma.auditEvent.findMany).mockResolvedValue([] as never) + + const from = new Date('2026-08-01T00:00:00.000Z') + const to = new Date('2026-08-24T00:00:00.000Z') + + await service.list({ actorId: 'admin-1', targetType: 'User', from, to }) + + expect(prisma.auditEvent.findMany).toHaveBeenCalledWith({ + where: { actorId: 'admin-1', targetType: 'User', occurredAt: { gte: from, lte: to } }, + orderBy: { occurredAt: 'desc' }, + take: 50, + skip: 0, + }) + }) + + it('omits absent filters rather than sending undefined predicates', async () => { + vi.mocked(prisma.auditEvent.findMany).mockResolvedValue([] as never) + + await service.list() + + expect(prisma.auditEvent.findMany).toHaveBeenCalledWith( + expect.objectContaining({ where: {} }) + ) + }) + + it('caps the page size so the whole trail cannot be pulled at once', async () => { + vi.mocked(prisma.auditEvent.findMany).mockResolvedValue([] as never) + + await service.list({ take: 100_000 }) + + expect(prisma.auditEvent.findMany).toHaveBeenCalledWith( + expect.objectContaining({ take: 200 }) + ) + }) + + it('rejects a non-positive page size', async () => { + vi.mocked(prisma.auditEvent.findMany).mockResolvedValue([] as never) + + await service.list({ take: 0 }) + + expect(prisma.auditEvent.findMany).toHaveBeenCalledWith( + expect.objectContaining({ take: 1 }) + ) + }) + }) + + describe('historyFor', () => { + it('returns one record history oldest first', async () => { + vi.mocked(prisma.auditEvent.findMany).mockResolvedValue([] as never) + + await service.historyFor('Wallet', 'w-1') + + expect(prisma.auditEvent.findMany).toHaveBeenCalledWith({ + where: { targetType: 'Wallet', targetId: 'w-1' }, + orderBy: { occurredAt: 'asc' }, + take: 50, + }) + }) + }) +}) diff --git a/tests/audit/audited-mutation.test.ts b/tests/audit/audited-mutation.test.ts new file mode 100644 index 0000000..b607540 --- /dev/null +++ b/tests/audit/audited-mutation.test.ts @@ -0,0 +1,561 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest' + +vi.mock('../../src/config/database', () => ({ + default: { $transaction: vi.fn() }, +})) + +vi.mock('../../src/utils/logger', () => ({ + default: { info: vi.fn(), warn: vi.fn(), error: vi.fn() }, +})) + +import prisma from '../../src/config/database' +import { + AuditPolicyError, + actorFromRequest, + auditedArchive, + auditedMutation, + auditedRestore, + systemActor, + workerActor, +} from '../../src/audit/audited-mutation' +import { REDACTED } from '../../src/audit/redaction' +import { ActorType } from '../../src/audit/types' + +/** + * A stand-in transaction client. Records the order of operations, so a test can + * assert the audit row was written inside the same transaction as the mutation + * rather than after it. + */ +function fakeTransaction() { + const calls: string[] = [] + const auditCreate = vi.fn(async () => { + calls.push('audit') + + return {} + }) + + const tx = { auditEvent: { create: auditCreate } } + + vi.mocked(prisma.$transaction).mockImplementation( + (async (callback: (client: unknown) => Promise) => callback(tx)) as never + ) + + return { calls, auditCreate, tx } +} + +/** The audit row a fake transaction received. */ +function auditRow(auditCreate: ReturnType): Record { + return (auditCreate.mock.calls[0][0] as { data: Record }).data +} + +describe('auditedMutation', () => { + beforeEach(() => { + vi.resetAllMocks() + }) + + describe('atomicity', () => { + it('runs the mutation and the audit write in one transaction', async () => { + const { calls, auditCreate } = fakeTransaction() + + await auditedMutation({ + action: 'account.deactivated', + actor: { type: ActorType.USER, id: 'u-1', role: 'LEARNER' }, + target: { type: 'User', id: 'u-1' }, + mutate: async () => { + calls.push('mutate') + + return { id: 'u-1' } + }, + }) + + expect(prisma.$transaction).toHaveBeenCalledOnce() + expect(calls).toEqual(['mutate', 'audit']) + expect(auditCreate).toHaveBeenCalledOnce() + }) + + it('hands the mutation the transaction client, not the global one', async () => { + const { tx } = fakeTransaction() + const seen: unknown[] = [] + + await auditedMutation({ + action: 'user.updated', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'User', id: 'u-1' }, + mutate: async (client) => { + seen.push(client) + + return null + }, + }) + + // A write through the global client would commit outside the transaction + // and escape the audit's all-or-nothing guarantee. + expect(seen).toEqual([tx]) + }) + + it('propagates a failed audit write, so an unaudited change cannot land', async () => { + const auditCreate = vi.fn().mockRejectedValue(new Error('audit constraint')) + vi.mocked(prisma.$transaction).mockImplementation( + (async (callback: (client: unknown) => Promise) => + callback({ auditEvent: { create: auditCreate } })) as never + ) + + await expect( + auditedMutation({ + action: 'wallet.status_changed', + actor: { type: ActorType.WORKER, id: 'provisioning' }, + target: { type: 'Wallet', id: 'w-1' }, + mutate: async () => ({ id: 'w-1' }), + }) + ).rejects.toThrow('audit constraint') + }) + + it('does not write an audit event when the mutation itself fails', async () => { + const { auditCreate } = fakeTransaction() + + await expect( + auditedMutation({ + action: 'wallet.status_changed', + actor: { type: ActorType.WORKER, id: 'provisioning' }, + target: { type: 'Wallet', id: 'w-1' }, + mutate: async () => { + throw new Error('lease lost') + }, + }) + ).rejects.toThrow('lease lost') + + // Auditing a change that never happened is as wrong as missing one. + expect(auditCreate).not.toHaveBeenCalled() + }) + + it('returns the mutation result unchanged', async () => { + fakeTransaction() + + const result = await auditedMutation({ + action: 'user.updated', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'User', id: 'u-1' }, + mutate: async () => ({ id: 'u-1', status: 'DEACTIVATED' }), + }) + + expect(result).toEqual({ id: 'u-1', status: 'DEACTIVATED' }) + }) + }) + + describe('attribution', () => { + it('records actor, action, target, reason, request id and source', async () => { + const { auditCreate } = fakeTransaction() + + await auditedMutation({ + action: 'account.deactivated', + actor: { type: ActorType.ADMIN, id: 'admin-1', role: 'ADMIN' }, + target: { type: 'User', id: 'learner-2' }, + reason: 'abuse report #91', + requestId: 'req-1', + correlationId: 'evt-7', + source: 'api.admin.deactivate', + mutate: async () => null, + }) + + expect(auditRow(auditCreate)).toMatchObject({ + actorType: 'ADMIN', + actorId: 'admin-1', + actorRole: 'ADMIN', + action: 'account.deactivated', + targetType: 'User', + targetId: 'learner-2', + reason: 'abuse report #91', + requestId: 'req-1', + correlationId: 'evt-7', + source: 'api.admin.deactivate', + }) + }) + + it('resolves the target id from the result, for a create', async () => { + const { auditCreate } = fakeTransaction() + + await auditedMutation({ + action: 'wallet.reserved', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'Wallet' }, + mutate: async () => ({ id: 'w-new' }), + resolveTargetId: (result) => result.id, + }) + + expect(auditRow(auditCreate).targetId).toBe('w-new') + }) + + it('prefers an explicit target id over a resolved one', async () => { + const { auditCreate } = fakeTransaction() + + await auditedMutation({ + action: 'wallet.updated', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'Wallet', id: 'w-explicit' }, + mutate: async () => ({ id: 'w-resolved' }), + resolveTargetId: (result) => result.id, + }) + + expect(auditRow(auditCreate).targetId).toBe('w-explicit') + }) + + it('merges metadata resolved from the result', async () => { + const { auditCreate } = fakeTransaction() + + await auditedMutation({ + action: 'wallet.status_changed', + actor: { type: ActorType.WORKER, id: 'provisioning' }, + target: { type: 'Wallet', id: 'w-1' }, + metadata: { from: 'RESERVED' }, + mutate: async () => ({ status: 'ACTIVE' }), + resolveMetadata: (result) => ({ to: result.status }), + }) + + expect(JSON.parse(auditRow(auditCreate).metadata as string)).toEqual({ + from: 'RESERVED', + to: 'ACTIVE', + }) + }) + + it('redacts metadata before it reaches the row', async () => { + const { auditCreate } = fakeTransaction() + + await auditedMutation({ + action: 'password.changed', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'User', id: 'u-1' }, + metadata: { newPassword: 'hunter2', method: 'reset-link' }, + mutate: async () => null, + }) + + const metadata = auditRow(auditCreate).metadata as string + + expect(metadata).not.toContain('hunter2') + expect(JSON.parse(metadata)).toMatchObject({ newPassword: REDACTED, method: 'reset-link' }) + }) + + it('hashes the IP and coarsens the User-Agent it is given', async () => { + const { auditCreate } = fakeTransaction() + + await auditedMutation({ + action: 'session.revoked', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'Session', id: 's-1' }, + ipAddress: '198.51.100.7', + userAgent: 'Mozilla/5.0 (X11; Linux x86_64) Firefox/121.0', + mutate: async () => null, + }) + + const row = auditRow(auditCreate) + + expect(row.userAgentFamily).toBe('Firefox') + expect(JSON.stringify(row)).not.toContain('198.51.100.7') + expect(row.actorIpHash).toMatch(/^[0-9a-f]{32}$/) + }) + + it('stores null metadata when there is none, rather than an empty object', async () => { + const { auditCreate } = fakeTransaction() + + await auditedMutation({ + action: 'session.revoked', + actor: { type: ActorType.USER, id: 'u-1' }, + target: { type: 'Session', id: 's-1' }, + mutate: async () => null, + }) + + expect(auditRow(auditCreate).metadata).toBeNull() + }) + }) + + describe('policy enforcement', () => { + it('requires a reason from an ADMIN actor', async () => { + const { auditCreate } = fakeTransaction() + const mutate = vi.fn() + + await expect( + auditedMutation({ + action: 'account.deactivated', + actor: { type: ActorType.ADMIN, id: 'admin-1', role: 'ADMIN' }, + target: { type: 'User', id: 'learner-2' }, + mutate, + }) + ).rejects.toThrow(AuditPolicyError) + + // Rejected before anything ran, so there is nothing to roll back. + expect(mutate).not.toHaveBeenCalled() + expect(auditCreate).not.toHaveBeenCalled() + expect(prisma.$transaction).not.toHaveBeenCalled() + }) + + it('treats a blank reason as no reason', async () => { + fakeTransaction() + + await expect( + auditedMutation({ + action: 'account.deactivated', + actor: { type: ActorType.ADMIN, id: 'admin-1' }, + target: { type: 'User', id: 'learner-2' }, + reason: ' ', + mutate: async () => null, + }) + ).rejects.toThrow(AuditPolicyError) + }) + + it('does not require a reason from a SYSTEM or WORKER actor', async () => { + fakeTransaction() + + await expect( + auditedMutation({ + action: 'export.purged', + actor: systemActor('lifecycle-sweep'), + target: { type: 'DataExportRequest', id: 'e-1' }, + mutate: async () => null, + }) + ).resolves.toBeNull() + + await expect( + auditedMutation({ + action: 'wallet.provisioned', + actor: workerActor('wallet-provisioning'), + target: { type: 'Wallet', id: 'w-1' }, + mutate: async () => null, + }) + ).resolves.toBeNull() + }) + + it('rejects a USER or ADMIN actor with no id, which would be unattributable', async () => { + fakeTransaction() + + await expect( + auditedMutation({ + action: 'user.updated', + actor: { type: ActorType.USER }, + target: { type: 'User', id: 'u-1' }, + mutate: async () => null, + }) + ).rejects.toThrow(/unattributable/) + }) + + it('rejects an empty action or target type', async () => { + fakeTransaction() + + await expect( + auditedMutation({ + action: ' ', + actor: systemActor('sweep'), + target: { type: 'User', id: 'u-1' }, + mutate: async () => null, + }) + ).rejects.toThrow(AuditPolicyError) + + await expect( + auditedMutation({ + action: 'user.updated', + actor: systemActor('sweep'), + target: { type: '' }, + mutate: async () => null, + }) + ).rejects.toThrow(AuditPolicyError) + }) + }) +}) + +describe('auditedArchive', () => { + beforeEach(() => { + vi.resetAllMocks() + }) + + it('stamps the archive patch and audits it in one transaction', async () => { + const { auditCreate } = fakeTransaction() + let received: unknown + + await auditedArchive({ + model: 'Module', + id: 'm-1', + reason: 'superseded by v2', + actor: { type: ActorType.ADMIN, id: 'admin-1', role: 'ADMIN' }, + archive: async (_tx, patch) => { + received = patch + + return { id: 'm-1' } + }, + }) + + expect(received).toMatchObject({ + archivedById: 'admin-1', + archivedReason: 'superseded by v2', + }) + expect((received as { archivedAt: Date }).archivedAt).toBeInstanceOf(Date) + + expect(auditRow(auditCreate)).toMatchObject({ + action: 'module.archived', + targetType: 'Module', + targetId: 'm-1', + reason: 'superseded by v2', + recordClass: 'ARCHIVABLE', + }) + }) + + it('derives a snake_case action name from the model', async () => { + const { auditCreate } = fakeTransaction() + + await auditedArchive({ + model: 'LearnerProfile', + id: 'p-1', + reason: 'account deactivated', + actor: systemActor('lifecycle-sweep'), + archive: async () => null, + }) + + expect(auditRow(auditCreate).action).toBe('learner_profile.archived') + }) + + it('refuses to archive a model the matrix does not classify as archivable', async () => { + const archive = vi.fn() + + await expect( + auditedArchive({ + model: 'Transaction', + id: 't-1', + reason: 'mistake', + actor: { type: ActorType.ADMIN, id: 'admin-1', role: 'ADMIN' }, + archive, + }) + ).rejects.toThrow(/IMMUTABLE, not ARCHIVABLE/) + + expect(archive).not.toHaveBeenCalled() + }) + + it('refuses to archive an unclassified model', async () => { + await expect( + auditedArchive({ + model: 'SomeFutureModel', + id: 'x-1', + reason: 'because', + actor: systemActor('sweep'), + archive: async () => null, + }) + ).rejects.toThrow(/no rule in the lifecycle matrix/) + }) + + it('requires a reason regardless of actor type', async () => { + // Unlike a plain audited mutation, an archive always needs one: the column + // is NOT NULL-checked in the database and unreviewable without it. + await expect( + auditedArchive({ + model: 'Module', + id: 'm-1', + reason: ' ', + actor: systemActor('sweep'), + archive: async () => null, + }) + ).rejects.toThrow(/requires a reason/) + }) +}) + +describe('auditedRestore', () => { + beforeEach(() => { + vi.resetAllMocks() + }) + + it('clears the archive columns and audits the restore', async () => { + const { auditCreate } = fakeTransaction() + let received: unknown + + await auditedRestore({ + model: 'Module', + id: 'm-1', + reason: 'withdrawn in error', + actor: { type: ActorType.ADMIN, id: 'admin-1', role: 'ADMIN' }, + restore: async (_tx, patch) => { + received = patch + + return { id: 'm-1' } + }, + }) + + expect(received).toEqual({ + archivedAt: null, + archivedById: null, + archivedReason: null, + }) + expect(auditRow(auditCreate).action).toBe('module.restored') + }) + + it('refuses to restore a non-archivable model', async () => { + await expect( + auditedRestore({ + model: 'User', + id: 'u-1', + reason: 'x', + actor: systemActor('sweep'), + restore: async () => null, + }) + ).rejects.toThrow(/MUTABLE, not ARCHIVABLE/) + }) +}) + +describe('actorFromRequest', () => { + it('builds a USER actor from an authenticated learner request', () => { + expect( + actorFromRequest({ + actor: { id: 'u-1', role: 'LEARNER' }, + requestId: 'req-1', + ip: '203.0.113.5', + headers: { 'user-agent': 'curl/8.4.0' }, + }) + ).toEqual({ + actor: { type: ActorType.USER, id: 'u-1', role: 'LEARNER' }, + requestId: 'req-1', + ipAddress: '203.0.113.5', + userAgent: 'curl/8.4.0', + }) + }) + + it('builds an ADMIN actor from a staff request', () => { + // It is the actor's authority that decides the scrutiny, not the endpoint. + expect(actorFromRequest({ actor: { id: 'a-1', role: 'ADMIN' } }).actor).toEqual({ + type: ActorType.ADMIN, + id: 'a-1', + role: 'ADMIN', + }) + }) + + it('falls back to ANONYMOUS for an unauthenticated request', () => { + const context = actorFromRequest({ requestId: 'req-2' }) + + expect(context.actor).toEqual({ type: ActorType.ANONYMOUS }) + expect(context.requestId).toBe('req-2') + }) + + it('nulls a missing request id, ip and User-Agent', () => { + expect(actorFromRequest({})).toEqual({ + actor: { type: ActorType.ANONYMOUS }, + requestId: null, + ipAddress: null, + userAgent: null, + }) + }) + + it('ignores a non-string User-Agent header', () => { + expect( + actorFromRequest({ headers: { 'user-agent': ['a', 'b'] } }).userAgent + ).toBeNull() + }) +}) + +describe('actor constructors', () => { + it('systemActor names the component', () => { + expect(systemActor('lifecycle-sweep')).toEqual({ + type: ActorType.SYSTEM, + id: 'lifecycle-sweep', + }) + }) + + it('workerActor names the worker', () => { + expect(workerActor('wallet-provisioning')).toEqual({ + type: ActorType.WORKER, + id: 'wallet-provisioning', + }) + }) +}) diff --git a/tests/audit/classification.test.ts b/tests/audit/classification.test.ts new file mode 100644 index 0000000..356dc42 --- /dev/null +++ b/tests/audit/classification.test.ts @@ -0,0 +1,278 @@ +import { describe, it, expect } from 'vitest' +import { readFileSync } from 'fs' +import { join } from 'path' +import { + Retention, + lifecycleRuleFor, + lifecycleRules, + modelsInClass, + recordClassFor, + requiresAudit, + retentionCutoff, +} from '../../src/audit/classification' +import { DataCategory, ErasureAction, RecordClass } from '../../src/audit/types' + +/** Model names declared in prisma/schema.prisma. */ +function schemaModels(): string[] { + const schema = readFileSync(join(process.cwd(), 'prisma', 'schema.prisma'), 'utf8') + const matches = schema.matchAll(/^model\s+(\w+)\s*\{/gm) + + return [...matches].map((match) => match[1]) +} + +describe('lifecycle classification', () => { + describe('coverage', () => { + it('classifies every model in the Prisma schema', () => { + const unclassified = schemaModels().filter((model) => !lifecycleRuleFor(model)) + + // A model with no rule has no retention, no erasure behaviour and no + // audit requirement. Add it to src/audit/classification.ts and document + // it in docs/DATA_LIFECYCLE.md. + expect(unclassified).toEqual([]) + }) + + it('does not classify models that no longer exist in the schema', () => { + const models = new Set(schemaModels()) + const stale = lifecycleRules() + .map((rule) => rule.model) + .filter((model) => !models.has(model)) + + expect(stale).toEqual([]) + }) + + it('assigns each model exactly one rule', () => { + const seen = new Set() + const duplicates: string[] = [] + + for (const rule of lifecycleRules()) { + if (seen.has(rule.model)) duplicates.push(rule.model) + seen.add(rule.model) + } + + expect(duplicates).toEqual([]) + }) + + it('uses all four lifecycle classes', () => { + for (const recordClass of Object.values(RecordClass)) { + expect(modelsInClass(recordClass).length).toBeGreaterThan(0) + } + }) + }) + + describe('internal consistency', () => { + it('gives every purgeable model a retention anchor', () => { + const missing = lifecycleRules() + .filter((rule) => rule.retentionDays !== null && !rule.retentionAnchor) + .map((rule) => rule.model) + + expect(missing).toEqual([]) + }) + + it('anchors archivable models on archivedAt', () => { + const wrong = lifecycleRules() + .filter( + (rule) => + rule.recordClass === RecordClass.ARCHIVABLE && rule.retentionAnchor !== 'archivedAt' + ) + .map((rule) => rule.model) + + expect(wrong).toEqual([]) + }) + + it('states a rationale for every rule', () => { + const undocumented = lifecycleRules() + .filter((rule) => rule.notes.trim().length < 20) + .map((rule) => rule.model) + + expect(undocumented).toEqual([]) + }) + }) + + describe('policy invariants', () => { + it('retains money records for the statutory window and never deletes them on erasure', () => { + const money = lifecycleRules().filter((rule) => rule.category === DataCategory.MONEY) + + expect(money.length).toBeGreaterThan(0) + + for (const rule of money) { + // A ledger a subject can erase is not a ledger. Money rows either + // survive erasure outright or disappear with their parent. + expect([ErasureAction.RETAIN, ErasureAction.CASCADE]).toContain(rule.onErasure) + + if (rule.retentionDays !== null) { + expect(rule.retentionDays).toBeGreaterThanOrEqual(Retention.ONE_YEAR) + } + } + }) + + it('keeps credentials verifiable indefinitely', () => { + const credentials = lifecycleRules().filter( + (rule) => rule.category === DataCategory.CREDENTIAL + ) + + expect(credentials.length).toBeGreaterThan(0) + + for (const rule of credentials) { + expect(rule.recordClass).toBe(RecordClass.IMMUTABLE) + expect(rule.retentionDays).toBeNull() + } + }) + + it('retains consent proof beyond the account it describes', () => { + const consent = lifecycleRules().filter((rule) => rule.category === DataCategory.CONSENT) + + expect(consent.length).toBeGreaterThan(0) + + for (const rule of consent) { + expect(rule.recordClass).toBe(RecordClass.IMMUTABLE) + expect(rule.onErasure).toBe(ErasureAction.RETAIN) + expect(rule.retentionDays).toBe(Retention.SEVEN_YEARS) + } + }) + + it('bounds how long security events are kept', () => { + const security = lifecycleRules().filter( + (rule) => rule.category === DataCategory.SECURITY + ) + + expect(security.length).toBeGreaterThan(0) + + for (const rule of security) { + // Indefinite retention of security data is the failure mode this + // category exists to prevent. ManagedKeyReference is the one exception: + // destroying a key handle orphans the funds held under it. + if (rule.model === 'ManagedKeyReference') { + expect(rule.retentionDays).toBeNull() + continue + } + + expect(rule.retentionDays).not.toBeNull() + expect(rule.retentionDays!).toBeLessThanOrEqual(Retention.SEVEN_YEARS) + } + }) + + it('gives the export artifact the shortest retention in the schema', () => { + const shortest = Math.min( + ...lifecycleRules() + .map((rule) => rule.retentionDays) + .filter((days): days is number => days !== null) + ) + + expect(lifecycleRuleFor('DataExportRequest')!.retentionDays).toBe(shortest) + }) + + it('audits every money, credential and consent mutation', () => { + const sensitive = [DataCategory.MONEY, DataCategory.CREDENTIAL, DataCategory.CONSENT] + + // Append-only history tables are exempt: a row in one of them *is* the + // audit record, and auditing it would only produce a second row saying + // the first was written. + const selfAuditing = new Set(['AuditEvent', 'AuditLog', 'PreferenceAuditLog']) + + const unaudited = lifecycleRules() + .filter((rule) => sensitive.includes(rule.category) && !rule.audited) + .map((rule) => rule.model) + .filter((model) => !selfAuditing.has(model)) + + // JobAttempt and WalletProvisioningJob are OPERATIONAL, not MONEY, so + // queue bookkeeping is not swept up by this rule. + expect(unaudited).toEqual([]) + }) + + it('does not require the audit tables to audit themselves', () => { + expect(requiresAudit('AuditEvent')).toBe(false) + expect(requiresAudit('AuditLog')).toBe(false) + expect(requiresAudit('PreferenceAuditLog')).toBe(false) + }) + + it('keeps every self-auditing history table immutable', () => { + // The exemption above is only safe because these tables cannot be edited: + // an unaudited *and* mutable history table would be rewritable in silence. + for (const model of ['AuditEvent', 'AuditLog', 'PreferenceAuditLog']) { + expect(lifecycleRuleFor(model)!.recordClass).toBe(RecordClass.IMMUTABLE) + } + }) + }) + + describe('archive classification', () => { + it('marks exactly the models that carry archive columns', () => { + expect([...modelsInClass(RecordClass.ARCHIVABLE)].sort()).toEqual([ + 'Avatar', + 'LearnerProfile', + 'Module', + 'ReferralCode', + 'WebhookEndpoint', + ]) + }) + + it('declares archivedAt on every archivable model in the schema', () => { + const schema = readFileSync(join(process.cwd(), 'prisma', 'schema.prisma'), 'utf8') + + for (const model of modelsInClass(RecordClass.ARCHIVABLE)) { + const block = schema.match(new RegExp(`model\\s+${model}\\s*\\{([\\s\\S]*?)\\n\\}`)) + + expect(block, `model ${model} not found in schema`).not.toBeNull() + expect(block![1], `${model} is ARCHIVABLE but has no archivedAt`).toContain('archivedAt') + expect(block![1]).toContain('archivedById') + expect(block![1]).toContain('archivedReason') + } + }) + + it('does not declare archive columns on models outside the archivable class', () => { + const schema = readFileSync(join(process.cwd(), 'prisma', 'schema.prisma'), 'utf8') + const archivable = new Set(modelsInClass(RecordClass.ARCHIVABLE)) + + const unexpected = lifecycleRules() + .filter((rule) => !archivable.has(rule.model)) + .filter((rule) => { + const block = schema.match(new RegExp(`model\\s+${rule.model}\\s*\\{([\\s\\S]*?)\\n\\}`)) + + return block ? /^\s*archivedAt\s/m.test(block[1]) : false + }) + .map((rule) => rule.model) + + expect(unexpected).toEqual([]) + }) + }) + + describe('recordClassFor', () => { + it('resolves a classified model', () => { + expect(recordClassFor('Transaction')).toBe(RecordClass.IMMUTABLE) + expect(recordClassFor('Module')).toBe(RecordClass.ARCHIVABLE) + expect(recordClassFor('DeviceToken')).toBe(RecordClass.DELETABLE) + expect(recordClassFor('User')).toBe(RecordClass.MUTABLE) + }) + + it('falls back to MUTABLE for an unknown model rather than throwing', () => { + // Called from inside an audit write, so it must not be able to fail the + // mutation it is describing. + expect(recordClassFor('SomeFutureModel')).toBe(RecordClass.MUTABLE) + }) + }) + + describe('retentionCutoff', () => { + const now = new Date('2026-08-24T00:00:00.000Z') + + it('subtracts the retention window from now', () => { + expect(retentionCutoff('EmailDelivery', now)?.toISOString()).toBe( + '2026-07-25T00:00:00.000Z' + ) + }) + + it('returns null for indefinitely retained models', () => { + expect(retentionCutoff('Credential', now)).toBeNull() + expect(retentionCutoff('User', now)).toBeNull() + }) + + it('returns null for an unclassified model', () => { + expect(retentionCutoff('SomeFutureModel', now)).toBeNull() + }) + + it('puts the audit-event cutoff seven years back', () => { + const cutoff = retentionCutoff('AuditEvent', now)! + const years = (now.getTime() - cutoff.getTime()) / (365.25 * 24 * 60 * 60_000) + + expect(years).toBeCloseTo(7, 1) + }) + }) +}) diff --git a/tests/audit/redaction.test.ts b/tests/audit/redaction.test.ts new file mode 100644 index 0000000..4188725 --- /dev/null +++ b/tests/audit/redaction.test.ts @@ -0,0 +1,385 @@ +import { describe, it, expect } from 'vitest' +import { + REDACTED, + RedactionLimits, + TRUNCATED, + hashIpAddress, + isDeniedKey, + isDeniedValue, + redactMetadata, + serializeMetadata, + userAgentFamily, +} from '../../src/audit/redaction' + +describe('audit redaction', () => { + describe('key deny-list', () => { + it.each([ + 'password', + 'newPassword', + 'password_hash', + 'refreshToken', + 'accessToken', + 'tokenHash', + 'apiKey', + 'API_KEY', + 'authorization', + 'Cookie', + 'clientSecret', + 'privateKey', + 'mnemonic', + 'passphrase', + 'jwt', + 'signature', + 'codeHash', + 'otp', + ])('denies the secret-bearing key %s', (key) => { + expect(isDeniedKey(key)).toBe(true) + }) + + it.each([ + 'email', + 'emailAddress', + 'phone', + 'phoneNumber', + 'msisdn', + 'fullName', + 'dateOfBirth', + 'ipAddress', + 'userAgent', + 'fingerprint', + 'cardNumber', + 'ssn', + ])('denies the personal-identifier key %s', (key) => { + expect(isDeniedKey(key)).toBe(true) + }) + + it.each([ + 'statusCode', + 'failureCode', + 'errorCode', + 'referralCode', + 'attemptCount', + 'walletId', + 'moduleId', + 'requestId', + 'amountStroops', + 'assetCode', + 'status', + 'reason', + ])('allows the operational key %s', (key) => { + // Over-redaction is its own failure: an audit trail nobody can read is + // not reviewable. These are the keys investigators actually need. + expect(isDeniedKey(key)).toBe(false) + }) + + it('normalizes separators and case before matching', () => { + expect(isDeniedKey('USER_AGENT')).toBe(true) + expect(isDeniedKey('user-agent')).toBe(true) + expect(isDeniedKey('userAgent')).toBe(true) + }) + }) + + describe('value deny-list', () => { + it('denies a Stellar secret seed', () => { + expect(isDeniedValue('SBQWY3DNPFWGSZTFNV4WQZLBOJ7SFQNDBQFXHTOYIY5QYVCSFRCFUKPP')).toBe(true) + }) + + it('denies a Stellar public key', () => { + expect(isDeniedValue('GCKFBEIYV2U22IO2BJ4KVJOIP7XPWQGQFKKWXR6DOSJBV7STMAQSMTGG')).toBe(true) + }) + + it('denies a JWT', () => { + expect( + isDeniedValue('eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N_XgL0n3I9PlFUP0THsR8U') + ).toBe(true) + }) + + it('denies a bearer credential', () => { + expect(isDeniedValue('Bearer abcdef0123456789abcdef')).toBe(true) + }) + + it('denies an email address', () => { + expect(isDeniedValue('learner@example.com')).toBe(true) + }) + + it('denies an E.164 phone number', () => { + expect(isDeniedValue('call +2348012345678 now')).toBe(true) + }) + + it('denies an IPv4 address', () => { + expect(isDeniedValue('203.0.113.42')).toBe(true) + }) + + it('denies a long opaque hex blob', () => { + expect(isDeniedValue('a'.repeat(64))).toBe(true) + }) + + it('denies a PEM private key header', () => { + expect(isDeniedValue('-----BEGIN RSA PRIVATE KEY-----')).toBe(true) + }) + + it('allows an identifier, a status and a stroop amount', () => { + expect(isDeniedValue('550e8400-e29b-41d4-a716-446655440000')).toBe(false) + expect(isDeniedValue('PENDING_DELETION')).toBe(false) + expect(isDeniedValue('10000000')).toBe(false) + expect(isDeniedValue('api.account.deactivate')).toBe(false) + }) + }) + + describe('redactMetadata', () => { + it('returns null for empty input', () => { + expect(redactMetadata(null).value).toBeNull() + expect(redactMetadata(undefined).value).toBeNull() + }) + + it('keeps safe fields untouched', () => { + const { value, redactedPaths } = redactMetadata({ + walletId: 'w-1', + from: 'RESERVED', + to: 'ACTIVE', + attempt: 3, + terminal: false, + }) + + expect(value).toEqual({ + walletId: 'w-1', + from: 'RESERVED', + to: 'ACTIVE', + attempt: 3, + terminal: false, + }) + expect(redactedPaths).toEqual([]) + }) + + it('replaces a denied key without inspecting its value', () => { + const { value, redactedPaths } = redactMetadata({ password: 'hunter2', userId: 'u-1' }) + + expect(value).toMatchObject({ password: REDACTED, userId: 'u-1' }) + expect(redactedPaths).toEqual(['password']) + }) + + it('replaces a secret hiding under an innocuous key', () => { + // The key deny-list cannot catch this one; the value deny-list must. + const { value, redactedPaths } = redactMetadata({ + note: 'recovery is SBQWY3DNPFWGSZTFNV4WQZLBOJ7SFQNDBQFXHTOYIY5QYVCSFRCFUKPP', + }) + + expect(value!.note).toBe(REDACTED) + expect(redactedPaths).toEqual(['note']) + }) + + it('redacts inside nested objects and arrays, reporting the path', () => { + const { value, redactedPaths } = redactMetadata({ + session: { device: 'pixel-7', ipAddress: '203.0.113.9' }, + recipients: [{ email: 'a@b.com' }, { email: 'c@d.com' }], + }) + + expect((value!.session as Record).ipAddress).toBe(REDACTED) + expect(redactedPaths).toContain('session.ipAddress') + expect(redactedPaths).toContain('recipients[0].email') + expect(redactedPaths).toContain('recipients[1].email') + }) + + it('records that redaction happened, so a reader is not misled', () => { + const { value } = redactMetadata({ token: 'abc', keep: 'yes' }) + + expect(value!._redacted).toEqual(['token']) + }) + + it('omits the marker when nothing was redacted', () => { + const { value } = redactMetadata({ keep: 'yes' }) + + expect(value).not.toHaveProperty('_redacted') + }) + + it('collapses values past the depth limit', () => { + const deep = { a: { b: { c: { d: { e: { f: 'too far' } } } } } } + const { value, truncated } = redactMetadata(deep) + + expect(truncated).toBe(true) + expect(JSON.stringify(value)).toContain(TRUNCATED) + }) + + it('caps array length', () => { + const { value, truncated } = redactMetadata({ + ids: Array.from({ length: 100 }, (_, index) => `id-${index}`), + }) + + expect((value!.ids as unknown[]).length).toBe(RedactionLimits.maxArrayLength) + expect(truncated).toBe(true) + }) + + it('caps object breadth', () => { + const wide = Object.fromEntries( + Array.from({ length: 100 }, (_, index) => [`k${index}`, index]) + ) + const { value, truncated } = redactMetadata(wide) + + expect(Object.keys(value!).length).toBe(RedactionLimits.maxKeys) + expect(truncated).toBe(true) + }) + + it('truncates an over-long string', () => { + const { value, truncated } = redactMetadata({ note: 'x'.repeat(1000) }) + + expect(value!.note).toBe(`${'x'.repeat(RedactionLimits.maxStringLength)}${TRUNCATED}`) + expect(truncated).toBe(true) + }) + + it('survives a circular reference instead of throwing', () => { + const circular: Record = { name: 'loop' } + circular.self = circular + + const { value, truncated } = redactMetadata(circular) + + expect(value!.name).toBe('loop') + expect(truncated).toBe(true) + }) + + it('renders a BigInt as a decimal string', () => { + // Stroop amounts arrive as BigInt and are not JSON-serializable. + expect(redactMetadata({ amountStroops: 10_000_000n }).value).toEqual({ + amountStroops: '10000000', + }) + }) + + it('renders a Date as an ISO timestamp', () => { + expect( + redactMetadata({ archivedAt: new Date('2026-08-24T10:00:00.000Z') }).value + ).toEqual({ archivedAt: '2026-08-24T10:00:00.000Z' }) + }) + + it('keeps an error message but drops the stack', () => { + const { value } = redactMetadata({ cause: new Error('lease lost') }) + + expect(value!.cause).toEqual({ name: 'Error', message: 'lease lost' }) + expect(JSON.stringify(value)).not.toContain('at ') + }) + + it('replaces a function or symbol, which is always a caller mistake', () => { + const { value, redactedPaths } = redactMetadata({ + callback: () => undefined, + marker: Symbol('x'), + }) + + expect(value!.callback).toBe(REDACTED) + expect(value!.marker).toBe(REDACTED) + expect(redactedPaths).toEqual(['callback', 'marker']) + }) + + it('normalizes a non-finite number to null', () => { + expect(redactMetadata({ ratio: Number.NaN, size: Infinity }).value).toEqual({ + ratio: null, + size: null, + }) + }) + }) + + describe('serializeMetadata', () => { + it('returns null for empty input', () => { + expect(serializeMetadata(null)).toBeNull() + expect(serializeMetadata({})).toBeNull() + }) + + it('produces parseable JSON with secrets already replaced', () => { + const serialized = serializeMetadata({ password: 'hunter2', userId: 'u-1' })! + const parsed = JSON.parse(serialized) + + expect(parsed).toMatchObject({ password: REDACTED, userId: 'u-1' }) + expect(serialized).not.toContain('hunter2') + }) + + it('drops an oversized payload rather than storing an unparseable prefix', () => { + const serialized = serializeMetadata({ + blob: Array.from({ length: 20 }, () => 'y'.repeat(250)), + })! + + expect(() => JSON.parse(serialized)).not.toThrow() + expect(Buffer.byteLength(serialized)).toBeLessThanOrEqual( + RedactionLimits.maxSerializedBytes + ) + }) + + it('flags that a cap fired, so the row is not read as complete', () => { + const serialized = serializeMetadata({ note: 'z'.repeat(1000) })! + + expect(JSON.parse(serialized)._truncated).toBe(true) + }) + }) + + describe('hashIpAddress', () => { + const secret = 'test-secret' + + it('returns null for a missing address', () => { + expect(hashIpAddress(null, secret)).toBeNull() + expect(hashIpAddress(undefined, secret)).toBeNull() + expect(hashIpAddress('', secret)).toBeNull() + }) + + it('never returns the address itself', () => { + const hash = hashIpAddress('203.0.113.42', secret)! + + expect(hash).not.toContain('203') + expect(hash).toMatch(/^[0-9a-f]{32}$/) + }) + + it('is stable, so events from one source can be correlated', () => { + expect(hashIpAddress('203.0.113.42', secret)).toBe(hashIpAddress('203.0.113.42', secret)) + }) + + it('separates different addresses', () => { + expect(hashIpAddress('203.0.113.42', secret)).not.toBe( + hashIpAddress('203.0.113.43', secret) + ) + }) + + it('is keyed, so a rotated secret breaks correlation with older hashes', () => { + // The point of HMAC over a bare digest: the IPv4 space is small enough + // that an unkeyed hash is reversible by enumeration. + expect(hashIpAddress('203.0.113.42', 'secret-a')).not.toBe( + hashIpAddress('203.0.113.42', 'secret-b') + ) + }) + + it('ignores surrounding whitespace', () => { + expect(hashIpAddress(' 203.0.113.42 ', secret)).toBe(hashIpAddress('203.0.113.42', secret)) + }) + }) + + describe('userAgentFamily', () => { + it('returns null for a missing User-Agent', () => { + expect(userAgentFamily(null)).toBeNull() + expect(userAgentFamily('')).toBeNull() + }) + + it.each([ + ['Mozilla/5.0 (Windows NT 10.0) Chrome/120.0.0.0 Safari/537.36', 'Chrome'], + ['Mozilla/5.0 (Windows NT 10.0) Chrome/120 Safari/537.36 Edg/120.0', 'Edge'], + ['Mozilla/5.0 (X11; Linux) Firefox/121.0', 'Firefox'], + ['Mozilla/5.0 (Macintosh) Version/17.0 Safari/605.1.15', 'Safari'], + ['okhttp/4.12.0', 'Android'], + ['LearnaultApp/1.0 CFNetwork/1494 Darwin/23.4.0', 'iOS'], + ['curl/8.4.0', 'curl'], + ['PostmanRuntime/7.36.0', 'Postman'], + ['Googlebot/2.1', 'Bot'], + ])('reduces %s to %s', (ua, family) => { + expect(userAgentFamily(ua)).toBe(family) + }) + + it('resolves Edge before Chrome, which it also advertises', () => { + expect(userAgentFamily('Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0')).toBe('Edge') + }) + + it('falls back to Other for an unrecognized agent', () => { + expect(userAgentFamily('SomeInternalClient/2.0')).toBe('Other') + }) + + it('discards the version and platform detail it was given', () => { + const family = userAgentFamily('Mozilla/5.0 (Windows NT 10.0; Win64) Chrome/120.0.6099.109')! + + expect(family).toBe('Chrome') + expect(family).not.toContain('120') + expect(family).not.toContain('Windows') + }) + }) +}) diff --git a/tests/integration/audit-immutability.test.ts b/tests/integration/audit-immutability.test.ts new file mode 100644 index 0000000..099b124 --- /dev/null +++ b/tests/integration/audit-immutability.test.ts @@ -0,0 +1,228 @@ +import { describe, it, expect, beforeAll, afterAll } from 'vitest' +import { readFileSync } from 'fs' +import { join } from 'path' +import { randomUUID } from 'crypto' +import { Pool } from 'pg' +import { AUDIT_PURGE_SETTING } from '../../src/audit/audit-event.service' + +/** + * Database-level proof that audit events are immutable. + * + * The unit tests assert the *service* offers no way to edit an event. This + * asserts the database refuses even when someone bypasses the service and + * writes raw SQL — which is the only guarantee worth having, since an audit + * trail the application can quietly rewrite is not an audit trail. + * + * Skipped when no test database is reachable. Bring one up with + * `pnpm stack:up` (or any Postgres on DATABASE_URL) to run it. + */ + +const MIGRATION = join( + process.cwd(), + 'prisma', + 'migrations', + '20260824090000_auditable_data_lifecycle', + 'migration.sql' +) + +let pool: Pool | undefined +let available = false + +/** + * Apply the immutability DDL from the shipped migration. + * + * `tests/globalSetup.ts` prepares the schema with `prisma db push`, which + * creates tables from schema.prisma and never executes migration SQL — so the + * triggers do not exist in a test database by default. Extracting them from the + * real migration file means this test verifies the artifact that actually ships, + * not a copy of it that could drift. + */ +async function applyImmutabilityDdl(db: Pool): Promise { + const sql = readFileSync(MIGRATION, 'utf8') + + const start = sql.indexOf('CREATE OR REPLACE FUNCTION "audit_events_reject_mutation"') + const end = sql.indexOf('-- ARCHIVE COLUMNS') + + if (start === -1 || end === -1 || end <= start) { + throw new Error('Could not locate the immutability DDL in the migration file') + } + + await db.query(sql.slice(start, end)) +} + +async function insertEvent(db: Pool, action = 'test.event'): Promise { + const id = randomUUID() + + await db.query( + `INSERT INTO "audit_events" + ("id", "actorType", "action", "recordClass", "targetType", "targetId") + VALUES ($1, 'SYSTEM', $2, 'IMMUTABLE', 'User', $3)`, + [id, action, randomUUID()] + ) + + return id +} + +beforeAll(async () => { + const connectionString = process.env.DATABASE_URL + if (!connectionString) return + + const candidate = new Pool({ connectionString, connectionTimeoutMillis: 3000 }) + + try { + await candidate.query('SELECT 1 FROM "audit_events" LIMIT 1') + await applyImmutabilityDdl(candidate) + pool = candidate + available = true + } catch { + await candidate.end().catch(() => undefined) + } +}) + +afterAll(async () => { + if (!pool) return + + // The purge setting is the only sanctioned way to remove rows, so cleanup has + // to use it too — which is itself a small confirmation that it works. + await pool.query(`SET LOCAL "${AUDIT_PURGE_SETTING}" = 'on'`).catch(() => undefined) + await pool + .query( + `BEGIN; SET LOCAL "${AUDIT_PURGE_SETTING}" = 'on'; + DELETE FROM "audit_events" WHERE "action" LIKE 'test.%'; COMMIT;` + ) + .catch(() => undefined) + await pool.end().catch(() => undefined) +}) + +describe.skipIf(!process.env.DATABASE_URL)('audit_events immutability (database)', () => { + it('accepts an append', async () => { + if (!available) return expect(available).toBe(false) + + const id = await insertEvent(pool!) + const { rows } = await pool!.query('SELECT "id" FROM "audit_events" WHERE "id" = $1', [id]) + + expect(rows).toHaveLength(1) + }) + + it('rejects an UPDATE', async () => { + if (!available) return expect(available).toBe(false) + + const id = await insertEvent(pool!) + + await expect( + pool!.query('UPDATE "audit_events" SET "reason" = $1 WHERE "id" = $2', ['tampered', id]) + ).rejects.toThrow(/immutable/i) + }) + + it('rejects an UPDATE even inside the purge escape hatch', async () => { + if (!available) return expect(available).toBe(false) + + const id = await insertEvent(pool!) + const client = await pool!.connect() + + try { + await client.query('BEGIN') + await client.query(`SET LOCAL "${AUDIT_PURGE_SETTING}" = 'on'`) + + // The escape hatch exists for the retention purge only. It must not + // become a way to edit history. + await expect( + client.query('UPDATE "audit_events" SET "reason" = $1 WHERE "id" = $2', ['x', id]) + ).rejects.toThrow(/immutable/i) + } finally { + await client.query('ROLLBACK').catch(() => undefined) + client.release() + } + }) + + it('rejects a DELETE without the purge setting', async () => { + if (!available) return expect(available).toBe(false) + + const id = await insertEvent(pool!) + + await expect( + pool!.query('DELETE FROM "audit_events" WHERE "id" = $1', [id]) + ).rejects.toThrow(/retention purge/i) + }) + + it('rejects a TRUNCATE, which bypasses row-level triggers', async () => { + if (!available) return expect(available).toBe(false) + + await expect(pool!.query('TRUNCATE TABLE "audit_events"')).rejects.toThrow( + /may not be truncated/i + ) + }) + + it('allows a DELETE when the retention purge sets the session variable', async () => { + if (!available) return expect(available).toBe(false) + + const id = await insertEvent(pool!) + const client = await pool!.connect() + + try { + await client.query('BEGIN') + await client.query(`SET LOCAL "${AUDIT_PURGE_SETTING}" = 'on'`) + const result = await client.query('DELETE FROM "audit_events" WHERE "id" = $1', [id]) + await client.query('COMMIT') + + expect(result.rowCount).toBe(1) + } finally { + await client.query('ROLLBACK').catch(() => undefined) + client.release() + } + }) + + it('confines the purge setting to its own transaction', async () => { + if (!available) return expect(available).toBe(false) + + const id = await insertEvent(pool!) + const client = await pool!.connect() + + try { + // SET LOCAL, not SET: the permission must not leak to later statements on + // a pooled connection that some unrelated request picks up next. + await client.query('BEGIN') + await client.query(`SET LOCAL "${AUDIT_PURGE_SETTING}" = 'on'`) + await client.query('COMMIT') + + await expect( + client.query('DELETE FROM "audit_events" WHERE "id" = $1', [id]) + ).rejects.toThrow(/retention purge/i) + } finally { + client.release() + } + }) +}) + +describe.skipIf(!process.env.DATABASE_URL)('archive constraints (database)', () => { + it('rejects an archived row with no reason', async () => { + if (!available) return expect(available).toBe(false) + + // The check constraint is what makes "archive behaviour is deterministic" + // true for writers that skip the helper in src/audit/audited-mutation.ts. + await expect( + pool!.query( + `INSERT INTO "Module" + ("id", "title", "description", "category", "difficulty", "archivedAt", "updatedAt") + VALUES ($1, 't', 'd', 'c', 'easy', now(), now())`, + [randomUUID()] + ) + ).rejects.toThrow(/archive_reason_check/i) + }) + + it('accepts an archived row that states a reason', async () => { + if (!available) return expect(available).toBe(false) + + const id = randomUUID() + + await pool!.query( + `INSERT INTO "Module" + ("id", "title", "description", "category", "difficulty", + "archivedAt", "archivedReason", "updatedAt") + VALUES ($1, 't', 'd', 'c', 'easy', now(), 'superseded', now())`, + [id] + ) + + await pool!.query('DELETE FROM "Module" WHERE "id" = $1', [id]) + }) +}) diff --git a/tests/services/webhook.service.spec.ts b/tests/services/webhook.service.spec.ts index 2926f37..25f98fd 100644 --- a/tests/services/webhook.service.spec.ts +++ b/tests/services/webhook.service.spec.ts @@ -21,6 +21,14 @@ vi.mock('@prisma/client', () => ({ PrismaClient: class { webhookEndpoint = mockPrismaInstance.webhookEndpoint webhookDelivery = mockPrismaInstance.webhookDelivery + + // src/config/database.ts applies the archive-exclusion extension. This + // mock returns itself: WebhookEndpoint is archivable, but every + // expectation here asserts on the delegate calls rather than on the + // `where` the extension would add. + $extends() { + return this + } }, }))