Skip to content

feat(audit): add auditable data lifecycle and archive policy - #148

Merged
3m1n3nc3 merged 1 commit into
learnault:mainfrom
Cedarich:feat/auditable-data-lifecycle-archive-policy
Aug 24, 2026
Merged

feat(audit): add auditable data lifecycle and archive policy#148
3m1n3nc3 merged 1 commit into
learnault:mainfrom
Cedarich:feat/auditable-data-lifecycle-archive-policy

Conversation

@Cedarich

Copy link
Copy Markdown
Contributor

Summary

Implements #124: soft-delete, archive, retention, erasure, and immutable audit
behaviour for sensitive data.

Full policy and rationale: docs/DATA_LIFECYCLE.md.

What landed

Lifecycle matrix — all 37 models in schema.prisma classified as MUTABLE,
ARCHIVABLE, DELETABLE, or IMMUTABLE, each with a data category, retention window
and anchor column, erasure behaviour, audit requirement, and a written rationale.
src/audit/classification.ts is the machine-readable source of truth;
tests/audit/classification.test.ts parses the schema and fails on any
unclassified model, so the two cannot drift.

Retention by category: money and consent 7y, credentials indefinite, security
events bounded (30–90d for auth material, 7y for audit events), content archived
then purged at 365d, operational journals 30–90d, export artifacts 7d — the
shortest in the schema, because that row is a full personal-data dump.

Immutable audit events — new audit_events table recording actor
(type/id/role), action, target (type/id), record class, reason, request id,
correlation id, source, redacted metadata, keyed IP hash, and UA family.

Enforcement is in the database, not convention:

  • UPDATE rejected unconditionally — no escape hatch
  • DELETE rejected unless the transaction set learnault.audit_purge, which
    only purgeExpired() does, and which deletes by timestamp so it cannot
    target one inconvenient event
  • TRUNCATE rejected by a statement-level trigger, since it bypasses row triggers
  • SET LOCAL (not SET) so the permission cannot leak to a later request on a
    pooled connection

No FK to User, deliberately: Cascade would let erasure destroy the trail and
SetNull would mutate an immutable row.

Redaction on write — audit rows can't be scrubbed later, so metadata is
filtered on the way in. Two independent passes: key names, and value shapes
(Stellar seed/public key, JWT, bearer credential, email, E.164, IPv4, 40+ char
hex blob, PEM header) so a secret nested under an innocuous key is still caught.
Structural caps bound depth/breadth/length/size. Redacted paths are recorded in
_redacted so a reader isn't misled.

Over-redaction is treated as its own failure — statusCode, failureCode,
referralCode, amountStroops stay readable. to is deliberately allowed: it's
the standard key for a status transition, and the value-level email pattern
catches an actual recipient anyway.

IP hashing uses HMAC, not a bare digest — the IPv4 space is small enough to
enumerate. If AUDIT_IP_HASH_SECRET is unset in production the hash is omitted
entirely
rather than falling back to a value published in this repo.

Audited mutation helperauditedMutation commits the change and its audit
event in one transaction. A failed audit write rolls the mutation back, which is
the opposite of auditEventService.record() (fire-and-forget, for standalone
events like a failed login). ADMIN actors must supply a reason, checked before the
transaction opens. auditedArchive/auditedRestore wrap the archive patch the
same way and refuse models the matrix doesn't classify as ARCHIVABLE.

Archived rows excluded by default — a Prisma client extension injects
archivedAt: null into findFirst/findMany/count/aggregate/groupBy on
archivable models, applied once in src/config/database.ts so the guarantee holds
for code that's never heard of this module. Two documented carve-outs:
findUnique (a silent filter turns a found row into null and reads as "deleted"
to code holding the id — use assertActive) and writes (archive/restore must see
their own row). Opting out via includeArchived() is visible at the call site.

archivedAt/archivedById/archivedReason added to LearnerProfile, Module,
Avatar, ReferralCode, WebhookEndpoint, each with a CHECK constraint so an
archived row always states why — because "archive" lands from several call sites
and a reason-less archive is indistinguishable from an accident months later.

Verification

  • pnpm test1073 passed, 3 skipped, 59 files. 222 of those are new.
  • pnpm lint — clean.
  • tsc --noEmit — no new errors. (Four files have pre-existing errors from the
    earlier reward/amount/bonusAmount*Stroops rename, unrelated to this
    change and confirmed present on main.)
  • prisma validate — schema valid; client generates.

Test coverage maps to the acceptance criteria: immutability (no mutating API
surface, purge uses the session variable, DB-level UPDATE/DELETE/TRUNCATE
rejection), visibility (default exclusion and every carve-out), attribution
(actor distinct from target, transaction atomicity, rollback on audit failure,
ADMIN-reason policy), and redaction (key and value deny-lists, caps, IP hashing,
UA coarsening, and explicit over-redaction guards).

Not yet verified — needs a reviewer with a database

No Postgres was reachable in this environment (Docker daemon down), so:

  1. migrate deploy output is not attached. The migration SQL is unexecuted.
    Please run it and confirm before merge.
  2. tests/integration/audit-immutability.test.ts has not actually run. It
    skips without a reachable DB.

Worth knowing about that test: tests/globalSetup.ts prepares schemas with
prisma db push, which creates tables from schema.prisma and never executes
migration SQL
— so the triggers don't exist in a test database by default. The
test therefore extracts the immutability DDL from the shipped migration file and
applies it itself, which means it verifies the artifact that actually ships rather
than a copy. The migration's trigger DDL uses DROP ... IF EXISTS so it is
re-appliable. CI has no Postgres service either, so this test will skip there too
until one is added.

Reviewer notes

  • src/config/database.ts casts the extended client back to PrismaClient. The
    extension only rewrites a query's where — it adds no methods and changes no
    result shapes — and widening every injection site to a union of both client
    types makes the overloaded $transaction signature stop resolving.
  • tests/services/webhook.service.spec.ts gained a $extends() stub on its mock
    PrismaClient, since the real client configuration now calls it.
  • audit_logs is now marked legacy/superseded. It still works and existing
    writers are untouched; new code should write audit_events. Migrating those
    call sites is deliberately out of scope here.
  • No existing service was converted to auditedMutation in this PR — the
    primitives and policy land first, so Phase 1 profile/account lifecycle work can
    build on them without a large simultaneous refactor.

Closes #124

@3m1n3nc3
3m1n3nc3 merged commit 15a9f44 into learnault:main Aug 24, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Add Auditable Data Lifecycle and Archive Policy

2 participants