diff --git a/backend/docs/DATABASE_MIGRATIONS.md b/backend/docs/DATABASE_MIGRATIONS.md new file mode 100644 index 00000000..f4f9db26 --- /dev/null +++ b/backend/docs/DATABASE_MIGRATIONS.md @@ -0,0 +1,130 @@ +# Database Migrations + +The schema is owned by TypeORM migrations. `synchronize` is enabled **only** +when `NODE_ENV=development`; every other environment (including an unset +`NODE_ENV`) gets its schema exclusively from the migration files in +`src/migrations/`. + +## Files + +| File | Purpose | +| --- | --- | +| `src/config/database.config.ts` | Single source of truth for the connection options, shared by the Nest app and the CLI. | +| `src/data-source.ts` | `DataSource` instance the TypeORM CLI loads (`-d src/data-source.ts`). | +| `src/database/run-migrations.ts` | Run / revert / baseline, each under a PostgreSQL advisory lock. | +| `src/migrate.ts` | Standalone deploy entry point (`node dist/migrate.js `). | +| `src/migrations/` | Generated migration files, applied in timestamp order. | + +`src/data-source.ts` loads `.env` itself, so CLI commands pick up the same +`DB_HOST` / `DB_PORT` / `DB_USERNAME` / `DB_PASSWORD` / `DB_DATABASE` values the +application uses. + +## Commands + +Run these from `backend/`: + +```bash +# Apply all pending migrations +npm run migration:run + +# Revert the most recently applied migration +npm run migration:revert + +# List applied ([X]) and pending ([ ]) migrations +npm run migration:show + +# Generate a migration from the diff between entities and the live database +npm run migration:generate -- src/migrations/DescriptiveName + +# Create an empty migration to hand-write (e.g. a data backfill) +npm run migration:create -- src/migrations/DescriptiveName +``` + +`migration:generate` diffs your **entities** against a **live database**, so +point it at a database that is already up to date with the committed +migrations, then commit the file it writes. + +## Baselining a database built by `synchronize` + +Any database created before this workflow existed already has the tables, but +no row in the `migrations` table. Running `InitialSchema` against it would fail +on `CREATE TABLE "verifications"` — the table is already there. + +Such a database must be **baselined once**: record the initial migration as +applied without replaying its DDL. + +```bash +# 1. Confirm the live schema really does match the initial migration. +# Silence here ("No changes in database schema were found") means they agree. +npm run migration:generate -- src/migrations/BaselineCheck + +# 2. If — and only if — step 1 reported no changes, mark migrations as applied +# without executing them. +npm run migration:baseline:prod +``` + +If step 1 *does* emit a migration, the live schema has drifted from the +entities. Delete the generated file, reconcile the difference deliberately, and +only then baseline. Rehearse the whole procedure against a restored copy of the +production database before touching production. + +Fresh databases need none of this — `migration:run` handles them. + +## Production + +**Every schema change against a shared database must go through a `:prod` +command or the application's own startup path.** Those are the only routes that +take the advisory lock. The plain `migration:run` / `migration:revert` / +`migration:baseline` scripts drive the TypeORM CLI directly, take no lock, and +are meant for a developer's local database. + +| Command | Effect | +| --- | --- | +| `npm run migration:run:prod` | `node dist/migrate.js run` — apply pending migrations | +| `npm run migration:revert:prod` | `node dist/migrate.js revert` — revert the last migration | +| `npm run migration:baseline:prod` | `node dist/migrate.js baseline` — record migrations as applied without running them | +| `npm run migration:show:prod` | List applied / pending migrations (read-only, no lock) | + +None of these need `ts-node`, so they work on a host installed with +`npm install --production`. + +Two paths apply pending migrations on a deploy, and both take the same lock, so +either order is safe: + +1. **On startup.** Whenever `NODE_ENV` is not `development`, `main.ts` calls + `runPendingMigrations` before `app.listen()`, so the app never serves + traffic against a stale schema. +2. **From the deploy script.** `scripts/deploy-backend.sh` runs + `npm run migration:run:prod` after the build. + +### Why the advisory lock + +TypeORM decides which migrations are pending *before* recording them and takes +no cross-process lock of its own. During a rolling deploy, with more than one +replica, or when an operator runs a revert while an instance is booting, two +processes can therefore each conclude the same migration is pending and run its +non-idempotent DDL twice. `withMigrationLock` wraps every schema operation in a +PostgreSQL session-level advisory lock (`MIGRATION_ADVISORY_LOCK_KEY`): the +second process blocks until the first commits and releases, then re-reads the +migrations table and finds the work already done. + +The lock is held on its own connection rather than the one TypeORM uses for the +DDL. An advisory lock is a mutex between *sessions* and only has to be held for +the duration of the work it guards; holding it on a dedicated session also +keeps it alive across the several connections TypeORM opens and closes while +migrating. + +This relies on a session-pooled connection. Behind a transaction-pooling proxy +(PgBouncer in `transaction` mode), session-level advisory locks are not held +across statements — point migrations at a direct connection there. + +`migrationsRun` is deliberately left `false` everywhere, because TypeORM's own +bootstrap hook runs migrations inside `DataSource.initialize()`, before this +lock can be taken. + +## Adding an entity + +1. Write the entity as usual — it is picked up by the `*.entity.{ts,js}` glob. +2. Run `npm run migration:generate -- src/migrations/AddYourEntity` against a + database that is current. +3. Review the generated SQL, then commit the migration alongside the entity. diff --git a/backend/package-lock.json b/backend/package-lock.json index cc478524..45741a3c 100644 --- a/backend/package-lock.json +++ b/backend/package-lock.json @@ -34,6 +34,7 @@ "cache-manager-redis-store": "^3.0.1", "class-transformer": "^0.5.1", "class-validator": "^0.14.0", + "dotenv": "^16.6.1", "groq-sdk": "^0.3.0", "ioredis": "^5.3.2", "ipfs-http-client": "^60.0.0", @@ -2253,6 +2254,18 @@ "rxjs": "^7.1.0" } }, + "node_modules/@nestjs/config/node_modules/dotenv": { + "version": "16.4.5", + "resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.4.5.tgz", + "integrity": "sha512-ZmdL2rui+eB2YwhsWzjInR8LldtZHGDoQ1ugH85ppHKwpUHL7j7rN0Ti9NCnGiQbhaZ11FpR+7ao1dNsmduNUg==", + "license": "BSD-2-Clause", + "engines": { + "node": ">=12" + }, + "funding": { + "url": "https://dotenvx.com" + } + }, "node_modules/@nestjs/core": { "version": "10.4.22", "resolved": "https://registry.npmjs.org/@nestjs/core/-/core-10.4.22.tgz", @@ -5956,9 +5969,9 @@ } }, "node_modules/dotenv": { - "version": "16.4.5", - "resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.4.5.tgz", - "integrity": "sha512-ZmdL2rui+eB2YwhsWzjInR8LldtZHGDoQ1ugH85ppHKwpUHL7j7rN0Ti9NCnGiQbhaZ11FpR+7ao1dNsmduNUg==", + "version": "16.6.1", + "resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz", + "integrity": "sha512-uBq4egWHTcTt33a72vpSG0z3HnPuIl6NqYcTrKEg2azoEyl2hpW0zqlxysq2pK9HlDIHyHyakeYaYnSAwd8bow==", "license": "BSD-2-Clause", "engines": { "node": ">=12" @@ -13611,18 +13624,6 @@ } } }, - "node_modules/typeorm/node_modules/dotenv": { - "version": "16.6.1", - "resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz", - "integrity": "sha512-uBq4egWHTcTt33a72vpSG0z3HnPuIl6NqYcTrKEg2azoEyl2hpW0zqlxysq2pK9HlDIHyHyakeYaYnSAwd8bow==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=12" - }, - "funding": { - "url": "https://dotenvx.com" - } - }, "node_modules/typeorm/node_modules/glob": { "version": "10.5.0", "resolved": "https://registry.npmjs.org/glob/-/glob-10.5.0.tgz", diff --git a/backend/package.json b/backend/package.json index 08f981d0..aea2b967 100644 --- a/backend/package.json +++ b/backend/package.json @@ -12,6 +12,18 @@ "start:dev": "nest start --watch", "start:debug": "nest start --debug --watch", "start:prod": "node dist/main", + "typeorm": "typeorm-ts-node-commonjs -d src/data-source.ts", + "migration:generate": "npm run typeorm -- migration:generate", + "migration:create": "typeorm-ts-node-commonjs migration:create", + "migration:run": "npm run typeorm -- migration:run", + "migration:revert": "npm run typeorm -- migration:revert", + "migration:show": "npm run typeorm -- migration:show", + "migration:baseline": "npm run typeorm -- migration:run --fake", + "typeorm:prod": "typeorm -d dist/data-source.js", + "migration:run:prod": "node dist/migrate.js run", + "migration:revert:prod": "node dist/migrate.js revert", + "migration:baseline:prod": "node dist/migrate.js baseline", + "migration:show:prod": "npm run typeorm:prod -- migration:show", "lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix", "test": "jest", "test:watch": "jest --watch", @@ -45,6 +57,7 @@ "cache-manager-redis-store": "^3.0.1", "class-transformer": "^0.5.1", "class-validator": "^0.14.0", + "dotenv": "^16.6.1", "groq-sdk": "^0.3.0", "ioredis": "^5.3.2", "ipfs-http-client": "^60.0.0", diff --git a/backend/src/app.module.ts b/backend/src/app.module.ts index 3dc8aa76..ea33cbf4 100644 --- a/backend/src/app.module.ts +++ b/backend/src/app.module.ts @@ -6,6 +6,7 @@ import { ThrottlerModule } from '@nestjs/throttler'; import { APP_GUARD } from '@nestjs/core'; import { ThrottlerGuard } from '@nestjs/throttler'; import { redisStore } from 'cache-manager-redis-store'; +import { buildDatabaseOptions } from './config/database.config'; import { IdentityModule } from './identity/identity.module'; import { VerificationModule } from './verification/verification.module'; import { AccessControlModule } from './access-control/access-control.module'; @@ -37,17 +38,8 @@ import { RequestContextMiddleware } from './common/logger/logger.middleware'; LoggerModule, TypeOrmModule.forRootAsync({ imports: [ConfigModule], - useFactory: (configService: ConfigService) => ({ - type: 'postgres', - host: configService.get('DB_HOST'), - port: configService.get('DB_PORT'), - username: configService.get('DB_USERNAME'), - password: configService.get('DB_PASSWORD'), - database: configService.get('DB_DATABASE'), - entities: [__dirname + '/**/*.entity{.ts,.js}'], - synchronize: configService.get('NODE_ENV') === 'development', - logging: configService.get('NODE_ENV') === 'development', - }), + useFactory: (configService: ConfigService) => + buildDatabaseOptions((key) => configService.get(key)), inject: [ConfigService], }), ThrottlerModule.forRootAsync({ diff --git a/backend/src/config/database.config.spec.ts b/backend/src/config/database.config.spec.ts new file mode 100644 index 00000000..f9a7629e --- /dev/null +++ b/backend/src/config/database.config.spec.ts @@ -0,0 +1,69 @@ +import { buildDatabaseOptions, isDevelopment } from './database.config'; + +const reader = (env: Record) => (key: string) => + env[key]; + +describe('buildDatabaseOptions', () => { + it('enables synchronize only when NODE_ENV is development', () => { + const options = buildDatabaseOptions(reader({ NODE_ENV: 'development' })); + + expect(options.synchronize).toBe(true); + }); + + it.each(['production', 'staging', 'test', undefined])( + 'disables synchronize when NODE_ENV is %s', + (nodeEnv) => { + const options = buildDatabaseOptions(reader({ NODE_ENV: nodeEnv })); + + expect(options.synchronize).toBe(false); + }, + ); + + it.each(['development', 'production', undefined])( + 'never lets TypeORM auto-run migrations when NODE_ENV is %s', + (nodeEnv) => { + // Migrations are applied explicitly by `runPendingMigrations`, which + // holds an advisory lock that `migrationsRun` cannot take. + const options = buildDatabaseOptions(reader({ NODE_ENV: nodeEnv })); + + expect(options.migrationsRun).toBe(false); + }, + ); + + it('reads connection settings from the environment', () => { + const options = buildDatabaseOptions( + reader({ + NODE_ENV: 'production', + DB_HOST: 'db.internal', + DB_PORT: '6543', + DB_USERNAME: 'validfi', + DB_PASSWORD: 's3cret', + DB_DATABASE: 'validfi', + }), + ); + + expect(options).toMatchObject({ + type: 'postgres', + host: 'db.internal', + port: 6543, + username: 'validfi', + password: 's3cret', + database: 'validfi', + }); + }); + + it('points at the migrations directory and tracking table', () => { + const options = buildDatabaseOptions(reader({ NODE_ENV: 'production' })); + + expect(options.migrationsTableName).toBe('migrations'); + expect(options.migrations).toEqual([ + expect.stringContaining('migrations'), + ]); + }); +}); + +describe('isDevelopment', () => { + it('is false when NODE_ENV is unset', () => { + expect(isDevelopment(reader({}))).toBe(false); + }); +}); diff --git a/backend/src/config/database.config.ts b/backend/src/config/database.config.ts new file mode 100644 index 00000000..bbf90ec8 --- /dev/null +++ b/backend/src/config/database.config.ts @@ -0,0 +1,45 @@ +import { join } from 'path'; +import { DataSourceOptions } from 'typeorm'; + +/** + * Reads a configuration value. Lets the same options be built from either + * Nest's ConfigService or raw `process.env` (TypeORM CLI). + */ +export type EnvReader = (key: string) => string | undefined; + +/** + * Strict check: anything other than an explicit `development` is treated as a + * non-development environment, so an unset NODE_ENV never enables auto sync. + */ +export const isDevelopment = (get: EnvReader): boolean => + get('NODE_ENV') === 'development'; + +/** + * Single source of truth for the database connection, shared by the Nest + * application and the TypeORM CLI data source so both always agree on where + * entities and migrations live. + */ +export const buildDatabaseOptions = (get: EnvReader): DataSourceOptions => { + const development = isDevelopment(get); + + return { + type: 'postgres', + host: get('DB_HOST') ?? 'localhost', + port: parseInt(get('DB_PORT') ?? '5432', 10), + username: get('DB_USERNAME') ?? 'postgres', + password: get('DB_PASSWORD') ?? 'postgres', + database: get('DB_DATABASE') ?? 'securedata', + entities: [join(__dirname, '..', '**', '*.entity{.ts,.js}')], + migrations: [join(__dirname, '..', 'migrations', '*{.ts,.js}')], + migrationsTableName: 'migrations', + // Auto schema sync is a development-only convenience; every other + // environment gets its schema exclusively from migrations. + synchronize: development, + // Migrations are never run by TypeORM's own bootstrap hook: outside + // development the app applies them explicitly in `main.ts` via + // `runPendingMigrations`, which serialises concurrent instances behind a + // PostgreSQL advisory lock that `migrationsRun` cannot take. + migrationsRun: false, + logging: development, + }; +}; diff --git a/backend/src/data-source.ts b/backend/src/data-source.ts new file mode 100644 index 00000000..696c68c9 --- /dev/null +++ b/backend/src/data-source.ts @@ -0,0 +1,22 @@ +import { config as loadEnv } from 'dotenv'; +import { DataSource, DataSourceOptions } from 'typeorm'; +import { buildDatabaseOptions } from './config/database.config'; + +// The CLI runs outside Nest, so the .env file has to be loaded by hand. +loadEnv(); + +/** + * Options used by the TypeORM CLI. Schema changes must go through migrations, + * so the CLI never synchronizes or auto-runs migrations as a side effect. + */ +export const dataSourceOptions: DataSourceOptions = { + ...buildDatabaseOptions((key) => process.env[key]), + synchronize: false, + migrationsRun: false, +}; + +/** + * Data source for `npm run typeorm -- `, e.g. + * `npm run migration:generate -- src/migrations/AddSomething`. + */ +export const AppDataSource = new DataSource(dataSourceOptions); diff --git a/backend/src/database/migration-commands.spec.ts b/backend/src/database/migration-commands.spec.ts new file mode 100644 index 00000000..781f5ace --- /dev/null +++ b/backend/src/database/migration-commands.spec.ts @@ -0,0 +1,49 @@ +import { + DEFAULT_MIGRATION_COMMAND, + MIGRATION_COMMANDS, + resolveMigrationCommand, +} from './migration-commands'; +import { + baselineMigrations, + revertLastMigration, + runPendingMigrations, +} from './run-migrations'; + +describe('resolveMigrationCommand', () => { + it.each([ + ['run', runPendingMigrations], + ['revert', revertLastMigration], + ['baseline', baselineMigrations], + ])('resolves %s', (name, expected) => { + expect(resolveMigrationCommand(name)).toBe(expected); + }); + + it('defaults to running pending migrations', () => { + expect(resolveMigrationCommand(DEFAULT_MIGRATION_COMMAND)).toBe( + runPendingMigrations, + ); + }); + + it.each(['toString', 'constructor', 'hasOwnProperty', '__proto__'])( + 'rejects the inherited property %s', + (name) => { + // A plain `name in MIGRATION_COMMANDS` would accept these and exit + // successfully without migrating anything. + expect(() => resolveMigrationCommand(name)).toThrow('Unknown command'); + }, + ); + + it('rejects an unknown command and lists the valid ones', () => { + expect(() => resolveMigrationCommand('bogus')).toThrow( + 'Unknown command "bogus". Expected one of: run, revert, baseline.', + ); + }); + + it('exposes exactly the three supported commands', () => { + expect(Object.keys(MIGRATION_COMMANDS)).toEqual([ + 'run', + 'revert', + 'baseline', + ]); + }); +}); diff --git a/backend/src/database/migration-commands.ts b/backend/src/database/migration-commands.ts new file mode 100644 index 00000000..794757fe --- /dev/null +++ b/backend/src/database/migration-commands.ts @@ -0,0 +1,39 @@ +import { Logger } from '@nestjs/common'; +import { DataSource } from 'typeorm'; +import { + baselineMigrations, + revertLastMigration, + runPendingMigrations, +} from './run-migrations'; + +export type MigrationCommand = ( + dataSource: DataSource, + logger: Logger, +) => Promise; + +export const MIGRATION_COMMANDS: Record = { + run: runPendingMigrations, + revert: revertLastMigration, + baseline: baselineMigrations, +}; + +export const DEFAULT_MIGRATION_COMMAND = 'run'; + +/** + * Resolves a command name to its handler. + * + * Deliberately an own-property check: `'toString' in MIGRATION_COMMANDS` is + * true, so a plain `in` would let a typo resolve to something off + * `Object.prototype` and exit successfully without migrating anything. + */ +export function resolveMigrationCommand(name: string): MigrationCommand { + if (!Object.prototype.hasOwnProperty.call(MIGRATION_COMMANDS, name)) { + throw new Error( + `Unknown command "${name}". Expected one of: ${Object.keys( + MIGRATION_COMMANDS, + ).join(', ')}.`, + ); + } + + return MIGRATION_COMMANDS[name]; +} diff --git a/backend/src/database/run-migrations.spec.ts b/backend/src/database/run-migrations.spec.ts new file mode 100644 index 00000000..a87d902e --- /dev/null +++ b/backend/src/database/run-migrations.spec.ts @@ -0,0 +1,179 @@ +import { Logger } from '@nestjs/common'; +import { DataSource, MigrationExecutor } from 'typeorm'; +import { + baselineMigrations, + MIGRATION_ADVISORY_LOCK_KEY, + revertLastMigration, + runPendingMigrations, + withMigrationLock, +} from './run-migrations'; + +const LOCK = 'SELECT pg_advisory_lock($1)'; +const UNLOCK = 'SELECT pg_advisory_unlock($1)'; + +const pendingMigrations = jest.fn().mockResolvedValue([]); + +jest.mock('typeorm', () => ({ + ...jest.requireActual('typeorm'), + MigrationExecutor: jest.fn(), +})); + +describe('migration helpers', () => { + const silentLogger = { + log: jest.fn(), + warn: jest.fn(), + error: jest.fn(), + } as unknown as Logger; + + type Harness = { + dataSource: DataSource; + calls: string[]; + query: jest.Mock; + release: jest.Mock; + }; + + const buildHarness = ( + overrides: { + runMigrations?: jest.Mock; + undoLastMigration?: jest.Mock; + onQuery?: (sql: string) => void; + } = {}, + ): Harness => { + const calls: string[] = []; + + const query = jest.fn(async (sql: string) => { + calls.push(sql); + overrides.onQuery?.(sql); + return []; + }); + const release = jest.fn(); + + const dataSource = { + createQueryRunner: () => ({ connect: jest.fn(), query, release }), + runMigrations: jest.fn(async (...args: unknown[]) => { + calls.push('runMigrations'); + return overrides.runMigrations?.(...args) ?? []; + }), + undoLastMigration: jest.fn(async (...args: unknown[]) => { + calls.push('undoLastMigration'); + return overrides.undoLastMigration?.(...args); + }), + } as unknown as DataSource; + + return { dataSource, calls, query, release }; + }; + + beforeEach(() => { + jest.clearAllMocks(); + pendingMigrations.mockResolvedValue([]); + (MigrationExecutor as unknown as jest.Mock).mockImplementation(() => ({ + getPendingMigrations: pendingMigrations, + })); + }); + + describe('withMigrationLock', () => { + it('brackets the work with the advisory lock', async () => { + const { dataSource, calls, query } = buildHarness(); + + await withMigrationLock(dataSource, silentLogger, async () => { + calls.push('work'); + }); + + expect(calls).toEqual([LOCK, 'work', UNLOCK]); + expect(query).toHaveBeenCalledWith(LOCK, [MIGRATION_ADVISORY_LOCK_KEY]); + expect(query).toHaveBeenCalledWith(UNLOCK, [MIGRATION_ADVISORY_LOCK_KEY]); + }); + + it('releases the lock when the work throws', async () => { + const { dataSource, calls, release } = buildHarness(); + + await expect( + withMigrationLock(dataSource, silentLogger, async () => { + throw new Error('boom'); + }), + ).rejects.toThrow('boom'); + + expect(calls).toContain(UNLOCK); + expect(release).toHaveBeenCalled(); + }); + + it('releases the query runner even when the unlock query fails', async () => { + const { dataSource, release } = buildHarness({ + onQuery: (sql) => { + if (sql === UNLOCK) { + throw new Error('connection lost'); + } + }, + }); + + await expect( + withMigrationLock(dataSource, silentLogger, async () => undefined), + ).rejects.toThrow('connection lost'); + + expect(release).toHaveBeenCalled(); + }); + + it('returns the value produced by the work', async () => { + const { dataSource } = buildHarness(); + + await expect( + withMigrationLock(dataSource, silentLogger, async () => 'done'), + ).resolves.toBe('done'); + }); + }); + + describe('runPendingMigrations', () => { + it('runs migrations inside the lock, in a single transaction', async () => { + const { dataSource, calls } = buildHarness({ + runMigrations: jest.fn().mockResolvedValue([{ name: 'InitialSchema1' }]), + }); + + await runPendingMigrations(dataSource, silentLogger); + + expect(calls).toEqual([LOCK, 'runMigrations', UNLOCK]); + expect(dataSource.runMigrations).toHaveBeenCalledWith({ + transaction: 'all', + }); + }); + }); + + describe('revertLastMigration', () => { + it('reverts inside the lock', async () => { + const { dataSource, calls } = buildHarness(); + + await revertLastMigration(dataSource, silentLogger); + + expect(calls).toEqual([LOCK, 'undoLastMigration', UNLOCK]); + expect(dataSource.undoLastMigration).toHaveBeenCalledWith({ + transaction: 'all', + }); + }); + }); + + describe('baselineMigrations', () => { + it('records pending migrations as applied without running their DDL', async () => { + pendingMigrations.mockResolvedValue([{ name: 'InitialSchema1' }]); + const { dataSource, calls } = buildHarness(); + + await baselineMigrations(dataSource, silentLogger); + + expect(calls).toEqual([LOCK, 'runMigrations', UNLOCK]); + expect(dataSource.runMigrations).toHaveBeenCalledWith({ + transaction: 'all', + fake: true, + }); + expect(silentLogger.warn).toHaveBeenCalledWith( + expect.stringContaining('InitialSchema1'), + ); + }); + + it('does nothing when every migration is already recorded', async () => { + const { dataSource, calls } = buildHarness(); + + await baselineMigrations(dataSource, silentLogger); + + expect(calls).toEqual([LOCK, UNLOCK]); + expect(dataSource.runMigrations).not.toHaveBeenCalled(); + }); + }); +}); diff --git a/backend/src/database/run-migrations.ts b/backend/src/database/run-migrations.ts new file mode 100644 index 00000000..f0a093af --- /dev/null +++ b/backend/src/database/run-migrations.ts @@ -0,0 +1,125 @@ +import { Logger } from '@nestjs/common'; +import { DataSource, MigrationExecutor } from 'typeorm'; + +/** + * Arbitrary but fixed key identifying the schema-migration lock. Every process + * that migrates this database must use the same value. + */ +export const MIGRATION_ADVISORY_LOCK_KEY = 4071982411; + +/** + * Runs `work` while holding a PostgreSQL session-level advisory lock, so that + * only one process at a time can change the schema. + * + * TypeORM decides which migrations are pending before recording them and takes + * no cross-process lock of its own, so two instances starting at once (a + * rolling deploy, a replica set, or an operator running a revert while an + * instance boots) can each decide the same migration is pending and run its + * non-idempotent DDL twice. The lock serialises them: the loser blocks until + * the winner commits and releases, then re-reads the migrations table and + * finds the work already done. + * + * The lock is deliberately held on its own connection rather than the one + * TypeORM uses for the DDL — an advisory lock is a mutex between *sessions*, + * and it only has to be held for the duration of the work it guards. Holding + * it on a dedicated session also keeps it alive across the several connections + * TypeORM opens and closes internally while migrating. + * + * Note this needs a session-pooled connection — under a transaction-pooling + * proxy such as PgBouncer in `transaction` mode, session-level advisory locks + * are not held across statements. + */ +export async function withMigrationLock( + dataSource: DataSource, + logger: Logger, + work: () => Promise, +): Promise { + const queryRunner = dataSource.createQueryRunner(); + await queryRunner.connect(); + + try { + logger.log('Waiting for the schema migration lock...'); + await queryRunner.query('SELECT pg_advisory_lock($1)', [ + MIGRATION_ADVISORY_LOCK_KEY, + ]); + + try { + return await work(); + } finally { + await queryRunner.query('SELECT pg_advisory_unlock($1)', [ + MIGRATION_ADVISORY_LOCK_KEY, + ]); + } + } finally { + // Nested so that a failing unlock cannot leak the connection back to a + // pool that will never see it returned. + await queryRunner.release(); + } +} + +/** Applies every pending migration, in one transaction, under the lock. */ +export async function runPendingMigrations( + dataSource: DataSource, + logger: Logger = new Logger('Migrations'), +): Promise { + await withMigrationLock(dataSource, logger, async () => { + const applied = await dataSource.runMigrations({ transaction: 'all' }); + + if (applied.length === 0) { + logger.log('Schema is up to date; no migrations to apply.'); + } else { + logger.log( + `Applied ${applied.length} migration(s): ${applied + .map((migration) => migration.name) + .join(', ')}`, + ); + } + }); +} + +/** Reverts the most recently applied migration, under the lock. */ +export async function revertLastMigration( + dataSource: DataSource, + logger: Logger = new Logger('Migrations'), +): Promise { + await withMigrationLock(dataSource, logger, async () => { + await dataSource.undoLastMigration({ transaction: 'all' }); + logger.log('Reverted the last migration.'); + }); +} + +/** + * Records pending migrations as applied *without* running their DDL, under the + * lock. + * + * This is the one-time baseline step for a database whose schema was built by + * `synchronize` and so has the tables but no rows in the migrations table. + * Only run it after confirming the live schema already matches the entities — + * see `docs/DATABASE_MIGRATIONS.md`. + */ +export async function baselineMigrations( + dataSource: DataSource, + logger: Logger = new Logger('Migrations'), +): Promise { + await withMigrationLock(dataSource, logger, async () => { + // A fake run resolves to an empty array — TypeORM records the migration + // and skips the bookkeeping it does for real runs — so the names have to + // be read before faking them. + const pending = await new MigrationExecutor( + dataSource, + ).getPendingMigrations(); + + if (pending.length === 0) { + logger.log('Nothing to baseline; every migration is already recorded.'); + return; + } + + await dataSource.runMigrations({ transaction: 'all', fake: true }); + + logger.warn( + `Recorded ${pending.length} migration(s) as applied without running them: ${pending + .map((migration) => migration.name) + .join(', ')}`, + ); + }); +} diff --git a/backend/src/main.ts b/backend/src/main.ts index 16686523..bbdf8b2f 100644 --- a/backend/src/main.ts +++ b/backend/src/main.ts @@ -2,8 +2,11 @@ import { NestFactory } from '@nestjs/core'; import { ValidationPipe } from '@nestjs/common'; import { ConfigService } from '@nestjs/config'; import { SwaggerModule, DocumentBuilder } from '@nestjs/swagger'; +import { DataSource } from 'typeorm'; import { AppModule } from './app.module'; import { AllExceptionsFilter } from './common/filters/http-exception.filter'; +import { isDevelopment } from './config/database.config'; +import { runPendingMigrations } from './database/run-migrations'; import { StructuredLoggerService } from './common/logger/logger.service'; async function bootstrap() { @@ -18,6 +21,12 @@ async function bootstrap() { const apiPrefix = configService.get('API_PREFIX') ?? 'api/v1'; const nodeEnv = configService.get('NODE_ENV') ?? 'development'; + // Outside development the schema comes from migrations only, so bring the + // database up to date before the app starts accepting traffic. + if (!isDevelopment((key) => configService.get(key))) { + await runPendingMigrations(app.get(DataSource)); + } + app.setGlobalPrefix(apiPrefix); app.enableCors({ diff --git a/backend/src/migrate.ts b/backend/src/migrate.ts new file mode 100644 index 00000000..a5b767b8 --- /dev/null +++ b/backend/src/migrate.ts @@ -0,0 +1,35 @@ +import { Logger } from '@nestjs/common'; +import { AppDataSource } from './data-source'; +import { + DEFAULT_MIGRATION_COMMAND, + resolveMigrationCommand, +} from './database/migration-commands'; + +/** + * Standalone migration entry point for deployments: + * `node dist/migrate.js [run|revert|baseline]`. + * + * Runs without Nest and without ts-node, so it works on a host installed with + * `npm install --production`. Every command goes through the same advisory + * lock the application takes on startup, so no schema operation can race + * another one. + */ +async function migrate(): Promise { + const logger = new Logger('Migrations'); + const command = resolveMigrationCommand( + process.argv[2] ?? DEFAULT_MIGRATION_COMMAND, + ); + + await AppDataSource.initialize(); + + try { + await command(AppDataSource, logger); + } finally { + await AppDataSource.destroy(); + } +} + +migrate().catch((error) => { + new Logger('Migrations').error('Migration run failed', error?.stack ?? error); + process.exit(1); +}); diff --git a/backend/src/migrations/1787556591758-InitialSchema.ts b/backend/src/migrations/1787556591758-InitialSchema.ts new file mode 100644 index 00000000..8efe3060 --- /dev/null +++ b/backend/src/migrations/1787556591758-InitialSchema.ts @@ -0,0 +1,183 @@ +import { MigrationInterface, QueryRunner } from "typeorm"; + +export class InitialSchema1787556591758 implements MigrationInterface { + name = 'InitialSchema1787556591758' + + public async up(queryRunner: QueryRunner): Promise { + await queryRunner.query(`CREATE EXTENSION IF NOT EXISTS "uuid-ossp"`); + await queryRunner.query(`CREATE TYPE "public"."verifications_status_enum" AS ENUM('pending', 'approved', 'rejected', 'expired')`); + await queryRunner.query(`CREATE TABLE "verifications" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "identityId" character varying NOT NULL, "walletAddress" character varying NOT NULL, "proofHash" character varying NOT NULL, "verificationCommitment" character varying NOT NULL, "status" "public"."verifications_status_enum" NOT NULL DEFAULT 'pending', "reason" character varying, "expiresAt" TIMESTAMP WITH TIME ZONE, "metadata" json, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_2127ad1b143cf012280390b01d1" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_2e31645aac05e5b108d321042d" ON "verifications" ("identityId") `); + await queryRunner.query(`CREATE INDEX "IDX_64099fd2aafd220c32a480a439" ON "verifications" ("walletAddress") `); + await queryRunner.query(`CREATE TYPE "public"."role_assignments_role_enum" AS ENUM('viewer', 'verifier', 'issuer', 'admin')`); + await queryRunner.query(`CREATE TABLE "role_assignments" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "granteeAddress" character varying NOT NULL, "role" "public"."role_assignments_role_enum" NOT NULL, "resourceId" character varying NOT NULL DEFAULT '*', "grantedByAddress" character varying NOT NULL, "isActive" boolean NOT NULL DEFAULT true, "expiresAt" TIMESTAMP WITH TIME ZONE, "grantedAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_fc2df9835ac1d2a34839f113783" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_c9d9f1f1983bde0720a6047d18" ON "role_assignments" ("granteeAddress") `); + await queryRunner.query(`CREATE INDEX "IDX_0fe58a810ad39c1f2eb41ac5e4" ON "role_assignments" ("resourceId") `); + await queryRunner.query(`CREATE UNIQUE INDEX "IDX_7af9f9b4f800572efdc68dd6a6" ON "role_assignments" ("granteeAddress", "resourceId", "role") `); + await queryRunner.query(`CREATE TABLE "identities" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "walletAddress" character varying NOT NULL, "documentHash" character varying NOT NULL, "ipfsCid" character varying NOT NULL, "verificationStatus" boolean NOT NULL DEFAULT false, "revoked" boolean NOT NULL DEFAULT false, "revocationReason" character varying, "metadata" json, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_7b2f8cccf4ac6a2d7e6e9e8b1f6" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE TYPE "public"."health_authorities_authtype_enum" AS ENUM('api_key', 'oauth2', 'mutual_tls', 'jwt_bearer')`); + await queryRunner.query(`CREATE TYPE "public"."health_authorities_status_enum" AS ENUM('active', 'inactive', 'suspended', 'pending_verification')`); + await queryRunner.query(`CREATE TABLE "health_authorities" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "name" character varying NOT NULL, "apiUrl" character varying NOT NULL, "authType" "public"."health_authorities_authtype_enum" NOT NULL, "status" "public"."health_authorities_status_enum" NOT NULL DEFAULT 'pending_verification', "apiKey" character varying, "clientId" character varying, "clientSecret" character varying, "tokenUrl" character varying, "certificatePath" character varying, "jurisdiction" character varying, "accessToken" character varying, "tokenExpiresAt" TIMESTAMP WITH TIME ZONE, "metadata" json, "credentialsIssued" integer NOT NULL DEFAULT '0', "lastConnectedAt" TIMESTAMP WITH TIME ZONE, "lastError" character varying, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_091c95d7437b6fa771e71c9d4f9" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_a12fc85b426c020664339756b8" ON "health_authorities" ("name") `); + await queryRunner.query(`CREATE TYPE "public"."issuance_records_format_enum" AS ENUM('w3c_vc', 'fhir', 'hl7', 'custom_json', 'smart_health_card')`); + await queryRunner.query(`CREATE TYPE "public"."issuance_records_status_enum" AS ENUM('pending', 'processing', 'issued', 'failed', 'revoked')`); + await queryRunner.query(`CREATE TABLE "issuance_records" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "authorityId" uuid NOT NULL, "patientWalletAddress" character varying NOT NULL, "credentialType" character varying NOT NULL, "format" "public"."issuance_records_format_enum" NOT NULL DEFAULT 'custom_json', "status" "public"."issuance_records_status_enum" NOT NULL DEFAULT 'pending', "healthData" json NOT NULL, "issuedCredential" json, "credentialHash" character varying, "externalRequestId" character varying, "expirationDate" TIMESTAMP WITH TIME ZONE, "failureReason" character varying, "retryCount" integer NOT NULL DEFAULT '0', "issuerNotes" character varying, "metadata" json, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_39147b30d97d0f56755a0582e87" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_8440c70760bbd133bc7284806e" ON "issuance_records" ("authorityId") `); + await queryRunner.query(`CREATE INDEX "IDX_706ccfe85d207fabd3b966c5ba" ON "issuance_records" ("patientWalletAddress") `); + await queryRunner.query(`CREATE TABLE "shared_data" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "ownerAddress" character varying NOT NULL, "recipientAddress" character varying NOT NULL, "documentHash" character varying NOT NULL, "encryptedKey" text NOT NULL, "accessExpiry" integer NOT NULL, "isActive" boolean NOT NULL DEFAULT true, "metadata" json, "sharedAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_accd282bee89d94697d4c5d7a18" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE TABLE "credentials" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "type" character varying NOT NULL, "issuer" character varying NOT NULL, "holder" character varying, "data" json NOT NULL, "status" character varying NOT NULL DEFAULT 'active', "systemSource" character varying, "contentHash" character varying, "duplicateOfId" character varying, "isDuplicate" boolean NOT NULL DEFAULT false, "duplicateMetadata" json, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_1e38bc43be6697cdda548ad27a6" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_9c5dd783b8d84dfbda8fd7b98b" ON "credentials" ("contentHash") `); + await queryRunner.query(`CREATE TABLE "credential_versions" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "credentialId" character varying NOT NULL, "versionNumber" integer NOT NULL, "schemaVersion" character varying NOT NULL DEFAULT '1.0.0', "documentHash" character varying NOT NULL, "ipfsCid" character varying NOT NULL, "verificationStatus" boolean NOT NULL DEFAULT false, "revoked" boolean NOT NULL DEFAULT false, "metadata" json, "changeReason" character varying, "changedBy" character varying NOT NULL, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "UQ_2d3ddc62358cf21e617ce02a2f1" UNIQUE ("credentialId", "versionNumber"), CONSTRAINT "PK_043316e51edc6dbb84234b3ca53" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_8e4ee079a148d3e7fa5ff54b81" ON "credential_versions" ("credentialId") `); + await queryRunner.query(`CREATE INDEX "IDX_5fa4d1928dc2f0627286589b34" ON "credential_versions" ("versionNumber") `); + await queryRunner.query(`CREATE INDEX "IDX_ad6b1c640fb1846122557fb962" ON "credential_versions" ("changedBy") `); + await queryRunner.query(`CREATE TYPE "public"."credential_migrations_status_enum" AS ENUM('pending', 'success', 'failed', 'conflict', 'resolved')`); + await queryRunner.query(`CREATE TABLE "credential_migrations" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "credentialId" character varying NOT NULL, "fromVersion" integer NOT NULL, "toVersion" integer, "fromSchemaVersion" character varying NOT NULL, "toSchemaVersion" character varying NOT NULL, "status" "public"."credential_migrations_status_enum" NOT NULL DEFAULT 'pending', "conflictReason" character varying, "resolutionNotes" json, "migratedBy" character varying NOT NULL, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_e7353148b7b7117d980495905a6" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_8080932dda76eda6b6b6f5c3d9" ON "credential_migrations" ("credentialId") `); + await queryRunner.query(`CREATE INDEX "IDX_f07b0520b34fb6b2a02b092c87" ON "credential_migrations" ("status") `); + await queryRunner.query(`CREATE INDEX "IDX_77f05bf0e293a614e9f50699e2" ON "credential_migrations" ("migratedBy") `); + await queryRunner.query(`CREATE TYPE "public"."sharing_history_events_action_enum" AS ENUM('SHARED', 'EXTENDED', 'REVOKED', 'RESTORED')`); + await queryRunner.query(`CREATE TABLE "sharing_history_events" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "sharedDataId" character varying NOT NULL, "ownerAddress" character varying NOT NULL, "recipientAddress" character varying NOT NULL, "documentHash" character varying NOT NULL, "action" "public"."sharing_history_events_action_enum" NOT NULL, "metadata" json, "timestamp" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_03946915ba39ca6e78a993fffeb" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE TYPE "public"."credential_exports_format_enum" AS ENUM('json', 'csv', 'pdf', 'vc')`); + await queryRunner.query(`CREATE TYPE "public"."credential_exports_status_enum" AS ENUM('pending', 'processing', 'completed', 'failed')`); + await queryRunner.query(`CREATE TABLE "credential_exports" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "requestedBy" character varying NOT NULL, "format" "public"."credential_exports_format_enum" NOT NULL, "status" "public"."credential_exports_status_enum" NOT NULL DEFAULT 'pending', "credentialIds" json, "includeMetadata" boolean NOT NULL DEFAULT false, "includeVersionHistory" boolean NOT NULL DEFAULT false, "encryptSensitiveData" boolean NOT NULL DEFAULT true, "encryptionKeyId" character varying, "fileHash" text, "storagePath" text, "credentialCount" integer NOT NULL DEFAULT '0', "fileSizeBytes" bigint NOT NULL DEFAULT '0', "errorMessage" text, "validationResults" json, "completedAt" TIMESTAMP, "expiresAt" TIMESTAMP, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_3644c4337dde931ac4b1650647a" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_97e2b96eee5be7d900f71e51cc" ON "credential_exports" ("requestedBy") `); + await queryRunner.query(`CREATE TYPE "public"."audit_logs_operationtype_enum" AS ENUM('issued', 'revoked', 'verified', 'updated', 'deleted', 'shared', 'accessed', 'role_assigned', 'role_revoked')`); + await queryRunner.query(`CREATE TYPE "public"."audit_logs_status_enum" AS ENUM('success', 'failure')`); + await queryRunner.query(`CREATE TABLE "audit_logs" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "sequence" BIGSERIAL NOT NULL, "actorId" character varying NOT NULL, "operationType" "public"."audit_logs_operationtype_enum" NOT NULL, "targetCredentialId" character varying, "clientIp" character varying, "status" "public"."audit_logs_status_enum" NOT NULL DEFAULT 'success', "metadata" json, "previousHash" character varying(64), "hash" character varying(64) NOT NULL, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_1bb179d048bbc581caa3b013439" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE UNIQUE INDEX "IDX_a1a8da97e76ef043712fdeb694" ON "audit_logs" ("sequence") `); + await queryRunner.query(`CREATE INDEX "IDX_2dc33f7f3c22e2e7badafca1d1" ON "audit_logs" ("actorId") `); + await queryRunner.query(`CREATE INDEX "IDX_3f521f4fa46f1e21f3becf021b" ON "audit_logs" ("operationType") `); + await queryRunner.query(`CREATE INDEX "IDX_acdc6af76706ad7d8aa4029e28" ON "audit_logs" ("targetCredentialId") `); + await queryRunner.query(`CREATE INDEX "IDX_2961862b2704794af56359ff0a" ON "audit_logs" ("status") `); + await queryRunner.query(`CREATE INDEX "IDX_c69efb19bf127c97e6740ad530" ON "audit_logs" ("createdAt") `); + await queryRunner.query(`CREATE TABLE "access_permissions" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "grantorAddress" character varying NOT NULL, "granteeAddress" character varying NOT NULL, "resourceId" character varying NOT NULL, "accessExpiry" integer NOT NULL, "isActive" boolean NOT NULL DEFAULT true, "metadata" json, "grantedAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_82523ed02665644d1f7c79a64fe" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE TABLE "indexed_verifications" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "verification_id" bigint NOT NULL, "identity_id" bigint NOT NULL, "verifier" character varying(255) NOT NULL, "subject" character varying(255) NOT NULL, "verified" boolean NOT NULL, "proofHash" text, "zkProof" text, "verified_at" bigint NOT NULL, "ledger_timestamp" bigint NOT NULL, "ledger_sequence" bigint NOT NULL, "transaction_hash" character varying(255) NOT NULL, "contract_id" character varying(255) NOT NULL, "indexed_at" TIMESTAMP NOT NULL DEFAULT now(), "metadata" json, CONSTRAINT "PK_2697c5dab48bb1bf74e586e7aac" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_8a2cc7affaf3bc4a1144944da9" ON "indexed_verifications" ("verification_id") `); + await queryRunner.query(`CREATE INDEX "IDX_20133f03833a769b4277650b16" ON "indexed_verifications" ("identity_id") `); + await queryRunner.query(`CREATE INDEX "IDX_4e0b540710dbc25b421058d4a3" ON "indexed_verifications" ("verifier") `); + await queryRunner.query(`CREATE INDEX "IDX_0dccc53b17457a35d45e4a9ce7" ON "indexed_verifications" ("subject") `); + await queryRunner.query(`CREATE INDEX "IDX_3e6ec076a04ea8f3e2f34ba4c3" ON "indexed_verifications" ("ledger_sequence") `); + await queryRunner.query(`CREATE INDEX "IDX_0e0a146c0c15e8399a7b57d027" ON "indexed_verifications" ("transaction_hash") `); + await queryRunner.query(`CREATE TABLE "indexed_identities" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "identity_id" bigint NOT NULL, "owner" character varying(255) NOT NULL, "documentHash" character varying(255) NOT NULL, "ipfsCid" text NOT NULL, "verificationStatus" boolean NOT NULL DEFAULT false, "revoked" boolean NOT NULL DEFAULT false, "created_at" bigint NOT NULL, "ledger_timestamp" bigint NOT NULL, "ledger_sequence" bigint NOT NULL, "transaction_hash" character varying(255) NOT NULL, "contract_id" character varying(255) NOT NULL, "indexed_at" TIMESTAMP NOT NULL DEFAULT now(), "metadata" json, CONSTRAINT "PK_ce2e6e3ef7d8090c8fa2ea71756" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_d41457404b5f0a360f25385801" ON "indexed_identities" ("identity_id") `); + await queryRunner.query(`CREATE INDEX "IDX_7e467b972045b3faf5bd748cae" ON "indexed_identities" ("owner") `); + await queryRunner.query(`CREATE INDEX "IDX_fe3a18528380f95d4a8d74bda4" ON "indexed_identities" ("documentHash") `); + await queryRunner.query(`CREATE INDEX "IDX_e0c9fc81dff0a1cc64c25e2973" ON "indexed_identities" ("ledger_sequence") `); + await queryRunner.query(`CREATE INDEX "IDX_7fd6ac7d9f7d086f7aedaee886" ON "indexed_identities" ("transaction_hash") `); + await queryRunner.query(`CREATE TABLE "indexed_data_sharing" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "sharing_id" bigint NOT NULL, "identity_id" bigint NOT NULL, "owner" character varying(255) NOT NULL, "recipient" character varying(255) NOT NULL, "encrypted_data" text NOT NULL, "encryption_key_hash" text NOT NULL, "access_revoked" boolean NOT NULL DEFAULT false, "shared_at" bigint NOT NULL, "ledger_timestamp" bigint NOT NULL, "ledger_sequence" bigint NOT NULL, "transaction_hash" character varying(255) NOT NULL, "contract_id" character varying(255) NOT NULL, "indexed_at" TIMESTAMP NOT NULL DEFAULT now(), "metadata" json, CONSTRAINT "PK_6582f2ef54b2498f414fec88ffc" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_f5401af24836b589ac9640b4bc" ON "indexed_data_sharing" ("sharing_id") `); + await queryRunner.query(`CREATE INDEX "IDX_a4ffe5a7b6c474d9336f42cd1a" ON "indexed_data_sharing" ("identity_id") `); + await queryRunner.query(`CREATE INDEX "IDX_21dbbd85da6bf2d09b01960933" ON "indexed_data_sharing" ("owner") `); + await queryRunner.query(`CREATE INDEX "IDX_42c146a31098d47ee019d07691" ON "indexed_data_sharing" ("recipient") `); + await queryRunner.query(`CREATE INDEX "IDX_d3aae4fd70e25ac21d491c74cd" ON "indexed_data_sharing" ("ledger_sequence") `); + await queryRunner.query(`CREATE INDEX "IDX_b2dc8aea87149ab251162c77dd" ON "indexed_data_sharing" ("transaction_hash") `); + await queryRunner.query(`CREATE TABLE "indexed_access_controls" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "access_id" bigint NOT NULL, "identity_id" bigint NOT NULL, "grantor" character varying(255) NOT NULL, "grantee" character varying(255) NOT NULL, "access_granted" boolean NOT NULL, "expires_at" bigint, "granted_at" bigint NOT NULL, "ledger_timestamp" bigint NOT NULL, "ledger_sequence" bigint NOT NULL, "transaction_hash" character varying(255) NOT NULL, "contract_id" character varying(255) NOT NULL, "indexed_at" TIMESTAMP NOT NULL DEFAULT now(), "metadata" json, CONSTRAINT "PK_5d1d5f6a9d44cb909389f303b2a" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_0422607e1bf990304144128063" ON "indexed_access_controls" ("access_id") `); + await queryRunner.query(`CREATE INDEX "IDX_8b516aad3e21ccfad9d4752797" ON "indexed_access_controls" ("identity_id") `); + await queryRunner.query(`CREATE INDEX "IDX_99627ee134548e9c4ce2e679c2" ON "indexed_access_controls" ("grantor") `); + await queryRunner.query(`CREATE INDEX "IDX_44b7ebe0d26d0b73b975facea0" ON "indexed_access_controls" ("grantee") `); + await queryRunner.query(`CREATE INDEX "IDX_35b780e6d74cb3063054bc807b" ON "indexed_access_controls" ("ledger_sequence") `); + await queryRunner.query(`CREATE INDEX "IDX_bf109db3e86614b72aa743d441" ON "indexed_access_controls" ("transaction_hash") `); + await queryRunner.query(`CREATE TYPE "public"."backup_records_backuptype_enum" AS ENUM('full', 'incremental', 'selective')`); + await queryRunner.query(`CREATE TYPE "public"."backup_records_status_enum" AS ENUM('pending', 'in_progress', 'completed', 'failed', 'restored')`); + await queryRunner.query(`CREATE TABLE "backup_records" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "walletAddress" character varying NOT NULL, "backupType" "public"."backup_records_backuptype_enum" NOT NULL DEFAULT 'full', "status" "public"."backup_records_status_enum" NOT NULL DEFAULT 'pending', "ipfsCid" character varying NOT NULL, "sizeBytes" bigint NOT NULL, "manifest" json, "checksum" character varying, "errorMessage" character varying, "restoreAttempts" integer NOT NULL DEFAULT '0', "lastRestoreTest" TIMESTAMP WITH TIME ZONE, "isRotatable" boolean NOT NULL DEFAULT true, "retentionDays" integer NOT NULL DEFAULT '0', "createdAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_13c40e36547fe8bc4903891715b" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_a0d8320885495e194aa13f222b" ON "backup_records" ("walletAddress") `); + await queryRunner.query(`CREATE TABLE "backup_configs" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "walletAddress" character varying NOT NULL, "autoBackupEnabled" boolean NOT NULL DEFAULT true, "backupIntervalHours" integer NOT NULL DEFAULT '24', "retentionDays" integer NOT NULL DEFAULT '30', "maxBackupsToKeep" integer NOT NULL DEFAULT '10', "healthMonitoringEnabled" boolean NOT NULL DEFAULT true, "healthCheckIntervalHours" integer NOT NULL DEFAULT '6', "alertsEnabled" boolean NOT NULL DEFAULT true, "alertWebhookUrl" json, "rotationEnabled" boolean NOT NULL DEFAULT true, "rotationCheckIntervalDays" integer NOT NULL DEFAULT '7', "restorationTestEnabled" boolean NOT NULL DEFAULT true, "restorationTestIntervalDays" integer NOT NULL DEFAULT '30', "createdAt" TIMESTAMP NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "UQ_f7ae51b8b3f0328c8a1b7e0289a" UNIQUE ("walletAddress"), CONSTRAINT "PK_25d918dbd367dcb0831412b7b85" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_f7ae51b8b3f0328c8a1b7e0289" ON "backup_configs" ("walletAddress") `); + await queryRunner.query(`CREATE TYPE "public"."backup_alerts_alerttype_enum" AS ENUM('backup_failed', 'backup_health_warning', 'backup_rotation_executed', 'restoration_test_failed', 'storage_quota_warning')`); + await queryRunner.query(`CREATE TYPE "public"."backup_alerts_severity_enum" AS ENUM('low', 'medium', 'high', 'critical')`); + await queryRunner.query(`CREATE TABLE "backup_alerts" ("id" uuid NOT NULL DEFAULT uuid_generate_v4(), "walletAddress" character varying NOT NULL, "alertType" "public"."backup_alerts_alerttype_enum" NOT NULL, "severity" "public"."backup_alerts_severity_enum" NOT NULL DEFAULT 'low', "message" text NOT NULL, "metadata" json, "acknowledged" boolean NOT NULL DEFAULT false, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), CONSTRAINT "PK_922a9e071b34879abfc1934c6a5" PRIMARY KEY ("id"))`); + await queryRunner.query(`CREATE INDEX "IDX_01a416e1df5c81b855c98be1ee" ON "backup_alerts" ("walletAddress") `); + await queryRunner.query(`ALTER TABLE "issuance_records" ADD CONSTRAINT "FK_8440c70760bbd133bc7284806e2" FOREIGN KEY ("authorityId") REFERENCES "health_authorities"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`); + } + + public async down(queryRunner: QueryRunner): Promise { + await queryRunner.query(`ALTER TABLE "issuance_records" DROP CONSTRAINT "FK_8440c70760bbd133bc7284806e2"`); + await queryRunner.query(`DROP INDEX "public"."IDX_01a416e1df5c81b855c98be1ee"`); + await queryRunner.query(`DROP TABLE "backup_alerts"`); + await queryRunner.query(`DROP TYPE "public"."backup_alerts_severity_enum"`); + await queryRunner.query(`DROP TYPE "public"."backup_alerts_alerttype_enum"`); + await queryRunner.query(`DROP INDEX "public"."IDX_f7ae51b8b3f0328c8a1b7e0289"`); + await queryRunner.query(`DROP TABLE "backup_configs"`); + await queryRunner.query(`DROP INDEX "public"."IDX_a0d8320885495e194aa13f222b"`); + await queryRunner.query(`DROP TABLE "backup_records"`); + await queryRunner.query(`DROP TYPE "public"."backup_records_status_enum"`); + await queryRunner.query(`DROP TYPE "public"."backup_records_backuptype_enum"`); + await queryRunner.query(`DROP INDEX "public"."IDX_bf109db3e86614b72aa743d441"`); + await queryRunner.query(`DROP INDEX "public"."IDX_35b780e6d74cb3063054bc807b"`); + await queryRunner.query(`DROP INDEX "public"."IDX_44b7ebe0d26d0b73b975facea0"`); + await queryRunner.query(`DROP INDEX "public"."IDX_99627ee134548e9c4ce2e679c2"`); + await queryRunner.query(`DROP INDEX "public"."IDX_8b516aad3e21ccfad9d4752797"`); + await queryRunner.query(`DROP INDEX "public"."IDX_0422607e1bf990304144128063"`); + await queryRunner.query(`DROP TABLE "indexed_access_controls"`); + await queryRunner.query(`DROP INDEX "public"."IDX_b2dc8aea87149ab251162c77dd"`); + await queryRunner.query(`DROP INDEX "public"."IDX_d3aae4fd70e25ac21d491c74cd"`); + await queryRunner.query(`DROP INDEX "public"."IDX_42c146a31098d47ee019d07691"`); + await queryRunner.query(`DROP INDEX "public"."IDX_21dbbd85da6bf2d09b01960933"`); + await queryRunner.query(`DROP INDEX "public"."IDX_a4ffe5a7b6c474d9336f42cd1a"`); + await queryRunner.query(`DROP INDEX "public"."IDX_f5401af24836b589ac9640b4bc"`); + await queryRunner.query(`DROP TABLE "indexed_data_sharing"`); + await queryRunner.query(`DROP INDEX "public"."IDX_7fd6ac7d9f7d086f7aedaee886"`); + await queryRunner.query(`DROP INDEX "public"."IDX_e0c9fc81dff0a1cc64c25e2973"`); + await queryRunner.query(`DROP INDEX "public"."IDX_fe3a18528380f95d4a8d74bda4"`); + await queryRunner.query(`DROP INDEX "public"."IDX_7e467b972045b3faf5bd748cae"`); + await queryRunner.query(`DROP INDEX "public"."IDX_d41457404b5f0a360f25385801"`); + await queryRunner.query(`DROP TABLE "indexed_identities"`); + await queryRunner.query(`DROP INDEX "public"."IDX_0e0a146c0c15e8399a7b57d027"`); + await queryRunner.query(`DROP INDEX "public"."IDX_3e6ec076a04ea8f3e2f34ba4c3"`); + await queryRunner.query(`DROP INDEX "public"."IDX_0dccc53b17457a35d45e4a9ce7"`); + await queryRunner.query(`DROP INDEX "public"."IDX_4e0b540710dbc25b421058d4a3"`); + await queryRunner.query(`DROP INDEX "public"."IDX_20133f03833a769b4277650b16"`); + await queryRunner.query(`DROP INDEX "public"."IDX_8a2cc7affaf3bc4a1144944da9"`); + await queryRunner.query(`DROP TABLE "indexed_verifications"`); + await queryRunner.query(`DROP TABLE "access_permissions"`); + await queryRunner.query(`DROP INDEX "public"."IDX_c69efb19bf127c97e6740ad530"`); + await queryRunner.query(`DROP INDEX "public"."IDX_2961862b2704794af56359ff0a"`); + await queryRunner.query(`DROP INDEX "public"."IDX_acdc6af76706ad7d8aa4029e28"`); + await queryRunner.query(`DROP INDEX "public"."IDX_3f521f4fa46f1e21f3becf021b"`); + await queryRunner.query(`DROP INDEX "public"."IDX_2dc33f7f3c22e2e7badafca1d1"`); + await queryRunner.query(`DROP INDEX "public"."IDX_a1a8da97e76ef043712fdeb694"`); + await queryRunner.query(`DROP TABLE "audit_logs"`); + await queryRunner.query(`DROP TYPE "public"."audit_logs_status_enum"`); + await queryRunner.query(`DROP TYPE "public"."audit_logs_operationtype_enum"`); + await queryRunner.query(`DROP INDEX "public"."IDX_97e2b96eee5be7d900f71e51cc"`); + await queryRunner.query(`DROP TABLE "credential_exports"`); + await queryRunner.query(`DROP TYPE "public"."credential_exports_status_enum"`); + await queryRunner.query(`DROP TYPE "public"."credential_exports_format_enum"`); + await queryRunner.query(`DROP TABLE "sharing_history_events"`); + await queryRunner.query(`DROP TYPE "public"."sharing_history_events_action_enum"`); + await queryRunner.query(`DROP INDEX "public"."IDX_77f05bf0e293a614e9f50699e2"`); + await queryRunner.query(`DROP INDEX "public"."IDX_f07b0520b34fb6b2a02b092c87"`); + await queryRunner.query(`DROP INDEX "public"."IDX_8080932dda76eda6b6b6f5c3d9"`); + await queryRunner.query(`DROP TABLE "credential_migrations"`); + await queryRunner.query(`DROP TYPE "public"."credential_migrations_status_enum"`); + await queryRunner.query(`DROP INDEX "public"."IDX_ad6b1c640fb1846122557fb962"`); + await queryRunner.query(`DROP INDEX "public"."IDX_5fa4d1928dc2f0627286589b34"`); + await queryRunner.query(`DROP INDEX "public"."IDX_8e4ee079a148d3e7fa5ff54b81"`); + await queryRunner.query(`DROP TABLE "credential_versions"`); + await queryRunner.query(`DROP INDEX "public"."IDX_9c5dd783b8d84dfbda8fd7b98b"`); + await queryRunner.query(`DROP TABLE "credentials"`); + await queryRunner.query(`DROP TABLE "shared_data"`); + await queryRunner.query(`DROP INDEX "public"."IDX_706ccfe85d207fabd3b966c5ba"`); + await queryRunner.query(`DROP INDEX "public"."IDX_8440c70760bbd133bc7284806e"`); + await queryRunner.query(`DROP TABLE "issuance_records"`); + await queryRunner.query(`DROP TYPE "public"."issuance_records_status_enum"`); + await queryRunner.query(`DROP TYPE "public"."issuance_records_format_enum"`); + await queryRunner.query(`DROP INDEX "public"."IDX_a12fc85b426c020664339756b8"`); + await queryRunner.query(`DROP TABLE "health_authorities"`); + await queryRunner.query(`DROP TYPE "public"."health_authorities_status_enum"`); + await queryRunner.query(`DROP TYPE "public"."health_authorities_authtype_enum"`); + await queryRunner.query(`DROP TABLE "identities"`); + await queryRunner.query(`DROP INDEX "public"."IDX_7af9f9b4f800572efdc68dd6a6"`); + await queryRunner.query(`DROP INDEX "public"."IDX_0fe58a810ad39c1f2eb41ac5e4"`); + await queryRunner.query(`DROP INDEX "public"."IDX_c9d9f1f1983bde0720a6047d18"`); + await queryRunner.query(`DROP TABLE "role_assignments"`); + await queryRunner.query(`DROP TYPE "public"."role_assignments_role_enum"`); + await queryRunner.query(`DROP INDEX "public"."IDX_64099fd2aafd220c32a480a439"`); + await queryRunner.query(`DROP INDEX "public"."IDX_2e31645aac05e5b108d321042d"`); + await queryRunner.query(`DROP TABLE "verifications"`); + await queryRunner.query(`DROP TYPE "public"."verifications_status_enum"`); + } + +} diff --git a/scripts/deploy-backend.sh b/scripts/deploy-backend.sh index b37d66d9..2631e667 100755 --- a/scripts/deploy-backend.sh +++ b/scripts/deploy-backend.sh @@ -41,7 +41,7 @@ echo "" # Run database migrations echo -e "${YELLOW}Running database migrations...${NC}" -npm run typeorm migration:run +npm run migration:run:prod echo -e "${GREEN}Migrations complete${NC}" echo ""