A production-grade conversational speech data collection platform. Contributors record paired conversations through a browser-based interface, producing studio-quality WAV files for AI/ML training data.
Live: voice.usergy.ai | API: api.voice.usergy.ai | LiveKit: livekit.voice.usergy.ai
| Layer | Technology |
|---|---|
| Frontend | Next.js 15 (App Router), React 19, Tailwind CSS v4, Zustand |
| Backend | Express.js, TypeScript, Zod validation |
| Database | PostgreSQL 15 + Prisma ORM |
| Storage | Azure Blob Storage (chunked uploads, SAS tokens) |
| Real-time | LiveKit (WebRTC rooms for live conversation) |
| Auth | Email OTP (AWS SES) + Google OAuth + JWT (access/refresh rotation) |
| Audio | AudioWorklet PCM capture, EBU R128 normalization, ffmpeg |
| Monorepo | Turborepo + npm workspaces |
voice-platform/
├── apps/
│ ├── api/ # Express backend (port 3001)
│ │ └── src/
│ │ ├── config/env.ts # Zod-validated environment variables
│ │ ├── index.ts # Express app, middleware pipeline, startup
│ │ ├── lib/
│ │ │ ├── email.ts # AWS SES email sending
│ │ │ ├── jwt.ts # JWT sign/verify helpers
│ │ │ ├── livekit.ts # LiveKit token generation
│ │ │ ├── logger.ts # Winston structured logger + request context
│ │ │ ├── otp.ts # OTP generation and hashing
│ │ │ ├── prisma.ts # Prisma client singleton
│ │ │ ├── request-context.ts # AsyncLocalStorage request correlation
│ │ │ ├── sms.ts # AWS SNS SMS sending
│ │ │ └── storage.ts # Azure Blob: upload chunks, merge, SAS URLs
│ │ ├── middlewares/
│ │ │ ├── admin.ts # Admin role check
│ │ │ ├── auth.ts # JWT verification + user attachment
│ │ │ ├── csrf.ts # Origin/Referer validation on mutations
│ │ │ ├── error-handler.ts # Global error handler with AppError
│ │ │ ├── rate-limit.ts # Tiered rate limiting (OTP, upload, general)
│ │ │ └── request-id.ts # X-Request-ID injection
│ │ ├── routes/
│ │ │ ├── admin.ts # Dashboard stats, recordings/users/earnings CRUD
│ │ │ ├── auth.ts # OTP, magic link, Google OAuth, refresh, logout
│ │ │ ├── consent.ts # Recording & data usage consent management
│ │ │ ├── health.ts # Liveness + readiness with dependency checks
│ │ │ ├── profile.ts # User profile, stats, payment details
│ │ │ ├── recordings.ts # Chunk upload, finalize, status
│ │ │ ├── sessions.ts # Create, join, start, end recording sessions
│ │ │ └── webhooks.ts # LiveKit webhook receiver
│ │ ├── services/
│ │ │ ├── auth.service.ts # Auth business logic (OTP, Google, tokens)
│ │ │ └── qa.service.ts # Audio quality analysis (EBU R128, 15 metrics)
│ │ └── workers/
│ │ ├── merge.worker.ts # Chunk stitching → 24-bit WAV (atomic claim, 3 retries)
│ │ └── qa.worker.ts # Automated QA scoring → earnings calculation
│ │
│ └── web/ # Next.js frontend (port 3000)
│ ├── public/
│ │ └── audio-processor.worklet.js # AudioWorklet PCM capture (runs in audio thread)
│ └── src/
│ ├── app/
│ │ ├── admin/ # Admin dashboard (stats, recordings, users, earnings)
│ │ ├── dashboard/ # User dashboard (sessions, stats)
│ │ ├── join/[code]/ # Join session via invite code
│ │ ├── login/ # Email OTP + Google Sign-In
│ │ ├── profile/ # Profile + payment details
│ │ ├── sessions/[id]/ # Recording room (consent → health check → mic check → recording)
│ │ └── verify/ # OTP verification
│ ├── components/
│ │ ├── RecordingErrorBoundary.tsx # Crash recovery during recording
│ │ ├── auth/GoogleSignInButton.tsx
│ │ ├── recording/ # AudioLevelBar, ConsentStep, HealthCheckStep, RecordingTimer, UploadProgress
│ │ └── ui/ # Button, Input (design system)
│ ├── hooks/
│ │ ├── useAudioLevel.ts # Real-time mic level via AnalyserNode
│ │ ├── useAuth.ts # Auth state + token refresh
│ │ ├── useChunkUploader.ts # Upload queue, retries, IndexedDB offline buffer
│ │ ├── useLiveKit.ts # LiveKit room connection + shared stream
│ │ ├── usePCMRecorder.ts # AudioWorklet lossless PCM recording
│ │ └── useRecorder.ts # MediaRecorder fallback
│ └── lib/
│ ├── api-client.ts # Typed API client
│ ├── auth-store.ts # Zustand auth state
│ ├── recording-store.ts # Zustand recording state (sessionStorage-persisted)
│ └── utils.ts # Utility functions
│
├── packages/
│ ├── db/ # Prisma schema, migrations, seed
│ │ └── prisma/
│ │ └── schema.prisma # 18 models, 11 enums
│ ├── shared/ # Shared types & utilities
│ └── tsconfig/ # Shared TypeScript configs
│
├── infra/
│ ├── docker-compose.yml # LiveKit + Redis production stack
│ └── livekit.yaml # LiveKit server configuration
│
├── docs/ # Documentation (see below)
│ ├── API.md
│ ├── ARCHITECTURE.md
│ ├── DATABASE.md
│ ├── DEPLOYMENT.md
│ ├── SECURITY.md
│ └── WORKERS.md
│
├── .env.example # Environment variable template
├── turbo.json # Turborepo task configuration
└── package.json # Root workspace config
- Node.js 20+
- PostgreSQL 15+
- Azure Storage Account (or Azurite for local dev)
- AWS account (SES for emails, optional SNS for SMS)
- LiveKit server (optional for local dev)
# Clone and install
git clone <repo-url>
cd voice-platform
npm install
# Configure environment
cp .env.example .env
# Edit .env with your credentials (see .env.example for all variables)
# Database setup
npm run db:generate # Generate Prisma client
npm run db:push # Push schema to PostgreSQL
# Start development servers (API + Web)
npm run devThe API runs on http://localhost:3001 and the web app on http://localhost:3000.
# API only
cd apps/api && npm run dev
# Web only
cd apps/web && npm run dev
# Prisma Studio (database GUI)
cd packages/db && npx prisma studio┌─────────────────────────────────────────────────────────────────┐
│ Participant's Browser │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ AudioWorklet │ │ LiveKit │ │ Chunk Uploader │ │
│ │ PCM Capture │───▶│ (WebRTC) │ │ (10s intervals) │ │
│ │ (lossless) │ │ real-time │ │ WAV → Azure │ │
│ └──────┬───────┘ │ conversation│ └────────┬─────────┘ │
│ │ └──────────────┘ │ │
│ └─────────────────────────────────────────┘ │
└───────────────────────────────┬─────────────────────────────────┘
│ HTTPS (chunked WAV upload)
▼
┌─────────────────────────────────────────────────────────────────┐
│ Express API │
│ │
│ Auth ─── Sessions ─── Recordings ─── Admin ─── Health │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌──────────────────┐ │
│ │ Merge Worker │───▶│ QA Worker │───▶│ Earnings Engine │ │
│ │ (ffmpeg + │ │ (EBU R128, │ │ ($8/hr, auto- │ │
│ │ EBU R128) │ │ 15 metrics)│ │ calculate) │ │
│ └──────┬───────┘ └─────────────┘ └──────────────────┘ │
│ │ │
└─────────┼───────────────────────────────────────────────────────┘
│
▼
┌──────────────┐ ┌──────────────┐ ┌──────────────────┐
│ Azure Blob │ │ PostgreSQL │ │ LiveKit │
│ Storage │ │ (18 models) │ │ (WebRTC) │
└──────────────┘ └──────────────┘ └──────────────────┘
- AudioWorklet PCM Capture — Lossless Float32 samples captured in audio thread
- WAV Encoding — Each 10-second chunk encoded as standalone WAV in browser
- Chunked Upload — Upload queue with 5 retries, offline IndexedDB buffer
- Merge Worker — Atomic job claim, ffmpeg concat, EBU R128 normalization to -23 LUFS
- QA Worker — 15 audio metrics (loudness, clipping, DC offset, silence, true peak)
- Earnings — Auto-calculated at $8/hour for passed recordings
| Decision | Choice | Why |
|---|---|---|
| Recording | AudioWorklet PCM (Float32) | Truly lossless, no codec, consistent cross-browser |
| Fallback | MediaRecorder (PCM/FLAC/Opus) | For browsers without AudioWorklet support |
| Upload | Chunked (10s intervals) | Zero data loss on browser crash, offline buffer |
| Output | 24-bit PCM WAV, 48kHz, mono | Studio quality for AI training |
| Normalization | EBU R128 (-23 LUFS) | Broadcast standard loudness |
| Storage | Azure Blob | SAS token scoping, cost-effective |
| Real-time | LiveKit WebRTC | Low-latency conversation with shared mic stream |
| Auth | JWT access (15m) + refresh (30d) | Short-lived access, rotated refresh tokens |
| Queue | DB-backed MergeJob table | No Redis dependency for job queue |
| Document | Description |
|---|---|
| API Reference | All endpoints, request/response, auth requirements |
| Architecture | System design, flows, state machines, middleware |
| Database Schema | All 18 models, enums, relationships, indexes |
| Security | Auth, CSRF, rate limiting, CSP, data protection |
| Deployment | Production setup, environment, infrastructure |
| Workers | Merge and QA worker internals, monitoring |
# Run full stack
npm run dev
# Type checking
cd apps/api && npx tsc --noEmit
cd apps/web && npx tsc --noEmit
# Database operations
npm run db:generate # Regenerate Prisma client after schema changes
npm run db:push # Push schema changes to database
npm run db:migrate # Create and apply migration
# Prisma Studio
cd packages/db && npx prisma studioProprietary — UsergyAI. All rights reserved.