Skip to content

jota2rz/vdj-video-sync

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

54 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

VDJ Video Sync

Build Server Build Plugin

Synchronize local video playback with VirtualDJ — a C++ plugin sends real-time deck state to a Go server, which serves a browser-based video player that matches and syncs videos by exact filename, exact stem, filename similarity, BPM or random.

100% vibe coded with Claude Opus 4.6

Architecture

VDJ Plugin (C++ DLL)  ──HTTP POST──▶  Go Server  ──SSE──▶  Browser Clients
        │                               │                    ├─ /dashboard
        │  deck state every 50ms        │                    ├─ /library
        │  (filename, BPM, pitch,       │                    ├─ /transitions
        │   volume, elapsed, play/      │                    ├─ /overlay
        │   pause, audible)             │                    └─ /player
        │                               └─ SQLite (config, BPM cache
        │                                           & transition effects)
        └── VirtualDJ 8 DSP Plugin

Features

Note: Mobile devices are not supported. The server UI requires simultaneous decoding of multiple video streams, triple-buffered transitions, canvas mirroring, and real-time playback rate adjustments — all of which exceed mobile browser video decoding performance. A desktop or laptop browser is required.

Video Matching (6-level tiered fallback)

Level Type Description
0 Exact Video filename matches song filename exactly (with extension)
1 Stem Filename match without extension
2 Fuzzy ≥70% filename similarity (Levenshtein-based)
3 BPM + Fuzzy Closest BPM among videos with ≥30% name similarity; random pick from top 5
4 BPM Closest BPM match; random pick from top 5 closest candidates
5 Random Any random video (stable pick by song hash)
  • Half-time BPM correction: automatically detects and corrects half-time BPM readings
  • BPM from audio analysis (AAC & Opus codecs, pure Go — no ffmpeg)
  • Filename BPM fallback: parses BPM from filename (e.g. loop_128bpm.mp4) when audio analysis is unavailable

Playback Synchronization

  • Elapsed time sync — Levels 0-1 sync to VDJ elapsed time; levels 2+ sync to server-tracked independent position
  • Playback rate — BPM-based rate matching: (pitch / 100) × (deckBPM / videoBPM), clamped to 0.25–4.0x
  • Drift correction — Soft catch-up (±15% rate adjustment) when drift >150ms, hard seek when >2000ms
  • Video looping — Levels 0-1 loop with transition when video ends before the song; levels 2+ pick a different random video

Transition System

  • Server-driven triple-buffered design: 3 transition videos preloaded at all times
  • Server chooses which buffer to play — all clients show the same transition
  • Configurable duration (1-10 seconds) and enable/disable toggle from the control bar
  • Uses fetch() + blob URLs to bypass Chrome's 6-media-preload limit
  • Syncs playback rate to the incoming deck's BPM + pitch
  • CSS transition effects — animated "in" and "out" effects applied during transitions with a phased timeline (15% in → 70% hold → 15% out)
  • 12 built-in effects: Fade, Dissolve, Flash, Zoom, Iris, Glitch (in/out pairs)
  • Custom effects with full CSS @keyframes support
  • Server picks a random enabled effect per direction for each transition

Overlay System

  • Customizable CSS/JS overlay elements rendered on top of or behind video transitions
  • 5 built-in seed elements: BPM, Song Name, Artist, Progress Bar, Custom Logo
  • Two z-index layers: elements can be configured to show above transitions (z=100) or behind transitions (z=16)
  • Show Over Transition — per-element toggle; behind-layer elements defer updates until the transition's "in" effect finishes
  • Custom Logo — upload a PNG logo with a default highlight sweep CSS animation (fully customizable via the overlay CSS editor)
  • Data-driven visibility — elements automatically fade in when data is available and fade out when empty (e.g., BPM hides when no BPM data, song name hides when no track is loaded)
  • No audible deck — all overlay elements fade out when no deck is audible and playing
  • Each element has editable CSS and JavaScript; HTML is read-only
  • Full CRUD overlay management API with SSE broadcast on changes
  • Overlay modal with live editing (HTML above CSS for quick reference)

Dashboard (/dashboard)

  • Real-time deck status cards (decks 1-2 always shown, 3-4 appear/hide dynamically)
  • Deck cards show Song Name – Artist and elapsed / total time (MM:SS)
  • Embedded master video player with match type, playback rate, and BPM info
  • Canvas-mirrored per-deck video previews (~15 fps)
  • Active deck pulse animation
  • Deck limit warning banner (decks > 4)
  • BPM analysis overlay with progress indicator
  • Auto-scaling player and deck previews

Loop Video

  • Designate any song video as the "Loop Video" from the Library page
  • "Use Loop Video" toggle in the control bar activates the loop video on top of all deck videos
  • Loop video plays indefinitely until the toggle is disabled
  • Server-synced transitions — server picks the CSS effect and broadcasts it to all clients, so every browser shows the same transition animation
  • Deck switches happen silently underneath when loop video is active
  • Loop icon indicator (Heroicons) shown next to the designated video in the song list
  • Auto-cleared if the loop video file is deleted from disk
  • Warning banner when the toggle is enabled but no loop video is selected

Video Library (/library)

  • Tabbed browser: Song Videos / Transition Videos
  • Search filter
  • Video preview with click-to-pause
  • "Set Loop Video" — designate a song video as the loop video (emerald button)
  • "Force Master Video" — force a video on the active deck with transition
  • "Force Deck 1-4" — force a video on a specific deck
  • Auto-refreshes when server detects file changes on disk
  • Warning banners when no song or transition videos are found

Transition Effects (/transitions)

  • Browse and manage CSS "in" and "out" transition effects
  • 12 built-in effects (Fade, Dissolve, Flash, Zoom, Iris, Glitch — each with in/out variants)
  • Create custom effects with full CSS @keyframes and .transition-active class
  • Enable/disable individual effects — server picks randomly from enabled effects
  • Live preview with shuffleable sample videos
  • Built-in effects are read-only and cannot be deleted
  • Warning banner when all effects in a direction are disabled
  • Real-time sync across tabs via SSE

Standalone Player (/player)

  • Fullscreen video output (opens in new tab, no UI)
  • Same transition, sync, and overlay logic as the embedded dashboard player
  • Configurable aspect ratio from the settings modal (default 16:9)
  • Responsive sizing: constrains to viewport in both landscape and portrait orientations
  • "Waiting for track..." fallback when no video is matched

Control Bar

  • Transitions enabled/disabled toggle
  • Transition Videos toggle — when off, deck switches use CSS effects only (no video overlay); the old deck's "out" effect plays while the new deck is revealed underneath
  • Transition Duration ± buttons (1-10 seconds)
  • Use Loop Video toggle — when on, plays the designated loop video on top of all deck content; dimmed/locked when no loop video is set
  • Overlay enabled/disabled toggle — show or hide the overlay layer
  • Warning banner when loop video is enabled with no video selected
  • Config changes sync across all tabs via BroadcastChannel + SSE

Settings

  • Videos directory and transition videos directory configurable from the UI
  • Aspect ratio selector — choose between 16:9, 4:3, 21:9, 1:1, or 9:16; applies to player, dashboard, and library previews
  • Config persisted in SQLite, synced to all clients via SSE
  • Graceful server shutdown from the UI

Real-time Communication

  • Plugin → Server: HTTP POST every 50ms per deck (JSON)
  • Server → Browser: Server-Sent Events (SSE) via SharedWorker (single connection shared across all tabs to stay within HTTP/1.1 connection limits)
  • Cross-tab sync: BroadcastChannel for instant same-browser config propagation
  • Loop video cleanup: server auto-clears loop video config when the file is deleted from disk
  • Event types: deck-update, transition-pool, transition-play, deck-visibility, analysis-status, library-updated, config-updated, transitions-updated, overlay-updated, loop-video-transition

VDJ Plugin

  • VirtualDJ 8 DSP plugin (no audio modification — pass-through)
  • Polls deck state every 50ms in a background thread
  • Sends: deck number, filename, title, artist, BPM, pitch, volume, elapsed time, total time, playing, audible
  • Duplicate/mirrored deck detection (filters VDJ master-bus mirrors)
  • Change detection to minimize redundant HTTP traffic
  • Paused-deck seek detection (>50ms threshold to filter VDJ clock jitter)

Project Structure

├── .github/workflows/          # CI/CD pipelines
│   ├── build-plugin.yml        # Plugin builds (Windows x64, macOS arm64/amd64)
│   └── build-server.yml        # Server builds (Windows x64, macOS Universal, Linux x64)
│
├── docs/                       # GitHub Pages site (download page)
│
├── plugin/                     # C++ VirtualDJ plugin
│   ├── CMakeLists.txt
│   ├── src/
│   │   ├── main.cpp            # DllGetClassObject entry point
│   │   ├── VideoSyncPlugin.h
│   │   ├── VideoSyncPlugin.cpp
│   │   ├── VdjVideoSync.def    # DLL exports
│   │   └── Info.plist.in       # macOS bundle plist template
│   └── vendor/
│       └── httplib.h           # cpp-httplib (downloaded automatically by CI; for local builds, download manually)
│
├── server/                     # Go server
│   ├── main.go                 # HTTP server, routing, CLI flags
│   ├── go.mod
│   ├── Makefile
│   ├── internal/
│   │   ├── bpm/                # Audio BPM analysis (AAC/Opus → onset detection)
│   │   │   ├── bpm.go          # MP4 parsing, codec detection, autocorrelation
│   │   │   └── cache.go        # SQLite-backed BPM cache
│   │   ├── browser/            # Auto-open dashboard in browser on startup
│   │   ├── config/             # Thread-safe key-value config (SQLite-backed)
│   │   ├── db/                 # Database init & schema
│   │   ├── handlers/           # HTTP & SSE handlers
│   │   ├── models/             # Shared data types
│   │   ├── sse/                # Pub/sub hub for Server-Sent Events
│   │   ├── transitions/        # Transition effects CRUD store
│   │   ├── overlay/            # Overlay elements CRUD store
│   │   └── video/              # Video scanner, matcher, directory watcher
│   ├── templates/              # Templ templates (.templ → _templ.go)
│   │   ├── layouts/            # Base HTML layout
│   │   ├── pages/              # Dashboard, Library, Player, Transitions
│   │   └── components/         # Header (nav), ControlBar (bottom)
│   └── static/
│       ├── css/                # Tailwind CSS (input.css → output.css)
│       └── js/
│           ├── app.js          # All client-side logic (~4100 lines)
│           └── sse-worker.js   # SharedWorker for SSE connection sharing
│
├── VirtualDJ8_SDK_20211003/    # VDJ SDK headers (downloaded automatically by CI; for local builds, download manually)
│
└── LICENSE.md

Prerequisites

Plugin (C++)

  • CMake 3.20+
  • C++17 compiler: MSVC (Visual Studio 2022) on Windows, Clang/Xcode on macOS
  • cpp-httplib — download httplib.h and place it in plugin/vendor/httplib.h
  • VirtualDJ SDK — download the SDK headers and place them in VirtualDJ8_SDK_20211003/

Note: cpp-httplib and the VirtualDJ SDK are not distributed with this project. cpp-httplib is MIT-licensed but too large to vendor in git; the VDJ SDK has no clear open-source license. Download both before building.

Server (Go)

Building

Plugin

Windows:

cd plugin
cmake -B build -A x64
cmake --build build --config Release
# Output: build/out/Release/VdjVideoSync.dll
# Copy to: %USERPROFILE%/AppData/Local/VirtualDJ/Plugins64/SoundEffect/

macOS:

cd plugin
cmake -B build
cmake --build build --config Release
# Output: build/out/VdjVideoSync.bundle
# Copy to: ~/Library/Application Support/VirtualDJ/Plugins64/SoundEffect/

Based on szemek/virtualdj-plugins-examples for XCode compatibility.

Server

The Go Server compiles natively on Windows, macOS, and Linux — all dependencies are pure Go (no CGo).

cd server

# Full build (generate templates + build CSS + compile Go binary)
make build

# Run after building
make run

# Development mode (watches templ, tailwind, and Go files)
make dev

# Clean generated files
make clean

Or manually:

cd server
templ generate
tailwindcss -i static/css/input.css -o static/css/output.css --minify
# -p 1 and -gcflags prevent OOM from concentus/SILK codec compilation
go build -p 1 -gcflags="github.com/lostromb/concentus/...=-N -l" -o vdj-video-sync-server .    # Linux / macOS
go build -p 1 -gcflags="github.com/lostromb/concentus/...=-N -l" -o vdj-video-sync-server.exe .  # Windows
./vdj-video-sync-server -port :8090 -videos ./videos

You can also cross-compile from any OS:

# -p 1 and -gcflags prevent OOM from concentus/SILK codec compilation
GOOS=windows GOARCH=amd64 go build -p 1 -gcflags="github.com/lostromb/concentus/...=-N -l" -o vdj-video-sync-server.exe .
GOOS=darwin  GOARCH=arm64 go build -p 1 -gcflags="github.com/lostromb/concentus/...=-N -l" -o vdj-video-sync-server .
GOOS=linux   GOARCH=amd64 go build -p 1 -gcflags="github.com/lostromb/concentus/...=-N -l" -o vdj-video-sync-server .

Usage

  1. Start the server — the dashboard opens automatically in your default browser
  2. Put VdjVideoSync.dll at Plugins64/SoundEffect/
  3. Launch VirtualDJ and enable the Master Effect called VdjVideoSync
  4. (Optional) To change the server IP or port, open Effect Controls and click Set IP or Set Port — values are validated and saved automatically
  5. Open http://localhost:8090/player in a separate window/tab/screen for fullscreen video output
  6. Place video files in the configured videos directory (.mp4 with AAC or Opus SILK or CELT audio, this means it's YouTube compatible)
  7. Place transition videos in the transition videos directory

Server flags

Flag Default Description
-port :8090 HTTP listen port
-db vdj-video-sync.db SQLite database path
-videos ./videos Directory containing video files
-transition-videos ./transition-videos Directory containing transition video files
-debug false Enable debug logging (also disables auto-open browser)
-no-browser false Do not open the dashboard in a browser on startup

Headless / server environments: On Linux, the browser is not opened when neither $DISPLAY nor $WAYLAND_DISPLAY is set. On any platform you can pass -no-browser to skip the attempt entirely.

Dependencies

Go Server

Package Purpose
templ Type-safe HTML templating
go-mp4 MP4 container parsing — extracts audio tracks from video files
concentus (Go port) Pure-Go Opus decoder (SILK + CELT) (YouTube compatibility) — decodes audio for BPM analysis
go-aac Pure-Go AAC decoder — decodes audio for BPM analysis
sqlite Pure-Go SQLite driver

All dependencies are pure Go — no CGo, no ffmpeg, no native libraries required.

VirtualDJ C++ Plugin

Library Purpose
cpp-httplib Single-header HTTP client for sending deck state (downloaded automatically by CI; for local builds, download manually)
VirtualDJ SDK Plugin interface headers (downloaded automatically by CI; for local builds, download manually)

Frontend

Technology Purpose
Tailwind CSS v4 Utility-first CSS framework
Vanilla JavaScript No framework — single app.js file

AI

Technology Purpose
Claude Opus 4.6 Vibe-coded the entire application — architecture, plugin, server, frontend, and this README

License

GPL-3.0

About

Browser-based Automatic BPM-synced video mixing for VirtualDJ

Topics

Resources

License

Stars

Watchers

Forks

Contributors