Scene Ripper

Automatic scene detection, AI-powered analysis, and algorithmic video remixing for filmmakers and video artists.

What It Does

Scene Ripper analyzes video files to detect scene boundaries, enriches clips with AI-generated metadata (descriptions, shot types, transcripts), and lets you remix clips into new sequences using algorithmic composition or AI assistance.

Key Features:

• Automatic scene detection with adjustable sensitivity
• Text-based scene detection for karaoke, subtitles, and slides
• Intention-first workflow (one-click sequence creation from URLs)
• AI-powered clip analysis (descriptions, shot types, transcription, colors)
• Film language analysis with scene reports and editing suggestions
• Integrated chat agent for AI-assisted editing
• Thumbnail grid browser with collapsible source headers
• 21 sequencing modes including sort, similarity, audio, text, gaze, reference, and drawing-guided workflows
• DINOv2 Visual Match for visual similarity workflows (details)
• Export individual clips or complete sequences
• YouTube/Internet Archive import
• Project save/load with full state persistence

Installation

Detailed install guide: docs/user-guide/installation.md

Release process for maintainers: docs/releases.md

Requirements

• macOS 13 Ventura or newer on Apple Silicon for the downloadable app
• Windows 10/11 64-bit or a recent x86_64 Linux desktop for packaged builds
• Python 3.11+ when running from source
• At least 5 GB free disk space for the app, FFmpeg, cached thumbnails, and optional model/runtime downloads; 10 GB+ is recommended if you use local transcription, face detection, or large video imports

Downloaded runtimes and caches live under the platform app-support directory: ~/Library/Application Support/Scene Ripper/ on macOS, %LOCALAPPDATA%\Scene Ripper\ on Windows, and ~/.local/share/scene-ripper/ on Linux. Logs live in ~/Library/Logs/Scene Ripper/ on macOS and under the app-support logs/ directory elsewhere.

Linux

Option 1: AppImage (Recommended)

Download the latest AppImage from Releases. Open the newest release, expand Assets, and choose the Scene_Ripper-*-x86_64.AppImage file:

# Download (replace VERSION with actual version)
wget https://github.com/dvschultz/algorithmic-filmmaking/releases/download/vVERSION/Scene_Ripper-VERSION-x86_64.AppImage

# Make executable and run
chmod +x Scene_Ripper-*-x86_64.AppImage
./Scene_Ripper-*-x86_64.AppImage

Option 2: From Source

# Install system dependencies first (see System Dependencies below)
# Ubuntu/Debian:
sudo apt install python3 python3-pip ffmpeg \
    gstreamer1.0-plugins-good gstreamer1.0-plugins-bad \
    gstreamer1.0-plugins-ugly gstreamer1.0-libav

# Clone and install
git clone https://github.com/dvschultz/algorithmic-filmmaking.git
cd algorithmic-filmmaking
pip install -r requirements.txt

# Run
python main.py

macOS

Downloadable app (Apple Silicon)

Download the latest DMG from Releases. Open the newest release, expand Assets, choose the Apple Silicon macOS DMG, open it, and drag Scene Ripper.app into Applications.

The downloadable macOS app requires macOS 13 Ventura or newer and Apple Silicon. The app bundle declares this minimum system version so older macOS versions should refuse launch before startup.

On first launch, the app downloads FFmpeg and other optional dependencies as needed. macOS may ask for Keychain access when you save or use API keys; allow it if you want Scene Ripper to read stored credentials. You can avoid Keychain prompts by using environment variables such as OPENAI_API_KEY, GEMINI_API_KEY, or YOUTUBE_API_KEY.

If you plan to use face detection features such as Rose Hobart on a new Mac, install Xcode Command Line Tools first:

xcode-select --install

From source

# Install Xcode Command Line Tools first if you want face detection features
xcode-select --install

# Install FFmpeg
brew install ffmpeg

# Clone and install
git clone https://github.com/dvschultz/algorithmic-filmmaking.git
cd algorithmic-filmmaking
pip install -r requirements.txt

# Run
python main.py

Windows

Downloadable app

Download the latest Windows installer from Releases. Open the newest release, expand Assets, choose SceneRipper-Setup-*.exe, and run the installer.

The first run downloads FFmpeg and other optional dependencies when features need them. Early unsigned builds may trigger a Windows SmartScreen warning.

From source

1. Install Python 3.11+ from python.org
2. Install FFmpeg from ffmpeg.org and add to PATH
3. Clone and run:

git clone https://github.com/dvschultz/algorithmic-filmmaking.git
cd algorithmic-filmmaking
pip install -r requirements.txt
python main.py

Quick Start

Using Make (recommended for development):

make run        # Launch the GUI
make watch      # Auto-restart on code changes (requires watchdog)
make test       # Run tests

Or run directly:

python main.py

Tab-Based Workflow:

The app uses a 5-tab workflow, though you can skip or revisit tabs as needed:

Tab	Purpose
Collect	Import videos (local files, YouTube, Internet Archive)
Cut	Detect scenes and browse clips
Analyze	Enrich clips with AI metadata (descriptions, transcripts, etc.)
Sequence	Arrange clips using various sequencing modes
Render	Export final video or EDL

Basic Flow:

1. Collect: Import a video file (drag-drop or file browser)
2. Cut: Adjust sensitivity and click "Detect Scenes"
3. Analyze: (Optional) Run analysis to add descriptions, transcripts, shot types
4. Sequence: Add clips to the timeline manually or generate one with a sequencer algorithm
5. Render: Export as MP4 or EDL for external editors

User Guides

• Chat Agent
• Agent Tools Reference
• Analysis
• Sequencer Algorithms
• Local AI Models
• API Keys
• Prompt Reference
• Reporting Bugs

Features

Scene Detection

• PySceneDetect with AdaptiveDetector
• Sensitivity slider (1.0 sensitive → 10.0 less sensitive)
• Text-based detection mode for karaoke, subtitles, and presentation slides—uses OCR to detect text changes as scene boundaries with configurable ROI and similarity threshold
• Background processing keeps UI responsive
• Progress reporting

AI-Powered Analysis

Enrich clips with metadata using local or cloud AI models:

Analysis	Description	Models
Describe	Natural language description of clip content	GPT-4o, Claude, Gemini, Moondream (local)
Classify	Shot type detection (close-up, medium, wide, etc.)	CLIP (local), VideoMAE (cloud)
Cinematography	Full film language analysis (shot size, angle, movement, lighting, composition)	Gemini (video or frame)
Transcribe	Speech-to-text	faster-whisper (local)
Colors	Dominant color palette extraction	OpenCV
Objects	Object detection	YOLO

Supports both local (free, private) and cloud (higher quality) processing tiers.

Shot Type Classification Tiers:

• CPU (local): CLIP zero-shot classification with ensemble prompts—runs on thumbnails, free and private
• Cloud: VideoMAE model on Replicate—analyzes video segments for significantly better accuracy, requires API key

Cinematography Analysis

Rich VLM-powered analysis providing professional-grade film language breakdown:

Category	Properties
Shot Size	Granular classification: ELS, VLS, LS, MLS, MS, MCU, CU, BCU, ECU, Insert
Camera Angle	low_angle, eye_level, high_angle, dutch_angle, birds_eye, worms_eye + emotional effect
Camera Movement	static, pan, tilt, track, handheld, crane, arc (video mode only)
Composition	Subject position, headroom, lead room, visual balance
Lighting	Style (high/low key), direction, quality (hard/soft), color temperature
Technical	Dutch tilt detection, lens type estimation, depth of field
Derived	Emotional intensity, suggested pacing

Analysis Modes:

• Frame mode: Fast analysis from thumbnails
• Video mode: Analyzes full clip segment for camera movement detection (requires Gemini API)

Film Language Analysis

Generate scene reports with cinematography analysis:

• Pacing metrics: Shot duration distribution, rhythm analysis
• Visual consistency: Color palette coherence, lighting continuity
• Continuity warnings: Potential jump cuts, axis crosses
• Editing suggestions: AI-powered recommendations for improving flow

Chat Agent

Integrated AI assistant that can help with editing tasks:

• Navigate between tabs
• Select and filter clips by metadata
• Run analysis operations
• Build sequences programmatically
• Answer questions about your project

Configure your preferred LLM provider (OpenAI, Anthropic, Gemini, Ollama) in Settings.

Sequencing Modes

Multiple ways to arrange clips:

• Manual: Drag-drop clips to timeline
• Arrange: Sort by color, duration, brightness, volume, shot scale, gaze, or original order
• Connect: Chain clips by visual similarity, match cut boundaries, follow a reference guide, or build LLM-guided associations
• Audio: Cut to music with Staccato or pull spoken phrases with Cassette Tape
• Text and AI: Generate poetry, story structures, transcript phrase sequences, or drawing-guided edits
• Find: Filter a specific person with Rose Hobart or build gaze-based edits with Eyes Without a Face

Sorting options:

• Shot Type: Filter by Wide Shot, Full Shot, Medium Shot, Close-up, or Extreme Close-up
• Duration: Sort by clip length (shortest first or longest first)
• Color: Arrange by dominant color (rainbow, warm to cool, cool to warm)

Intention-First Workflow

Start from a creative intention and let the app handle the rest:

1. Click a sequence card or create a new intention
2. Paste YouTube URLs or select local files in the import dialog
3. The app automatically downloads, detects scenes, generates thumbnails, analyzes clips, and builds your sequence
4. All in one flow—no tab-switching required

Clip Browser

• Thumbnail grid of detected scenes
• Collapsible source headers: Clips grouped by source with expand/collapse, showing clip count and selection state—reduces clutter when working with many sources
• Duration labels on each clip
• Click to preview, double-click for full playback
• Filter by metadata (shot type, has transcript, etc.)

Export

• Individual clip export (frame-accurate)
• Batch export (all or selected)
• Sequence export (timeline → single video)
• EDL export for external NLE software
• Dataset export for ML training

Source Import

• Local video files (drag-drop or browser)
• YouTube via yt-dlp
• Internet Archive collections
• Automatic scene detection after download

Projects

• Save/load project state (sources, clips, sequences, settings)
• JSON-based project format
• Auto-recovery support

Project Structure

algorithmic-filmmaking/
├── main.py              # Entry point
├── requirements.txt     # Dependencies
├── core/
│   ├── scene_detect.py  # PySceneDetect wrapper
│   ├── ffmpeg.py        # FFmpeg operations
│   ├── thumbnail.py     # Thumbnail generation
│   ├── downloader.py    # yt-dlp wrapper
│   ├── sequence_export.py
│   └── remix/
│       └── shuffle.py   # Constrained shuffle
├── models/
│   ├── clip.py          # Source, Clip dataclasses
│   └── sequence.py      # Sequence, Track, SequenceClip
└── ui/
    ├── main_window.py   # Main application window
    ├── clip_browser.py  # Thumbnail grid
    ├── video_player.py  # Preview player
    └── timeline/        # Timeline components

Configuration

API Keys (Optional)

Cloud AI features require API keys. Configure in Settings → API Keys:

Provider	Features	Get Key
OpenAI	GPT-4o descriptions, chat	platform.openai.com
Anthropic	Claude descriptions, chat	console.anthropic.com
Google	Gemini descriptions (supports video input), chat	aistudio.google.com
Replicate	VideoMAE shot classification (cloud tier)	replicate.com
YouTube	Search and metadata	console.cloud.google.com

Keys are stored securely in your system keyring. Environment variables (e.g., ANTHROPIC_API_KEY) take priority.

Local-only operation: Scene detection, transcription (Whisper), and local vision (Moondream) work without any API keys.

Technology Stack

Component	Technology
UI Framework	PySide6 (Qt 6)
Scene Detection	PySceneDetect
Video Processing	FFmpeg
Video Download	yt-dlp
Computer Vision	OpenCV
Transcription	faster-whisper
LLM Integration	LiteLLM (multi-provider)
Local Vision	Moondream 2B

Requirements

Core:

PySide6>=6.6
scenedetect[opencv]>=0.6.4
opencv-python>=4.8
numpy>=1.24
yt-dlp>=2024.1

AI Features (optional but recommended):

litellm              # Multi-provider LLM support
faster-whisper       # Local transcription
transformers         # Local vision (Moondream)

See requirements.txt for the full list.

System Dependencies

macOS:

brew install ffmpeg

Linux (Ubuntu/Debian):

# FFmpeg and Qt multimedia backend (GStreamer)
sudo apt install ffmpeg \
    gstreamer1.0-plugins-good \
    gstreamer1.0-plugins-bad \
    gstreamer1.0-plugins-ugly \
    gstreamer1.0-libav

Linux (Fedora):

sudo dnf install ffmpeg \
    gstreamer1-plugins-good \
    gstreamer1-plugins-bad-free \
    gstreamer1-plugins-ugly-free \
    gstreamer1-libav

Linux (Arch):

sudo pacman -S ffmpeg \
    gst-plugins-good \
    gst-plugins-bad \
    gst-plugins-ugly \
    gst-libav

Windows: Download FFmpeg from ffmpeg.org and add to PATH

CLI

Command-line interface for batch processing:

# Scene detection
python -m cli.main detect video.mp4 --threshold 3.0

# Transcription
python -m cli.main transcribe video.mp4

# YouTube search
python -m cli.main youtube search "query"

# Export project
python -m cli.main export project.json --format edl

# Full help
python -m cli.main --help

Roadmap

Complete:

• Scene detection with AdaptiveDetector
• Text-based scene detection (karaoke/subtitle mode)
• Thumbnail browser with filtering
• Collapsible source headers in clip browser
• Video preview player
• Individual and batch clip export
• URL import (YouTube, Internet Archive)
• Multi-track timeline
• Sequence export (MP4, EDL)
• Shuffle remix algorithm
• Linux support (AppImage packaging)
• AI-powered descriptions (GPT-4o, Claude, Gemini, Moondream)
• Shot type classification and filtering
• Cinematography analysis (VLM-powered film language breakdown)
• Transcription (faster-whisper)
• Color analysis with direction control
• Integrated chat agent
• 21 sequencer algorithms
• CLI for batch processing
• Project save/load
• Intention-first workflow
• Film language analysis / scene reports

In Progress:

• Timeline playback preview
• Object detection (YOLO)

Future:

• FAISS vector similarity search
• Model/view timeline virtualization
• Motion-based ordering