mm-tools is a pruned, local-first multimedia workstation monorepo. It contains the runtime source, per-project setup entrypoints, and browser/CLI frontends used by this tool suite. It does not contain checkpoints, virtual environments, caches, generated media, or training/evaluation material.
Root-authored mm-tools code is distributed under the root AGPLv3 LICENSE.
Imported upstream attributions and revisions are recorded in the commit ledger.
The all-in-one frontend for local multimedia inference. Cutting out the cloud.
Runtime inference is local. Network access is needed only while installing or downloading the model artifacts required.
This repo was created for protecting open-source runtimes under a documentable heirarchy with AGPL.
Install the host tools required by the projects you plan to run:
uvfor Python environments and packagesffmpegandffprobefor audio/video projectsbun- sandwich optional- Rust/Cargo for
img2svgand the optional CrisperWhisper launcher - a GPU or CPU, and likely a monitor
Then download the checkpoints once (the dependencies):
Due to the size of the model weights. Make sure you're sitting on a speedy drive.
cd ~ && git clone https://github.com/CommanderTurtle/mm-tools multimedia && cd multimedia
cd models
uv venv --python 3.12.10 --seed --managed-python
source .venv/bin/activate
uv pip install -U huggingface_hub hf_transfer
uv run download_models.py
# Choose workers first, then select project bundles or press Enter for all.
# Non-interactive full install: uv run download_models.py all --yes --workers 24
deactivate # when doneThe downloader is resumable. Run it again after an interrupted transfer. New
source-import bundles are named 4d, lingbot, lingbot-5090, fire3d,
aukspeech, sculpting, worldsculpt, nvidia-sim, v2v, and
yue2; use uv run download_models.py --list for the complete current menu.
The source imports, single-GPU adapters, setup/start contracts, and private
browser studios are included here; model artifacts remain external and
resumable. LingBot deliberately separates its shared encoder/VAE bundle from
the official 1.3B single-5090 DiT and does not retain the unusable 14B DiT.
Model downloads remain governed by each model's native license and any access terms accepted for their usage. ARDY uses an ungated, pre-merged INT8 LLM2Vec encoder that preserves the exact Llama 3 + MNTP + supervised-adapter pipeline; the original gated BF16 assembly is not downloaded. 4DAnyone's neutral SMPL-X body model remains a separate manual download. See the model inventory for every included artifact, output path, and explicit exclusion. The ability to self-host a tool rivaling Adobe in 2026 was the purpose for this repo.
Assuming one has a nice GPU, you can host a wide variety of tools. Emulating the top-tier closed source software out there, solely with open-source runtimes. It wouldn't be without the community that building something like this would be possible.
~16-20gb vram recommended for most tools. The largest quality-first music paths benefit from 24-32 GB. Each project has its own upper bound, and none is authoritative if you run them one at a time. (only ~8-9gb for LongCat)
So. What can you do?
2-longcat.mp4
compatible with vox
- an in-house built translation pipeline with native web-ui. Compatibile with vox system-audio driver. (This exact fork)
compatible with vox
whisper.mp4
2-video-to-gif-avif.mp4
because who says ffmpeg is fine in cli?
view what a combined workflow looks like:
2-ideogram+img2svg-compact.1.mp4
2-redesign.mp4
2-muscriptor.mp4
(uses minimized, stripped comfyUI pipeline directly, but custom WebUI)
minimax.mp4
Run all at once.
The install sequence is simple for each project. Requiring WSL if on Windows.
Jump into a project folder, run its setup once, and start it. Setup creates the ignored .env and isolated .venv automatically.
cd ~/multimedia/PROJECT
./setupwithuv.sh
./startwithuv.shVox - Native Audio/Translate Routing for this repo.
See other projects by me!
https://github.com/CommanderTurtle/persephone just my integration of own internal config + continuity with omp. gpu & local-only for just about everything so far
-
Diogenes - hosting (linux, service manager), maintained fork of Odysseus for backends and control center — this can install and integrate the following if not already installed:
-
sandwich - node package management/auditing for all these package.jsons
-
libriarian - wikifier (makes wikis, dreaming agents)
-
retrieval - skill labrador that fetches necessary skills from archive on disk, rather than bloating agent prompt with full list
-
persephone - adds omp (pi agent) gateway layer
-
leetcoder - allows hermes to puppeteer pi’s for small coding tasks (delegation)
Lots of ideas from stablyai/orca, which does all these things, and where i got a lot of inspiration for the ecosystem
Still WIP. Doing lots of editing as I continue development. Especially with Diogenes automatically setting up my config upon install
- orc - a multisite/multiframework F# engine for modular websites in py, ts, c#, js, and ruby.
- reactor - a lightweight rust-poller that allows for a live fsharp repo in 7+ languages. Side-hosts vite dev server, zensical serve, jekyll serve, netdocs serving (C#), and more with preview
- tools - just some lightweight tools that help making maintaining websites easy in modular languages
- regedited - for databases
- macrohelp - for macros
- firebending - an Anything-MCP built on macrohelp (TBD)
- app/adspace - ad simulator
- app/countku - count in haiku
- app/webclip - scrape the web to markdown
extra
For Python projects, choose the setup entrypoint in that project only. No root setup wrapper exists.
Local Ideogram 4 generation source plus the fuzzy-mask editing browser: ObjectClear, BiRefNet, OpenCV, and optional private-Comfy Ideogram masked editing.
cd ~/multimedia/ideogram
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8174The standalone Ideogram generator is local_generate.py; add src to PYTHONPATH or install the local package when using that lane. The editor loads Ideogram only when explicitly selected. Its private engine reuses bundled Comfy code (not the Minimax service), supports local caption drafting and configurable crop processing, and preserves the original image outside the feathered mask. See Ideogram architecture and setup.
Native VTracer-based raster-to-SVG CLI and browser studio.
cd ~/multimedia/img2svg
./setupwithrust
./startwithrust # http://127.0.0.1:417Local LongCat voice synthesis and cloning. The HTTP service and browser workbench own independent model lifecycles.
cd ~/multimedia/longcat
./setupwithuv.sh
./starthttp.sh # machine API: :8230
./startwithuv.sh # browser UI: :8231Native ACE-Step 1.5 music generation, editing, repainting, extension, and audio-to-audio controls.
cd ~/multimedia/acestep
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8250The browser process preloads the native XL-SFT DiT and 4B 5 Hz language model: the quality-first ACE combination for a 24 GB+ workstation. It reads the shared native ACE-Step model tree directly and releases all weights on Ctrl+C.
Standalone MiniMax Music 3 text-to-song production desk using the official graph: full FP16 DiT, pruned INT8 text encoder, DAV decoder, real CFG/top-k/sampler controls, and an animated lyrics-and-spectrum performance view. It does not require a separately running ComfyUI.
cd ~/multimedia/music/minimax
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8254The browser owns a private loopback song engine. Its separate Prompt Guide tab uses the bundled Comfy CLIPLoader(type=krea2) → TextGenerate path with the local SergiusFlavius/Qwen3-VL-4B-Instruct-heretic-NVFP4 checkpoint, but begins fully off and unloaded; enabling it starts a second loopback-only engine. A Guided Brief Lab adds one-click recipes plus expandable genre, instrumentation, key/mode, BPM, meter, groove, harmony, vocal, form, production, and listening-context vocabulary. It creates plain-English copypasta without invoking a model or silently changing controls. Guide results remain detached, with manual copy controls for MiniMax's Global Metadata, Vocal Details, Arrangement, and tuning notes. Load models and Unload control song-model residency explicitly, while Ctrl+C unloads all weights and closes the web app plus both private engines. Links in the studio hand covers/editing to ACE-Step, loops to Foundation-1, and audio reverse engineering to MuScriptor.
Foundation-1 text-to-audio with the enhanced Stable Audio Gradio controls, local T5 conditioning, MIDI analysis, and optional TorchAO controls.
cd ~/multimedia/stableaudio
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8251Two-stage symbolic composition: generate harmony MIDI, then orchestrate a MIDI sketch with baseline or GRPO checkpoints.
cd ~/multimedia/symphony
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8252The web process is model-free while idle. Each generation request owns and releases its model in a short-lived worker.
Prompt-conditioned local singing synthesis with the highest-quality Pro weights only, explicit load/unload controls, browser playback, and WAV downloads.
cd ~/multimedia/vocalrender
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8253Audio-to-MIDI/score transcription with a browser piano roll.
cd ~/multimedia/muscriptor
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8222Single-file CLI:
./startwithuv.sh INPUT.mp3Sheet-music image/PDF encoding and SMT-backed beKern/MIDI/SVG conversion.
cd ~/multimedia/musvit
./setupwithuv.sh
./startwithuv.sh # browser studio: :8223
./startwithuv.sh SCORE_IMAGE_OR_PDF # direct CLIUse musvit_embed.py directly for the embedding lane.
Flat-image decomposition into editable layers using local detection, segmentation, inpainting, OCR, and Qwen Image Layered components.
cd ~/multimedia/redesign
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8173Review .env before launch. Native Diffusers is the primary image lane; an existing private ComfyUI listener is optional.
Local EraX text translation, language classification, and a shared INT4 multimodal lane for image explanation, OCR translation, custom visual prompts, and spoken-language arbitration.
cd ~/multimedia/translate
./setupwithuv.sh
./starthttp.sh # http://127.0.0.1:8176
./startwithuv.sh # standalone browser UI: :8177The setup prompt accepts GPU or CPU; TRANSLATE_ACCELERATOR remains available for noninteractive runs. The browser UI owns an independent model lifecycle. Its explicit external-load button can attach to an already-loaded starthttp.sh instance, but it never falls back between them automatically.
Private VideoSmaller-style FFmpeg compression.
cd ~/multimedia/video-compact
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8240Local trim/crop/resize and animated GIF or AVIF conversion.
cd ~/multimedia/video-to-gif-avif
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8241The host FFmpeg build must expose the requested GIF/AVIF encoder and AVIF muxer.
Local CrisperWhisper 2.0 transcription in normalized or literal mode. The HTTP service and browser workbench use separate ports and model lifecycles.
cd ~/multimedia/whisper
./setupwithuv.sh
./starthttp.sh # machine API: :8172
./startwithuv.sh # browser UI: :8173ReDesign also defaults to 8173; change CW2_UI_PORT when both browser services run together.
| Port | Runtime |
|---|---|
417 |
img2svg |
8172 |
CrisperWhisper HTTP |
8173 |
CrisperWhisper UI or ReDesign |
8174 |
Ideogram/ObjectClear editor |
8176 |
Translate |
8177 |
Translate standalone UI |
8222 |
MuScriptor |
8223 |
MuSViT local score studio |
8230 |
LongCat HTTP |
8231 |
LongCat UI |
8240 |
Video Compact |
8241 |
Video to GIF/AVIF |
8250 |
ACE-Step 1.5 |
8251 |
Stable Audio Foundation-1 |
8252 |
SymphonyGen local studio |
8253 |
VocalRender local studio |
8254 |
MiniMax Music 3 local studio |
Services bind to the configured private-LAN interface and do not add public-facing authentication by default. Keep them behind the host firewall or set the supported bearer token where provided.
The current expansion uses one shared, dependency-light product surface with project-native adapters. Every studio has responsive dark/light themes, searchable task cards, complete native controls, presets and drafts, raw streaming uploads, a persistent single-GPU queue, cancellation, SSE progress, model load/unload health, an output library, side-by-side comparison, and API examples. There are no hosted assets, analytics, CORS opt-ins, or automatic model downloads at runtime.
| Project | Canonical surface | Default port |
|---|---|---|
| AuK | 16 speech generation, cloning, editing, emotion, timing, and nonverbal workflows; Vox-compatible speech route | 8260 |
| 4DAnyone | monocular 4D human reconstruction and reusable result inspection | 8261 |
| TRELLIS.2 + Pixal3D | image/multiview sculpting, local refinement, and interactive GLB inspection | 8262 |
| WorldSculpt | complete multi-object scene reconstruction and scene inspection | 8263 |
| ID-V2V | identity-preserving transfer, normal/depth guidance, and reusable preprocessing | 8265 |
| ARDY + SOMA-X | text-to-motion, motion inspection, parametric bodies, and hands | 8266 |
| LingBot World V2 | 1.3B single-5090 world generation and visual camera-path authoring | 8267 |
| Fire3D | RGB/RGBD scene reconstruction, native-release evaluation, and GLB inspection | 8268 |
| YuE2 | composition, planning, continuation/cover, score realization/editing, transcription, resume, and VAE decode | 8270 |
sculpting-1-compact.mp4
The only Yue2 frontend with stem splitting!
- (Share with Friends via app.shel.sh/make & a.shel.sh superlinks, entirely local)
Another:
https://a.shel.sh/#media:rZ9jBas~g*mK]Q-()SB$IFDo]Xl-b5y[P,/@f/FwenhdLmh~Aaw