Chord-Conditioned Diffusion for Interactive Instrumental Generation

Deep Learning project (HS25, ETH Zurich) implementing discrete diffusion models for generating musical improvisations from chord progressions using OctupleMIDI tokenization and Transformer-based denoising.

Goal: Generate harmonically consistent musical improvisations from chord inputs using a discrete diffusion model with cross-attention conditioning.

---

Quick Start

1. Download Data

bash sbatches_cluster/run_download_lakh_midi.sh

Downloads Lakh MIDI dataset (~1.7 GB compressed). See scripts/0_data_download/ for details.

2. Prepare Data

Run the 4-phase data preparation pipeline:

bash sbatches_cluster/1_data_prep_phase_ab.sh       # Phase A-B: Filter candidates
bash sbatches_cluster/2_data_prep_phase_c.sh       # Phase C: Deduplication & split
bash sbatches_cluster/3_data_prep_phase_df.sh       # Phase D-F: Feature extraction
bash sbatches_cluster/4_data_prep_phase_gk.sh       # Phase G-K: Tokenization

Output: ~111K training examples in data/lakh_ab_full/dataset_v3/

See scripts/1_data_preparation/README.md for detailed pipeline documentation.

3. Train Model

# Recommended: Masked diffusion with cross-attention
sbatch sbatches_cluster/6_train_masked_crossattention.sh

# Alternative: Auxiliary losses version
sbatch sbatches_cluster/8_train_auxiliary_losses.sh

Output: Trained models saved to models/lakh_model*/

See scripts/2_training/README.md for model variants and configuration.

4. Generate MIDI

python scripts/3_midi_generation/generate_v2.py \
  --checkpoint models/lakh_model2/octuple_diffusion_best.pt \
  --chords "C:maj G:maj A:min F:maj"

---

Repository Structure

scripts/
├── 0_data_download/          # Download scripts
├── 1_data_preparation/       # 4-phase data prep pipeline
├── 2_training/               # Training scripts (baseline, masked, improved)
├── 3_midi_generation/        # Generation scripts
├── 4_analysis/               # Visualization and basic analysis tools
└── 5_metrics_analysis/       # Benchmark Suite (FAD, CLAP, Symbolic metrics)

sbatches_cluster/             # SLURM cluster scripts
legacy_experiments/           # Unused/experimental code

---

Model Architecture

OctupleDiffusionModel: Transformer-based discrete diffusion model

• Input: OctupleMIDI tokens (8 attributes: pitch, position, bar, velocity, duration, program, tempo, time_sig)
• Conditioning: Chord progressions via cross-attention
• Diffusion: Masked (absorbing state) corruption over 50-128 timesteps
• Output: 8 separate prediction heads (one per attribute)

Model Variants:

• train_baseline.py: Discrete diffusion (multinomial noise)
• train_masked_crossattention.py: Masked diffusion + cross-attention (recommended)
• train_masked_concatenation.py: Masked diffusion + input concatenation
• train_auxiliary_losses.py: Cross-attention + enhanced losses (chord conditioning, smoothness, focal loss)

---

Baselines

To evaluate the proposed method, we compare our model against two state-of-the-art symbolic music generation systems:

1. FIGARO

Reference: Ruan et al., FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control (ICLR 2024).

Description: FIGARO is a description-to-sequence modeling framework that leverages a learned latent representation to achieve fine-grained control over symbolic music generation. It serves as a strong baseline for high-quality, controllable generation tasks. We utilize the official pre-trained weights and inference configuration provided by the authors for comparison.

2. MuseMorphose

Reference: Wu et al., MuseMorphose: Full-Song and Fine-Grained Music Style Transfer with One Transformer VAE (ICASSP 2022).

Description: MuseMorphose employs a Transformer-based VAE architecture designed to handle long-term structure and fine-grained style transfer. It represents a significant benchmark for models attempting to capture complex structural dependencies in full-song generation.

Comparison Framework

All models are evaluated on the held-out Test Set of the LMD-matched (Lakh MIDI) dataset. To ensure a fair comparison:

• We generate an equal number of samples ($N=1000$) for each model.
• All symbolic outputs are rendered to audio using the same synthesis pipeline (FluidSynth with FluidR3_GM.sf2) before computing audio-based metrics (FAD, CLAP).
• Symbolic metrics are computed on the raw MIDI outputs aligned with the ground truth chord progressions.

---

Data Format

Octuple Token: [bar, pos, pitch, dur, vel, chord, tempo, mode]

• 8-bar sliding windows with 50% overlap
• Chord-conditioned (61 chord vocabulary)
• Transposed to C major / A minor for key normalization

See scripts/1_data_preparation/README.md for complete data format specification.

---

Cluster Usage

All scripts are designed for SLURM cluster execution. See sbatches_cluster/QUICK_START_CLUSTER.md for cluster-specific instructions.

Monitor training:

tail -f output/train_model_2_*.out  # or train_model_improved_*.out for auxiliary losses

---

Documentation

• Data Preparation: scripts/1_data_preparation/README.md
• Training: scripts/2_training/README.md
• Cluster Guide: sbatches_cluster/QUICK_START_CLUSTER.md

---

Requirements

pip install torch numpy pandas pretty_midi mido symusic tqdm

See requirements.txt for complete list.

---