An Open, Modular Exploration Towards Systematic World–Action Model Pretraining

What is OpenWAM

OpenWAM is an open research stack for systematically developing World-Action Models (WAMs). It turns tightly coupled design choices into modular components and controlled experiments. It consists of:

• OpenWAM-Infra: A modular infrastructure for composing and comparing model, representation, training, inference, deployment, and evaluation choices.
• OpenWAM-Study: Controlled studies that derive practical principles for inheriting world knowledge, coupling world and action learning, and scaling across domains.
• OpenWAM-α: An open pretrained WAM that applies these principles at scale, trained on 518.5M frames (about 6,400 hours) of egocentric human and robot data.

News

• [2026/09/06] 🤖 OpenWAM is integrated into XPolicyLab.
• [2026/09/06] 🤗 We release all pretrained and finetuned models on huggingface.
• [2026/09/06] 🔥 OpenWAM Codebase Release！

Installation

Create an environment with conda:

# Requires Python >= 3.10
conda create -n openwam python=3.10
conda activate openwam

or with venv:

# Requires Python >= 3.10 (check with `python3 --version`)
python3 -m venv .venv
source .venv/bin/activate

We recommend using PyTorch 2.7.1 with CUDA 12.8 (others may also work):

pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu128

Then install OpenWAM:

pip install -e .

deepspeed ships as a source distribution and is compiled during pip install, so a C compiler (gcc) must be on the PATH.

Cosmos-Predict2.5 Extras (Optional) — needed only for experiments with the cosmos_predict25_2b video backbone

With your environment activated:

git submodule update --init third_party/cosmos-predict2.5
bash scripts/install_cosmos_predict25.sh

The script installs the upstream cosmos packages into the active environment and compiles transformer-engine (CUDA toolkit with nvcc required), then automatically restores the package versions OpenWAM pins.

Assets Preparation

The downloaders are interactive by default; every menu step also has a flag (--name, --source, --root, --yes, see --help) so they can run unattended, and the default storage location is resolved relative to the repository regardless of the working directory. Component downloaders store assets under assets/ and update the matching YAML path; the released-checkpoint downloader keeps each checkpoint's self-contained config unchanged.

1. Video Backbone

python scripts/download_assets/download_video_backbone.py

Supported video backbones

Wan2.2-TI2V-5B ✅	Wan2.1-VACE-1.3B ✅	Wan2.1-I2V-14B-480P ✅
Cosmos-Predict2.5-2B ✅	Cosmos3-Edge ✅

Weights are saved under assets/video_backbone_ckpt/ and the selected configs/model/video_backbone/ file is updated with the downloaded path.

2. Benchmark Data

python scripts/download_assets/download_benchmark_data.py

Supported benchmarks

RoboTwin2.0 ✅	RoboDojo ✅	RoboDojo-Real ✅
LIBERO ✅	VLABench ✅	EBench ✅
RoboCasa365 ✅	RoboCasa_GR1 ✅

Data is saved under assets/benchmark_data/<benchmark>/. Normalization statistics are prepared when needed, and the selected dataloader configuration is updated.

3. VLM Backbone (Optional)

Required only by tri_system:

python scripts/download_assets/download_vlm_backbone.py

Supported VLM backbones

Qwen3-VL-2B-Instruct ✅

Weights are saved under assets/vlm_backbone_ckpt/, and the selected configuration is updated.

4. Visual Encoders (Optional)

Required only for video backbones that use an external encoder:

python scripts/download_assets/download_visual_encoder.py

Supported visual encoders

DINOv3 ViT-B/16 ✅	V-JEPA 2.1 ViT-G/16 ✅
Wan2.2 VAE ✅	FLUX.2 VAE ✅

Weights are saved under assets/visual_encoder_ckpt/, and the selected encoder configuration is updated.

5. Released OpenWAM Checkpoints

Use this downloader to obtain OpenWAM-Alpha releases or OpenWAM-Study checkpoints from the OpenWAM collection:

python scripts/download_assets/download_openwam_checkpoints.py

Checkpoints are saved under assets/openwam_ckpt/openwam_alpha/ or assets/openwam_ckpt/openwam_study/<type>/. Each checkpoint directory contains its own config and can be deployed directly with:

bash scripts/deploy.sh <ckpt_dir_path>

For fine-tuning, set training.finetune_ckpt_path in configs/train.yaml to the downloaded checkpoint directory. Benchmark data is still required.

Quick Start

Quick Start provides a minimal end-to-end example: prepare the assets, train or fine-tune a policy, deploy its checkpoint, and run a first inference check.

The example uses the DualSystem JointSelfAttention architecture, the Wan2.2-TI2V-5B video backbone, and the Mutual attention mask.

Component	Selection	Configuration
Architecture	dual_system / joint_self_attn	configs/model/dual_system.yaml
Video backbone	wan22_ti2v_5b	configs/model/video_backbone/wan22_ti2v_5b.yaml
Attention mask	mutual	model.architecture.attention_mask_mode
Dataset	libero	configs/dataloader/libero.yaml

Resource recommendation: We recommend 8 GPUs with 80 GB VRAM each for training. This configuration supports normal training for all architectures using Wan2.2-5B or smaller video backbones. More GPUs are better when available and can further improve training throughput.

From Scratch Training

1. Download LIBERO and let the downloader update its dataloader configuration:

   python scripts/download_assets/download_benchmark_data.py

   Select LIBERO in the interactive menu.

2. Download Wan2.2-TI2V-5B:

   python scripts/download_assets/download_video_backbone.py

   Select Wan2.2-TI2V-5B and the desired model source.

3. Start a debug run with the complete model selection:

   bash scripts/train.sh \
     dataloader=libero \
     model=dual_system \
     model/video_backbone=wan22_ti2v_5b \
     model.architecture.variant=joint_self_attn \
     model.architecture.attention_mask_mode=mutual \
     training.debug=true

   training.debug=true runs 20 steps, saves at steps 10 and 20, and uses a constant learning rate. Check the run output, then set training.debug=false for normal training. Training defaults and CLI overrides are defined in configs/train.yaml. Debug outputs use training.output_path, whose default is outputs/openwam_checkpoints.

4. Deploy the debug checkpoint and inspect one input-output cycle:

   bash scripts/deploy.sh <debug_ckpt_dir_path>

   Keep the server running, then open another terminal and run the two inference helpers. Deployment enables compile by default, so the first inference may take longer while compilation warms up; later requests are typically faster:

   python scripts/inference_test/inference_single_test.py \
     --server ws://127.0.0.1:8848 --test --state-dim 10
   
   python scripts/inference_test/inference_continuous_test.py \
     --server ws://127.0.0.1:8848 --test --state-dim 10

   The single-request helper checks ping, one prediction, and reset. The continuous helper sends repeated predictions over one connection and reports the returned action dimension and latency. Stop the deployment process after the checks.

OpenWAM-α Fine-Tuning

1. Download LIBERO as shown above.

2. Download the OpenWAM-Alpha foundation checkpoint:

   python scripts/download_assets/download_openwam_checkpoints.py

   Select OpenWAM_Alpha and OpenWAM-Alpha-Pretrain-Foundation-Model. Keep the resulting directory as <foundation_ckpt_dir_path>.

3. Start fine-tuning from that directory:

   bash scripts/train.sh \
     dataloader=libero \
     training.finetune_ckpt_path=<foundation_ckpt_dir_path>

   The default model configuration already matches the required setup above. You can set the same field in configs/train.yaml instead of passing it on the command line. num_frames=33 and video_stride=4 in configs/dataloader/libero.yaml produce the 32-step action horizon expected by the sampler; no separate action_chunk override is needed.

4. Deploy the resulting checkpoint directory:

   bash scripts/deploy.sh <ckpt_dir_path>

   Install and run the LIBERO client according to the LIBERO evaluation guide.

OpenWAM Usage Guidance

OpenWAM is configured through composable Hydra YAML files. Select an architecture, backbone, dataloader, and runtime behavior by changing configuration values or overriding them on the command line. The task guides below are the maintained entry points for using and extending the repository:

Guide	Use it when you want to…
Training and deployment	choose a model/dataloader, prepare assets, train, fine-tune, resume, or deploy a policy
Architecture extension	extend a video, visual, VLM, action backbone, or WAM architecture
Benchmark integration	extend a dataloader and connect a benchmark client to the WebSocket protocol
OpenWAM-α fine-tuning	fine-tune the released foundation checkpoint to execute downstream task

💡 Agent tip: Pick the guide that matches your task and feed it directly to your agent — less explaining, more building.

Installation and Assets Preparation above cover environment setup and model or dataset downloads. Benchmark-specific environment and evaluation details remain in the benchmarks directory.

Star History

License

OpenWAM is released under the Apache License 2.0.

Citation

If you use OpenWAM, please cite:

@article{wang2026openwam,
  title   = {OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining},
  author  = {Yuran Wang and Siqiao Huang and Mingleyang Li and Chenhao Zhang and Jiaqi Liang and Weiyang Jin and Yue Chen and Xuemin Chi and Donghao Zhou and Qize Yu and Yu-Kai Wang and Yuhan Rui and Shenzhe Yao and Zhen Yuan and Zhenhao Shen and Kefei Zhu and Zijie Zhu and Ning Gao and Xiaowei Chi and Guanqi He and Shanghang Zhang and Hao Dong and Lin Shao and Hang Zhao},
  year    = {2026},
  journal = {arXiv preprint arXiv: 2609.07398}
}