This project was created by Yuezhe Yang for the paper "UltraGS: Real-Time Physically-Decoupled Gaussian Splatting for Ultrasound Novel View Synthesis" (Paper Link).
Ultrasound imaging is a cornerstone of non-invasive clinical diagnostics, yet its limited field of view poses challenges for novel view synthesis. We present UltraGS, a real-time framework that adapts Gaussian Splatting to sensorless ultrasound imaging by integrating explicit radiance fields with lightweight, physics-inspired acoustic modeling. UltraGS employs depth-aware Gaussian primitives with learnable fields of view to improve geometric consistency under unconstrained probe motion, and introduces PD Rendering, a differentiable acoustic operator that combines low-order spherical harmonics with first-order wave effects for efficient intensity synthesis. We further present a clinical ultrasound dataset acquired under real-world scanning protocols. Extensive evaluations across three datasets demonstrate that UltraGS establishes a new performance-efficiency frontier, achieving state-of-the-art results in PSNR (up to 29.55) and SSIM (up to 0.89) while achieving real-time synthesis at 64.69 fps on a single GPU.
To validate the effectiveness of our model, we conducted extensive experiments on three ultrasound datasets from different sources.
The details of the datasets used and their specific links are provided here for easy access to the data.
Ultrasound in the Wild Dataset (Wild Dataset): This dataset contains ten knee ultrasound samples acquired using a handheld Butterfly iQ+ probe at 30 FPS. The longitudinal suprapatellar sweeps include realistic motion artifacts and trajectory variations, making the dataset suitable for evaluating robustness under uncontrolled scanning conditions.
Wild Dataset: We used this dataset to evaluate the performance of UltraGS under handheld and unconstrained scanning scenarios.
Phantom Dataset: This dataset contains nine lumbar spine phantom scans acquired using a KUKA robotic manipulator. The tilted and perpendicular sweeps introduce spinous process occlusions, which are useful for testing geometric reconstruction under controlled but challenging conditions.
Phantom Dataset: We used this dataset to validate reconstruction quality under controlled ultrasound acquisition.
Clinical Ultrasound Examination Dataset (Clinical Dataset): This is our in-house clinical ultrasound dataset collected using the Canon i900 ultrasound system with institutional research ethics approval (Ethics Approval No.: SL2024-KY-29-01). It contains six challenging cases with unconstrained probe motion, anatomical occlusions, and realistic clinical scanning variability, including wrist joint and kidney scans.
Clinical Dataset: We have open-sourced this dataset on Google Drive, and used it to validate the performance of UltraGS under real-world clinical ultrasound examination scenarios.
Our experimental platform is configured with an RTX 3090 GPU, and the released code has been tested in a PyTorch 2.0 environment with CUDA 11.7.
Run the installation commands:
conda env create --file environment.yml
conda activate ultrags
pip install -r requirements.txtData preprocessing is divided into two stages:
- Prepare ultrasound images and estimate the relative camera poses for each sequence. Although each ultrasound image is a 2D cross-section of the scanned anatomy, we follow the preprocessing strategy used in the Wild Dataset and treat the image sequence as an open scene for SfM reconstruction. The camera parameters can be estimated with COLMAP, for example using the
colmap2nerf.pyscript from Instant-NGP to export the poses into a NeRF-styletransforms_train.jsonfile. The COLMAP poses are used only for initialization in UltraGS. - Organize each processed sequence into the UltraGS input format.
Each processed case should contain the following files:
case_name/
├── images/
├── transforms_train.json
└── points3d.plyThe points3d.ply file is optional. If it is not provided, UltraGS will initialize a random point cloud for training.
Once you have completed the data preparation, you can use train.py to train your own model.
python train.py -s /path/to/processed/ultrasound/case -m /path/to/outputAfter completing the training, you can use render.py and metrics.py to evaluate the model.
python render.py -s /path/to/processed/ultrasound/case -m /path/to/output --skip_train
python metrics.py -m /path/to/outputTo visualize the learned reconstruction along a camera trajectory, run:
python render.py -s /path/to/processed/ultrasound/case -m /path/to/output --render_path@article{yang2025ultrags,
title={UltraGS: Gaussian Splatting for Ultrasound Novel View Synthesis},
author={Yang, Yuezhe and Cai, Wenjie and Yang, Dexin and Dong, Yufang and Dong, Xingbo and Jin, Zhe},
journal={arXiv preprint arXiv:2511.07743},
year={2025}
}