Skip to content

[Example] Add Qwen2.5-Omni offline inference example - #54

Open
Sakura-Tears07 wants to merge 1 commit into
SandAI-org:mainfrom
Sakura-Tears07:dev
Open

[Example] Add Qwen2.5-Omni offline inference example#54
Sakura-Tears07 wants to merge 1 commit into
SandAI-org:mainfrom
Sakura-Tears07:dev

Conversation

@Sakura-Tears07

Copy link
Copy Markdown

Add a vLLM-Omni based example under example/inference/qwen2.5-omni, aligned with the wan2.2-vae layout for MagiCompiler profiling.

MagiCompiler is toggled through VLLM_OMNI_MAGI_COMPILER inside vLLM-Omni.

PR Category

  • New Feature
  • Optimization
  • Breaking Change
  • Bug Fix
  • Development / Refactoring
  • Documentation
  • Chore
  • Testing

Description

This PR adds a Qwen2.5-Omni offline inference and profiling example based on vLLM-Omni.

Main changes:

  • Add the example under example/inference/qwen2.5-omni.
  • Add a Qwen2.5-Omni inference wrapper.
  • Add warm-up execution before profiling.
  • Add repeated inference runs with NVTX ranges.
  • Add optional Nsight Systems profiling.
  • Support text and video inputs.
  • Support local model checkpoints and custom deployment configurations.
  • Update .gitignore for generated audio files and profiling reports.

MagiCompiler is enabled using:

VLLM_OMNI_MAGI_COMPILER=1

Set it to 0 to disable MagiCompiler for comparison.

Add a vLLM-Omni based example under example/inference/qwen2.5-omni, aligned with the wan2.2-vae layout for MagiCompiler profiling. Magi is toggled via VLLM_OMNI_MAGI_COMPILER inside vLLM-Omni.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants