[codex] add ERNIE image diffusers runner#1115
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces support for the ERNIE-Image text-to-image model by adding dedicated configuration files, a text encoder, a transformer model wrapper, a scheduler, a VAE decoder, and a runner pipeline, along with comprehensive unit tests. The review feedback highlights three key areas for improvement: resolving a potential runtime dtype mismatch in the VAE decoder by explicitly casting batch norm statistics to the latent's dtype, replacing an assertion with a proper ValueError for runtime validation in the runner, and correcting an inconsistent variable reference from model_cls to self.model_cls in the pipeline initialization.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
@wangshankun could you review this PR? Let me know if there is a contribution guide to follow. |
Summary
Adds ERNIE-Image text-to-image support through a Diffusers-backed LightX2V runner.
This PR introduces:
ernie_imageconfigs and model pipeline registrationErnieImageRunnerwith a decomposed generation path instead of defaulting toErnieImagePipeline.__call__unload_modulescleanupScope
This is still intentionally Diffusers-backed. It uses
diffusers.ErnieImagePipeline.from_pretrainedas the underlying loader and then routes runtime access through LightX2V wrapper boundaries. It does not yet implement native ERNIE weight mapping, native transformer infer classes, quantization, LoRA, or distributed execution.Validation
Local checks:
python -m unittest test_cases.test_ernie_image_runner-> 17 tests OKruff check configs/ernie_image lightx2v/infer.py lightx2v/pipeline.py lightx2v/utils/set_config.py lightx2v/models/runners/ernie_image lightx2v/models/input_encoders/hf/ernie_image lightx2v/models/networks/ernie_image lightx2v/models/schedulers/ernie_image lightx2v/models/video_encoders/hf/ernie_image test_cases/test_ernie_image_runner.py-> all checks passedpython -m py_compile ...for the ERNIE runner/wrapper/test files -> passedH100 smoke regression after the component-container migration:
0.0, max abs diff00.0, max abs diff0unload_modules=true+ tensor return: generation completed, progress reached(100.0, 100), andpipe/components/model/text_encoder/scheduler/vaewere all unloaded