Skip to content

Fix vLLM compatibility in GreedyProbsCalculator and WhiteboxModelvLLM#454

Open
dmndxld wants to merge 1 commit into
IINemo:mainfrom
dmndxld:fix/vllm-compatibility
Open

Fix vLLM compatibility in GreedyProbsCalculator and WhiteboxModelvLLM#454
dmndxld wants to merge 1 commit into
IINemo:mainfrom
dmndxld:fix/vllm-compatibility

Conversation

@dmndxld

@dmndxld dmndxld commented Apr 21, 2026

Copy link
Copy Markdown

Summary

Fix two bugs preventing vLLM models from working with estimate_uncertainty() and UEManager.

Changes

1. GreedyProbsCalculator vLLM compatibility (stat_calculators/greedy_probs.py)

  • Skip output_attentions and output_hidden_states when model is vLLM (not supported by vLLM engine)
  • Skip embeddings extraction and attention processing for vLLM models
  • Without this fix, using vLLM with any estimator triggers NotImplementedError in get_embeddings_from_output()

2. WhiteboxModelvLLM missing model_path attribute (model_adapters/whitebox_model_vllm.py)

  • Call super().__init__(model_path, model_type) to properly initialize base class
  • Add model_path parameter (defaults to vLLM engine's model_name)
  • Without this fix, estimate_uncertainty() raises AttributeError: 'WhiteboxModelvLLM' object has no attribute 'model_path'

Test plan

  • Verified with estimate_uncertainty() using WhiteboxModelvLLM + SemanticEntropy
  • flake8 and black checks pass locally

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant