Skip to content

LogProbs.top_logprobs typed as Dict but API returns List[Dict] #443

@bilelomrani1

Description

@bilelomrani1

Describe the bug

LogProbs.top_logprobs is typed as Optional[Dict[str, float]] but the API returns a list[dict[str, float]] (one dict per token). This causes PydanticSerializationUnexpectedValue when calling model_dump() on the response.

The field is defined at together/types/chat/chat_completion.py:47:

top_logprobs: Optional[Dict[str, float]] = None

The correct type should be Optional[List[Dict[str, float]]].

To Reproduce

import warnings
import together

warnings.filterwarnings("error")

client = together.Together()
response = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Say hello."}],
    logprobs=3,
    max_tokens=5,
)

# The value is a list, not a dict:
lp = response.choices[0].logprobs
print(type(lp.top_logprobs))
# <class 'list'>
print(lp.top_logprobs)
# [{'Hello': -2.6e-06, 'hello': -13.5, ' Hello': -13.875}, {'.': -4.8e-05, ...}, ...]

# model_dump() triggers a Pydantic serialization warning:
response.model_dump()
# UserWarning: PydanticSerializationUnexpectedValue(
#   Expected `dict[str, float]` - serialized value may not be as expected
#   [field_name='top_logprobs', input_type=list])

Expected behavior

model_dump() should succeed without warnings. The type annotation should match what the API returns: Optional[List[Dict[str, float]]].

Environment:

  • OS: macOS
  • Python: 3.12
  • together SDK version: 2.7.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions