Skip to content

Bug: evo-objects rejects PyArrow large_string values #230

Description

@davidknight-seequent

Bug Description

evo-objects rejects PyArrow large_string values produced by pandas 3 during dataframe upload.

When uploading dataframes through evo-objects, string columns can be inferred by PyArrow as large_string instead of string. evo-objects currently treats large_string as unsupported and raises a TypeError during table format detection and category upload.

Observed Error

Unsupported column type large_string from DataType(large_string)

Impact

Any workflow that calls save_dataframe or upload_dataframe on a dataframe containing text columns may fail if PyArrow infers those columns as large_string.

This affects:

  • String attribute uploads
  • Lookup table uploads for category attributes
  • Any table format inference path that relies on ArrowTableFormat.from_schema

Acceptance Criteria

  • Related functionality is unaffected
  • A test case is added to prevent regression

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workingevo-objectsAffects the evo-objects package

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions