core: fix ambiguous column error when searching or sorting#983
core: fix ambiguous column error when searching or sorting#983mmzeynalli merged 3 commits intosmithyhq:mainfrom
Conversation
|
@nurikk could you give me minimal code example in which the current code (without your patch) fails? I understand your description, but would like to test it myself as well. |
here you go: so basically issue here:
results in but if you allow search only by one field, it works fine
|
mmzeynalli
left a comment
There was a problem hiding this comment.
I just checked on my local. Your patch (and current main) has a gap. If one model has 2 different FKs to another model, you get error:
# tests/test_models.py
class Order(Base):
__tablename__ = "orders"
id = Column(Integer, primary_key=True)
billing_address_id = Column(Integer, ForeignKey("addresses.id"))
shipping_address_id = Column(Integer, ForeignKey("addresses.id"))
billing_address = relationship("Address", foreign_keys=[billing_address_id])
shipping_address = relationship("Address", foreign_keys=[shipping_address_id])
def test_search_two_fks_to_same_model() -> None:
"""Searching on two different relationships to the same model should produce two JOINs."""
class OrderAdmin(ModelView, model=Order):
column_searchable_list = ["billing_address.name", "shipping_address.name"]
stmt = OrderAdmin().search_query(select(Order), "example")
stmt_str = str(stmt)
# Must have two separate joins to addresses (via different FKs)
assert stmt_str.count("JOIN") == 2Gives FAILED tests/test_models.py::test_search_two_fks_to_same_model - sqlalchemy.exc.AmbiguousForeignKeysError: Can't determine join between 'orders' and 'addresses'; tables have more th...
What works though, is:
def _join_relationship_paths(
self,
stmt: Select,
field_path: str,
joined_paths: Set[str],
) -> Tuple[Select, Any]:
"""Join relationship paths and return the statement and target model.
This helper method navigates through relationship paths (e.g., 'user.profile')
and joins each relationship only once, tracking which paths have been joined
to avoid duplicate JOINs. Joins via the relationship attribute so SQLAlchemy
uses the correct foreign key constraint.
Args:
stmt: The SQLAlchemy select statement to modify
field_path: The field path (e.g., 'user.profile.role')
joined_paths: Set tracking which relationship paths have been joined
Returns:
Tuple of (modified statement, target model class)
"""
model = self.model
parts = field_path.split(".")
current_path = ""
for part in parts[:-1]:
current_path = f"{current_path}.{part}" if current_path else part
relationship_attr = getattr(model, part)
next_model = relationship_attr.mapper.class_
if current_path not in joined_paths:
stmt = stmt.join(relationship_attr)
joined_paths.add(current_path)
model = next_model
return stmt, model| def test_search_multi_fields_no_duplicate_joins(client: TestClient) -> None: | ||
| class AddressAdmin(ModelView, model=Address): | ||
| column_searchable_list = [User.id, User.name] | ||
|
|
||
| admin.add_view(AddressAdmin) | ||
|
|
||
| with session_maker() as session: | ||
| user = User(name="Alice") | ||
| address = Address(user=user) | ||
| session.add_all([user, address]) | ||
| session.commit() | ||
|
|
||
| response = client.get("/admin/address/list?search=Alice") | ||
| assert response.status_code == 200 | ||
|
|
||
|
|
||
| def test_sort_multi_fields_no_duplicate_joins(client: TestClient) -> None: | ||
| class AddressAdmin(ModelView, model=Address): | ||
| column_sortable_list = [Address.id, User.id, User.name] | ||
|
|
||
| admin.add_view(AddressAdmin) | ||
|
|
||
| with session_maker() as session: | ||
| user1 = User(name="Bob") | ||
| user2 = User(name="Alice") | ||
| address1 = Address(user=user1) | ||
| address2 = Address(user=user2) | ||
| session.add_all([user1, user2, address1, address2]) | ||
| session.commit() | ||
|
|
||
| response = client.get("/admin/address/list?sortBy=user.name&sort=asc") | ||
| assert response.status_code == 200 | ||
|
|
||
|
|
||
| def test_sort_and_search_together_no_duplicate_joins(client: TestClient) -> None: | ||
| class AddressAdmin(ModelView, model=Address): | ||
| column_searchable_list = [User.name, User.id] | ||
| column_sortable_list = [Address.id, User.id, User.name] | ||
|
|
||
| admin.add_view(AddressAdmin) | ||
|
|
||
| with session_maker() as session: | ||
| user1 = User(name="Alice") | ||
| user2 = User(name="Bob") | ||
| user3 = User(name="Charlie") | ||
| address1 = Address(user=user1) | ||
| address2 = Address(user=user2) | ||
| address3 = Address(user=user3) | ||
| session.add_all([user1, user2, user3, address1, address2, address3]) | ||
| session.commit() | ||
|
|
||
| response = client.get("/admin/address/list?sortBy=user.name&sort=asc&search=o") | ||
| assert response.status_code == 200 |
There was a problem hiding this comment.
test names are a little misleading. This names should be in test_models file, as you check for non-multiple joins. Here I would name test_sort_and_search_together_no_ambigious_column_error
There was a problem hiding this comment.
Moved the unit-level no_duplicate_joins tests to test_models.py and kept a single HTTP integration test here, renamed to test_sort_and_search_together_no_ambigious_column_error.
|
Also, there is such warning in tests: |
|
@nurikk any updates? We would like to release new version soon. |
…e fields from same relationship Fixes SQLite 'ambiguous column name' error that occurred when users configured column_searchable_list or column_sortable_list with multiple fields from the same related model (e.g., ['user.id', 'user.name']). The issue happened because: - Each field was independently joining the same table multiple times - sort_query runs before search_query, so duplicate joins could occur across methods - SQLite cannot resolve ambiguous column references with duplicate table joins Solution: - Added _get_joined_entities() to inspect existing joins in the SQL statement - Added _join_relationship_paths() as DRY helper to track and prevent duplicate joins - Refactored both search_query and sort_query to use the helper - Joins are now tracked both by relationship path (within method) and by inspecting the statement (cross-method), ensuring no duplicates Users can now safely use multiple fields from the same relationship in searchable and sortable columns without encountering database errors.
Join via the relationship attribute instead of the target class so SQLAlchemy picks the right foreign key when a model has multiple relationships to the same target (e.g. billing/shipping addresses). This lets us drop the statement introspection helper since SQLAlchemy already dedupes joins on the same relationship attribute across calls. Also guards search_query against or_() being called with no arguments.
|
Addressed in latest commit:
|
Replace early-return guard with or_(false(), *expressions) as requested in review — the canonical SQLAlchemy pattern for a potentially empty or_() clause.
Fixes SQLite "ambiguous column name" error raised when
column_searchable_listorcolumn_sortable_listreferences multiple fields from the same relationship (e.g.["user.name", "user.email"]).Cause
_join_relationship_pathsjoined via the target model class, which caused SQLAlchemy to emit duplicate JOINs across fields — and also failed withAmbiguousForeignKeysErrorwhen a model had multiple foreign keys to the same target (e.g.billing_address+shipping_addressboth pointing atAddress).Fix
stmt.join(Model.relation)) so SQLAlchemy picks the right FK automatically.search_queryagainstor_()being called with no arguments.Tests
test_search_multi_fields_no_duplicate_joins— search alone with multiple fields from same relationshiptest_sort_multi_fields_no_duplicate_joins— sort alone with multiple fields from same relationshiptest_sort_then_search_no_duplicate_joins— cross-method dedup (sort then search)test_search_two_fks_to_same_model— model with two FKs to the same targettest_sort_and_search_together_no_ambigious_column_error— HTTP-level regression guard