Skip to content

Look up namespaced SQL tables by their schema-qualified key - #1794

Open
DMZ22 wants to merge 1 commit into
frictionlessdata:mainfrom
DMZ22:sql-namespace-table-lookup
Open

Look up namespaced SQL tables by their schema-qualified key#1794
DMZ22 wants to merge 1 commit into
frictionlessdata:mainfrom
DMZ22:sql-namespace-table-lookup

Conversation

@DMZ22

@DMZ22 DMZ22 commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Writing a resource to a table inside a namespace (schema) raised KeyError:

resource.write(sql_url, control=formats.SqlControl(table="reprex", namespace="datapackage"))
# KeyError: 'reprex'   (frictionless/formats/sql/adapter.py, write_row_stream)

Cause. SqlAdapter.__init__ builds MetaData(schema=self.control.namespace). When a schema is set, SQLAlchemy keys every table by its schema-qualified name — Table.key is "<namespace>.<name>", and metadata.tables is keyed the same way:

>>> md = sa.MetaData(schema="ns")
>>> t = sa.Table("reprex", md, sa.Column("x", sa.Integer))
>>> list(md.tables)        # ['ns.reprex']
>>> t.key                  # 'ns.reprex'
>>> md.tables["reprex"]    # KeyError

But every lookup in the adapter used the bare name — self.metadata.tables[table_name] — so write_row_stream, read_schema, read_cell_stream, delete_resource and write_resource_with_metadata all KeyError the moment a namespace is in play.

Fix. Route the six lookups through a small helper that prepends the namespace when one is configured, matching how the tables are actually keyed:

def _table_key(self, table_name: str) -> str:
    if self.control.namespace:
        return f"{self.control.namespace}.{table_name}"
    return table_name

No behaviour changes when namespace is None (the helper returns the name unchanged), so the existing non-namespaced paths are untouched.

Verification

Reproduced and fixed end-to-end on SQLite with an attached schema (no server needed):

without fix:  KeyError: 'table'   at adapter.py write_row_stream
with fix:     wrote + read back -> [['id','name'], [1,'english'], [2,'中国人']]

Added test_sql_adapter_namespace_issue_1602 in the adapter spec's Bugs section — it attaches a second SQLite database as a schema, writes a package into it, and reads the rows back. Reverting only adapter.py while keeping the test makes it fail with the original KeyError, so it guards the fix. The full SQL adapter suite is 38 passed / 0 failed (59 skipped are the server-backed backends). ruff format --check is clean, and ruff check reports the same count on adapter.py before and after, so this adds no new lint.

The SQL adapter builds its metadata with MetaData(schema=namespace), so
SQLAlchemy keys each table as "<namespace>.<name>" (Table.key). Every
metadata.tables lookup used the bare table name, which raised
"KeyError: '<table>'" as soon as a namespace was set — writing, reading or
deleting a table in a schema was impossible.

Resolve the lookup key through a helper that prepends the namespace when
one is configured, matching how the tables are actually keyed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KeyError: 'table_name' when writing a resource to postgres table in schema

1 participant