Summary
The temporary Spark-repr migration shim added in PR #1192 (April 2026) has reached its July 2026 expiry date. The in-code comment on _LEGACY_SPARK_REPR_RE explicitly says "remove after July 2026" — that date has passed (today is 2026-08-01).
What to remove
In datacontract/export/sql_type_converter.py:
-
The _LEGACY_SPARK_REPR_RE constant and its preceding comment (lines 11–13):
# invalid physicalType from `import spark` in v0.11.0–v0.12.1 (#1048) (e.g. StringType() instead of string)
# remove after July 2026
_LEGACY_SPARK_REPR_RE = re.compile(r"^(?:[A-Z][a-zA-Z]*)?Type\(.*\)$")
-
The legacy-note branch inside _warn_cannot_map_type (lines 30–34):
if isinstance(raw_type, str) and _LEGACY_SPARK_REPR_RE.match(raw_type):
warning_message += (
"\nNote: `datacontract import spark` emitted invalid physicalType values in v0.11.0–v0.12.1 (December 2025 until June 2026). "
"Re-import datacontracts from these versions to fix."
)
Only remove this if block — the rest of _warn_cannot_map_type (the logger.warning call etc.) must stay.
-
The import re line at the top — but only if re is no longer used anywhere else in the file. Run grep -n '\bre\b' datacontract/export/sql_type_converter.py to verify before removing it.
Context
- Shim was introduced to help users who imported data contracts with versions v0.11.0–v0.12.1, which emitted invalid Spark
physicalType values like StringType() instead of string.
- The shim appended a user-facing note to the "cannot map type" warning so affected users could re-import to fix their contracts.
- Enough time has passed (8+ months since v0.12.1) that the migration notice is no longer necessary.
Summary
The temporary Spark-repr migration shim added in PR #1192 (April 2026) has reached its July 2026 expiry date. The in-code comment on
_LEGACY_SPARK_REPR_REexplicitly says "remove after July 2026" — that date has passed (today is 2026-08-01).What to remove
In
datacontract/export/sql_type_converter.py:The
_LEGACY_SPARK_REPR_REconstant and its preceding comment (lines 11–13):The legacy-note branch inside
_warn_cannot_map_type(lines 30–34):Only remove this
ifblock — the rest of_warn_cannot_map_type(thelogger.warningcall etc.) must stay.The
import reline at the top — but only ifreis no longer used anywhere else in the file. Rungrep -n '\bre\b' datacontract/export/sql_type_converter.pyto verify before removing it.Context
physicalTypevalues likeStringType()instead ofstring.