feat[next]: dtype-generic type system foundations by havogt · Pull Request #2660 · GridTools/gt4py

havogt · 2026-06-16T15:04:23Z

Preparation for dtype-generic field operators in gt4py.next (first step of the series): writing a field operator once and instantiating it at several scalar dtypes, spelled with a value-constrained typing.TypeVar such as TypeVar("T", float32, float64). The architecture is monomorphization at call time — the operator is type-checked once in terms of its type variables and then specialized to a concrete dtype for each call.

This PR adds only the (inert) type-system foundation; nothing produces a type variable yet, so there is no user-visible behavior change.

Newly introduced type:

ts.TypeVarType — a DataType (TypeSpec) carrying a name and a non-empty tuple of scalar constraints. Two occurrences of the same name within one signature denote the same type; it coexists with DeferredType rather than replacing it.
FieldType.dtype is widened to also allow a TypeVarType.

Supporting (dtype-scoped) utilities the later steps build on:

type_info.is_generic — recursive over composite types.
bind_type_vars / substitute_type_vars.
TypeVar-aware variants of the dtype predicates, promote, and extract_dtype / extract_dims.

Design and decisions are recorded in ADR 0023.

Add the type-system machinery for dtype-generic operators, inert until the frontend produces it: - ts.TypeVarType (value-constrained scalar type variable); widen FieldType.dtype to ScalarType | ListType | TypeVarType. - type_info.is_generic (recursive), bind_type_vars / substitute_type_vars (dtype-scoped: name -> ScalarType), and TypeVar-aware dtype predicates / promote / extract_dtype / extract_dims. - Route CompiledProgramsPool._is_generic through type_info.is_generic, resolving its TODO (behavior-preserving for scan operators). Widening FieldType.dtype requires narrowing at the concrete-IR lowering sites (dace, global_tmps), where the D4 invariant guarantees no TypeVarType, plus a return-type annotation in FOAST type deduction. Records the decision in ADR 0023. See the implementation roadmap for the PR series.

havogt · 2026-06-16T17:11:55Z

-                type_info.extract_dtype,
-                result_collection_constructor=as_tuple,
-            )(tmp_expr.type)
+            ) = cast(


check if we can express this without a cast

havogt · 2026-06-16T17:13:20Z

-                self.dc_node, self.gt_type.dtype, field_origin, it_indices
-            )
+            dtype = self.gt_type.dtype
+            assert isinstance(dtype, (ts.ScalarType, ts.ListType))


Can all these assert be avoided by better concept definition?

The problem with widening the type is that now everything needs to either support the generic case or assert like here, even though after specialization none of these cases can occur. A general generic type would solve this automatically, before specialization things need to support the generic, e.g. GenericType[ScalarType], afterwards things can stay they way they are.

- canonicalize TypeVarType.constraints (order carries no meaning for a value-constrained TypeVar) so the type's identity is order-insensitive - unify the type-variable traversals: add _type_params / tree_map_type_params and reexpress is_generic and substitute_type_vars on them; reword the is_generic docstring (deep check, not the negation of is_concrete) - document the non-scalar case in bind_var and the dtype-union narrowing assert in the dace lowering - ADR 0023: tighten, rename the example, drop the external design-investigation reference, and add it to the ADR index under a new Type System section

Put the type-variable utility doctests under an 'Examples:' section (matching the rest of type_info.py), make the _type_params summary a sentence, and drop docstring and comment text describing rationale, callers, or future plans rather than behavior.

Replace the mutating closure-based walk with pure module-level helpers (_bind_var / _merge_bindings / _bind) that return and merge per-leaf bindings; keeps the tolerant structural recursion (mismatches deferred to the signature checks).

havogt · 2026-06-19T09:04:49Z

+    return tuple(sorted(constraints, key=lambda c: c.kind))
+
+
+class TypeVarType(DataType):


Suggested change

class TypeVarType(DataType):

class DTypeVarType(DataType):

for now?

…pe-system

Upstream main now occupies the 0023 slot with the Fingerprinting ADR, so move this ADR to the next free number and update the index accordingly.

tehrengruber · 2026-06-22T00:29:31Z

-                self.dc_node, self.gt_type.dtype, field_origin, it_indices
-            )
+            dtype = self.gt_type.dtype
+            assert isinstance(dtype, (ts.ScalarType, ts.ListType))


The problem with widening the type is that now everything needs to either support the generic case or assert like here, even though after specialization none of these cases can occur. A general generic type would solve this automatically, before specialization things need to support the generic, e.g. GenericType[ScalarType], afterwards things can stay they way they are.

Route substitution through the existing `tree_map_type` rather than a bespoke type-parameter map, handling the dtype / element-type / signature rewrite in the leaf function. Drop the upfront `is_generic` check: the traversal runs unconditionally and reconstructs the tree. Addresses the review comment on the `tree_map_type_params` helper.

is_arithmetic hand-rolled its own TypeVarType handling under a comment claiming it could not reuse _is_field_or_scalar_of_kind. That helper takes a kind *set*, so arithmetic is just membership in the union of the floating point and integral kinds; all(constraint in union) is correct even for a type variable mixing float and integral constraints. Route it through the shared helper via _ARITHMETIC_KINDS, removing the duplicated constraint fold.

The positional/keyword/return sub-type layout of FunctionType was spelled out independently in _type_params, substitute_type_vars and is_compatible_type. Extract it into _function_type_arg_groups (the grouped layout, kept grouped so is_compatible_type still enforces per-group arity), _function_type_children (the flat enumeration) and _map_function_type (structure-preserving rebuild), and route the three call sites through them.

The dace lowering narrows FieldType.dtype past the widened TypeVarType with plain 'assert isinstance(dtype, (ts.ScalarType, ts.ListType))'. Introduce a type_info.is_concrete_dtype TypeIs guard and route those asserts through it, so the narrowing is self-documenting (the dtype is concrete, i.e. not a type variable) and the intent is stated once.

havogt · 2026-06-25T08:54:49Z

+    )
+
+
+def _type_params(symbol_type: ts.TypeSpec) -> tuple[ts.TypeSpec, ...]:


Can this be used as a generic type_tree_map?

havogt changed the title ~~feat[next]: dtype-generic type system foundations (Stage 0)~~ feat[next]: dtype-generic type system foundations Jun 16, 2026

havogt commented Jun 17, 2026

View reviewed changes

havogt added 4 commits June 17, 2026 11:42

Merge upstream main into dtype-generic type system foundations

f787966

refactor[next]: make bind_type_vars functional

be62638

Replace the mutating closure-based walk with pure module-level helpers (_bind_var / _merge_bindings / _bind) that return and merge per-leaf bindings; keeps the tolerant structural recursion (mismatches deferred to the signature checks).

havogt commented Jun 19, 2026

View reviewed changes

havogt added 2 commits June 19, 2026 11:51

Merge remote-tracking branch 'upstream/main' into dtype-generics-2-ty…

d68e060

…pe-system

docs[next]: renumber dtype-generic ADR to 0024

339f1dd

Upstream main now occupies the 0023 slot with the Fingerprinting ADR, so move this ADR to the next free number and update the index accordingly.

havogt requested a review from tehrengruber June 19, 2026 10:00

tehrengruber reviewed Jun 22, 2026

View reviewed changes

havogt commented Jun 22, 2026

View reviewed changes

Comment thread src/gt4py/next/type_system/type_specifications.py

havogt commented Jun 22, 2026

View reviewed changes

Comment thread src/gt4py/next/type_system/type_specifications.py

havogt mentioned this pull request Jun 22, 2026

[prototype] Approach 3: PartialTypeSpec lazy generic types (vs Stage-0) havogt/gt4py#66

Draft

havogt added 4 commits June 23, 2026 20:23

havogt commented Jun 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat[next]: dtype-generic type system foundations#2660

feat[next]: dtype-generic type system foundations#2660
havogt wants to merge 11 commits into
GridTools:mainfrom
havogt:dtype-generics-2-type-system

havogt commented Jun 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

havogt Jun 16, 2026

Uh oh!

havogt Jun 16, 2026

Uh oh!

tehrengruber Jun 22, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

havogt Jun 19, 2026

Uh oh!

tehrengruber Jun 22, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

havogt Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		return tuple(sorted(constraints, key=lambda c: c.kind))


		class TypeVarType(DataType):

		)


		def _type_params(symbol_type: ts.TypeSpec) -> tuple[ts.TypeSpec, ...]:

Uh oh!

Conversation

havogt commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

havogt Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

havogt Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

tehrengruber Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

havogt Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

tehrengruber Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

havogt Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

havogt commented Jun 16, 2026 •

edited

Loading