Skip to content

Commit e524f49

Browse files
authored
fix: array_concat widens container variant for mixed List/LargeList inputs (#21704)
## Which issue does this PR close? - Closes #21702. ## Rationale for this change `array_concat` hit an internal cast error when given a mix of `List` and `LargeList` (or `FixedSizeList` and `LargeList`) arguments: ```sql > select array_concat(make_array(1, 2), arrow_cast([3, 4], 'LargeList(Int64)')); DataFusion error: Internal error: could not cast array of type List(Int64) to arrow_array::array::list_array::GenericListArray<i64>. ``` `ArrayConcat::coerce_types` was coercing only the base element type, leaving the outer container alone. When the resolved return type is `LargeList`, `array_concat_inner` later tries to downcast each arg to `GenericListArray<i64>`, which fails for any `List` argument that slipped through. ## What changes are included in this PR? In `ArrayConcat::coerce_types`, after coercing the base type, also promote each input's outermost `List` to `LargeList` when the return type is a `LargeList`. `FixedSizeList` inputs already go through `FixedSizedListToList` first and then get promoted too. Per-arg dimensionality is preserved, so nested cases keep working with `align_array_dimensions`. ## Are these changes tested? Yes, added sqllogictests in `array_concat.slt` covering: - `List` + `LargeList` - `LargeList` + `List` - `FixedSizeList` + `LargeList` - Three-way mix `List`, `LargeList`, `List` Each one also asserts `arrow_typeof(...) = LargeList(Int64)`. ## Are there any user-facing changes? Queries that previously returned an internal cast error now return the concatenated `LargeList` as expected. No API changes.
1 parent c470bb1 commit e524f49

2 files changed

Lines changed: 47 additions & 2 deletions

File tree

datafusion/functions-nested/src/concat.rs

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -317,10 +317,23 @@ impl ScalarUDFImpl for ArrayConcat {
317317
}
318318

319319
fn coerce_types(&self, arg_types: &[DataType]) -> Result<Vec<DataType>> {
320-
let base_type = base_type(&self.return_type(arg_types)?);
320+
let return_type = self.return_type(arg_types)?;
321+
let base_type = base_type(&return_type);
321322
let coercion = Some(&ListCoercion::FixedSizedListToList);
323+
// When the return type is a `LargeList`, the outer container of every
324+
// input must be widened to `LargeList` as well. Otherwise
325+
// `array_concat_inner` would later try to downcast a `List` argument
326+
// to `GenericListArray<i64>` and fail.
327+
let promote_to_large_list = matches!(return_type, DataType::LargeList(_));
322328
let arg_types = arg_types.iter().map(|arg_type| {
323-
coerced_type_with_base_type_only(arg_type, &base_type, coercion)
329+
let coerced =
330+
coerced_type_with_base_type_only(arg_type, &base_type, coercion);
331+
match coerced {
332+
DataType::List(field) if promote_to_large_list => {
333+
DataType::LargeList(field)
334+
}
335+
other => other,
336+
}
324337
});
325338

326339
Ok(arg_types.collect())

datafusion/sqllogictest/test_files/array/array_concat.slt

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,38 @@ select
121121
----
122122
[1, 2, 3] List(Utf8View)
123123

124+
# Concatenating mixed list and large list — return type widens to LargeList
125+
query ?T
126+
select
127+
array_concat(make_array(1, 2), arrow_cast([3, 4], 'LargeList(Int64)')),
128+
arrow_typeof(array_concat(make_array(1, 2), arrow_cast([3, 4], 'LargeList(Int64)')));
129+
----
130+
[1, 2, 3, 4] LargeList(Int64)
131+
132+
# Reverse argument order: LargeList first, plain list second
133+
query ?T
134+
select
135+
array_concat(arrow_cast([1, 2], 'LargeList(Int64)'), make_array(3, 4)),
136+
arrow_typeof(array_concat(arrow_cast([1, 2], 'LargeList(Int64)'), make_array(3, 4)));
137+
----
138+
[1, 2, 3, 4] LargeList(Int64)
139+
140+
# FixedSizeList combined with LargeList — also widens to LargeList
141+
query ?T
142+
select
143+
array_concat(arrow_cast([1, 2], 'FixedSizeList(2, Int64)'), arrow_cast([3, 4], 'LargeList(Int64)')),
144+
arrow_typeof(array_concat(arrow_cast([1, 2], 'FixedSizeList(2, Int64)'), arrow_cast([3, 4], 'LargeList(Int64)')));
145+
----
146+
[1, 2, 3, 4] LargeList(Int64)
147+
148+
# Three-way mix: List, LargeList, List
149+
query ?T
150+
select
151+
array_concat(make_array(1, 2), arrow_cast([3], 'LargeList(Int64)'), make_array(4, 5)),
152+
arrow_typeof(array_concat(make_array(1, 2), arrow_cast([3], 'LargeList(Int64)'), make_array(4, 5)));
153+
----
154+
[1, 2, 3, 4, 5] LargeList(Int64)
155+
124156
# array_concat with NULL elements inside arrays
125157
query ?
126158
select array_concat([1, NULL, 3], [NULL, 5]);

0 commit comments

Comments
 (0)