[UUID 6/8] UUID multi-stage engine (planner + runtime) - #18874
Open
xiangfu0 wants to merge 5 commits into
Open
Conversation
This was referenced Jun 29, 2026
xiangfu0
force-pushed
the
uuid-split/06-mse-planner-runtime
branch
3 times, most recently
from
June 30, 2026 09:14
7a7cf63 to
22f38d3
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #18874 +/- ##
============================================
- Coverage 66.65% 66.61% -0.05%
Complexity 1423 1423
============================================
Files 3443 3446 +3
Lines 218632 218968 +336
Branches 34793 34885 +92
============================================
+ Hits 145726 145857 +131
- Misses 61192 61376 +184
- Partials 11714 11735 +21
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
xiangfu0
force-pushed
the
uuid-split/06-mse-planner-runtime
branch
7 times, most recently
from
July 7, 2026 07:08
763c96a to
96284ae
Compare
xiangfu0
force-pushed
the
uuid-split/06-mse-planner-runtime
branch
11 times, most recently
from
July 14, 2026 08:03
3e1c95c to
f41f8ea
Compare
xiangfu0
force-pushed
the
uuid-split/06-mse-planner-runtime
branch
10 times, most recently
from
July 21, 2026 08:10
1cf12f0 to
bd7f7af
Compare
xiangfu0
force-pushed
the
uuid-split/06-mse-planner-runtime
branch
from
July 22, 2026 08:11
bd7f7af to
c6669fe
Compare
This was referenced Jul 27, 2026
xiangfu0
force-pushed
the
uuid-split/06-mse-planner-runtime
branch
8 times, most recently
from
August 9, 2026 09:16
f5ffcae to
f263ef9
Compare
Part 5/8 of splitting apache#18140 (logical UUID type). Rebased onto latest master; stacked on uuid-split/04-sse-predicates-cast. Downstream references use the UuidKey class merged in apache#18869.
… path getConvertedKey had `case UUID` falling through to BYTES, returning the raw byte[]. The other reduce path converts group keys via ColumnDataType#convert, and UUID is the one type whose converted form is not its stored bytes -- it yields a java.util.UUID. PredicateRowMatcher casts that directly (see apache#18872), so the byte[] made GROUP BY ... HAVING over a UUID column fail with "ClassCastException: class [B cannot be cast to class java.util.UUID". Delegating to columnDataType.convert(...) keeps the two paths identical by construction rather than by duplicated knowledge. Covered by a new UuidAggregationTest integration test rather than a unit test. The unit-level BaseQueriesTest harness cannot reach this code, which is why it was uncovered; a query-level test goes through the real broker reduce. Verified by reverting the fix: testGroupByUuidColumnWithHaving and testGroupByUuidColumnWithHavingReturningFinalResult both fail with the ClassCastException and both pass with it. The test also covers GROUP BY key rendering, DISTINCT (BytesDistinctTable no longer hard-codes hex) and DISTINCTCOUNT / DISTINCTCOUNTHLL / DISTINCTCOUNTBITMAP over a UUID column.
Both callers key on UuidKey already (NoDictionary{Single,Multi}ColumnGroupKey
Generator, via UuidKey.fromBytes), so UuidKey.fromObject accepted five input
types where exactly one is ever passed, and ran an instanceof chain per row in
the group-by loop.
Casting directly matches the sibling maps -- DoubleToIdMap casts to double --
and keeps the input type deterministic, which is what was asked for on the
equivalent PredicateRowMatcher branch in apache#18872.
…cal string The previous version rendered each UUID as its 36-char canonical string so that DISTINCTCOUNTHLL(uuidCol) would equal DISTINCTCOUNTHLL(CAST(uuidCol AS STRING)). No other logical type provides that guarantee: the scan path switches on the stored type, so TIMESTAMP offers its raw millis and BOOLEAN its int, and neither matches a CAST to STRING. The UUID rendering was inventing a cross-type equivalence at the cost of a String allocation per row in the aggregation loop. UUID now hashes its stored 16 bytes. Verified byte[] is content-hashed rather than identity-hashed by both HyperLogLog (clearspring MurmurHash) and UltraLogLogUtils.OBJECT_FUNNEL (putBytes). A minimal guard is still needed at each site, because unlike LONG or INT the stored BYTES type is not a scalar case in this family: the scan path has no `case BYTES`, and the dictionary path reads BYTES as serialized sketch state. - AggregationFunctionUtils: the three UUID blocks are gone; the BYTES guard now excludes UUID so it falls through to the scalar path, which offers dictionary.get(i) -- the stored byte[] -- exactly as the scan path does. - DistinctCountBitmap hashes Arrays.hashCode(bytes). - DistinctCountThetaSketch cannot take scalar bytes (BYTES there means "serialized sketch"), so it surfaces the stored hex rendering instead. Replaces testUuidDistinctCountHllMatchesStringDistinctCountHll, which asserted the invariant being dropped, with one pinning the new behaviour.
Part 6/8 of splitting apache#18140 (logical UUID type). Rebased onto latest master; stacked on uuid-split/05-agg-groupby-distinct. Downstream references use the UuidKey class merged in apache#18869.
xiangfu0
force-pushed
the
uuid-split/06-mse-planner-runtime
branch
from
August 9, 2026 09:20
f263ef9 to
0615256
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parent tracking issue: #16619
What
Multi-stage (v2) engine support for UUID.
Changes
expressions.protoUUID / UUID_ARRAY enum values + planner type resolution, Rex parsing, proto serde,TypeFactoryHashJoinOperator/EnrichedHashJoinOperator+UuidLookupTable(LookupTable.normalizeKey),OneUuidKeyGroupIdGenerator,ServerPlanRequestUtilsEnables
UUID equi-joins and group-by in the multi-stage engine.
Depends on
#18869 and #18871.
About this PR / how to review
This is part 6 of 8 splitting #18140 (first-class logical
UUIDtype) into layered PRs, as requested there.The split is enabled by the v1 design:
DataType.UUIDhas stored typeBYTES, so most paths handle it automatically; each PR adds explicit UUID semantics to one subsystem. Head branch lives onxiangfu0/pinot.This PR is stacked on #18873 (branch
uuid-split/05-agg-groupby-distinct). Because GitHub PRs to apache must base onmaster, the Files-changed tab is cumulative (it includes layers 1–6) until the PRs below it merge. Review the commit titled[UUID 6/8] UUID multi-stage engine (planner + runtime)— that is this layer's change. Each parent merge shrinks this diff after a rebase.Full stack (merge bottom → top)
After #18869 lands, PRs 2/4/7 (#18870, #18872, #18875) only depend on it and can be reviewed in parallel; PRs 5/6 (#18873, #18874) also need #18871; PR 8 (#18876) needs #18874.
Full feature description, v1 design contract, scope exclusions, and benchmark numbers: #18140.