Add parquet DELTA read tests and nested benchmark axis - #23771
Draft
vyasr wants to merge 3 commits into
Draft
Conversation
…_input_values=33) Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> (cherry picked from commit 1e99ed8)
… pages Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> (cherry picked from commit 5e0fef0)
Nested DELTA pages have no benchmark today. The DeltaBinaryList* gtests are correctness-only and TPC-H has no nested DELTA columns, so the 3-warp decode_delta_binary_kernel<Flat=false> path is currently unmeasured -- which makes it impossible to say whether replacing it regresses anything. nesting=0 keeps the flat columns these benchmarks have always used; nesting=N wraps the leaf type in N levels of LIST, which is what puts the decoder on its repetition path. Setting the encoding needed to become recursive. The writer only honours the request on the schema node whose physical type matches: it skips nodes named "list" and reaches the values through lists_column_view::child_column_index, so a request left on the outermost column never lands on a LIST's leaf. Verified the nested configuration measures what it claims, rather than assuming it: under nsys, nesting=1 with DELTA_BINARY_PACKED runs decode_delta_binary_kernel<unsigned char, (bool)0> -- the Flat=false instantiation -- for 95.5% of GPU time (65.8 ms x 209 launches), and no compute_nz_idx_kernel appears, which is the expected shape while nested pages have no pre-pass. Reported noise is 0.19%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit c4fbfec)
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
nestingaxis to DELTA parquet read benchmarks so nested LIST paths are measuredTesting
git diff --check upstream/main..HEADbuild-cudf-cpp -j0 -DBUILD_BENCHMARKS=ON./PARQUET_TEST --gtest_filter="ParquetWriterTest.DeltaBinaryNzIdxTailIteration33:ParquetReaderTest.DeltaBinaryBoundaryWordMerge"./PARQUET_READER_NVBENCH --benchmark parquet_read_delta_binary --devices 0 --axis encoding=DELTA_BINARY_PACKED --axis io_type=DEVICE_BUFFER --axis cardinality=1000 --axis run_length=1 --axis nesting=1 --axis data_size=1048576 --profile --min-samples 1 --timeout 30 --csv stdoutNote: the full build and focused checks were run before rebasing this extracted branch onto latest
upstream/main; after the rebase,git diff --check upstream/main..HEADwas rerun successfully.