Skip to content

feat(schema): add Explore query JSON Schema generator and Rust batch validator - #49

Merged
bwebs merged 7 commits into
mainfrom
bryan/explore-schema
Sep 24, 2026
Merged

bwebs merged 7 commits into
mainfrom
bryan/explore-schema

Conversation

@bwebs

@bwebs bwebs commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Adds strongly-typed Looker Explore query JSON Schema generation (lkr schema generate) and high-performance Rust/PyO3 + rayon query validation (lkr schema validate), along with the developer history validation harness in scripts/schema/.

Key Changes

  • lkr schema generate: Generates Draft 2020-12 JSON Schemas for any LookML Explore (--model, --explore), extracting visible dimensions, measures, filters, parameters, typed filter expression definitions ($defs), sort regexes, and dynamic fields (CustomDimension, CustomMeasure, TableCalculation).
  • lkr schema validate: High-throughput Rust + PyO3 + rayon validator (lkr/schema/rust/src/lib.rs) validating query payloads against Explore schemas, including ${view.field} formula reference validation across dynamic_fields and filter_expression, read-only Query sanitization, and column_limit without pivots warnings.
  • scripts/schema/: Developer harness for fetching and batch-validating completed queries from system__activity::history.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a strongly-typed Looker Explore query schema generator and validator under the lkr schema CLI command. It features a high-performance validator implemented in Rust using PyO3 and Rayon for parallel batch validation, along with a testing harness that validates real queries from Looker's system activity history. The review feedback is highly constructive, identifying a potential ReDoS vulnerability in the date-time filter regex, suggesting memory allocation optimizations in the Rust validator, recommending stricter type validation for custom measure filters, and advising better error handling for missing or malformed local explore files.

Comment thread lkr/schema/filter_expressions.py Outdated
Comment thread lkr/schema/rust/src/lib.rs Outdated
Comment thread lkr/schema/rust/src/lib.rs Outdated
Comment thread lkr/schema/main.py Outdated
@bwebs
bwebs merged commit 9fe4b75 into main Sep 24, 2026
1 check passed
@bwebs
bwebs deleted the bryan/explore-schema branch September 24, 2026 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant