feat(schema): add Explore query JSON Schema generator and Rust batch validator

.github/workflows/ci.yml

@@ -15,6 +15,11 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@v4
 
+      - name: Setup Rust toolchain
+        uses: dtolnay/rust-toolchain@stable
+        with:
+          components: clippy, rustfmt, llvm-tools-preview
+
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
@@ -29,9 +34,16 @@ jobs:
         run: |
           uv sync --extra all
 
-      - name: Run with pytest
+      - name: Run Rust clippy, unit tests, and build extension
+        run: |
+          cargo clippy --manifest-path lkr/schema/rust/Cargo.toml -- -D warnings
+          cargo test --no-default-features --manifest-path lkr/schema/rust/Cargo.toml
+          make schema-rs
+
+      - name: Run pytest
         run: |
           uv run pytest tests/ -v
+          uv run pytest tests/test_schema.py -v --cov=lkr.schema --cov=scripts.schema --cov-report=term-missing
 
       - name: Run ruff
         run: |
@@ -40,4 +52,3 @@ jobs:
       - name: Run ty
         run: |
           uv run ty check
-

.gitignore

@@ -2,4 +2,8 @@
 __pycache__
 .venv
 tmp
-.env
+.env
+lkr/schema/rust/target/
+coverage.xml
+.coverage
+*.profraw

.pre-commit-config.yaml

@@ -0,0 +1,23 @@
+repos:
+  - repo: https://github.com/astral-sh/uv-pre-commit
+    rev: 0.7.12
+    hooks:
+      - id: uv-lock
+        args: ["--default-index", "https://pypi.org/simple", "--no-config"]
+  - repo: local
+    hooks:
+      - id: ruff-check
+        name: ruff check
+        entry: uv run ruff check .
+        language: system
+        pass_filenames: false
+      - id: ty-check
+        name: ty check
+        entry: uv run ty check
+        language: system
+        pass_filenames: false
+      - id: test-and-coverage
+        name: pytest & rust coverage
+        entry: make coverage
+        language: system
+        pass_filenames: false

AGENTS.md

@@ -0,0 +1,7 @@
+# AGENTS.md
+
+## Pre-Commit & Testing Workflow
+
+- **Do NOT run tests, coverage, `ty check`, or `ruff check` after every individual file edit or change.**
+- Linting (`uv run ruff check .`), type-checking (`uv run ty check`), Rust unit tests/clippy (`make test-rs`), and Python/Rust test coverage (`make coverage`) are configured via [`.pre-commit-config.yaml`](.pre-commit-config.yaml) (powered by [`astral-sh/uv-pre-commit`](https://github.com/astral-sh/uv-pre-commit)) and run automatically at commit time.
+- Focus on completing your logical unit of work first; let `pre-commit` (or a single verification pass right before committing) run the full check and coverage suite.

Makefile

@@ -1,4 +1,4 @@
-.PHONY: docs test-deps codemode-test codemode-start
+.PHONY: docs test-deps codemode-test codemode-start schema-rs test-rs coverage-py coverage-rs coverage
 
 docs:
 	uv run typer lkr/main.py utils docs --output lkr.md
@@ -9,6 +9,20 @@ test-deps:
 codemode-test:
 	uv run pytest tests/test_codemode.py
 
+schema-rs:
+	cargo build --release --manifest-path lkr/schema/rust/Cargo.toml && cp lkr/schema/rust/target/release/lib_schema_rs.so lkr/schema/_schema_rs.so
+
+test-rs:
+	cargo clippy --manifest-path lkr/schema/rust/Cargo.toml -- -D warnings
+	cargo test --no-default-features --manifest-path lkr/schema/rust/Cargo.toml
+
+coverage-py:
+	uv run pytest tests/test_schema.py --cov=lkr.schema --cov=scripts.schema --cov-report=term-missing --cov-report=xml:coverage.xml
+
+coverage-rs:
+	RUSTFLAGS="-C instrument-coverage" LLVM_PROFILE_FILE="target/cov-%p-%m.profraw" cargo test --no-default-features --manifest-path lkr/schema/rust/Cargo.toml
+
+coverage: test-rs coverage-py coverage-rs
 
 download-swagger:
 	uv run python lkr/codemode/download_swagger.py

TODO.md

@@ -0,0 +1,33 @@
+# `lkr schema` Notes & Findings from `system__activity::history` (`vdAt9IRgn89vVzHSLxOczt`)
+
+## Answered Questions & Implemented Pipeline Steps
+
+Using `system__activity::history` (starting from query slug `vdAt9IRgn89vVzHSLxOczt`, schema saved in `tmp/system_activity_history.json`), we inspected real completed queries across multiple Looker models and explores and updated `lkr/schema/` accordingly:
+
+1. **Resolving `system__activity::history` `query.id` to `get_query` (`lkr/schema/history.py`)**:
+   - In `system__activity::history`, `query.id` is the internal database integer ID (e.g. `314`, `324`, `142769`), whereas Looker API 4.0 `sdk.query(query_id)` (`GET /queries/{query_id}`) requires the string `query.slug` (e.g. `"P2BfdfcRGtgx3QWn48RwnHFSsCPwWPKY"`).
+   - `resolve_history_queries` starts from `vdAt9IRgn89vVzHSLxOczt` (or custom `--history-slug`), queries `system__activity::history` for `["query.id", "query.slug", "query.model", "query.view"]` filtered by `--model`, `--explore`, and/or `--query-id`, calls `sdk.query(slug)`, strips read-only `Query` fields via `sanitize_read_query_to_write_query`, and validates each query via `validate_query`.
+
+2. **Real-world `dynamic_fields` Shapes (`lkr/schema/dynamic_fields.py` & `lkr/schema/validator.py`)**:
+   - **`TableCalculation`**: Can be expression-based (`expression`) OR shortcut-based (`calculation_type` such as `rank_of_column`, `percent_of_column_sum`, `running_total`, `percent_of_previous` + `args: [<field>]` + `based_on` / `source_field`, with no `expression`). Both forms and Looker UI metadata (`category`, `_kind_hint`, `_type_hint`, `is_disabled`, `__PARAMETER_LINE_NUMS`) are now supported and validated.
+   - **`CustomMeasure`**: Often omits `type` when derived from an existing measure (`based_on: "<view.measure>"`, `measure: "<name>"`) and may include `filters` (dict of `<field>: <filter_expr>`) or `filter_expression` (Looker custom filter formula). Both `based_on` and custom measure `filters` are now validated against the explore schema.
+   - **`CustomDimension`**: Supports `dimension`, `expression`, `based_on`, `calculation_type` + `args`, and Looker UI hints.
+   - **`${view.field}` Formula Reference Validation**: `_validate_expression_field_refs` validates that all `${view.field}` references inside `dynamic_fields[*].expression`, `dynamic_fields[*].filter_expression`, and `body.filter_expression` resolve to non-hidden explore fields or defined `dynamic_fields` `.name`s (ignoring Looker table calculation functions like `row()`, `offset()`, `pivot_index()`, `now()`).
+
+3. **Real-world `sorts` & Dashboard `filters` (`lkr/schema/fields.py` & `lkr/schema/filter_expressions.py`)**:
+   - Pivoted column sorts in Looker can omit `asc`/`desc` (`"<field> 0"` as well as `"<field> desc 0"`).
+   - `type: tier` dimensions in LookML emit companion sort fields `"<dimension>__sort_"` in `body.sorts` (`users.age_tier__sort_`, `user_order_facts.lifetime_orders_tier__sort_`), which are validated against visible dimensions.
+   - Unfiltered dashboard tiles frequently pass `""` (empty string) in `body.filters`.
+
+## TODO Bucket
+
+- [ ] **Capture `case_sensitive` on string filter properties (`x-case-sensitive: bool`)**:
+  - **API Limitation**: Looker's REST API (`LookmlModel`, `LookmlModelExplore`, and `LookmlModelExploreField` from `sdk.lookml_model_explore(model, explore)`) does **not** serialize `case_sensitive` at the model, explore, or field level.
+  - **Sourcing via LookML Parser**: Parse the underlying `.lkml` files (using `project_name` + `source_file` from the explore/field metadata with a LookML parser library) to resolve the effective boolean using Looker's inheritance cascade:
+    `field.case_sensitive ?? explore.case_sensitive ?? model.case_sensitive ?? True`
+  - **Schema Representation (Option B)**: In `build_filter_property_schema` (`lkr/schema/filter_expressions.py`), annotate string filter properties in `body.filters.properties.<field>` with:
+    - `"x-case-sensitive": bool` (machine-readable JSON Schema extension tag)
+    - Appended `[case_sensitive: true|false]` in `description` (for LLM/agent visibility).
+
+- [ ] **Strongly type `body.vis_config`**:
+  - Replace the untyped Swagger passthrough (`"additionalProperties": {"type": "any", "format": "any"}`) with strongly-typed schemas discriminated by `vis_config.type` (e.g., `looker_grid`, `looker_column`, `looker_bar`, `looker_line`, `looker_area`, `looker_pie`, `single_value`, maps, etc.).

lkr.md

@@ -11,14 +11,14 @@ $ lkr [OPTIONS] COMMAND [ARGS]...
 **Options**:
 
 * `--version`: Show the version and exit.
-* `--client-id TEXT`: [env var: LOOKERSDK_CLIENT_ID]
-* `--client-secret TEXT`: [env var: LOOKERSDK_CLIENT_SECRET]
-* `--base-url TEXT`: [env var: LOOKERSDK_BASE_URL]
-* `--log-level [DEBUG|INFO|WARNING|ERROR|CRITICAL]`: [env var: LOG_LEVEL]
+* `--client-id <str>`: [env var: LOOKERSDK_CLIENT_ID]
+* `--client-secret <str>`: [env var: LOOKERSDK_CLIENT_SECRET]
+* `--base-url <str>`: [env var: LOOKERSDK_BASE_URL]
+* `--log-level <DEBUG|INFO|WARNING|ERROR|CRITICAL>`: [env var: LOG_LEVEL]
 * `--quiet`
 * `--force-oauth`
 * `--dev`
-* `--oauth-account TEXT`: OAuth account to lookup in DB and use regardless of what&#x27;s active
+* `--oauth-account <str>`: OAuth account to lookup in DB and use regardless of what&#x27;s active
 * `--install-completion`: Install completion for the current shell.
 * `--show-completion`: Show completion for the current shell, to copy it or customize the installation.
 * `--help`: Show this message and exit.
@@ -27,6 +27,7 @@ $ lkr [OPTIONS] COMMAND [ARGS]...
 
 * `db-template`
 * `auth`: Authentication commands for LookML Repository
+* `schema`: Generate and validate strongly-typed...
 * `mcp`
 * `observability`
 * `tools`
@@ -42,7 +43,7 @@ $ lkr db-template [OPTIONS]
 
 **Options**:
 
-* `--import-error TEXT`: [default: No module named &#x27;lkr.db_template.main&#x27;]
+* `--import-error <str>`: [default: No module named &#x27;lkr.db_template.main&#x27;]
 
 ## `lkr auth`
 
@@ -77,8 +78,8 @@ $ lkr auth login [OPTIONS]
 
 **Options**:
 
-* `-I, --instance-name TEXT`: Name of the Looker instance to login or switch to
-* `-p, --port INTEGER RANGE`: Port to run the local OAuth redirect web server on  [1&lt;=x&lt;=65535]
+* `-I, --instance-name <str>`: Name of the Looker instance to login or switch to
+* `-p, --port <int range>`: Port to run the local OAuth redirect web server on  [1&lt;=x&lt;=65535]
 * `--help`: Show this message and exit.
 
 ### `lkr auth logout`
@@ -93,7 +94,7 @@ $ lkr auth logout [OPTIONS]
 
 **Options**:
 
-* `--instance-name TEXT`: Name of the Looker instance to logout from. If not provided, logs out from all instances.
+* `--instance-name <str>`: Name of the Looker instance to logout from. If not provided, logs out from all instances.
 * `--all`: Logout from all instances
 * `--help`: Show this message and exit.
 
@@ -125,6 +126,64 @@ $ lkr auth list [OPTIONS]
 
 * `--help`: Show this message and exit.
 
+## `lkr schema`
+
+Generate and validate strongly-typed Looker Explore query schemas
+
+**Usage**:
+
+```console
+$ lkr schema [OPTIONS] COMMAND [ARGS]...
+```
+
+**Options**:
+
+* `--help`: Show this message and exit.
+
+**Commands**:
+
+* `generate`: Generate a strongly-typed run_inline_query...
+* `validate`: Validate a query (--query or --query-file)...
+
+### `lkr schema generate`
+
+Generate a strongly-typed run_inline_query JSON Schema for --model and --explore.
+
+**Usage**:
+
+```console
+$ lkr schema generate [OPTIONS]
+```
+
+**Options**:
+
+* `-m, --model <str>`: LookML model name  [required]
+* `-e, --explore <str>`: LookML explore name  [required]
+* `-f, --explore-file <path>`: Path to a local lookml_model_explore JSON file (e.g. tmp/order_items.json)
+* `-o, --output <path>`: Optional file path to write the generated JSON schema
+* `--help`: Show this message and exit.
+
+### `lkr schema validate`
+
+Validate a query (--query or --query-file) against an Explore schema.
+
+**Usage**:
+
+```console
+$ lkr schema validate [OPTIONS]
+```
+
+**Options**:
+
+* `-m, --model <str>`: LookML model name
+* `-e, --explore <str>`: LookML explore name
+* `-f, --explore-file <path>`: Path to a local lookml_model_explore JSON file (e.g. tmp/order_items.json)
+* `--schema <str>`: Inline JSON schema string (optional; if omitted, generates schema from --model and --explore)
+* `--schema-file <path>`: Path to a generated JSON schema file (optional; if omitted, generates schema from --model and --explore)
+* `-q, --query <str>`: Inline query JSON string (either full run_inline_query payload or body WriteQuery)
+* `--query-file <path>`: Path to a query JSON file (either full run_inline_query payload or body WriteQuery)
+* `--help`: Show this message and exit.
+
 ## `lkr mcp`
 
 **Usage**:
@@ -182,10 +241,10 @@ $ lkr observability embed [OPTIONS]
 
 **Options**:
 
-* `--host TEXT`: Host to bind to  [env var: HOST; default: 0.0.0.0]
-* `--port INTEGER`: Port to bind to  [env var: PORT; default: 8080]
-* `--timeout INTEGER`: Timeout for the health check  [env var: TIMEOUT; default: 120]
-* `--event-prefix TEXT`: Event prefix  [env var: EVENT_PREFIX; default: lkr-observability]
+* `--host <str>`: Host to bind to  [env var: HOST; default: 0.0.0.0]
+* `--port <int>`: Port to bind to  [env var: PORT; default: 8080]
+* `--timeout <int>`: Timeout for the health check  [env var: TIMEOUT; default: 120]
+* `--event-prefix <str>`: Event prefix  [env var: EVENT_PREFIX; default: lkr-observability]
 * `--help`: Show this message and exit.
 
 ## `lkr tools`
@@ -216,8 +275,8 @@ $ lkr tools user-attribute-updater [OPTIONS]
 
 **Options**:
 
-* `--host TEXT`: [env var: HOST; default: 127.0.0.1]
-* `--port INTEGER`: [env var: PORT; default: 8080]
+* `--host <str>`: [env var: HOST; default: 127.0.0.1]
+* `--port <int>`: [env var: PORT; default: 8080]
 * `--help`: Show this message and exit.
 
 ### `lkr tools schedule-download-deprecation`
@@ -232,10 +291,10 @@ $ lkr tools schedule-download-deprecation [OPTIONS]
 
 **Options**:
 
-* `--limit INTEGER`: Search batch size  [default: 500]
-* `--model-offset INTEGER`: Offset for model columns  [default: 0]
+* `--limit <int>`: Search batch size  [default: 500]
+* `--model-offset <int>`: Offset for model columns  [default: 0]
 * `--csv`: Output as CSV instead of a table
-* `--csv-file-name TEXT`: Name for the output CSV file (without extension)  [default: schedule_download_deprecation]
+* `--csv-file-name <str>`: Name for the output CSV file (without extension)  [default: schedule_download_deprecation]
 * `--unfiltered`: Show all rows, including those with no missing permissions
 * `--email`: Use Email instead of Name
 * `--help`: Show this message and exit.
@@ -268,19 +327,20 @@ If --file / -f is specified (or folder_name is a file), only that single file is
 **Usage**:
 
 ```console
-$ lkr tools lookml push [OPTIONS] FOLDER_NAME
+$ lkr tools lookml push [OPTIONS] {folder_name}
 ```
 
 **Arguments**:
 
-* `FOLDER_NAME`: Local folder name / Looker project ID to push  [required]
+* `folder_name`: Local folder name / Looker project ID to push  [required]
 
 **Options**:
 
-* `--project-id, --project TEXT`: Looker project ID to push to (if different from folder name)
-* `-f, --file TEXT`: Single file relative path (or absolute path) to push
+* `--project-id, --project <str>`: Looker project ID to push to (if different from folder name)
+* `-f, --file <str>`: Single file relative path (or absolute path) to push
 * `--deploy`: Commit and deploy to production after push
-* `--message TEXT`: Commit message when deploying  [default: push from lkr cli]
+* `--reset / --no-reset`: Reset developer workspace to production before pushing  [default: reset]
+* `--message <str>`: Commit message when deploying  [default: push from lkr cli]
 * `--help`: Show this message and exit.
 
 #### `lkr tools lookml pull`
@@ -291,19 +351,19 @@ If --file / -f is specified, only that single file is pulled without deleting lo
 **Usage**:
 
 ```console
-$ lkr tools lookml pull [OPTIONS] FOLDER_NAME
+$ lkr tools lookml pull [OPTIONS] {folder_name}
 ```
 
 **Arguments**:
 
-* `FOLDER_NAME`: Local folder name / Looker project ID to pull into  [required]
+* `folder_name`: Local folder name / Looker project ID to pull into  [required]
 
 **Options**:
 
-* `--project-id, --project TEXT`: Looker project ID to pull from (if different from folder name)
-* `-f, --file TEXT`: Single file relative path to pull from Looker
+* `--project-id, --project <str>`: Looker project ID to pull from (if different from folder name)
+* `-f, --file <str>`: Single file relative path to pull from Looker
 * `--deploy`: Commit and deploy to production on Looker after pull
-* `--message TEXT`: Commit message when deploying  [default: pull from lkr cli then commit and deploy]
+* `--message <str>`: Commit message when deploying  [default: pull from lkr cli then commit and deploy]
 * `--help`: Show this message and exit.
 
 #### `lkr tools lookml deploy`
@@ -313,17 +373,17 @@ Commit dev workspace and deploy Looker project to production.
 **Usage**:
 
 ```console
-$ lkr tools lookml deploy [OPTIONS] [FOLDER_NAME]
+$ lkr tools lookml deploy [OPTIONS] [folder_name]
 ```
 
 **Arguments**:
 
-* `[FOLDER_NAME]`: Local folder name / Looker project ID to deploy
+* `folder_name`: Local folder name / Looker project ID to deploy
 
 **Options**:
 
-* `--project-id, --project TEXT`: Looker project ID to deploy (if folder_name not specified)
-* `--message TEXT`: Commit message  [default: commit and deploy from lkr cli]
+* `--project-id, --project <str>`: Looker project ID to deploy (if folder_name not specified)
+* `--message <str>`: Commit message  [default: commit and deploy from lkr cli]
 * `--help`: Show this message and exit.
 
 ## `lkr code-mode`
@@ -353,10 +413,11 @@ $ lkr code-mode sandbox [OPTIONS]
 
 **Options**:
 
-* `-c, --code TEXT`: Execute Python code directly in the sandbox
-* `-f, --file TEXT`: Execute Python code from a file in the sandbox
+* `-c, --code <str>`: Execute Python code directly in the sandbox
+* `-f, --file <str>`: Execute Python code from a file in the sandbox
 * `--dev-mode`: Run in dev mode
-* `-v, --var TEXT`: Inject variable as key=value pair (e.g. -v project=my_project)
+* `--allow-update-session`: Allow calling update_session inside code-mode
+* `-v, --var <str>`: Inject variable as key=value pair (e.g. -v project=my_project)
 * `--help`: Show this message and exit.
 
 ### `lkr code-mode run`