Skip to content

Integrate JiT compilation into CLI#2110

Open
rafal-hawrylak wants to merge 1 commit into
mainfrom
cli_jit_integration
Open

Integrate JiT compilation into CLI#2110
rafal-hawrylak wants to merge 1 commit into
mainfrom
cli_jit_integration

Conversation

@rafal-hawrylak
Copy link
Copy Markdown
Collaborator

No description provided.

@rafal-hawrylak rafal-hawrylak self-assigned this Mar 9, 2026
@rafal-hawrylak rafal-hawrylak marked this pull request as ready for review March 9, 2026 20:28
@rafal-hawrylak rafal-hawrylak requested a review from a team as a code owner March 9, 2026 20:28
@rafal-hawrylak rafal-hawrylak enabled auto-merge (squash) March 9, 2026 20:28
Copy link
Copy Markdown
Contributor

@kolina kolina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's test compatibility with different Dataform core versions.

Also this change seems worth it to bump a minor version of Dataform core

Comment thread cli/api/commands/compile.ts
Comment thread cli/api/commands/compile.ts
Comment thread cli/api/commands/compile.ts
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/tests/jit/jit_run_test.ts
Comment thread cli/api/commands/compile.ts Outdated
Comment thread packages/@dataform/cli/worker.ts Outdated
Comment thread protos/execution.proto
Comment thread tests/api/api.spec.ts Outdated
Copy link
Copy Markdown
Collaborator

@ikholopov-omni ikholopov-omni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rafal-hawrylak rafal-hawrylak force-pushed the cli_jit_integration branch 13 times, most recently from 10bb4e6 to 824d7d9 Compare April 3, 2026 14:08
@rafal-hawrylak rafal-hawrylak force-pushed the cli_jit_integration branch 3 times, most recently from 2466f28 to 38cd3b9 Compare April 7, 2026 09:24
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/api/dbadapters/bigquery.ts Outdated
Comment thread cli/vm/jit_worker.ts Outdated
Comment thread cli/console.ts Outdated
Comment thread cli/index.ts
Comment thread packages/@dataform/cli/worker.ts
@rafal-hawrylak rafal-hawrylak force-pushed the cli_jit_integration branch 4 times, most recently from 807dae4 to 82adbdd Compare April 18, 2026 22:21
Copy link
Copy Markdown
Contributor

@kolina kolina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No outstanding issues from my side except question about access of JiT compiled code to other files

Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/api/commands/run.ts Outdated
Comment thread cli/vm/jit_worker.ts Outdated
Comment thread packages/@dataform/cli/worker.ts
feat: enable JiT compilation in CLI run, build and compile commands
Connect main CLI execution flow to the JiT worker process

Add comprehensive JiT test suite

Finalize AoT build logic to preserve JiT-specific fields

Update CLI console and entry points for integrated JiT support

Add file_name field (=8) to JitCompilationRequest proto for target source
file resolution in the JiT worker.
@rafal-hawrylak
Copy link
Copy Markdown
Collaborator Author

Blocked on review of https://github.com/dataform-co/dataform/pull/2109/changes

Unblocked

@rafal-hawrylak
Copy link
Copy Markdown
Collaborator Author

No outstanding issues from my side except question about access of JiT compiled code to other files

Handled the access of external files to be on pair with GCP approach to provide portability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants