Update ort and ort-genai package

[baijumeswani]

Package reference mapping after this PR:

Binding / package	Runtime dependencies	Installation-time native packages	Bundled native content
C++ Microsoft.AI.Foundry.Local.Runtime	Microsoft.ML.OnnxRuntime 1.28.0; Microsoft.ML.OnnxRuntimeGenAI.Foundry 0.15.1	Restored by NuGet as declared dependencies.	foundry_local for Windows x64/ARM64, Linux x64/ARM64, and macOS ARM64. Windows also includes Microsoft.Windows.AI.MachineLearning.dll 2.1.70.
C# Microsoft.AI.Foundry.Local	Betalgo.Ranul.OpenAI 9.1.0; Microsoft.Extensions.Logging 9.0.9; Microsoft.AI.Foundry.Local.Runtime at the SDK package version. netstandard2.0 also uses System.Memory 4.6.0, Microsoft.Bcl.AsyncInterfaces 9.0.9, and System.Threading.Channels 9.0.9.	NuGet transitively restores ONNX Runtime 1.28.0 and ONNX Runtime GenAI 0.15.1 through the runtime package.	Native libraries come from Microsoft.AI.Foundry.Local.Runtime.
JavaScript foundry-local-sdk	adm-zip ^0.6.0; node-addon-api ^8.2.2	The npm install script downloads Microsoft.ML.OnnxRuntime 1.28.0 and Microsoft.ML.OnnxRuntimeGenAI.Foundry 0.15.1 for the active RID.	The Node addon and foundry_local library are shipped under prebuilds/. Windows artifacts also include the reg-free WinML runtime.
Python foundry-local-sdk	cffi>=1.16; typing_extensions>=4.5; pydantic>=2.0.0; requests>=2.32.4; openai>=2.24.0; onnxruntime==1.28.0; onnxruntime-genai-core==0.15.1	No NuGet downloads. Pip installs onnxruntime and onnxruntime-genai-core on every RID.	The wheel includes foundry_local. Windows wheels also include the reg-free WinML runtime.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview	Aug 4, 2026 3:54pm

[Copilot]

Pull request overview

Updates ONNX Runtime dependencies and standardizes native package consumption on the CPU package.

Changes:

• Bumps ORT to 1.28.0 and ORT GenAI to 0.15.0.
• Replaces Foundry/GPU-specific ORT packages with Microsoft.ML.OnnxRuntime.
• Updates pipeline prefetching, packaging, and documentation.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
sdk_v2/js/script/install-native.cjs	Uses the standard ORT package.
sdk_v2/deps_versions.json	Updates shared dependency versions.
sdk_v2/cpp/nuget/pack.py	Updates ORT package help text.
sdk_v2/cpp/nuget/Microsoft.AI.Foundry.Local.Runtime.nuspec	Changes the runtime dependency.
sdk_v2/cpp/cmake/FindOnnxRuntime.cmake	Simplifies standard ORT acquisition.
.pipelines/v2/templates/steps-prefetch-nuget.yml	Updates package prefetching.
.pipelines/v2/templates/steps-build-windows.yml	Updates parameter documentation.
.pipelines/v2/templates/steps-build-linux.yml	Removes GPU-package prefetching.
.pipelines/v2/sdk_v2-pipeline-plan.md	Documents CPU-only Linux builds.
.pipelines/v2/sdk_v2-js-pipeline-plan.md	Documents the standard ORT package.
.pipelines/foundry-local-packaging.yml	Synchronizes pipeline version pins.

Comments suppressed due to low confidence (1)

.pipelines/v2/templates/steps-prefetch-nuget.yml:132

• After removing the GPU package, this list contains only two packages, so “All four packages” is now incorrect in the bash branch as well.

      declare -a entries=(
        "genai:Microsoft.ML.OnnxRuntimeGenAI.Foundry:${{ parameters.genaiVersion }}"
        "ort:Microsoft.ML.OnnxRuntime:${{ parameters.ortVersion }}"
      )

[Copilot]

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

sdk_v2/cpp/src/inferencing/generative/tokenizer.h:20

• The mutex is the only protection against the shared tokenizer's documented non-reentrant encode path, but no added test performs concurrent encodes through one model/tokenizer. Without such a regression test, removing or narrowing this lock would leave all updated tests green while restoring the intermittent corruption this wrapper is intended to prevent. Add a focused multi-threaded test that repeatedly encodes on the same shared tokenizer and validates completion/results.

/// A single tokenizer is created per model and shared across all concurrent sessions of that model
/// (see ModelLoadManager). The underlying ort-extensions BPE encode path is not reentrant (it mutates
/// shared pre-tokenizer state), so Encode/ApplyChatTemplate are serialized internally. Callers use this
/// type exactly as they would a plain tokenizer and do not need to know that access is synchronized.

sdk_v2/cpp/test/sdk_api/chat_session_test.cc:218

• Making every arithmetic expectation non-fatal means the test can pass when delayed history commit is broken: a model given each follow-up without prior context can still return deterministic non-empty text, increment TurnCount, and satisfy the turn-2/turn-3 equality check. Keep at least one hard, ISA-robust assertion whose answer depends on earlier context (for example, use a prompt with a wider output-token margin as the comment itself suggests).

  EXPECT_NE(r2.GetFinishReason(), FOUNDRY_LOCAL_FINISH_NONE);
  EXPECT_NE(r2.GetFinishReason(), FOUNDRY_LOCAL_FINISH_ERROR);
  expect_contains_soft(t2, "5", "Turn 2");

sdk_v2/cpp/test/sdk_api/responses_test.cc:498

• This test no longer proves that the streaming turn's context is available to the following non-streaming turn. previous_response_id is copied into response metadata and a fresh session can still produce non-empty output, so the assertions pass even if streaming session check-in/history chaining is broken. Retain a robust content-dependent assertion for the second turn (the removed password check or an equally deterministic prompt).

  ASSERT_TRUE(second_response.contains("output_text"));
  EXPECT_FALSE(second_response["output_text"].get<std::string>().empty()) << "Expected non-empty output_text";

[baijumeswani]

This PR fixes another issue uncovered when upgrading to onnxruntime-genai 0.15.1:

Fix: Windows heap corruption from concurrent tokenizer encode across parallel sessions

Symptom

Intermittent, Windows-only native crashes in the sdk_v2 C# integration tests:  0xC0000005  access violations and  invalid string_view position  errors during model inference. Failures were flaky (only a handful of tests, non-deterministic) and never reproduced on Linux. CI failed sporadically on the Windows leg only.

Root cause

The ORT GenAI 0.15.1 tokenizer's encode path is not reentrant. After ort-extensions #1068,  KernelBpeTokenizer::Tokenize / SpmTokenize  mutate shared pre-tokenizer state ( cached_splitters_ , whose  reg_splitter  holds a  std::u32string_view  into a member string). Two threads encoding on the same tokenizer instance stomp on that shared buffer.

Foundry's  ModelLoadManager  caches one  OgaTokenizer  per model and shares it across all concurrent sessions of that model. When multiple sessions ran inference in parallel (exactly what the C# test suite does), their  Encode  /  ApplyChatTemplate  calls raced on that single shared tokenizer, corrupting the native heap. Linux happened not to trip the corruption, which is why it presented as Windows-only.

Scope of the race — what is and isn't affected

• Affected (shared, mutable, non-reentrant): the encode path.  Encode  and  ApplyChatTemplate  both mutate the shared BPE pre-tokenizer state. These were the only unsafe operations.
• Not affected: the decode path. Each session/generator creates its own  OgaTokenizerStream  with its own detokenizer  cache_ .  TokenizerStream::Decode  routes through  OrtxDetokenizeCached  — a separate ort-extensions code path that does not touch  cached_splitters_ . Per-session streams are never shared, so decode needs no locking.
• Not in use: non-streaming  Tokenizer::Decode  /  DecodeBatch . These genai APIs would read the shared tokenizer directly, but foundry never calls them. (Flagged for future reviewers: introducing a direct non-streaming decode on the shared tokenizer would need the same synchronization consideration.)
• Unaffected:  GetEosTokenIds()  reads immutable model metadata, not the mutable encode state.

Fix

Introduced an  fl::Tokenizer  wrapper ( tokenizer.h  /  tokenizer.cc ) that owns the  OgaTokenizer  plus a  std::mutex  and serializes the encode path internally:

•  Encode()  and  ApplyChatTemplate()  take the lock (both mutate the shared BPE state, so they share one mutex).
•  Oga()  exposes the raw tokenizer unlocked, used only for the two safe operations: creating per-session decode streams and reading immutable EOS ids.

 GenAIModelInstance  now holds an  fl::Tokenizer  and exposes it via  model.Tokenizer() . Callers use  model.Tokenizer().Encode(...)  /  .ApplyChatTemplate(...)  and do not need to know the access is synchronized — the thread-safety is an implementation detail of the wrapper. The decode-only special-token tokenizer (tool-call detection) is left as a raw  OgaTokenizer , since it is only ever used to seed per-session streams.

This keeps genai 0.15.1 (no dependency bump required); the corruption is fixed on the foundry side by removing the concurrent-encode data race.

Validation

• C++ unit tests:  ChatTemplateTest  8/8 pass.
• C# integration suite: 102/102 pass, run repeatedly in parallel with zero heap corruption (previously intermittent).
• Code review: all encode call sites confirmed routed through the lock; all  Oga()  (unlocked) call sites confirmed to be decode/immutable-only; destruction order and exception safety verified.

[baijumeswani]

In addition, a few non-deterministic tests have been re-rewritten to test the scenario it was supposed to rather than check for substrings in the reponses.