[api][integrations] Support models' native structured output (foundation + OpenAI)

[weiqingy]

Linked issue: #280

Purpose of change

Today output_schema is honored only by prompt-engineering and parsing the response text; no integration uses a provider's native structured-output API. This PR adds the foundation for native structured output at the chat-model connection layer, plus the OpenAI implementation, in Java and Python. It's the first in a small stack under #280 (Azure/Ollama, Anthropic, DashScope follow; ReActAgent final-output wiring is a separate follow-up).

How it works:

• The output schema is carried to the connection via a reserved key (__structured_output_schema__) in the existing modelParams/kwargs map, so the abstract chat() signature is unchanged.
• Each connection declares a boolean capability (supportsNativeStructuredOutput() / supports_native_structured_output), default false.
• A connection applies the native API only when a schema is present, no tools are bound, the schema is a POJO/BaseModel (not RowTypeInfo), and the setup is same-language. The key is always stripped before the SDK call so it can't leak into a request.
• The prompt path is kept as the fallback. In the ReAct loop tools are always bound, so the native path stays dormant and existing behavior is unchanged.

OpenAI applies response_format json_schema strict. Other connections only strip the reserved key for now. The same-language guard avoids marshaling a schema object across the Pemja bridge, where native structured output can't work anyway (a Java Class is not a Python BaseModel).

Tests

Unit tests with the SDK mocked (no network): native applied with schema and no tools (Java + Python); not applied when tools are bound or for RowTypeInfo; the reserved key never reaches a provider SDK; the same-language threading guard; and existing ReActAgent prompt-path tests remain green.

API

Yes — additive only. BaseChatModelConnection gains a public reserved-key constant and a protected capability method (default false); no existing signatures change.

Documentation

• doc-needed
• doc-not-needed
• doc-included

[weiqingy]

The failing check was unrelated to this PR — it was a transient CI infra failure:

• it-java [ubuntu-latest] [java-17] [flink-2.3]: The Install java setup step failed before any test ran — actions/setup-java@v4 got an HTTP 403 downloading the Temurin JDK 17 tarball from Adoptium's release server, so the actual IT steps were skipped. The other it-java matrix legs (flink-2.0/2.1/2.2) passed in the same run. Evidence: failed job.

Re-triggered CI (empty commit) to clear it — all checks are green now.

[GreatEugenius]

Hi @weiqingy, thanks for the PR. Overall, the approach looks good.

One thought on the strip: right now every non-native connection has to pop the key itself (the 4 non-OpenAI Python ones do it purely to avoid leaking; the Java ones don't strip at all and are only safe incidentally). Since both languages already funnel through BaseChatModelSetup.chat() before connection.chat(...), could we strip there based on the flag?

# Python
if not connection.supports_native_structured_output:
    merged_kwargs.pop(STRUCTURED_OUTPUT_SCHEMA_KEY, None)

// Java
if (!connection.supportsNativeStructuredOutput()) {
    params.remove(BaseChatModelConnection.STRUCTURED_OUTPUT_SCHEMA_KEY);
}

That removes the 4 leak-only pops on the Python side, closes the Java asymmetry, and gives the flag a real caller on both. Native connections would still pop themselves (to discard when tools are bound). Not blocking.

[weiqingy]

Thanks for the review, @GreatEugenius — this is a nice simplification. Done in f35e432.

[GreatEugenius]

LGTM. The centralized strip in BaseChatModelSetup.chat reads cleanly, and I believe this lays a solid foundation for the follow-up wiring (ReActAgent final-output). Thanks @weiqingy for the quick turnaround — looking forward to the rest of the stack! @wenjin272, could you please take a look?

[weiqingy]

Thanks, @GreatEugenius!

@wenjin272, could you take a look when you get a chance?