ONVIF AI Service

[ocampana-videotec]

Initial draft pull request to gather feedback on the AI specs being written right now.

Current status

• AI.xml is been written
• Proposal to add OAuth 2.1 in Security.xml

Tentative scheduling

• First complete draft circulated with ONVIF members and test tool vendor by the F2F meeting in Prague
• First hackathon at the plugfest in Mexico City

Topics to be included

• Transport (MCP)
• Security (OAuth 2.1)
• Minimum set of data and tools common to all the AI-native devices (potential candidate: Axis' proposal of observations)

---

I expect this PR to be used as a shared notepad about the specs begin developed. Once ready, this will be closed a new a formal PR will be created.

[sujithhanwha]

Image is missing.

[willysagefalk]

PR specifies a persistent HTTP GET /mcp carrying newline-delimited notifications. MCP 2026-07-28 explicitly removed the GET stream. Requests use POST; long-lived notifications use the SSE response to subscriptions/listen. Independent notifications/message events cannot simply be placed on a permanent GET stream. Official MCP Streamable HTTP specification.

[willysagefalk]

server/discover is incorrectly mandatory as the first call.

A server must support it, but a modern client may call any operation directly and handle UnsupportedProtocolVersionError. The PR currently says the client shall call it first.

[ocampana-videotec]

server/discover is incorrectly mandatory as the first call.

A server must support it, but a modern client may call any operation directly and handle UnsupportedProtocolVersionError. The PR currently says the client shall call it first.

@willysagefalk Do you have a reference for that?

[willysagefalk]

@ocampana-videotec Yes. The normative reference is the MCP 2026-07-28 specification, Discovery — When to Call:

https://modelcontextprotocol.io/specification/2026-07-28/server/discover#when-to-call

It states that calling server/discover is optional for clients: a client may invoke any RPC directly and handle UnsupportedProtocolVersionError if the requested version is unsupported. The requirement is that servers MUST implement server/discover.

The same section only recommends calling it first for a dual-era client over stdio, as a backward-compatibility probe. Since the ONVIF proposal uses HTTP POST, that stdio-specific recommendation does not apply.

I therefore suggest replacing:

The ONVIF Client shall call server/discover as the first request...

with something along these lines:

The ONVIF Device shall support server/discover. An ONVIF Client may call server/discover before other requests to retrieve the supported protocol versions and capabilities.

[ocampana-videotec]

@ocampana-videotec Yes. The normative reference is the MCP 2026-07-28 specification, Discovery — When to Call:

Corrected.

[ocampana-videotec]

@willysagefalk , following our conversation, I just added chapter 9 about metadata.

As you can see it defines abstract concepts and for Observations and Provenance it proposes two specialization. SOSA/SSN was included to accommodate @DammnAI 's feedback in https://github.com/onvif/wg_AI/pull/9 , while the JLDS-based provenance is my proposal based on an already existing specification.

If any member wants to model an Observation, they can simply propose a similar derivation.

[willysagefalk]

Thanks for adding the metadata chapter. I suggest that we now use this PR as the working integration point and complete the model here. I reviewed head a7d7d490.

The following changes are needed before the draft can become an implementable specification.

Blocking implementation issues

1. Replace the persistent GET /mcp data channel

   The draft still describes notifications/message over a permanent GET /mcp connection. In MCP 2026-07-28, each Streamable HTTP message is sent by POST and a long-lived notification stream is opened by a subscriptions/listen request; that operation replaced the former HTTP GET stream.

   Please define the metadata delivery flow using subscriptions/listen, including the requested/acknowledged notification filter, subscription identifier, cancellation/reconnection behavior, and the exact notification type. If ONVIF needs notifications that carry complete observation payloads rather than standard resource-change notifications, this must be defined as an explicit, negotiated ONVIF extension with its own schema.

   References:

   • https://modelcontextprotocol.io/specification/2026-07-28/basic/transports
   • https://modelcontextprotocol.io/specification/2026-07-28/basic/patterns/subscriptions

2. Separate JSON-LD context from validation-schema identification

   The text currently reuses the LinkML schema id as the JSON-LD @context. These artifacts have different functions: @context maps JSON terms to IRIs, while a schema identifier selects the LinkML/JSON Schema used for structural validation.

   Please define separate, versioned identifiers, for example @context, schema, and, if needed, semanticContract. Retrieval should be through authenticated MCP resources or a trusted ONVIF registry, with immutable versions and integrity binding rather than fetching an arbitrary URI supplied by a payload.

   Reference: https://www.w3.org/TR/json-ld11/

3. Use a named Data Integrity cryptosuite exactly as specified

   The current proofValue description defines H(canonical(unsecuredDocument)) || H(canonical(proofConfig)). For ecdsa-rdfc-2019, W3C defines the reverse order: proofConfigHash || transformedDocumentHash.

   Please select a concrete cryptosuite and reference its complete transformation, hashing, serialization, verification-method, and proof-value rules rather than constructing a generic algorithm from the Security Baseline. Add at least one signing and verification test vector.

   Reference: https://www.w3.org/TR/vc-di-ecdsa/#hashing-ecdsa-rdfc-2019

4. Repair yaml/ai.yaml

   The file currently identifies itself as the Recording model:

   id: https://w3id.org/onvif/schema/recording
   name: onvif_recording_model

   It also uses tai:Capabilities without declaring tai. Please give the AI model its own namespace/name, declare every prefix, and validate imports and references. This should be covered by schema-generation/validation CI.

Information-model changes requiring WG agreement

5. Do not use Observation as both the evidence concept and the top-level payload container

   The draft defines Observation as “the top-level abstract container for every piece of information.” That makes an observation simultaneously an acquisition of evidence, a transport envelope, and—when AI inference is involved—a claim about the world. These roles have different identity, provenance, confidence, and lifecycle.

   I suggest defining the core boundary explicitly:

   • InformationEnvelope: serialization container for one or more information entities.
   • Observation: acquisition of evidence, with source/procedure, phenomenon time, result time, and observation location.
   • Statement: a typed proposition about a subject.
   • Assertion: an agent's attributed claim that a Statement is valid, with assertion time, confidence, provenance/authenticity, and supporting observations.
   • PhysicalObject: stable reference to the real-world subject.

   This matters for RE-ID: detections/tracks are observations; “Track A has the same identity as Track B” is a statement; the RE-ID service's confidence-bearing output is an assertion supported by those observations.

   SOSA can remain a concrete observation specialization or mapping, while the ONVIF model also supports inference, fusion, verification, and human observations without forcing all of them into a sensor-only pattern.

6. Replace SOSAObservation.hasResult: Any with testable result types

   Any prevents generated clients, schema validation, and the ONVIF test tool from determining what is conformant. Define a typed Result hierarchy/union. A direct measurement may use a typed value result; a derived conclusion should reference a typed Assertion/Statement structure.

7. Separate observation location from source coverage

   Where an observation applies is not the same as the area a source can observe. Please model these separately and use structured references for coordinate frame, geometry/spatial region, units, transforms, and topology. Avoid unconstrained zone or coordinate strings.

8. Add end-to-end normative examples and conformance points

   Please include at least:

   • one direct sensor observation;
   • one derived AI assertion;
   • one RE-ID assertion supported by two observations;
   • one signed JSON-LD example;
   • one subscriptions/listen delivery sequence.

   For each SHALL/MUST, identify the responsible role, observable pass/fail behavior, capability discovery, unsupported behavior/fault, and expected test-tool coverage.

One editorial cleanup: the PR description should describe the minimum information model in vendor-neutral ONVIF terms.

The abstract-class and specialization direction is useful. The main work now is to make the boundaries explicit and ensure the transport, JSON-LD, schema, and signature mechanisms can be implemented and tested consistently.

[ocampana-videotec]

Sorry @willysagefalk but we need to find a more efficient way to tackle the document, we cannot just copy&paste tons of lines from an LLM. For the future, we need IMHO to be shorter a more focused.

Point 1: I will study it, it looks correct.

Point 2: I have no idea right now. It's generated by the linkml tools...

Point 3: I intentionally got rid of ecdsa-rdfc-2019 in order to point our specs to the ONVIF Security Baseline.

Point 4: The yaml file is broken, I know it very weak, there is even a note in the AI.xml that that part will have to be removed and be put in the ai.yaml. We are simply too far now to allocate time to it.

Point 5: what is the problem here? beyond a style issue, I really see no problem for the observation to be to top level container.

Point 6: how is it defined in SOSA? We do not want to diverge. If this is not good enough, then we have a reason to ditch SOSA

Point 7: this looks like point 5. What is the use case we are trying to solve?

Point 8: I agree and in fact we are starting to work on Annex A. The problem is that before creating an example we must clarify how it will work.

Therefore, the immediately actionable items are: 1, 3 and 6.

[willysagefalk]

I suggest extending the model with a neutral Assertion type alongside Observation, together with an ObjectObservation specialization:

InformationEntity
├── Observation
│ ├── ObjectObservation
│ └── SOSAObservation
└── Assertion
├── Statement
└── Claim

ObjectObservation represents an observation concerning one persistent PhysicalObject. It groups the Claims produced within that observation context and may include observed positions or other object-related information.

Assertion is not an Observation. It expresses a proposition, relationship or conclusion and can reference Observations as supporting evidence. Statement and Claim are concrete specializations of Assertion.

For example, in RE-ID:

ObjectObservation A
└── contains Claims about Object/Track A

ObjectObservation B
└── contains Claims about Object/Track B

Re-identification Claim
├── proposition: A and B represent the same PhysicalObject
├── supportedBy: ObjectObservation A, ObjectObservation B
├── producedBy: RE-ID service
└── confidence: 0.82

SOSAObservation remains available for SOSA interoperability, but it does not need to be the foundation of the ONVIF information model. ObjectObservation and SOSAObservation describe different aspects, so a sensor-based ObjectObservation may also be mapped to SOSA.

[ocampana-videotec]

@willysagefalk let's start clarifying the definition of the abstract classes involved in metadata streaming. Once we clarify them, we can specialize stuff like ObjectObservation, PhysicaObject and so on.

My definitions:

• Observation: is the top-level abstract container for every piece of information emitted by an ONVIF AI Device
• Assertion: I do not have a definition yet
• Claim: I do not have a definition yet
• Statement: I do not have a definition yet

I do have a few ideas I am working for for statement, but I do not have a clear understanding of the difference between Claim and Assertion. My idea is that, if we are talking about abstract classes, assertions, claims and statements are the same thing.

If that's the case we could have

Observation
+- timing
+- location
+- provenance
+- assertion (or how we are going to call it)

To my eyes, InformationEntity is an empy container, it's just a waste of bandwidth. We do not need to wrap a rich container (Observation) with an empty conatiner (InformationEntitiy)

From assertion we could derive a statement, and for the statement use the idea of the predicate we were discussing at the Budapest meeting.

[willysagefalk]

InformationEntity is not intended as a serialized wrapper around an Observation, so it adds no bandwidth. It is only the neutral abstract superclass that allows both Observations and Assertions to be top-level information types.

An Observation represents an observation/perception context and may contain Assertions produced during that observation. An Assertion represents a proposition or conclusion and may reference one or more Observations as evidence. For example, a RE-ID Assertion that two observations concern the same physical object cannot naturally belong to only one of those observations.

I agree that we should first settle these two abstract definitions. We can then clarify the precise distinction between Statement and Claim and add specializations such as ObjectObservation.

[ocampana-videotec]

I am still having difficulties to understand clearly the difference between an Assertion and an Observation

I am good with "An Observation represents an observation/perception context and may contain Assertions produced during that observation". But then I see a logical problem in "An Assertion represents a proposition or conclusion and $\color{red}{\text{may}}$ reference one or more Observations as evidence".

Why may? Observations may exist without Assertions, but in my understanding Assertions cannot exist outside of an Observation. It is a must, not a may in my opinion.

Assertions without observation are, de facto, Observations.

Do you agree on this?

[willysagefalk]

I do not fully agree. The key is to separate the statement from the process or context that produced it.

An Observation is an event/context: something was perceived at a particular time and place.
An Assertion is a proposition: something claims that a fact is true.

There are then two different relationships:

Produced during an Observation
A direct perception assertion may belong to an Observation:
“Object 17 is a person.”
Uses Observations as evidence
A derived assertion may reference earlier Observations:
“The person in Observation A is the same person as in Observation B.”

The second relationship should remain may, because assertions can also come from sources that are not observations:

“Camera 1 is installed in Room 5” — configuration assertion.
“Zone A is restricted” — semantic or policy assertion.
“Object X and Object Y are the same person” — inference based on observations.
“The operator identified this person as John” — human assertion.

These are propositions, but they are not themselves observations. Calling them Observations would collapse the distinction between perception, inference, configuration, and human knowledge.

The current wording could nevertheless be clearer:

An Observation represents a perception context and may contain Assertions produced during that Observation. An Assertion represents a proposition or conclusion. An Assertion may be produced during an Observation and may reference one or more Observations or other information entities as evidence.