Add API-level tests for the MCP tool surface - #97
Merged
Conversation
An LLM getting a tool call wrong is a different problem from the tool being wrong. These drive POST /mcp as a real MCP client would, and read every write back — cross-checking against the REST services on the same ApiService wherever they see the same row, since a tool that reports success while writing nothing is indistinguishable from a working one on the MCP side alone. Rust rather than python_tests/ because the SDK already wires auth with refresh and TLS against the OS trust store (rustls-tls-native-roots). A Python client needed a hand-rolled token exchange and a CA-bundle probe, since a venv's certifi does not carry the dev IdP's issuer. The McpClient here is a private test helper; no new dependency, no public API, and the crate ships identically with or without the module. Coverage is measured, not declared: try_call_tool records each (tool, field) pair it sends and the sweep diffs that against the live tools/list schema — 125 fields across 37 tools. A parameter added server-side fails the audit until something drives it. Hence one sequential sweep rather than 74 tests: cargo test parallelises with no ordering hook, so a shared registry could not be read reliably at the end of any individual test. Also adds three polling range-query tests to the Python datapoint suite. They prove the point exists through a second read path first, so an empty range result cannot be excused as eventual consistency — which is what caught /timeseries/data/list answering 200 with a zero-byte body. mcp_event_update_by_uuid_reindexes_the_external_id is red on purpose: renaming an event by UUID writes the new externalId but never reindexes it, so the event stays reachable only under the old one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
olavgg
approved these changes
Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An LLM getting a tool call wrong is a different problem from the tool being wrong. These drive POST /mcp as a real MCP client would, and read every write back — cross-checking against the REST services on the same ApiService wherever they see the same row, since a tool that reports success while writing nothing is indistinguishable from a working one on the MCP side alone.
Rust rather than python_tests/ because the SDK already wires auth with refresh and TLS against the OS trust store (rustls-tls-native-roots). A Python client needed a hand-rolled token exchange and a CA-bundle probe, since a venv's certifi does not carry the dev IdP's issuer. The McpClient here is a private test helper; no new dependency, no public API, and the crate ships identically with or without the module.
Coverage is measured, not declared: try_call_tool records each (tool, field) pair it sends and the sweep diffs that against the live tools/list schema — 125 fields across 37 tools. A parameter added server-side fails the audit until something drives it. Hence one sequential sweep rather than 74 tests: cargo test parallelises with no ordering hook, so a shared registry could not be read reliably at the end of any individual test.
Also adds three polling range-query tests to the Python datapoint suite. They prove the point exists through a second read path first, so an empty range result cannot be excused as eventual consistency — which is what caught /timeseries/data/list answering 200 with a zero-byte body.
mcp_event_update_by_uuid_reindexes_the_external_id is red on purpose: renaming an event by UUID writes the new externalId but never reindexes it, so the event stays reachable only under the old one.