flightcheck: re-point WD-REST-001 + WD-ENV-001 to Declarative Agent components API - #317
Open
Dawn Jeong (daeunJe0ng) wants to merge 12 commits into
Open
Dawn Jeong (daeunJe0ng) wants to merge 12 commits into
Dawn Jeong (daeunJe0ng) wants to merge 12 commits into
Conversation
Dawn Jeong (daeunJe0ng)
marked this pull request as ready for review
September 23, 2026 05:26
…test (AB#7852506, AB#7852495, AB#7852511) Extract the shared minimalBots components connection-reference reader into checks/_da_connection_refs.py as the single source that DV-CONN-001 (active agent), ENV-004 (environment-wide, de-duped by logical name) and the Workday shared-parameter checks (WD-ENV-001/WD-REST-001) all read through, so the per-check re-point PRs stop adding divergent copies of it. The module is the superset of PR microsoft#317's reader plus a public read_all_agents_connection_references() that de-dupes by connection-reference logical name for ENV-004. Brings the validated/documented mock builders the checks need and a direct pure-logic contract test (18 cases) covering normalization, None-gating (SKIP), fail-loudly ValueError, JSON-string sharedConnectionParameters parsing, and the Workday shared-parameter sweep. Foundation for the 6 DA re-point PRs to rebase onto (single shared reader, no add/add collision). Full flightcheck suite: 1142 passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e25cf992-09c9-49b8-9971-2a19cb2fcb05
… (7852506) Read the Workday SOAP connection reference from the Declarative Agent minimalBots components API (runner.agentbuilder.fetch_components) instead of the Custom Agent Dataverse connectionreferences query, so DV-CONN-001 works against DA-GA agents that no longer expose Dataverse. Verdict simplifies to found + bound (connectionId present); the Dataverse-only statuscode and multiple-ref branches are dropped since the components shape carries neither. BAP owner echo is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Distinguish a genuinely-absent connectionReferenceChanges (missing key -> no references -> FAILED not-found) from a present-but-non-list payload (a shape we do not understand -> raise ValueError so the dispatcher degrades DV-CONN-001 to a WARNING). Mirrors native_agent._connection_references, the shipped precedent this check re-uses; the prior code collapsed both to [] and could report a confident "reference not found" on an unparseable 200 response. Adds a malformed-changeset test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…list) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8246cd2c-37d0-4000-8fe3-fa9b082669e0
…7852511) Re-point WD-REST-001 and WD-ENV-001 to AgentBuilder components sharedConnectionParameters so Workday REST and token configuration no longer depend on Dataverse/minimalbots legacy reads. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8246cd2c-37d0-4000-8fe3-fa9b082669e0
Dawn Jeong (daeunJe0ng)
force-pushed
the
users/dawnjeong/flightcheck-da-wd-scp-7852511
branch
from
September 24, 2026 00:16
c04f2cd to
c1bd684
Compare
…selection (WD-REST-001/WD-ENV-001) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8246cd2c-37d0-4000-8fe3-fa9b082669e0
…rams sweep across all agents (AB#7852511)
The shared _da_connection_refs reader iterated every configured agent, so
DV-CONN-001 (a HIGH-priority "Workday SOAP connection reference bound to a
connection you own" check) could PASS/FAIL on a Workday reference belonging
to a non-active agent in a multi-agent config - a false verdict on the agent
under check.
Split the reader in two:
* read_active_agent_connection_references now returns only the active agent's
references (config agent.botId), matching DV-CONN-001's intent and its
function name.
* _all_agents_connection_references keeps the every-agent behavior the
Workday shared-parameter sweep (WD-REST-001 / WD-ENV-001) needs.
Row shape is unchanged (still carries botid + sharedconnectionparameters), so
no caller changes. Single-agent configs are unaffected. Added a multi-agent
regression test: the active agent has no Workday reference while another
configured agent does; DV-CONN-001 must FAIL "not found" rather than pass on
the other agent's binding.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e25cf992-09c9-49b8-9971-2a19cb2fcb05
…nents reader (WD-REST-001/WD-ENV-001) Live AgentBuilder returns connectionReference.sharedConnectionParameters as a JSON string, not a nested object. The reader required a dict and raised ValueError on the string shape, degrading WD-REST-001 and WD-ENV-001 to WARNING instead of a clean PASS against a real Workday-bound agent. json.loads the string before validating, and add a live-shape test plus mock helper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e25cf992-09c9-49b8-9971-2a19cb2fcb05
This was referenced Sep 24, 2026
Dawn Jeong (daeunJe0ng)
added a commit
that referenced
this pull request
Sep 24, 2026
…test (#326) Extract the shared minimalBots components connection-reference reader into checks/_da_connection_refs.py as the single source that DV-CONN-001 (active agent), ENV-004 (environment-wide, de-duped by logical name) and the Workday shared-parameter checks (WD-ENV-001/WD-REST-001) all read through, so the per-check re-point PRs stop adding divergent copies of it. The module is the superset of PR #317's reader plus a public read_all_agents_connection_references() that de-dupes by connection-reference logical name for ENV-004. Brings the validated/documented mock builders the checks need and a direct pure-logic contract test (18 cases) covering normalization, None-gating (SKIP), fail-loudly ValueError, JSON-string sharedConnectionParameters parsing, and the Workday shared-parameter sweep. Foundation for the 6 DA re-point PRs to rebase onto (single shared reader, no add/add collision). Full flightcheck suite: 1142 passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e25cf992-09c9-49b8-9971-2a19cb2fcb05
…ghtcheck-da-wd-scp-7852511 # Conflicts: # solutions/ess-maker-skills/scripts/flightcheck/checks/workday.py # solutions/ess-maker-skills/scripts/flightcheck/checks/workday_extension.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Re-points WD-REST-001 (AB#7852511) and WD-ENV-001 (AB#7852509) to the Declarative Agent components API.
This PR stacks on the shared reader in #326 and on #304. Merge order: #326 -> #304 -> this PR. Until they merge, GitHub shows their commits in this diff.
What changed
flightcheck/checks/_da_connection_refs.pyreader (provided by flightcheck: add canonical DA connection-reference reader + contract test #326) for Declarative Agent connection references.read_active_agent_connection_referencesreturns only the active agent's references (configagent.botId). DV-CONN-001 consumes this._all_agents_connection_referencesreturns every configured agent's references. The Workday shared-parameter sweep (WD-REST-001 / WD-ENV-001) consumes this, because it must inspect each configured agent's own Workday reference.sharedConnectionParameters.values.restBaseUrifrom the Workday connection reference in the components payload.sharedConnectionParameters.valueskeys:tenantNametoken:ResourceUritoken:WorkdayTokenUritoken:WorkdayClientIdAGENTBUILDERand do not require a Dataverse endpoint.sharedConnectionParametersbuilder: wrapper shape fromtools/ess-ca-to-da/reference/hr/agent.yml, Workday keys from the Workday connector docs. No live components cassette captures these keys yet. Added PASS / FAIL / SKIP / WARNING coverage for both checks.Design rationale
DV-CONN-001 and ENV-004 already use
agentbuilder.AgentBuilderClient.fetch_components(agent_id)against the Declarative Agent components endpoint.This PR uses the same endpoint and shared reader pattern instead of introducing a parallel minimalBots reader. The enclosing
connectionReferenceChanges[].connectionReferenceshape is cassette-backed, but nestedsharedConnectionParameters.valuesis documented-tier only: wrapper shape from the referenceagent.yml, Workday keys from the connector docs, and no live components response captured yet. Workday PASS paths remain unverified live until a Workday-bound DA agent is available.DV-CONN-001 scoping: an earlier revision of the shared reader iterated every configured agent, so DV-CONN-001 (a HIGH-priority "Workday SOAP connection reference bound to a connection you own" check) could PASS or FAIL on a Workday reference belonging to a non-active agent in a multi-agent config - a false verdict on the agent under check. The reader now returns only the active agent's references, matching the check's intent and its function name. The Workday shared-parameter sweep keeps the every-agent behavior it needs via the separate
_all_agents_connection_references. Single-agent configs are unaffected; the change only matters for multi-agent configs.The implementation does not guess alternate API fields. If the components payload shape is malformed, the reader raises
ValueError, and the owning check reports a WARNING instead of returning a false PASS or FAIL.Testing
Run from
C:\Users\dawnjeong\worktrees\fc-pr-g2with:Results:
python -m pytest tests\flightcheck -q->1135 passed in 8.45spython -m ruff check solutions\ess-maker-skills\scripts->All checks passed!Added a multi-agent regression test in
tests/flightcheck/checks/test_workday_extension.py: the active agent has no Workday reference while another configured agent does; DV-CONN-001 must FAIL "not found" rather than pass on the other agent's binding. This pins the single-active scoping described above.Update 2026-09-23: parse JSON-string
sharedConnectionParametersLive AgentBuilder returns
connectionReference.sharedConnectionParametersas a JSON string, not a nested object. The reader required a dict and raisedValueErroron the string shape, which would degrade WD-REST-001 and WD-ENV-001 to WARNING instead of a clean PASS against a real Workday-bound agent. The reader nowjson.loadsthe string before validating the shape. Added a live-shape test (test_shared_parameters_json_string_shape_passes, WD-ENV-001 passes on the string form) and a mock helpershared_connection_parameters_json_string. Re-run:python -m pytest tests\flightcheck -q->1136 passed; ruff clean.Live DA-environment validation (partially complete)
Live run 2026-09-22 against ESS GA env
258bd301(prod ring; non-Workday agent): both checks executed against a real DA agent and readconnectionReferenceChanges. With no Workday connection reference present, WD-REST-001 returned the documentedrestBaseUri-missing FAIL with the reconnect remediation (checked below). WD-ENV-001 also FAILED and listed the missing keys, but because no Workday connref exists the 'connref present, one key missing' sub-case was not isolated, so that box stays unchecked. All Workday PASS paths need a Workday-bound DA agent (none reachable this run).To be checked once a Declarative Agent that exposes Workday
sharedConnectionParametersin its components payload is provisioned. Blocked on a published Workday-bound DA agent in the test env; these components reads use the prod-audience AgentBuilder token and do NOT depend on the TIP2 (Test-In-Production ring 2) auth issue (TIP2 only gates the separate WD-RUN-001 active probe, which is not in this PR). Each box is marked when the scenario is confirmed on a live DA env.restBaseUriis present and trimmed to/api-> PASSED.restBaseUriis missing or empty -> FAILED with reconnect-the-Workday-connection remediation.tenantName,token:ResourceUri,token:WorkdayTokenUri, andtoken:WorkdayClientIdare all present -> PASSED.connectionReferenceChanges/sharedConnectionParameters.valuesshape -> WARNING (fail loudly).Sync 2026-09-24: Merged latest
upstream/main(merge9c487083). Two conflicts, both import-block collisions inflightcheck/checks/:workday.pyandworkday_extension.py. This PR (stacked on the DV-CONN-001 re-point) imports the components reader from_da_connection_refs; main addedagent_scopehelpers +auth.query_all. Kept BOTH import groups. Inworkday_extension.pythe merged body uses the reader path, so main'squery_allimport was dead (ruff F401) and was removed with its stale comment - same supersession#314handled. Ruff clean;tests/flightcheck1220 passed. Net delta over main is ~674 lines re-pointing WD-REST-001 + WD-ENV-001 to the Declarative Agent components API - not superseded by main's independent Workday DA work.