adk-telemetry: attribute Connect + FlightCheck by connector (ADO 7943641)

[amilandi]

Summary

Adds a bounded connector dimension to every Connect capability event and every FlightCheck event so PMs can report Workday vs ServiceNow adoption and reliability separately, instead of one generic connect wedge.

Closes ADO 7943641.

Taxonomy

connector ∈ { "workday", "servicenow", "legacy", "unknown", "" }

value	meaning
workday / servicenow	attributed to that backend
legacy	caller-supplied sentinel for pre-1.4.0 connect events surfaced retroactively
unknown	out-of-taxonomy value passed by a caller (should be zero — investigate)
"" (empty)	event is not connector-scoped (topic authoring, workflow deletion, cross-cutting FlightCheck runs / categories)

Wiring

Connect (emit_capability.py shim): new --connector <workday|servicenow|legacy> flag; parsed positional-free (before or after the capability, = form also supported). Applied to worker mode, sync mode, and the detached subprocess round-trip. Skill wiring moved from connect/SKILL.md's top-of-file (which fired before the maker picked WD vs SN) into connect/step1.md §1.3 branches so the attribution is known when the emit fires.

FlightCheck (derived on-emit):

• Run event: derive_connector_from_scope(scope) — workday* / servicenow scopes attribute; full and cross-cutting scopes emit empty (per-check attribution provides the finer split).
• Check event: derive_connector_from_category(category) — prefix-match on Workday* / ServiceNow*; cross-cutting categories (Environment, Authentication, Prerequisites, etc.) emit empty.

Schema version bumps

• adk_telemetry.SCHEMA_VERSION 1.3.0 → 1.4.0
• flightcheck.telemetry.TELEMETRY_SCHEMA_VERSION 1.1 → 1.2

Additive change — old cubes / queries continue to work; the new dim simply appears with empty values for events that pre-date the merge.

Dashboards

None in this PR (per PM ask). Once events land in Kusto with the new dim, we can add per-connector tiles to the FlightCheck + ADK Aria dashboards without a code change.

Docs

• scripts/telemetry_queries.kql §9: connector-split adoption + reliability queries (Connect starts by connector, FlightCheck runs & per-check reliability, legacy → attributed rollout tracker).
• emit_capability.py --help documents the new flag.

Tests

38 new assertions across tests/test_adk_telemetry.py and tests/flightcheck/test_telemetry.py:

• normalize_connector taxonomy (known / empty / legacy / unknown / case+whitespace).
• Connector round-trips through every emit (emit_capability_use, emit_flightcheck_run/result/error).
• Shim --connector plumbing (positional-before, positional-after, = form, dangling flag, worker mode round-trip, no-flag → empty).
• FlightCheck derivation from scope + category, including cross-cutting → empty.
• Envelope-level assertion that check events carry the derived connector across a mixed WD + SN + cross-cutting run.
• Extends the "wired capabilities" reverse scanner to include step*.md (not just SKILL.md) so deferred emits like the new connect wiring stay visible to the dead-capability guard.

Full suite (tests/test_adk_telemetry.py + tests/flightcheck/test_telemetry.py): 224 passed.

Not in scope

• Dashboards (deferred; see above).
• emit_api_call connector stamping — not in the AC. Some API calls are Workday/ServiceNow-specific; happy to do a follow-up if PM wants Connect-flow API attribution too.
• main-ca cherry-pick of this change to keep CA-mode kits attribution-consistent.

---

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com
Copilot-Session: 70ddf006-7f8d-48e7-9afa-3fbae73b3864

[nkemms]

Local review findings (full first-pass threshold):

1. ADK FlightCheck connector attribution is not wired through the production callers. The new connector parameters on emit_flightcheck_run() and emit_flightcheck_result() are covered by direct helper tests, but the actual checkpoint and normal-scope paths in flightcheck/cli.py still call them without a connector. As a result, the ADK FlightCheck event family remains unattributed in real runs even though the separate ESSMakerKit.FlightCheck.* events derive connector correctly. Please derive the connector in the CLI runtime path and pass it to both ADK emitters, with a caller-level test.

2. Run-level Workday derivation omits real Workday scopes. _WORKDAY_SCOPES recognizes workday, workdaytenant, and workdayextension, while SCOPE_MAP also defines workdayda and topics (Workday Topics). Runs using either omitted scope emit an empty connector. Please include both scopes and add coverage for them.

3. KQL query 9a uses the wrong tenant field for ADK events. adk_capability_use dimensions use the ADK snake_case field tenant_id, but the new query calculates dcount(tenantId). tenantId belongs to the legacy ESSMakerKit event shape, so the ADK tenant count will be incorrect. Please change the ADK query to tenant_id.

4. The rollout query does not accurately distinguish unattributed, legacy, unknown, and attributed events. Omitted connector values remain empty (and historical rows may have a missing/null field); they are not automatically emitted as legacy. Query 9d checks only connector == "", counts unknown as attributed, and describes legacy behavior that occurs only when that value is explicitly supplied. Please use null-safe/isempty() handling and report unknown separately, or simplify the rollout categories so their labels match the emitted data.

5. The detached parent-process connector forwarding lacks direct coverage. Worker-mode parsing/forwarding is tested, but the parent test exercises only the no-connector path and does not assert that --connector <value> is included in the spawned Popen argv. Please add a parent-path assertion for the connector form so the full async boundary is protected.

Targeted telemetry tests passed locally: 224 passed. The branch also merged cleanly with the current target. Open sibling conflicts appear to be merge-order coordination rather than additional defects.

[nkemms]

Re-reviewed the latest update. The five previously reported findings are addressed: production callers now forward connector attribution, all Workday scopes are covered, the ADK tenant field is corrected, rollout buckets handle missing and unknown values, and the detached parent path is tested.

The branch currently conflicts with updated main in the shared telemetry files. When resolving, preserve both this PR's connector attribution and the existing toolkit_git_sha / toolkit_git_branch dimensions.

Follow-up: single-checkpoint attribution currently derives from the first result row; prerequisite-failure sentinels can therefore leave an otherwise connector-specific run unattributed. This does not need to hold this PR.

266 targeted tests passed locally.

[nkemms]

Reviewed the latest update; the reported findings are addressed. Remaining target-branch reconciliation and attribution edge are documented as follow-up.

[amilandi]

Thanks — merge conflicts against main resolved in 7ae8506 (pushed). The merge preserves both:

• this PR's connector attribution (derived connector on adk.flightcheck.run / .result / .capability.use + on the legacy ESSMakerKit.FlightCheck.Run / .Check events), and
• the toolkit_git_sha / toolkit_git_branch common dimensions from main.

Schema versions bumped:

• adk_telemetry.SCHEMA_VERSION → 1.5.0 (added 1.5.0 note for connector; kept 1.4.0 note for toolkit_git_*).
• flightcheck.telemetry.TELEMETRY_SCHEMA_VERSION → 1.3 (added 1.3 note for connector; kept 1.2 note for toolkitGitSha / toolkitGitBranch).

Version-gate tests updated. 244 targeted tests pass locally (tests/test_adk_telemetry.py + tests/flightcheck/test_telemetry.py). Full suite failures on this branch are the same pre-existing failures on origin/main (installer/maker-profile and da-setup-router modules) — unrelated to this PR.

Follow-up on single-checkpoint prerequisite-failure attribution acknowledged and tracked; keeping it out of this PR per your note.

[nkemms]

Re-approved after the latest main merge. The conflict resolution preserves connector attribution and the existing toolkit Git dimensions; focused validation passed.