Avoid re-resolving one-shot UI elements - #841
Merged
Zach Teutsch (zateutsch) merged 3 commits intoSep 16, 2026
Merged
Conversation
Retain the live UIA provider element inside the internal UiElement context so one-shot operations can act on the element they already resolved. Keep the serialized DTO fallback and stale-element behavior intact, and add deterministic resolution-count coverage across all affected operations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot started reviewing on behalf of
Nikola Metulev (nmetulev)
September 14, 2026 23:12
View session
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Provider errors are incorrectly reported as stale, and several optimized paths lack regression assertions.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Retains live UI Automation provider elements to avoid redundant tree resolution during one-shot operations.
Changes:
- Adds internal live-element context with serialized fallback.
- Reuses retained elements across actions, inspection, capture, and window resolution.
- Adds regression instrumentation and tests.
File summaries
| File | Description |
|---|---|
UiAutomationService.Screenshot.cs |
Reuses retained elements during cropping. |
UiAutomationService.cs |
Implements retention, fallback tracking, and reuse. |
UiAutomationService.CaptureSupport.cs |
Reuses elements for window resolution. |
UiElementContext.cs |
Stores the live provider element. |
UiElement.cs |
Adds internal element context. |
RealUiAutomationTests.Patterns.cs |
Verifies serialized fallback resolution. |
RealUiAutomationTests.Actions.cs |
Tests retained-element actions and staleness. |
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Only translate UIA element-unavailable failures to stale errors and preserve other provider COM failures. Extend zero-resolution coverage to scoped inspection, ancestor inspection, and slug screenshot cropping. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot started reviewing on behalf of
Nikola Metulev (nmetulev)
September 14, 2026 23:41
View session
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The implementation is focused and correct, and the build, six targeted regressions, all 399 UI Automation tests, and three public API tests passed.
Review details
- Files reviewed: 9/9 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Contributor
Build Metrics ReportBinary Sizes
Test Results✅ 5811 passed, 18 skipped out of 5829 tests in 892.3s (+2 tests, +48.0s vs. baseline) Test Coverage✅ 87.6% line coverage, 80.8% branch coverage · ✅ no change vs. baseline CLI Startup Time65ms median (x64, Try This BuildInstalls the MSIX for your architecture, replacing any previously installed build. Needs the GitHub CLI — the command offers to install it and sign you in if it is missing. & ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) 841Switching between builds often?Put the tool on your PATH once: & ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) -AddToPathThen this build is just: winapp-pr 841Run Updated 2026-09-15 00:33:07 UTC · commit |
This was referenced Sep 15, 2026
Zach Teutsch (zateutsch)
approved these changes
Sep 16, 2026
Zach Teutsch (zateutsch)
enabled auto-merge (squash)
September 16, 2026 19:00
Zach Teutsch (zateutsch)
deleted the
nmetulev-retain-resolved-ui-elements
branch
September 16, 2026 19:08
Nikola Metulev (nmetulev)
added a commit
that referenced
this pull request
Sep 17, 2026
## Description
Read `FontWeight`, `FontName`, `FontSize`, `ForegroundColor`,
`IsItalic`, and `StrikethroughStyle` across the element's whole
TextPattern document. Uniform values are invariant strings; UIA's mixed
and reserved-not-supported COM tokens become `Mixed` and `NotSupported`,
while elements without TextPattern return `Unavailable`. Provider
failures retain the existing error contract.
Unknown case-sensitive property names now fail with `invalid_arguments`
before querying the app. Omitting `--property` includes all six
attributes. The published `elementId` and string-valued `properties`
JSON envelope and existing geometry formats are unchanged.
## Usage Example
```powershell
winapp ui get-property Document -a myapp --property FontWeight --json
# {"elementId":"Document","properties":{"FontWeight":"700"}}
winapp ui get-property Document -a myapp --json
```
## Related Issue
Fixes #822. Independent of #841, #842, and #844. No query predicates,
explicit actions, selection/caret/range APIs, batch mode, or
persistence.
## Type of Change
- New feature
- Test update
- Documentation
## Checklist
- [x] New unit, CLI, and real-provider tests
- [x] Tested locally on Windows: 26 formatting/property library tests
and 28 CLI/property/public-API tests pass
- [x] `scripts/build-cli.ps1 -SkipTests` completes NativeAOT x64/arm64
publishing, npm/NuGet/MSIX packaging, and generated docs/schema
- [x] Canonical UI automation docs and shipped skill/reference updated
## Additional Notes
Real-app coverage reads native RichEdit uniform and mixed formatting
independently of selection, a Button without TextPattern, and an
HWND-based provider returning UIAutomationCore's actual
reserved-not-supported token through the repository COM projection.
Background fixtures assert they never activate. Unit tests cover
invariant conversion, actual COM sentinel identity, unknown names, and
provider-error propagation; CLI tests pin string serialization and
scrubbed errors.
Pre-human-review gate is pending automated review and CI; this PR is not
being merged by the implementation session.
---------
Co-authored-by: Nikola Metulev <711864+nmetulev@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Zach Teutsch <88554871+zateutsch@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #840
Retains the live UI Automation provider element in an internal, non-serialized context on
UiElement, allowing one-shot property, value, invoke, set-value, focus, scrolling, scoped inspection/capture, and element-window operations to reuse the element already found by selector resolution.Serialized or externally constructed
UiElementinstances continue through the existingResolveComElementfallback, and stale retained elements preserve the explicit stale-element error.Deterministic instrumentation verifies:
Validation:
RealUiAutomationTestspassedUiCommandTestsran: 555 passed, 1 opt-in recording test skippedscripts/build-cli.ps1 -SkipTestscompleted successfully, including NativeAOT publish and packagingThe unfiltered
scripts/build-cli.ps1was also run; compilation and packaging succeeded, while unrelated existing environment-dependent tests failed in live NuGet feed, crash-dump, and recording-output cases.