Skip to content

feat(core): allow keyboard press without a locate target - #2938

Merged
quanruzhuoxiu merged 1 commit into
mainfrom
fix/playwright-cut-shortcut
Aug 12, 2026
Merged

feat(core): allow keyboard press without a locate target#2938
quanruzhuoxiu merged 1 commit into
mainfrom
fix/playwright-cut-shortcut

Conversation

@quanruzhuoxiu

@quanruzhuoxiu quanruzhuoxiu commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • allow the recommended aiKeyboardPress signature to receive undefined as its locate argument
  • omit locating and the preliminary click when the shortcut should operate on the currently focused element
  • guide the planner to omit locate for Copy or Cut after text has already been selected
  • document the API in English and Chinese while preserving eager model configuration validation
  • cover direct Playwright and Puppeteer calls and an aiAct-planned targetless Backspace with a generated report

Scope

This PR targets main directly and contains only the optional-locate API, planner guidance, documentation, and targetless tests. It does not contain the focusElement implementation or browser-side Cut command mapping from #2962.

Validation

  • pnpm run lint
  • pnpm exec nx build @midscene/core
  • pnpm exec nx build @midscene/web
  • pnpm exec tsc --noEmit -p packages/web-integration/tsconfig.json
  • pnpm test -- tests/unit-test/action-keyboard-press.test.ts tests/unit-test/prompt/prompt.test.ts in packages/core (46 tests passed)
  • PUPPETEER_EXECUTABLE_PATH='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome' pnpm test -- tests/unit-test/agent-keyboard-press-targetless-e2e.test.ts in packages/web-integration (2 tests passed)

Merge order

Merge this PR into main first. #2962 is an independent browser-side follow-up.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deploying midscene with  Cloudflare Pages  Cloudflare Pages

Latest commit: 31fdd7a
Status: ✅  Deploy successful!
Preview URL: https://d2118b5f.midscene.pages.dev
Branch Preview URL: https://fix-playwright-cut-shortcut.midscene.pages.dev

View logs

@quanruzhuoxiu quanruzhuoxiu changed the title fix(web-integration): preserve selection for cut shortcuts fix(web-integration): preserve selections for keyboard shortcuts Aug 10, 2026
@quanru
quanru force-pushed the fix/playwright-cut-shortcut branch 2 times, most recently from dd44f9d to c59890c Compare August 12, 2026 07:35
@quanruzhuoxiu quanruzhuoxiu changed the title fix(web-integration): preserve selections for keyboard shortcuts feat(core): allow keyboard press without a locate target Aug 12, 2026
@quanruzhuoxiu
quanruzhuoxiu changed the base branch from main to fix/web-keyboard-focus-selection August 12, 2026 07:36
Co-authored-by: quanruzhuoxiu <robin.lifeos@gmail.com>
@quanru
quanru force-pushed the fix/playwright-cut-shortcut branch from c59890c to 31fdd7a Compare August 12, 2026 07:51
@quanruzhuoxiu
quanruzhuoxiu changed the base branch from fix/web-keyboard-focus-selection to main August 12, 2026 07:52
@quanruzhuoxiu
quanruzhuoxiu merged commit 2e3ea16 into main Aug 12, 2026
19 checks passed
@quanruzhuoxiu
quanruzhuoxiu deleted the fix/playwright-cut-shortcut branch August 12, 2026 08:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants