docs: add cross-origin iframe guidance (CDP WebSocket path) to forge references - #15
docs: add cross-origin iframe guidance (CDP WebSocket path) to forge references#15Wadehl wants to merge 1 commit into
Conversation
…references When target data is inside a cross-origin iframe, browser-act's eval is blocked by the browser's same-origin policy. This commit documents the verified fallback: connect directly to the iframe's CDP WebSocket target, bypassing cross-origin enforcement at the protocol level. Changes: - exploration_extraction.md: add "Cross-Origin iframe" section after DOM Extraction, covering browser-act capability boundary (state/click ok, eval blocked), CDP WebSocket direct-connect pattern with Python code examples, and decision rule for when to use state+click vs CDP - output_template.md: add "CDP Direct" component type with SKILL.md template block, CDP-specific Python wrapper template (connects to CDP and prints result directly, no eval wrapping), and extend Filling Specification browser-act#17 to include the new label
|
Thanks for identifying and documenting this scenario. Cross-origin iframe access is now supported natively by the BrowserAct browser-act --session eval The CLI resolves the iframe selector to a concrete frame ID. BrowserAct Core then maps that frame to its owning CDP target For that reason, we do not plan to merge the proposed CDP Direct approach as written. The documentation could instead be Thanks again for reporting the original usability gap. |
Background
During real-world use of browser-act-skill-forge, a gap was encountered: when the target data is rendered inside a cross-origin iframe, the forge references provided no guidance. The
evalcommand is blocked by the browser's same-origin policy, and without a documented fallback, exploration stalls with no clear next step.Root Cause
The exploration and output references only cover same-origin scenarios. Cross-origin iframes create a hard split in browser-act's command set:
stateandclickwork — OS-level accessibility, not subject to same-origin policyevalis blocked at the JS layerThis split was not documented, leaving the forge without a defined path for this case.
Verified Solution
Chrome DevTools Protocol (CDP) exposes every frame — including cross-origin iframes — as an independent debugging target listed at
http://localhost:9222/json. By connecting directly to the iframe'swebSocketDebuggerUrlvia the Pythonwebsocketslibrary, arbitrary JS can be executed inside the iframe without touching the browser's same-origin enforcement. This was validated end-to-end: virtual scroll triggering, DOM querying, and structured data extraction all work through this path.Changes
references/exploration_extraction.mdstate/clickok,evalblocked)state+clickwhen the a11y tree is sufficient; use CDP when JS execution inside the iframe is required.pyfiles and labeledCDP Directin the generated Skillreferences/output_template.mdCDP Directas a new named component type under Capability Componentseval "$(...)"wrapping)CDP Directlabel appliesDesign Rationale
The CDP path is Python-native: the WebSocket connection must live in Python, so the script cannot follow the standard "print JS string → eval" contract. Making this a distinct labeled type prevents misuse and keeps the
eval "$(...)"wrapper convention unambiguous for all other component types.