Conversation
Generate images from a prompt (text-to-image) or edit an uploaded image
with a prompt (image edit) in a new Doubao conversation, wait for
generation, and download every generated candidate at full resolution.
- UI_SELECTOR strategy, all anchors verified against the live logged-in
doubao.com DOM (2026-08-31): hidden input[type=file] upload via CDP
setFileInputFiles, tiptap ProseMirror / Semi textarea composer,
synthetic KeyboardEvent('Enter') submit (the 2026-08 composer's send
control is a div.send-btn-wrapper with no button semantics, and CDP
key events do not reach the tiptap keymap), rc_gen_image result
extraction with signed-CDN download outside the page.
- Poll loop recovers by re-entering the conversation URL when the
owned-session tab gets recycled mid-wait.
- Expected failures surface as CliError subclasses (ArgumentError /
CommandExecutionError / TimeoutError).
- 15 new tests (JSDOM script coverage + mocked command flows); docs
updated (docs/adapters/browser/doubao.md, docs/adapters/index.md).
…bmit check Fresh conversations now sometimes get /chat/local_<digits> URLs instead of numeric-only ids (observed live 2026-08-31 23:5x); the submit success check only matched digits and falsely reported the prompt as unsubmitted.
Bursts of automated image requests can trip Doubao risk control, which injects a captcha iframe; image submissions then silently never generate. Run detectDoubaoVerificationScript on the submit-failure and timeout paths so the command reports the challenge immediately instead of burning the whole timeout window. Export the shared detector from utils.js for reuse.
The mocked page.wait resolved instantly, so the timeout-path test spun the poll loop hot for the whole real-time deadline and exhausted the worker heap. Advance fake system time from the wait mock instead, and dispatch remaining evaluate mocks by script content.
Author
|
Two live findings from continued testing tonight (both handled in follow-up commits):
Current checks: |
Live evidence (2026-09-01, logged-in doubao.com): a submit can be swallowed silently — the URL still flips to /chat/local_<id> and the user bubble renders optimistically, but the server never receives the message, so 'ask' waits for a reply that never comes and 'doubao ask' times out. Two additional live findings shape the fix: - The send control is button#flow-end-msg-send again; an in-page synthetic .click() on it is intermittently swallowed, and a trusted CDP coordinate click lands in the void when the bridge window is hidden (innerWidth/innerHeight collapse to 0). - The composer only clears on a real send, which makes composer-clear the reliable post-submit signal. clickDoubaoSendButton() surfaces a collapsed viewport via Page.bringToFront (raw CDP), prefers the JS click (immune to window visibility), and keeps the trusted CDP click as the other flavor. sendDoubaoMessage() now verifies the composer cleared and escalates: button click -> native Enter -> button click with the native flavor, throwing a diagnostic error when every flavor is swallowed. Verified live: doubao ask round-trips (3+4 -> 7, 9x9 -> 81) after the fix.
…s variants
The PR's original submit path dispatched a synthetic
KeyboardEvent('Enter') on the composer. Live re-verification on
2026-09-01 showed that mechanism regressed: Enter no longer reaches the
submit handler, the send control is button#flow-end-msg-send again, and
the swallowed submit still flips the URL to /chat/local_<id> with an
optimistic user bubble — the old success check then polled a dead
conversation until timeout.
- Submit through clickDoubaoSendButton (verified composer-clear),
with the in-page script (button click, synthetic Enter fallback) as
the second flavor and a native-flavor re-click as the third.
- Result extraction: one generated asset is served as several ~tplv
templates on random CDN hosts (p3/p11/p26) — ds_wm_* is a 384px
thumbnail, i_pre_wm_* the 2048px+ variant. Dedup by asset pathname
and prefer the hi-res template instead of returning duplicate rows
with a random size; wait for the key set to stabilize so the hi-res
variant (landing a beat later) is not missed.
Verified live on 2026-09-01: two text-to-image runs return 4 unique
2048x2048 images each; opencli validate doubao PASS.
Author
|
Updated the branch with 2026-09-01 live re-verification (details in the PR description's Update 2026-09-01 section):
Live evidence (all logged-in, 2026-09-01): two text-to-image runs → 4 unique 2048×2048 images each; |
The 2026-09 doubao.com composer renders input[type=file] inside a Radix menu with class="hidden" until the composer "+" trigger opens. A display:none input cannot open a file chooser, so the extension's setFileInput (click + Page.fileChooserOpened interception) timed out after 5s on every --image run. Best-effort openDoubaoComposerMenu() natively clicks the "+" trigger (CDP trusted click; synthetic events are ignored by the page) and waits for the input to become visible before handing off to the existing setFileInput flow. Verified live against logged-in doubao.com on 2026-09-13: edit mode round-trip succeeds (upload -> prompt -> submit -> generated image downloaded); text-to-image mode unaffected (4/4 images).
…stream by default Doubao watermarks are applied by the Imagex CDN template, not baked into pixels. The generation SSE stream carries every variant per image, including image.image_ori_raw.url on the unmarked image_raw template with its own server-side signature (the DOM lk3s signature is bound to its template, so rewriting the suffix on a watermarked URL fails with 403). edit-image now wraps page JSON.parse while the run's patches stream in, collecting image_ori_raw URLs keyed by asset path, and swaps each downloaded URL for the raw original by default. --watermark opts back into the watermarked CDN variants. Falls back to the watermarked URL when the hook missed the asset, so behavior degrades gracefully. Note: the watermarked variant carries the AI-generated content label required by Chinese regulation (AI content labeling measures, effective 2025-09); the flag exists so downstream users can make an informed choice. Verified live against logged-in doubao.com on 2026-09-13: text-to-image returned 4/4 unmarked full-resolution images (1728x2304); edit mode (upload + edit prompt) also round-tripped unmarked at full resolution.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
opencli doubao edit-image: one command covering Doubao web image generation (text-to-image) and editing (image + prompt), each in a fresh conversation, waiting for generation and downloading every candidate at full resolution (2048px).Implementation notes (UI_SELECTOR, all anchors verified against the live logged-in doubao.com DOM on 2026-08-31):
setFileInputFileson the hiddeninput[type=file](accept includes png/jpeg/jpg/webp).button#flow-end-msg-send(re-verified 2026-09-01). Submission is verified by the composer clearing — the URL flips to/chat/local_<id>and the user bubble renders optimistically even when the submit event was swallowed, so neither is a success signal on its own.img[src*="rc_gen_image"]inside the message list; the signed CDN URL downloads fine outside the page. One asset is served as several~tplvtemplates on random CDN hosts (ds_wm_*384px thumbnail,i_pre_wm_*2048px+ variant), so rows are deduped by asset pathname with the hi-res template preferred.ArgumentError/CommandExecutionError/TimeoutError), including Doubao-side generation-failure text.Independently mergeable — only imports pre-existing
utils.jsexports; no dependency on #2133 (though they touch the same composer generation).Update 2026-09-01 (live re-verification + hardening)
Re-testing against the logged-in doubao.com UI on 2026-09-01 surfaced three issues, addressed in the two new commits:
button#flow-end-msg-sendagain, and a syntheticKeyboardEvent('Enter')no longer reaches the submit handler. Worse, a swallowed submit still flips the URL to/chat/local_<id>and renders the user bubble optimistically — the original URL-based success check then polls a conversation the server never saw until timeout.edit-imagenow submits throughclickDoubaoSendButton()(shared withask/send, see below) and treats composer-clear as the real-sent signal, re-clicking with the other click flavor when text is still stuck..click()is intermittently swallowed, and a trusted CDP coordinate click lands in the void when the bridge window is hidden (innerWidth/innerHeightcollapse to 0 — same class of issue as fix(chatgpt): adapt to 2026-08 chatgpt.com UI and zero-size bridge viewports #2436).clickDoubaoSendButton()inutils.jssurfaces a collapsed viewport viaPage.bringToFront(raw CDP), prefers the JS click (immune to window visibility), and keeps the trusted CDP click as the other flavor;sendDoubaoMessage()gains the same verify-and-escalate treatment (button click → native Enter → native-flavor button click) with a diagnostic error if every flavor is swallowed. Livedoubao askround-trips verified after the fix (3+4 → 7, 9×9 → 81).~tplvtemplates on different CDN hosts (p3/p11/p26), which defeated the full-URL dedup — runs returned 6 rows for 4 images, randomly 384px or 2048px. Rows are now deduped by asset pathname with thei_pre_wm_*hi-res template preferred, and the poll waits for the key set to stabilize so the hi-res variant (which lands a beat after the 384px thumbnail) is not missed. Two live text-to-image runs each returned 4 unique 2048×2048 images.Tests updated for the new submit flow (57/57 doubao tests green);
opencli validate doubaoPASS.Related issue: none
Update 2026-09-13 (upload menu fix for the 2026-09 composer)
Another live drift, caught by re-running both modes against the logged-in doubao.com UI on 2026-09-13:
class="hidden"(display:none) until the composer "+" trigger opens its menu. The extension'ssetFileInputworks by clicking the input and interceptingPage.fileChooserOpened, and adisplay:noneinput never opens a chooser — every--imagerun failed withPage.fileChooserOpened not received within 5s. Text-to-image (no--image) was and remains unaffected.edit-image.jsgains a best-effortopenDoubaoComposerMenu()step beforesetFileInput: it natively clicks the "+" trigger via a trusted CDP click (page.nativeClick— syntheticpointerdown/clicksequences are ignored by the page) and waits until the file input drops itshiddenclass, then hands off to the unchangedsetFileInputflow. No-op on older DOMs where the input is already visible; existing mocks and assertions are unaffected (the menu probe returns null on the test page mock and the step is skipped)./chat/local_<id>), generation poll, and download of the edited image.Raw original download (291d9b0, same day)
While integrating, one more property of the watermark surfaced: it is applied by the Imagex CDN template, not baked into pixels. The generation SSE stream carries every variant per image, including
image.image_ori_raw.urlon the unmarkedimage_rawtemplate with its own server-side signature — the DOMlk3ssignature is bound to its template, so rewriting the suffix on a watermarked URL fails with 403 (verified).edit-imagenow wraps pageJSON.parsewhile the run's patches stream in, collectsimage_ori_rawURLs keyed by asset path, and downloads the unmarked raw original by default;--watermarkopts back into the watermarked CDN variants. Falls back to the watermarked URL when the hook misses an asset, so behavior degrades gracefully. Existing mocks are unaffected (evaluatereturns undefined in tests → the hook is a no-op).Live 2026-09-13: text-to-image 4/4 unmarked full-resolution (1728x2304, was 384-2048 compressed watermarked); edit mode round-tripped unmarked at full resolution.
Type of Change
Checklist
Documentation (if adding/modifying an adapter)
docs/adapters/(if new adapter) — N/A, existing adapterdocs/adapters/index.mdtabledocs/.vitepress/config.mts(if new adapter) — N/A, existing adapterREADME.md/README.zh-CN.mdwhen command discoverability changed — N/A (README lists adapters, not commands; discoverability is viaopencli list)<prompt>positional;--image/--out/--timeoutnamed flags)CliErrorsubclasses instead of rawErrorChecks run locally
Screenshots / Output
Sample run (2026-08-31, live):
Edit mode output has the same shape (single result when Doubao returns one candidate).