feat(panel): hardware buttons, keyboard, and idle-freeze fix for the portable emulator panel - #24
Merged
Merged
Conversation
Bring the Windows/Linux panel to input parity with the macOS sidebar, all over
the existing gRPC connection (no adb shell):
- Hardware buttons (Home/Back/Recents/Power) in the sidebar, forwarded via
EmulatorInput::key ("GoHome"/"GoBack"/"AppSwitch"/"Power"). Verified live on
Windows: pressing Home from the app drawer returns to the home screen.
- Keyboard: the video_frame is `focusable`, and KeyDown events map floem keys
(characters + Enter/Backspace/Tab/Esc/Delete/arrows) to EmulatorInput::key, so
clicking into the panel routes typing to the device.
- Fix EmulatorGrpcClient::send_key: send a keydown+keyup pair instead of a lone
`keypress`, which the emulator ignores for non-character keys (Home/Back/…) —
this is why keys never reached the device before (the macOS panel sidesteps it
by shelling out to `adb`). This is the fix that makes the buttons actually work.
… screen The emulator emits a screenshot frame only when the screen *changes*, so an idle screen legitimately produces none. The 10s "no frame" watchdog treated that as a dead socket and stopped the stream for good — the panel froze (last frame stuck) until a manual Stop→Start, which read as "taps/swiping stopped working" (input still reached the device, but nothing repainted). Wrap connect+stream in a reconnect loop: on a stall/end, drop the stream and reconnect (a fresh stream repaints the current frame immediately), only exiting when the panel (UI receiver) is gone. Timeout bumped 10s→20s to cut idle churn. Native-resolution probe is cached across reconnects.
dev-josias
changed the title
This was referenced Jul 1, 2026
dev-josias
added a commit
that referenced
this pull request
Jul 1, 2026
* docs: record agent-CLI backends + emulator hardware keys/keyboard Reflect what landed on main toward v0.3.0: - AI assistant now supports external agent CLIs — Claude Code (per-action approval), Codex (sandboxed + consent, macOS/Linux), Gemini (read-only) — PR #25. Added to README features, the Pages feature grid, and the CLAUDE.md status/positioning. - Emulator panel gained hardware buttons (Home/Back/Power) + keyboard and an idle-freeze fix (PR #24/#26); updated the preview wording everywhere. Download badge stays 0.2.0 until a v0.3.0 tag is cut (per CLAUDE.md). * docs(site): next-level landing page redesign Rebuilt the GitHub Pages landing page for impact + conversion: - Animated aurora backdrop, glassmorphism nav pill, gradient hero headline, and an OS-detected primary CTA ("Download for macOS/ Windows/Linux" via navigator). - Product screenshot framed in a floating app-window with a gradient glow border + thumbnail switcher. - Trust/stats strip, bento feature grid with hover glow, and a new AI-agents spotlight with a live mock terminal + approval card that shows the actual Claude Code per-action approval flow. - Scroll-reveal animations (IntersectionObserver), smooth scroll, prefers-reduced-motion + responsive/mobile fallbacks. - Verified in-browser: renders clean, no console errors, reveals + OS-detection working. Download badge stays v0.2.0 until the tag. Content-only (no code); real download/GitHub links + screenshots + Lapce attribution + Apache-2.0 all preserved. * docs(site): reskin landing page in UMIDE's own theme colors Swap the generic blue/violet palette for UMIDE's actual identity: - One Dark base #282c34 (the editor background) with a subtle depth gradient from the secondary bg #21252b. - Brand blue from the logo (#3da4eb / #196eae) + theme blue #61afef as the primary accent and hero/CTA gradient (blue → cyan #56b6c2). - Theme purple #c678dd and cyan for the aurora + secondary accents; theme green #98c379 for the 'live' pip. The page now visually matches the app it ships. * docs(site): add 'See it in action' demo-video section + lightbox A dedicated demo section with a dimmed product poster, a pulsing gradient play button, and a responsive video lightbox (YouTube iframe or inline mp4). Publishing the demo is a one-line change: set the DEMO_VIDEO JS constant to a YouTube ID or an mp4 path and commit. Until then the play button gracefully links to the repo. Also adds a 'Demo' nav link. Verified in-browser: poster + play button render, lightbox opens/embeds/ closes (× · backdrop · Esc) and clears its slot, no console errors.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Brings the portable Windows/Linux Android emulator panel to input parity with the macOS sidebar (hardware buttons + keyboard) and fixes a panel-freeze bug, all over the existing gRPC connection (no
adb shell).1. Hardware buttons
Home / Back / Recents / Power in the panel sidebar (🏠 ◀ ⊞ ⏻), forwarded via
EmulatorInput::key("GoHome"/"GoBack"/"AppSwitch"/"Power").2. Keyboard
The
video_frameis nowfocusable;KeyDownevents map floem keys (characters + Enter/Backspace/Tab/Esc/Delete/arrows) toEmulatorInput::key, so clicking into the panel routes typing to the device.3. Fix
send_key(the fix that makes keys actually work)send_keysent a lonekeypress, which the emulator ignores for non-character keys (Home/Back/…) — the same way the DOM only fireskeypressfor character keys. Now it sends a proper keydown + keyup pair. (The macOS panel dodged this by shelling out toadb; the portable panel uses gRPC, so the event type matters.) This is why keys never reached the device before.4. Fix panel freeze on an idle screen (
emulator_stream.rs)The emulator emits a frame only when the screen changes, so an idle screen produces none. The 10s "no-frame" watchdog (from #22) treated that as a dead socket and stopped the stream permanently → the panel froze on the last frame until a manual Stop→Start, which read as "taps/swiping stopped working" (input still reached the device; nothing repainted). Now the stream reconnects on a stall/end instead of giving up (a fresh stream repaints the current frame immediately); it only exits when the panel closes. Timeout 10s→20s to cut idle churn; native-resolution probe cached across reconnects.
Verified live on Windows (real
Pixel_9a, headless-gpu host)send_keykeydown+keyup fix).no frame for 20s, reconnecting(notstopping) and the on-screen clock keeps advancing; a swipe afterward navigates the device. Stream stays live, touch works.cargo fmtclean (CI confirms Clippy/build on all three OSes).Not independently confirmed (low risk)
send_keypath +focusable, so it should work; worth a 2-second manual check (click into the panel, focus a device text field, type).Note
Item 4 touches
emulator_stream.rs(the #22 watchdog area) — heads-up if that's being iterated on elsewhere.🤖 Generated with Claude Code