Skip to content

Unattended remote access and remote control - #45

Open
Rambomst wants to merge 9 commits into
Ylianst:mainfrom
Rambomst:feature/unattended-remote-access
Open

Unattended remote access and remote control#45
Rambomst wants to merge 9 commits into
Ylianst:mainfrom
Rambomst:feature/unattended-remote-access

Conversation

@Rambomst

@Rambomst Rambomst commented May 30, 2026

Copy link
Copy Markdown

Turns the Android agent into a full unattended remote-control agent: background screen sharing and remote input, running as a persistent foreground service.

Unattended operation

  • specialUse foreground service + boot receiver, so it runs and reconnects without the app open.
  • Shared AgentController runtime for lifecycle; unattended setup prompts and settings.

Remote control (previously view-only)

  • Accessibility service provides background capture (takeScreenshot) and remote input: mouse/touch as gestures, keyboard as text + cursor/navigation, and Android global actions (app drawer, notifications, quick settings, lock, power).

Platform & tooling

  • Merged latest upstream (Android SDK 37), file-download fix, edge-to-edge/system-bar handling.
  • GitHub Actions PR build workflow; developer docs under docs/.

Known limitations

  • Sideloads are flagged by Play Protect (accessibility/remote-control category); install via adb install or enterprise deployment. On Android 13+ the accessibility toggle is a Restricted Setting unless installed via ADB/EMM.
  • Unattended capture is ~2-3 fps (Android throttles AccessibilityService.takeScreenshot).
  • Secure surfaces and the lock screen are redacted by Android and can't be captured or controlled by a non-privileged app.

Disclaimer: developed with an AI coding assistant.

Rambomst added 3 commits May 30, 2026 21:57
- Add foreground service, boot receiver, and accessibility remote control
- Route agent lifecycle through shared runtime controller
- Add unattended setup prompts and settings status entries
- Improve remote desktop refresh and session notifications
- Update Gradle and scanner dependency versions
…tter battery usage

- Add retry backoff logic with delay for agent reconnection attempts
- Introduce `shouldKeepForegroundServiceRunning` for better service control
- Optimise frame encoding with idle frame delay adjustments
- Enhance user disconnect handling via shared preferences
- Adjust remote desktop frame capturing to dynamically handle delays
- MeshAgent: stop starting projection immediately after the agent control
  connection authenticates.
- AgentRuntime: require an active desktop tunnel before starting screen
  projection.
@Rambomst

Copy link
Copy Markdown
Author

Ive had this running on two devices without any issues for the past week or so. My initial commit introduce some battery issues but that was resolved and now its back to basically using nothing.

…s disabled

- Implement user consent prompt via dialogs and notifications for remote desktop access.
- Add new notification channel for "Approve/Deny" screen sharing actions.
- Ensure thread-safe handling and lifecycle management for capture processes.
- Remove redundant battery optimization request logic.
- Optimize screenshot capture flow with throttling and backoff mechanisms.
@Rambomst

Rambomst commented Jun 14, 2026

Copy link
Copy Markdown
Author

A few fixes to the unattended consent flow, with Play Store approval in mind.

  • Consent is properly enforced now. With Automatic Consent off, every session has to be approved (in-app dialog, or Approve/Deny on the notification), so there's no silent screen capture, which is the main thing Google flags for apps like this.
  • Removed the battery optimization popup that auto-fired on launch, since auto-firing that intent is a known rejection trigger. It's only reachable from Settings now.
  • Capture only runs during an active session and shows a persistent notification plus a per-session one, so it stays transparent rather than looking like spyware.

To hopefully get this through review we need to do this in Play Console:

  • Complete the Accessibility API declaration and add a prominent in-app disclosure, framed as person-driven remote support as Google restricts accessibility use for automation as of the Jan 2026 policy change.
  • Fill in the foreground service types declaration for the specialUse service which needs a short justification and a demo video. Something like this:

MeshCentral Agent is a remote management/support tool. The foreground service keeps a persistent connection to the management server open so an administrator can reach the device on demand for remote desktop, file transfer, and commands. It has to run continuously and indefinitely; it isn't a finite data sync, media playback, location job, or any other predefined types, so specialUse applies.

  • There might also be a need to justify the battery optimization exemption, but that should be fine since this agent wouldn't work properly without it. If they did have an issue with it we can change it pretty easily but no harm in trying if this was already going through review.

How to sideload this with the accessibility changes:

Play Store > profile icon > Play Protect > gear/settings icon > toggle off "Scan apps with Play Protect" > install the APK > turn it back on after.

Then it blocks the accessibility access until we do this:
Settings > Apps > Mesh Central > App info⋮(top-right) > Allow restricted settings (authenticate), then the toggle works.

@GlitchedCod

Copy link
Copy Markdown

I hope that this get official one day, it's really a useful feature. The app itself without that is basically useless, it's not ok to ask a user how to go to the home menu, find all the apps, start the agent and then allow the screen sharing. Plus you can't see when a device is online on the MeshCentral dashboard since the app is not running in the background if it wasn't started at the launch of the device.

It would also be cool to add the "device administrator" rights to the app so that it can be set as the device owner, disable FRP on the device and so that the user can't uninstall it unless the device is wiped.

…ility handling

- Add `.github/workflows/android-pr-build.yml` to automate pull request builds, including debug APK upload.
- Refactor `MeshAccessibilityService` screenshot handling with callback extraction.
- Prevent unintended `NullPointerException` by introducing safe access and cleanup methods.
- Harden navigation logic in fragments with exception handling.
- Improve encoded bitmap memory handling during desktop frame capture.
- Update remote desktop input documentation to reflect accessibility feature improvements.
- Enhance WebSocket handling with safe null checks and connection closures.
- Add enhanced test coverage for edge cases and validation logic.
@Rambomst

Copy link
Copy Markdown
Author

@GlitchedCod Yeah I hope this gets merged as well, I brought it up at the community meeting last night so hopefully it will get some traction.

As for your request around device administrator functionality, that would be out of scope for this PR. One thing of note around that, I don't think that sort of functionality would ever make it to the play store build, so if it was to be development it would be something that needs to get manually deployed to each device you want to manage via ADB.

@marclaporte

Copy link
Copy Markdown

I know @kambereBr is testing this PR. yay!

I look forward to this as I have an Android device which I need to manage from time to time :-)

- Add `android:fitsSystemWindows="true"` to adjust layout for system bars.
- Set `app:statusBarForeground` in `AppBarLayout` for better visual consistency.
…le gesture execution

- Add drag gesture support with motion path tracking and duration-based gestures.
- Introduce gesture queueing mechanism to ensure sequential input handling.
- Optimise screenshot capture timing with dynamic wake handling and backoff adjustments.
- Enhance text input handling with precise cursor control and focused field management.
- Refine global action handling and keyboard interaction for seamless remote desktop experience.
@Rambomst Rambomst changed the title Add unattended remote access support Unattended remote access and remote control for the Android agent Aug 28, 2026
@Rambomst Rambomst changed the title Unattended remote access and remote control for the Android agent Unattended remote access and remote control Aug 28, 2026
@GlitchedCod

Copy link
Copy Markdown

I know @kambereBr is testing this PR. yay!

I look forward to this as I have an Android device which I need to manage from time to time :-)

That is so cool ! This is definitely gonna be a game changer. Fun fact I actually read this merge request yesterday to see if they were any update onto it.

@Rambomst

Copy link
Copy Markdown
Author

I have created a MeshCentral PR which implements some UI changes to facilitate interactions with this agent, things like buttons to go home/back/recents menu/etc.

MeshCentral #8123

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants