fix(sync): copy peer files from SAF dir into internal dir before pull - #292
TimeToBuildBob wants to merge 1 commit into
Conversation
When using a SAF/Syncthing directory for sync, copySyncFilesToSafDir mirrors internal → SAF (outbound) but nothing copies other hosts' databases FROM the SAF directory INTO the app-private syncDir before syncBoth runs. So pull_all always finds zero peers — the internal dir only ever contains our own staging. Add copyPeerFilesFromSafDir() that mirrors SAF → internal for all hostname directories except our own (our own staging is authoritative in the internal dir). Call it in syncBothAsync's lambda before syncBoth, inside the executor thread. Errors per entry are logged and skipped so a partially-accessible SAF directory does not abort an otherwise healthy sync pass. Fixes ActivityWatch#291. Git-Session-Id: 0e07
|
Closing for now per #291 (comment): Erik asked whether Android pull should wait for sync v2 (JSONL + zst compression + read-from-sync-staging) to avoid blowing up app storage. The architectural fix here (SAF → internal pre-copy before pull) is correct and ready, but should land only once the sync v2 storage model is in place. Will reopen when sync v2 is ready. |
|
| errors++ | ||
| continue | ||
| } | ||
| inp.use { FileOutputStream(File(destDir, name)).use { out -> it.copyTo(out) } } |
There was a problem hiding this comment.
Partial Copies Replace Valid Data
If a SAF read fails or is interrupted, FileOutputStream has already truncated the existing peer database. The error handlers leave that partial file in place, and syncBoth immediately tries to consume it. A transient provider error or a file being updated by Syncthing can therefore replace the last valid peer snapshot with a corrupt SQLite database and cause the pull to fail. Copy to a temporary file and replace the destination only after the copy succeeds.
| errors++ | ||
| continue | ||
| } | ||
| inp.use { FileOutputStream(File(destDir, name)).use { out -> it.copyTo(out) } } |
There was a problem hiding this comment.
SAF Names Escape Sync Directory
Provider-controlled DocumentFile.name values are used directly in File paths at the hostname, directory, and file levels. Because a user-selected third-party document provider can return names containing .. or path separators, a malicious or nonconforming provider can make this write escape syncDir and overwrite other app files, including app-private files when the internal-storage fallback is active. Reject unsafe path components and verify that every canonical destination remains inside the intended root.
How this was verified: SAF display names flow unchanged through File(destDir, name) into a writable file sink, while .. resolves outside the internal sync directory.
|
|
||
| val localHostDir = File(destRoot, hostname) | ||
| localHostDir.mkdirs() | ||
| val (c, e) = copyFromSafDirectory(hostDir, localHostDir) |
There was a problem hiding this comment.
The inbound copy creates and overwrites entries but never removes internal peer files that have disappeared from the SAF tree. Since syncDir persists between runs and remote discovery scans retained hostname and device databases, removing a peer or renaming its hostname leaves the old snapshot participating in later pulls and accumulating indefinitely. Reconcile copied peer directories with the current SAF tree while preserving the intentionally skipped local hostname.
Problem
When using a SAF/Syncthing folder for sync (Android 11+ setup),
pull_allinsidesyncBothalways reports "pulled 0 peers" regardless of how many peers have written their databases into the Syncthing-shared folder.Root cause:
copySyncFilesToSafDirmirrors internal → SAF (making our data visible to Syncthing) but nothing mirrors SAF → internal beforesyncBothruns. The app-privatesyncDironly ever contains our own staging files, solist_remote_dbsfinds zero peers.Reported in #291.
Fix
Add
copyPeerFilesFromSafDir()that reads all hostname directories from the SAF tree into the internalsyncDirbeforesyncBothis called:syncBothAsynclambda, inside the executor thread, immediately beforesyncBothThe reverse-copy helper
copyFromSafDirectorymirrorsDocumentFile → FileusingContentResolver.openInputStream+FileOutputStream, structurally parallel to the existingmirrorDirectory(File → DocumentFile).Testing
Manual: configure Syncthing on two devices pointing at the same SAF-granted folder. After this change,
syncBothon either device should report non-zero peer count and actually pull events from the other device's database.