Skip to content

aw-sync: leftovers after #685/#686 — orphaned 2-level staging db, stale -synced-from- buckets, walker enters dot-dirs #689

Description

@ErikBjare

Collecting the cleanup items that #685 / #686 / #687 deliberately leave behind, so they do not get lost once those merge.

1. The orphaned 2-level staging database

#682's root cause is that aw-sync daemon staged local data at {sync_dir}/{device_id}/test.db instead of {sync_dir}/{hostname}/{device_id}/test.db. #685 fixes where new pushes go but does not touch what is already there, as @TimeToBuildBob notes on that PR.

So every user who ran the daemon has an orphaned database at the sync root that:

Wanted: a one-time migration, or at minimum detection. aw-sync status (#687) already classifies sync-dir entries by layout, so it is the natural place to report "this is an orphaned legacy staging db, N bytes, safe to delete" — and a --clean or documented manual step to remove it. Deleting it from inside a synced folder propagates to every peer, which is the desired outcome but should be an explicit user action, not a silent one.

2. Stale -synced-from- buckets inside staging databases

Before #648, the push path re-exported buckets that had themselves been synced from another host. That fix stopped new ones being created; nothing prunes the existing ones. My root staging db still contains, for example:

aw-watcher-window_erb-laptop2-arch-synced-from-erb-laptop2-arch   652321 events
aw-watcher-afk_erb-main2-arch-synced-from-erb-main2-arch          373525 events
aw-watcher-window_erb-main3-synced-from-erb-main3                 190854 events

These are second-hand copies sitting in a file that is replicated to every device. They are filtered on read by is_synced_bucket, so they are not a correctness problem any more — they are dead weight, and they make the "is this staging db healthy?" question harder to answer. Worth reporting in status and offering to prune.

3. The walker descends into dot-directories

#685's collect_db_files recurses into every directory at depth 0 and 1 with no name filter, so it walks .git, .stfolder, .stversions and similar. On my sync folder that means walking a real .git tree on every pass.

No .db files are found there in practice, so this is not currently a correctness bug — but find_remotes now propagates I/O errors rather than swallowing them (deliberately, per its doc comment), so an unreadable entry under a dot-directory would fail the whole pass, and per the daemon issue filed alongside this one, that kills the daemon. Skipping .-prefixed directories is a one-line guard that removes the class.

Also worth considering while in there: Syncthing versioning layouts. .stversions/{hostname}/{device_id}/test.db is deep enough to escape the current depth limit, and Simple File Versioning appends a timestamp to the filename so the extension stops being db — but that is luck, not design, and a .stversions with Trash Can versioning at a shallower path would be picked up as a peer.

4. test.db as the staging filename

Raised in #684 item 6 and not addressed. Users are told to inspect this folder; every database in it being called test.db reads as leftover scaffolding. Renaming needs a fallback that still reads test.db, so it is a small migration rather than a rename.

cc @TimeToBuildBob — items 1 and 2 are the ones with user-visible cost (disk + replication); 3 is a one-liner; 4 can wait.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions