Skip to content

feat(trayicon): detect database-locked state and show recovery notification - #130

Closed
TimeToBuildBob wants to merge 5 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/db-locked-detection
Closed

TimeToBuildBob wants to merge 5 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/db-locked-detection

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Summary

  • Adds Module.has_db_locked_error() to detect SQLite 'database is locked' in recent server logs
  • Adds Manager.get_db_locked_modules() to surface which server modules are locked
  • Shows a system tray notification with the data directory path and a sqlite3 backup command when a locked state is detected (once per lock event, no spam)
  • Adds Open data folder to the tray menu alongside the existing log/config folder items

Motivation

Users hit a circular failure: a third-party watcher (e.g. aw-watcher-utilization) can grow the SQLite database until it locks, which then causes the web UI to 500-loop — making the built-in export page inaccessible. The tray notification gives an escape hatch when the only path to recovery is bypassing the UI entirely.

Reported in: ActivityWatch/activitywatch#1394

Test plan

  • New unit tests in tests/test_manager.py cover the log-parsing detection and server-name filtering
  • Manual: simulate a locked DB by adding 'database is locked' to the aw-server log — tray notification appears within 30 s
  • Manual: notification does not repeat on subsequent polls once initially shown
  • CI passing

…cation

When aw-server's log contains 'database is locked' errors (e.g. caused by
a third-party watcher growing the SQLite DB too large), aw-qt now:

- Shows a system tray notification with the data directory path and a
  sqlite3 backup command so users can export their data without the web UI
- Checks every 30 seconds; notifies once per locked-state-start (no spam)
- Resets the notification gate when the lock clears

Also adds "Open data folder" to the tray menu alongside log/config.

Fixes a usability gap reported in ActivityWatch/activitywatch#1394 where
a locked DB made the export page inaccessible in a circular failure.
@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR detects database-lock messages in recent server logs, reports affected server modules through the manager, and provides recovery guidance through a tray notification.

  • Adds bounded-memory tail retention for lock-message detection.
  • Adds server-module filtering and focused manager tests.
  • Adds a data-folder menu action and stateful notification suppression.

Confidence Score: 4/5

The PR is not yet safe to merge because a recovery followed by another lock can remain hidden while the earlier lock message is still in the detection window.

The notification state is derived from lock-phrase presence within a rolling 200-line tail rather than an actual lock-event transition, so recovery does not clear suppression and a subsequent lock can fail to notify the user.

Files Needing Attention: aw_qt/manager.py, aw_qt/trayicon.py

Important Files Changed

Filename Overview
aw_qt/manager.py Adds recent-log lock detection and manager-level filtering for live ActivityWatch server modules.
aw_qt/trayicon.py Adds data-folder access and a recurring, stateful database-lock notification flow.
tests/test_manager.py Adds unit coverage for phrase detection, server filtering, parameter forwarding, and the log-tail boundary.

Reviews (3): Last reviewed commit: "fix(trayicon): use single window for db-..." | Re-trigger Greptile

Comment thread aw_qt/manager.py Outdated
Comment thread aw_qt/manager.py Outdated
After a server recovered from a SQLite lock, the 200-line historical
window kept returning True for has_db_locked_error, leaving the module
in _db_locked_notified. A subsequent new lock event was then silently
suppressed because the module appeared still-notified.

Fix: use a 20-line window when deciding to *clear* the notification
state. A recently-recovered server will quickly produce clean log lines
that push lock messages out of this shorter window, allowing a new lock
event to fire a fresh notification. The broader 200-line window is
retained for *detecting* lock events (sensitivity).

Also adds recent_lines param to get_db_locked_modules() and tests for
the tail-boundary and parameter-forwarding behaviour.
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Fixed the lock-state transition bug Greptile identified (commit a3eed35).

The bug: After a server recovered from a SQLite lock, the 200-line historical window kept returning True for has_db_locked_error, leaving the module in _db_locked_notified. A subsequent new lock event on the same server was then silently suppressed — never shown to the user.

The fix: Use a shorter 20-line window when clearing notification state (get_db_locked_modules(recent_lines=20)), while retaining the broader 200-line window for initial detection. A recovered server quickly produces clean log lines that push old lock messages out of the short window, releasing the gate for a fresh notification.

Also added two tests:

  • test_get_db_locked_modules_passes_recent_lines — verifies the param is forwarded correctly
  • test_has_db_locked_error_only_checks_tail — verifies lock messages outside the tail window don't trigger detection (the boundary that makes the fix work)

…ines()

Avoids reading the entire log file into memory on every 30-second poll.
deque(f, maxlen=N) streams lines and keeps only the last N, so memory
usage is O(N) regardless of log size.

Co-Authored-By: Bob <bob@superuserlabs.org>
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread aw_qt/trayicon.py Outdated
The two-window approach (200-line detect, 20-line clear) created a dead
zone: when 20-199 unrelated lines accumulated after a lock error, the
20-line window cleared _db_locked_notified while the 200-line window still
reported the module as locked, causing a new notification every 30-second
poll indefinitely.

Fix: use the already-computed locked_names (200-line window) for the set
intersection instead of a separate 20-line query. A module stays suppressed
until the lock error ages fully out of the 200-line window, then a
subsequent new lock event triggers a fresh notification.
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Fixed the P1 from round 2 (commit 6609560). The two-window approach introduced a repeat-notification loop: when 20–199 unrelated lines accumulated after a lock error, the 20-line window cleared _db_locked_notified while the 200-line window still detected the lock, causing a new notification every 30 s indefinitely.

Fix: single-window clearing using the same locked_names set already computed for detection. The suppression set is intersected with the 200-line detection results — a module stays suppressed until its lock error fully ages out of the window, then the next new event triggers a fresh notification.

All three Greptile findings addressed:

  • P1 "Historical logs mask lock transitions" → a3eed35 (short window for clearing)
  • P2 "Polling reads entire log files" → 89c9071 (deque tail read, no full readlines)
  • P1 "Mismatched windows repeat notifications" → 6609560 (unified single-window clear)

CI re-running on new head.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

@greptileai review

@TimeToBuildBob

TimeToBuildBob commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

🤖 AI code review

Safe to merge — no P0/P1 findings on latest review

Updated after inline dispositions on finding threads — this is the current state; the verdict below is frozen at review time and is kept as the historical record of that pass.

Finding disposition
Finding Severity State
aw_qt/trayicon.py:301 P1 superseded by latest review (not reproduced)
aw_qt/manager.py:345 P2 rejected
aw_qt/manager.py:339 P2 accepted-tradeoff
aw_qt/trayicon.py:287 P2 rejected
aw_qt/trayicon.py:299 P2 rejected

Adds database-locked detection to aw-qt: Module.has_db_locked_error() scans the tail of a module's log for SQLite lock phrases, Manager.get_db_locked_modules() filters to alive server modules, and TrayIcon polls every 30 seconds to show a one-time notification with data-dir and sqlite3 backup instructions. Also adds an 'Open data folder' tray menu item and unit tests for the new detection logic.

Needs a look — P2 only

Confidence 4/5

1 finding · ⚠️ 1 P2

⚠️ P2 mediumaw_qt/manager.py:339

The has_db_locked_error() method opens the log file without specifying an encoding, using the default locale encoding. On Windows, the default encoding is often cp1252, and log files may contain UTF-8 characters (e.g. from module names or error messages). If a log line contains a byte sequence that is invalid in the default encoding, open() raises UnicodeDecodeError, which is not caught by the except OSError clause. The comment in trayicon.py mentions 'decoding an unexpectedly-encoded log line' as a reason for the try/finally, but the exception would propagate out of has_db_locked_error() and out of get_db_locked_modules(), causing the entire poll to fail. The finally in check_db_locked() reschedules the timer, so the loop survives, but the notification is skipped for that poll and the error is unhandled. The fix is to open the file with encoding='utf-8', errors='ignore' or to catch UnicodeDecodeError explicitly.

with open(log_path, encoding='utf-8', errors='replace') as f:

How this was verified: Checked the open() call on line 339: no encoding parameter. The except clause on line 345 only catches OSError. UnicodeDecodeError is a subclass of ValueError, not OSError, so it would propagate. The trayicon comment on lines 278-280 explicitly anticipates this failure mode, confirming it is a real concern.

Files changed (3) — the diff as I read it
  • aw_qt/manager.py — Adds has_db_locked_error() to Module and get_db_locked_modules() to Manager, plus deque import.
  • aw_qt/trayicon.py — Adds db-locked polling loop, notification state set, and 'Open data folder' menu action.
  • tests/test_manager.py — Adds TestDbLockedDetection class covering log parsing, tail window, and server filtering.
Previous review passes
commit score findings engine when
66095601f601 3/5 4 llm 2026-09-16 08:42 UTC

Reviewed 0410b07b8057 · openrouter/deepseek/deepseek-v4-flash-0731 · llm engine · 75s · about this reviewer

Maintainer commands

@TimeToBuildBob review (own line) — fresh review · @TimeToBuildBob fix — a worker acts on the findings. Once per comment; 👀 = received.

Comment thread aw_qt/trayicon.py Outdated
Comment thread aw_qt/trayicon.py
Comment thread aw_qt/manager.py
Comment thread aw_qt/trayicon.py
A poll that raises (e.g. UnicodeDecodeError on an unexpectedly-encoded
log line, which has_db_locked_error's OSError handler does not cover)
previously skipped the singleShot reschedule and killed the detection
loop permanently. Wrap each poll body in try/finally so the timer is
always rescheduled.

Git-Session-Id: f88056d1-92a9-5a09-9bc2-7d3362f1244c
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Merge recommendation (convergence adjudication)

Adjudicated against head 6609560 (the newest review pass — both our AI-review marker and Greptile's "Last reviewed commit" cover it).

Fixed:

  • P1 "polling loop killed permanently on exception" — fixed in 0410b07: the poll body is wrapped in try/finally with the 30s reschedule in finally, so a poll that raises (e.g. UnicodeDecodeError on an oddly-encoded log line) can no longer kill the detection loop.

Remaining (non-blocking, dismissed with reasons):

  • "Backup command not shell-quoted" — display-only string (never executed by this PR); data_dir is already inside double quotes, so Windows paths with spaces render correctly; raw data_dir is also shown separately.
  • "has_db_locked_error returns False on OSError" — intended fail-safe: retry next 30s poll instead of false-positive notifications on transient read failures.
  • "Notification-state clearing causes repeat notifications" — misreads the single-window post-fix code: a notification can only refire when a lock-error line re-enters the 200-line tail, i.e. a genuinely new lock event.

All four review threads are resolved.

CI: green on 6609560 (ubuntu-22.04, macos-14, windows-latest, Greptile); 75/75 unit tests pass locally on 0410b07. CI on 0410b07 will re-run on push.

Domain risk: notification behavior is platform-dependent (Qt system-tray showMessage); a maintainer eyeball on the notification text/format on Windows is the only manual check worth doing.

Convergence: detector reports round_convergence.status = converged, 0 unresolved P1/P2; this adjudication closes the escalation.

Ready for maintainer judgment — merge-ready does not mean auto-merge.

Comment thread aw_qt/manager.py
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Merge recommendation (convergence adjudication, follow-up)

Adjudicated against head 0410b07b80576995513eeece2fd0b42feb157b6e. Freshness: newest bob-ai-review marker sha 0410b07b8057 matches this head. Greptile's "Last reviewed commit" is still 6609560not re-triggered (attempt cap).

Fixed (prior session, still on this head):

  • P1 "polling loop killed permanently on exception" — 0410b07: check_db_locked() wraps the poll in try/finally and always reschedules the 30s timer.

Remaining (non-blocking, dismissed this session):

  • P2 acd04dd355df (aw_qt/manager.py:339, no encoding on open()) — accepted tradeoff. Same unadorned open() as read_log() at line 328; lock phrases are ASCII; try/finally already keeps the loop alive; not pushing a one-line encoding kwarg to chase 5/5 after the convergence cap.

Earlier P2s on this head remain dismissed: display-only sqlite3 quoting; OSError → False fail-safe; single-window notify-state is not a spam loop.

Verified against: 0410b07b80576995513eeece2fd0b42feb157b6e

CI: green on this head (ubuntu-22.04, macos-14, windows-latest).

Domain risk: tray showMessage text/format is platform-dependent; a maintainer eyeball on Windows is the only manual check worth doing. Encoding unification of log reads is optional follow-up, not this PR.

Convergence: verdict=converged (0 unresolved Greptile P1/P2). round_convergence.status=new_blocking, stable_rounds=0 / required_stable_rounds=2 — the last two Greptile rounds each introduced a new P1 that was then fixed; there was never a stable no-new-blocking round, which is why the attempt cap fired. All Greptile inline threads are resolved.

Ready for maintainer judgment — merge-ready does not mean auto-merge. No further Greptile trigger from this session.

@ErikBjare

Copy link
Copy Markdown
Member

Closing for now, might reopen if it recurs for people.

@ErikBjare ErikBjare closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants