From f8405b88f020481d7bd608ecd2db70f849bc37e5 Mon Sep 17 00:00:00 2001 From: Konst Date: Sun, 19 Jul 2026 00:37:44 +0200 Subject: [PATCH] sources: add a generic IMAP source MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds an IMAP mailbox source for providers the Gmail source cannot reach (the existing one goes through the gog CLI / Gmail API, so a plain IMAP mailbox — GMX, Fastmail, a company server — has no path in today). One source per mailbox, each with an independent cursor (:), registered from sync.imap. Some mail carries the part worth acting on only as an inline image — a scan, a photo, a rendered document — so the body text is useless to the inbox consumer. sync.imap.match singles those messages out by sender or by an image's Content-ID/filename, and sync.imap.vision runs a multimodal pass over the image at ingest time so what lands in the inbox is plain text. match.only_matched drops everything else at the source, turning the mailbox into a single-purpose notifier rather than a second inbox. Both blocks are inert by default: with no match rules configured nothing is singled out, no model is ever called, and this is a plain mailbox reader. The vision prompt and the answer key are deliberately configured as a pair: the prompt tells the model which label to emit and the parser reads the line after exactly that label, so changing one without the other would silently yield unknown_answer every time — a failure that looks like the model degrading rather than a config mistake. Two tests pin the coupling, one moving both (works) and one moving only the prompt (degrades as expected). Disabled by default; passwords are read from sync.imap.passwords keyed by username so no credential belongs in the tracked config. An account with no password is skipped with a warning instead of failing the whole sync. 25 tests pass. --- config.example.yaml | 11 + docs/config.md | 17 ++ docs/sources.md | 81 +++++++ nerve/config.py | 135 +++++++++++ nerve/sources/imap.py | 497 ++++++++++++++++++++++++++++++++++++++ nerve/sources/registry.py | 59 +++++ tests/test_imap_source.py | 437 +++++++++++++++++++++++++++++++++ 7 files changed, 1237 insertions(+) create mode 100644 nerve/sources/imap.py create mode 100644 tests/test_imap_source.py diff --git a/config.example.yaml b/config.example.yaml index 1ab39970..c32be920 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -100,6 +100,17 @@ sync: enabled: true accounts: [] # Set in config.local.yaml schedule: "*/15 * * * *" + # Plain IMAP mailboxes — for providers the Gmail source can't reach. + # One source per account. Passwords go in config.local.yaml under + # sync.imap.passwords, keyed by username. See docs/sources.md for the + # optional pass that reads mail whose payload is only legible as an image. + # imap: + # enabled: false + # accounts: + # - host: imap.example.net + # username: me@example.net + # label: mailbox # source name becomes imap:mailbox + # schedule: "*/30 * * * *" github: enabled: true schedule: "*/15 * * * *" diff --git a/docs/config.md b/docs/config.md index 74deb176..52c829f9 100644 --- a/docs/config.md +++ b/docs/config.md @@ -218,6 +218,23 @@ Sources pull data from external services on a schedule. See [sources.md](sources | `sync.gmail.schedule` | cron | `*/15 * * * *` | Fetch frequency | | `sync.gmail.keyring_password` | string | - | gog keyring password | +**IMAP-specific** (see [sources.md](sources.md) for the image pass): + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `sync.imap.enabled` | bool | `false` | Enable IMAP mailboxes | +| `sync.imap.accounts` | list | `[]` | Mailboxes: `host`, `username`, `label`, `port`, `mailbox` | +| `sync.imap.passwords` | dict | `{}` | Password per username — keep in `config.local.yaml` | +| `sync.imap.schedule` | cron | `*/30 * * * *` | Fetch frequency | +| `sync.imap.initial_lookback_days` | int | `1` | `SINCE` window on first run | +| `sync.imap.match.sender_contains` | list | `[]` | Substrings matched against `From:` | +| `sync.imap.match.attachment_contains` | list | `[]` | Substrings matched against an image's Content-ID / filename | +| `sync.imap.match.only_matched` | bool | `false` | Drop everything that did not match | +| `sync.imap.vision.enabled` | bool | `false` | Run a multimodal pass over a matched message's image | +| `sync.imap.vision.model` | string | *(empty)* | Falls back to `memory.fast_model` | +| `sync.imap.vision.prompt` | string | *(empty)* | What to ask about the image — required when vision is enabled | +| `sync.imap.vision.answer_key` | string | *(empty)* | Label the prompt asks the model to emit; empty = first non-empty line | + **GitHub-specific:** | Key | Type | Default | Description | diff --git a/docs/sources.md b/docs/sources.md index f24a0749..a9411d71 100644 --- a/docs/sources.md +++ b/docs/sources.md @@ -72,6 +72,70 @@ A persistent cron job (`inbox-processor`) runs every 15 minutes: - **Two-step fetch:** Search returns metadata only; body + `internalDate` are fetched per-message via `gog gmail get` (up to 5 concurrent) - **Default schedule:** `*/15 * * * *` (every 15 min) +### IMAP +- **Adapter:** `nerve/sources/imap.py` — plain `imaplib`, no CLI or provider API +- **Scope:** Any IMAP mailbox the Gmail source cannot reach (that one goes through the `gog` CLI / Gmail API), e.g. GMX, Fastmail, a company server +- **Instances:** One source per configured account (`imap: