Skip to content

sources: add a generic IMAP source - #236

Open
constkolesnyak wants to merge 1 commit into
ClickHouse:mainfrom
constkolesnyak:feat/imap-source-v2
Open

sources: add a generic IMAP source#236
constkolesnyak wants to merge 1 commit into
ClickHouse:mainfrom
constkolesnyak:feat/imap-source-v2

Conversation

@constkolesnyak

Copy link
Copy Markdown
Contributor

What

Add a generic IMAP sync source (nerve/sources/imap.py) that polls one or more IMAP mailboxes and surfaces new emails as source messages.

  • Connects via IMAP4_SSL, authenticates per-account with credentials stored in config.local.yaml
  • Fetches UNSEEN messages from INBOX, condenses each into a compact representation (sender, subject, plain-text body) for downstream processing
  • Accounts with no password in config are skipped with a warning instead of failing the entire sync
  • Includes unit tests (tests/test_imap_source.py)

Config example (in config.local.yaml):

sync:
  imap:
    accounts:
      - username: you@example.com
        host: imap.example.com
    passwords:
      you@example.com: your-password

Why

Gmail's IMAP access can be locked down by organization policy; a generic IMAP source lets any IMAP-accessible mailbox feed into nerve.

Rebased on current upstream/main (2026-07-30).

Adds an IMAP mailbox source for providers the Gmail source cannot reach
(the existing one goes through the gog CLI / Gmail API, so a plain IMAP
mailbox — GMX, Fastmail, a company server — has no path in today). One
source per mailbox, each with an independent cursor
(<UIDVALIDITY>:<max_uid>), registered from sync.imap.

Some mail carries the part worth acting on only as an inline image — a
scan, a photo, a rendered document — so the body text is useless to the
inbox consumer. sync.imap.match singles those messages out by sender or by
an image's Content-ID/filename, and sync.imap.vision runs a multimodal pass
over the image at ingest time so what lands in the inbox is plain text.
match.only_matched drops everything else at the source, turning the mailbox
into a single-purpose notifier rather than a second inbox.

Both blocks are inert by default: with no match rules configured nothing is
singled out, no model is ever called, and this is a plain mailbox reader.

The vision prompt and the answer key are deliberately configured as a pair:
the prompt tells the model which label to emit and the parser reads the line
after exactly that label, so changing one without the other would silently
yield unknown_answer every time — a failure that looks like the model
degrading rather than a config mistake. Two tests pin the coupling, one
moving both (works) and one moving only the prompt (degrades as expected).

Disabled by default; passwords are read from sync.imap.passwords keyed by
username so no credential belongs in the tracked config. An account with no
password is skipped with a warning instead of failing the whole sync.

25 tests pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant