diff --git a/README.md b/README.md index bd9d1a1..1c7a3ce 100644 --- a/README.md +++ b/README.md @@ -22,9 +22,10 @@ single-process path bypasses all transport overhead. pip install eventforge ``` -Core install pulls only `pydantic` + `typing-extensions`. The TCP -transport is stdlib. Optional integrations (Pydantic Logfire, etc.) -have their own extras documented in `docs/`. +Core install pulls only `pydantic` + `typing-extensions`. The Memory and +TCP transports are stdlib. Optional extras: `eventforge[redis]` and +`eventforge[nats]` add the broker-backed transports; `eventforge[logfire]` +adds Pydantic Logfire emission. ## Concepts @@ -54,7 +55,8 @@ the network on a TCP transport. `@task` composes an `Executor` + an optional ## Transport Layer -A `Transport` is the wire under a `MessageQueue`. Two ship in the box: +A `Transport` is the wire under a `MessageQueue`. Two ship in the core; two +more are available behind optional extras: - `MemoryTransport` -- in-process, thread-safe, the default. Zero serialization, zero sockets. @@ -62,6 +64,16 @@ A `Transport` is the wire under a `MessageQueue`. Two ship in the box: length-prefixed JSON over a TCP socket (`[4-byte big-endian length][JSON]`), for cross-process / cross-machine delivery. The server must be `.start()`ed; the client must be `.connect()`ed. +- `RedisTransport` -- broker-backed pub/sub over Redis `PUBLISH`/`PSUBSCRIBE`. + `pip install eventforge[redis]`, then + `from eventforge.transports.redis import RedisTransport`. +- `NatsTransport` -- broker-backed pub/sub over NATS. + `pip install eventforge[nats]`, then + `from eventforge.transports.nats import NatsTransport`. + +The two optional transports lazy-import their dependency, so a plain +`import eventforge` never requires `redis` or `nats`. They serialize the same +JSON wire as TCP. All transports implement one small ABC, so they are interchangeable under `MessageQueue`, `RPCServer`, and `RPCClient`: @@ -90,10 +102,15 @@ local = MessageQueue() transport = TCPServerTransport(host="127.0.0.1", port=9090) transport.start() networked = MessageQueue(transport=transport) + +# broker-backed: same MessageQueue API again +from eventforge.transports.redis import RedisTransport +brokered = MessageQueue(transport=RedisTransport(host="localhost", port=6379)) ``` -Because the ABC is this small, you could add a Redis or NATS transport by -implementing those six methods -- nothing else in the stack needs to know. +The `RedisTransport` and `NatsTransport` above are exactly this pattern; the +ABC is small enough that adding another (Kafka, ZeroMQ, ...) is just those six +methods -- nothing above the transport needs to know. ## Messaging diff --git a/docs/index.md b/docs/index.md index 263c9d6..b8632ac 100644 --- a/docs/index.md +++ b/docs/index.md @@ -30,8 +30,10 @@ lives. `@task` composes an `Executor` + a `MessageQueue` + `Observers`. pip install eventforge ``` -Core install pulls only `pydantic` + `typing-extensions`. The TCP transport -is stdlib. +Core install pulls only `pydantic` + `typing-extensions`. The Memory and TCP +transports are stdlib. Optional extras: `eventforge[redis]` / +`eventforge[nats]` for the broker-backed transports, `eventforge[logfire]` +for Logfire emission. ## Quick Start @@ -225,6 +227,8 @@ eventforge/ base.py # Transport ABC memory.py # In-memory transport (default) tcp.py # JSON-over-TCP server/client transports + redis.py # Redis pub/sub transport (optional: [redis]) + nats.py # NATS pub/sub transport (optional: [nats]) queue.py # MessageQueue with pub-sub remote.py # RemoteQueue push-only switchboard executor.py # Executor (local request-reply) diff --git a/docs/rpc.md b/docs/rpc.md index ce269aa..2ee9750 100644 --- a/docs/rpc.md +++ b/docs/rpc.md @@ -355,7 +355,7 @@ class RPCResponse(BaseModel): ```python from eventforge import MessageQueue, RPCServer, RPCClient -# Shared queue (in production, use Redis/ZMQ transport) +# Shared queue (in production, use a TCP, Redis, or NATS transport) queue = MessageQueue() # User service diff --git a/docs/task.md b/docs/task.md index 9c8b1a0..fda1379 100644 --- a/docs/task.md +++ b/docs/task.md @@ -134,7 +134,7 @@ Disable with `publish_result=False`. topic=None, # Topic name (defaults to function name) executor=None, # Executor (default) -- how the body runs # in-process (inline / thread / process) - on_execute=None, # List of Observer instances + on_execute=None, # list of observers (AvgMeters, etc.) on_success=None, # Callable[[TaskContext], None] on_failure=None, # Callable[[TaskContext], None] on_complete=None, # Callable[[TaskContext], None] @@ -151,7 +151,7 @@ Disable with `publish_result=False`. | `queue` | `MessageQueue` | `None` | Queue for pub-sub. If provided with topic, subscribes automatically | | `topic` | `str` | Function name | Topic for queue subscription and result publishing | | `executor` | `Executor` | `Executor()` | Local execution backend: how the body runs in-process (inline / thread / process) | -| `on_execute` | `List[Observer]` | `[]` | Observers called on start/end/error | +| `on_execute` | `list` | `[]` | Observers (AvgMeters, etc.) wired to the start / success / failure / complete lifecycle | | `on_success` | `Callable` | `None` | Called on successful execution | | `on_failure` | `Callable` | `None` | Called on failed execution | | `on_complete` | `Callable` | `None` | Called after execution (success or failure) |