[server] Extend the robust resync logic to regular cluster log sync #1924

[michaelvlach]

Introduce a /cluster/logs endpoint and log-stream format so followers can catch up missing raft entries before falling back to full snapshot sync. Add append failure tracking in raft (with a force-resync heartbeat flag after repeated append failures) to proactively trigger follower resync when a node is persistently unreachable. Also add cluster_max_chunk_size configuration (default 65,536) and wire it into snapshot/log streaming and extraction paths, updating tests and config initializers accordingly.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
agdb	Ready	Preview	Sep 9, 2026 10:33pm UTC

[Copilot]

🟡 Changes recommended

The new log catch-up implementation has correctness and scalability issues (missing stream/index validation and full-response buffering) that should be addressed before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR extends agdb_server’s cluster resync mechanism by adding a log catch-up path (via a new /cluster/logs streaming endpoint) so followers can fetch missing raft entries before falling back to a full snapshot install, and introduces append-failure tracking to proactively trigger resync. It also adds a new cluster_max_chunk_size configuration knob and wires it into snapshot/log streaming and snapshot extraction.

Changes:

• Add /api/v1/cluster/logs endpoint with a binary, chunked log-stream format for catch-up.
• Implement “log catch-up first, snapshot fallback” in follower resync + add append-failure thresholding to trigger a force-resync heartbeat.
• Introduce cluster_max_chunk_size (default 65,536) and propagate it through server/app/test config initializers and streaming/extraction buffers.

File summaries

File	Description
agdb_server/tests/tls/mod.rs	Adds cluster_max_chunk_size to test config initialization.
agdb_server/tests/routes/misc_routes.rs	Updates multiple route tests to include cluster_max_chunk_size in config.
agdb_server/tests/routes/cluster_test_extra.rs	Adds an integration test covering log catch-up after repeated append failures.
agdb_server/src/routes/cluster.rs	Adds /cluster/logs handler + threads cluster_max_chunk_size into snapshot streaming.
agdb_server/src/raft.rs	Adds append-failure tracking and a force_resync flag carried by heartbeats; updates tests accordingly.
agdb_server/src/config.rs	Parses cluster_max_chunk_size and includes it in default config + config tests.
agdb_server/src/cluster.rs	Implements log catch-up download/apply path before snapshot resync; wires chunk size into snapshot extraction.
agdb_server/src/cluster_log.rs	Updates cluster log tests’ config initialization for the new config field.
agdb_server/src/app.rs	Registers the new /cluster/logs route.
agdb_api/src/test_server/test_cluster.rs	Updates test cluster server config to include cluster_max_chunk_size.
agdb_api/src/test_server.rs	Updates test server config; fixes next_port() overflow behavior via wrapping_add.
agdb_api/src/api_types/config_impl.rs	Adds DEFAULT_CLUSTER_MAX_CHUNK_SIZE and the cluster_max_chunk_size config field + serialization.

Review details

• Files reviewed: 12/12 changed files
• Comments generated: 5
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.