diff --git a/Cargo.lock b/Cargo.lock index a4e6268c..7487360e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3581,7 +3581,6 @@ dependencies = [ "tempfile", "thiserror 2.0.12", "tokio", - "ureq", "uuid", ] diff --git a/Makefile b/Makefile index 07dc0075..98e0b2e6 100644 --- a/Makefile +++ b/Makefile @@ -395,7 +395,7 @@ docs-clean: clean: rm -rf dist docs/site probing/server/web-assets web/dist web/target - rm -rf python/probing/libs python/probing/shim/hccl + rm -rf python/probing/bundled_web python/probing/libs python/probing/shim/hccl rm -rf .pytest_cache .coverage coverage.xml coverage.lcov cargo clean rm -f coverage.lcov coverage.json diff --git a/docs/src/design/federation.md b/docs/src/design/federation.md index 6ed1580a..c28988ce 100644 --- a/docs/src/design/federation.md +++ b/docs/src/design/federation.md @@ -233,17 +233,20 @@ flowchart LR RW --> EX[Execute shards] EX --> TG[Inject federation tags] TG --> MG[Merge / re-aggregate] - MG --> OUT[DataFrame + FanoutMeta] + MG --> OUT[QueryOutcome DataFrame + QueryQuality] ``` **Conventions** | Item | Semantics | |------|-----------| -| Response | `dataframe` + `meta.nodes_queried` + `meta.nodes_failed` | +| Core result | `QueryOutcome` always carries `data` and typed `QueryQuality`; partialness is not a task-local side channel | +| HTTP response | `dataframe` + `meta.nodes_queried` + `meta.nodes_failed` | | Peer set | Snapshot of `cluster.nodes`, **excluding** coordinator listen addrs | | Peer execution | Always `probe.*`; peers must not fan-out again | -| Concurrency | Parallel peer requests; latency ≈ slowest peer + coordinator merge | +| Execution | SQL, extension capture, node discovery, and heartbeat HTTP share one async `FanoutService` | +| Isolation | Peer network I/O runs on a dedicated Tokio runtime (`PROBING_FANOUT_WORKER_THREADS`), separate from Axum/DataFusion caller workers | +| Concurrency | Shared bounded peer requests; latency ≈ slowest peer + coordinator merge | | Timeout | Failed peer → `nodes_failed`; default **30s** (`PROBING_REMOTE_QUERY_TIMEOUT_SECS`) | | Strict fan-out | `PROBING_FANOUT_STRICT=1` fails query on any `nodes_failed` or `peer_batches_dropped` (no partial merge) | | Partial HTTP | Cluster/fan-out APIs return **503** with partial `dataframe` when `meta.partial=true` (non-strict only) | diff --git a/docs/src/design/federation.zh.md b/docs/src/design/federation.zh.md index 4bdcc92e..3148c1de 100644 --- a/docs/src/design/federation.zh.md +++ b/docs/src/design/federation.zh.md @@ -240,17 +240,20 @@ flowchart LR RW --> EX[各分片执行] EX --> TG[注入联邦标签] TG --> MG[合并 / 二次聚合] - MG --> OUT[DataFrame + FanoutMeta] + MG --> OUT[QueryOutcome DataFrame + QueryQuality] ``` **统一约定** | 项 | 语义 | |----|------| -| 响应体 | `dataframe` + `meta.nodes_queried` + `meta.nodes_failed` | +| Core 返回值 | `QueryOutcome` 始终同时携带 `data` 与类型化 `QueryQuality`;partial 不再是 task-local 旁路状态 | +| HTTP 响应体 | `dataframe` + `meta.nodes_queried` + `meta.nodes_failed` | | Peer 集合 | `cluster.nodes` 快照,**排除** coordinator 自身 listen addr | | Peer 执行 | 永远 `probe.*`;禁止 peer 再 fan-out(防递归) | -| 并发 | 各 peer 并行请求;总延迟 ≈ 最慢 peer + coordinator 合并 | +| 统一执行 | SQL、Extension 采集、节点发现和心跳 HTTP 共用一个异步 `FanoutService` | +| 资源隔离 | Peer 网络 I/O 使用独立 Tokio runtime(`PROBING_FANOUT_WORKER_THREADS`),不占用 Axum/DataFusion 调用线程 | +| 并发 | 共用有界并发;总延迟 ≈ 最慢 peer + coordinator 合并 | | 超时 | 失败 peer → `nodes_failed`;默认 **30s**(`PROBING_REMOTE_QUERY_TIMEOUT_SECS`) | | 严格 fan-out | `PROBING_FANOUT_STRICT=1`:任一 `nodes_failed` 或 `peer_batches_dropped` 则整查失败(无 partial merge) | | Partial HTTP | 集群/fan-out API 在 `meta.partial=true` 时返回 **503** 及 partial `dataframe`(非 strict 模式) | diff --git a/docs/src/design/modularity.md b/docs/src/design/modularity.md index a2c0ab82..742dd420 100644 --- a/docs/src/design/modularity.md +++ b/docs/src/design/modularity.md @@ -156,20 +156,22 @@ New work should extend **one** of these contracts instead of adding cross-module `global..`. - Do **not** query other collectors from inside `scan()`; join at SQL layer. -### 3.2 `ProbeExtension` — config + imperative HTTP +### 3.2 `ProbeExtension` — typed config + HTTP contracts **Where:** `probing/core/src/core/probe_extension.rs` **Derive:** `#[derive(ProbeExtension)]` in `probing-macros` | Capability | Mechanism | |------------|-----------| -| Config keys | `probing..