Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
282 changes: 212 additions & 70 deletions crates/persisting-pchronicle-cli/src/server/catalog.rs

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion crates/persisting-pchronicle-cli/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -664,7 +664,7 @@ async fn serve_catalog_issue_grant_revoke_rewrites_config() -> Result<()> {
fs::write(
&catalog,
r#"
[libraries.prod]
[datasets.prod]
uri = "s3://bucket/prod"
access_key = "BACKEND_AK"
secret_key = "BACKEND_SK"
Expand Down
2 changes: 2 additions & 0 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,8 @@ nav:
- pChronicle reference: pchronicle/reference/index.md
- Product terminology: pchronicle/reference/terminology.md
- pChronicle CLI: pchronicle/reference/cli.md
- Single-machine and self-service cases: pchronicle/reference/cases-self.md
- Cluster platform and Catalog Server cases: pchronicle/reference/cases-platform.md
- Query model: pchronicle/reference/query-model.md
- AgenticMD format: pchronicle/reference/agenticmd.md
- Run data formats: pchronicle/reference/formats/index.md
Expand Down
92 changes: 92 additions & 0 deletions docs/src/pchronicle/reference/cases-platform.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# pChronicle 集群平台与 Catalog Server 场景

本文覆盖平台化部署。Catalog 配置只管理用户、Dataset 和授权;Warehouse 的服务参数仍由 `pchronicle serve` 提供。

## P01:从空配置创建 Catalog 用户

```bash
pchronicle serve catalog user create \
--catalog-config ./catalog.toml alice
```

如果文件不存在,命令创建配置文件、生成用户 AK/SK,并只在本次输出 secret。

## P02:登记 Dataset

```bash
pchronicle serve catalog dataset create \
--catalog-config ./catalog.toml \
prod s3://bucket/prod \
--endpoint http://127.0.0.1:9000 \
--region us-west-2 \
--ak BACKEND_AK \
--sk BACKEND_SK
```

该命令只登记 Dataset,不创建或删除后端数据。

## P03:授权用户

```bash
pchronicle serve catalog grant \
--catalog-config ./catalog.toml \
alice prod \
--permission read \
--permission query \
--permission analyze
```

预期:配置中出现独立的 `[[grants]]` 记录。

## P04:启动 Catalog Server

```bash
pchronicle serve \
--catalog-config ./catalog.toml \
--listen 127.0.0.1:8081
```

父进程负责用户认证、Dataset 列表和 ticket;查询数据面在授权 mounts 的 worker 中执行。

## P05:访问授权 Dataset

```bash
pchronicle alias add team catalog://127.0.0.1:8081 \
--ak USER_AK --sk USER_SK
pchronicle query @team/prod \
--sql 'SELECT COUNT(*) AS runs FROM dataset.runs'
```

预期:授权用户可以查询 `prod`;未授权用户或未知 Dataset 返回相同的 404 资源错误。

## P06:撤销授权

```bash
pchronicle serve catalog revoke \
--catalog-config ./catalog.toml \
alice prod --permission query
```

预期:后续查询被拒绝,但 `read` 和其它仍保留的权限不受影响。

## P07:RustFS Warehouse 回归

准备 RustFS,并设置:

```bash
export PCHRONICLE_RUSTFS_ENDPOINT=http://127.0.0.1:9000
export PCHRONICLE_RUSTFS_ACCESS_KEY=rustfsadmin
export PCHRONICLE_RUSTFS_SECRET_KEY=rustfsadmin
export PCHRONICLE_RUSTFS_BUCKET=pchronicle-cases
```

然后运行 RustFS 回归测试,验证 Dataset 写入、Catalog discovery、SQL 查询、Explorer 和 refresh 行为。

平台验收重点:

- Catalog 文件可从空文件开始构建;
- 用户、Dataset 和 grants 修改是确定性的;
- Dataset 后端凭据只在授权 ticket 中使用;
- Worker 只收到当前用户被授权的 mounts;
- Catalog refresh 不影响已完成查询的 snapshot;
- RustFS 上的 Warehouse 行为与本地 Dataset 一致。
92 changes: 92 additions & 0 deletions docs/src/pchronicle/reference/cases-platform.zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# pChronicle 集群平台与 Catalog Server 场景

本文覆盖平台化部署。Catalog 配置只管理用户、Dataset 和授权;Warehouse 的服务参数仍由 `pchronicle serve` 提供。

## P01:从空配置创建 Catalog 用户

```bash
pchronicle serve catalog user create \
--catalog-config ./catalog.toml alice
```

如果文件不存在,命令创建配置文件、生成用户 AK/SK,并只在本次输出 secret。

## P02:登记 Dataset

```bash
pchronicle serve catalog dataset create \
--catalog-config ./catalog.toml \
prod s3://bucket/prod \
--endpoint http://127.0.0.1:9000 \
--region us-west-2 \
--ak BACKEND_AK \
--sk BACKEND_SK
```

该命令只登记 Dataset,不创建或删除后端数据。

## P03:授权用户

```bash
pchronicle serve catalog grant \
--catalog-config ./catalog.toml \
alice prod \
--permission read \
--permission query \
--permission analyze
```

预期:配置中出现独立的 `[[grants]]` 记录。

## P04:启动 Catalog Server

```bash
pchronicle serve \
--catalog-config ./catalog.toml \
--listen 127.0.0.1:8081
```

父进程负责用户认证、Dataset 列表和 ticket;查询数据面在授权 mounts 的 worker 中执行。

## P05:访问授权 Dataset

```bash
pchronicle alias add team catalog://127.0.0.1:8081 \
--ak USER_AK --sk USER_SK
pchronicle query @team/prod \
--sql 'SELECT COUNT(*) AS runs FROM dataset.runs'
```

预期:授权用户可以查询 `prod`;未授权用户或未知 Dataset 返回相同的 404 资源错误。

## P06:撤销授权

```bash
pchronicle serve catalog revoke \
--catalog-config ./catalog.toml \
alice prod --permission query
```

预期:后续查询被拒绝,但 `read` 和其它仍保留的权限不受影响。

## P07:RustFS Warehouse 回归

准备 RustFS,并设置:

```bash
export PCHRONICLE_RUSTFS_ENDPOINT=http://127.0.0.1:9000
export PCHRONICLE_RUSTFS_ACCESS_KEY=rustfsadmin
export PCHRONICLE_RUSTFS_SECRET_KEY=rustfsadmin
export PCHRONICLE_RUSTFS_BUCKET=pchronicle-cases
```

然后运行 RustFS 回归测试,验证 Dataset 写入、Catalog discovery、SQL 查询、Explorer 和 refresh 行为。

平台验收重点:

- Catalog 文件可从空文件开始构建;
- 用户、Dataset 和 grants 修改是确定性的;
- Dataset 后端凭据只在授权 ticket 中使用;
- Worker 只收到当前用户被授权的 mounts;
- Catalog refresh 不影响已完成查询的 snapshot;
- RustFS 上的 Warehouse 行为与本地 Dataset 一致。
65 changes: 65 additions & 0 deletions docs/src/pchronicle/reference/cases-self.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# pChronicle 单机与自助使用场景

本文覆盖不依赖 Catalog Server 的基础工作流。每个案例都可以在一台开发机上独立执行,Dataset 可以是本地目录或对象存储 URI。

## 准备

```bash
mkdir -p /tmp/pchronicle-cases
cd /tmp/pchronicle-cases
pchronicle onboard
```

## S01:浏览本地 Dataset

```bash
pchronicle ls ./trajectory-data
pchronicle status ./trajectory-data
```

预期:命令列出 Dataset 中的 runs、steps 和 tool calls;空 Dataset 返回明确的空结果。

## S02:执行 SQL 查询

```bash
pchronicle query ./trajectory-data \
--sql 'SELECT COUNT(*) AS runs FROM dataset.runs'
```

预期:查询成功并返回确定的 runs 数量。

## S03:运行内建分析

```bash
pchronicle analysis overview ./trajectory-data
```

预期:输出运行数、步骤数、工具调用数和时间范围。

## S04:导入和导出

```bash
pchronicle import input.jsonl --output ./trajectory-data
pchronicle export ./trajectory-data --output output.jsonl
```

预期:导出内容可以再次导入,记录的 ID 和事件顺序保持一致。

## S05:本地 Warehouse

```bash
pchronicle serve ./trajectory-data --listen 127.0.0.1:8081
```

预期:Web UI、`/api/query/tables`、`/api/catalog` 和 Explorer API 可用;未启用 Catalog 时不需要用户凭据。

## S06:对象存储 Dataset

```bash
export AWS_ENDPOINT_URL_S3=http://127.0.0.1:9000
export AWS_ACCESS_KEY_ID=rustfsadmin
export AWS_SECRET_ACCESS_KEY=rustfsadmin
pchronicle ls s3://bucket/trajectory
```

预期:pChronicle 通过 S3 兼容接口发现并查询 Dataset。endpoint 和凭据不会写入 Dataset URI。
65 changes: 65 additions & 0 deletions docs/src/pchronicle/reference/cases-self.zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# pChronicle 单机与自助使用场景

本文覆盖不依赖 Catalog Server 的基础工作流。每个案例都可以在一台开发机上独立执行,Dataset 可以是本地目录或对象存储 URI。

## 准备

```bash
mkdir -p /tmp/pchronicle-cases
cd /tmp/pchronicle-cases
pchronicle onboard
```

## S01:浏览本地 Dataset

```bash
pchronicle ls ./trajectory-data
pchronicle status ./trajectory-data
```

预期:命令列出 Dataset 中的 runs、steps 和 tool calls;空 Dataset 返回明确的空结果。

## S02:执行 SQL 查询

```bash
pchronicle query ./trajectory-data \
--sql 'SELECT COUNT(*) AS runs FROM dataset.runs'
```

预期:查询成功并返回确定的 runs 数量。

## S03:运行内建分析

```bash
pchronicle analysis overview ./trajectory-data
```

预期:输出运行数、步骤数、工具调用数和时间范围。

## S04:导入和导出

```bash
pchronicle import input.jsonl --output ./trajectory-data
pchronicle export ./trajectory-data --output output.jsonl
```

预期:导出内容可以再次导入,记录的 ID 和事件顺序保持一致。

## S05:本地 Warehouse

```bash
pchronicle serve ./trajectory-data --listen 127.0.0.1:8081
```

预期:Web UI、`/api/query/tables`、`/api/catalog` 和 Explorer API 可用;未启用 Catalog 时不需要用户凭据。

## S06:对象存储 Dataset

```bash
export AWS_ENDPOINT_URL_S3=http://127.0.0.1:9000
export AWS_ACCESS_KEY_ID=rustfsadmin
export AWS_SECRET_ACCESS_KEY=rustfsadmin
pchronicle ls s3://bucket/trajectory
```

预期:pChronicle 通过 S3 兼容接口发现并查询 Dataset。endpoint 和凭据不会写入 Dataset URI。
21 changes: 20 additions & 1 deletion docs/src/pchronicle/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ pchronicle serve \
Every listener must use a loopback address. A bare single Dataset is mounted as
`default`; with several Datasets, use `NAME=DATASET` when a stable mount name is
needed. Control requires a mount named `default`.
`--catalog-config FILE` serves a path Directory instead of opening libraries
`--catalog-config FILE` serves a path Directory instead of opening Datasets
in the parent process. Pair it with `alias add NAME catalog://127.0.0.1:PORT --ak --sk`.
`pchronicle serve catalog issue|grant|revoke` rewrites that file and does not
start HTTP; `issue` prints the user secret once. Restart serve after changing
Expand Down Expand Up @@ -308,3 +308,22 @@ Use [Discover and query](../guides/discover-and-query.md) for the locate-then-SQ
workflow, [Import and export](../guides/exchange.md) for interchange, and
[Serve Datasets locally](../guides/serve.md) for the read-only server. Snapshot
construction is explained in [Snapshot design](../design/catalog.md).

#### Catalog management

Catalog configuration contains only users, Datasets, and grants. Management commands create the file when it does not exist.

```text
pchronicle serve catalog user create --catalog-config FILE NAME
pchronicle serve catalog user list --catalog-config FILE
pchronicle serve catalog user remove --catalog-config FILE NAME
pchronicle serve catalog dataset create --catalog-config FILE NAME URI [OPTIONS]
pchronicle serve catalog dataset list --catalog-config FILE
pchronicle serve catalog dataset show --catalog-config FILE NAME
pchronicle serve catalog dataset remove --catalog-config FILE NAME
pchronicle serve catalog grant --catalog-config FILE USER DATASET --permission PERMISSION...
pchronicle serve catalog revoke --catalog-config FILE USER DATASET --permission PERMISSION...
pchronicle serve catalog grants --catalog-config FILE
```

`user create` generates AK/SK and prints the secret once. `dataset create` registers the URI and storage credentials without creating or deleting backend data. `grant` and `revoke` manage `read`, `query`, `analyze`, `write`, and `admin` permissions.
Loading
Loading