feat(cloud): publish archives with explicit admission evidence

CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## Unreleased
 
+- Add explicit `cloud publish --admission-policy=archive-v1 --observation-order` with frozen source, integrity, and enrichment evidence and immutable warnings. Preserve strict publication defaults, canonical archive contents, privacy, and existing size limits.
 - Preserve newly fetched GitHub numeric IDs exactly across REST and GraphQL syncs, including IDs above the floating-point precision limit. Previously stored IDs are not repaired.
 
 ## 0.9.5 - 2026-09-09

docs/cloud-archives.md

@@ -91,12 +91,58 @@ gitcrawl cloud publish \
   --json
 ```
 
-A later publish verifies the candidate through the publisher-only status projection. It skips repeated ingest only when the digest, source sync, schema, resolved publication profile, generation timestamp, and coverage match.
+A later publish verifies the candidate through the publisher-only status projection. It skips repeated ingest only when the digest, source sync, schema, resolved publication profile, immutable warnings, generation timestamp, and coverage match.
 
 Cutover requires reader-authenticated `GET /sqlite`. Gitcrawl validates the cutover acknowledgement, polls the scoped reader projection until its digest, profile, generation, and dataset coverage match, rechecks the publisher metadata, downloads the bound SQLite image, and verifies its hash before reporting success. Without `--stage-only`, a successful publish moves unpinned reads to the complete snapshot.
 
 Incomplete local enrichment fails before remote mutation. `--allow-incomplete` is the explicit override. `--observation-order` publishes durable fetch ordering only after the remote operator fence is enabled.
 
+## Archive admission
+
+To publish a raw archive with explicitly reported source and enrichment gaps:
+
+```bash
+gitcrawl cloud publish \
+  --remote URL \
+  --archive gitcrawl/openclaw__openclaw \
+  --admission-policy=archive-v1 \
+  --observation-order \
+  --stage-only \
+  --json
+```
+
+This opt-in policy requires the remote capabilities
+`gitcrawl.archive-admission.v1` and `gitcrawl.observation-order.v1`.
+An older remote or a disabled observation fence fails preflight before upload.
+Do not combine the policy with `--allow-incomplete`. Omitting the policy retains
+the strict default.
+
+Archive admission still requires SQLite integrity, compatible canonical tables,
+repositories, referential closure, full bodies, and the existing privacy scrub.
+Use the full runtime archive, not a lossy portable export. Native portable
+profiles that declare excerpts or excluded patches/history are rejected even
+when current bodies fit the excerpt limit. Empty patch rows in a full runtime
+are not proof of loss. The existing
+4 GiB SQLite, 512 MiB gzip, eight-part, staging, and cutover limits still apply.
+Admission does not request summaries, embeddings, or other model work.
+
+The JSON result includes typed `admission` evidence and stable `warnings`.
+Repository inventory observations distinguish unsupported, unknown, missing,
+partial, complete, and empty observations. Child observations also report stale
+reservations relative to the archived thread. Complete observations describe
+the recorded scope, not current GitHub state. Workflow freshness and current
+remote freshness remain unknown; export clocks never establish either.
+Source gaps, PR detail/file gaps, and all six enrichment metrics remain visible.
+Incomplete current revisions retain their actual coverage counts and
+`complete: false`; admission does not make stale enrichment current for readers.
+
+The sanitized SQLite copy stores the same deterministic evidence in
+`portable_metadata.cloud_admission_v1` before hashing. Existing source markers
+are not trusted, and strict exports remove them. The original database is not
+changed. Canonically sorted warnings are part of immutable snapshot identity,
+including stage-only replay and reader verification. Warnings alone never
+enable admission.
+
 ## Privacy and retention
 
 Digest-scoped bundles can contain private issue and pull request text. Bundle metadata declares both message-body and source-code sensitivity because patch text is retained.

go.mod

@@ -9,7 +9,7 @@ require (
 	github.com/charmbracelet/lipgloss v1.1.0
 	github.com/charmbracelet/x/ansi v0.11.8
 	github.com/mattn/go-isatty v0.0.24
-	github.com/openclaw/crawlkit v0.15.0
+	github.com/openclaw/crawlkit v0.15.1
 	github.com/zalando/go-keyring v0.2.8
 	golang.org/x/sys v0.47.0
 	modernc.org/sqlite v1.58.0

go.sum

@@ -62,8 +62,8 @@ github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc
 github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
 github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
 github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
-github.com/openclaw/crawlkit v0.15.0 h1:KAmdew2UQPZm/gOfqiLw/WhhDY7Y2UZj0EcgQ5s7dl4=
-github.com/openclaw/crawlkit v0.15.0/go.mod h1:JhEsXnoxozd2qp1p4Dw8ZhKzQoYm3WbrlvO0ILVx8cs=
+github.com/openclaw/crawlkit v0.15.1 h1:B281AyJgVCTj7bMuONW57DXNV4sQyj9F5LLEaUc71l0=
+github.com/openclaw/crawlkit v0.15.1/go.mod h1:JhEsXnoxozd2qp1p4Dw8ZhKzQoYm3WbrlvO0ILVx8cs=
 github.com/pelletier/go-toml/v2 v2.4.3 h1:GTRvJQutkOSftxIFD5xw9aepkYNuPWmVJpffdDPYVpY=
 github.com/pelletier/go-toml/v2 v2.4.3/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=

internal/cli/app.go

@@ -5406,7 +5406,7 @@ Usage:
 	"cloud": `gitcrawl cloud manages Worker-backed remote archives.
 
 Usage:
-  gitcrawl cloud publish --remote URL --archive id [--allow-incomplete] [--observation-order] [--stage-only] [--json]
+  gitcrawl cloud publish --remote URL --archive id [--allow-incomplete | --admission-policy=archive-v1] [--observation-order] [--stage-only] [--json]
 `,
 	"whoami": `gitcrawl whoami prints the configured remote archive identity.
 

internal/cli/app_test.go

@@ -437,7 +437,7 @@ func TestCloudSQLiteSnapshotDropsLocalCodeCorpus(t *testing.T) {
 		`, repoID, repoID, repoID, repoID, repoID, repoID); err != nil {
 		t.Fatalf("seed private cloud payloads: %v", err)
 	}
-	snapshotPath, cleanup, err := cloudSQLiteSnapshotPath(ctx, st.DB(), dbPath)
+	snapshotPath, _, cleanup, err := cloudSQLiteSnapshotPath(ctx, st.DB(), dbPath, gitcrawlCloudPublishOptions{})
 	if err != nil {
 		t.Fatalf("cloud snapshot: %v", err)
 	}
@@ -1083,6 +1083,11 @@ func TestGitcrawlPublisherStatusMatchesExactMetadata(t *testing.T) {
 }
 
 func TestCloudPublishSendsLocalRows(t *testing.T) {
+	t.Run("legacy allow incomplete", func(t *testing.T) { testCloudPublishSendsLocalRows(t, false) })
+	t.Run("archive admission", func(t *testing.T) { testCloudPublishSendsLocalRows(t, true) })
+}
+
+func testCloudPublishSendsLocalRows(t *testing.T, archiveAdmission bool) {
 	ctx := context.Background()
 	dir := t.TempDir()
 	cfgPath := filepath.Join(dir, "config.toml")
@@ -1107,10 +1112,21 @@ func TestCloudPublishSendsLocalRows(t *testing.T) {
 	var publishedDatasets []crawlremote.DatasetCoverage
 	var publisherStatusSnapshotIDs []string
 	mutationCounter := 0
+	uploadRequests := 0
 	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		if r.Method == http.MethodGet && r.URL.EscapedPath() == "/v1/contract" {
 			w.Header().Set("content-type", "application/json")
-			_ = json.NewEncoder(w).Encode(testSnapshotPublishContract())
+			contract := testSnapshotPublishContract()
+			if archiveAdmission {
+				contract.Apps[0].Capabilities = append(contract.Apps[0].Capabilities, gitcrawlArchiveAdmissionCapability, gitcrawlObservationOrderCapability)
+				for index := range contract.Apps[0].IngestTables {
+					table := &contract.Apps[0].IngestTables[index]
+					if table.Name == "threads" || table.Name == "thread_revisions" {
+						table.Columns = append(table.Columns, "observation_sequence")
+					}
+				}
+			}
+			_ = json.NewEncoder(w).Encode(contract)
 			return
 		}
 		if got := r.Header.Get("authorization"); got != "Bearer publish-token" {
@@ -1127,6 +1143,7 @@ func TestCloudPublishSendsLocalRows(t *testing.T) {
 			return
 		}
 		if r.Method == http.MethodPut && r.URL.EscapedPath() == "/v1/apps/gitcrawl/archives/gitcrawl%2Fopenclaw__openclaw/sqlite" {
+			uploadRequests++
 			uploadKind := r.Header.Get("x-crawl-sqlite-upload")
 			payload, err := io.ReadAll(r.Body)
 			if err != nil {
@@ -1338,6 +1355,7 @@ func TestCloudPublishSendsLocalRows(t *testing.T) {
 				SourceSyncAt:       body.Manifest.SourceSyncAt,
 				DatasetGeneratedAt: fmt.Sprint(body.Rows[0][5]),
 				CoverageComplete:   true,
+				Warnings:           slices.Clone(body.Manifest.Warnings),
 			}
 		}
 		seenTables[body.Table] = body
@@ -1360,15 +1378,20 @@ func TestCloudPublishSendsLocalRows(t *testing.T) {
 	app := New()
 	var out bytes.Buffer
 	app.Stdout = &out
-	if err := app.Run(ctx, []string{
+	args := []string{
 		"--config", cfgPath,
 		"cloud", "publish",
 		"--remote", server.URL,
 		"--archive", "gitcrawl/openclaw__openclaw",
 		"--token-env", tokenEnv,
-		"--allow-incomplete",
 		"--json",
-	}); err != nil {
+	}
+	if archiveAdmission {
+		args = append(args, "--admission-policy=archive-v1", "--observation-order")
+	} else {
+		args = append(args, "--allow-incomplete")
+	}
+	if err := app.Run(ctx, args); err != nil {
 		t.Fatalf("cloud publish: %v", err)
 	}
 
@@ -1425,6 +1448,24 @@ func TestCloudPublishSendsLocalRows(t *testing.T) {
 		privacy["includes_source_code"] != true {
 		t.Fatalf("missing sqlite bundle privacy output: %#v", payload)
 	}
+	if archiveAdmission {
+		if len(publishedSnapshot.Warnings) == 0 || payload["admission"] == nil {
+			t.Fatal("archive publication lost persistent admission evidence")
+		}
+		for _, dataset := range publishedDatasets {
+			if dataset.Dataset == "thread_revisions" && (dataset.EligibleCount != 3 || dataset.Complete) {
+				t.Fatalf("archive admission hid incomplete revision coverage: %+v", dataset)
+			}
+		}
+	}
+	beforeMutations, beforeUploads := mutationCounter, uploadRequests
+	out.Reset()
+	if err := app.Run(ctx, append(args, "--stage-only")); err != nil {
+		t.Fatalf("stage-only replay: %v", err)
+	}
+	if mutationCounter != beforeMutations || uploadRequests != beforeUploads {
+		t.Fatal("identical staged snapshot was uploaded or ingested again")
+	}
 }
 
 func TestCloudPublishRejectsMissingSnapshotCapabilityBeforeUpload(t *testing.T) {
@@ -1480,7 +1521,20 @@ func TestCloudPublishRejectsMissingRequestedCapabilityBeforeUpload(t *testing.T)
 		name              string
 		args              []string
 		missingCapability string
+		archiveAdmission  bool
 	}{
+		{
+			name:              "archive admission",
+			args:              []string{"--admission-policy=archive-v1", "--observation-order", "--stage-only"},
+			missingCapability: gitcrawlArchiveAdmissionCapability,
+			archiveAdmission:  true,
+		},
+		{
+			name:              "archive observation fence",
+			args:              []string{"--admission-policy=archive-v1", "--observation-order", "--stage-only"},
+			missingCapability: gitcrawlObservationOrderCapability,
+			archiveAdmission:  true,
+		},
 		{
 			name:              "observation order",
 			args:              []string{"--observation-order", "--stage-only"},
@@ -1515,6 +1569,15 @@ func TestCloudPublishRejectsMissingRequestedCapabilityBeforeUpload(t *testing.T)
 			server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 				if r.Method == http.MethodGet && r.URL.EscapedPath() == "/v1/contract" {
 					contract := testSnapshotPublishContract()
+					if test.archiveAdmission {
+						contract.Apps[0].Capabilities = append(contract.Apps[0].Capabilities, gitcrawlArchiveAdmissionCapability, gitcrawlObservationOrderCapability)
+						for index := range contract.Apps[0].IngestTables {
+							table := &contract.Apps[0].IngestTables[index]
+							if table.Name == "threads" || table.Name == "thread_revisions" {
+								table.Columns = append(table.Columns, "observation_sequence")
+							}
+						}
+					}
 					capabilities := make([]string, 0, len(contract.Apps[0].Capabilities))
 					for _, capability := range contract.Apps[0].Capabilities {
 						if capability != test.missingCapability {
@@ -1537,9 +1600,11 @@ func TestCloudPublishRejectsMissingRequestedCapabilityBeforeUpload(t *testing.T)
 				"--remote", server.URL,
 				"--archive", "gitcrawl/openclaw__openclaw",
 				"--token-env", tokenEnv,
-				"--allow-incomplete",
 				"--json",
 			}
+			if !test.archiveAdmission {
+				args = append(args, "--allow-incomplete")
+			}
 			args = append(args, test.args...)
 			err := New().Run(ctx, args)
 			if err == nil || !strings.Contains(err.Error(), test.missingCapability) {