fix: bound portable subscriber refresh without background repacks

CHANGELOG.md

@@ -1,7 +1,11 @@
 # Changelog
 
-## 0.9.3 - Unreleased
+## 0.9.3 - 2026-08-29
 
+- Fix repeated gzip-only portable initialization and add a locked, bounded, preservation-first `portable refresh` subscriber command with automatic Git maintenance disabled across portable operations.
+- Preserve writable portable runtime data across refresh and ordinary reads, keep prepared raw replicas updating through publisher transitions, report gzip subscriber status from the usable artifact/runtime, and retain explicit Git config isolation with safe scope diagnostics.
+- Reject credential-bearing portable remote URLs while retaining SSH transport usernames.
+- Report actionable portable Git failure categories without exposing raw remote or credential-helper diagnostics.
 - Update the minimum Go toolchain to 1.26.7, SQLite to v1.57.0, and the current compatible Go dependency closure; keep Go 1.27 on hold until managed CodeQL supports it.
 - Refresh govulncheck, deadcode, the Dockerfile frontend, and the docs build's Node runtime.
 - Stabilize the coverage gate with direct tests for immutable SQLite reads, PR detail lookups, and vector scopes.

README.md

@@ -62,6 +62,7 @@ Octopool owns pooled live `gh` reads. Gitcrawl keeps local mirror, search, clust
 | Job | Command | Guide |
 | --- | --- | --- |
 | Check archive health | `gitcrawl status` / `gitcrawl doctor` | [Configuration](docs/configuration.md) |
+| Refresh a portable subscriber | `gitcrawl portable refresh --expected-remote URL` | [Portable stores](docs/portable-stores.md#routine-subscriber-refresh) |
 | Mirror GitHub threads | `gitcrawl sync owner/repo` | [Sync](docs/sync.md) |
 | Search threads or indexed code | `gitcrawl search ...` | [Search](docs/search.md) |
 | Build and inspect clusters | `gitcrawl refresh`, `clusters`, `tui` | [Clustering](docs/clustering.md) |

docs/automation.md

@@ -60,6 +60,26 @@ Run `gitcrawl refresh owner/repo` on a cron, systemd timer, or `launchd` agent e
 
 For multiple repos, loop in a small shell script — gitcrawl is happy to run sequentially against a shared SQLite file.
 
+### Portable subscribers
+
+After one-time initialization, schedule `portable refresh`, not repeated
+`init` or `doctor`. It retains configuration and refuses unsafe checkout state:
+
+```bash
+gitcrawl --config /path/to/config.toml portable refresh \
+  --expected-remote https://github.com/example/archive-store.git \
+  --git /absolute/path/to/git --timeout 2m \
+  --min-free-bytes 2147483648 --max-growth-bytes 2147483648 \
+  --json > refresh.json 2> refresh.log
+```
+
+Treat nonzero exits as refusals requiring inspection. Do not automatically
+delete locks, temporary packs, backups or SQLite sidecars and retry. A `partial`
+result identifies an interrupted advancement, not a rollback. A
+`mirror_result` of `preserved-local` means local runtime writes remain intact.
+See [Portable stores](/portable-stores/#routine-subscriber-refresh) for the
+admission rules, sampled growth budget and exact JSON contract.
+
 ## Agent recipes
 
 ### "Look up an issue without burning quota"

docs/commands.md

@@ -136,6 +136,7 @@ octopool gh api repos/openclaw/openclaw/pulls/123
 
 | Command | Purpose | Docs |
 | --- | --- | --- |
+| `gitcrawl portable refresh --expected-remote URL [--store-dir PATH --portable-db PATH --branch main --git PATH --timeout 2m --min-free-bytes N --max-growth-bytes N --json]` | Validate and fast-forward a clean configured subscriber without regenerating config or invoking repair | [Portable stores](/portable-stores/#routine-subscriber-refresh) |
 | `gitcrawl portable prune [--body-chars --no-vacuum --include-sync-failures --no-publish --json]` | Build a compact portable v2 backup and (optionally) `VACUUM` for publishing | [Portable stores](/portable-stores/#publishing-gitcrawl-portable-prune) |
 | `gitcrawl portable export --profile current-state-v1 --output-dir PATH [--repository owner/repo --body-chars --database-name --public-path --max-bytes --json]` | Create a validated, optionally repository-scoped database-and-manifest generation without changing or publishing the active database | [Portable stores](/portable-stores/#derived-generations-gitcrawl-portable-export) |
 

docs/portable-stores.md

@@ -55,9 +55,144 @@ size and SHA-256, runs SQLite `quick_check`, and only then atomically replaces
 the runtime mirror. Legacy raw SQLite stores continue to use the same manifest
 without compression fields.
 
+Initialization validates portable arguments before invoking Git and validates
+the artifact before saving configuration. Repeated initialization and a
+publisher's raw-to-gzip transition do not require a raw `.db` in the checkout.
+Use `init` for setup; it still regenerates configuration on success.
+
+## Routine subscriber refresh
+
+Use the strict subscriber command for scheduled updates:
+
+```bash
+gitcrawl --config /path/to/config.toml portable refresh \
+  --expected-remote https://github.com/example/archive-store.git \
+  --git /absolute/path/to/git \
+  --timeout 2m \
+  --min-free-bytes 2147483648 \
+  --max-growth-bytes 2147483648 \
+  --json
+```
+
+This uses the configured logical database and its existing checkout. Optional
+`--store-dir` and `--portable-db` assert that those configured paths match the
+intended store; they do not reconfigure it. `--expected-remote` is required
+because legacy configs do not pin an origin URL. `--branch` defaults to `main`;
+the checkout must be on that branch and track the matching origin branch.
+No credentials belong in the URL or command line; use Git's credential helper.
+HTTP(S) URL userinfo and passwords in any URL are refused. SSH transport
+usernames such as `ssh://git@host/repo` and `git@host:repo` remain supported.
+
+Refresh takes a nonblocking advisory lock for the canonical store path. All
+Gitcrawl portable Git writers, legacy recovery, runtime promotion and CLI
+writable-runtime sessions share that lease. Symlink aliases converge. Its
+permanent sibling `.STORE.gitcrawl.lock` file is never removed or stolen based
+on age; the operating system releases ownership when the process exits.
+An occupied lock is a refusal, not a reason to kill another process.
+
+Strict refresh refuses dirty indexes/worktrees, **all untracked or ignored
+checkout files**, hidden index entries, unrelated origins, divergent history,
+active Git locks or operation state, orphan/temporary packs, hooks, filters,
+attributes, submodules, linked worktrees, alternate object stores, config
+includes/redirections and unsupported extensions. Resolve these deliberately
+outside the subscriber command. It never resets, cleans, prunes, repacks,
+reclones, deletes backups or sidecars, or invokes reader/doctor auto-repair.
+
+Admission examines every exposed Git configuration scope. Refusals identify
+the scope and unsafe category without printing keys or values. Single-valued
+hooks-path, fsmonitor, attributes-file and SSH-command settings overridden by
+the portable runner are inert; actual checkout hooks/attributes and exposed
+filter definitions still cause refusal. For a dedicated subscriber that must
+exclude machine-wide filters, Unix callers can set
+`GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1` for this invocation (Windows:
+`GIT_CONFIG_GLOBAL=NUL GIT_CONFIG_NOSYSTEM=1`). `GIT_CONFIG_SYSTEM` may also point
+to the platform null device. These scope-removal controls survive sanitization;
+arbitrary config paths, repository redirection and config injection do not.
+Disabling global configuration also removes global credential helpers, so the
+remaining authentication configuration must support the intended remote.
+No global files are changed, and repository-local safety checks still apply.
+
+Fetch requests only the intended branch into `FETCH_HEAD`, with no pruning,
+tags, submodule recursion or remote-tracking ref updates. Both existing HEAD
+and the tracking ref must be ancestors of the frozen fetched commit. Only its
+manifest and selected artifact are extracted into private staging beside the
+runtime mirror. Existing raw/gzip digest, expanded-size, SQLite `quick_check`
+and semantic-identity validation run before a fast-forward checkout. Tracking
+ref advancement uses compare-and-swap. Configuration, refs, store identity,
+Git metadata, cleanliness and capacity are checked again at mutation boundaries.
+
+The validated runtime generation is atomically renamed outside the checkout.
+A runtime with bytes differing from its recorded source digest, or any SQLite
+sidecars, is preserved as `preserved-local`; it is never overwritten to make
+the subscriber appear fresh. Newly materialized raw replicas record their digest
+even without a manifest. Verified disposable replicas use immutable SQLite reads
+so ordinary reads do not create sidecars that would claim local ownership.
+Writable CLI opens record local ownership before
+SQLite migrations or writes. Ordinary reads retain that ownership and the
+original source identity across later publisher generations; local closures,
+vectors and other runtime work survive repeated reads. A corrupt locally owned
+runtime reports an error without replacement; a corrupt disposable replica
+still follows the legacy recovery path. Other Gitcrawl CLI writers obey the lease, but
+external Git/SQLite writers do not. Observable changes cause refusal; this is
+not a universal filesystem transaction or protection against a hostile writer.
+
+| Flag | Default | Meaning |
+| --- | --- | --- |
+| `--expected-remote URL` | required | Expected identity of `origin` |
+| `--store-dir PATH` | configured store | Assert canonical store directory |
+| `--portable-db PATH` | configured logical DB | Assert clean relative slash path; never the gzip path |
+| `--branch NAME` | `main` | Expected local and origin branch |
+| `--git PATH` | `GITCRAWL_PORTABLE_GIT`, then resolved process PATH | Absolute executable; no login shell is used |
+| `--timeout DURATION` | `2m` | Total operation deadline, plus bounded process cleanup |
+| `--min-free-bytes N` | `2147483648` (2 GiB) | Free-space reserve on store and staging filesystems |
+| `--max-growth-bytes N` | `2147483648` (2 GiB) | Positive logical-file growth budget |
+| `--json` | off | Structured success, no-op, refusal or partial result |
+
+Admission requires reserve **plus the full growth budget** available on both
+filesystems. Growth is the sum of positive per-path size deltas under the
+checkout (including `.git`) and runtime database directory, relative to
+admission. Deleting/shrinking old files does not credit the budget. Metadata
+scans run at stage boundaries and every 100 ms in flight, with a 200,000-file
+scan limit; they never read or copy historical pack contents. Blob extraction
+is capped at its frozen Git size, and manifest-based staging estimates allow
+for inflation and semantic-identity copies. A budget is a sampled cancellation
+boundary, **not an OS quota**: rapid Git or filesystem writes can overshoot
+between scans. Reserve is also affected by unrelated processes. Use filesystem
+quotas where a hard physical allocation ceiling is required.
+
+On cancellation, owned Git process groups receive a graceful termination
+request, then are forced after at most 750 ms and reaped (pipe cleanup adds at
+most one second). Windows starts Git suspended, assigns a kill-on-close job,
+then resumes it; console break is best-effort before job termination. Unrelated
+processes are never targeted. Only this operation's private staging is removed.
+Filesystem and SQLite cancellation is cooperative; a kernel I/O stall can
+delay cancellation or process reaping beyond the requested deadline.
+Fetched objects and `FETCH_HEAD` may remain after failure. If checkout, tracking
+ref or mirror advancement has already started, JSON reports `partial`; no
+successful rollback is claimed, and an interrupted Git operation may need
+operator inspection before another strict refresh.
+
+JSON reports `stage`, `result` (`updated`, `no-op`, `refused`, `partial`),
+`before_commit`, `after_commit`, `target_commit`, `artifact_id`, `sha256`,
+`artifact_bytes`, `mirror_destination`, `mirror_result`, `capacity` and
+`elapsed_ms`. A refusal includes a bounded `reason` and exits nonzero. Stderr
+contains stage/elapsed/result diagnostics; stdout remains data. A no-op means
+the refs already match; a missing runtime may still be materialized. Git failures
+use fixed, credential-safe repository, authentication, connection or disk-space
+diagnostics when recognized, with general troubleshooting guidance otherwise;
+raw remote/helper stderr is never included.
+
+All portable Git entry points explicitly set `maintenance.auto=false` and
+`gc.auto=0`, including init, implicit reader fetch/merge and retained recovery
+clone/reset helpers. Fetch also uses `--no-auto-maintenance`. Hooks, fsmonitor
+and recursive submodule operations are disabled in the portable runner.
+Unsupported Git options fail closed; there is no fallback that drops safety
+flags. Set `GITCRAWL_PORTABLE_GIT` to an absolute path to select Git for legacy
+portable commands as well. Gitcrawl does not change global or local Git policy.
+
 ## How read-only commands behave
 
-Read-only commands (`search`, `threads`, `clusters`, `cluster-detail`, `neighbors`, the TUI) refresh the portable-store checkout before reading, so they always see the latest published data:
+Read-only commands (`search`, `threads`, `clusters`, `cluster-detail`, `neighbors`, the TUI) normally refresh the portable-store checkout before reading. A locally owned writable runtime continues serving its local data instead of accepting publisher replacement:
 
 - The refresh is best-effort and non-interactive
 - SSH attempts are bounded so an offline remote does not hang the CLI
@@ -68,6 +203,23 @@ Read-only commands (`search`, `threads`, `clusters`, `cluster-detail`, `neighbor
 
 If the remote is unreachable, the read still answers from the local checkout.
 
+`status` is observational: it uses an existing runtime without fetching,
+repairing, migrating, or promoting anything. It reports `state: stale` when
+that runtime's recorded source generation differs from the checkout, with a
+warning for writable local state. Before a gzip-only subscriber has a runtime,
+status validates and reads the artifact in disposable temporary storage; its
+inventory reports the gzip path and compressed on-disk bytes, with a warning
+explaining that distinction. It never reports an absent logical `.db` as an
+empty, current subscriber.
+
+This is the **legacy reader recovery contract**, separate from strict
+`portable refresh`: a marked malformed store can still be backed up, reset and
+recloned with the existing recovery backoff. Its historical stale-index-lock
+check requires both age and an external open-file probe. Do not use these
+commands as a preservation-first subscriber verification step. They now share
+the canonical store lease and maintenance suppression, but do not inherit
+strict refresh's capacity limits or clean-only policy.
+
 ## How write commands behave
 
 Write commands (`sync`, `embed`, `refresh`, `portable prune`, `cluster`, neighbor generation) open a **writable runtime mirror** alongside the portable checkout so new GitHub data, vectors, and overrides persist without partially mutating the published portable store. When this redirect engages, gitcrawl prints one stderr notice naming both the runtime mirror and the checkout database.