Git-backed workspace sync (9/13)

docs/config.md

@@ -179,6 +179,66 @@ is too old to read shows up as a startup warning on the instance and in `nerve
 doctor`. If a key is ever retired upstream, `--strict-keys` flags it here first;
 drop the key, or install the ref you deploy (`nerve @ git+...@v1.2.3`) instead.
 
+## Git-Backed Workspace Sync
+
+The workspace can be a git repository whose remote is a shared **config repo**.
+Config changes are proposed as PRs, reviewed and merged there; the instance pulls
+the merged result and reloads, with no restart and no editing on the box.
+
+```bash
+nerve config sync                 # git pull --ff-only the workspace, then validate
+nerve config sync --branch main
+nerve config sync --no-validate
+nerve config sync --no-strict-env # tolerate ${VAR}s your shell doesn't have
+```
+
+Enable periodic pulls in the daemon (opt-in):
+
+```yaml
+workspace_sync:
+  enabled: true          # off by default
+  branch: main           # empty = current tracking branch
+  interval_minutes: 1    # also the upper bound on how stale a box can be
+  validate: true         # validate the pulled bundle before applying
+  strict_env: true       # unset required ${VAR} in the bundle blocks the merge
+```
+
+Each sync is fetch, then validate, then fast-forward merge. The fetched bundle is
+validated in a throwaway git worktree and the live working tree is fast-forwarded
+only if that passes, so an invalid bundle never lands on disk and there is nothing
+for a later reload or restart to pick up (`POST /api/config/sync` returns 400 and
+leaves the workspace untouched). A pull that changed something reloads cron and MCP
+config, so the merged change takes effect immediately. CI on the PR is still the
+first line of defense. The remote and credentials come from git itself, so
+configure `git remote` and auth in the workspace as usual.
+
+**Keep the config subtree clean.** Sync refuses to merge while
+`<workspace>/config/` has local changes: an edited or deleted tracked file, a
+staged change, or an untracked file. Validation judges a clean checkout of the
+fetched commit, but the merge lands in your working tree, and `--ff-only` only
+refuses when the incoming commit touches the same path. Anything else would survive
+the merge unchecked, leaving a bundle on disk that is not the one that passed. The
+case that matters most is an untracked `config/cron/gates/*.py`: the daemon imports
+and runs gate plugins and validation never loads them, so a box meant to run only
+reviewed config would be running local code. Commit, discard or push local edits;
+the failure message names the paths. Files matched by `.gitignore` inside `config/`
+are warnings rather than refusals, since the shared repo can never carry them.
+
+Sync validates more strictly than CI: an unset required `${VAR}` blocks the merge.
+CI has no secrets, so it reports those as info, but the daemon does have them, and a
+bundle with an unresolved required variable is one it will refuse to load on its
+next restart. If a shared change adds a `${VAR}` this box legitimately does not set,
+use `workspace_sync.strict_env: false` rather than letting every sync fail.
+`nerve config sync` runs in your shell, which may not carry the daemon's environment
+(systemd `EnvironmentFile`, docker `--env-file`), so pass `--no-strict-env` there
+for a one-off. Warnings never block a merge: an unrecognized cron gate type, an
+unknown key, or a skipped validation.
+
+`workspace_sync` changes need a daemon restart. The sync loop reads the current
+config object every cycle, so it adds no staleness of its own, but nothing refreshes
+that object while the process runs. Turning `enabled` on needs a restart in any
+case, because the sync task is only created at startup.
+
 ## Migrating an Existing Install
 
 Installs from before the workspace-config layout are migrated automatically and

nerve/cli.py

@@ -1195,6 +1195,50 @@ def config_group() -> None:
     """Configuration commands."""
 
 
+@config_group.command("sync")
+@click.option("--branch", default="", help="Branch to pull (default: current tracking branch).")
+@click.option("--no-validate", is_flag=True, help="Skip validating the pulled bundle.")
+@click.option(
+    "--no-strict-env", is_flag=True,
+    help="Allow unset ${ENV_VAR} references in the pulled bundle. Normally they "
+         "block the merge, because the daemon refuses to load a config with an "
+         "unresolved required variable. Use this when your shell doesn't carry "
+         "the daemon's environment (systemd/docker). Otherwise follows "
+         "workspace_sync.strict_env.",
+)
+@click.pass_context
+def config_sync(
+    ctx: click.Context, branch: str, no_validate: bool, no_strict_env: bool,
+) -> None:
+    """Pull the workspace from its git remote (the shared config repo).
+
+    Fast-forward only. This moves the files; it does not tell a running daemon
+    about them. The daemon applies them on its next sync cycle, or immediately
+    via POST /api/config/sync.
+    """
+    from nerve.sync_service import sync_workspace
+
+    config = ctx.obj["config"]
+    if config is None:
+        raise click.ClickException("Config could not be loaded; run 'nerve doctor'.")
+    # Raw values, not Path(...): sync_workspace coerces inside its own
+    # never-raises guard, so a bad `workspace` reports instead of traceback.
+    result = sync_workspace(
+        config.workspace, ctx.obj["config_dir"], branch=branch,
+        validate=not no_validate,
+        strict_env=config.workspace_sync.strict_env and not no_strict_env,
+    )
+    for warning in result.validation_warnings:
+        click.secho(f"  [WARN] {warning}", fg="yellow")
+    if result.ok:
+        click.secho(result.message, fg="green")
+    else:
+        for err in result.validation_errors:
+            click.secho(f"  [ERR] {err}", fg="red")
+        click.secho(result.message, fg="red")
+        ctx.exit(1)
+
+
 @config_group.command("validate")
 @click.option(
     "--workspace", "workspace", type=click.Path(), default=None,

nerve/config.py

@@ -176,8 +176,7 @@ def _resolve_env_refs(merged: dict[str, Any]) -> dict[str, Any]:
 #
 # ${ENV_VAR} interpolation is applied once at the end. This keeps existing
 # single-file installs working unchanged (layer 1 absent → prior behavior) while
-# letting shared settings live in the workspace. Lockdown mode (a later story)
-# will drop layers 2/3 so only the tracked workspace layer applies.
+# letting shared settings live in the workspace.
 
 
 def _read_yaml_mapping(path: Path, *, strict: bool = False) -> dict[str, Any]:
@@ -947,6 +946,44 @@ def from_dict(cls, d: dict, workspace: Path | None = None) -> CronConfig:
         )
 
 
+@dataclass
+class WorkspaceSyncConfig:
+    """Git-backed sync of the workspace from a remote (the config repo).
+
+    Opt-in. When enabled, the daemon periodically ``git pull --ff-only``s the
+    workspace so config changes reviewed & merged via PR on the remote land on
+    the instance and hot-reload. The remote/branch come from git itself; only the
+    cadence and enablement live here.
+    """
+
+    enabled: bool = False
+    branch: str = ""               # empty → git's default (current tracking branch)
+    # How often the daemon pulls. This is also the upper bound on how long a
+    # merged config change takes to reach the instance, since the pull is what
+    # applies it, so it is deliberately short; a fetch that finds nothing new is
+    # one round trip. Raise it for a large fleet pointed at one repo.
+    interval_minutes: int = 1
+    validate: bool = True          # validate the pulled bundle before applying
+    # Treat an unset required ${VAR} in the pulled bundle as invalid. On by
+    # default: the daemon refuses to load such a config, so merging it only
+    # defers the failure to the next restart. Configurable because a shared repo
+    # can introduce a ${VAR} that this particular box legitimately does not set,
+    # and the alternative is a machine whose every sync fails until someone
+    # notices.
+    strict_env: bool = True
+
+    @classmethod
+    @_coerced
+    def from_dict(cls, d: dict) -> WorkspaceSyncConfig:
+        return cls(
+            enabled=d.get("enabled", False),
+            branch=str(d.get("branch", "") or ""),
+            interval_minutes=d.get("interval_minutes", 1),
+            validate=d.get("validate", True),
+            strict_env=d.get("strict_env", True),
+        )
+
+
 @dataclass
 class BackupConfig:
     """Scheduled backup of Nerve state to a local directory.
@@ -1863,6 +1900,7 @@ class NerveConfig:
     sync: SyncConfig = field(default_factory=SyncConfig)
     memory: MemoryConfig = field(default_factory=MemoryConfig)
     cron: CronConfig = field(default_factory=CronConfig)
+    workspace_sync: WorkspaceSyncConfig = field(default_factory=WorkspaceSyncConfig)
     backup: BackupConfig = field(default_factory=BackupConfig)
     sessions: SessionsConfig = field(default_factory=SessionsConfig)
     retention: RetentionConfig = field(default_factory=RetentionConfig)
@@ -2068,6 +2106,7 @@ def _build_from_dict(cls, d: dict) -> NerveConfig:
             sync=SyncConfig.from_dict(d.get("sync", {})),
             memory=MemoryConfig.from_dict(d.get("memory", {})),
             cron=CronConfig.from_dict(d.get("cron", {}), workspace=workspace),
+            workspace_sync=WorkspaceSyncConfig.from_dict(d.get("workspace_sync", {})),
             backup=BackupConfig.from_dict(d.get("backup", {})),
             sessions=SessionsConfig.from_dict(d.get("sessions", {})),
             retention=RetentionConfig.from_dict(d.get("retention", {})),

nerve/config_validate.py

@@ -118,6 +118,38 @@ def validate_config_bundle(
     env_names = ", ".join(sorted(set(missing)))
     if missing:
         msg = f"references unset environment variable(s): {env_names}"
+        # Say which layer asked for them. The refs are collected from the merged
+        # config, so a variable named only by this host's config.yaml /
+        # config.local.yaml is otherwise indistinguishable from one the portable
+        # bundle needs — and when this runs as sync's gate, that reads as a defect
+        # in an incoming change that has nothing to do with it.
+        #
+        # Which is why the workspace layer is scanned separately rather than
+        # inferred from the machine one: a var both layers name would otherwise be
+        # reported as the machine's alone, sending the reader to a file whose only
+        # fault is that it also needs the variable, while the portable config that
+        # equally needs it goes unmentioned. Only claim exclusivity when the
+        # workspace config genuinely does not ask for the variable.
+        from_machine: list[str] = []
+        cfg._interpolate_env(machine, from_machine)
+        from_workspace: list[str] = []
+        cfg._interpolate_env(ws_settings, from_workspace)
+        unset = set(missing)
+        machine_only = sorted((set(from_machine) - set(from_workspace)) & unset)
+        both_layers = sorted(set(from_machine) & set(from_workspace) & unset)
+        clauses = []
+        if machine_only:
+            clauses.append(
+                f"{', '.join(machine_only)} referenced only by this machine's "
+                f"config.yaml/config.local.yaml, not by the workspace config"
+            )
+        if both_layers:
+            clauses.append(
+                f"{', '.join(both_layers)} referenced by both the workspace config "
+                f"and this machine's config.yaml/config.local.yaml"
+            )
+        if clauses:
+            msg += " — " + "; ".join(clauses)
         (result.errors if strict_env else result.info).append(msg)
 
     # Unknown / misspelled keys: warnings by default (forward-compat / example

nerve/cron/service.py

@@ -252,6 +252,9 @@ def __init__(self, config: NerveConfig, engine: AgentEngine, db: Database):
         self._jobs: list[CronJob] = []
         self._source_runners: list[SourceRunner] = []
         self._job_locks: dict[str, asyncio.Lock] = {}
+        # Serialize reload() so the file watcher, the sync loop, and the HTTP
+        # route can't interleave scheduler mutations.
+        self._reload_lock = asyncio.Lock()
 
     async def start(self) -> None:
         """Load jobs and start the scheduler."""
@@ -405,23 +408,34 @@ async def reload(self) -> dict:
         All-or-nothing: the complete change set — every trigger included — is
         built before the scheduler is touched, so a reload the daemon cannot
         carry out raises with the running schedule exactly as it was. That
-        covers the gate registry too; see :meth:`_reload_from_disk`.
+        covers the gate registry too; see :meth:`_reload_locked`.
+
+        Serialized via a lock so concurrent callers (file watcher, sync loop,
+        HTTP route) can't interleave scheduler mutations. The two properties are
+        separate: the lock keeps two reloads from overlapping, and the planning
+        pass keeps a single failing one from applying half of itself.
 
         Returns a summary dict: ``{"added", "removed", "updated", "enabled"}``,
         where "updated" is every enabled job that already had a trigger. It says
         what the reload rescheduled, not which jobs the file changed.
         """
+        async with self._reload_lock:
+            return await self._reload_locked()
+
+    async def _reload_locked(self) -> dict:
+        """Reload under the lock, undoing the one change that isn't local.
+
+        GATE_REGISTRY is process-global and has to be replaced *before* jobs are
+        rebuilt, because a CronJob builds its gates from it at construction time.
+        That puts it ahead of every check that can still refuse the reload, so it
+        needs undoing by hand — leaving the scheduler untouched is not enough when
+        a deleted plugin's gate has already been unregistered. The next CronJob
+        built from disk (run_job, rotate_session) would then fail to build for
+        want of a gate type, off a reload that returned 400: the instance would
+        lose jobs it had never agreed to change.
+        """
         from nerve.cron.gates import GATE_REGISTRY
 
-        # The one piece of reload state that isn't local: GATE_REGISTRY is
-        # process-global and has to be replaced *before* jobs are rebuilt,
-        # because a CronJob builds its gates from it at construction time. That
-        # puts it ahead of every check that can still refuse the reload, so it
-        # needs undoing by hand — leaving the scheduler untouched is not enough
-        # when a deleted plugin's gate has already been unregistered. The next
-        # CronJob built from disk (run_job, rotate_session) would then fail to
-        # build for want of a gate type, off a reload that returned 400: the
-        # instance would lose jobs it had never agreed to change.
         gates_before = dict(GATE_REGISTRY)
         try:
             return await self._reload_from_disk(gates_before)
@@ -435,7 +449,7 @@ async def reload(self) -> dict:
     async def _reload_from_disk(
         self, gates_before: dict[str, type["CronGate"]],
     ) -> dict:
-        """The body of :meth:`reload`. Only call it through there.
+        """The body of :meth:`reload`. Only call it through :meth:`_reload_locked`.
 
         Takes the pre-reload registry snapshot so it can report which gates
         vanished once the change is committed, and so its caller can restore

nerve/gateway/routes/__init__.py

@@ -35,6 +35,7 @@
     codex,
     workflow_runs,
     review_loops,
+    config,
 )
 
 __all__ = [
@@ -67,4 +68,5 @@ def register_all_routes() -> APIRouter:
     router.include_router(codex.router)
     router.include_router(workflow_runs.router)
     router.include_router(review_loops.router)
+    router.include_router(config.router)
     return router

nerve/gateway/routes/config.py

@@ -0,0 +1,56 @@
+"""Config / workspace-sync routes."""
+
+from __future__ import annotations
+
+from fastapi import APIRouter, Depends, HTTPException
+
+from nerve.gateway.auth import require_auth
+from nerve.gateway.routes._deps import get_deps
+
+router = APIRouter()
+
+
+@router.post("/api/config/sync")
+async def sync_workspace_route(user: dict = Depends(require_auth)):
+    """Pull the workspace from its git remote and apply changes (cron + MCP).
+
+    Fast-forward only; validates the pulled bundle before applying.
+    """
+    import asyncio
+
+    from nerve.config import get_config
+    from nerve.gateway.server import _cron_service
+    from nerve.sync_service import _apply_sync, sync_workspace
+
+    config = get_config()
+    # Hand over the raw configured values: sync_workspace coerces them inside its
+    # own never-raises guard, so a `workspace` that is null or a list reports a
+    # 400 rather than a 500 with a stack trace.
+    result = await asyncio.to_thread(
+        sync_workspace,
+        config.workspace,
+        config.config_dir or config.workspace,
+        branch=config.workspace_sync.branch,
+        validate=config.workspace_sync.validate,
+        strict_env=config.workspace_sync.strict_env,
+    )
+    if not result.ok:
+        raise HTTPException(
+            status_code=400,
+            detail={
+                "message": result.message,
+                "errors": result.validation_errors,
+                "warnings": result.validation_warnings,
+            },
+        )
+    if result.changed:
+        deps = get_deps()
+        await _apply_sync(deps.engine, _cron_service)
+    return {
+        "ok": True,
+        "changed": result.changed,
+        "message": result.message,
+        "old_rev": result.old_rev,
+        "new_rev": result.new_rev,
+        "warnings": result.validation_warnings,
+    }