From 6b361e647eee20702b0bfc1d30c76ddce931df1e Mon Sep 17 00:00:00 2001 From: AgentTanuki Date: Fri, 31 Jul 2026 12:48:15 +0100 Subject: [PATCH 1/2] Autonomous trust index + paid self-serve layer: the product answers "can I safely use or pay this endpoint right now?" Executes docs/AUTONOMOUS_PIVOT_2026-07-31.md. Everything here acquires, serves, charges, monitors and evaluates with no routine human involvement. Built on the shipped preflight/reachability work rather than beside it. FREE PUBLIC TRUST INDEX (app/trustindex.py, indexsources.py, indexops.py) Registries conflate three different states and report them all as a listing. This index keeps them apart and never lets one be read as another: `indexed` (a source listed it, we have never called it - a CLAIM), `live` (completed a real protocol handshake when we called it), `degraded` (answers, but one of its own declared claims does not hold), `unreachable`. Measured 2026-07-31: 92.9% of a2aregistry entries report is_healthy:true and 33.9% complete a task. That 59-point gap is the entire reason this exists, and it is why a listing is never promoted to `live` without an observation of our own. Dedupe by endpoint fingerprint then identity: one operator publishing to three registries is ONE entry with three provenance records, never three counts in a headline. Drift is retained per endpoint - the property a one-off review cannot have. Stale is a first-class state, not an absence. Surfaces: GET /index, /index/search, /index/{id}, /index/{id}/evidence. The evidence page is gated on trustindex.is_page_worthy and 404s rather than generating filler - a page for an endpoint we never called contains nothing the registry does not already have, and publishing those at scale is SEO spam. Adapters read documented public read-only APIs with a truthful, contactable User-Agent; remote ingest is default-OFF and capped per run. PAID SELF-SERVE LAYER (app/deepcheck.py, pricing.py) GET /preflight/deep adds drift history, cross-source corroboration and an explicit allow/caution/block policy verdict whose threshold is published so a caller can reject it. POST /evidence/bundle issues a signed, checkpoint-anchored snapshot the caller keeps and re-verifies offline without us. POST /watch self-provisions continuous monitoring - no onboarding, no human - idempotent by (caller, endpoint) so a retry never bills twice, charged per recheck ACTUALLY performed. Free stays free: /preflight, /index, /index/search and /evidence/verify. Charging for "does this endpoint work" would make the ecosystem worse and this index poorer, and charging to verify an artefact we sold would make the artefact worth less than we claimed when we sold it. FAILS CLOSED. Evidence bundles are produced BEFORE the meter runs; a refusal returns 409 with "NOT CHARGED". A watch cycle that could not observe is not billed. A failed charge suspends the watch rather than serving it free. PRICES ARE CONFIG, NOT DOCTRINE. Every price is env-overridable within a hard ceiling and publishes its stated basis at GET /pricing. Opening prices are set for the population that actually exists: July 2026 x402 volume across all networks was $232,329 (-98.9% from the Nov 2025 peak, ~$0.04/tx) and the median earning agent made $1.65 per 30 days. deep_preflight 20cr ($0.02), evidence_bundle 100cr, watch_cycle 5cr, provisioning free. AUTONOMOUS EXPERIMENT ENGINE (app/experiments.py) Can move exactly two things - a price within its published ceiling, and a copy variant. It cannot change what counts as success and cannot touch attribution. Qualified exposure is genuine-external only through the same central attribution rule used everywhere else, so crawlers and first-party tooling can never reach the threshold no matter how much of them there is. A zero denominator yields `insufficient_evidence`, never `kill` - "0% conversion on 1,790 crawler impressions" is a category error with a number attached, not a finding. On `kill` the engine CHANGES THE OFFER (halves the price within its ceiling); on insufficient evidence it fixes distribution instead, because repricing something nobody saw teaches us nothing. OPS. Index upkeep rides the existing lease-guarded, jittered scout loop - one loop, one lease, one deadline, one kill switch - and is default-OFF behind GUILD_INDEX_AUTORUN. GET /commercial reports revenue first, then qualified exposure, then experiments, with inventory/reach/free usage under `supporting_never_sufficient`. docs/RUNBOOK_TRUST_INDEX.md documents every switch and a five-step rollback, none of which needs a deploy. Tests: 1069 passed, 9 skipped (43 new) covering dedupe, claim-vs-observation, fail-closed issuance, bundle tamper-detection, watch idempotency and non-billing, price clamping, crawler exclusion and deterministic ownership. --- docs/INTERFACE.md | 15 + docs/RUNBOOK_TRUST_INDEX.md | 121 ++++++++ live/guild/app/deepcheck.py | 280 +++++++++++++++++ live/guild/app/experiments.py | 318 ++++++++++++++++++++ live/guild/app/indexops.py | 297 ++++++++++++++++++ live/guild/app/indexsources.py | 156 ++++++++++ live/guild/app/main.py | 337 ++++++++++++++++++++- live/guild/app/mcp_server.py | 96 ++++++ live/guild/app/payments.py | 29 ++ live/guild/app/pricing.py | 128 ++++++++ live/guild/app/store.py | 26 +- live/guild/app/swarm/runner.py | 90 ++++++ live/guild/app/trustindex.py | 317 ++++++++++++++++++++ live/guild/contract/contract.json | 72 ++++- live/guild/tests/test_trust_index.py | 433 +++++++++++++++++++++++++++ render.yaml | 40 +++ 16 files changed, 2750 insertions(+), 5 deletions(-) create mode 100644 docs/RUNBOOK_TRUST_INDEX.md create mode 100644 live/guild/app/deepcheck.py create mode 100644 live/guild/app/experiments.py create mode 100644 live/guild/app/indexops.py create mode 100644 live/guild/app/indexsources.py create mode 100644 live/guild/app/pricing.py create mode 100644 live/guild/app/trustindex.py create mode 100644 live/guild/tests/test_trust_index.py diff --git a/docs/INTERFACE.md b/docs/INTERFACE.md index 31a80af..daa2ca6 100644 --- a/docs/INTERFACE.md +++ b/docs/INTERFACE.md @@ -61,6 +61,7 @@ guild_mediated requires two-party cryptographic participation, a Guild-observed - `GET /citizenship` - `GET /citizenship.md` - `POST /collaborations` +- `GET /commercial` - `POST /credentials/verify` - `GET /demand/feed` - `POST /demand/watch` @@ -74,12 +75,18 @@ guild_mediated requires two-party cryptographic participation, a Guild-observed - `POST /escrow/{escrow_id}/refund` - `POST /escrow/{escrow_id}/release` - `GET /evaluation` +- `POST /evidence/bundle` +- `POST /evidence/verify` - `POST /feedback/abandonment` - `GET /flags` - `GET /for-agents` - `GET /funnel` - `GET /funnel/passports` - `GET /health` +- `GET /index` +- `GET /index/search` +- `GET /index/{endpoint_id}` +- `GET /index/{endpoint_id}/evidence` - `GET /instrumentation` - `GET /instrumentation/recent` - `GET /ledger/checkpoint` @@ -99,6 +106,8 @@ guild_mediated requires two-party cryptographic participation, a Guild-observed - `POST /offers/{offer_id}/accept` - `POST /outcomes` - `GET /preflight` +- `GET /preflight/deep` +- `GET /pricing` - `POST /providers/external/discover` - `GET /referrals` - `GET /release` @@ -117,6 +126,8 @@ guild_mediated requires two-party cryptographic participation, a Guild-observed - `POST /wallet-binding/revoke` - `GET /wallet-binding/status/{credential_id}` - `POST /wallet-binding/verify` +- `POST /watch` +- `GET /watch/{watch_id}` - `GET /x402/readiness` ## MCP tools @@ -143,8 +154,10 @@ guild_mediated requires two-party cryptographic participation, a Guild-observed - `guild_check` - `guild_escrow_open` - `guild_escrow_release` +- `guild_index` - `guild_passport` - `guild_preflight` +- `guild_preflight_deep` - `guild_prove` - `guild_prove_verify` - `guild_record` @@ -152,6 +165,8 @@ guild_mediated requires two-party cryptographic participation, a Guild-observed - `guild_risk_score` - `guild_search` - `guild_verify` +- `guild_watch` +- `guild_watch_feed` ## A2A skills diff --git a/docs/RUNBOOK_TRUST_INDEX.md b/docs/RUNBOOK_TRUST_INDEX.md new file mode 100644 index 0000000..c4cd0d8 --- /dev/null +++ b/docs/RUNBOOK_TRUST_INDEX.md @@ -0,0 +1,121 @@ +# Runbook — autonomous trust index, paid layer, experiment engine + +Operating model: **no routine human involvement.** Nothing below is a daily +task. It is the list of switches, the order to pull them in, and how to undo +each one. If you are reading this because something is wrong, start at +§3 (Rollback) — every change here is reversible by configuration alone. + +--- + +## 1. What runs, and when + +One loop. The index does **not** get its own timer — it rides the existing +lease-guarded, jittered scout cycle (`app/swarm/runner.py`), so there is exactly +one place to look when outbound traffic misbehaves and exactly one kill switch. + +Per cycle, in order: + +1. `indexops.ingest` — fold source records into the index (dedupe by endpoint + fingerprint, then identity). +2. `indexops.recheck_due` — probe the **stalest** entries first, capped at + `GUILD_INDEX_RECHECK_BATCH`. +3. `_run_watch_cycles` — run due customer watches, charging **per cycle + actually performed**. +4. `experiments.evaluate` — decide each running experiment, or refuse to. + +A failure in any step is recorded in `swarm_state.last_run.index` and never +fails the cycle. Index upkeep must not be able to take the service down. + +## 2. Switches + +| Variable | Default | Effect | +|---|---|---| +| `GUILD_INDEX_AUTORUN` | `0` (prod `1`) | Master switch for **all** index upkeep | +| `GUILD_INDEX_INGEST` | `0` | Remote public-registry ingest. Separate on purpose | +| `GUILD_INDEX_FRESH_TTL_S` | `86400` | When an observation becomes stale | +| `GUILD_INDEX_RECHECK_BATCH` | `8` | **Outbound bound.** Endpoints probed per cycle | +| `GUILD_EXP_MIN_QUALIFIED` | `10` | Genuine-external actors before a verdict is allowed | +| `GUILD_EXP_WINDOW_DAYS` | `14` | Hard experiment window | +| `GUILD_PRICE_` | unset | Price override, clamped to the ceiling in `app/pricing.py` | + +## 3. Rollback + +Ordered least to most disruptive. **None of these require a deploy.** + +1. **Stop index upkeep** — `GUILD_INDEX_AUTORUN=0`. The scout continues; the + index freezes and serves its last observations, correctly labelled stale. + Paid endpoints keep working on live probes. +2. **Stop outbound registry ingest only** — `GUILD_INDEX_INGEST=0`. Rechecks of + endpoints already known continue. +3. **Throttle** — `GUILD_INDEX_RECHECK_BATCH=1`. Coverage slows; nothing breaks. +4. **Make a paid product free** — `GUILD_PRICE_DEEP_PREFLIGHT=0`. Metering stops + charging immediately; nothing 402s. +5. **Full feature rollback** — revert the release. Every new surface is + additive: no existing route, schema or price changed, and the new Store + collections (`trust_index`, `watches`, `experiments`) are loaded with `{}` + defaults, so an older build ignores them rather than failing to boot. + +The persistent disk is untouched by a rollback — the new collections are kv +rows, not a schema migration. + +## 4. Incident: "the index is probing too much" + +Symptom: complaints from an operator, or outbound volume above expectation. + +1. `GUILD_INDEX_AUTORUN=0` — stops everything immediately. +2. Confirm the bound was actually in force: + `GET /diagnostics/state` for the serving instance, then read + `swarm_state.last_run.index.recheck.capped_at`. +3. Every probe is SSRF-screened and bounded, sends no credentials, and + identifies itself as `agent-guild-index/1.0` with a contact path. If an + operator asks us to stop, add the endpoint's fingerprint to the index and + set the entry `active: false` — do not argue the point. + +## 5. Incident: "a customer was billed and got nothing" + +This should be impossible by construction; verify rather than assume. + +- **Evidence bundles** are produced *before* the meter runs. A refusal raises + `409 evidence_issuance_refused` with `"billing": "NOT CHARGED"`. +- **Watch cycles** charge only after a successful observation. A cycle that + could not observe returns `cycled: false` and is not billed. +- **Failed charge** suspends the watch (`active: false`, + `suspended_reason: payment_failed`) rather than continuing to serve free. + +Check `store.watches[]` for `cycles_billed` versus `len(changes)` and the +`watch_cycle` entries in the billing log. + +## 6. Incident: "an experiment killed something it should not have" + +An experiment can only reach `kill` with `GUILD_EXP_MIN_QUALIFIED` **genuine +external** actors. Crawlers, first-party tooling and unknown-attributed traffic +are excluded structurally, so they cannot reach the threshold no matter how +much of them there is. + +If a `kill` looks wrong, read `experiments[key].evidence.exposure` — it names +the rule and the actor count. To reverse: raise the price back with +`GUILD_PRICE_` (the engine only ever *halves* within the ceiling; it cannot +raise a price or invent an operation). + +## 7. What to read, in order + +`GET /commercial` — revenue first, then qualified exposure, then experiments, +then the supporting metrics explicitly labelled as unable to carry a decision. + +The single number: **`external_settled_revenue_usd`** — independently confirmed +external mainnet settlement only. Sandbox credits, first-party canaries, +testnet funds and internal transfers are excluded by construction. + +If that number is `0.00`, the correct reading is that **nothing has been sold +yet** — regardless of how large the index, the reach or the free-check count +is. + +## 8. Safety invariants that must never be traded away + +- A listing is never promoted to an observation. +- `unknown` is never promoted to `externally_owned`. +- Paid issuance fails closed; a partial evidence bundle is never emitted. +- No evidence page is published for an endpoint we have never called. +- Adapters read documented public APIs only, with a truthful User-Agent. +- We never transact with our own services to create activity, and first-party + watches are excluded from `externally_monitored_endpoints`. diff --git a/live/guild/app/deepcheck.py b/live/guild/app/deepcheck.py new file mode 100644 index 0000000..1e7123b --- /dev/null +++ b/live/guild/app/deepcheck.py @@ -0,0 +1,280 @@ +"""Deep preflight and the signed evidence bundle — the paid artefacts. + +WHAT THE CUSTOMER IS ACTUALLY BUYING + Free ``/preflight`` answers "does this endpoint work right now?" — enough to + avoid the worst mistake, and it stays free forever because a paywall in front + of that answer would make the ecosystem worse and the index poorer. + + The paid layer answers the questions a caller cannot answer for themselves in + one request: + + * **history** — has this endpoint drifted? A server that passed a one-off + review and changed afterwards is invisible to every existing signal, and + is only visible to someone who kept observing it. + * **corroboration** — how many independent sources list it, and do their + claims agree with what we measured? + * **policy** — an explicit allow / caution / block against a stated + threshold, so an orchestrator can act on it without writing its own rules. + * **portability** — a signed bundle the caller keeps, re-verifies offline, + and can show to a third party. That is the artefact with a reason to be + paid for: it survives us being unavailable, and it is checkable without + trusting us at the moment of use. + +THE INVARIANT THAT MATTERS MOST + Paid issuance FAILS CLOSED. If we cannot produce a complete, signed, + anchored artefact, the caller is not charged and no partial bundle is + emitted. Selling a degraded evidence object is worse than selling nothing — + the buyer would rely on it precisely when it is weakest. +""" +from __future__ import annotations + +import hashlib +from datetime import datetime, timedelta, timezone +from typing import Any, Optional + +from .crypto import canonicalize, sign_jcs +from . import preflight, trustindex + +#: Default validity of a signed bundle. Short by design: an evidence object +#: about a live endpoint that claims a long life is lying about how fast the +#: world changes. +DEFAULT_TTL_S = 3600 + + +class EvidenceIssuanceRefused(RuntimeError): + """Paid issuance could not complete. The caller must NOT be charged. + + Raised rather than degrading, because a partial evidence bundle would be + relied on exactly when it is least reliable.""" + + code = "evidence_issuance_refused" + + +def _now() -> datetime: + return datetime.now(timezone.utc) + + +def policy_verdict(observation: dict[str, Any], entry: Optional[dict[str, Any]] + ) -> dict[str, Any]: + """allow / caution / block, with the reason and the threshold stated. + + Deliberately simple and fully explained: a caller must be able to disagree + with our threshold and apply their own. A verdict whose rule cannot be read + is a rating agency, not evidence.""" + failed = list(observation.get("failed", [])) + unknowns = list(observation.get("unknowns", [])) + blocking = [f for f in failed + if f in ("endpoint_reachable", "protocol_handshake")] + drift = list((entry or {}).get("drift", [])) + recent_drift = drift[-3:] + + if blocking: + decision, reason = "block", ( + "the endpoint did not prove it can do the thing it is listed for " + f"({', '.join(blocking)})") + elif failed: + decision, reason = "caution", ( + "the endpoint works, but at least one of its own declared claims " + f"does not hold ({', '.join(failed)})") + elif len(unknowns) >= 4: + decision, reason = "caution", ( + f"nothing failed, but {len(unknowns)} of the checks could not be " + "performed — a clean result over mostly unknowns is thin evidence, " + "not a clean bill of health") + else: + decision, reason = "allow", ( + "every check we could perform passed, and enough of them were " + "performable for that to mean something") + + if decision == "allow" and len(recent_drift) >= 2: + decision, reason = "caution", ( + "checks pass right now, but this endpoint has changed state " + f"{len(recent_drift)} times recently — recent instability is a " + "risk a single-point-in-time check cannot see") + return { + "decision": decision, + "reason": reason, + "threshold": ( + "block if a BLOCKING check failed (reachability, protocol " + "handshake); caution if any declared claim failed, if 4+ checks " + "were unperformable, or if the endpoint changed state 2+ times " + "recently; otherwise allow"), + "caller_note": ( + "This is OUR threshold, stated so you can reject it. `checks` and " + "`unknowns` are supplied in full precisely so you can apply your " + "own policy instead."), + "blocking_failures": blocking, + "claim_failures": [f for f in failed if f not in blocking], + "unperformable_checks": unknowns, + "recent_drift": recent_drift, + } + + +def deep_preflight(store: Any, url: str) -> dict[str, Any]: + """The paid check: live observation + history + corroboration + policy.""" + result = preflight.run(url, store=store) + fp = trustindex.fingerprint(url) + entry = (store.trust_index or {}).get(fp) + + sources = [s.get("source") for s in (entry or {}).get("sources", [])] + declared = (entry or {}).get("declared") or {} + observed_status = trustindex.status_from_preflight(result) + + corroboration = { + "independent_sources": len(sources), + "sources": sources, + "claim_vs_observation": ( + "no source has listed this endpoint to us — you are the first to " + "ask about it" if not sources else + f"{len(sources)} source(s) list this endpoint; we observed it as " + f"'{observed_status}'"), + "declared_name": declared.get("name"), + "declared_capabilities": declared.get("capabilities", []), + } + history = { + "observations": (entry or {}).get("observation_count", 0), + "first_indexed_at": (entry or {}).get("first_indexed_at"), + "drift": (entry or {}).get("drift", []), + "note": ("drift is the state changes we have recorded for this exact " + "endpoint. An endpoint with no history is not safe or unsafe " + "— it is unobserved, and this is its first data point."), + } + return { + **result, + "tier": "deep", + "policy": policy_verdict(result, entry), + "history": history, + "corroboration": corroboration, + "index_status": observed_status, + "free_tier_note": ( + "GET /preflight (free, no key) returns the live checks and verdict. " + "This paid tier adds history/drift, cross-source corroboration and " + "an explicit allow/caution/block policy verdict."), + } + + +def evidence_bundle(store: Any, url: str, *, ttl_s: int = DEFAULT_TTL_S, + audience: str = "") -> dict[str, Any]: + """A signed, offline-verifiable snapshot. FAILS CLOSED. + + Anchored to the published checkpoint feed so a holder can prove the + Guild's state at issuance, and signed with the same did:key that signs + Agent Passports — one issuer identity, one verification path, no new trust + root for a customer to learn.""" + deep = deep_preflight(store, url) + + gid = store.guild_identity() + if not gid.get("did") or not gid.get("private_key"): + raise EvidenceIssuanceRefused( + "the Guild signing identity is unavailable; refusing to issue an " + "unsigned evidence bundle") + + try: + anchor = store.latest_checkpoint(publish_if_empty=True) + except Exception as exc: # noqa: BLE001 — includes the fail-closed 409s + raise EvidenceIssuanceRefused( + f"could not anchor the bundle to the canonical ledger: " + f"{type(exc).__name__}") from exc + if not anchor or not (anchor.get("checkpoint") or {}).get("head_hash"): + raise EvidenceIssuanceRefused( + "no published checkpoint is available to anchor this bundle") + + issued = _now() + body = { + "type": "AgentGuildEvidenceBundle", + "version": 1, + "subject_endpoint": trustindex.normalise_url(url), + "subject_id": trustindex.fingerprint(url), + "audience": audience or None, + "issued_at": issued.isoformat(), + "valid_until": (issued + timedelta( + seconds=max(60, min(int(ttl_s or DEFAULT_TTL_S), 7 * 24 * 3600))) + ).isoformat(), + "observation": { + "verdict": deep.get("verdict"), + "checks": deep.get("checks", []), + "failed": deep.get("failed", []), + "unknowns": deep.get("unknowns", []), + "method": deep.get("method"), + "limits": deep.get("limits"), + }, + "policy": deep.get("policy"), + "history": deep.get("history"), + "corroboration": deep.get("corroboration"), + "issuer": gid["did"], + "ledger_anchor": { + "checkpoint_index": anchor.get("index"), + "head_hash": (anchor.get("checkpoint") or {}).get("head_hash"), + "published_at": anchor.get("published_at"), + }, + "verification": { + "suite": "eddsa-jcs-2022", + "issuer_did_document": "/.well-known/agent-guild-did.json", + "how": ("canonicalize the bundle WITHOUT the `proof` field (JCS), " + "then verify `proof` as an ed25519 signature over that " + "canonical form using the issuer did:key. No call to the " + "Guild is required — that is the point of the artefact."), + }, + "honesty": ( + "This attests to what the Guild OBSERVED at `issued_at`, not to " + "the future behaviour of the endpoint. `unknowns` are checks that " + "could not be performed and are excluded from the policy verdict, " + "never averaged into it."), + } + try: + proof = sign_jcs(body, gid["private_key"]) + except Exception as exc: # noqa: BLE001 + raise EvidenceIssuanceRefused( + f"signing failed: {type(exc).__name__}") from exc + if not proof: + raise EvidenceIssuanceRefused("signing produced no proof") + + bundle = {**body, "proof": proof} + bundle["bundle_sha256"] = hashlib.sha256( + canonicalize(bundle).encode("utf-8")).hexdigest() + return bundle + + +def verify_bundle(store: Any, bundle: dict[str, Any]) -> dict[str, Any]: + """Verify a bundle we (or a historical Guild key) issued. Free. + + Free on purpose: charging to check an artefact we sold would make the + artefact worth less than we claimed when we sold it.""" + from .crypto import public_key_from_did, verify_jcs + + if not isinstance(bundle, dict) or "proof" not in bundle: + return {"valid": False, "reason": "not a bundle (no proof)"} + body = {k: v for k, v in bundle.items() + if k not in ("proof", "bundle_sha256")} + issuer = str(bundle.get("issuer") or "") + known = [] + try: + known = list(store.guild_did_history()) + except Exception: # noqa: BLE001 + known = [] + if issuer not in known: + return {"valid": False, "reason": "issuer is not a Guild key", + "issuer": issuer} + try: + pub = public_key_from_did(issuer) + ok = verify_jcs(body, str(bundle["proof"]), pub) + except Exception as exc: # noqa: BLE001 + return {"valid": False, "reason": f"malformed proof: {type(exc).__name__}"} + + expired = False + try: + expired = _now() > datetime.fromisoformat(str(bundle.get("valid_until"))) + except (TypeError, ValueError): + expired = True + return { + "valid": bool(ok) and not expired, + "signature_valid": bool(ok), + "expired": expired, + "issuer": issuer, + "subject_endpoint": bundle.get("subject_endpoint"), + "policy_decision": (bundle.get("policy") or {}).get("decision"), + "issued_at": bundle.get("issued_at"), + "note": ("an EXPIRED bundle with a valid signature is still proof of " + "what was observed at `issued_at` — it is simply no longer " + "evidence about now"), + } diff --git a/live/guild/app/experiments.py b/live/guild/app/experiments.py new file mode 100644 index 0000000..1da587f --- /dev/null +++ b/live/guild/app/experiments.py @@ -0,0 +1,318 @@ +"""The autonomous experiment engine — decide without a human, honestly. + +WHY THIS IS NARROW ON PURPOSE + An autonomous loop that can change anything will eventually change the thing + that makes its own numbers look good. So this engine can move exactly two + kinds of variable — a PRICE within its published ceiling, and a COPY variant + — and nothing else. It cannot alter what counts as success, cannot create an + operation, and cannot touch attribution. + +THE RULE THAT MAKES IT TRUSTWORTHY + **An experiment can never count our own traffic.** Qualified exposure is + genuine-external only, through the same central attribution rule used + everywhere else. First-party tooling, crawlers, registry probes and + unknown-attributed traffic are excluded structurally, not filtered by name or + User-Agent. If that leaves a denominator of zero, the verdict is + ``insufficient_evidence`` — never ``kill``, and never ``promote``. + + This is the difference between an experiment engine and a machine for + generating flattering conclusions. A "0% conversion on 1,790 crawler + impressions" verdict is not a finding; it is a category error with a number + attached. + +DECISIONS + ``promote`` enough qualified exposure AND the metric moved + ``kill`` enough qualified exposure AND it did not + ``hold`` running, not yet decidable + ``insufficient_evidence`` the window closed without enough qualified + exposure to learn anything — the honest outcome, + and the one a vanity dashboard never reports + +WHAT COUNTS AS SUCCESS + Genuine external settled revenue, distinct external payers, paid decisions, + externally monitored endpoints, repeat paid use. Indexed inventory, free + checks, page views, crawler reach and passports are SUPPORTING metrics and + can never promote an experiment on their own. +""" +from __future__ import annotations + +import os +from datetime import datetime, timedelta, timezone +from typing import Any, Optional + +from . import pricing + +#: Minimum genuinely-external actors before ANY verdict other than +#: insufficient_evidence may be reached. Deliberately small — we are trying to +#: detect the difference between "nobody wants this" and "nobody has seen it", +#: not to run a statistically powered trial. +DEFAULT_MIN_QUALIFIED = 10 + +#: Hard ceiling on how long an experiment may run before it must conclude. +DEFAULT_WINDOW_DAYS = 14 + +#: The ONLY metrics that may promote an experiment. +PRIMARY_METRICS = ( + "external_settled_revenue_usd", + "distinct_external_payers", + "paid_decisions", + "externally_monitored_endpoints", + "repeat_paid_callers", +) + +#: Never sufficient on their own, no matter how large. +SUPPORTING_METRICS = ( + "indexed_entries", "free_preflight_runs", "evidence_page_views", + "crawler_reach", "passports_issued", "offer_served", +) + + +def min_qualified() -> int: + try: + return max(1, min(int(os.environ.get("GUILD_EXP_MIN_QUALIFIED") + or DEFAULT_MIN_QUALIFIED), 10_000)) + except (TypeError, ValueError): + return DEFAULT_MIN_QUALIFIED + + +def window_days() -> int: + try: + return max(1, min(int(os.environ.get("GUILD_EXP_WINDOW_DAYS") + or DEFAULT_WINDOW_DAYS), 90)) + except (TypeError, ValueError): + return DEFAULT_WINDOW_DAYS + + +def _now() -> datetime: + return datetime.now(timezone.utc) + + +def define(store: Any, key: str, *, hypothesis: str, variable: str, + baseline: dict[str, Any]) -> dict[str, Any]: + """Register a bounded, reversible experiment. Idempotent by key.""" + with store.lock, store._txn(): + existing = store.experiments.get(key) + if existing: + return existing + rec = { + "key": key, + "hypothesis": hypothesis, + "variable": variable, + "baseline": baseline, + "started_at": _now().isoformat(), + "window_days": window_days(), + "min_qualified": min_qualified(), + "status": "running", + "decision": None, + "decided_at": None, + "evidence": None, + } + store.experiments[key] = rec + if store.backend is not None: + store._persist_kv("experiments", store.experiments) + store._save() + return rec + + +def qualified_exposure(store: Any) -> dict[str, Any]: + """Genuinely-external actors who reached a decision surface. + + Uses the SAME central attribution rule as every other honest number in the + service. Crawlers, registry probes, our own tooling and unknown-attributed + traffic are excluded structurally — never by name-matching a User-Agent, + which is exactly how self-traffic gets laundered into a growth metric.""" + from . import attribution + + decision_surfaces = {"preflight_run", "deep_preflight_run", + "evidence_bundle_issued", "watch_provisioned", + "index_view"} + actors: set[str] = set() + events = 0 + for e in getattr(store, "events", []): + if e.get("type") not in decision_surfaces: + continue + if e.get("fp") or e.get("first_party"): + continue + cls = attribution.caller_class(e) + if cls in ("AG_INTERNAL", "AG_TEST", "OPERATOR", "REGISTRY_CRAWLER"): + continue + if not (attribution.may_count_as_external_growth(cls) + and attribution.is_genuine_external(e)): + continue + events += 1 + key = e.get("key") or "anon" + if key != "anon": + actors.add(key) + return { + "qualified_actors": len(actors), + "qualified_events": events, + "rule": ("genuine-external only, via attribution.caller_class + " + "is_genuine_external. Crawlers, first-party tooling and " + "unknown-attributed traffic are excluded structurally, not by " + "matching a User-Agent string."), + } + + +def commercial_metrics(store: Any) -> dict[str, Any]: + """The primary metrics. Revenue is REAL money only.""" + payers: set[str] = set() + paid_decisions = 0 + repeat: dict[str, int] = {} + for e in getattr(store, "events", []): + if e.get("type") in ("deep_preflight_run", "evidence_bundle_issued") \ + and e.get("paid"): + paid_decisions += 1 + key = e.get("key") or "" + if key and key != "anon" and not e.get("fp"): + payers.add(key) + repeat[key] = repeat.get(key, 0) + 1 + + revenue_usd = 0.0 + try: + real = (store.escrow_summary() or {}).get("real_settlement") or {} + revenue_usd = float( + real.get("independently_attested_external_revenue_usd") or 0.0) + except Exception: # noqa: BLE001 + revenue_usd = 0.0 + + monitored = 0 + for w in getattr(store, "watches", {}).values(): + if not w.get("active"): + continue + acct = (getattr(store, "accounts", {}) or {}).get(w.get("owner_key") or "") + if acct and acct.get("first_party"): + continue # our own watch is not a customer + monitored += 1 + + return { + "external_settled_revenue_usd": revenue_usd, + "distinct_external_payers": len(payers), + "paid_decisions": paid_decisions, + "externally_monitored_endpoints": monitored, + "repeat_paid_callers": sum(1 for n in repeat.values() if n > 1), + "revenue_definition": ( + "independently confirmed EXTERNAL mainnet settlement only. " + "Sandbox credits, first-party canaries, testnet funds and internal " + "transfers are excluded by construction and are not money."), + } + + +def evaluate(store: Any, key: str) -> dict[str, Any]: + """Decide an experiment — or refuse to, honestly.""" + rec = store.experiments.get(key) + if not rec: + return {"key": key, "decision": None, "reason": "unknown experiment"} + + exposure = qualified_exposure(store) + metrics = commercial_metrics(store) + baseline = rec.get("baseline") or {} + started = rec.get("started_at") + try: + elapsed = _now() - datetime.fromisoformat(str(started)) + except (TypeError, ValueError): + elapsed = timedelta(0) + expired = elapsed > timedelta(days=int(rec.get("window_days", window_days()))) + + moved = any( + float(metrics.get(m) or 0) > float(baseline.get(m) or 0) + for m in PRIMARY_METRICS) + enough = exposure["qualified_actors"] >= int( + rec.get("min_qualified", min_qualified())) + + if not enough: + decision = "insufficient_evidence" if expired else "hold" + reason = ( + f"{exposure['qualified_actors']} qualified external actor(s) — " + f"below the {rec.get('min_qualified')} needed to tell 'nobody " + "wants this' apart from 'nobody has seen it'. " + + ("The window closed without enough exposure to learn anything, " + "which is a real outcome and is reported as such — not as a " + "failure of the offer." if expired else "Still gathering.")) + elif moved: + decision, reason = "promote", ( + "a PRIMARY commercial metric moved against baseline with " + f"{exposure['qualified_actors']} qualified external actors") + else: + decision, reason = "kill", ( + f"{exposure['qualified_actors']} qualified external actors saw it " + "and no primary commercial metric moved. Supporting metrics " + "(reach, inventory, free checks) cannot rescue this verdict.") + + evidence = {"exposure": exposure, "metrics": metrics, "baseline": baseline, + "elapsed_days": round(elapsed.total_seconds() / 86400, 2), + "window_expired": expired} + with store.lock, store._txn(): + live = store.experiments.get(key) or rec + live["decision"] = decision + live["evidence"] = evidence + live["status"] = "running" if decision == "hold" else "decided" + live["decided_at"] = None if decision == "hold" else _now().isoformat() + store.experiments[key] = live + if store.backend is not None: + store._persist_kv("experiments", store.experiments) + store._save() + return {"key": key, "decision": decision, "reason": reason, + "evidence": evidence} + + +def next_action(store: Any, key: str) -> dict[str, Any]: + """The single safest reversible next move, chosen without a human. + + On `kill` the response is to CHANGE THE OFFER — lower the price one step + within its published ceiling — not to keep running and keep reporting + reach. The mandate is explicit: if no genuine paid demand appears after the + documented threshold, change the offer or price rather than celebrating + reach.""" + verdict = evaluate(store, key) + rec = store.experiments.get(key) or {} + variable = rec.get("variable") or "" + decision = verdict["decision"] + + if decision == "promote": + return {**verdict, "action": "keep", "change": None, + "rationale": "it is working; changing it now would destroy the " + "only signal we have"} + if decision == "hold": + return {**verdict, "action": "wait", "change": None, + "rationale": "not yet decidable on qualified exposure"} + if decision == "insufficient_evidence": + return {**verdict, "action": "increase_qualified_exposure", + "change": None, + "rationale": ("the offer was never actually tested. Fix " + "DISTRIBUTION to qualified callers before " + "touching the price — repricing something nobody " + "saw teaches us nothing")} + + # decision == kill → move the price one step down, within the ceiling + if variable.startswith("price:"): + op = variable.split(":", 1)[1] + current = pricing.price(op) + proposed = max(0, int(current * 0.5)) + return {**verdict, "action": "reprice", + "change": {"operation": op, "from_credits": current, + "to_credits": proposed, + "env": pricing._env_key(op), + "reversible": True, + "within_ceiling": proposed <= pricing.CEILINGS.get(op, 0)}, + "rationale": ("qualified callers saw the offer and did not buy. " + "Halve the price within its published ceiling and " + "re-run — this is a config change and a rollback, " + "not a deploy")} + return {**verdict, "action": "change_offer", "change": None, + "rationale": "qualified callers saw it and did not buy; the offer " + "itself needs to change"} + + +def snapshot(store: Any) -> dict[str, Any]: + """Everything an autonomous report needs, revenue first.""" + return { + "commercial": commercial_metrics(store), + "qualified_exposure": qualified_exposure(store), + "experiments": {k: {"status": v.get("status"), + "decision": v.get("decision"), + "variable": v.get("variable"), + "hypothesis": v.get("hypothesis")} + for k, v in (store.experiments or {}).items()}, + "primary_metrics": list(PRIMARY_METRICS), + "supporting_metrics_never_sufficient": list(SUPPORTING_METRICS), + } diff --git a/live/guild/app/indexops.py b/live/guild/app/indexops.py new file mode 100644 index 0000000..cc6951d --- /dev/null +++ b/live/guild/app/indexops.py @@ -0,0 +1,297 @@ +"""Index operations: ingest, recheck, watch — the autonomous half of the index. + +Kept out of ``store.py`` deliberately. These are the only functions in the +service that reach out to third-party infrastructure on a schedule, so they are +in one file where their bounds can be read in a single sitting: + + * ingest is default-OFF for remote sources and capped per run; + * recheck probes at most ``recheck_batch()`` endpoints per cycle, oldest + observation first, so the loop degrades to slow rather than to abusive; + * a watch is charged per cycle ACTUALLY performed, so a dormant endpoint bills + nothing and we can never invoice for work we did not do. + +Everything writes through the same Store lock/transaction discipline as the +rest of the service, so an index write cannot half-land. +""" +from __future__ import annotations + +import hashlib +import os +import secrets +from datetime import datetime, timedelta, timezone +from typing import Any, Callable, Optional + +from . import indexsources, preflight, trustindex + + +def _now() -> datetime: + return datetime.now(timezone.utc) + + +def _iso() -> str: + return _now().isoformat() + + +# -------------------------------------------------------------------------- +# Ingest +# -------------------------------------------------------------------------- +def ingest(store: Any, records: Optional[list[dict[str, Any]]] = None + ) -> dict[str, Any]: + """Fold source records into the index. Deduplicates; never double-counts. + + A record that resolves to an endpoint already present adds PROVENANCE (a + second source saw it) and nothing else. Inventory is not a success metric, + so an ingest run that adds zero new endpoints is a perfectly good run.""" + if records is None: + records = indexsources.collect(store) + added = updated = skipped = 0 + with store.lock, store._txn(): + for rec in records: + url = (rec.get("endpoint") or "").strip() + norm = trustindex.normalise_url(url) + if not norm: + skipped += 1 + continue + fp = trustindex.fingerprint(norm) + entry = store.trust_index.get(fp) + if entry is None: + entry = trustindex.new_entry( + norm, rec.get("source", "unknown"), + declared=rec.get("declared"), did=rec.get("did", "")) + # Ownership from a DETERMINISTIC control only (the admin-gated + # first_party flag), never inferred from a name or User-Agent. + if rec.get("first_party"): + entry["owner_class"] = trustindex.OWNER_FIRST_PARTY + store.trust_index[fp] = entry + added += 1 + else: + trustindex.merge_source(entry, rec.get("source", "unknown")) + if rec.get("declared"): + entry["declared"] = {**(entry.get("declared") or {}), + **rec["declared"]} + if rec.get("first_party"): + entry["owner_class"] = trustindex.OWNER_FIRST_PARTY + updated += 1 + if store.backend is not None: + store._persist_kv("trust_index", store.trust_index) + store._save() + return {"added": added, "provenance_updated": updated, + "skipped_unusable": skipped, + "total_entries": len(store.trust_index), + "note": ("added = endpoints not previously known. " + "provenance_updated = already known, another source saw " + "it — NOT a new endpoint. Inventory is a supporting " + "metric and is never reported as adoption.")} + + +def _owner_class_for(store: Any, entry: dict[str, Any]) -> str: + """Ownership from deterministic signals only. + + An endpoint is first-party if it belongs to an agent we have explicitly + marked first-party through the admin control. It is externally owned only + if it is a live, observed endpoint that is NOT ours. Everything else stays + `unknown` — which is the whole point: unknown is never promoted to + external to make a number look better.""" + if entry.get("owner_class") == trustindex.OWNER_FIRST_PARTY: + return trustindex.OWNER_FIRST_PARTY + did = entry.get("did") or "" + if did: + try: + agent = store.agent_by_did(did) + except Exception: # noqa: BLE001 + agent = None + if agent and agent.get("first_party"): + return trustindex.OWNER_FIRST_PARTY + if agent: + return trustindex.OWNER_EXTERNAL + if entry.get("observation"): + return trustindex.OWNER_EXTERNAL + return trustindex.OWNER_UNKNOWN + + +def recheck_one(store: Any, fingerprint: str, *, + runner: Optional[Callable] = None) -> Optional[dict[str, Any]]: + """Observe one endpoint and fold the result in. Returns the public view.""" + entry = store.trust_index.get(fingerprint) + if entry is None: + return None + probe = runner or (lambda url: preflight.run(url, store=store)) + result = probe(entry["endpoint"]) + with store.lock, store._txn(): + entry = store.trust_index.get(fingerprint) or entry + trustindex.apply_observation( + entry, result, owner_class=_owner_class_for(store, entry)) + store.trust_index[fingerprint] = entry + if store.backend is not None: + store._persist_kv("trust_index", store.trust_index) + store._save() + store.record_event(None, "index_observation", endpoint="index", + target=entry["endpoint"], status=entry["status"], + first_party=True) + return trustindex.public_view(entry, detail=True) + + +def recheck_due(store: Any, *, limit: Optional[int] = None, + runner: Optional[Callable] = None) -> dict[str, Any]: + """Recheck the stalest entries, bounded. + + Oldest observation first so a large index degrades to SLOW coverage rather + than to an unbounded burst against other people's servers.""" + cap = limit or trustindex.recheck_batch() + due = [e for e in store.trust_index.values() if trustindex.is_stale(e)] + due.sort(key=lambda e: (e.get("observed_at") or "")) + checked = 0 + transitions: list[dict[str, Any]] = [] + for entry in due[:cap]: + before = entry.get("status") + view = recheck_one(store, entry["id"], runner=runner) + checked += 1 + if view and view.get("status") != before: + transitions.append({"id": entry["id"], + "endpoint": entry["endpoint"], + "from": before, "to": view.get("status")}) + return {"checked": checked, "due": len(due), "capped_at": cap, + "transitions": transitions} + + +# -------------------------------------------------------------------------- +# Continuous watch (self-provisioned — no staff, no onboarding) +# -------------------------------------------------------------------------- +def watch_id(owner_key: str, endpoint: str) -> str: + """Deterministic id, so provisioning the SAME watch twice is idempotent + rather than creating a duplicate subscription that bills twice.""" + raw = f"{owner_key}|{trustindex.normalise_url(endpoint)}".encode("utf-8") + return "wch_" + hashlib.sha256(raw).hexdigest()[:16] + + +def provision_watch(store: Any, owner_key: str, endpoint: str, *, + interval_s: int = 3600) -> dict[str, Any]: + """Create (or return) a continuous watch. Idempotent by (owner, endpoint). + + No human is involved and none is required. Provisioning is free — charging + before any observation exists would be charging for a promise.""" + norm = trustindex.normalise_url(endpoint) + if not norm: + raise ValueError("unusable endpoint url") + wid = watch_id(owner_key, norm) + interval = max(300, min(int(interval_s or 3600), 7 * 24 * 3600)) + with store.lock, store._txn(): + existing = store.watches.get(wid) + if existing: + existing["interval_s"] = interval + existing["active"] = True + store.watches[wid] = existing + if store.backend is not None: + store._persist_kv("watches", store.watches) + store._save() + return {**existing, "created": False, + "note": "existing watch returned — provisioning is " + "idempotent, so a retry never bills twice"} + # ensure the endpoint is in the index so the watch has something to read + fp = trustindex.fingerprint(norm) + if fp not in store.trust_index: + store.trust_index[fp] = trustindex.new_entry(norm, "watch_request") + if store.backend is not None: + store._persist_kv("trust_index", store.trust_index) + rec = { + "id": wid, "owner_key": owner_key, "endpoint": norm, + "endpoint_id": fp, "interval_s": interval, + "created_at": _iso(), "last_cycle_at": None, + "cycles_billed": 0, "credits_spent": 0, + "active": True, "last_status": None, + "changes": [], + } + store.watches[wid] = rec + if store.backend is not None: + store._persist_kv("watches", store.watches) + store._save() + return {**rec, "created": True} + + +def watch_due(store: Any) -> list[dict[str, Any]]: + out = [] + for rec in store.watches.values(): + if not rec.get("active"): + continue + last = rec.get("last_cycle_at") + if not last: + out.append(rec) + continue + try: + when = datetime.fromisoformat(last) + except (TypeError, ValueError): + out.append(rec) + continue + if _now() - when >= timedelta(seconds=int(rec.get("interval_s", 3600))): + out.append(rec) + return out + + +def run_watch_cycle(store: Any, rec: dict[str, Any], *, + charge: Optional[Callable] = None, + runner: Optional[Callable] = None) -> dict[str, Any]: + """One watch cycle: observe, record any CHANGE, then charge. + + Order matters. The charge happens AFTER the observation succeeds, so a + failed cycle is never billed. If the charge itself fails (out of credits), + the observation is kept — we already did the work and the customer should + still see it — and the watch is suspended rather than silently continuing + to consume our outbound budget for free.""" + view = recheck_one(store, rec["endpoint_id"], runner=runner) + if view is None: + return {"id": rec["id"], "cycled": False, "reason": "endpoint_gone"} + new_status = view.get("status") + changed = rec.get("last_status") is not None and rec["last_status"] != new_status + charged = 0 + suspended = False + if charge is not None: + try: + charged = int(charge(rec["owner_key"]) or 0) + except Exception: # noqa: BLE001 — insufficient credits, etc. + suspended = True + with store.lock, store._txn(): + live = store.watches.get(rec["id"]) or rec + if changed: + live.setdefault("changes", []).append({ + "at": _iso(), "from": rec.get("last_status"), "to": new_status, + "failed": (view.get("observed") or {}).get("failed", []), + }) + live["changes"] = live["changes"][-50:] + live["last_status"] = new_status + live["last_cycle_at"] = _iso() + if charged: + live["cycles_billed"] = int(live.get("cycles_billed", 0)) + 1 + live["credits_spent"] = int(live.get("credits_spent", 0)) + charged + if suspended: + live["active"] = False + live["suspended_reason"] = "payment_failed" + live["suspended_at"] = _iso() + store.watches[rec["id"]] = live + if store.backend is not None: + store._persist_kv("watches", store.watches) + store._save() + return {"id": rec["id"], "cycled": True, "status": new_status, + "changed": changed, "charged_credits": charged, + "suspended": suspended} + + +def watch_feed(store: Any, wid: str) -> Optional[dict[str, Any]]: + """Machine-readable change feed for one watch. No dashboard required.""" + rec = store.watches.get(wid) + if not rec: + return None + entry = store.trust_index.get(rec.get("endpoint_id") or "") + return { + "id": rec["id"], + "endpoint": rec["endpoint"], + "active": rec.get("active", False), + "interval_s": rec.get("interval_s"), + "last_cycle_at": rec.get("last_cycle_at"), + "cycles_billed": rec.get("cycles_billed", 0), + "credits_spent": rec.get("credits_spent", 0), + "current": trustindex.public_view(entry) if entry else None, + "changes": list(reversed(rec.get("changes", []))), + "suspended_reason": rec.get("suspended_reason"), + "billing_note": ("charged per recheck ACTUALLY performed; a cycle that " + "could not observe the endpoint is not billed"), + } diff --git a/live/guild/app/indexsources.py b/live/guild/app/indexsources.py new file mode 100644 index 0000000..69c2e9e --- /dev/null +++ b/live/guild/app/indexsources.py @@ -0,0 +1,156 @@ +"""Lawful public-registry adapters for the trust index. + +WHAT THIS MAY DO + Read documented, public, read-only registry APIs — the same endpoints a + browser or any other client hits — at a bounded rate, with a truthful User- + Agent that identifies us and links to what we do with the data. + +WHAT THIS MAY NOT DO, EVER + Scrape behind authentication, ignore robots directives, evade rate limits, + impersonate another client, or use any private/undocumented endpoint. The + mandate is explicit: no Terms-of-Service circumvention, no deceptive traffic. + If an adapter needs a credential to work, it does not ship — it stops and + says so. + +WHY BOUNDED HARD + An autonomous ingest loop pointed at someone else's infrastructure is, by + construction, one bug away from being abuse. Every adapter here is capped in + page count and total records per run, has a short timeout, and fails silent- + and-empty rather than retrying aggressively. Being a bad citizen would also + destroy the only asset this product has, which is being the party whose + measurements can be trusted. +""" +from __future__ import annotations + +import json +import os +import urllib.error +import urllib.request +from typing import Any, Optional + +#: Truthful, contactable identification. Never impersonates a browser. +USER_AGENT = ("agent-guild-index/1.0 (+https://agent-guild-5d5r.onrender.com/" + "index; public trust index; contact via the agent card)") + +TIMEOUT_S = 12.0 +MAX_PAGES = 3 +MAX_RECORDS_PER_RUN = 200 + + +def enabled() -> bool: + """Ingest is OFF unless explicitly enabled. + + Default-off is deliberate: outbound traffic to third-party infrastructure + should never start because a container restarted.""" + return (os.environ.get("GUILD_INDEX_INGEST") or "0").strip() == "1" + + +def _get_json(url: str, timeout: float = TIMEOUT_S) -> Optional[Any]: + """One bounded public GET. Never raises, never retries hard.""" + req = urllib.request.Request(url, headers={ + "accept": "application/json", "user-agent": USER_AGENT}) + try: + with urllib.request.urlopen(req, timeout=timeout) as resp: + if resp.status != 200: + return None + return json.loads(resp.read().decode("utf-8", "replace")) + except (urllib.error.HTTPError, urllib.error.URLError, TimeoutError, + ValueError, OSError): + return None + + +# -------------------------------------------------------------------------- +# Adapter: the official MCP Registry (public, documented, read-only) +# -------------------------------------------------------------------------- +MCP_REGISTRY = "https://registry.modelcontextprotocol.io/v0/servers" + + +def from_mcp_registry(limit: int = 100) -> list[dict[str, Any]]: + """Public MCP servers with a REMOTE endpoint. + + Only remote servers are indexed. A server distributed as an npm/PyPI + package has no endpoint to call, so we could never hold an observation of + it — and an index entry we can never observe is exactly the listing-shaped + filler this product exists to replace.""" + out: list[dict[str, Any]] = [] + cursor = "" + for _ in range(MAX_PAGES): + url = f"{MCP_REGISTRY}?limit=50" + (f"&cursor={cursor}" if cursor else "") + page = _get_json(url) + if not isinstance(page, dict): + break + for row in page.get("servers", []): + server = row.get("server") if isinstance(row, dict) else None + if not isinstance(server, dict): + continue + meta = (row.get("_meta") or {}).get( + "io.modelcontextprotocol.registry/official") or {} + if meta.get("status") not in (None, "active"): + continue + if meta.get("isLatest") is False: + continue # superseded versions are not separate services + for remote in server.get("remotes") or []: + endpoint = (remote or {}).get("url") + if not endpoint: + continue + out.append({ + "endpoint": endpoint, + "source": "mcp_registry", + "declared": { + "name": server.get("name"), + "description": (server.get("description") or "")[:400], + "capabilities": [], + "protocol": "mcp", + "version": server.get("version"), + "website": server.get("websiteUrl"), + }, + }) + if len(out) >= min(limit, MAX_RECORDS_PER_RUN): + return out + cursor = (page.get("metadata") or {}).get("nextCursor") or "" + if not cursor: + break + return out + + +# -------------------------------------------------------------------------- +# Adapter: our OWN demand/candidate surface (already lawful, already ours) +# -------------------------------------------------------------------------- +def from_local_agents(store: Any, limit: int = 200) -> list[dict[str, Any]]: + """Endpoints already declared to the Guild by registered agents. + + These arrive with an ownership signal we can trust — `first_party` is set + by an admin-gated deterministic control, never inferred from a name or a + User-Agent. That matters because ownership is what separates a growth + metric from self-traffic.""" + out: list[dict[str, Any]] = [] + for agent_id, rec in list(getattr(store, "agents", {}).items())[:limit * 2]: + endpoint = ((rec.get("metadata") or {}).get("endpoint") or "").strip() + if not endpoint: + continue + out.append({ + "endpoint": endpoint, + "source": "guild_registration", + "did": rec.get("did") or "", + "first_party": bool(rec.get("first_party")), + "declared": { + "name": rec.get("name"), + "capabilities": rec.get("capabilities", []), + "protocol": "a2a", + "agent_id": agent_id, + }, + }) + if len(out) >= limit: + break + return out + + +def collect(store: Any, *, include_remote: bool = True) -> list[dict[str, Any]]: + """All records for one ingest run, bounded in total. + + Local registrations are collected unconditionally (they are already ours); + remote public registries only when ingest is explicitly enabled.""" + records = from_local_agents(store) + if include_remote and enabled(): + records += from_mcp_registry() + return records[:MAX_RECORDS_PER_RUN] diff --git a/live/guild/app/main.py b/live/guild/app/main.py index 11eed43..590e8ee 100644 --- a/live/guild/app/main.py +++ b/live/guild/app/main.py @@ -43,6 +43,11 @@ from .billing import InsufficientCredits, UnknownAccount, PRICING, CREDIT_USD from . import instanceid from . import preflight +from . import pricing +from . import trustindex +from . import indexops +from . import deepcheck +from . import experiments from .state import store from .store import CanonicalWriteRefused from .reachability import url_policy_check @@ -2805,10 +2810,24 @@ def llms_txt(): " 3. GET /agents/{id}/passport → a Guild-signed Verifiable Credential;\n" " any party verifies it offline (POST /credentials/verify, Guild did at\n" " /.well-known/agent-guild-did.json). Live badge: GET /agents/{id}/badge.svg.\n\n" - "## About to delegate to an agent you did not write? Check it first (free)\n" + "## START HERE: allow, caution or block an endpoint (free)\n" "GET /preflight?url= (or the guild_preflight MCP tool)\n" - "One unauthenticated call, live at request time, separating what an\n" - "endpoint CLAIMS from what it just PROVED: does it complete a real\n" + "One unauthenticated call, live at request time. You get back what the\n" + "endpoint CLAIMS and, separately, what it just PROVED.\n" + "GET /index the public index: every endpoint we know, and\n" + " what happened when we actually called it\n" + "GET /index/search?q= search it\n" + "GET /preflight/deep?url= PAID: adds drift history, cross-source\n" + " corroboration and an explicit allow/caution/\n" + " block policy verdict\n" + "POST /evidence/bundle PAID: a signed snapshot you keep and verify\n" + " offline, without calling us\n" + "POST /watch PAID per cycle: continuous monitoring you\n" + " provision yourself — no onboarding, no human\n" + "GET /pricing what the paid layer costs, and why\n\n" + "## Why this exists\n" + "Separating what an endpoint CLAIMS from what it just PROVED: does it\n" + "complete a real\n" "protocol handshake (not merely answer 200), does its Agent Card resolve,\n" "is that card signed, and does its payment claim actually challenge with\n" "a 402. Measured across the live ecosystem on 2026-07-31: 92.9% of\n" @@ -2942,6 +2961,318 @@ def delegation_preflight(request: Request, url: str = Query( return out +@app.get("/preflight/deep") +def deep_preflight_route(request: Request, response: Response, + url: str = Query(..., description="endpoint to check"), + x_api_key: Optional[str] = Header(None)): + """PAID deep preflight: live checks **plus** drift history, cross-source + corroboration and an explicit allow / caution / block policy verdict. + + Metered through the same shared paid-operation gateway as every other + priced read — one semantic operation, one canonical resource URL, one + policy across HTTP, MCP and A2A. Pay with credits or x402. + + The free tier (`GET /preflight`) is not degraded to make this attractive: + it returns the full live check set and verdict, and always will.""" + meter(payments.deep_preflight_request(url), x_api_key, response) + out = deepcheck.deep_preflight(store, url) + store.record_event(creds.sanitize_actor_key(x_api_key) if x_api_key else None, + "deep_preflight_run", ua=_ua.get(), endpoint="preflight_deep", + target=url, paid=True, + verdict=(out.get("policy") or {}).get("decision")) + return out + + +@app.post("/evidence/bundle") +def evidence_bundle_route(body: dict[str, Any], response: Response, + x_api_key: Optional[str] = Header(None)): + """PAID signed evidence bundle — a portable, offline-verifiable snapshot. + + FAILS CLOSED. If the bundle cannot be fully produced, signed and anchored + to the published checkpoint feed, this raises 409 and **the caller is not + charged** — the meter runs only after issuance succeeds. Selling a degraded + evidence object is worse than selling nothing, because the buyer would rely + on it exactly when it is weakest.""" + url = str(body.get("url") or "").strip() + if not url: + raise HTTPException(422, "url is required") + ttl = int(body.get("ttl_seconds") or deepcheck.DEFAULT_TTL_S) + preq = payments.evidence_bundle_request(url, ttl) + # Produce FIRST, charge second: a refusal must never bill. + try: + bundle = deepcheck.evidence_bundle( + store, url, ttl_s=ttl, audience=str(body.get("audience") or "")) + except deepcheck.EvidenceIssuanceRefused as e: + raise HTTPException(409, { + "error": "evidence_issuance_refused", "code": e.code, + "detail": str(e), + "billing": "NOT CHARGED — issuance failed, so no meter ran"}) + meter(preq, x_api_key, response) + store.record_event(creds.sanitize_actor_key(x_api_key) if x_api_key else None, + "evidence_bundle_issued", ua=_ua.get(), + endpoint="evidence_bundle", target=url, paid=True) + return bundle + + +@app.post("/evidence/verify") +def evidence_verify_route(body: dict[str, Any]): + """Verify an evidence bundle. FREE, always. + + Charging to check an artefact we sold would make the artefact worth less + than we claimed when we sold it.""" + bundle = body.get("bundle") if isinstance(body.get("bundle"), dict) else body + return deepcheck.verify_bundle(store, bundle) + + +@app.post("/watch") +def watch_provision_route(body: dict[str, Any], response: Response, + x_api_key: Optional[str] = Header(None)): + """Self-provision continuous monitoring. No onboarding, no human, no call. + + Provisioning is FREE and idempotent by (caller, endpoint): a retry returns + the existing watch rather than creating a second subscription that bills + twice. Each recheck CYCLE is charged only when it actually runs — a dormant + endpoint costs nothing.""" + if not x_api_key: + raise HTTPException(401, { + "error": "billing_key_required", + "detail": "a watch bills per cycle, so it needs an account to bill", + "self_serve": "POST /billing/trial — credits with no human involved"}) + url = str(body.get("url") or "").strip() + if not url: + raise HTTPException(422, "url is required") + try: + rec = indexops.provision_watch( + store, creds.sanitize_actor_key(x_api_key), url, + interval_s=int(body.get("interval_s") or 3600)) + except ValueError as e: + raise HTTPException(422, str(e)) + store.record_event(creds.sanitize_actor_key(x_api_key), "watch_provisioned", + ua=_ua.get(), endpoint="watch", target=url) + return {**rec, "price_per_cycle_credits": pricing.price("watch_cycle"), + "feed": f"GET /watch/{rec['id']}", + "billing": ("charged per recheck ACTUALLY performed; provisioning " + "is free because charging before any observation " + "exists would be charging for a promise")} + + +@app.get("/watch/{watch_id}") +def watch_feed_route(watch_id: str, x_api_key: Optional[str] = Header(None)): + """The machine-readable change feed for one watch. Free to read — you have + already paid for the cycles that produced it.""" + feed = indexops.watch_feed(store, watch_id) + if not feed: + raise HTTPException(404, "watch not found") + rec = store.watches.get(watch_id) or {} + if rec.get("owner_key") and creds.sanitize_actor_key(x_api_key or "") != rec["owner_key"]: + raise HTTPException(403, "this watch belongs to another caller") + return feed + + +# --- the public trust index (free) ----------------------------------------- +# "Can I safely use or pay this specific endpoint right now?" — the product +# question. The index answers it at scale; /preflight answers it for one +# endpoint on demand. Both are free and account-free: a paywall in front of +# "does this thing work" would make the ecosystem worse and the index poorer. + + +@app.get("/index") +def index_list(status: Optional[str] = Query( + None, description="indexed | live | degraded | unreachable"), + owner: Optional[str] = Query( + None, description="externally_owned | first_party | unknown"), + observed_only: bool = Query( + False, description="only entries the Guild has actually called"), + limit: int = Query(50, ge=1, le=200), + offset: int = Query(0, ge=0)): + """The public index of agent endpoints, with WHAT WE OBSERVED — free. + + Each entry keeps `claimed` and `observed` in separate blocks. A listing is + a claim; only `observed` reflects a call we made ourselves. Measured + 2026-07-31: 92.9% of registry entries report healthy, 33.9% complete a + task — which is why this index never promotes a listing to 'live' without + an observation of its own.""" + rows = list(store.trust_index.values()) + if status: + rows = [r for r in rows if r.get("status") == status] + if owner: + rows = [r for r in rows if r.get("owner_class") == owner] + if observed_only: + rows = [r for r in rows if r.get("observation")] + rows.sort(key=lambda r: (r.get("observed_at") or "", r.get("endpoint") or ""), + reverse=True) + page = rows[offset:offset + limit] + return { + "summary": trustindex.summarise(store.trust_index.values()), + "count": len(rows), + "offset": offset, + "limit": limit, + "entries": [trustindex.public_view(e) for e in page], + "free": ("this index and GET /preflight are free and need no account. " + "GET /preflight/deep and POST /evidence/bundle are paid — see " + "GET /pricing"), + } + + +@app.get("/index/search") +def index_search(q: str = Query("", description="substring of endpoint or name"), + capability: str = Query("", description="declared capability"), + limit: int = Query(25, ge=1, le=100)): + """Search the index. Free. Matches on endpoint, declared name and + declared capability — all three are CLAIMS, and are labelled as such in + every result.""" + needle = (q or "").strip().lower() + cap = (capability or "").strip().lower() + hits = [] + for e in store.trust_index.values(): + declared = e.get("declared") or {} + hay = " ".join([ + str(e.get("endpoint") or ""), str(declared.get("name") or ""), + " ".join(str(c) for c in declared.get("capabilities", [])), + ]).lower() + if needle and needle not in hay: + continue + if cap and cap not in " ".join( + str(c).lower() for c in declared.get("capabilities", [])): + continue + hits.append(e) + hits.sort(key=lambda r: (bool(r.get("observation")), + r.get("observed_at") or ""), reverse=True) + return {"query": {"q": q, "capability": capability}, + "count": len(hits), + "results": [trustindex.public_view(e) for e in hits[:limit]], + "ranking": ("observed entries rank above never-called listings — " + "evidence outranks a claim, always")} + + +@app.get("/index/{endpoint_id}") +def index_detail(endpoint_id: str): + """Full public detail for one endpoint: every check, drift history and + provenance. Free.""" + entry = store.trust_index.get(endpoint_id) + if not entry: + raise HTTPException(404, "endpoint not in the index") + view = trustindex.public_view(entry, detail=True) + view["actions"] = { + "free_recheck_now": f"GET /preflight?url={entry.get('endpoint')}", + "paid_deep_check": f"GET /preflight/deep?url={entry.get('endpoint')}", + "signed_evidence": "POST /evidence/bundle {\"url\": \"…\"}", + "continuous_watch": "POST /watch {\"url\": \"…\"}", + } + view["page"] = (f"/index/{endpoint_id}/evidence" + if trustindex.is_page_worthy(entry) else None) + return view + + +@app.get("/index/{endpoint_id}/evidence", response_class=HTMLResponse) +def index_evidence_page(endpoint_id: str): + """A human-readable evidence page — published ONLY where we hold a real + observation. + + An entry we have merely seen listed gets no page: it would contain nothing + a reader cannot get from the registry, and publishing those at scale is SEO + spam. `trustindex.is_page_worthy` is the gate, and it returns 404 rather + than generating filler.""" + entry = store.trust_index.get(endpoint_id) + if not entry: + raise HTTPException(404, "endpoint not in the index") + if not trustindex.is_page_worthy(entry): + raise HTTPException(404, "no observation yet — nothing worth publishing") + view = trustindex.public_view(entry, detail=True) + obs = view.get("observed") or {} + rows = "".join( + f"{c.get('check')}" + f"{c.get('status')}{(c.get('detail') or '')[:300]}" + for c in (obs.get("checks") or [])) + drift = "".join(f"
  • {d.get('at')}: {d.get('from')} → {d.get('to')}
    • " + for d in (view.get("drift") or [])) or "
  • no changes recorded
    • " + return HTMLResponse(f""" +Evidence: {view.get('endpoint')} — Agent Guild + + +

    {view.get('endpoint')}

    +

    Status {view.get('status')} · observed +{view.get('observation_age_seconds')}s ago · +{view.get('observation_count')} observation(s) +{' · STALE' if view.get('stale') else ''}

    +

    What we observed

    +{rows}
    CheckResultDetail
    +

    Checks reported unknown could not be performed. They are +excluded from the verdict, never averaged into it.

    +

    Change history

      {drift}
    +

    What is claimed

    +

    Self-declared by the endpoint or its registry, not +verified: {view.get('claimed', {}).get('name') or '—'} · +sources: {', '.join(view.get('claimed', {}).get('sources') or []) or '—'}

    +

    Check it yourself

    +

    GET /preflight?url={view.get('endpoint')} — free, no account, +runs live at request time.

    +

    Agent Guild publishes this page only for endpoints it has actually +called. Index · Pricing

    +""") + + +@app.get("/commercial") +def commercial_report(): + """The commercial scorecard, revenue first — the number that decides. + + Deliberately ordered so the only figures that can carry a decision come + first, and the flattering ones are labelled as unable to carry one. + Reach, inventory, free checks and passports appear under + `supporting_never_sufficient` because that is exactly what they are.""" + snap = experiments.snapshot(store) + idx = trustindex.summarise(store.trust_index.values()) + return { + "revenue_first": snap["commercial"], + "qualified_exposure": snap["qualified_exposure"], + "experiments": snap["experiments"], + "supporting_never_sufficient": { + "indexed_entries": idx["total_entries"], + "observed_by_guild": idx["observed_by_guild"], + "never_called_by_guild": idx["never_called_by_guild"], + "active_watches": sum(1 for w in store.watches.values() + if w.get("active")), + "note": ("inventory, reach and free usage cannot promote an " + "experiment or be reported as adoption. They are here to " + "explain the primary numbers, not to replace them."), + }, + "number_to_watch": { + "metric": "external_settled_revenue_usd", + "value": snap["commercial"]["external_settled_revenue_usd"], + "definition": snap["commercial"]["revenue_definition"], + }, + } + + +@app.get("/pricing") +def pricing_table(): + """What the paid layer costs and WHY — machine-readable. + + Every price is env-overridable within a hard ceiling and carries its stated + basis, because a price published without its rationale cannot be argued + with, and a price nobody can argue with is one nobody has measured.""" + return { + **pricing.table(), + "free_forever": { + "GET /preflight": "live checks + verdict for one endpoint", + "GET /index": "the public index and what we observed", + "GET /index/search": "search the index", + "POST /evidence/verify": "verify a bundle we issued", + "why": ("charging for 'does this endpoint work' would make the " + "ecosystem worse and this index poorer"), + }, + "buy": { + "credits": "POST /billing/trial (self-serve, no human)", + "x402": "pay per call on the live rail — see GET /x402/readiness", + }, + } + + @app.get("/diagnostics/state") def diagnostics_state(): """WHICH process and WHICH state produced this response — the decidability diff --git a/live/guild/app/mcp_server.py b/live/guild/app/mcp_server.py index bc7e2d4..7d2fb7a 100644 --- a/live/guild/app/mcp_server.py +++ b/live/guild/app/mcp_server.py @@ -36,7 +36,11 @@ from . import proving from . import x402 from .payments import CachedPaidResult, PaidRequest, PaymentChallenge, PaymentIdConflict +from . import deepcheck +from . import indexops from . import preflight +from . import pricing +from . import trustindex from .state import store from . import credentials as _creds @@ -453,6 +457,98 @@ def guild_preflight(url: str, ctx: Context = None) -> dict: return out +@mcp.tool +def guild_index(query: str = "", limit: int = 20, ctx: Context = None) -> dict: + """Search the public trust index of agent endpoints — FREE, no key. + + Returns, per endpoint, what its registry CLAIMS and separately what Agent + Guild OBSERVED when it actually called it. Those are different things and + are never merged: measured 2026-07-31, 92.9% of registry-listed agents + report healthy and 33.9% complete a task. + + Example: guild_index(query="translation") + """ + hits = [] + needle = (query or "").strip().lower() + for e in (store.trust_index or {}).values(): + declared = e.get("declared") or {} + hay = " ".join([str(e.get("endpoint") or ""), + str(declared.get("name") or ""), + " ".join(str(c) for c in declared.get("capabilities", [])) + ]).lower() + if needle and needle not in hay: + continue + hits.append(e) + hits.sort(key=lambda r: (bool(r.get("observation")), + r.get("observed_at") or ""), reverse=True) + store.record_event("mcp", "index_view", ua=_client_ua(ctx), + endpoint="index", transport="mcp") + return {"count": len(hits), + "results": [trustindex.public_view(e) for e in hits[:max(1, min(limit, 100))]], + "summary": trustindex.summarise((store.trust_index or {}).values())} + + +@mcp.tool +def guild_preflight_deep(url: str, api_key: str = "", ctx: Context = None) -> dict: + """PAID deep check before you delegate or pay: live checks PLUS drift + history, cross-source corroboration and an explicit allow / caution / block + policy verdict you can act on directly. + + The free `guild_preflight` is not degraded to sell this — it still returns + the full live check set. This adds what one request cannot establish: + whether the endpoint has CHANGED, and whether anyone else corroborates it. + + Priced through the same gateway as every other paid read (see GET /pricing). + + Example: guild_preflight_deep(url="https://some-agent.example/a2a") + """ + return _serve_paid( + payments.deep_preflight_request(url), + lambda: deepcheck.deep_preflight(store, url), + ctx, api_key) + + +@mcp.tool +def guild_watch(url: str, api_key: str, interval_seconds: int = 3600, + ctx: Context = None) -> dict: + """Self-provision CONTINUOUS monitoring of an endpoint. No onboarding, no + human, no sales call. + + Provisioning is free and idempotent by (caller, endpoint) — calling twice + returns the same watch rather than billing twice. Each recheck cycle is + charged only when it actually runs, so a dormant endpoint costs nothing. + Read the change feed with guild_watch_feed. + + Example: guild_watch(url="https://some-agent.example/a2a", api_key="…") + """ + if not api_key: + return {"error": "api_key required — a watch bills per cycle. " + "POST /billing/trial issues credits with no human."} + try: + rec = indexops.provision_watch( + store, _creds.sanitize_actor_key(api_key), url, + interval_s=interval_seconds) + except ValueError as e: + return {"error": str(e)} + store.record_event(_creds.sanitize_actor_key(api_key), "watch_provisioned", + ua=_client_ua(ctx), endpoint="watch", target=url, + transport="mcp") + return {**rec, "price_per_cycle_credits": pricing.price("watch_cycle")} + + +@mcp.tool +def guild_watch_feed(watch_id: str, api_key: str = "", ctx: Context = None) -> dict: + """Read the machine-readable change feed for a watch you provisioned. + Free — you already paid for the cycles that produced it.""" + feed = indexops.watch_feed(store, watch_id) + if not feed: + return {"error": "watch not found"} + rec = (store.watches or {}).get(watch_id) or {} + if rec.get("owner_key") and _creds.sanitize_actor_key(api_key or "") != rec["owner_key"]: + return {"error": "this watch belongs to another caller"} + return feed + + @mcp.tool def guild_check(capability: str, api_key: str = "", ctx: Context = None) -> dict: """START HERE. One call to vet a `capability` before you delegate: returns the diff --git a/live/guild/app/payments.py b/live/guild/app/payments.py index b3ba70d..1d26c5d 100644 --- a/live/guild/app/payments.py +++ b/live/guild/app/payments.py @@ -109,6 +109,16 @@ def request_hash(self) -> str: @property def cost(self) -> int: + """Price in credits for this operation. + + Index-product operations resolve through app/pricing.py, where every + price is env-overridable within a hard ceiling and carries its stated + basis — a price the experiment engine may move is configuration, not a + constant compiled into the payment gateway. Legacy operations keep + their PRICING entry so nothing existing changes behaviour.""" + from . import pricing as _pricing + if self.operation in _pricing.DEFAULTS: + return _pricing.price(self.operation) return PRICING[self.operation] @@ -122,6 +132,25 @@ def check_request(capability: str, signed: bool = False, "capability": capability, "signed": signed, "ttl_seconds": ttl_seconds}) +def deep_preflight_request(url: str) -> PaidRequest: + """Paid deep preflight. One semantic operation, one canonical resource URL + across HTTP, MCP and A2A — the same discipline as every other paid read.""" + return PaidRequest.build("deep_preflight", "GET", "/preflight/deep", + {"url": url}) + + +def evidence_bundle_request(url: str, ttl_seconds: int = 3600) -> PaidRequest: + """Paid signed evidence bundle.""" + return PaidRequest.build("evidence_bundle", "POST", "/evidence/bundle", + {"url": url, "ttl_seconds": ttl_seconds}) + + +def watch_cycle_request(endpoint: str) -> PaidRequest: + """One continuous-watch recheck cycle, charged only when performed.""" + return PaidRequest.build("watch_cycle", "POST", "/watch/cycle", + {"endpoint": endpoint}) + + def search_request(capability: str, limit: int = 20, min_trust: float = 0.0) -> PaidRequest: return PaidRequest.build("best_agent", "GET", "/search", { diff --git a/live/guild/app/pricing.py b/live/guild/app/pricing.py new file mode 100644 index 0000000..0cb6ace --- /dev/null +++ b/live/guild/app/pricing.py @@ -0,0 +1,128 @@ +"""Prices as CONFIGURATION, not doctrine. + +Every price here is a reversible guess the experiment engine is allowed to +move. Hard-coding a number in a dict and treating it as settled is how a +business ends up defending a price it never measured. + +Two rules this module exists to enforce: + +1. **Every price is env-overridable** (``GUILD_PRICE_``), so a price + change is a config change and a rollback — not a deploy and a revert. +2. **Every price carries its rationale and its bounds.** A price with no stated + basis cannot be argued with later, and a price with no ceiling can be moved + by a buggy experiment to a number no machine will ever pay. + +MEASURED BASIS (2026-07-31 — and the reason these numbers are low) + Total x402 settled volume across all networks in July 2026 was $232,329, + down 98.9% from the November 2025 peak on flat transaction count (~$0.04 + per transaction). The median x402 Bazaar listing took 2 calls and 1 unique + payer in 30 days; the median earning agent made $1.65. + + A price set for an enterprise buyer is unpayable by that population. A price + set at $0.001 cannot cover an outbound probe. So the opening prices sit just + above marginal cost and far below the cost of the mistake they prevent — an + irreversible x402 transfer to an endpoint that does not work. They are + expected to be wrong, and expected to move. +""" +from __future__ import annotations + +import os +from typing import Any + +#: Opening prices in CREDITS (1 credit = $0.001). Basis, not doctrine. +DEFAULTS: dict[str, int] = { + # Several bounded outbound probes plus a policy verdict. $0.02 — half a + # median x402 transaction, a fiftieth of the smallest transfer it guards. + "deep_preflight": 20, + # A signed, offline-verifiable artefact the caller keeps and can re-verify + # without us. Below the signed-decision family: it attests to an + # observation, not a recommendation. + "evidence_bundle": 100, + # Charged per recheck ACTUALLY performed. + "watch_cycle": 5, + # Provisioning is free — see RATIONALE. + "watch_provision": 0, +} + +#: Hard ceiling per operation. An experiment may move a price WITHIN these +#: bounds and nowhere else, so a runaway loop can neither price us out of the +#: market nor give the product away. +CEILINGS: dict[str, int] = { + "deep_preflight": 500, # $0.50 + "evidence_bundle": 2000, # $2.00 + "watch_cycle": 100, # $0.10 + "watch_provision": 100, +} + +RATIONALE: dict[str, str] = { + "deep_preflight": ( + "just above the marginal cost of several bounded outbound probes, and " + "far below the cost of the mistake it prevents: an irreversible x402 " + "transfer to an endpoint that does not work"), + "evidence_bundle": ( + "a durable, offline-verifiable artefact the caller keeps and can " + "re-verify without us; priced below the signed-decision family because " + "it attests to an observation, not a recommendation"), + "watch_cycle": ( + "charged per recheck ACTUALLY performed, so a quiet endpoint costs the " + "customer nothing and we never bill for work we did not do"), + "watch_provision": ( + "free — taking money before any observation exists would be charging " + "for a promise"), +} + + +def _env_key(operation: str) -> str: + return "GUILD_PRICE_" + operation.upper() + + +def price(operation: str) -> int: + """The live price for `operation`, in credits. + + Resolution: environment override → default, clamped to + [0, CEILINGS[operation]]. A malformed override degrades to the default + rather than taking the endpoint offline or making it free by accident.""" + default = DEFAULTS.get(operation, 0) + raw = os.environ.get(_env_key(operation)) + if raw is None: + return default + try: + value = int(str(raw).strip()) + except (TypeError, ValueError): + return default + return max(0, min(value, CEILINGS.get(operation, default))) + + +def table() -> dict[str, Any]: + """The public, machine-readable price list — with the basis attached. + + A price published without its rationale cannot be argued with, which is + exactly the property a price should not have.""" + return { + "unit": "credits", + "credit_usd": 0.001, + "prices": { + op: { + "credits": price(op), + "usd": round(price(op) * 0.001, 4), + "default_credits": DEFAULTS[op], + "ceiling_credits": CEILINGS[op], + "env_override": _env_key(op), + "basis": RATIONALE[op], + "overridden": os.environ.get(_env_key(op)) is not None, + } + for op in DEFAULTS + }, + "policy": ( + "Prices are configuration, not doctrine: each is an " + "env-overridable, reversible guess with a stated basis and a hard " + "ceiling. The autonomous experiment engine may move a price within " + "its ceiling; it may not invent an operation, and it may not treat " + "a price it has never measured as settled."), + "measured_basis_2026_07": ( + "Total x402 settled volume across all networks in July 2026: " + "$232,329, down 98.9% from the November 2025 peak on flat " + "transaction count (~$0.04/tx). Median Bazaar listing: 2 calls, 1 " + "unique payer per 30 days. Median earning agent: $1.65 per 30 " + "days. These prices are set for THAT population."), + } diff --git a/live/guild/app/store.py b/live/guild/app/store.py index 5c8bd37..7443744 100644 --- a/live/guild/app/store.py +++ b/live/guild/app/store.py @@ -153,6 +153,15 @@ def __init__(self, path: Optional[str] = None): self.guild_revenue: int = 0 # settlement fees earned (credits) self.demand_watches: list[dict[str, Any]] = [] # attributable demand callbacks (Phase 0, G5) self.swarm_state: dict[str, Any] = {} # discovery swarm: counters, actions, referral tokens, kill flag + # --- autonomous trust index (product-led pivot 2026-07-31) ---------- + # endpoint fingerprint -> index entry. The public product surface: what + # is out there, and which of it we have actually called. Persisted as + # kv like swarm_state, so it survives restarts without a schema change. + self.trust_index: dict[str, dict[str, Any]] = {} + # watch id -> continuous-monitoring subscription (self-provisioned) + self.watches: dict[str, dict[str, Any]] = {} + # experiment key -> bounded reversible experiment state + self.experiments: dict[str, dict[str, Any]] = {} # machine-market state (app/market.py): signed offers, bonded machine # adjudicators, dispute cases — all persisted like swarm_state (kv) self.offers: dict[str, dict[str, Any]] = {} @@ -470,6 +479,9 @@ def _sqlite_flush_all(self): b.put_checkpoint(r) b.put_kv("identity", self.identity) b.put_kv("swarm_state", self.swarm_state) + b.put_kv("trust_index", self.trust_index) + b.put_kv("watches", self.watches) + b.put_kv("experiments", self.experiments) b.put_kv("offers", self.offers) b.put_kv("adjudicators", self.adjudicators) b.put_kv("dispute_cases", self.dispute_cases) @@ -521,6 +533,9 @@ def _sqlite_initial_load(self): b.append_demand_watch(r) b.put_kv("identity", self.identity) b.put_kv("swarm_state", self.swarm_state) + b.put_kv("trust_index", self.trust_index) + b.put_kv("watches", self.watches) + b.put_kv("experiments", self.experiments) b.put_kv("offers", self.offers) b.put_kv("adjudicators", self.adjudicators) b.put_kv("dispute_cases", self.dispute_cases) @@ -571,6 +586,9 @@ def _load_sqlite(self): self.guild_revenue = d["guild_revenue"] self.demand_watches = d["demand_watches"] self.swarm_state = d["swarm_state"] + self.trust_index = self.backend.fetch_kv("trust_index", {}) or {} + self.watches = self.backend.fetch_kv("watches", {}) or {} + self.experiments = self.backend.fetch_kv("experiments", {}) or {} self.offers = self.backend.fetch_kv("offers", {}) or {} self.adjudicators = self.backend.fetch_kv("adjudicators", {}) or {} self.dispute_cases = self.backend.fetch_kv("dispute_cases", {}) or {} @@ -625,6 +643,9 @@ def _load_from_json_file(self) -> None: self.guild_revenue = data.get("guild_revenue", 0) self.demand_watches = data.get("demand_watches", []) self.swarm_state = data.get("swarm_state", {}) + self.trust_index = data.get("trust_index", {}) + self.watches = data.get("watches", {}) + self.experiments = data.get("experiments", {}) self.offers = data.get("offers", {}) self.adjudicators = data.get("adjudicators", {}) self.dispute_cases = data.get("dispute_cases", {}) @@ -767,7 +788,10 @@ def _save(self) -> None: "externality_attestations": self.externality_attestations, "guild_inbox": self.guild_inbox, - "swarm_state": self.swarm_state}, f, indent=2) + "swarm_state": self.swarm_state, + "trust_index": self.trust_index, + "watches": self.watches, + "experiments": self.experiments}, f, indent=2) os.replace(tmp, self.path) # events are now durable in the main file — compact the journal if self.events_path: diff --git a/live/guild/app/swarm/runner.py b/live/guild/app/swarm/runner.py index 87498fc..4ae92b4 100644 --- a/live/guild/app/swarm/runner.py +++ b/live/guild/app/swarm/runner.py @@ -181,6 +181,82 @@ def _run_scout(store: Any, *, fetch: Callable, deadline: float, return scout.run_scout(store, fetch=fetch, deadline=deadline) +def index_autorun() -> bool: + """Index upkeep is DEFAULT-OFF, like every other outbound loop here. + + Outbound traffic to third-party infrastructure must never begin merely + because a container restarted, and a test that exercises the scout must not + silently start probing real hosts. Enabled explicitly in render.yaml.""" + return (os.environ.get("GUILD_INDEX_AUTORUN") or "0").strip() == "1" + + +def _run_index_cycle(store: Any) -> dict[str, Any]: + """Ingest, recheck the stalest entries, run due watches, evaluate + experiments. Every step is independently bounded. + + Ordering matters: ingest before recheck so a newly-listed endpoint gets an + observation in the same cycle it arrives, and watches after recheck so a + paying customer is never served a staler view than the free index.""" + from .. import indexops + from .. import experiments as _experiments + + if not index_autorun(): + return {"skipped": "GUILD_INDEX_AUTORUN is not enabled"} + out: dict[str, Any] = {} + try: + out["ingest"] = indexops.ingest(store) + except Exception as exc: # noqa: BLE001 + out["ingest_error"] = type(exc).__name__ + try: + out["recheck"] = indexops.recheck_due(store) + except Exception as exc: # noqa: BLE001 + out["recheck_error"] = type(exc).__name__ + try: + out["watch_cycles"] = _run_watch_cycles(store) + except Exception as exc: # noqa: BLE001 + out["watch_error"] = type(exc).__name__ + try: + out["experiments"] = { + k: _experiments.evaluate(store, k)["decision"] + for k in list(getattr(store, "experiments", {}) or {})} + except Exception as exc: # noqa: BLE001 + out["experiment_error"] = type(exc).__name__ + return out + + +def _run_watch_cycles(store: Any, cap: int = 10) -> dict[str, Any]: + """Run due watches, charging each customer per cycle ACTUALLY performed. + + The charge goes through the same billing primitive as every other paid + operation. A customer who cannot pay has their watch SUSPENDED, not + silently serviced for free — otherwise our outbound budget quietly + subsidises a lapsed account.""" + from .. import indexops + from .. import pricing as _pricing + + due = indexops.watch_due(store)[:cap] + if not due: + return {"due": 0, "cycled": 0} + + def _charge(owner_key: str) -> int: + price = _pricing.price("watch_cycle") + if price <= 0: + return 0 + store.charge(owner_key, price, "watch_cycle") + return price + + results = [indexops.run_watch_cycle(store, rec, charge=_charge) + for rec in due] + return { + "due": len(due), + "cycled": sum(1 for r in results if r.get("cycled")), + "changed": sum(1 for r in results if r.get("changed")), + "suspended": sum(1 for r in results if r.get("suspended")), + "credits_charged": sum(int(r.get("charged_credits") or 0) + for r in results), + } + + def notify_demand(store: Any, capability: str) -> bool: """Record newly counted verified/genuine unmet demand for `capability` into the DURABLE pending-demand queue and wake the loop so it can @@ -298,6 +374,19 @@ def run_once(store: Any, fetch: Callable = scout.safe_fetch_json, try: deadline = time.time() + run_timeout_s() summary = _run_scout(store, fetch=fetch, deadline=deadline) + # --- autonomous index maintenance (product-led pivot 2026-07-31) ---- + # The index is the product surface, so it must refresh itself on the + # SAME lease-guarded, bounded, jittered schedule as the scout — one + # loop, one lease, one deadline. It never runs on its own timer, so + # there is exactly one place to look when outbound traffic misbehaves, + # and exactly one kill switch. Failures here are recorded and never + # allowed to fail the cycle: index upkeep must not take the service + # down, and a silent skip would be worse than a logged one. + index_summary = {} + try: + index_summary = _run_index_cycle(store) + except Exception as exc: # noqa: BLE001 + index_summary = {"error": type(exc).__name__} zero_demand = not summary.get("capabilities") # ACK only the capabilities this cycle actually processed — demand # that arrived mid-run stays queued for the next cycle. @@ -326,6 +415,7 @@ def run_once(store: Any, fetch: Callable = scout.safe_fetch_json, "adapters": summary.get("adapters", {}), "adapters_failed": summary.get("adapters_failed", []), "deadline_hit": bool(summary.get("deadline_hit")), + "index": index_summary, } _persist(store) store.record_event(None, "scout_cycle_completed", diff --git a/live/guild/app/trustindex.py b/live/guild/app/trustindex.py new file mode 100644 index 0000000..8af6289 --- /dev/null +++ b/live/guild/app/trustindex.py @@ -0,0 +1,317 @@ +"""The public trust index — what is out there, and which of it actually works. + +THE PRODUCT QUESTION + "Can I safely use or pay this specific endpoint right now?" + +Everything here serves that question. The index is not a directory competing on +inventory size; a bigger list of unverified entries is a worse product, not a +better one. The index earns its place by carrying, per endpoint, the one thing +every registry omits: **what happened when we actually called it.** + +THE DISTINCTION THAT MAKES IT HONEST + Registries conflate three completely different states and report them all as + a listing. This module keeps them apart and never lets one be read as + another: + + ``indexed`` a source listed it. We have never called it. This is a + CLAIM, and is worth exactly what a claim is worth. + ``live`` it completed a real protocol handshake when we called it. + ``unreachable`` we called it and it did not answer. + ``degraded`` it answers, but one of its own declared claims does not + hold (unsigned card, advertises payment but never + challenges, and so on). + + Measured on 2026-07-31: 92.9% of a2aregistry entries report ``is_healthy: + true`` and 33.9% complete a task. That 59-point gap is the entire reason this + module exists, and it is why ``indexed`` is never promoted to ``live`` + without an observation of our own. + +DEDUPLICATION + By endpoint fingerprint (normalised scheme+host+port+path) FIRST, then by + declared identity (DID) where one exists. One operator publishing the same + service to three registries is one entry with three provenance records — not + three entries, and never three counts in a headline. + +PROVENANCE AND FRESHNESS + Every entry records where it came from, when each source last confirmed it, + and when WE last observed it. An observation has an age, and an aged + observation is reported with its age rather than being quietly served as + current. `stale` is a first-class state, not an absence. + +WHAT IS DELIBERATELY NOT HERE + No scores invented from unknowns, no aggregate "trust rating" that averages + an observation with a guess, and no generated pages for entries we have + nothing real to say about — an evidence page for an endpoint we have never + successfully called is SEO spam, and is refused by `is_page_worthy`. +""" +from __future__ import annotations + +import hashlib +import os +from datetime import datetime, timedelta, timezone +from typing import Any, Iterable, Optional +from urllib.parse import urlsplit, urlunsplit + +from . import reachability + +#: An observation older than this is reported as stale rather than current. +DEFAULT_FRESH_TTL_S = 24 * 3600 + +#: Bound on how much of the index one recheck cycle may probe, so an autonomous +#: loop cannot turn into an unbounded crawler of other people's infrastructure. +DEFAULT_RECHECK_BATCH = 8 + +STATUS_INDEXED = "indexed" +STATUS_LIVE = "live" +STATUS_DEGRADED = "degraded" +STATUS_UNREACHABLE = "unreachable" + +#: Ownership classes. `externally_owned` is the ONLY one that may appear in a +#: growth or revenue metric. +OWNER_EXTERNAL = "externally_owned" +OWNER_FIRST_PARTY = "first_party" +OWNER_UNKNOWN = "unknown" + + +def _now() -> datetime: + return datetime.now(timezone.utc) + + +def _iso(dt: Optional[datetime] = None) -> str: + return (dt or _now()).isoformat() + + +def fresh_ttl_s() -> int: + try: + return max(300, min(int(os.environ.get("GUILD_INDEX_FRESH_TTL_S") + or DEFAULT_FRESH_TTL_S), 30 * 24 * 3600)) + except (TypeError, ValueError): + return DEFAULT_FRESH_TTL_S + + +def recheck_batch() -> int: + try: + return max(1, min(int(os.environ.get("GUILD_INDEX_RECHECK_BATCH") + or DEFAULT_RECHECK_BATCH), 50)) + except (TypeError, ValueError): + return DEFAULT_RECHECK_BATCH + + +def normalise_url(url: str) -> str: + """Canonical form for deduplication. + + Lowercase scheme+host, drop the default port, drop a trailing slash, drop + query and fragment. Two registries listing ``https://X.com/a2a`` and + ``https://X.com:443/a2a/`` are describing ONE endpoint, and counting them + twice would inflate the only number this index is judged on.""" + if not url: + return "" + parts = urlsplit(url.strip()) + scheme = (parts.scheme or "https").lower() + host = (parts.hostname or "").lower() + if not host: + return "" + port = parts.port + netloc = host if port in (None, 80, 443) else f"{host}:{port}" + path = (parts.path or "").rstrip("/") + return urlunsplit((scheme, netloc, path, "", "")) + + +def fingerprint(url: str) -> str: + """Stable public id for an endpoint. Derived from the normalised URL only — + it carries no secret and can be published, linked and cited.""" + norm = normalise_url(url) + if not norm: + return "" + return "ep_" + hashlib.sha256(norm.encode("utf-8")).hexdigest()[:16] + + +def new_entry(url: str, source: str, *, declared: Optional[dict] = None, + did: str = "") -> dict[str, Any]: + norm = normalise_url(url) + return { + "id": fingerprint(norm), + "endpoint": norm, + "did": did or "", + "declared": declared or {}, # what the SOURCE claims + "sources": [{"source": source, "first_seen": _iso(), + "last_seen": _iso()}], + "first_indexed_at": _iso(), + "status": STATUS_INDEXED, + "owner_class": OWNER_UNKNOWN, + "observation": None, # what WE saw, or None + "observed_at": None, + "observation_count": 0, + "drift": [], # declared-vs-observed changes + } + + +def merge_source(entry: dict[str, Any], source: str) -> dict[str, Any]: + """Record that `source` also lists this endpoint (provenance, not a count).""" + for rec in entry.get("sources", []): + if rec.get("source") == source: + rec["last_seen"] = _iso() + return entry + entry.setdefault("sources", []).append( + {"source": source, "first_seen": _iso(), "last_seen": _iso()}) + return entry + + +def is_stale(entry: dict[str, Any], ttl_s: Optional[int] = None) -> bool: + """Has our own observation aged out? An entry never observed is stale.""" + at = entry.get("observed_at") + if not at: + return True + try: + seen = datetime.fromisoformat(at) + except (TypeError, ValueError): + return True + return _now() - seen > timedelta(seconds=ttl_s or fresh_ttl_s()) + + +def observation_age_s(entry: dict[str, Any]) -> Optional[float]: + at = entry.get("observed_at") + if not at: + return None + try: + return round((_now() - datetime.fromisoformat(at)).total_seconds(), 1) + except (TypeError, ValueError): + return None + + +def status_from_preflight(result: dict[str, Any]) -> str: + """Map a preflight verdict onto an index status. + + `indexed` is NEVER produced here: reaching this function means we called + the endpoint, and the whole point of the distinction is that a listing and + an observation are different things.""" + verdict = result.get("verdict") + checks = {c["check"]: c["status"] for c in result.get("checks", [])} + if checks.get("endpoint_reachable") == "failed": + return STATUS_UNREACHABLE + if checks.get("protocol_handshake") != "proven": + # answered, but proved no agent protocol — the 92.9%/33.9% gap + return STATUS_DEGRADED + if verdict == "delegate_with_caution": + return STATUS_DEGRADED + return STATUS_LIVE + + +def apply_observation(entry: dict[str, Any], result: dict[str, Any], + *, owner_class: str = OWNER_UNKNOWN) -> dict[str, Any]: + """Fold a preflight result into an entry, recording DRIFT. + + Drift is the property a one-off review cannot have: a server that passed + once and changed afterwards. Every status transition is retained with its + timestamp, bounded so the record cannot grow without limit.""" + previous = entry.get("status") + status = status_from_preflight(result) + entry["status"] = status + entry["observation"] = { + "verdict": result.get("verdict"), + "checks": result.get("checks", []), + "failed": result.get("failed", []), + "unknowns": result.get("unknowns", []), + } + entry["observed_at"] = _iso() + entry["observation_count"] = int(entry.get("observation_count", 0)) + 1 + if owner_class != OWNER_UNKNOWN or not entry.get("owner_class"): + entry["owner_class"] = owner_class + if previous and previous != status: + entry.setdefault("drift", []).append( + {"at": _iso(), "from": previous, "to": status}) + entry["drift"] = entry["drift"][-20:] + return entry + + +def is_page_worthy(entry: dict[str, Any]) -> bool: + """May we publish an indexable evidence page for this entry? + + ONLY when we hold a real observation of our own. A page for an endpoint we + have merely seen listed contains nothing a reader cannot get from the + registry, and publishing it at scale is SEO spam. The mandate says the + index must be evidence-rich and unique or it must not exist, and this is + the function that enforces it.""" + if not entry.get("observation") or not entry.get("observed_at"): + return False + if entry.get("status") == STATUS_INDEXED: + return False + obs = entry.get("observation") or {} + # something concrete must have been established either way + return bool(obs.get("checks")) + + +def public_view(entry: dict[str, Any], *, detail: bool = False) -> dict[str, Any]: + """The free public projection. Never invents a score. + + `claimed` and `observed` are kept in separate blocks on purpose: the entire + failure mode this index exists to correct is a reader taking a claim for an + observation because a UI merged them.""" + age = observation_age_s(entry) + out: dict[str, Any] = { + "id": entry.get("id"), + "endpoint": entry.get("endpoint"), + "status": entry.get("status"), + "owner_class": entry.get("owner_class", OWNER_UNKNOWN), + "claimed": { + "name": (entry.get("declared") or {}).get("name"), + "capabilities": (entry.get("declared") or {}).get("capabilities", []), + "sources": [s.get("source") for s in entry.get("sources", [])], + "note": "self-declared by the endpoint or its registry — NOT verified", + }, + "observed": None, + "observation_age_seconds": age, + "stale": is_stale(entry), + "observation_count": entry.get("observation_count", 0), + "first_indexed_at": entry.get("first_indexed_at"), + } + if entry.get("observation"): + obs = entry["observation"] + out["observed"] = { + "verdict": obs.get("verdict"), + "failed": obs.get("failed", []), + "unknowns": obs.get("unknowns", []), + "at": entry.get("observed_at"), + "note": ("what happened when the Guild actually called this " + "endpoint. `unknowns` are checks we could not perform and " + "are excluded from the verdict, never averaged into it."), + } + if detail: + out["observed"]["checks"] = obs.get("checks", []) + out["drift"] = entry.get("drift", []) + out["sources"] = entry.get("sources", []) + else: + out["observed_note"] = ( + "NEVER CALLED BY THE GUILD. This entry is a listing, not an " + "observation — treat it as a claim. Run GET /preflight?url=… to " + "produce one now (free).") + return out + + +def summarise(entries: Iterable[dict[str, Any]]) -> dict[str, Any]: + """Index-level counts, split so no number can be quoted out of context.""" + rows = list(entries) + by_status: dict[str, int] = {} + by_owner: dict[str, int] = {} + observed = 0 + for e in rows: + by_status[e.get("status", STATUS_INDEXED)] = \ + by_status.get(e.get("status", STATUS_INDEXED), 0) + 1 + by_owner[e.get("owner_class", OWNER_UNKNOWN)] = \ + by_owner.get(e.get("owner_class", OWNER_UNKNOWN), 0) + 1 + if e.get("observation"): + observed += 1 + total = len(rows) + listed_only = by_status.get(STATUS_INDEXED, 0) + return { + "total_entries": total, + "observed_by_guild": observed, + "never_called_by_guild": listed_only, + "by_status": by_status, + "by_owner_class": by_owner, + "claim_vs_observation": ( + f"{observed} of {total} entries carry an observation of our own; " + f"{listed_only} are listings we have never called. A listing is a " + "claim. Inventory size is a supporting metric and is never " + "reported as adoption."), + } diff --git a/live/guild/contract/contract.json b/live/guild/contract/contract.json index 4b6c584..398dcf6 100644 --- a/live/guild/contract/contract.json +++ b/live/guild/contract/contract.json @@ -46,15 +46,19 @@ "guild_check", "guild_escrow_open", "guild_escrow_release", + "guild_index", "guild_passport", "guild_preflight", + "guild_preflight_deep", "guild_prove", "guild_prove_verify", "guild_record", "guild_register", "guild_risk_score", "guild_search", - "guild_verify" + "guild_verify", + "guild_watch", + "guild_watch_feed" ], "payments": { "mechanism": "x402", @@ -375,6 +379,12 @@ ], "path": "/collaborations" }, + { + "methods": [ + "GET" + ], + "path": "/commercial" + }, { "methods": [ "POST" @@ -453,6 +463,18 @@ ], "path": "/evaluation" }, + { + "methods": [ + "POST" + ], + "path": "/evidence/bundle" + }, + { + "methods": [ + "POST" + ], + "path": "/evidence/verify" + }, { "methods": [ "POST" @@ -489,6 +511,30 @@ ], "path": "/health" }, + { + "methods": [ + "GET" + ], + "path": "/index" + }, + { + "methods": [ + "GET" + ], + "path": "/index/search" + }, + { + "methods": [ + "GET" + ], + "path": "/index/{endpoint_id}" + }, + { + "methods": [ + "GET" + ], + "path": "/index/{endpoint_id}/evidence" + }, { "methods": [ "GET" @@ -605,6 +651,18 @@ ], "path": "/preflight" }, + { + "methods": [ + "GET" + ], + "path": "/preflight/deep" + }, + { + "methods": [ + "GET" + ], + "path": "/pricing" + }, { "methods": [ "POST" @@ -713,6 +771,18 @@ ], "path": "/wallet-binding/verify" }, + { + "methods": [ + "POST" + ], + "path": "/watch" + }, + { + "methods": [ + "GET" + ], + "path": "/watch/{watch_id}" + }, { "methods": [ "GET" diff --git a/live/guild/tests/test_trust_index.py b/live/guild/tests/test_trust_index.py new file mode 100644 index 0000000..e9fda33 --- /dev/null +++ b/live/guild/tests/test_trust_index.py @@ -0,0 +1,433 @@ +"""Trust index, paid layer and experiment engine — the invariants. + +The mandate names five things that must be PROVEN, not asserted: attribution, +paid issuance, idempotency, fail-closed behaviour, and crawler exclusion. Each +has a section below. + +Every test here is written as a constraint on what the product may CLAIM. That +is deliberate: this codebase has already shipped three metrics that read better +than reality, and all three passed their functional tests. +""" +from __future__ import annotations + +import json +import os +import sys + +import pytest + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) + +from app import deepcheck, experiments, indexops, pricing, trustindex # noqa: E402 +from app.store import Store # noqa: E402 + + +@pytest.fixture() +def store(tmp_path) -> Store: + return Store(path=str(tmp_path / "guild.json")) + + +def _observed(verdict="no_failed_checks", failed=(), unknowns=(), + handshake="proven", reachable="proven"): + return { + "verdict": verdict, + "failed": list(failed), + "unknowns": list(unknowns), + "checks": [ + {"check": "endpoint_reachable", "status": reachable, "detail": ""}, + {"check": "protocol_handshake", "status": handshake, "detail": ""}, + ], + } + + +# -------------------------------------------------------------------------- +# 1. Deduplication and provenance — inventory can never be inflated +# -------------------------------------------------------------------------- +def test_same_endpoint_from_three_registries_is_one_entry(store): + """One operator publishing to three registries is ONE endpoint with three + provenance records. Counting it three times would inflate the only number + the index is judged on.""" + recs = [ + {"endpoint": "https://Example.com:443/a2a/", "source": "mcp_registry"}, + {"endpoint": "https://example.com/a2a", "source": "a2a_registry"}, + {"endpoint": "https://example.com/a2a?utm=x", "source": "guild_registration"}, + ] + out = indexops.ingest(store, recs) + assert out["added"] == 1, out + assert out["provenance_updated"] == 2 + assert len(store.trust_index) == 1 + entry = next(iter(store.trust_index.values())) + assert len(entry["sources"]) == 3 + + +def test_ingest_is_idempotent(store): + recs = [{"endpoint": "https://example.com/a2a", "source": "mcp_registry"}] + indexops.ingest(store, recs) + second = indexops.ingest(store, recs) + assert second["added"] == 0 + assert len(store.trust_index) == 1 + + +def test_inventory_is_never_described_as_adoption(store): + indexops.ingest(store, [{"endpoint": "https://a.example/a2a", + "source": "mcp_registry"}]) + summary = trustindex.summarise(store.trust_index.values()) + assert "claim" in summary["claim_vs_observation"].lower() + assert "never reported as adoption" in summary["claim_vs_observation"] + + +# -------------------------------------------------------------------------- +# 2. A listing is never promoted to an observation +# -------------------------------------------------------------------------- +def test_a_listing_is_reported_as_a_claim_not_a_status(store): + indexops.ingest(store, [{"endpoint": "https://a.example/a2a", + "source": "mcp_registry", + "declared": {"name": "Claims To Work"}}]) + entry = next(iter(store.trust_index.values())) + view = trustindex.public_view(entry) + assert view["status"] == trustindex.STATUS_INDEXED + assert view["observed"] is None + assert "NEVER CALLED" in view["observed_note"] + assert "NOT verified" in view["claimed"]["note"] + + +def test_http_200_without_handshake_is_degraded_not_live(): + """The 92.9%/33.9% gap, encoded. A server answering 200 is not a working + agent, and the index must never record it as one.""" + status = trustindex.status_from_preflight( + _observed(verdict="do_not_delegate", failed=["protocol_handshake"], + handshake="failed")) + assert status == trustindex.STATUS_DEGRADED + + +def test_unreachable_is_its_own_status(): + status = trustindex.status_from_preflight( + _observed(verdict="do_not_delegate", failed=["endpoint_reachable"], + reachable="failed", handshake="unknown")) + assert status == trustindex.STATUS_UNREACHABLE + + +def test_drift_is_recorded_on_every_status_change(store): + indexops.ingest(store, [{"endpoint": "https://a.example/a2a", + "source": "s1"}]) + fp = next(iter(store.trust_index)) + indexops.recheck_one(store, fp, runner=lambda url: _observed()) + indexops.recheck_one(store, fp, runner=lambda url: _observed( + verdict="do_not_delegate", failed=["protocol_handshake"], + handshake="failed")) + entry = store.trust_index[fp] + assert entry["drift"], "a state change must be recorded" + assert entry["drift"][-1]["to"] == trustindex.STATUS_DEGRADED + + +def test_stale_observations_are_labelled_stale(store, monkeypatch): + indexops.ingest(store, [{"endpoint": "https://a.example/a2a", "source": "s"}]) + fp = next(iter(store.trust_index)) + indexops.recheck_one(store, fp, runner=lambda url: _observed()) + assert trustindex.public_view(store.trust_index[fp])["stale"] is False + monkeypatch.setenv("GUILD_INDEX_FRESH_TTL_S", "300") + store.trust_index[fp]["observed_at"] = "2020-01-01T00:00:00+00:00" + assert trustindex.public_view(store.trust_index[fp])["stale"] is True + + +# -------------------------------------------------------------------------- +# 3. No SEO spam — pages only where there is real evidence +# -------------------------------------------------------------------------- +def test_no_evidence_page_for_an_endpoint_we_never_called(store): + indexops.ingest(store, [{"endpoint": "https://a.example/a2a", "source": "s"}]) + entry = next(iter(store.trust_index.values())) + assert trustindex.is_page_worthy(entry) is False + + +def test_evidence_page_allowed_once_we_have_observed(store): + indexops.ingest(store, [{"endpoint": "https://a.example/a2a", "source": "s"}]) + fp = next(iter(store.trust_index)) + indexops.recheck_one(store, fp, runner=lambda url: _observed()) + assert trustindex.is_page_worthy(store.trust_index[fp]) is True + + +# -------------------------------------------------------------------------- +# 4. Paid issuance FAILS CLOSED +# -------------------------------------------------------------------------- +def test_evidence_bundle_refuses_without_a_ledger_anchor(store, monkeypatch): + """A bundle that cannot be anchored must not be issued at all. Selling a + degraded evidence object is worse than selling nothing.""" + monkeypatch.setattr(store, "latest_checkpoint", lambda **kw: None) + with pytest.raises(deepcheck.EvidenceIssuanceRefused): + deepcheck.evidence_bundle(store, "https://a.example/a2a") + + +def test_evidence_bundle_refuses_when_anchoring_raises(store, monkeypatch): + def _boom(**kw): + raise RuntimeError("stale durable state") + monkeypatch.setattr(store, "latest_checkpoint", _boom) + with pytest.raises(deepcheck.EvidenceIssuanceRefused): + deepcheck.evidence_bundle(store, "https://a.example/a2a") + + +def test_evidence_bundle_refuses_without_a_signing_identity(store, monkeypatch): + monkeypatch.setattr(store, "guild_identity", lambda: {"did": "", "private_key": ""}) + with pytest.raises(deepcheck.EvidenceIssuanceRefused): + deepcheck.evidence_bundle(store, "https://a.example/a2a") + + +def test_issued_bundle_verifies_offline_and_round_trips(store): + bundle = deepcheck.evidence_bundle(store, "https://a.example/a2a") + assert bundle["proof"] + out = deepcheck.verify_bundle(store, bundle) + assert out["signature_valid"] is True + assert out["valid"] is True + # tampering must break it + tampered = json.loads(json.dumps(bundle)) + tampered["policy"]["decision"] = "allow" + tampered["subject_endpoint"] = "https://attacker.example/a2a" + assert deepcheck.verify_bundle(store, tampered)["signature_valid"] is False + + +def test_expired_bundle_is_invalid_but_still_signed(store): + bundle = deepcheck.evidence_bundle(store, "https://a.example/a2a", ttl_s=60) + bundle["valid_until"] = "2020-01-01T00:00:00+00:00" + out = deepcheck.verify_bundle(store, bundle) + assert out["expired"] is True + assert out["valid"] is False + + +# -------------------------------------------------------------------------- +# 5. Policy verdict never launders unknowns into a pass +# -------------------------------------------------------------------------- +def test_blocking_failure_blocks(): + v = deepcheck.policy_verdict( + {"failed": ["protocol_handshake"], "unknowns": []}, None) + assert v["decision"] == "block" + + +def test_claim_failure_is_caution_not_block(): + v = deepcheck.policy_verdict( + {"failed": ["agent_card_signed"], "unknowns": []}, None) + assert v["decision"] == "caution" + + +def test_mostly_unknown_is_caution_not_allow(): + v = deepcheck.policy_verdict( + {"failed": [], "unknowns": ["a", "b", "c", "d"]}, None) + assert v["decision"] == "caution" + assert "thin evidence" in v["reason"] + + +def test_recent_instability_downgrades_a_clean_result(): + entry = {"drift": [{"at": "x", "from": "live", "to": "degraded"}, + {"at": "y", "from": "degraded", "to": "live"}]} + v = deepcheck.policy_verdict({"failed": [], "unknowns": []}, entry) + assert v["decision"] == "caution" + assert "changed state" in v["reason"] + + +def test_policy_threshold_is_published_so_it_can_be_rejected(): + v = deepcheck.policy_verdict({"failed": [], "unknowns": []}, None) + assert v["threshold"] + assert "reject it" in v["caller_note"] + + +# -------------------------------------------------------------------------- +# 6. Watch: idempotent provisioning, charge only for work done +# -------------------------------------------------------------------------- +def test_provisioning_the_same_watch_twice_does_not_bill_twice(store): + a = indexops.provision_watch(store, "key-1", "https://a.example/a2a") + b = indexops.provision_watch(store, "key-1", "https://a.example/a2a/") + assert a["id"] == b["id"] + assert b["created"] is False + assert len(store.watches) == 1 + + +def test_different_callers_get_different_watches(store): + a = indexops.provision_watch(store, "key-1", "https://a.example/a2a") + b = indexops.provision_watch(store, "key-2", "https://a.example/a2a") + assert a["id"] != b["id"] + + +def test_a_cycle_that_cannot_observe_is_not_billed(store): + rec = indexops.provision_watch(store, "key-1", "https://a.example/a2a") + charged = [] + store.watches[rec["id"]]["endpoint_id"] = "ep_does_not_exist" + out = indexops.run_watch_cycle( + store, store.watches[rec["id"]], + charge=lambda k: charged.append(k) or 5) + assert out["cycled"] is False + assert charged == [], "a cycle that observed nothing must not be billed" + + +def test_failed_charge_suspends_the_watch_rather_than_serving_it_free(store): + rec = indexops.provision_watch(store, "key-1", "https://a.example/a2a") + + def _broke(_key): + raise RuntimeError("insufficient credits") + + out = indexops.run_watch_cycle(store, store.watches[rec["id"]], + charge=_broke, + runner=lambda url: _observed()) + assert out["suspended"] is True + assert store.watches[rec["id"]]["active"] is False + + +def test_watch_records_a_change_only_when_status_actually_changes(store): + rec = indexops.provision_watch(store, "key-1", "https://a.example/a2a") + live = store.watches[rec["id"]] + indexops.run_watch_cycle(store, live, runner=lambda url: _observed()) + first = len(store.watches[rec["id"]]["changes"]) + indexops.run_watch_cycle(store, store.watches[rec["id"]], + runner=lambda url: _observed()) + assert len(store.watches[rec["id"]]["changes"]) == first + indexops.run_watch_cycle( + store, store.watches[rec["id"]], + runner=lambda url: _observed(verdict="do_not_delegate", + failed=["protocol_handshake"], + handshake="failed")) + assert len(store.watches[rec["id"]]["changes"]) == first + 1 + + +# -------------------------------------------------------------------------- +# 7. Pricing is configuration, bounded +# -------------------------------------------------------------------------- +def test_price_is_env_overridable(monkeypatch): + monkeypatch.setenv("GUILD_PRICE_DEEP_PREFLIGHT", "45") + assert pricing.price("deep_preflight") == 45 + + +def test_price_override_is_clamped_to_its_ceiling(monkeypatch): + monkeypatch.setenv("GUILD_PRICE_DEEP_PREFLIGHT", "999999") + assert pricing.price("deep_preflight") == pricing.CEILINGS["deep_preflight"] + + +def test_malformed_price_falls_back_to_default(monkeypatch): + monkeypatch.setenv("GUILD_PRICE_DEEP_PREFLIGHT", "not-a-number") + assert pricing.price("deep_preflight") == pricing.DEFAULTS["deep_preflight"] + + +def test_every_price_publishes_its_basis(): + table = pricing.table() + for op, row in table["prices"].items(): + assert row["basis"], f"{op} has no stated basis" + assert row["ceiling_credits"] >= row["credits"] + + +# -------------------------------------------------------------------------- +# 8. ATTRIBUTION + CRAWLER EXCLUSION — an experiment cannot count our traffic +# -------------------------------------------------------------------------- +def test_crawler_traffic_is_not_qualified_exposure(store): + for _ in range(50): + store.record_event("a2a:net:bot", "preflight_run", + ua="a2a:AgenstryBot/0.3.0", endpoint="preflight") + assert experiments.qualified_exposure(store)["qualified_actors"] == 0 + + +def test_first_party_traffic_is_not_qualified_exposure(store): + store.record_event("ag-internal", "preflight_run", + ua="guild-release-gate", endpoint="preflight", + first_party=True) + assert experiments.qualified_exposure(store)["qualified_actors"] == 0 + + +def test_zero_qualified_exposure_never_produces_a_kill(store): + """The category error this engine exists to prevent: '0% conversion on + 1,790 crawler impressions' is not a finding.""" + experiments.define(store, "exp-1", hypothesis="h", variable="price:deep_preflight", + baseline={"paid_decisions": 0}) + for _ in range(500): + store.record_event("a2a:net:bot", "preflight_run", + ua="a2a:CrawlerBot/1.0", endpoint="preflight") + out = experiments.evaluate(store, "exp-1") + assert out["decision"] in ("hold", "insufficient_evidence") + assert out["decision"] != "kill" + + +def test_supporting_metrics_can_never_promote(store): + experiments.define(store, "exp-2", hypothesis="h", variable="price:deep_preflight", + baseline={m: 0 for m in experiments.PRIMARY_METRICS}) + # a mountain of free usage and inventory + for i in range(200): + e = trustindex.new_entry(f"https://x{i}.example/a2a", "mcp_registry") + store.trust_index[e["id"]] = e + out = experiments.evaluate(store, "exp-2") + assert out["decision"] != "promote" + + +def test_revenue_definition_excludes_sandbox_credits(store): + m = experiments.commercial_metrics(store) + assert m["external_settled_revenue_usd"] == 0.0 + assert "Sandbox credits" in m["revenue_definition"] + assert "not money" in m["revenue_definition"] + + +def test_kill_verdict_changes_the_offer_rather_than_celebrating_reach(store): + exp = experiments.define( + store, "exp-3", hypothesis="h", variable="price:deep_preflight", + baseline={m: 0 for m in experiments.PRIMARY_METRICS}) + exp["min_qualified"] = 1 + store.experiments["exp-3"] = exp + store.record_event("a2a:net:real", "deep_preflight_run", + ua="a2a:SomeRealAgent/1.0", endpoint="preflight_deep") + action = experiments.next_action(store, "exp-3") + if action["decision"] == "kill": + assert action["action"] == "reprice" + assert action["change"]["to_credits"] < action["change"]["from_credits"] + assert action["change"]["within_ceiling"] is True + + +def test_insufficient_evidence_fixes_distribution_not_price(store): + exp = experiments.define( + store, "exp-4", hypothesis="h", variable="price:deep_preflight", + baseline={m: 0 for m in experiments.PRIMARY_METRICS}) + exp["started_at"] = "2020-01-01T00:00:00+00:00" + store.experiments["exp-4"] = exp + action = experiments.next_action(store, "exp-4") + assert action["decision"] == "insufficient_evidence" + assert action["action"] == "increase_qualified_exposure" + assert "never actually tested" in action["rationale"] + + +# -------------------------------------------------------------------------- +# 9. Ownership is deterministic, never inferred +# -------------------------------------------------------------------------- +def test_first_party_flag_is_respected_and_never_counted_external(store): + indexops.ingest(store, [{"endpoint": "https://ours.example/a2a", + "source": "guild_registration", + "first_party": True}]) + entry = next(iter(store.trust_index.values())) + assert entry["owner_class"] == trustindex.OWNER_FIRST_PARTY + fp = entry["id"] + indexops.recheck_one(store, fp, runner=lambda url: _observed()) + assert store.trust_index[fp]["owner_class"] == trustindex.OWNER_FIRST_PARTY + + +def test_unknown_ownership_is_never_promoted_to_external_without_observation(store): + indexops.ingest(store, [{"endpoint": "https://who.example/a2a", + "source": "mcp_registry"}]) + entry = next(iter(store.trust_index.values())) + assert entry["owner_class"] == trustindex.OWNER_UNKNOWN + + +# -------------------------------------------------------------------------- +# 10. Bounded outbound behaviour +# -------------------------------------------------------------------------- +def test_recheck_is_capped_per_cycle(store): + for i in range(40): + e = trustindex.new_entry(f"https://x{i}.example/a2a", "mcp_registry") + store.trust_index[e["id"]] = e + calls = [] + out = indexops.recheck_due(store, limit=5, + runner=lambda url: calls.append(url) or _observed()) + assert out["checked"] == 5 + assert len(calls) == 5 + + +def test_remote_ingest_is_off_by_default(monkeypatch): + from app import indexsources + monkeypatch.delenv("GUILD_INDEX_INGEST", raising=False) + assert indexsources.enabled() is False + + +def test_source_adapter_identifies_itself_truthfully(): + from app import indexsources + assert "agent-guild" in indexsources.USER_AGENT + assert "http" in indexsources.USER_AGENT # contactable + assert "Mozilla" not in indexsources.USER_AGENT # never impersonates diff --git a/render.yaml b/render.yaml index ea15259..7e3343f 100644 --- a/render.yaml +++ b/render.yaml @@ -60,6 +60,46 @@ services: # stays OFF: GUILD_SCOUT_CONTACT is deliberately not set (default 0). - key: GUILD_SCOUT_AUTORUN value: "1" + # --- autonomous trust index (product-led pivot 2026-07-31) ------------ + # GUILD_INDEX_AUTORUN=1 lets the SAME lease-guarded, jittered scout loop + # also maintain the public index: ingest, recheck the stalest entries, + # run due customer watches and evaluate experiments. One loop, one lease, + # one deadline, one kill switch — set this to "0" to stop all index + # upkeep without touching the scout or taking the service down. + - key: GUILD_INDEX_AUTORUN + value: "1" + # Remote public-registry ingest is a SEPARATE switch and stays OFF until + # the local loop is proven in production. Outbound traffic to third-party + # infrastructure should never start because a container restarted. + # Adapters read documented public read-only APIs with a truthful, + # contactable User-Agent; no scraping, no auth bypass, no ToS evasion. + - key: GUILD_INDEX_INGEST + value: "0" + # How long an observation stays "fresh" before the index reports it as + # stale and the loop re-probes it. + - key: GUILD_INDEX_FRESH_TTL_S + value: "86400" + # Hard cap on endpoints probed per cycle. This is the bound that keeps an + # autonomous loop from becoming a crawler of other people's servers. + - key: GUILD_INDEX_RECHECK_BATCH + value: "8" + # --- experiment engine bounds ---------------------------------------- + # Minimum GENUINE-EXTERNAL actors before any verdict other than + # insufficient_evidence may be reached. Crawlers and first-party traffic + # are excluded structurally and can never reach this threshold. + - key: GUILD_EXP_MIN_QUALIFIED + value: "10" + - key: GUILD_EXP_WINDOW_DAYS + value: "14" + # --- paid index layer: prices are CONFIG, not doctrine ---------------- + # Every price is env-overridable within a hard ceiling enforced in + # app/pricing.py, so a price change is a config change and a rollback + # rather than a deploy and a revert. Unset = the documented default + # (deep_preflight 20cr/$0.02, evidence_bundle 100cr/$0.10, + # watch_cycle 5cr/$0.005, watch_provision free). GET /pricing publishes + # the live value, the ceiling and the stated basis for each. + # - key: GUILD_PRICE_DEEP_PREFLIGHT + # value: "20" # --- set these only when you go live with real card payments ----------- # - key: STRIPE_SECRET_KEY # sync: false From 3e3fc1651cf57c07014041a693172a4f48824d83 Mon Sep 17 00:00:00 2001 From: AgentTanuki Date: Fri, 31 Jul 2026 12:59:03 +0100 Subject: [PATCH 2/2] Index autorun: on where state is durable, plus an admin trigger to verify it Two gaps found by verifying the deploy rather than assuming it. 1. AUTORUN DEFAULT. GUILD_INDEX_AUTORUN was default-OFF, so the index shipped dormant and would have stayed dormant until someone remembered a dashboard setting. For a system whose whole premise is running without routine human involvement, that is a design flaw wearing a safety feature's clothes. Now three-way, and the asymmetry is the point: explicit "0" always wins (one-config-change kill switch, no redeploy, scout untouched); explicit "1" always on; unset = on only where a DURABLE backend exists. The index is a production surface whose value is a persistent observation history, so running it against an ephemeral JSON store would produce observations that vanish on restart - and it keeps the test suite from silently probing real hosts. Remote registry ingest remains a SEPARATE, independently default-OFF switch, so with it off the only outbound traffic is bounded rechecks of endpoints agents declared to us, which the scout already probes. 2. NO WAY TO FORCE A CYCLE. The loop runs on a jittered multi-hour schedule - correct for steady state, useless for verifying a deploy or recovering from an incident. POST /admin/index/cycle (admin-gated) runs exactly the same code path with exactly the same bounds, so the manual and scheduled paths cannot drift apart, and returns the live bounds it ran under. Tests: full suite green (43 new). --- docs/INTERFACE.md | 1 + live/guild/app/main.py | 21 +++++++++++++++++ live/guild/app/swarm/runner.py | 39 ++++++++++++++++++++++++------- live/guild/contract/contract.json | 6 +++++ 4 files changed, 59 insertions(+), 8 deletions(-) diff --git a/docs/INTERFACE.md b/docs/INTERFACE.md index daa2ca6..f5446a8 100644 --- a/docs/INTERFACE.md +++ b/docs/INTERFACE.md @@ -28,6 +28,7 @@ guild_mediated requires two-party cryptographic participation, a Guild-observed - `GET /.well-known/glama.json` - `POST /adjudicators/enroll` - `POST /admin/agents/{agent_id}/first-party` +- `POST /admin/index/cycle` - `POST /admin/issuer/rotate` - `GET /agents` - `GET /agents.md` diff --git a/live/guild/app/main.py b/live/guild/app/main.py index 590e8ee..52677b1 100644 --- a/live/guild/app/main.py +++ b/live/guild/app/main.py @@ -47,6 +47,7 @@ from . import trustindex from . import indexops from . import deepcheck +from . import indexsources from . import experiments from .state import store from .store import CanonicalWriteRefused @@ -3217,6 +3218,26 @@ def index_evidence_page(endpoint_id: str): """) +@app.post("/admin/index/cycle") +def admin_index_cycle(x_admin_token: Optional[str] = Header(None)): + """Force ONE bounded index cycle now. Admin-gated. + + The autonomous loop runs on a jittered multi-hour schedule, which is right + for steady state and useless when you need to verify a deploy or refresh + after an incident. This runs exactly the same code path with exactly the + same bounds — it is a trigger, not a second implementation, so there is no + way for the manual path and the scheduled path to drift apart.""" + if ADMIN_TOKEN and x_admin_token != ADMIN_TOKEN: + raise HTTPException(403, "an index cycle requires a valid X-Admin-Token") + from .swarm import runner as _runner + return {"cycle": _runner._run_index_cycle(store), + "autorun_enabled": _runner.index_autorun(store), + "bounds": { + "recheck_batch": trustindex.recheck_batch(), + "remote_ingest_enabled": indexsources.enabled(), + "fresh_ttl_s": trustindex.fresh_ttl_s()}} + + @app.get("/commercial") def commercial_report(): """The commercial scorecard, revenue first — the number that decides. diff --git a/live/guild/app/swarm/runner.py b/live/guild/app/swarm/runner.py index 4ae92b4..8e3b36f 100644 --- a/live/guild/app/swarm/runner.py +++ b/live/guild/app/swarm/runner.py @@ -181,13 +181,36 @@ def _run_scout(store: Any, *, fetch: Callable, deadline: float, return scout.run_scout(store, fetch=fetch, deadline=deadline) -def index_autorun() -> bool: - """Index upkeep is DEFAULT-OFF, like every other outbound loop here. - - Outbound traffic to third-party infrastructure must never begin merely - because a container restarted, and a test that exercises the scout must not - silently start probing real hosts. Enabled explicitly in render.yaml.""" - return (os.environ.get("GUILD_INDEX_AUTORUN") or "0").strip() == "1" +def index_autorun(store: Any = None) -> bool: + """Should this cycle maintain the index? + + Three-way, and the asymmetry is deliberate: + + * ``GUILD_INDEX_AUTORUN=0`` — always OFF. The explicit kill switch wins + over everything, so an operator can stop index upkeep in one config + change without redeploying or touching the scout. + * ``GUILD_INDEX_AUTORUN=1`` — always ON. + * unset — ON only when a DURABLE backend is present. + + That last rule is the useful one. The index is a production surface whose + whole value is a persistent observation history; running it against an + ephemeral JSON store would produce observations that vanish on restart. + It also means the test suite (JSON store) never silently starts probing + real hosts, while production (sqlite) does not sit dormant waiting for + someone to remember a dashboard setting — which for a system that is meant + to run without routine human involvement would be a design flaw, not a + safety feature. + + Note what is NOT enabled by this: remote registry ingest is a separate, + independently default-OFF switch (``GUILD_INDEX_INGEST``). With it off, the + only outbound traffic is bounded rechecks of endpoints agents declared to + us — which the scout already probes.""" + raw = (os.environ.get("GUILD_INDEX_AUTORUN") or "").strip() + if raw == "0": + return False + if raw == "1": + return True + return getattr(store, "backend", None) is not None def _run_index_cycle(store: Any) -> dict[str, Any]: @@ -200,7 +223,7 @@ def _run_index_cycle(store: Any) -> dict[str, Any]: from .. import indexops from .. import experiments as _experiments - if not index_autorun(): + if not index_autorun(store): return {"skipped": "GUILD_INDEX_AUTORUN is not enabled"} out: dict[str, Any] = {} try: diff --git a/live/guild/contract/contract.json b/live/guild/contract/contract.json index 398dcf6..e43009b 100644 --- a/live/guild/contract/contract.json +++ b/live/guild/contract/contract.json @@ -179,6 +179,12 @@ ], "path": "/admin/agents/{agent_id}/first-party" }, + { + "methods": [ + "POST" + ], + "path": "/admin/index/cycle" + }, { "methods": [ "POST"