Document tenant-managed SitePrefix architecture and operations

[chet]

This feature changes what “site prefix” means for operators and tenants, and it removes a database invariant that has made IP globally unique. The documentation needs to explain both the normal workflow and the boundaries that keep duplicate address space isolated.

Current state

• Operator-configured site_fabric_prefixes, REST IPBlocks, Core VpcPrefixes, and NetworkPrefixes have related but distinct jobs.
• Existing architecture material describes the current globally unique flow.
• OpenAPI endpoint details and CLI help will be updated with their implementation PRs, but they do not replace an architecture and operations guide.

General idea

• Document the configured SitePrefix, tenant SitePrefix, VpcPrefix, NetworkPrefix, REST projection, and IPAM namespace relationships.
• Add tenant examples for create, inspect status, carve an exact VpcPrefix, use it from a VPC, understand peering rejection, and retire resources in dependency order.
• Add operator guidance for site/profile eligibility, quotas/limits, both feature gates, schema/agent/Core/REST deployment order, readiness/preflight, canary selection, metrics/logs, and stuck Provisioning/Deleting diagnosis.
• Explain the site-wide-rendering restriction, physical-fabric attestation, RD/RT and gateway-import checks, effective ACL qualification, and why mixed safe/unsafe active FNN profiles remain deferred.
• Document gate-off as a freeze: existing resources remain visible and protected, additions stop, and safe deletion/drain remains available.
• Explain why PXE/cloud-init, reverse DNS, and raw-IP lookup need VPC-aware or fail-closed behavior.
• Document rollback before and after duplicate rows exist, including the fact that a global exclusion cannot be recreated over unresolved duplicates.
• State the deferred scope clearly: tenant IPv6, ETV/Flat, direct tenant-root segments, north/south BYOIP, tenant DNS views, shared routing profiles, and web UI.
• Link the final GitHub Initiative, API reference, CLI help, and qualification commands.

Acceptance criteria

• A tenant can follow the documented API or nicocli flow without learning provider-only IPBlock internals.
• An SRE can determine whether a site is eligible, enable each gate in order, observe DPU readiness, stop a rollout, and diagnose a stuck lifecycle.
• A physical-network operator can verify route-server RD preservation and prove that no external VRF combines overlap-enabled native RTs.
• A reviewer can trace every prefix from SitePrefix authority through VpcPrefix, NetworkPrefix, VPC/VRF, REST lease, and DPU policy.
• Upgrade and rollback instructions distinguish the pre-duplicate and post-duplicate cases.
• Terminology is consistent across the book, API descriptions, CLI help, and Initiative.
• Examples match the shipped request/response fields and qualification commands.

Not in this issue

• A web-console guide for a UI that does not exist yet.
• Public/BYOIP routing procedures.
• Repeating generated OpenAPI reference pages.

Likely areas

• book/src and/or docs architecture and operations pages
• configuration reference for FNN eligibility and feature gates
• tenant workflow examples
• links to generated admin CLI and nicocli help
• rollout and rollback runbook

Conditional issue draft

Do not file this issue with the initial twenty unless #3888 finds that ambiguity rejection and the deployed underlay identity are insufficient.

Planning

• Size: M
• Risk: Medium
• Best fit: Mid/senior engineer with SRE, networking, REST, and security review

Dependencies

• Native blocked by: Qualify and gate tenant-managed SitePrefixes across Core, DPU, and REST #3902 — Qualify and gate tenant-managed SitePrefixes across Core, DPU, and REST
• Coding can start: None for the outline
• Enablement: Documentation must be complete before the canary
• Parallel work: Every workstream

Part of #3883.