docs(0017): align backfill tasks + overview docs to local CH mirror#48
Merged
Conversation
Align the Stream 1/2 backfill tasks with the post-ADR-0007 reality: the local staging store is a ClickHouse mirror of the Hetzner prices.* schema (same local-CH -> Hetzner shape BE uses), not a local Postgres. - 0017: dual-role local CH (soroban_events input + prices.* mirror); add the archival/historical USD asset price requirement (volume_quote / volume_quote_usd / close_usd + oracle_prices). - 0053: Postgres -> local CH mirror across summary/steps/AC; volume_quote per 0058, USD columns left DEFAULT 0 for enrichment (0026); repoint provisioning blocker 0050 -> 0063. - 0028: SDEX cloud-push source is the 0027 local CH (not Postgres), target is Hetzner CH prices.* (not RDS), push via 0052 mTLS client, ReplacingMergeTree(version) idempotency. Replace personal username with "operator" throughout.
Update the stale two-secret cert/key assumption to the decided shape:
a single {cert,key,ca} JSON bundle secret per identity per env, named
by MTLS_SECRET_NAME (0063 Design Decision #3 / 0052's client).
- 0052: the cert-load step reads one bundle secret, not two.
- 0050: prices-api receives one {cert,key,ca} bundle per env.
Sweep both architecture overviews for staleness contradicting shipped
reality (0027 shipped local ClickHouse; ADR 0007 pivoted off RDS).
- Backfill sink: local Postgres -> local ClickHouse across the §7
two-stream section, data-flow diagrams, ASCII flow boxes, the
workstation subgraph, resumability prose, and diagram legends.
- Credentials: "2 secrets per env" -> single {cert,key,ca} JSON bundle
secret per identity (named by MTLS_SECRET_NAME).
Intentional historical notes ("prior Postgres design", "RDS removed",
changelog rows) left intact.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
prices.*schema (same local-CH → Hetzner shape BE uses), not a local Postgres; rows are pushed to Hetzner CH via the 0052 mTLS client withReplacingMergeTree(version)idempotency.volume_quote/volume_quote_usd/close_usd+oracle_prices), flagging the historical-oracle open question; 0053 populatesvolume_quote(0058) and leaves USD columnsDEFAULT 0for enrichment (0026).{cert,key,ca}JSON bundle secret per identity (named byMTLS_SECRET_NAME) in 0050, 0052, and both overview docs.