docs(lore-0063): correct prices.* sizing from 64k ground-truth measurement#49
Merged
karczuRF merged 3 commits intoJun 22, 2026
Merged
Conversation
…ement Ran a fresh 64,000-ledger backfill (62016000-62079999, cached partition, fully local) through the real prices-clickhouse pipeline and measured system.parts: 114 MiB / ~1,872 B/ledger. Combined with task 0060's 10k+100k runs, the real prices.* footprint is ~1.9-3.7 KB/ledger (~3.5-6 GB/yr), superseding the task-0046 ~74 B/ledger / ~0.45 GB/yr estimate by ~25-50×. Driver is trading-pair diversity, not ledger count. Cost-share with BE accordingly moves from ~1%/$1-2 to ~10-15%/$8-11 per env/mo. Adds a shared-vs-dedicated-container cost table: a dedicated prices CH container (ADR 0007 Alt-3) costs ~2x and breaks BE's in-cluster price_usd_series JOIN, so the shared prices DB stays correct (sidecar = task-0047-RED fallback only). Saves the measurement as notes/G-64k-sizing-remeasure.md and refreshes the schema-overview §1.2/§8.3/§8.5 + revision history.
Record the local-only ground-truth test over 128,000 cached ledgers (62848000-62975999) through the production prices-clickhouse schema: SDEX extraction verified (16.5M trades), AMM path proven via Aquarius (864 ticks; Phoenix/Soroswap 0 due to in-window pool-discovery limit), oracle captured. Full-schema footprint 4.13 KiB/ledger (516.87 MiB / 128k, ~22 GiB/yr at this high-activity range); parse 18.7 ms/ledger. Link forward from G-64k-sizing-remeasure.
The schema overview never documented the `close_usd` column or its BE-facing read-surface views, so a reader searching the doc could not find the historical USD asset price BE requested. Add the `close_usd Decimal(38,14) DEFAULT 0` column to the `price_ohlcv_*` DDL and a §3.2 subsection covering the derived VIEWs (`price_usd_series` / `_1h`, `usd_reference` / `_1h`, `identity_by_contract`), the read-time ok / no_asset_price / no_reference status discriminator, caller-owned grain selection, and the load-bearing USDC-issuer literal. Source of truth: packages/prices-clickhouse/schema/views.sql.
karczuRF
deleted the
feat/0063_provision-prices-db-on-hetzner-ch-self-served
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Re-measured the
prices.*ClickHouse footprint with a fresh 64,000-ledger backfill (window62016000-62079999) and corrected the docs + cost-share, which were still citing the superseded task-0046 estimate.Fully local / prepare-only: docker ClickHouse 25.6, cached partition (no S3 fetch), no prod infra touched.
Key finding — sizing was wrong by ~25–50×
Cost-share / architecture impact
$1-2to ~10-15%/$8-11per env/mo.price_usd_seriesJOIN (0199 contract) → sharedpricesDB stays correct; sidecar remains the task-0047-RED fallback only._1h/_4hretention cap) is trivial for the shared box.Changes
lore/1-tasks/active/0063_…/notes/G-64k-sizing-remeasure.md— full measurement record, three-window comparison, corrected projection, cost table, reproduction block.docs/database-schema/database-schema-overview.md§1.2, §8.3 (sizing table + "superseded" callout), §8.5 (cost lines), revision-history row.G-provisioning-plan.mdforward-links the new note.Caveats (documented in the note)
close_usd/USD-series enrichment not run → that column compresses to ~0 here (adds only a few % when populated; it's a column, not a new row class).