From edabf60984ac660a629b761a64b5262bb066b9d5 Mon Sep 17 00:00:00 2001 From: Oluwatomilola Date: Tue, 25 Aug 2026 14:22:05 +0100 Subject: [PATCH] feat: add DataStore reference documentation and update meta.json --- apps/docs/content/meta.json | 5 +- apps/docs/content/reference/data-store.mdx | 168 +++++++++++++++++++++ 2 files changed, 172 insertions(+), 1 deletion(-) create mode 100644 apps/docs/content/reference/data-store.mdx diff --git a/apps/docs/content/meta.json b/apps/docs/content/meta.json index 835dd92..19e940e 100644 --- a/apps/docs/content/meta.json +++ b/apps/docs/content/meta.json @@ -8,7 +8,10 @@ "concepts/liquidation" ] }, - { "label": "Reference", "pages": ["reference/glossary"] }, + { + "label": "Reference", + "pages": ["reference/data-store", "reference/glossary"] + }, { "label": "Resources", "pages": [ diff --git a/apps/docs/content/reference/data-store.mdx b/apps/docs/content/reference/data-store.mdx new file mode 100644 index 0000000..e107688 --- /dev/null +++ b/apps/docs/content/reference/data-store.mdx @@ -0,0 +1,168 @@ +--- +title: DataStore contract +description: Read and write SO4 protocol state directly from Soroban with the DataStore API, typed keys, authorization rules, and storage TTL guidance. +updated: 2026-08-25 +status: stable +--- + +DataStore is the protocol's typed key-value contract: handlers write protocol state under deterministic `BytesN<32>` keys, and integrators read that state through Soroban RPC without depending on the indexer. + +This page describes the deployed testnet contract at `CCZ3VKBEDLNBO2JM3EXL3SNBDJOV5BTN52FVQPER7F6D5GCE53PITQ3J`. The source of truth is the generated `DataStoreClient` from the contracts repository at the commit used for the deployment. DataStore has no per-key owner: every address with the `CONTROLLER` role can write every namespace. + +## Storage model {#storage-model} + +DataStore wraps each caller-supplied 32-byte key in a typed `DataKey` before storing it. The wrapper prevents a `u128` entry from colliding with an `i128`, address, boolean, bytes32, or set entry that uses the same raw key. + +| Storage | Values | Archival and rent | +| --- | --- | --- | +| Instance | `Initialized`, `RoleStore`, `InstanceU128`, `InstanceI128` cache entries | All instance keys share the contract instance entry. If it is archived, all instance state becomes unavailable together and the contract needs restoration. DataStore does not currently extend this entry automatically. | +| Persistent | All ordinary scalar values, address and bytes32 sets, nonce, keeper metrics, position managers, and fee configuration | Each typed key is independently recoverable after archival and independently charged. Ordinary scalar reads and writes do not extend TTL. The `B32Set` helpers extend the set entry to `518400` ledgers when its remaining TTL is below `259200` ledgers. | +| Temporary | None | DataStore does not use temporary storage. Do not infer temporary semantics from a missing value. A missing persistent value may be archived, or may have its type-specific default. | + +`set_u128_config` writes the same value to persistent storage and the instance cache. `get_u128_cached` reads persistent storage and refreshes the instance cache; it is therefore self-healing after a plain `set_u128`, delta, increment, or decrement. Cache reads are not a substitute for restoring an archived persistent entry. + +## API coverage {#api-coverage} + +The generated binding exposes 46 public functions. The list below is grouped only where parameter and authorization behavior is identical; each function name is included so the list can be checked against the generated `DataStoreClient`. + +### Initialization + +| Function | Parameters and result | Auth and errors | +| --- | --- | --- | +| `initialize` | `admin: Address`, `role_store: Address`; no result | `admin` signs. `AlreadyInitialized` if called twice. | + +### `u128` values + +All `u128` values are unsigned integers. Their units come from the key namespace: pool and token amounts use token-native smallest units; USD values use the protocol's 30-decimal `FLOAT_PRECISION`; fee values use the unit documented by the consuming handler; ledger values use ledger sequences. A missing read returns `0`. + +| Functions | Parameters and result | Auth and errors | +| --- | --- | --- | +| `get_u128`, `get_u128_batch` | `key: BytesN<32>` or `keys: Vec`; `u128` or values in input order | Read-only. | +| `get_u128_instance` | `key: BytesN<32>`; `u128` | Read-only instance cache. Missing returns `0`. | +| `set_u128_instance`, `set_u128` | `caller: Address`, `key`, `value: u128`; returns `value` | Caller signs and must hold `CONTROLLER`; `NotInitialized` or `Unauthorized`. | +| `set_u128_config` | `caller`, `key`, `value`; returns `value` | `CONTROLLER`; `NotInitialized` or `Unauthorized`. Writes persistent and instance entries. | +| `get_u128_cached` | `key`; `u128` | Read-only persistent read plus cache refresh. | +| `remove_u128` | `caller`, `key`; no result | `CONTROLLER`; `NotInitialized` or `Unauthorized`. | +| `apply_delta_to_u128` | `caller`, `key`, `delta: i128`; returns next `u128` | `CONTROLLER`; `Underflow` when a negative delta exceeds the stored value. Positive overflow saturates. | +| `increment_u128`, `decrement_u128` | `caller`, `key`, `amount: u128`; returns next `u128` | `CONTROLLER`; `Underflow` for an excessive decrement. Increment saturates on overflow. | + +### `i128`, address, boolean, and bytes32 values + +`i128` values are signed protocol quantities, commonly signed deltas or funding values. `Address` values are Stellar account or contract addresses. `BytesN<32>` values are opaque hashes such as order keys. Missing `i128`, bool, or bytes32 reads return `0`, `false`, or 32 zero bytes. A missing address returns `None`. + +| Functions | Parameters and result | Auth and errors | +| --- | --- | --- | +| `get_i128`, `get_i128_instance` | `key`; `i128` | Read-only; persistent or instance respectively. | +| `set_i128`, `set_i128_instance` | `caller`, `key`, `value: i128`; returns `value` | `CONTROLLER`; `NotInitialized` or `Unauthorized`. | +| `remove_i128` | `caller`, `key`; no result | `CONTROLLER`; `NotInitialized` or `Unauthorized`. | +| `apply_delta_to_i128` | `caller`, `key`, `delta: i128`; returns next `i128` | `CONTROLLER`; `NotInitialized` or `Unauthorized`. Uses saturating addition. | +| `get_address` | `key`; `Option
` | Read-only. | +| `set_address` | `caller`, `key`, `value: Address`; returns address | `CONTROLLER`; `NotInitialized` or `Unauthorized`. | +| `remove_address` | `caller`, `key`; no result | `CONTROLLER`; `NotInitialized` or `Unauthorized`. | +| `get_bool` | `key`; `bool` | Read-only. | +| `set_bool` | `caller`, `key`, `value: bool`; returns bool | `CONTROLLER`; `NotInitialized` or `Unauthorized`. | +| `remove_bool` | `caller`, `key`; no result | `CONTROLLER`; `NotInitialized` or `Unauthorized`. | +| `get_bytes32` | `key`; `BytesN<32>` | Read-only. | +| `set_bytes32` | `caller`, `key`, `value: BytesN<32>`; returns value | `CONTROLLER`; `NotInitialized` or `Unauthorized`. | + +### Sets and counters + +Set pagination uses a half-open range: `start` is inclusive and `end` is exclusive, both clamped to the set length. Adding a duplicate is a no-op. Removing an absent member is a no-op. Address sets do not extend TTL; bytes32 sets do. + +| Functions | Parameters and result | Auth and errors | +| --- | --- | --- | +| `add_address_to_set`, `remove_address_from_set` | `caller`, `set_key`, `value: Address`; no result | `CONTROLLER`; `NotInitialized` or `Unauthorized`. Persistent. | +| `get_address_set_count`, `contains_address` | `set_key` and, for `contains_address`, `value`; `u32` or bool | Read-only persistent. | +| `get_address_set_at` | `set_key`, `start: u32`, `end: u32`; `Vec
` | Read-only persistent. | +| `add_bytes32_to_set`, `remove_bytes32_from_set` | `caller`, `set_key`, `value: BytesN<32>`; no result | `CONTROLLER`; `NotInitialized` or `Unauthorized`. Persistent and TTL-renewing for the set entry. | +| `get_bytes32_set_count`, `contains_bytes32` | `set_key` and, for `contains_bytes32`, `value`; `u32` or bool | Read-only persistent and TTL-renewing for the set entry. | +| `get_bytes32_set_at` | `set_key`, `start`, `end`; `Vec>` | Read-only persistent and TTL-renewing for the set entry. | +| `get_nonce` | no parameters; `u64` | Read-only persistent `u128` nonce value narrowed to `u64`. Missing returns `0`. | +| `increment_nonce` | `caller`; next `u64` | Caller signs and must hold `CONTROLLER`; `NotInitialized` or `Unauthorized`. | + +### Keeper, manager, and fee configuration + +| Functions | Parameters and result | Auth and errors | +| --- | --- | --- | +| `record_keeper_execution` | `caller`, `keeper: Address`, `executed_price: u128`, `expected_price: u128`; no result | `CONTROLLER`; records count and variance in `u128` persistent keys. | +| `get_keeper_stats` | `keeper`; `(execution_count, total_variance_bps, slash_penalty_amount)` | Read-only. Prices use the consumer's price scale; variance is basis points; penalty is token-native units. | +| `get_position_manager` | `owner`, `market`; `Option
` | Read-only persistent. | +| `set_position_manager` | `owner`, `market`, `manager`; returns manager | `owner` signs. No `CONTROLLER` role is required because the owner controls this delegation. Persistent. | +| `get_liquidation_execution_fee` | `market`; `u128` | Read-only persistent, token-native fee units. Missing returns `0`. | +| `set_liquidation_execution_fee` | `caller`, `market`, `fee: u128`; returns fee | `CONTROLLER`; `LiquidationExecutionFeeTooHigh` above `1000 * 10^7` smallest token units. | +| `get_min_execution_fee` | no parameters; `u128` | Read-only persistent, token-native fee units. Missing returns `0`. | +| `set_min_execution_fee` | `caller`, `fee: u128`; returns fee | `CONTROLLER`; `NotInitialized` or `Unauthorized`. | + +Every controller-gated write also requires the caller's Soroban authorization entry. `Unauthorized` means the caller is not a controller, not that the key belongs to another subsystem. `NotInitialized` means the DataStore instance has not been linked to its RoleStore. + +## Key derivation {#key-derivation} + +The raw `BytesN<32>` key is the SHA-256 digest of a tagged, length-prefixed byte sequence. For a string or address, append a 2-byte big-endian byte length followed by its UTF-8 bytes. For a bool, append `01` for true or `00` for false. For a `u32` tier, append its 4-byte big-endian representation. The typed storage wrapper is added by DataStore after this raw key is derived. + +For example, the pool amount key is: + +```text +sha256(utf8("POOL_AMOUNT") with a 2-byte length prefix + || utf8(market address) with a 2-byte length prefix + || utf8(token address) with a 2-byte length prefix) +``` + +The common key families are: + +| Value | Derivation tag and arguments | Getter | +| --- | --- | --- | +| Market metadata | `MARKET`, `market` | `get_address` or the market consumer's typed reader | +| Pool amount | `POOL_AMOUNT`, `market`, `token` | `get_u128` | +| Open interest | `OPEN_INTEREST`, `market`, `collateral_token`, `is_long` | `get_u128` | +| Market list | `MARKET_LIST` | `get_bytes32_set_at` | +| Position | `POSITION`, `account`, `market`, `collateral_token`, `is_long` | Consumer-specific position reader | +| Funding factor | `FUNDING_FACTOR`, `market` | `get_i128` or `get_u128` according to the value written | +| Max leverage | `MAX_LEVERAGE`, `market` | `get_u128` | +| Position manager | `POSITION_MANAGER`, `owner`, `market` | `get_address` | +| Nonce | `NONCE` | `get_nonce` | +| Minimum execution fee | `MIN_EXECUTION_FEE` | `get_min_execution_fee` | + +The repository's `scripts/compute_key.py` mirrors these encodings. This example derives the TETH/TUSDC pool amount key using the committed testnet addresses: + +```bash +$market = 'CCBUUSYZJTGVA6PYUNQDFPZFHTBZ2QSHOUO7YAGRQVA46T3ZLSIYULS4' +$token = 'CBAN5YU3KRDKPTQ2H76D6S7HQFPRBGUD524F65BUM2RQCITPTRLKWKES' +# Hash: 2-byte big-endian length + UTF-8 bytes for each part. +$parts = @('POOL_AMOUNT', $market, $token) +$bytes = [System.Collections.Generic.List[byte]]::new() +foreach ($part in $parts) { + $raw = [Text.Encoding]::UTF8.GetBytes($part) + $length = [BitConverter]::GetBytes([UInt16]$raw.Length) + if ([BitConverter]::IsLittleEndian) { [Array]::Reverse($length) } + $bytes.AddRange($length) + $bytes.AddRange($raw) +} +(([Security.Cryptography.SHA256]::Create().ComputeHash($bytes.ToArray()) | + ForEach-Object ToString x2) -join '') +``` + +The command returns `43ef499df0a57784930a854a17ecb1298acd38ebfebaa336b3d88bf609d16d5`. This is the raw key passed to the typed `DataKey::U128` wrapper, not the wrapper's XDR encoding. + +Do not concatenate unprefixed strings or hash the binary contract address. The helpers hash the printable Stellar strkey, including its length prefix. + +## Testnet read {#testnet-read} + +To read a value directly, derive the raw key, encode it as a Soroban `BytesN<32>` argument, and simulate the corresponding getter against `https://soroban-testnet.stellar.org`. For a pool amount, the complete inputs are: + +```text +DataStore: CCZ3VKBEDLNBO2JM3EXL3SNBDJOV5BTN52FVQPER7F6D5GCE53PITQ3J +market: CCBUUSYZJTGVA6PYUNQDFPZFHTBZ2QSHOUO7YAGRQVA46T3ZLSIYULS4 +token: CBAN5YU3KRDKPTQ2H76D6S7HQFPRBGUD524F65BUM2RQCITPTRLKWKES +getter: get_u128 +network: Test SDF Network ; September 2015 +``` + +The returned integer is in the token's smallest unit. For TUSDC, divide by $10^6$ to display a token amount. A direct RPC read must include the `DataKey::U128` typed contract-data entry in its footprint; a raw `BytesN<32>` ledger lookup is not the same key. Record the ledger sequence with the value because persistent entries can later archive. + +For a read-only client, the equivalent contract call is `get_u128(key)`. For a write, use the generated binding, have a `CONTROLLER` address sign the authorization entry, simulate first, and submit only after checking the footprint and rent cost. The developer pages for contract clients and reading data are deferred until their prerequisite content lands. + +## Operational cautions {#operational-cautions} + +A zero result is not proof that a protocol value was intentionally configured to zero. It can mean the typed key is wrong, the value was never written, or the persistent entry is archived. Restore an archived entry before recreating it. Monitor the contract instance, scalar DataStore entries, and their related indexes independently. + +Granting `CONTROLLER` grants write access to all DataStore namespaces. Integrators should treat every controller contract as equally trusted, even when application conventions assign a key family to a particular handler.