Skip to content

IC-1033: add Rabby Pipeline block tracing - #7

Open
maxim-inj wants to merge 2 commits into
masterfrom
feat/rabby-pipeline
Open

IC-1033: add Rabby Pipeline block tracing#7
maxim-inj wants to merge 2 commits into
masterfrom
feat/rabby-pipeline

Conversation

@maxim-inj

@maxim-inj maxim-inj commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds Chaintable Pipeline Mode 2 support to the Injective EVM gateway through trace_debankBlock(blockNumberOrHash). The response is wire-compatible with types.DebankOutPut and combines EVM execution data with the gateway's virtualized Cosmos bank events so Rabby/Pipeline can observe native x/bank, MTS, ERC20, and Circle USDC movement.

This revision also closes the audit's response-correctness findings:

  • unsafe native-before-EVM block orderings are rejected before tracing or reading a cached trace;
  • reverted and parent-reverted logs no longer consume persisted receipt indexes;
  • reverted CREATE/CREATE2 writes with no destination no longer add 0x0000000000000000000000000000000000000000 to storage_contracts;
  • out-of-range tracer log positions are clamped consistently for timeline and receipt-index assignment.

Related deployment references:

Design and implementation

RPC surface

  • Registers a trace namespace and exposes trace_debankBlock(blockNumberOrHash).
  • Resolves concrete block numbers and hashes, rejects pending blocks, and loads the full block, receipts, parent header, and trace result.
  • Returns block_file, an Ethereum-compatible header, RLP-encoded state_diff, and Pipeline's validation_hash.

EVM tracing and state diff

For traceable layouts, the endpoint requests one native geth muxTracer replay containing:

  • erc7562Tracer with logs enabled for nested call/create frames, EVM logs, errors, gas/output data, and exact per-frame SSTORE markers;
  • prestateTracer for touched accounts and storage slots.

Injective's current gRPC trace path does not provide a usable prestate-tracer post-state diff. The gateway therefore queries historical eth_getProof and eth_getCode at the traced height, compares those values with the captured prestate, and RLP-encodes Pipeline's BlockStorageDiff. This covers account balance/nonce/code changes, creation/deletion, storage changes, new bytecode, and root-only Cosmos state transitions.

The tracer's effective stackTopItemsSize=3 default is explicit in the request. This also versions the trace-cache key, preventing an offline gateway from reusing a Pipeline trace cached before the ordered-replay validation was introduced.

Ordered Cosmos/EVM replay safety

The current injective-core QueryTraceBlockRequest and QueryTraceTxRequest protobuf contracts accept only MsgEthereumTx. They cannot execute a native Cosmos message in the same mutable replay context. Starting at H-1 and stripping a mixed block down to EVM messages can therefore produce different frames, logs, touched slots, and state diffs when an EVM transaction reads state changed by an earlier native message.

The gateway now fails closed instead of returning data from that different transition:

  1. Decode the original Comet block in exact transaction/message order.
  2. Scan backward from the final EVM message.
  3. If a native or undecodable message precedes a later EVM message, return ordered Cosmos/EVM block replay is unavailable before trace-cache lookup and before the gRPC replay.
  4. Continue supporting native-only blocks and native messages after the final EVM message, since they cannot affect an earlier EVM execution.

Full support for every mixed ordering requires a follow-up injective-core trace API that accepts and executes complete ordered Cosmos transactions from beginning-of-block state. The gateway cannot reproduce native keeper transitions remotely with the current EVM-only request contract.

Pipeline wire compatibility

  • Mirrors Pipeline's block, transaction, event, trace, header, validation, and RLP layouts.
  • Reproduces Pipeline's MD5 trace/event identifiers, SHA-1 validation checksum, call/log timeline positions, trace addresses, parent links, success/error collections, lower-cased contract addresses, and gas-fee fields.
  • Preserves opcode-level storage_contracts semantics for reverted or net-zero SSTORE operations when an execution address is known.
  • Supports contract creation, nested CALL/DELEGATECALL/CALLCODE/EXTDELEGATECALL, logs, revert reasons, and dynamic-fee metadata.

Correct receipt log indexes

The opcode trace may contain logs from frames whose effects were reverted, while the receipt contains only persisted logs. Receipt indexes are now consumed only by frames that are neither failed nor parent-failed. A reverted child log stays in error_events with index zero, and the next surviving trace log receives the actual nonzero receipt index. Tracer positions beyond the call count are clamped to the same terminal bucket used by timeline assignment, preventing duplicate unconsumed receipt events.

Correct storage-contract addresses for failed creates

erc7562Tracer can clear a reverted CREATE/CREATE2 frame's to address. Storage-address resolution now returns (address, known) and only inserts a contract when the execution address is known. The trace still reports self_storage_change=true, but a missing creation destination is skipped instead of being represented as the zero address. Real calls to the zero address remain distinguishable because a non-nil to pointer is still accepted.

Cosmos and token movement integration

  • Converts virtualized Cosmos transactions into Pipeline transactions with deterministic synthetic root traces.
  • Appends virtual bank logs to the relevant EVM or synthetic trace using 0x0000000000000000000000000000000000000800.
  • Keeps persisted Cosmos fee/bank events successful when an EVM payload reverts while placing the reverted EVM frame in error_traces.
  • Covers native x/bank transfers, canonical Injective MTS/BankERC20 transfers, storage-backed ERC20 logs, and Circle's Injective USDC implementation for supported replay layouts.

Short usage guide

Enable EVM gRPC tracing on the Injective node:

# app.toml
enable-grpc-tracing = true

or:

injectived start --evm.enable-grpc-tracing=true

Enable the gateway namespace and Cosmos event virtualization:

WEB3INJ_JSONRPC_API=eth,net,web3,debug,trace
WEB3INJ_VIRTUALIZE_COSMOS_EVENTS=true

Query by height or hash:

curl -H 'content-type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"trace_debankBlock","params":["0x1275"]}' \
  http://127.0.0.1:8545

See docs/PIPELINE_TRACING.md for the focused live compatibility test and the ordered-replay safety boundary.

Test coverage

New focused regressions cover:

  • a native Cosmos transaction before an EVM transaction;
  • a native message between two EVM messages in one Cosmos transaction;
  • an undecodable transaction before an EVM transaction;
  • the supported EVM-before-native ordering;
  • a reverted child log before a surviving log with a nonzero receipt index;
  • a successful child whose parent later reverts;
  • out-of-range log-position clamping and duplicate prevention;
  • reverted CREATE with SSTORE, a missing to, retained storage flags, and no zero-address storage contract.

Validation commands:

go test ./...
go vet ./...
make build
go test -race -count=1 \
  ./internal/evm/rpc/backend \
  ./internal/evm/rpc/debank \
  ./internal/evm/rpc/namespaces/ethereum/trace \
  ./internal/evm/rpc \
  ./e2e

All commands pass.

Live verification

Validated against a running local Injective chain with gRPC tracing and Cosmos event virtualization enabled.

Audit-specific fixtures

Scenario Block Verified result
Native MsgSend at tx 0, EVM transfer at tx 1 12948 (0x3294) Both trace_debankBlock and debug_traceBlockByNumber reject before replay with the exact tx/message position; no divergent response is emitted
Reverted CREATE constructor executes SSTORE then REVERT 13085 self_storage_change=true, storage_change=true, one error trace, and storage_contracts=[] (no zero address)
Existing Circle USDC transfer compatibility fixture 4725 (0x1275) Real Pipeline decoder/checksum/RLP test passes: 3 txs, 7 traces, 15 events

Existing compatibility fixtures

Scenario Result
Native x/bank transfer, block 4162 2 Pipeline txs, 2 traces, 14 virtual bank events
Storage ERC20 deploy/transfer, blocks 4200/4214 Creation/code diff, ERC20 log, exact SSTORE, storage diff
Canonical MTS FixedSupplyBankERC20, blocks 4282/4314 Nested bank-precompile calls plus ERC20 and virtual bank events
Circle USDC proxy mint/transfer, blocks 4724/4725 Proxy delegatecalls, bank precompile, and virtual bank events
Reverted ERC20 transfer, block 4776 EVM error trace plus 11 persisted successful virtual fee events
Net-zero ERC20 self-transfer, block 6245 self_storage_change=true, token in storage_contracts, zero net RLP storage diffs

Supported fixtures pass number and hash lookup. Responses were unmarshaled into the real github.com/Chaintable/pipeline/types.DebankOutPut, had validation hashes recomputed by Pipeline code, and had state_diff decoded using Pipeline's RLP types.

The gated live compatibility command is:

WEB3INJ_E2E_DEBANK=1 \
WEB3INJ_E2E_DEBANK_RPC=http://127.0.0.1:8645 \
WEB3INJ_E2E_DEBANK_BLOCK=0x1275 \
go test ./e2e -run '^TestDebankPipelineCompatibilityLive$' -count=1 -v

Summary by CodeRabbit

  • New Features

    • Added the trace_debankBlock RPC method for Pipeline Mode 2 block tracing.
    • Added support for detailed transaction traces, events, storage changes, validation data, and native bank effects.
    • Added replay-order validation to prevent unsupported mixed Cosmos/EVM block traces.
  • Documentation

    • Added configuration, usage, response format, compatibility, and troubleshooting guidance for pipeline tracing.
  • Tests

    • Added comprehensive unit and live compatibility coverage for tracing, state diffs, event handling, and replay ordering.

@linear

linear Bot commented Jul 21, 2026

Copy link
Copy Markdown

IC-1033

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds Pipeline Mode 2 tracing through trace_debankBlock, including Pipeline-compatible output types, EVM replay and state-diff construction, ordered replay validation, namespace registration, configuration, documentation, unit tests, and live compatibility validation.

Changes

Pipeline tracing

Layer / File(s) Summary
Pipeline contracts and tracer configuration
internal/evm/rpc/debank/types.go, internal/evm/rpc/debank/builder.go
Defines Pipeline output, trace, event, validation, state-diff, build-input, and tracer configuration types.
Debank output builder
internal/evm/rpc/debank/builder.go
Builds block artifacts, nested traces and events, storage contracts, and RLP-encoded historical state diffs.
Ordered replay validation
internal/evm/rpc/backend/tracing.go, internal/evm/rpc/backend/trace_helpers_test.go
Validates Cosmos/EVM message ordering before tracing and rejects unsupported native or undecodable messages preceding Ethereum replay.
Trace API and enablement
internal/evm/rpc/namespaces/ethereum/trace/api.go, internal/evm/rpc/apis.go, .env.example, README.md, docs/PIPELINE_TRACING.md
Exposes trace_debankBlock, registers the trace namespace, enables it in the example environment, and documents configuration and response behavior.
Compatibility and builder validation
internal/evm/rpc/debank/builder_test.go, internal/evm/rpc/apis_test.go, e2e/debank_pipeline_test.go
Tests namespace registration, tracer configuration, builder output, state diffs, frame handling, replay ordering, and live RPC compatibility.

Estimated code review effort: 4 (Complex) | ~75 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant TraceAPI
  participant EVMBackend
  participant DebankBuilder
  participant StateReader
  Client->>TraceAPI: trace_debankBlock(blockNumberOrHash)
  TraceAPI->>EVMBackend: TraceBlock(block, TraceConfig)
  EVMBackend-->>TraceAPI: TraceResults
  TraceAPI->>DebankBuilder: Build(block, parent, receipts, TraceResults, StateReader)
  DebankBuilder->>StateReader: GetProof and GetCode at traced height
  StateReader-->>DebankBuilder: Historical account state
  DebankBuilder-->>TraceAPI: debank.Output
  TraceAPI-->>Client: Pipeline-compatible response
Loading

Possibly related PRs

  • InjectiveLabs/evm-gateway#6: Changes TraceBlock context-height handling while this PR adds replay-order validation to the same tracing path.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 11.11% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly points to the new pipeline block tracing feature, matching the main change set.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/rabby-pipeline

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
internal/evm/rpc/namespaces/ethereum/trace/api.go (1)

31-84: 🚀 Performance & Scalability | 🔵 Trivial

Consider guarding the cost of this endpoint.

DebankBlock performs a full-block mux replay plus one eth_getProof and one eth_getCode per touched account, all on the request path. On busy blocks this is a heavy, unbounded operation exposed via a public namespace. Consider a per-request timeout via ctx, and/or rate limiting / result caching keyed by block hash, so a burst of trace_debankBlock calls can't degrade the node.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/evm/rpc/namespaces/ethereum/trace/api.go` around lines 31 - 84,
Bound the expensive DebankBlock request path by applying a per-request timeout
through the existing ctx flow and adding protection against repeated work, such
as rate limiting or caching results by block hash. Ensure bursts of
trace_debankBlock calls cannot trigger unbounded full-block replay and
proof/code lookups, while preserving the existing successful response and error
handling behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@internal/evm/rpc/namespaces/ethereum/trace/api.go`:
- Around line 31-84: Bound the expensive DebankBlock request path by applying a
per-request timeout through the existing ctx flow and adding protection against
repeated work, such as rate limiting or caching results by block hash. Ensure
bursts of trace_debankBlock calls cannot trigger unbounded full-block replay and
proof/code lookups, while preserving the existing successful response and error
handling behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4a5b7e6c-eb75-46cd-a013-3e6aada3c06a

📥 Commits

Reviewing files that changed from the base of the PR and between bd53603 and 7228ab5.

📒 Files selected for processing (10)
  • .env.example
  • README.md
  • docs/PIPELINE_TRACING.md
  • e2e/debank_pipeline_test.go
  • internal/evm/rpc/apis.go
  • internal/evm/rpc/apis_test.go
  • internal/evm/rpc/debank/builder.go
  • internal/evm/rpc/debank/builder_test.go
  • internal/evm/rpc/debank/types.go
  • internal/evm/rpc/namespaces/ethereum/trace/api.go

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Chaintable Pipeline “Mode 2” block tracing support to evm-gateway by introducing a new trace JSON-RPC namespace with trace_debankBlock, returning a Pipeline wire-compatible response that merges EVM tracing results with the gateway’s virtualized Cosmos bank events.

Changes:

  • Added a new trace JSON-RPC namespace exposing trace_debankBlock(blockNumberOrHash).
  • Implemented Pipeline-compatible response building, including mux-tracer decoding, trace/event shaping, and RLP-encoded state diff generation via post-state eth_getProof/eth_getCode.
  • Added unit + live e2e compatibility coverage and documentation for configuration/usage.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
README.md Adds a short pointer to Pipeline tracing docs and the new RPC method.
internal/evm/rpc/namespaces/ethereum/trace/api.go Implements trace_debankBlock handler and block/receipt/trace loading.
internal/evm/rpc/debank/types.go Introduces Pipeline wire-compatible JSON/RLP types and validation hash helper.
internal/evm/rpc/debank/builder.go Core builder that converts blocks/receipts/traces/virtual logs into Pipeline output + state diff.
internal/evm/rpc/debank/builder_test.go Unit tests for mux config, trace/event shaping, state diff encoding, and revert/virtual-log handling.
internal/evm/rpc/apis.go Registers the new trace namespace.
internal/evm/rpc/apis_test.go Verifies trace namespace registration.
e2e/debank_pipeline_test.go Adds an opt-in live e2e test validating the full Mode 2 wire contract.
docs/PIPELINE_TRACING.md Documents configuration and a focused live compatibility test command.
.env.example Adds trace to the example enabled JSON-RPC namespaces.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/evm/rpc/debank/builder.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
internal/evm/rpc/backend/tracing.go (1)

181-222: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Replay-order checks should not be skipped on cache reads

  • In OfflineRPCOnly, cached trace results can still be returned without the replay-order guard when the block isn’t already supplied, so stale pre-fix entries can slip through.
  • For cacheable requests that arrive without a resolved block, the current flow fetches and validates the block before checking the cache, so a cache hit can still fail on a transient RPC error. Consider deferring that fetch or storing a cache version/validation marker.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/evm/rpc/backend/tracing.go` around lines 181 - 222, Update the
trace-cache path around validateTraceBlockReplayOrder so cache lookup does not
require an upfront Tendermint block fetch, avoiding transient RPC failures
before cache hits. Ensure cached results carry and validate a
replay-order/version marker, reject legacy or unvalidated entries (including in
OfflineRPCOnly), and only resolve and validate the block when required before
returning a result.
🧹 Nitpick comments (1)
internal/evm/rpc/backend/tracing.go (1)

292-373: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Transaction decode is now duplicated on every TraceBlock call.

validateTraceBlockReplayOrder decodes every tx/message in the block, and traceBlockEthereumTransactions (Lines 348-373) decodes the same block's txs again immediately afterward. Every trace request now pays the tx-decode cost twice. Consider having validateTraceBlockReplayOrder return the decoded Ethereum messages/hashes (or a shared decode helper) so traceBlockEthereumTransactions can reuse that work instead of re-decoding.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/evm/rpc/backend/tracing.go` around lines 292 - 373, Avoid decoding
block transactions twice during TraceBlock handling by sharing the decoded
transaction/message results between validateTraceBlockReplayOrder and
traceBlockEthereumTransactions. Update these methods, or introduce a focused
shared helper, so validation and Ethereum message/hash extraction reuse one
decode pass while preserving existing validation errors and returned ordering.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@internal/evm/rpc/backend/tracing.go`:
- Around line 181-222: Update the trace-cache path around
validateTraceBlockReplayOrder so cache lookup does not require an upfront
Tendermint block fetch, avoiding transient RPC failures before cache hits.
Ensure cached results carry and validate a replay-order/version marker, reject
legacy or unvalidated entries (including in OfflineRPCOnly), and only resolve
and validate the block when required before returning a result.

---

Nitpick comments:
In `@internal/evm/rpc/backend/tracing.go`:
- Around line 292-373: Avoid decoding block transactions twice during TraceBlock
handling by sharing the decoded transaction/message results between
validateTraceBlockReplayOrder and traceBlockEthereumTransactions. Update these
methods, or introduce a focused shared helper, so validation and Ethereum
message/hash extraction reuse one decode pass while preserving existing
validation errors and returned ordering.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0bfca92f-13da-452d-9790-24daa74ff159

📥 Commits

Reviewing files that changed from the base of the PR and between 7228ab5 and 51abd87.

📒 Files selected for processing (5)
  • docs/PIPELINE_TRACING.md
  • internal/evm/rpc/backend/trace_helpers_test.go
  • internal/evm/rpc/backend/tracing.go
  • internal/evm/rpc/debank/builder.go
  • internal/evm/rpc/debank/builder_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • docs/PIPELINE_TRACING.md
  • internal/evm/rpc/debank/builder.go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants