Skip to content

resource_manager: add replayable per-second RU observations - #1539

Open
JmPotato wants to merge 1 commit into
pingcap:masterfrom
JmPotato:codex/ru-minute-peak
Open

JmPotato wants to merge 1 commit into
pingcap:masterfrom
JmPotato:codex/ru-minute-peak

Conversation

@JmPotato

@JmPotato JmPotato commented Sep 20, 2026 •

Copy link
Copy Markdown
Member

Issue Number: ref #1545

Clients report accumulated RU consumption asynchronously, so PD cannot reconstruct aligned one-second peaks from report-arrival times.

Add the optional Consumption.ru_by_second observation ledger. Each payload identifies a contiguous range of closed UTC seconds and carries immutable, signed RRU/WRU contributions, including zero seconds. The outer client ID, resource group, keyspace and source kind identify the producer; overlapping ranges can be replayed and deduplicated by the consumer. An absent ledger means unsupported observation, not zero consumption.

The existing scalar consumption fields retain their billing and token-allocation semantics. Regenerate the Go bindings and protocol compatibility lock.

Validation: make check and go test ./pkg/resource_manager. Local PD/TiDB SQL integration verified transport and reconstruction against raw confirmed-accounting logs; the PD follow-up contains the aggregation and end-to-end evidence.

Summary by CodeRabbit

  • New Features
    • Added per-second resource consumption observations, including UTC start times and repeated consumption buckets.
    • Consumption buckets now report signed read and write resource usage values.
    • These observations are informational and do not affect token allocation or billing.

Signed-off-by: JmPotato <github@ipotato.me>
@ti-chi-bot
ti-chi-bot Bot requested a review from TszKitLo40 September 20, 2026 12:33
@ti-chi-bot ti-chi-bot Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Sep 20, 2026
@coderabbitai

coderabbitai Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 87b503c8-1f14-436b-86c6-51a704e41982

📥 Commits

Reviewing files that changed from the base of the PR and between d80b6c5 and 2a45fb4.

⛔ Files ignored due to path filters (2)
  • pkg/resource_manager/resource_manager.pb.go is excluded by !**/*.pb.go
  • scripts/proto.lock is excluded by !**/*.lock
📒 Files selected for processing (1)
  • proto/resource_manager.proto

Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review.


📝 Walkthrough

Walkthrough

The protobuf schema adds an optional per-second RU observation ledger to Consumption. It defines timestamped buckets with signed read and write settlement values. The ledger is separate from token allocation and billing.

Changes

RU consumption ledger

Layer / File(s) Summary
Ledger message contract
proto/resource_manager.proto
Consumption adds optional field ru_by_second. New RUConsumptionBySecond and RUConsumptionBucket messages define the timestamp, bucket list, and signed rru and wru values.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Suggested reviewers: disksing

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding replayable per-second RU observations.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit hops through protobuf snow
New RU buckets line up in a row
Reads and writes keep signed score
The ledger observes, but bills no more
Timestamps guide each careful flow

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot

ti-chi-bot Bot commented Sep 20, 2026

Copy link
Copy Markdown

@coderabbitai[bot]: adding LGTM is restricted to approvers and reviewers in OWNERS files.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot

ti-chi-bot Bot commented Sep 20, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: coderabbitai[bot]
Once this PR has been reviewed and has the lgtm label, please assign kennytm, likidu for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

JmPotato added a commit to JmPotato/pd that referenced this pull request Sep 23, 2026
Clients can report RRU and WRU per UTC second in
Consumption.ru_by_second. Merge the seconds reported by every source of a
resource group and publish the busiest second of each completed minute
as minute-end-timestamped gauges, with its RRU/WRU components and a
coverage signal. Minutes with missing, conflicting or invalid seconds are
withheld rather than published as lower values.

A source that stops reporting expires after 180 seconds and withholds
only the minutes overlapping its unreported tail. Source and group counts
are capped, and an overflow withholds only the affected group. Metrics
flushers are serialized across leadership terms, and the timeline is
cleared when a term ends.

The kvproto replacement is temporary until pingcap/kvproto#1539 is
merged; that revision also adds WatchGCStates, answered as unimplemented.

Signed-off-by: JmPotato <github@ipotato.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/needs-triage-completed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant