Skip to content

Add initializeBucketCapacity to seed empty-bucket metrics#2666

Open
delthas wants to merge 2 commits into
development/8.5from
improvement/ARSN-610/init-bucket-capacity-metric
Open

Add initializeBucketCapacity to seed empty-bucket metrics#2666
delthas wants to merge 2 commits into
development/8.5from
improvement/ARSN-610/init-bucket-capacity-metric

Conversation

@delthas

@delthas delthas commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Context

Bucket quota metrics live in the __infostore collection, keyed bucket_<name>_<creationTimestamp>, and are written today only by the s3utils count-items cron. A freshly created bucket — or a quota enabled on an empty bucket — has no metric document until the next cron run (up to 24h), so CloudServer's bucket-quota check finds nothing in Scuba and fails open (ARTESCA-17063).

MongoClientInterface already touches __infostore internally (writeUUIDIfNotExists), so this is a natural home for the write.

Change

  • Add MongoClientInterface.initializeBucketCapacity(bucketName, creationDate, log, cb) — idempotently inserts a zero-value bucket metric document via updateOne({_id}, {$setOnInsert: zeroDoc}, {upsert: true}). An existing document (e.g. one written by count-items with real values) is never overwritten.
    • _id = bucket_<name>_<new Date(creationDate).getTime()>.
    • Document shape matches what count-items produces / Scuba reads: measuredOn plus all-zero usedCapacity and objectCount, built from the module-level emptyBucketCapacityMetrics constant.
  • Expose it through MetadataWrapper with a capability guard (no-op for non-mongodb backends, matching the existing getUUID/countItems pattern).

The consumer is CloudServer (CLDSRV-949, scality/cloudserver#6221), which calls this on createBucket and on bucketUpdateQuota of a verifiably-empty bucket.

Tests

Unit tests (MongoClientInterface.spec.js, real mongodb-memory-server): inserts a zero document and reads it back; confirms an existing document is not overwritten (idempotent); returns InternalError on update failure.

Issue: ARSN-610

@bert-e

bert-e commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Hello delthas,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request TBA
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@scality scality deleted a comment from bert-e Jul 6, 2026
@bert-e

bert-e commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 64.63415% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.70%. Comparing base (ff0aae1) to head (4fcedbc).
⚠️ Report is 9 commits behind head on development/8.5.

Files with missing lines Patch % Lines
lib/storage/metadata/MetadataWrapper.js 58.46% 27 Missing ⚠️
...orage/metadata/mongoclient/MongoClientInterface.ts 88.23% 2 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff                @@
##           development/8.5    #2666   +/-   ##
================================================
  Coverage            74.70%   74.70%           
================================================
  Files                  229      229           
  Lines                18682    18698   +16     
  Branches              3876     3909   +33     
================================================
+ Hits                 13956    13969   +13     
- Misses                4721     4724    +3     
  Partials                 5        5           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@delthas delthas force-pushed the improvement/ARSN-610/init-bucket-capacity-metric branch from 454e618 to 05db0fa Compare July 6, 2026 14:43
@scality scality deleted a comment from bert-e Jul 6, 2026
@delthas delthas force-pushed the improvement/ARSN-610/init-bucket-capacity-metric branch 2 times, most recently from 8406297 to 7f70f0e Compare July 7, 2026 08:54
Comment thread lib/storage/metadata/mongoclient/MongoClientInterface.ts
@delthas delthas marked this pull request as ready for review July 7, 2026 13:54
@scality scality deleted a comment from bert-e Jul 7, 2026
@delthas delthas force-pushed the improvement/ARSN-610/init-bucket-capacity-metric branch from 7f70f0e to 45c987c Compare July 7, 2026 14:13
Isolates prettier reformatting of files that were already
prettier-dirty on development/8.5, so the functional change commit
that follows stays prettier-clean.

Issue: ARSN-610
@delthas delthas force-pushed the improvement/ARSN-610/init-bucket-capacity-metric branch from 45c987c to 43a1d05 Compare July 8, 2026 09:38
@delthas delthas requested review from a team, SylvainSenechal and maeldonn July 8, 2026 09:39
Comment thread lib/storage/metadata/mongoclient/MongoClientInterface.ts Outdated
Comment thread lib/storage/metadata/mongoclient/MongoClientInterface.ts Outdated
Comment thread lib/storage/metadata/mongoclient/MongoClientInterface.ts Outdated
@scality scality deleted a comment from bert-e Jul 8, 2026
Add a MongoClientInterface method that idempotently inserts a
zero-value bucket capacity document into __infostore when none
exists (upsert with $setOnInsert, so an existing count-items
document with real values is never overwritten). Exposed through
MetadataWrapper (no-op for non-mongodb backends).

This lets CloudServer seed a metric document at bucket creation
(or when a quota is enabled on an empty bucket) so bucket quota
checks are served before the periodic count-items job runs.

Issue: ARSN-610
@delthas delthas force-pushed the improvement/ARSN-610/init-bucket-capacity-metric branch from 43a1d05 to 4fcedbc Compare July 9, 2026 08:59
@delthas delthas requested a review from maeldonn July 9, 2026 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants