fix(browser): handle absolute https Wikipedia URLs in throughput benchmark#194
Merged
Merged
Conversation
…hmark The CSS selector :not([href*=':']) rejected article body links rendered as absolute https:// URLs (e.g. https://en.wikipedia.org/wiki/Foo) because 'https:' contains a colon. Wikipedia now renders some body links in this format, causing ~54% of iterations to fail across all providers on the 2026-07-17 run. Three fixes: - Replace CSS :not() filter with JS-based isArticleLink() that only checks for namespace colons after /wiki/, handling relative, protocol-relative, and absolute URL formats - Skip goBack and dependent actions when click fails to avoid 30s timeouts on stub articles with no body links - Add stub-filtering retry logic to workflow URL resolution - Fix hardcoded expectedActions=50 in run.ts to use ACTIONS_PER_SESSION (10)
Contributor License AgreementAll contributors are covered by a CLA. |
Contributor
Storage Benchmark Results1MB Files
View full run · SVGs available as build artifacts |
Contributor
Snapshot/Fork Benchmark Resultssmall dataset
|
Contributor
Browser Benchmark Results
View full run · SVG available as build artifact |
Contributor
Browser Throughput Benchmark Results
View full run · SVG available as build artifact |
Contributor
Sandbox Benchmark ResultsSequential
Staggered
Burst
Sandbox Dax
View full run · SVGs available as build artifacts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The 2026-07-17 browser throughput benchmark run showed all 6 providers dropping to ~46% success rate with scores collapsing to ~7 (from ~50-77):
Root cause
The CSS selector
#mw-content-text a[href*="/wiki/"]:not([href*=":"])`` was rejecting article body links rendered as **absolute URLs** (https://en.wikipedia.org/wiki/Article`) becausehttps:contains a colon. Wikipedia now renders some body links in this absolute format (as part of the Parsoid Read Views rollout). The:not([href*=":"])filter was designed to exclude namespace pages (Help:, File:, Category:) but also rejects any URL withhttps:.When the click action failed to find a link,
goBacknavigated to a blank page and the finalwaitForSelectortimed out at 30s, producing the bimodal ~42s / ~4s task times and tanking scores across all providers.Additionally, the
expectedActions = 50hardcoded value inrun.tsmeant the "OK" count was always 0 (actual actions per session is 10).Fixes
throughput-benchmark.ts): Replace CSS:not()filter with JS-basedisArticleLink()that checks for namespace colons only after/wiki/, correctly handling relative, protocol-relative, and absolutehttps://URL formatsthroughput-benchmark.ts): When click fails, skip remaining dependent actions (6-10) instead of timing out for 40sbrowser-throughput-benchmarks.yml): Retry up to 5 times to find articles with body links during URL resolutionrun.ts): UseACTIONS_PER_SESSIONinstead of hardcoded50