Skip to content

Implementation of Readable iteration helpers that does not rely on Async Iteration#64429

Open
lukiano wants to merge 12 commits into
nodejs:mainfrom
lukiano:chore/faster-map-2
Open

Implementation of Readable iteration helpers that does not rely on Async Iteration#64429
lukiano wants to merge 12 commits into
nodejs:mainfrom
lukiano:chore/faster-map-2

Conversation

@lukiano

@lukiano lukiano commented Jul 11, 2026

Copy link
Copy Markdown

Hi, I'd like to offer an alternative implementation of Readable iteration helpers like find() that don't rely on for await (...) to consume items from the stream. This results in improved performance when the data is already available, such as when creating a Readable from an array, as shown in the attached screenshot (executed on a MacBook M2 Max).

The main con is increased duplication and code complexity. My understanding is that these helpers are still in the experimental phase, which I hope makes a change like this easier to accept.

There are many commits in the branch, but I'll squash them before merging.

The affected helpers are

  • map()
  • filter()
  • reduce()
  • find()
  • toArray()
  • some()
  • every()
  • drop()

Unaffected helpers:

  • flatmap()
  • take()

There's also a change to the from() method that increases the buffer watermark when the source is an array of data.
At first, I coded the changes manually, but eventually I had assistance from AI to keep backward compatibility. Still, there's a small break as can be seen in the updated test in test/parallel/test-stream-reduce.js.

The file benchmark/streams/operator-throughput.js allows interested parties to run the benchmarks on their computers or modify them to try other scenarios. I can remove it before merging.

stream-operator-throughput-summary

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance
  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem. labels Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants