gh-123471: Make concurrent iteration over itertools.islice thread-safe#144528
gh-123471: Make concurrent iteration over itertools.islice thread-safe#144528eendebakpt wants to merge 8 commits intopython:mainfrom
Conversation
| PyObject *item; | ||
| PyObject *it = lz->it; | ||
| Py_ssize_t stop = lz->stop; | ||
| Py_ssize_t oldnext; |
There was a problem hiding this comment.
I suggest to just use a critical section, performance of concurrent iteration is not important.
There was a problem hiding this comment.
The code owner @rhettinger has been hesitant with changes that effect performance. I have not measured it yet but both this PR or the critical section will probably be performance neutral on the normal build. For the FT build critical sections will probably be a bit slower (when running a single thread, I agree that performance of concurrent iteration does not matter).
I opened a second PR #148348 with the critical section approach (which is a simple change and similar to how most of the other iterations in this module have been handled). Looking at it now, I think I slightly prefer the critical section approach.
We make
itertools.islicesafe under free-threading (e.g. guard against crashes) by:cntandnextstepfield as a sentinel for iterator exhaustionNote: several other iterators in the
itertoolsmodule have been made safe using a critical section (e.g. #144402). That approach works here as well and would make the approach a bit more uniform and more readable. Downside of a critical section it that is scales less well.