[BUG]: destructor crashes with latest merge of #5958

[lgritz]

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

da6e071

Problem description

I haven't been able to track down the exact source of the bug, but my project's CI does one build variant against the top-of-tree pybind11 master, and has just started failing today.

I can confirm that it works for commit e44aae2, but fails for commit da6e071, which corresponds to the last merged PR, #5958.

In debug mode, I'm hitting assertions like:

/github/home/ext/dist/include/pybind11/detail/type_caster_base.h:380: pybind11::detail::values_and_holders::iterator::iterator(pybind11::detail::instance*, const type_vec*): Assertion `!types->empty()' failed.

Even in release modes (but with certain gcc hardening enabled), with gcc14 I'm getting

/usr/include/c++/14/bits/stl_vector.h:1149: constexpr std::vector< <template-parameter-1-1>, <template-parameter-1-2> >::const_reference std::vector< <template-parameter-1-1>, <template-parameter-1-2> >::operator[](size_type) const [with _Tp = pybind11::detail::type_info*; _Alloc = std::allocator<pybind11::detail::type_info*>; const_reference = pybind11::detail::type_info* const&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.
Aborted (core dumped)

These tests all pass as late as yesterday, in the immediately previous pybind11 commit.

Reproducible example code

Is this a regression? Put the last known working version here if it is.

Not a regression

[PhilipDeegan]

I'm also seeing a segfault (in release), but only on the macos github action

Assertion failed: (!types->empty()), function iterator, file type_caster_base.h, line 380.
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] *** Process received signal ***
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] Signal: Abort trap: 6 (6)
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] Signal code:  (0)
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 0] 0   libsystem_platform.dylib            0x00000001934656a4 _sigtramp + 56
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 1] 0   libsystem_pthread.dylib             0x000000019342b848 pthread_kill + 296
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 2] 0   libsystem_c.dylib                   0x00000001933349e4 abort + 124
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 3] 0   libsystem_c.dylib                   0x0000000193333c18 err + 0
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 4] 0   cpp_etc.cpython-312-darwin.so       0x00000001018d8460 _ZN8pybind116detail24deregister_instance_implEPvPNS0_8instanceE.cold.1 + 0
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 5] 0   cpp_etc.cpython-312-darwin.so       0x00000001018b0c8c _ZN8pybind116detail14clear_instanceEP7_object + 652
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 6] 0   cpp_etc.cpython-312-darwin.so       0x00000001018b082c pybind11_object_dealloc + 44
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 7] 0   Python                              0x0000000101f2d048 lru_list_elem_dealloc + 172
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 8] 0   Python                              0x0000000101f2b9a4 lru_cache_tp_clear + 428
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [ 9] 0   Python                              0x0000000101f2b67c lru_cache_dealloc + 80
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [10] 0   Python                              0x0000000101d367ac meth_dealloc + 364
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [11] 0   Python                              0x0000000101cfe45c list_dealloc + 228
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [12] 0   Python                              0x0000000101e7bf6c interpreter_clear + 1392
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [13] 0   Python                              0x0000000101e76da0 finalize_interp_clear + 128
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [14] 0   Python                              0x0000000101e754c4 Py_FinalizeEx + 236
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [15] 0   Python                              0x0000000101eac97c Py_RunMain + 500
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [16] 0   Python                              0x0000000101eace98 pymain_main + 40
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [17] 0   Python                              0x0000000101ead018 Py_BytesMain + 40
[sat12-bq154-b33998cb-56ae-4999-814f-a02f84ba6c9c-DED26C4AEF82:37597] [18] 0   dyld                                0x000000019308ab98 start + 6076

[lgritz]

FWIW, my example above with the assertion from type_baster_base.h was also from MacOS, and the other example I gave with the assertion in stl_vector.h was Linux. So it may be manifesting slightly differently depending on OS, compiler, C++ library (libstdc++ vs libc++).

[lgritz]

Hi, do any pybind11 maintainers have comments about this?

Is it possible that I'm doing something wrong? Any tips? Or should I consider pybind11 top-of-tree to be broken at the moment and stop testing against it until the next release?

[HinTak]

I'd appreciate some tips on
skia-python/skia-python#350 which is related to destructors I think also, but in 3.x (I.e. earlier than this), too.

[HinTak]

The pybind11 3.x segfault we have is perhaps interesting to compare - which I cannot reproduce locally - is maybe gcc vs clang related.

Mac os is clang (by default) and the rest is gcc. Does it help if you use gcc on mac os?

[lgritz]

Updated info:

With gcc14 on Linux, my app is crashing upon pybind11 destruction. But if I turn on STL assertions via

add_compile_definitions (_GLIBCXX_ASSERTIONS)

then I get not just a crash, but an assertion:

/usr/include/c++/14/bits/stl_vector.h:1149: constexpr std::vector< <template-parameter-1-1>, <template-parameter-1-2> >::const_reference std::vector< <template-parameter-1-1>, <template-parameter-1-2> >::operator[](size_type) const [with _Tp = pybind11::detail::type_info*; _Alloc = std::allocator<pybind11::detail::type_info*>; const_reference = pybind11::detail::type_info* const&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.
Aborted (core dumped)

Maybe this helps? Maybe turning on the assertions on your end will catch something in your CI?

[lgritz]

This appears to have been fixed by #5972. Closing.