[BUG]: static_pointer_cast where dynamic_pointer_cast is needed

[leakec]

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

3.0.2

Problem description

If a class is using virtual inheritance and its holder type is a shared pointer, then any method using that class fails due to the static_pointer_cast in pybind11/detail/holder_caster_foreign_helpers.h.

This used to work in pybind11 3.0.1, so I believe this is a new bug introduced in version 3.0.2.

Example code tarball for convenience:
example.tar.gz

Reproducible example code

Note, all the files listed below are part of the example.tar.gz attached above. Feel free to download and untar it if you would like to avoid copy-pasting all the files below.

example.h

#pragma once

#include <memory>

class Base: public std::enable_shared_from_this<Base> {
  public:
    Base();
    virtual ~Base();
    static std::shared_ptr<Base> create();
    virtual void method();
};

class Derived: public Base {
  public:
    using Base::Base;
    ~Derived();
    static std::shared_ptr<Derived> create();
    void method() override;
};

class Derived2: public virtual Derived {
  public:
    using Derived::Derived;
    ~Derived2();
    static std::shared_ptr<Derived2> create();
    void method() override;
    void method2(const std::shared_ptr<Derived2>& d2);
};

example.cc

#include "example.h"
#include <iostream>

Base::Base() {}
Base::~Base() {}
void Base::method() {
    std::cout << "Base" << std::endl;
}
std::shared_ptr<Base> Base::create() {
    return std::make_shared<Base>();
}

Derived::~Derived(){}
void Derived::method() {
    std::cout << "Derived" << std::endl;
}
std::shared_ptr<Derived> Derived::create() {
    return std::make_shared<Derived>();
}

Derived2::~Derived2(){}
void Derived2::method() {
    std::cout << "Derived2" << std::endl;
}
std::shared_ptr<Derived2> Derived2::create() {
    return std::make_shared<Derived2>();
}
void Derived2::method2(const std::shared_ptr<Derived2>& d2) {
    d2->method();
}

example_Py.cc

#include <pybind11/pybind11.h>
#include "example.h"
namespace py = pybind11;

PYBIND11_MODULE(example_Py, m) {
    py::class_<Base, std::shared_ptr<Base>>(m, "Base")
        .def(py::init<>(&Base::create))
        .def("method", &Base::method);

    py::class_<Derived, Base, std::shared_ptr<Derived>>(m, "Derived")
        .def(py::init<>(&Derived::create));

    py::class_<Derived2, Derived, std::shared_ptr<Derived2>>(m, "Derived2")
        .def(py::init<>(&Derived2::create))
        .def("method2", &Derived2::method2, py::arg("d2"));
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.25)
project(example)

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

find_package(Python REQUIRED COMPONENTS Interpreter Development)

# Find pybind11 using the path from Python
execute_process(
    COMMAND ${Python_EXECUTABLE} -c "import pybind11; import pathlib; print(pathlib.Path(pybind11.__file__).parent / \"share\" / \"cmake\" / \"pybind11\")"
    OUTPUT_VARIABLE pybind11_DIR
    OUTPUT_STRIP_TRAILING_WHITESPACE
)
find_package(pybind11 3.0 REQUIRED CONFIG)

# Build library
add_library(example example.cc)

# Add python module
pybind11_add_module(example_Py example_Py.cc)
target_link_libraries(example_Py PRIVATE example)

# Install the module
install(TARGETS example_Py DESTINATION .)

test.py

import example_Py

b = example_Py.Base()
b.method()

d = example_Py.Derived()
d.method()

d2 = example_Py.Derived2()
d2.method()
d2.method2(d2)

run

#!/bin/bash

# Clean old build
rm -rf build *.so

# Make and install
cmake -B build -DCMAKE_INSTALL_PREFIX=`pwd` .
cmake --build build -j 10
cmake --install build

# Test
python test.py

If you create the files listed above and run ./run, everything works as expected for pybind11 3.0.1, but fails to compile example_Py.cc for pybind11 3.0.2. The failure to compile happens because of method2 on Derived2. Without this method, everything passes. This is where a static_pointer_cast is used to try to convert a Base to a Derived2, but a dynamic_pointer_cast is needed since virtual inheritance is involved.

Is this a regression? Put the last known working version here if it is.

3.0.1

[Skylion007]

FYI @rwgk

[rwgk]

@oremanj The static_pointer_cast is in pybind11/detail/holder_caster_foreign_helpers.h introduced with #5862. Could you please take a look?

[lfos]

@oremanj - Are you able to take a look at the issue? I believe this may be causing build failures across a variety of Linux distributions, including Arch Linux and Gentoo.

[rwgk]

I'm taking a look, to see if some LLM can take care of this quickly.