Create and use StableTypeId instead of TypeId

[cart]

std::any::TypeId is not stable across binaries. This makes it unsuitable for use in scene files, networking, or dynamic plugins.

In the short term we get around this by just using std::any::type_name. But this isn't particularly efficient for certain serialization cases and Eq / Hash traits. Legion uses TypeIds extensively and we've (in the short term) replaced those with type_name to allow for dynamic plugin loading. But that probably incurs measurable overhead.

Its worth exploring the idea of a StableTypeId, which is just a wrapped integer. I think it makes sense to use a const-hashing algorithm on type_name to produce StableTypeIds

[cart]

This is even more important now that we're using bevy_ecs instead of legion. Dynamic plugin loading will be broken until we add this.

[Plecra]

How would a hashing algorithm deal with collisions? Obvious question, but I don't see how this issue can be addressed without resolving it 😶

[cart]

If you are discussing hash collisions where "a" and "b" hash to the same thing, i think we can choose one that is effectively "collision free" for the type names we would be using. The world runs on hashing strings.

For the case where there are two types that have the same "type name": I don't think thats a problem in practice. Within a crate there cannot be collisions. Across crates, crates.io enforces uniqueness.

I'd love your thoughts on this!

[Plecra]

I'm concerned about unsafe code making assumptions that any given TypeId must only have one representation, since I can't tell how that'd be avoided.

I think Rust's safety is best thought of as a binary property, and wouldn't like a solution that just made it really unlikely that you manage to write two types with the same hash (which could then proceed to cause UB). Especially as the algorithm would be public, and crates.io is an amazing attack vector.

That said, I could be completely unaware of some way of proving an algorithm "collision free" for type names, or there really is a way of verifying the hashes. (On second thought, can the StableTypeId just contain a pre-hashed &'static str? Yea, probably...) Ahw, same issue with the serialization

[cart]

Back when bevy used a legion fork, i solved the type id problem with &'static str. It definitely worked, but there are two issues:

1. Performance. Comparison (and hashing) is expensive and both are relatively common operations in the ECS implementation
2. We'd still be hashing the string because aspects of hecs rely on a hashed type id. The only difference between &'static str and StableTypeId is where the hash happens (run time vs compile time).

I think the cost/benefit here is quite low. The cost is "very very very unlikely collisions" and the benefit is dynamic plugin loading and no additional runtime cost. On the "malicious code" side, if you are taking a dependency on something, that is already trusted code. We will never be able to protect against malicious dependencies.

[Plecra]

What do you mean by that? There's a lot that can be done to defend from dangerous code, and Rust is an effort to empower developers in that sense.

As far as I'm aware, there shouldn't be any way for a forbid_unsafe, no_std crate to interact with the host machine, or mess with any memory I don't give it. That's incredibly useful for managing dependencies in a Foss context.

A soundness hole allows those otherwise sandboxed dependencies to trigger UB, and get up to whatever they like.

Instead, could this be done with a derive macro? The uniqueness of the IDs could then be verified at compile time, and compilation can fail if a collision is found. I do understand why you don't consider a priority - the chances are incredibly low. It's just a disappointing sacrifice to make (Side note: this flaw would mean that properly reviewing dependencies for safety would require a manual verification step anyway since a human can't know that a name is safe for their project)

[cart]

I actually wasn't thinking about the forbid_unsafe and no_std combo providing "sandboxing" capabilities. I'll need to look in to that (and the safety guarantees the combo provides). As a brief aside, its worth noting that currently Bevy has no goal to be no_std, although I'm not against that either (provided it doesn't impact the user experience).

"Sandboxed dependencies" is a nice feature thats may be worth pursuing, but Bevy's primary goals are not "running untrusted dependencies" or even "memory safety". Usability and productivity will take priority over almost everything. I don't think the concerns you have will be encountered in the "common case" (or even the top 99% of cases) and the cost of resolving them (appears to be) either major runtime overhead or major ergonomics costs.

I do want to resolve these problems, but ideally we find a way to do it that doesn't require high run time costs, macros, or user facing "boilerplate".

For example, a blunt force tool would be to add an opt-in cargo feature that uses TypeIds instead of StableTypeId. The cost there is that you wouldn't be able to use dynamic plugin loading.

Another option is exploring the "compile time dupe detection", but im afraid that breaks down in a world with dynamic dependency loading. "Run time dupe detection" seems reasonable though.

[Plecra]

It'd be a shame to have to use some kind of boilerplate. The ease of use of components at the moment is just brilliant.

And given a decent hashing algorithm, I'd be surprised if this issue ever actually mattered. However much I like the idea, you're probably right that it isn't worth it.

Could this decision be added to the documentation if a better solution isn't found? I think it's only fair to users to make the choice visible, since it does contradict the technical meaning of the safe API. Hopefully it'd also give traction to an RFC for a real StableTypeId 😁

[kazimuth]

fyi, different versions of a single crate can be included in a dependency tree, so even without plugin loading I believe you can have multiple types with the same type_name but different layouts :/ you can't directly depend on multiple versions of the same crate but your dependencies can, which could cause an issue with e.g. third-party plugins.

one solution to speed up comparison + hashing would be to have a global type name interner at run time. it's not going to be written often so it doesn't need to be particularly fancy. it wouldn't speed up serialization, though. it also doesn't solve the uniqueness issue.

you could potentially reduce the chance of conflicts by checking mem::size_of, mem::align_of, etc, and hashing those when you intern types. if you get a mismatch, panic. obviously that won't catch all issues though.

oh, alternative strategy: you could add something to the Properties derive that creates a more unique type name. like

impl Properties for $T {
    fn unique_name() -> &'static str {
         concat!(env!("CARGO_PKG_NAME"), "@", env!("CARGO_PKG_VERSION"), "::", std::module_path!(), "::$T"))
    }
}

or whatever, and then require that all types that are serialized / pass through plugins implement Property. Would just need to make sure that wouldn't slow down incremental compiles.

[mrobakowski]

I agree with @kazimuth. The docs for type_name specifically mention that multiple different types can have the same name and so the name cannot be considered a unique identifier. The unique_name that has been proposed above could include even more info, like which features of the crate were enabled (because afaik you can even include a single crate version multiple times with different features). There's a nice list of available env vars in the cargo book.

[zicklag]

Relevant to this discussion is #142. In order to have dynamically registered components, TypeId would need to become more of a ComponentId and would have to be suitable for components created outside of the Rust ecosystem as well.