Add mesh picking backend and MeshRayCast system parameter

[Jondolf]

Objective

Closes #15545.

bevy_picking supports UI and sprite picking, but not mesh picking. Being able to pick meshes would be extremely useful for various games, tools, and our own examples, as well as scene editors and inspectors. So, we need a mesh picking backend!

Luckily, bevy_mod_picking (which bevy_picking is based on) by @aevyrie already has a backend for it using bevy_mod_raycast. As a side product of adding mesh picking, we also get support for performing ray casts on meshes!

Solution

Upstream a large chunk of the immediate-mode ray casting functionality from bevy_mod_raycast, and add a mesh picking backend based on bevy_mod_picking. Huge thanks to @aevyrie who did all the hard work on these incredible crates!

All meshes are pickable by default. Picking can be disabled for individual entities by adding PickingBehavior::IGNORE, like normal. Or, if you want mesh picking to be entirely opt-in, you can set MeshPickingBackendSettings::require_markers to true and add a RayCastPickable component to the desired camera and target entities.

You can also use the new MeshRayCast system parameter to cast rays into the world manually:

fn ray_cast_system(mut ray_cast: MeshRayCast, foo_query: Query<(), With<Foo>>) {
    let ray = Ray3d::new(Vec3::ZERO, Dir3::X);

    // Only ray cast against entities with the `Foo` component.
    let filter = |entity| foo_query.contains(entity);

    // Never early-exit. Note that you can change behavior per-entity.
    let early_exit_test = |_entity| false;

    // Ignore the visibility of entities. This allows ray casting hidden entities.
    let visibility = RayCastVisibility::Any;

    let settings = RayCastSettings::default()
        .with_filter(&filter)
        .with_early_exit_test(&early_exit_test)
        .with_visibility(visibility);

    // Cast the ray with the settings, returning a list of intersections.
    let hits = ray_cast.cast_ray(ray, &settings);
}

This is largely a direct port, but I did make several changes to match our APIs better, remove things we don't need or that I think are unnecessary, and do some general improvements to code quality and documentation.

Changes Relative to bevy_mod_raycast and bevy_mod_picking

• Every Raycast and "raycast" has been renamed to RayCast and "ray cast" (similar reasoning as the "Naming" section in Ray Casting for Primitive Shapes #15724)
• Raycast system param has been renamed to MeshRayCast to avoid naming conflicts and to be explicit that it is not for colliders
• RaycastBackend has been renamed to MeshPickingBackend
• RayCastVisibility variants are now Any, Visible, and VisibleInView instead of Ignore, MustBeVisible, and MustBeVisibleAndInView
• NoBackfaceCulling has been renamed to RayCastBackfaces, to avoid implying that it affects the rendering of backfaces for meshes (it doesn't)
• SimplifiedMesh and RayCastBackfaces live near other ray casting API types, not in their own 10 LoC module
• All intersection logic and types are in the same intersections module, not split across several modules
• Some intersection types have been renamed to be clearer and more consistent
  • IntersectionData -> RayMeshHit
  • RayHit -> RayTriangleHit
• General documentation and code quality improvements

Removed / Not Ported

• Removed unused ray helpers and types, like PrimitiveIntersection
• Removed getters on intersection types, and made their properties public
• There is no 2d feature, and Raycast::mesh_query and Raycast::mesh2d_query have been merged into MeshRayCast::mesh_query, which handles both 2D and 3D
  • I assume this existed previously because Mesh2dHandle used to be in bevy_sprite. Now both the 2D and 3D mesh are in bevy_render.
• There is no debug feature or ray debug rendering
• There is no deferred API (RaycastSource)
• There is no CursorRayPlugin (the picking backend handles this)

Note for Reviewers

In case it's helpful, the first commit here is essentially a one-to-one port. The rest of the commits are primarily refactoring and cleaning things up in the ways listed earlier, as well as changes to the module structure.

It may also be useful to compare the original picking backend and bevy_mod_raycast to this PR. Feel free to mention if there are any changes that I should revert or something I should not include in this PR.

Testing

I tested mesh picking and relevant components in some examples, for both 2D and 3D meshes, and added a new mesh_picking example. I also stole ported over the ray-mesh intersection benchmark from bevy_mod_raycast.

---

Showcase

Below is a version of the 2d_shapes example modified to demonstrate 2D mesh picking. This is not included in this PR.

2024-10-09.23-17-54.mp4

And below is the new mesh_picking example:

2024-10-09.23-56-20.mp4

There is also a really cool new mesh_ray_cast example ported over from bevy_mod_raycast:

2024-10-10.03-41-38.mp4

[aevyrie]

This is a bit of a drive-by review, but the summary of changes look great. Lots of things I haven't had the energy to clean up. :)

Thanks for your effort!

[aevyrie]

It might be nice to include one of the raycasting examples, so people have a quick way to get familiar with the system param and raycast settings. I'm partial to thereflecting_laser one, because it looks cool, and shows how you can use the result of one raycast to create another one.

[alice-i-cecile]

Release note candidate right there :D Seems useful too!

[Jondolf]

Agreed, I'll add that! I've always though that example looks super cool :)

(And I'll fix CI...)

[tbillington]

Awesome work :) just a few opportunities to harden the engine against badly formed user data, and perf stuff.

[tbillington]

there is also sort_unstable_by_key, but the docs say better performance isn't always the case 🤷 https://doc.rust-lang.org/std/vec/struct.Vec.html#method.sort_unstable_by_key

probably not worth worry about

[bushrat011899]

I'm not familiar with the original bevy_mod_picking, but this looks good to me. I do have some comments, but I don't think they should block this from being merged.

[NthTensor]

Reviewing this today. Have we benchmarked the changes from the "Optimize ray-AABB intersection" commit? They look reasonable but I'd like to know how much of an improvement they are before merging.

[Jondolf]

@NthTensor Hard to get accurate traces on the overall effect, but with a simple benchmark for that function alone with 100,000 slightly randomized AABBs, I'm getting roughly a 15% improvement.

(note that this bench currently includes creating the AABBs, but that's the same before and after)

I did also briefly profile picking in the Caldera test scene with both the initial (roughly) one-to-one port, and the current version. Couldn't get entirely reliable results, but perf seemed to be basically identical. I think I might've slightly regressed some things (maybe removing 2d feature and handling both 2D and 3D by default? hard to say) while improving perf elsewhere, which roughly cancels out.

Either way, I don't think performance is really a big concern here. In the Caldera scene, update_hits for the mesh picking backend was just 0.34% of total frame time in my test.

[NthTensor]

Looks great, that pretty much satisfies me. As a policy, I don't like merging things with the word "optimize" without some discussion of performance.