Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ray Casting for Primitive Shapes #15724

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

Jondolf
Copy link
Contributor

@Jondolf Jondolf commented Oct 8, 2024

Objective

Closes #13618.

Add ray casting support for Bevy's primitive 2D and 3D shapes.

The scope here is to support the following:

  • Test if a local ray intersects a given shape.
  • Compute the distance to the closest point of intersection along a local ray.
  • Compute the distance and normal at the point of intersection.
  • World-space versions of the above, taking an isometry to transform the shape.

These should be supported for all of Bevy's primitive shapes, except where unreasonable, such as for 3D lines, which are infinitely thin.

The following are not in scope here:

  • Cast a ray into the world to query all shapes for intersections.
  • Speed up ray queries for a set of shapes using an acceleration structure such as a Bounding Volume Hierarchy (BVH).
  • Get the feature ID of the hit facet (vertex, edge, or face). This could be added later for some polyhedral shapes if desired.

The goal is purely to provide the core tools and implementations for performing efficient ray casts on individual shapes. Abstractions can be built on top by users and third party crates, and eventually Bevy itself once it has first-party colliders and physics.

Solution

Add PrimitiveRayCast2d and PrimitiveRayCast3d traits with the following methods:

impl PrimitiveRayCast2d {
    fn intersects_local_ray(&self, ray: Ray2d) -> bool;
    fn intersects_ray(&self, iso: Isometry2d, ray: Ray2d) -> bool;
    fn local_ray_distance(&self, ray: Ray2d, max_distance: f32, solid: bool) -> Option<f32>;
    fn ray_distance(&self, iso: Isometry2d, ray: Ray2d, max_distance: f32, solid: bool) -> Option<f32>;
    fn local_ray_cast(&self, ray: Ray2d, max_distance: f32, solid: bool) -> Option<RayHit2d>;
    fn ray_cast(&self, iso: Isometry2d, ray: Ray2d, max_distance: f32, solid: bool) -> Option<RayHit2d>;
}
// ...and similarly for `PrimitiveRayCast3d`

where RayHit2d looks like this:

// Note: We could store the point of intersection too,
//       but it is easily computed as `ray.get_point(distance)`.
pub struct RayHit2d {
    pub distance: f32,
    pub normal: Dir2,
}
// ...and similarly for `RayHit3d`

Usage then looks like this:

let ray = Ray3d::new(Vec3::new(-1.0, 0.0, 0.0), Vec3::X);
let sphere = Sphere::new(1.0);
let iso = Isometry3d::from_translation(Vec3::new(1.0, 0.0, 0.0));

let max_distance = f32::MAX;
let solid = true;

if let Some(hit) = sphere.ray_cast(iso, ray, max_distance, solid) {
    assert_eq!(hit.distance, 1.0);
    assert_eq!(hit.normal, Dir3::NEG_X);
    assert_eq!(ray.get_point(hit.distance), Vec3::ZERO);
}

Names are open for bikeshedding. I chose PrimitiveRayCastNd because we already have RayCastNd structs, and these traits are intended to return only the most minimal data required to represent an intersection and its geometry efficiently. Other APIs could be built on top to return more data if desired.

Let's go over a few relevant features and implementation details.

Solid and Hollow Shapes

The ray casting methods (excluding intersection tests) have a solid boolean argument. It controls how rays cast from the interior of a shape behave. If true, the ray cast will terminate with a distance of zero as soon as the ray origin is detected to be inside of the shape. Otherwise, the ray will travel until it hits the boundary.

This feature has somewhat unclear utility. One valid use case is determining how far a shape extends in some given direction, which could be used to figure out e.g. how far away an object picked up by the player should be held. Or maybe you have a circular room, and want to cast rays against its walls from the inside without discretizing the circle to be formed out of multiple separate shapes.

Some hollow shapes can actually be handled in most cases without this built-in support, by simply performing another ray cast in the opposite direction from outside the shape if the ray origin is detected to be inside of the shape. However, for shapes like annuli and tori, the amount by which to offset the ray origin isn't obvious, and doing two ray casts is also more expensive than just having built-in support.

For prior art, Parry has the same boolean argument, and Box2D also supports hollow shapes, although in Box2D's case these are just handled by using chain (i.e. polyline) shapes.

Local and World Space Ray Casts

Each method has a local and world space variant. Practically all ray casts should be performed in local space, since it makes the algorithms significantly more straightforward.

The world-space versions are primarily a user-facing abstraction, which actually just transforms the ray into local space and then transform the results back to world space.

Discussion

Do we want this yet?

Bevy doesn't have built-in physics or collision detection yet. Should we have something like ray casting?

I believe having ray casting support for Bevy's primitive shapes could be amazing for users, even without first-party colliders. So far, the answer to "how do I do ray casts?" has basically been "try to use Parry, or just use Avian or bevy_rapier for nicer ergonomics and to avoid having to deal with Nalgebra". While this PR doesn't add APIs or tools to cast rays into the world like physics engines do, it does provide a foundation on top of which both users and crates could build their own lightweight APIs. We could even extend this to perform ray casts on meshes in the future (and as part of a mesh picking backend).

I don't think we should necessarily build an entire collision detection and geometry library in the background and upstream it all at once, but rather build it out incrementally in logical chunks. I think ray casting is a perfect candidate for this. This could be released in a third party crate too, but I think it's something that could have immediate upstream value to users.

Does it make sense to have this in bevy_math?

We don't have a better place yet. I expect us to add more similar queries (like point queries) over time. I think we should add this in bevy_math for now, and split things out as we reach critical mass for geometry functionality.

Naming

One potentially contentious part is the naming. Should we go for RayCast or Raycast?

If we do a little statistical research on popular physics and game engines for prior art (only ones I found clearly):

  • Box2D uses RayCast
  • Parry (and Avian and Rapier) uses RayCast
  • Jolt uses RayCast
  • Bepu uses RayCast
  • Godot uses RayCast
  • Unity uses Raycast
  • Bullet uses Raycast
  • PhysX uses Raycast

There is no clear consensus, but in my experience, I've seen RayCast a lot more. That may be because I've seen Box2D, Rapier, and Godot more however. Many other people may be more familiar with Raycast because of Unity and its popularity.

Personally, I prefer RayCast especially in the context of other types of casts existing. In my opinion, shape casts look more strange when combined as one word, Shapecast, as opposed to ShapeCast. Bevy will eventually have shape casts as well, and I think we should be consistent in naming there.

It is also worth noting that the usage of "ray cast" vs. "raycast" vs. "ray-cast" is wildly inconsistent in many engines and literature, even if code uses one form consistently. Godot's docs have all three forms on one page, and so does Wikipedia. I would personally prefer if we followed Box2D's example here: code uses RayCast and ShapeCast, and documentation also uses this naming consistently as "ray cast" and "shape cast".

Split into smaller PRs

This is a very large PR with quite a lot of math-heavy code. I could split this up into several smaller PRs if it would help reviewers.

Maybe something like:

  • Add core traits
  • Implement for Circle, Sphere, Ellipse, and Annulus
  • Implement for Arc2d, CircularSector, and CircularSegment
  • Implement for Rectangle and Cuboid
  • Implement for Capsule2d and Capsule3d
  • Implement for Line2d and Segment2d
  • Implement for Triangle2d, Rhombus, polygons, and polylines
  • Implement for Triangle3d
  • Implement for Tetrahedron
  • Implement for Cone, ConicalFrustum, and Cylinder
  • Implement for Torus

That would be 11 PRs 😬

Or, I can just have this one mega PR. I'm fine with whatever reviewers would prefer.

Testing

Every primitive shape that supports ray casting has basic tests for various different cases like outside hits, inside hits, and missed hits. The shared world-space versions of the methods also have basic tests.

There are also new ray_cast_2d and ray_cast_3d examples to showcase ray casting for the various primitive shapes. These function as good hands-on tests. You can see videos of these in the "Showcase" section.

Performance

Below are mean times for 100,000 ray casts, with randomized shape definitions (within specified ranges) and rays. I am using the Parry collision detection library for comparison, as it is currently the most popular and feature-complete option in the Rust ecosystem. Pay attention to whether units are microseconds or milliseconds!

Shapes marked as "-" don't have a built-in ray casting implementation in Parry, and would require using e.g. convex hulls or compound shapes.

2D:

Shape Our implementation Parry
Circle 535.88 μs 585.60 μs
Circular arc 708.05 μs -
Circular sector 3.5025 ms -
Circular segment 3.3423 ms -
Ellipse 487.78 μs -
Annulus 631.67 μs -
Capsule 1.8968 ms 10.729 ms
Rectangle 566.97 μs 1.7967 ms
Rhombus 2.8950 ms -
Line 605.50 μs -
Line segment 721.86 μs 1.7208 ms
Regular polygon 6.1786 ms -
Triangle 3.4512 ms 4.6280 ms

3D:

Shape Our implementation Parry
Sphere 521.23 μs 534.39 μs
Cuboid 496.98 μs 1.1455 ms
Cylinder 1.5770 ms 6.8190 ms
Cone 1.9658 ms 6.4109 ms
Conical frustum 2.1117 ms -
Capsule 721.44 μs 9.8425 ms
Triangle 1.0066 ms 1.7841 ms
Tetrahedron 3.3739 ms -
Torus 506.93 μs -

As you can see, all of our implementations outperform those found in Parry, and we also have many more supported shapes. Of course Parry also has its own shapes that we don't have yet, like triangle meshes, heightfields, and half-spaces.

Why so much faster? For shapes like capsules, cylinders, and cones, Parry actually doesn't have custom analytic solutions. Instead, it uses a form of GJK-based ray casting. This works for arbitrary shapes with support maps, but is less efficient and robust than the analytic solutions I implemented. Sometimes Parry's approach even completely misses ray hits for shapes like capsules because of the algorithm failing to converge with the way it's configured in Parry.

For other shapes like balls and cuboids, it's harder to say, but I did find several simplifications and ways to make CPU go brrr in comparison to Parry, and spent time micro-optimizing common shapes like Triangle3d in particular. It could also just be that Glam is simply faster in some cases here.


Showcase

Below are the new ray_cast_2d and ray_cast_3d examples. Keep in mind that this does not involve mesh picking of any kind: each object here stores a component that stores the underlying primitive shape, and then a system performs ray casts.

ray_cast_2d.mp4
ray_cast_3d.mp4

Acknowledgments

Thank you @Olle-Lukowski for creating initial versions of a lot of the 2D ray casting implementations in bevy_contrib_raycast! They worked wonderfully as a base to build on top of.

@Jondolf Jondolf added C-Feature A new feature, making something new possible A-Math Fundamental domain-agnostic mathematical operations labels Oct 8, 2024
@alice-i-cecile alice-i-cecile added M-Needs-Release-Note Work that should be called out in the blog due to impact S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged A-Picking Pointing at and selecting objects of all sorts labels Oct 8, 2024
@alice-i-cecile alice-i-cecile added this to the 0.16 milestone Oct 8, 2024
@BenjaminBrienen
Copy link
Contributor

Related: #13618

@Jondolf Jondolf mentioned this pull request Oct 10, 2024
7 tasks
@janhohenheim
Copy link
Member

This is my favorite PR for 0.16 :) thanks for implementing this, this is great!

@IQuick143 IQuick143 self-requested a review October 12, 2024 21:01
// Note: We use `f32::EPSILON` to avoid division by zero later for rays parallel to the capsule's axis.
let a = (baba - bard * bard).max(f32::EPSILON);
let b = baba * rdoa - baoa * bard;
let c = baba * oaoa - baoa * baoa - radius_squared * baba;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can skip a multiplication by changing this to:

Suggested change
let c = baba * oaoa - baoa * baoa - radius_squared * baba;
let c = baba * (oaoa - radius_squared) - baoa * baoa;

Not sure how this affects numerical stability though, I assume not much.
Also I wonder if perhaps the paren is not actually good for perf.
Just an idea.


impl PrimitiveRayCast2d for RegularPolygon {
#[inline]
fn local_ray_cast(&self, ray: Ray2d, max_distance: f32, solid: bool) -> Option<RayHit2d> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure we could avoid iterating all sides (at least for incoming rays, not for rays already near the polygon) by raycasting against the bounding (circumscribed) circle and then only checking the segment of the polygon corresponding to the point on the circle that got hit.

For raycasts already inside the bounding circle, then uh, I'm not sure, we can revert to the for loop behaviour.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For rays inside the bounding circle, we could do a forwards and backwards raycast against the circle, and then check these two segments (which one is the real hit depends on whether the ray entered the polygon, or exited the polygon.)

};

// Create the edge
let segment = Segment2d::new(Dir2::new(end - start).unwrap(), start.distance(end));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a place for a new_and_length application.

let end = vertices[i + 1];

// Create the edge
let segment = Segment2d::new(Dir2::new(end - start).unwrap(), start.distance(end));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar here.

Comment on lines +117 to +127
let qab = sign * triple_scalar_product(q, a, b);
let qbc = sign * triple_scalar_product(q, b, c);
let qac = sign * triple_scalar_product(q, a, c);

// ABC
if qab >= 0.0 && qbc >= 0.0 && qac < 0.0 {
return Triangle3d::new(v[0], v[1], v[2]).local_ray_cast(ray, max_distance, solid);
}

let qad = sign * triple_scalar_product(q, a, d);
let qbd = sign * triple_scalar_product(q, b, d);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you've written that glam can optimise the determinant, but I think it may be worth it here to compute q.cross(a) and q.cross(b), and then replace each of these with one .dot(third_vector)

github-merge-queue bot pushed a commit that referenced this pull request Oct 13, 2024
# Objective

Closes #15545.

`bevy_picking` supports UI and sprite picking, but not mesh picking.
Being able to pick meshes would be extremely useful for various games,
tools, and our own examples, as well as scene editors and inspectors.
So, we need a mesh picking backend!

Luckily,
[`bevy_mod_picking`](https://github.com/aevyrie/bevy_mod_picking) (which
`bevy_picking` is based on) by @aevyrie already has a [backend for
it](https://github.com/aevyrie/bevy_mod_picking/blob/74f0c3c0fbc8048632ba46fd8f14e26aaea9c76c/backends/bevy_picking_raycast/src/lib.rs)
using [`bevy_mod_raycast`](https://github.com/aevyrie/bevy_mod_raycast).
As a side product of adding mesh picking, we also get support for
performing ray casts on meshes!

## Solution

Upstream a large chunk of the immediate-mode ray casting functionality
from `bevy_mod_raycast`, and add a mesh picking backend based on
`bevy_mod_picking`. Huge thanks to @aevyrie who did all the hard work on
these incredible crates!

All meshes are pickable by default. Picking can be disabled for
individual entities by adding `PickingBehavior::IGNORE`, like normal.
Or, if you want mesh picking to be entirely opt-in, you can set
`MeshPickingBackendSettings::require_markers` to `true` and add a
`RayCastPickable` component to the desired camera and target entities.

You can also use the new `MeshRayCast` system parameter to cast rays
into the world manually:

```rust
fn ray_cast_system(mut ray_cast: MeshRayCast, foo_query: Query<(), With<Foo>>) {
    let ray = Ray3d::new(Vec3::ZERO, Dir3::X);

    // Only ray cast against entities with the `Foo` component.
    let filter = |entity| foo_query.contains(entity);

    // Never early-exit. Note that you can change behavior per-entity.
    let early_exit_test = |_entity| false;

    // Ignore the visibility of entities. This allows ray casting hidden entities.
    let visibility = RayCastVisibility::Any;

    let settings = RayCastSettings::default()
        .with_filter(&filter)
        .with_early_exit_test(&early_exit_test)
        .with_visibility(visibility);

    // Cast the ray with the settings, returning a list of intersections.
    let hits = ray_cast.cast_ray(ray, &settings);
}
```

This is largely a direct port, but I did make several changes to match
our APIs better, remove things we don't need or that I think are
unnecessary, and do some general improvements to code quality and
documentation.

### Changes Relative to `bevy_mod_raycast` and `bevy_mod_picking`

- Every `Raycast` and "raycast" has been renamed to `RayCast` and "ray
cast" (similar reasoning as the "Naming" section in #15724)
- `Raycast` system param has been renamed to `MeshRayCast` to avoid
naming conflicts and to be explicit that it is not for colliders
- `RaycastBackend` has been renamed to `MeshPickingBackend`
- `RayCastVisibility` variants are now `Any`, `Visible`, and
`VisibleInView` instead of `Ignore`, `MustBeVisible`, and
`MustBeVisibleAndInView`
- `NoBackfaceCulling` has been renamed to `RayCastBackfaces`, to avoid
implying that it affects the rendering of backfaces for meshes (it
doesn't)
- `SimplifiedMesh` and `RayCastBackfaces` live near other ray casting
API types, not in their own 10 LoC module
- All intersection logic and types are in the same `intersections`
module, not split across several modules
- Some intersection types have been renamed to be clearer and more
consistent
	- `IntersectionData` -> `RayMeshHit` 
	- `RayHit` -> `RayTriangleHit`
- General documentation and code quality improvements

### Removed / Not Ported

- Removed unused ray helpers and types, like `PrimitiveIntersection`
- Removed getters on intersection types, and made their properties
public
- There is no `2d` feature, and `Raycast::mesh_query` and
`Raycast::mesh2d_query` have been merged into `MeshRayCast::mesh_query`,
which handles both 2D and 3D
- I assume this existed previously because `Mesh2dHandle` used to be in
`bevy_sprite`. Now both the 2D and 3D mesh are in `bevy_render`.
- There is no `debug` feature or ray debug rendering
- There is no deferred API (`RaycastSource`)
- There is no `CursorRayPlugin` (the picking backend handles this)

### Note for Reviewers

In case it's helpful, the [first
commit](281638e)
here is essentially a one-to-one port. The rest of the commits are
primarily refactoring and cleaning things up in the ways listed earlier,
as well as changes to the module structure.

It may also be useful to compare the original [picking
backend](https://github.com/aevyrie/bevy_mod_picking/blob/74f0c3c0fbc8048632ba46fd8f14e26aaea9c76c/backends/bevy_picking_raycast/src/lib.rs)
and [`bevy_mod_raycast`](https://github.com/aevyrie/bevy_mod_raycast) to
this PR. Feel free to mention if there are any changes that I should
revert or something I should not include in this PR.

## Testing

I tested mesh picking and relevant components in some examples, for both
2D and 3D meshes, and added a new `mesh_picking` example. I also
~~stole~~ ported over the [ray-mesh intersection
benchmark](https://github.com/aevyrie/bevy_mod_raycast/blob/dbc5ef32fe48997a1a7eeec7434d9dd8b829e52e/benches/ray_mesh_intersection.rs)
from `bevy_mod_raycast`.

---

## Showcase

Below is a version of the `2d_shapes` example modified to demonstrate 2D
mesh picking. This is not included in this PR.


https://github.com/user-attachments/assets/7742528c-8630-4c00-bacd-81576ac432bf

And below is the new `mesh_picking` example:


https://github.com/user-attachments/assets/b65c7a5a-fa3a-4c2d-8bbd-e7a2c772986e

There is also a really cool new `mesh_ray_cast` example ported over from
`bevy_mod_raycast`:


https://github.com/user-attachments/assets/3c5eb6c0-bd94-4fb0-bec6-8a85668a06c9

---------

Co-authored-by: Aevyrie <[email protected]>
Co-authored-by: Trent <[email protected]>
Co-authored-by: François Mockers <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Math Fundamental domain-agnostic mathematical operations A-Picking Pointing at and selecting objects of all sorts C-Feature A new feature, making something new possible M-Needs-Release-Note Work that should be called out in the blog due to impact S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Raycasting for primitives.
5 participants