Adds `IMU` sensor #619

pascal-roth · 2024-07-02T15:15:03Z

Description

Add IMU sensor with cfg class IMUCfg and data class IMUData. Compared to the Isaac Sim implementation of the IMU Sensor, this sensor directly accesses the PhysX view buffers for speed acceleration.

Fixes #440

Type of change

New feature (non-breaking change which adds functionality)

Checklist

I have run the pre-commit checks with ./isaaclab.sh --format
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
I have updated the changelog and the corresponding version in the extension's config/extension.toml file
I have added my name to the CONTRIBUTORS.md or my name already exists there

…re/imu

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py

source/extensions/omni.isaac.lab/test/sensors/check_imu_sensor.py

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu_data.py

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py

source/extensions/omni.isaac.lab/omni/isaac/lab/utils/math.py

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu_cfg.py

Mayankm96 · 2024-07-04T21:54:20Z

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py

+        # obtain the velocities of the sensors
+        lin_vel_w, ang_vel_w = self._view.get_velocities()[env_ids].split([3, 3], dim=-1)
+        # store the velocities
+        # note: we clone here because the obtained tensors are read-only
+        self._data.ang_vel_b[env_ids] = math_utils.quat_rotate_inverse(self._data.quat_w[env_ids], ang_vel_w)
+        self._data.lin_acc_b[env_ids] = math_utils.quat_rotate_inverse(
+            self._data.quat_w[env_ids],
+            (lin_vel_w - self._last_lin_vel_w[env_ids]) / max(self._dt, self.cfg.update_period),
+        )
+        self._last_lin_vel_w[env_ids] = lin_vel_w.clone()


The offset is only used for the pose. It isn't accounted for in calculating the linear and angular velocities/accelerations.

it is included, as with the offset I am updating the world pose and orientation which is later used when transforming the velocities and accelerations from world into the sensor frame

But that isn't the same right? From the robot dynamics notes at ETH:

you are correct, didn't thought that through

Mayankm96

The math is wrong for calculating the linear and angular quantities when an offset is present.

Also need to update the docs: https://github.com/isaac-sim/IsaacLab/blob/main/docs/source/api/lab/omni.isaac.lab.sensors.rst

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/__init__.py

jsmith-bdai

Sorry for late review on this one! Mainly some doc changes, but great work! This will be useful for some users at AI institute too 😄

jsmith-bdai · 2024-07-09T18:50:49Z

source/extensions/omni.isaac.lab/docs/CHANGELOG.rst

@@ -1,6 +1,11 @@
 Changelog
 ---------

+* Added IMU sensor implementation that directly accessess the physx view :class:`omni.isaac.lab.sensors.IMU`. The


Suggested change

* Added IMU sensor implementation that directly accessess the physx view :class:`omni.isaac.lab.sensors.IMU`. The

* Added IMU sensor implementation that directly accesses the physx view :class:`omni.isaac.lab.sensors.IMU`. The

jsmith-bdai · 2024-07-09T18:56:18Z

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py

+class IMU(SensorBase):
+    """The inertia measurement unit sensor.
+
+    The sensor can be attached to any RigidObject in the scene. The sensor provides the linear acceleration and angular


After #644 merges in, this should update to RigidObject or Articulation

Suggested change

The sensor can be attached to any RigidObject in the scene. The sensor provides the linear acceleration and angular

The sensor can be attached to any :class:`RigidObject` or :class:`Articulation` in the scene. The sensor provides the linear acceleration and angular

You might want to reformat this as I made the line much longer

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py

jsmith-bdai · 2024-07-09T18:58:39Z

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py

+        be able to access the data from the sensor. It also initializes the internal buffers to store the data.
+
+        Raises:
+            RuntimeError: If the number of imu prims in the view does not match the number of environments.


Where is this check happening?

jsmith-bdai · 2024-07-09T19:06:21Z

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py

+        # data buffers
+        self._data.pos_w = torch.zeros(self._view.count, 3, device=self._device)
+        self._data.quat_w = torch.zeros(self._view.count, 4, device=self._device)
+        self._data.quat_w[:, 0] = 1.0


🔥 You don't know how many times I've forgotten this and gotten some strange errors 😄

jsmith-bdai · 2024-07-09T19:27:03Z

source/extensions/omni.isaac.lab/omni/isaac/lab/utils/math.py

Can we add unit tests for the new methods? The simplest test is just converting to and from a convention and ensuring the new solution is close enough

jsmith-bdai · 2024-07-09T19:28:22Z

source/extensions/omni.isaac.lab/test/sensors/check_imu_sensor.py

+parser.add_argument(
+    "--terrain_type",
+    type=str,
+    default="generator",


Suggested change

default="generator",

default="generator",

choices=["generator", "usd", "plane"],

jsmith-bdai · 2024-07-09T19:29:36Z

source/extensions/omni.isaac.lab/test/sensors/test_imu.py

+        ),
+    )
+
+    # sensors - frame transformer (filled inside unit test)


Suggested change

# sensors - frame transformer (filled inside unit test)

# sensors - imu (filled inside unit test)

jsmith-bdai · 2024-07-09T19:33:10Z

source/extensions/omni.isaac.lab/test/sensors/test_imu.py

+            # read data from sim
+            self.scene.update(self.sim.get_physics_dt())
+
+            # skip first steo where initial velocity is zero


Suggested change

# skip first steo where initial velocity is zero

# skip first step where initial velocity is zero

jsmith-bdai · 2024-07-09T19:34:00Z

source/extensions/omni.isaac.lab/test/sensors/test_imu.py

Do we expect this to work on CPU and GPU? If so, I would use the pattern in test_rigid_object.py where we iterate over CPU and GPU devices?

pascal-roth added 22 commits December 11, 2023 12:15

add imu sensor outdated version

bd2eb53

update to physix prim view and update init

530ab8a

add check for sensor

2bcf653

run formatter

e11defb

adjust changelog

cbd47fa

change frame of vel and acc, only accept rigid body prim, typo

baf5132

Merge branch 'devel' into feature/imu

50135d3

remove outdated call as made in latest PR for other sensors

6394262

run formatter

4f30a56

Merge branch 'devel' into feature/imu

914a39b

Merge branch 'main' into feature/imu

c5ace7a

start update

ec7da0d

fixes

ef4c544

run formatter

d90a7f1

fix check imu sensor

b88bf40

Merge branch 'main' into feature/imu

3d5bceb

Merge branch 'feature/imu' of github.com:isaac-orbit/orbit into featu…

39594de

…re/imu

run formatter

b966d04

Merge branch 'main' into feature/imu

a040b72

move to new sensor location

45c0add

cleanup changelog, update imu sensor to isaaclab

e168ee4

visualization fix

6cb55fc

pascal-roth requested review from Mayankm96 and jsmith-bdai July 2, 2024 15:15

pascal-roth self-assigned this Jul 2, 2024

Mayankm96 reviewed Jul 2, 2024

View reviewed changes

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py Outdated Show resolved Hide resolved

Mayankm96 reviewed Jul 2, 2024

View reviewed changes

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py Outdated Show resolved Hide resolved

Mayankm96 reviewed Jul 2, 2024

View reviewed changes

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py Outdated Show resolved Hide resolved

Mayankm96 reviewed Jul 2, 2024

View reviewed changes

source/extensions/omni.isaac.lab/test/sensors/check_imu_sensor.py Show resolved Hide resolved

Mayankm96 reviewed Jul 2, 2024

View reviewed changes

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu_data.py Outdated Show resolved Hide resolved

Mayankm96 reviewed Jul 2, 2024

View reviewed changes

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu.py Show resolved Hide resolved

pascal-roth added 3 commits July 3, 2024 09:42

fixes and move utils to math utils

8d2f272

add unittest, move utils to math.utils

c76ca6d

formatter

d17fbeb

pascal-roth requested a review from Mayankm96 July 3, 2024 12:28

Mayankm96 reviewed Jul 4, 2024

View reviewed changes

source/extensions/omni.isaac.lab/omni/isaac/lab/utils/math.py Outdated Show resolved Hide resolved

Mayankm96 reviewed Jul 4, 2024

View reviewed changes

source/extensions/omni.isaac.lab/omni/isaac/lab/utils/math.py Outdated Show resolved Hide resolved

Mayankm96 reviewed Jul 4, 2024

View reviewed changes

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/imu_cfg.py Show resolved Hide resolved

Mayankm96 reviewed Jul 4, 2024

View reviewed changes

Mayankm96 requested changes Jul 4, 2024

View reviewed changes

Mayankm96 reviewed Jul 4, 2024

View reviewed changes

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/imu/__init__.py Outdated Show resolved Hide resolved

pascal-roth requested a review from Mayankm96 July 5, 2024 11:04

pascal-roth added 2 commits July 7, 2024 14:14

wip

10759fa

add doc and formatter

4fdb670

jsmith-bdai approved these changes Jul 9, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds `IMU` sensor #619

Adds `IMU` sensor #619

pascal-roth commented Jul 2, 2024 •

edited by Mayankm96

Loading

Mayankm96 Jul 4, 2024

pascal-roth Jul 5, 2024 •

edited

Loading

Mayankm96 Jul 6, 2024

pascal-roth Jul 7, 2024

Mayankm96 left a comment •

edited

Loading

jsmith-bdai left a comment

jsmith-bdai Jul 9, 2024

jsmith-bdai Jul 9, 2024

jsmith-bdai Jul 9, 2024

jsmith-bdai Jul 9, 2024

jsmith-bdai Jul 9, 2024

jsmith-bdai Jul 9, 2024

jsmith-bdai Jul 9, 2024

jsmith-bdai Jul 9, 2024

jsmith-bdai Jul 9, 2024

	* Added IMU sensor implementation that directly accessess the physx view :class:`omni.isaac.lab.sensors.IMU`. The
	* Added IMU sensor implementation that directly accesses the physx view :class:`omni.isaac.lab.sensors.IMU`. The

	The sensor can be attached to any RigidObject in the scene. The sensor provides the linear acceleration and angular
	The sensor can be attached to any :class:`RigidObject` or :class:`Articulation` in the scene. The sensor provides the linear acceleration and angular

	default="generator",
	default="generator",
	choices=["generator", "usd", "plane"],

	# sensors - frame transformer (filled inside unit test)
	# sensors - imu (filled inside unit test)

	# skip first steo where initial velocity is zero
	# skip first step where initial velocity is zero

Adds IMU sensor #619

Are you sure you want to change the base?

Adds IMU sensor #619

Conversation

pascal-roth commented Jul 2, 2024 • edited by Mayankm96 Loading

Description

Type of change

Checklist

Choose a reason for hiding this comment

pascal-roth Jul 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mayankm96 left a comment • edited Loading

Choose a reason for hiding this comment

jsmith-bdai left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Adds `IMU` sensor #619

Adds `IMU` sensor #619

pascal-roth commented Jul 2, 2024 •

edited by Mayankm96

Loading

pascal-roth Jul 5, 2024 •

edited

Loading

Mayankm96 left a comment •

edited

Loading