Tracking quality depends on camera position in the scene

[JensOstertag]

Problem Description

I am currently developing an application with a "large" scene (approx. 1000x1000 units). The XR Origin is placed at about the center of the scene, so at X 500 / Y 500.
I have noticed that in my scene, the tracking quality is terrible, the hands were teleporting a few centimeters every few seconds, but in the sample scenes, the tracking works flawlessly. You can see a video of the problem below.

Steps to reproduce

I built my own testing scene - an empty Unity scene, with an XR Origin, the XR Leap Provider Manager, the Physical Hands Manager, and the left and right hands, and the Physical Hands Manager is the Leap Provider for both hands.

When starting the application, the XR Origin is at position (0, 0, 0), everything behaves as expected. Then, while the application is still running, it is moved to (500, 0, 500), and I notice the same behaviour that I get in my "real" scene. Moving the camera even further away results in even worse tracking quality.

2025-01-16.16-55-44.mp4

(The hand's texture isn't loaded because I work in an URP project.)

Setup desciption

• Unity Version: 6000.0.32f1
• Unity Ultraleap Plugin Version: 7.1.0
• Ultraleap Tracking Service Version: 6.0.0
• Ultraleap Control Panel Version: 3.4.1
• Operating system: Windows 11 Home

Additional information

After changing a few parameters, I also found out this:

• It doesn't matter if the XR Origin is starting at (500, 0, 500) and is then moved to (0, 0, 0), tracking is always broken when the camera is far away from (0, 0, 0).
• The Physical Hands Manager isn't even required for this problem to occur. When the arm's Leap Provider is set directly to the XR Leap Provider Manager, tracking quality gets equally as bad if the hand is just moved further away from (0, 0, 0).

[MaxPalmer-UH]

Interesting, we'll take a look.

[MattFiler]

Hey @JensOstertag, I've run through a variety of setups in Unity 2022.3.44f1 and Unity 6000.0.32f1 using plugin v7.1.0, including:

• XROrigin or CameraOffset with LeapProviderManager using OpenXR (at origin and 500,0,500 and beyond)
• XROrigin or CameraOffset with LeapProviderManager using LeapC (at origin and 500,0,500 and beyond)

and I've not been able to replicate this issue.

Would it be possible to send us a project where this issue is occuring so that we can check against any other packages you have included that may be causing the issue?

[JensOstertag]

Hey, sure, no problem, I've created this repository right here: https://github.com/JensOstertag-Archive/unity-ultraleap-tracking
I've copied my project and then reduced the contents as much as possible (deleted assets and packages which aren't necessary), so now within Assets/Scenes/SampleScene.unity, there is an empty scene as shown in the above video, and the problem still occurs.
I'd really appreciate if you could invesitage this issue further, thank you in advance! Please let me know if you need further information.

Edit: On the new-project-setup branch of the linked repository, I've started from the beginning, creating a new project and setting it up according to https://developer.vive.com/resources/openxr/unity/tutorials/setup-and-installation/getting-started-with-openxr/ and https://docs.ultraleap.com/xr-and-tabletop/xr/unity/getting-started/index.html. At (500, 0, 500) I've noticed that it was way less laggy than previously at this position, but when moving the camera further away, let's say to (1500, 0, 1500) or so, the hands are still "lagging"....

[MattFiler]

Unfortunately I'm still not seeing the issue - the only thing I can think is that perhaps its being caused by the camera transforms handled in the Vive SDK, as I'm using a Quest 3 here. Have you tried just using the stock Unity OpenXR loader rather than the Vive specifc one, or trying on another headset?

[JensOstertag]

Thanks for the tip, but today I was able to test around a bit more, but I couldn't resolve the issues.

First, I've set up a new project with the stock Unity OpenXR loader, as you've recommended, and also attempted using a Quest headet, but the "lagging" remained.

Then, to exclude that any VR setup could cause the problems, I've set up a new "Universal 3D" project and added the Ultraleap package registry, download the "Ultraleap Tracking" and "Ultraleap Tracking Preview" packages from the package manager (as described in the documentation), and then I've set up a scene with the following content:

There are no VR-related changes in this scene or the whole project, compared to a stock Unity project setup. The camera is located within another gameobject, because the Service Providers add the TrackedPoseDriver to camera by default, which then didn't allow me to move the camera around. So when I moved this parent gameobject, the problem still occurred. I've also disabled the option to auto-create the TrackedPoseDriver and then moved the camera directly, which also changed nothing.

I've added this setup here: https://github.com/JensOstertag-Archive/unity-ultraleap-tracking/tree/non-vr-project

Finally, I did the same thing again for HDRP, which expectedly also did not change anything about the hand tracking problem.

[MattFiler]

Interesting, @MaxPalmer-UH are you able to see if you can recreate the issue using the linked project?

Out of interest @JensOstertag what are your specs? From your original vid it seemed you had a stable framerate so I'm assuming it's not a hardware problem, but just for the sake of completeness.

[JensOstertag]

My PC has an i7-11700K, 32GB RAM and an RTX 4060 Ti and used the Vive Pro as well as Quest 2 headset