Skip to content

MRTK3 should expose a method to query if HoloLens 2 eye tracking is calibrated #11570

Closed

Description

@AMollis

This issue has been migrated a new MRTK repository, and the status of this issue will now be tracked at the following location:


Overview

MRTK2 had exposed a way to query if HoloLens eye tracking is calibrated. This is useful, because eye tracking doesn't function on HoloLens 2 if the user hasn't calibrated, and apps may want to change UX based on this lack of calibration.

Tasks

  • Add an API to query if eye tracking is calibrated.
  • Add usage of new API to Eye Gaze sample scene
  • Add manual test for HL2

API Recommendation

Add standalone helper class like EyeCalibrationChecker into Input package. Log a "warning" message if this is used on non-UWP platform.

Other Possibilities

  1. Extend ActionBasedController and create a GazeController, and add bool IsCalibrated() to GazeController
  2. Possibly add bool IsCalibrated() member function to IGazeInteractor
  3. Something else?

Workaround

The following code can be used to query this information:

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

using System;
using UnityEngine;
using UnityEngine.Events;
#if WINDOWS_UWP
using Windows.Perception;
using Windows.Perception.People;
using Windows.Perception.Spatial;
using Windows.UI.Input.Spatial;
#endif

namespace Microsoft.MixedReality.Toolkit.Examples
{
    /// <summary>
    /// Checks whether the user is calibrated and prompts a notification to encourage the user to calibrate.
    /// </summary>
// TODO make sure this component menu is the same as other "input" things
    [AddComponentMenu("Scripts/MRTK/Input/EyeCalibrationChecker")]
    public class EyeCalibrationChecker : MonoBehaviour
    {
        [Tooltip("For testing purposes, you can manually assign whether the user is eye calibrated or not.")]
        [SerializeField]
        private bool editorTestUserIsCalibrated = true;

        public UnityEvent OnEyeCalibrationDetected;
        public UnityEvent OnNoEyeCalibrationDetected;

        private bool? prevCalibrationStatus = null;

        private void Update()
        {
            bool? calibrationStatus;

            if (Application.isEditor)
            {
                calibrationStatus = editorTestUserIsCalibrated;
            }
            else
            {
                calibrationStatus = CheckCalibrationStatus();
            }

            if (calibrationStatus.HasValue)
            {
                if (prevCalibrationStatus != calibrationStatus)
                {
                    if (!calibrationStatus.Value)
                    {
                        OnNoEyeCalibrationDetected.Invoke();
                    }
                    else
                    {
                        OnEyeCalibrationDetected.Invoke();
                    }
                    prevCalibrationStatus = calibrationStatus;
                }
            }
        }

        private bool CheckCalibrationStatus()
        {
#if WINDOWS_UWP

            if (MixedReality.OpenXR.PerceptionInterop.GetSceneCoordinateSystem(Pose.identity) is SpatialCoordinateSystem worldOrigin)
            {
                SpatialPointerPose pointerPose = SpatialPointerPose.TryGetAtTimestamp(worldOrigin, PerceptionTimestampHelper.FromHistoricalTargetTime(DateTimeOffset.Now));
                if (pointerPose != null)
                {
                    EyesPose eyes = pointerPose.Eyes;
                    if (eyes != null)
                    {
                        return eyes.IsCalibrationValid;
                    }
                }
            }
#endif // WINDOWS_UWP

            return true;
        }
    }
}
### Tasks
- [ ] Add an API to query if eye tracking is calibrated.
- [ ] Add usage of new API to Eye Gaze sample scene
- [ ] Add Unit Tests. Since Unit Tests will run on non-HL2s, tests should only validate behavior for non-HL2 devices.

Activity

  1. changed the title [-]MRTK3 should expose a method to query if Windows eye tracking is calibrated[/-] [+]MRTK3 should expose a method to query if HoloLens 2 eye tracking is calibrated[/+] on May 24, 2023
  2. marlenaklein-msft commented on Jul 4, 2023

    @marlenaklein-msft
    Contributor

    Closed with #11664

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions