feat: Update to WebXR #285

mister-ben · 2023-12-08T09:55:13Z

Brings @kevleyski 's changes in #273 into main:

For devices that support WebXR, use that
Add iOS device orientation permission check on entering 360 (note ESLint workaround as this is iOS Safari specfic)
Fall back to original WebVR polyfill if navigator.xr not available after initialising WebXR polyfill
Includes fixes for projections other than equirectangular that needed updates after three.js version bump

Also:
Removes husky again, because the pre-commit lint needs a bump to at least Node 15 to accept some code in dependencies (nullish coalescing assignment), and upgrading that is a whole other package of work.
Prevents Browserstack CI from trying to run on Chromium, as the video playback will fail.

…w Device API is more likely the better thing to do though

WebVR to WebXR

[KJSL] Swap webvr ro webxr polyfill packages

[KJSL] Add WebXR immersive VR support

chore: Swap webvr ro webxr polyfill packages

…olyfill if webxr immersive is not available)

chore: Swap webvr ro webxr polyfill packages (continue to use webvr p…

… operation to play/pause and exit VR/AR session)

chore: Swap webvr for webxr polyfill packages (adding basic controller…

…k on entering 360)

chore: Swap webvr ro webxr polyfill packages

…k on entering 360)

chore: Swap webvr to webxr polyfill packages

Co-authored-by: Gary Katsevman <[email protected]>

Sync

kevleyski · 2023-12-12T18:59:34Z

Great stuff, will give this a proper go later but looks good already, I'll see if anyone wants to help with that too

A bit of context and doco for anyone new

https://docs.google.com/presentation/d/19jz0guudowBpcBlMg2INdD6PX5w9tdJfS54Q8Twvqkc/edit

Credits:

Don't forget to give StreamShark some major love next time seeing them around who sponsored this effort!
https://streamshark.io

Shout out to Nik Lever too for his Udemy course on WebXR which is particularly good - head here https://www.udemy.com/course/learn-webxr/?couponCode=DEC23_SALE

misteroneill

Thanks for porting this over, @mister-ben!

I don't know this project well, so take my review with a grain of salt, but the changes look good to me.

src/plugin.js

amtins · 2023-12-18T17:53:27Z

src/plugin.js

+    let displays = [];
+
    if (window.navigator.getVRDisplays) {
      this.log('is supported, getting vr displays');
-      window.navigator.getVRDisplays().then((displays) => {
-        if (displays.length > 0) {
-          this.log('Displays found', displays);
-          this.vrDisplay = displays[0];
-
-          // Native WebVR Head Mounted Displays (HMDs) like the HTC Vive
-          // also need the cardboard button to enter fully immersive mode
-          // so, we want to add the button if we're not polyfilled.
-          if (!this.vrDisplay.isPolyfilled) {
-            this.log('Real HMD found using VRControls', this.vrDisplay);
-            this.addCardboardButton_();
-
-            // We use VRControls here since we are working with an HMD
-            // and we only want orientation controls.
-            this.controls3d = new VRControls(this.camera);
-          }
-        }

-        if (!this.controls3d) {
-          this.log('no HMD found Using Orbit & Orientation Controls');
-          const options = {
-            camera: this.camera,
-            canvas: this.renderedCanvas,
-            // check if its a half sphere view projection
-            halfView: this.currentProjection_.indexOf('180') === 0,
-            orientation: videojs.browser.IS_IOS || videojs.browser.IS_ANDROID || false
-          };
-
-          if (this.options_.motionControls === false) {
-            options.orientation = false;
-          }
+      window.navigator.getVRDisplays().then((displaysArray) => {
+        displays = displaysArray;
+      });
+    }

-          this.controls3d = new OrbitOrientationContols(options);
-          this.canvasPlayerControls = new CanvasPlayerControls(this.player_, this.renderedCanvas, this.options_);
-        }
+    // Detect WebXR is supported
+    if (window.navigator.xr) {
+      this.log('WebXR is supported');
+      window.navigator.xr.isSessionSupported('immersive-vr').then((supportsImmersiveVR) => {
+        if (supportsImmersiveVR) {
+          // We support WebXR show the enter VRButton
+          this.vrButton = VRButton.createButton(this.renderer);
+          document.body.appendChild(this.vrButton);
+          this.initImmersiveVR();
+          this.initXRPolyfill(displays);
+
+          // key or pointer click toogle playback
+          document.body.addEventListener('keydown', () => {
+            this.togglePlay_();
+          });
+          document.body.addEventListener('click', () => {
+            this.togglePlay_();
+          });

-        this.animationFrameId_ = this.requestAnimationFrame(this.animate_);
+        } else {
+          // fallback to older WebVR if WebXR immersive session is not available
+          this.initVRPolyfill(displays);
+        }
+        window.navigator.xr.setSession = (session) => {
+          this.currentSession = session;
+          this.renderer.xr.setSession(this.currentSession);
+        };
      });
+    } else {
+      // fallback to older WebVR if WebXR Device API is not available
+      this.initVRPolyfill(displays);
+    }


The variable displays is set in a promise, I don't see how this could work but maybe I'm missing something.

This seems to lead to an NPE on the property this.player_.vr().vrDisplay in the following file: https://github.com/mister-ben/videojs-vr/blob/4576704f9c807b632a744607bfed8a084dedf509/src/cardboard-button.js#L42-L49

hmm, and getVRDisplays is deprecated and only implemented on android firefox

amtins · 2023-12-18T19:12:18Z

The progress bar circle is visible

mister-ben · 2023-12-19T15:27:30Z

The progress bar circle is visible

I haven't seen that... which example / which browser was this?

amtins · 2023-12-20T16:25:13Z

I haven't seen that... which example / which browser was this?

@mister-ben I have this behavior with all the examples on Firefox and Chrome on desktop. To reproduce this case, simply :

open any example
open the developer console and select the WebRX tab
start playback, the media should be displayed in stereoscopic mode, if not, click on the Enter VR button
click on exit immersive in the WebRX tab

mister-ben · 2023-12-20T16:36:48Z

I've removed the cardboard button in the control bar for now, and used three.js's VRButton as an overlay button. I don't love the UX, if we want the cardboard button we need to include what's going on inside the VRButton class, it's quite different from what worked before. This seems ok as an interim compromise.

This works well on Android. iOS is a pain going in and out of immersive mode. It works sometimes, other times Safari doesn't understand the screen orientation and messes up the display. When exiting, the canvas is larger than the player. I don't know yet whether the issues are in three.js, the plugin, or Safari. Or a combination.

amtins · 2023-12-20T17:06:30Z

@mister-ben on Android on Firefox, moving the phone to change orientation works well, but on Chrome it doesn't. Also, on VR mode, the back and settings buttons don't respond to touch, as well as when using a google cardboard the toggle play doesn't work.

amtins · 2023-12-20T18:55:59Z

I see that the canvas is created here plugin.js#L783-L788, and then it's deleted somehow when entering immersive mode. When leaving immersive mode, the canvas is added to the player as the last child somehow, and this is the cause of #285 (comment). But I can't figure out how.

WCPetersen · 2024-02-11T18:42:00Z

This seems to break the EAC projections. Can anyone else confirm?

kevleyski · 2024-02-11T20:49:23Z

(do we have some good examples, happy to dive in and check on the headsets I have)On 12 Feb 2024, at 5:42 am, Will Petersen ***@***.***> wrote: This seems to break the EAC projections. Can anyone else confirm? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

WCPetersen · 2024-02-12T20:12:31Z

Yes, there are examples for both EAC and EAC_LR in the examples/samples dirs. They should be listed when you run npm start. Even without the headset, they're broken in the browser. I just want to check if that is happening with everyone or if I have a bad dependency (or something like that).

WCPetersen · 2024-02-19T17:36:44Z

src/plugin.js

+        const obj = (index === 0) ? self.controllers[0] : self.controllers[1];
+
+        if (obj) {
+          if (obj.controller) {


obj.controller and obj.grip are never set.

WCPetersen · 2024-02-19T17:40:28Z

src/plugin.js

+    // Not using the cardboard button in favour of three.js's vrbutton.
+    // if (!this.player_.controlBar.getChild('CardboardButton')) {
+    //   this.player_.controlBar.addChild('CardboardButton', {});
+    // }


If the cardboard button is being removed in favor of the three.js VRButton, shouldn't all references to addCardboardButton_(), the whole cardboard-button.js file, and the forceCardboard option be removed? Why go halfway?

WCPetersen · 2024-02-19T20:29:15Z

src/plugin.js

+    return controllers;
+  }
+
+  createText(message, height) {


Does createText ever get called?

kevleyski · 2024-02-19T20:31:33Z

Yeah controllers were working - I’ll try find some time to check if something changed On 20 Feb 2024, at 7:29 am, Will Petersen ***@***.***> wrote: @WCPetersen commented on this pull request. In src/plugin.js:

+ controller.userData.selectPressed = false;

+ controller.userData.index = i; + this.scene.add(controller); + + controllers.push(controller); + + const grip = this.renderer.xr.getControllerGrip(i); + + grip.add(controllerModelFactory.createControllerModel(grip)); + this.scene.add(grip); + } + + return controllers; + } + + createText(message, height) { Does this ever get called? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

blysik · 2024-07-24T22:02:18Z

As an Apple Vision Pro user, I would love to see this implemented. Any chance it can worked on more soon? Thanks!

jbroberg · 2024-07-24T22:57:10Z

As an Apple Vision Pro user, I would love to see this implemented. Any chance it can worked on more soon? Thanks!

We are actively working on it over here - once it stabilises we can have another go at merging upstream. Any feedback is welcome - right now we are focused on improving/making reliable the entering and exiting of immersive mode on HMDs, which is not working super reliably right now.

mister-ben · 2024-07-25T08:48:26Z

Closing in favour of the other work

Kevin Staunton-Lambert and others added 30 commits December 2, 2022 13:51

Updating node-sass version (to remove python2 dependency)

d742fd1

Husky update

d3fba93

[KJSL] Swap webvr ro webxr polyfill packages

139f797

Example of having both polyfills - rewritting plugin.js to use the ne…

a46b98d

…w Device API is more likely the better thing to do though

[KJSL] Swap webvr ro webxr polyfill packages

dc28b05

Merge pull request videojs#1 from kevleyski/main

f41dc2a

WebVR to WebXR

[KJSL] Swap webvr ro webxr polyfill packages

dbc1073

[KJSL] Swap webvr ro webxr polyfill packages

cd30806

Merge pull request videojs#2 from kevleyski/main

eecdbba

[KJSL] Swap webvr ro webxr polyfill packages

[KJSL] Swap webvr ro webxr polyfill packages

d6821ab

Merge pull request videojs#3 from kevleyski/main

842abed

[KJSL] Swap webvr ro webxr polyfill packages

[KJSL] Swap webvr ro webxr polyfill packages

6de443d

[KJSL] Swap webvr ro webxr polyfill packages

b318d28

[KJSL] Swap webvr ro webxr polyfill packages

6d4ff0a

Merge pull request videojs#4 from kevleyski/main

e02ce1d

[KJSL] Add WebXR immersive VR support

chore: Swap webvr ro webxr polyfill packages

70a1611

Merge pull request videojs#5 from kevleyski/main

e0d1c48

chore: Swap webvr ro webxr polyfill packages

chore: Swap webvr ro webxr polyfill packages (continue to use webvr p…

36e7cdc

…olyfill if webxr immersive is not available)

Merge pull request videojs#6 from kevleyski/main

c8330f7

chore: Swap webvr ro webxr polyfill packages (continue to use webvr p…

chore: Swap webvr ro webxr polyfill packages (adding basic controller…

56ad5f3

… operation to play/pause and exit VR/AR session)

Merge pull request videojs#7 from kevleyski/main

a8d1459

chore: Swap webvr for webxr polyfill packages (adding basic controller…

chore: Swap webvr ro webxr polyfill packages (add iOS permission chec…

856a183

…k on entering 360)

Merge pull request videojs#8 from kevleyski/main

8d88ec5

chore: Swap webvr ro webxr polyfill packages

chore: Swap webvr ro webxr polyfill packages (add iOS permission chec…

7c0c62b

…k on entering 360)

Merge pull request videojs#9 from kevleyski/main

46c7207

chore: Swap webvr to webxr polyfill packages

Update src/big-vr-play-button.js

7bf68a5

Co-authored-by: Gary Katsevman <[email protected]>

Update src/plugin.js

1cc47ef

Co-authored-by: Gary Katsevman <[email protected]>

Merge pull request videojs#1 from MetaCDN/main

13d2b02

Sync

chore: Swap webvr for webxr polyfill packages (PR fix ups)

2603836

chore: Swap webvr for webxr polyfill packages (PR fix ups)

ff858af

mister-ben force-pushed the from-metacdn branch from d319e4d to 4576704 Compare December 12, 2023 17:41

mister-ben marked this pull request as ready for review December 12, 2023 17:46

misteroneill approved these changes Dec 13, 2023

View reviewed changes

amtins reviewed Dec 18, 2023

View reviewed changes

src/plugin.js Outdated Show resolved Hide resolved

amtins reviewed Dec 18, 2023

View reviewed changes

mister-ben added 2 commits December 19, 2023 14:29

RoundedRectangle is not a constructor

7e62cd4

test immersive session support

aaf320a

mister-ben added 2 commits December 20, 2023 17:25

Don't enter immersive automatically

162d3fa

Use three.js's button only

f32085a

mister-ben marked this pull request as draft December 20, 2023 20:53

amtins approved these changes Dec 21, 2023

View reviewed changes

WCPetersen added a commit to WCPetersen/videojs-vr that referenced this pull request Feb 10, 2024

feat: Update to WebXR (videojs#285)

e4a615d

WCPetersen reviewed Feb 19, 2024

View reviewed changes

src/plugin.js

return controllers;

}

createText(message, height) {

Copy link

WCPetersen Feb 19, 2024 •

edited

Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does createText ever get called?

mister-ben closed this Jul 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Update to WebXR #285

feat: Update to WebXR #285

mister-ben commented Dec 8, 2023 •

edited

Loading

kevleyski commented Dec 12, 2023 •

edited

Loading

misteroneill left a comment

amtins Dec 18, 2023

amtins Dec 18, 2023 •

edited

Loading

mister-ben Dec 19, 2023

amtins commented Dec 18, 2023

mister-ben commented Dec 19, 2023

amtins commented Dec 20, 2023 •

edited

Loading

mister-ben commented Dec 20, 2023

amtins commented Dec 20, 2023

amtins commented Dec 20, 2023

WCPetersen commented Feb 11, 2024

kevleyski commented Feb 11, 2024 via email

WCPetersen commented Feb 12, 2024

WCPetersen Feb 19, 2024

WCPetersen Feb 19, 2024

WCPetersen Feb 19, 2024 •

edited

Loading

kevleyski commented Feb 19, 2024 via email

blysik commented Jul 24, 2024

jbroberg commented Jul 24, 2024

mister-ben commented Jul 25, 2024

feat: Update to WebXR #285

feat: Update to WebXR #285

Conversation

mister-ben commented Dec 8, 2023 • edited Loading

kevleyski commented Dec 12, 2023 • edited Loading

misteroneill left a comment

Choose a reason for hiding this comment

amtins Dec 18, 2023

Choose a reason for hiding this comment

amtins Dec 18, 2023 • edited Loading

Choose a reason for hiding this comment

mister-ben Dec 19, 2023

Choose a reason for hiding this comment

amtins commented Dec 18, 2023

mister-ben commented Dec 19, 2023

amtins commented Dec 20, 2023 • edited Loading

mister-ben commented Dec 20, 2023

amtins commented Dec 20, 2023

amtins commented Dec 20, 2023

WCPetersen commented Feb 11, 2024

kevleyski commented Feb 11, 2024 via email

WCPetersen commented Feb 12, 2024

WCPetersen Feb 19, 2024

Choose a reason for hiding this comment

WCPetersen Feb 19, 2024

Choose a reason for hiding this comment

WCPetersen Feb 19, 2024 • edited Loading

Choose a reason for hiding this comment

kevleyski commented Feb 19, 2024 via email

blysik commented Jul 24, 2024

jbroberg commented Jul 24, 2024

mister-ben commented Jul 25, 2024

mister-ben commented Dec 8, 2023 •

edited

Loading

kevleyski commented Dec 12, 2023 •

edited

Loading

amtins Dec 18, 2023 •

edited

Loading

amtins commented Dec 20, 2023 •

edited

Loading

WCPetersen Feb 19, 2024 •

edited

Loading