VR releasing one side button when both are pressed unpresses both

[moreApi]

Reproduce:

• checkout https://github.com/scenerygraphics/scenery/tree/vr-ui-tools
• start VRControllerExample
• (select a box with left side button)
• press and hold both side buttons to start scaling
• release one, keep the other one pressed
• press and hold both again
• -> scaling does not work

The log messages show that with the first release both buttons are released.

[skalarproduktraum]

The place this probably goes wrong should be

scenery/src/main/kotlin/graphics/scenery/controls/OpenVRHMD.kt

Lines 547 to 589 in 999a4b3

	while(VRSystem_PollNextEvent(event)) {
	val role = when(VRSystem_GetControllerRoleForTrackedDeviceIndex(event.trackedDeviceIndex())) {
	ETrackedControllerRole_TrackedControllerRole_LeftHand -> TrackerRole.LeftHand
	ETrackedControllerRole_TrackedControllerRole_RightHand -> TrackerRole.RightHand
	ETrackedControllerRole_TrackedControllerRole_Invalid -> TrackerRole.Invalid
	else -> TrackerRole.Invalid
	}
	logger.debug("Event ${idToEventType(event.eventType())} for role $role")
	if(event.eventType() == EVREventType_VREvent_ButtonPress) {
	val button = event.data().controller().button()
	OpenVRButton.values().find { it.internalId == button }?.let {
	logger.debug("Button pressed: $it on $role")
	val keyEvent = it.toKeyEvent(role)
	GlobalKeyEventDispatcher.getInstance().dispatchKeyEvent(keyEvent.first)
	inputHandler.keyPressed(keyEvent.first)
	keysDown.add(it to role)
	GlobalKeyEventDispatcher.getInstance().dispatchKeyEvent(keyEvent.second)
	}
	}
	if(event.eventType() == EVREventType_VREvent_ButtonUnpress) {
	val button = event.data().controller().button()
	OpenVRButton.values().find { it.internalId == button }?.let {
	logger.debug("Button unpressed: $it on $role")
	val keyEvent = it.toKeyEvent(role)
	GlobalKeyEventDispatcher.getInstance().dispatchKeyEvent(keyEvent.second)
	inputHandler.keyReleased(keyEvent.second)
	keysDown.remove(it to role)
	}
	}
	if(event.eventType() == EVREventType_VREvent_MouseMove) {
	val x = event.data().mouse().x()
	val y = event.data().mouse().y()
	val down = event.data().mouse().button()
	logger.debug("Touchpad moved $x $y, down=$down")
	}
	}

[moreApi]

Same behavior for button Side,Trigger and A and probably all others

[skalarproduktraum]

Okay, so this is probably not an oversight, but an actual semantic error -.-

[moreApi]

line 566 of the excerpt above is the cause of pressed keys being empty.
But withouyt that line no VR button works anymore.
At least for drag behaviors that is because https://github.com/scijava/ui-behaviour/blob/9cbc0df842c240e36af48b48fc294ea8c982a63d/src/main/java/org/scijava/ui/behaviour/MouseAndKeyHandler.java#L440 only does exact matching and since there is always this axis or proximity thing pressed it always returns false.
There is something about "irgnored keys" in that match function. But this whole thing looks like highe magic to me.

[skalarproduktraum]

Input handling is one of the darkest kinds of magic.

[skalarproduktraum]

Jokes aside - I'd argue here it'd likely be best to allow only the actual buttons to trigger events, and and ignore the proximity sensor. What do you think, @moreApi?

[moreApi]

I dont think we need to care about the proximity sensor 👍