Gesture button is 'stolen' on Wayland

[pallaswept]

I use right button as my gesture button (although I do dream of one day selecting the gesture button per-command, rather than globally, but that's a story for another time), and I have noticed that while mouse-actions is working, I cannot right-click (such as to open a context menu)

It turns out that the click is detected, determined too small to be a gesture, but not propagated as a normal click. Trace logs:

load start:

Start grab
root_loop
[2023-06-28T07:44:09Z TRACE rdev::linux::grab] setup_devices

left-click:

[2023-06-28T07:44:13Z TRACE mouse_actions::process_event] event=ClickEvent { button: Left, edges: [], modifiers: [], event_type: Press, shapes_angles: [], shapes_xy: [] }
[2023-06-28T07:44:13Z TRACE mouse_actions::process_event] candidates=[]
[2023-06-28T07:44:13Z TRACE mouse_actions::process_event] propagate = true
[2023-06-28T07:44:13Z TRACE mouse_actions::grab] normalized_points = PointHistory([])
[2023-06-28T07:44:13Z TRACE mouse_actions::process_event] event=ClickEvent { button: Left, edges: [], modifiers: [], event_type: Release, shapes_angles: [[]], shapes_xy: [PointHistory([])] }
[2023-06-28T07:44:13Z TRACE mouse_actions::process_event] candidates=[]
[2023-06-28T07:44:13Z TRACE mouse_actions::process_event] propagate = true

right-click

[2023-06-28T07:44:40Z TRACE mouse_actions::process_event] event=ClickEvent { button: Right, edges: [], modifiers: [], event_type: Press, shapes_angles: [], shapes_xy: [] }
[2023-06-28T07:44:40Z TRACE mouse_actions::process_event] candidates=[]
[2023-06-28T07:44:40Z TRACE mouse_actions::process_event] propagate = true
[2023-06-28T07:44:40Z TRACE mouse_actions::grab] normalized_points = PointHistory([])
[2023-06-28T07:44:40Z TRACE mouse_actions::process_event] event=ClickEvent { button: Right, edges: [], modifiers: [], event_type: Release, shapes_angles: [[]], shapes_xy: [PointHistory([Point { x: 10, y: 10 }])] }
[2023-06-28T07:44:40Z TRACE mouse_actions::process_event] candidates=[Binding {...long list of points from my bindings removed for clarity...]
[2023-06-28T07:44:40Z DEBUG mouse_actions::process_event] ----------------------------------------
[2023-06-28T07:44:40Z TRACE mouse_actions::process_event] shape size(0) <= 8 → ignore this event
[2023-06-28T07:44:40Z TRACE mouse_actions::process_event] simulate
[2023-06-28T07:44:40Z TRACE mouse_actions::process_event] propagate = false
[2023-06-28T07:44:40Z DEBUG mouse_actions::process_event] Process event duration : 10.885132ms
^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C

Note that second propagate = false, I guess this is the culprit. It appears that neither the mousedown nor mouseup event are forwarded past mouse-actions in this case. I've tested it with middle button and the behaviour is the same.

Note also the multiple ^C - I did not type those, I pressed Ctrl+C one time. I am about to log a separate issue for that.

[jersou]

I don't reproduce this bug !
propagate = false is normal, note simulate, it simulate a mouse click :

            let rdev_btn = config.shape_button.to_rdev_event();
            trace!("simulate");
            simulate(&EventType::ButtonPress(rdev_btn))
                .map_err(|err| error!("simulate err: {:?}", err))
                .ok();
            thread::sleep(time::Duration::from_millis(10));
            simulate(&EventType::ButtonRelease(rdev_btn))
                .map_err(|err| error!("simulate err: {:?}", err))
                .ok();

10ms between ButtonPress and ButtonRelease is maybe too short.
I am under X11, maybe it's a Wayland bug only ?

[pallaswept]

maybe it's a Wayland bug only ?

I think you're right. I logged into an X11 session and it worked OK.

[pallaswept]

I used sudo libinput debug-events to trace it and when the simulate event occurs, no input is detected. Obviously, it works OK under X11 but here on Wayland, no dice.

[pallaswept]

I had another few minutes to look at this, and it's no surprise now that it isn't working under Wayland. While rdev uses evdev to monitor input events, making it cross-compatible with X11 and Wayland, that simulate function uses the XTest extension to generate the output events, making it X11-only.

The good news is that the same crate that rdev uses to read input, can generate output, via evdev. There's an example of it here From a glance, it appears to essentially create a virtual mouse and use that to send the input events.

The bad news is, that probably means writing a replacement/alternative for simulate(). I know that you have made several improvements to upstream rdev, so I guess this would be another one, but I wonder if you plan to upstream your changes or just continue to run with the local fork?

I guess that another way around this would be making ydotool a dependency on Wayland, and calling that to generate the simulated events, but that's not so clean as using an inbuilt lib. mouse-actions is really fast and I think that it would be nice to keep it streamlined.

It's obvious that you're not usually using Wayland, and I am, so I could try to take a shot at this if you like? I will warn you that I have zero experience with Rust (outside of this project which I've been glued to since day one) , although having had a lot of experience with C (mostly embedded devices for robotics automation, but also some Linux stuff thanks to the Y2K bug) and a little with C++ (which I hate passionately), I find it fairly comfortable to adjust to Rust, at least reading the code is very clear to me, but I can't guarantee I'll be of any use when it comes to writing it. But I am offering to try, if it will be helpful for you. Or perhaps I should just get out of your way ;) and let you take care of this. Let me know.

[pallaswept]

I guess that another way around this would be making ydotool a dependency on Wayland, and calling that to generate the simulated events, but that's not so clean as using an inbuilt lib. mouse-actions is really fast and I think that it would be nice to keep it streamlined.

I tried this in a quick-and-drty way, and it doesn't work. Because mouse-actions can't tell that the event came from ydotool, it treats the simulated rightclick just like a real one, and goes into a loop of clicking, simulating a click, which makes it simulate a click, which makes it simulate a click.....

So, to be able to simulate mouse events, we would have to use evdev and filter out events which are generated by mouse-actions' virtual device.

[jersou]

I'll have to reread the above comments carefully, but a workaround (in the meantime) you may be suggesting is to add a binding to for the rigth click release event to ydotool click 0xC1 :

I check the code of rdev::simulate feature : it seems to be made for X11 only...

edit: ydotool use uinput, the crate https://docs.rs/crate/mouse-keyboard-input/0.3.7 could be used to emulate mouse button under Wayland
I re read your comment, and I'll try on Wayland, the workarroud works with xdotool click 3 on X11, but I suppose that xdotool doesn't use uinput...

[pallaswept]

I'll try on Wayland, the workarroud

Be careful my friend, it enters a very fast loop, be ready to kill mouse-actions before it crashes your compositor :)

Edit (I wanted to post that reply quickly in case maybe I can save you from a bad experience ;) )

the crate https://docs.rs/crate/mouse-keyboard-input/0.3.7 could be used to emulate mouse button under Wayland

This looks interesting, I was looking at the evdev-rs crate (which rdev uses also) but this would be more simple. The tricky part is that mouse-actions will need to be able to filter out any input from the virtual device, to avoid this infinite-loop scenario I had above, I'm still reading rdev's code to see if it can do this natively. I'm very new to rust so I'm slow, hopefully you will look at it and see an easy solution!

[jersou]

I've come to the same conclusions as you, even though I didn't test the "trap" as I was informed :-)... Thank you !

I suppose we can modify rdev (which is forked in this repo) to ignore the virtual device created by mouse-actions.
In the future, I'll have to see how to propose a PR for the changes made to this fork.

edit: in relation to this, there is work to be done on rdev and its device management, one of the biggest issues listed in the Readme : "TODO/Hight/fix rdev".

[jersou]

I test a quickfix POC : use a virtual device, click with it and ignore the next click :

It works under X11, I don't test on Wayland yet.

[jersou]

Some news:
I can only work on MA sporadically, but I'm progressing a little anyway: I'm on the main bug (in my opinion), the crash of MA when there is a peripheral addition. I have a fix for these crash that I need to clean up: adding a device no longer crashes MA or freezes the cursor, and the new device is correctly captured if it's a mouse.

For this fix, the management of monitored devices is modified (in the rdev fork), it will then be possible to ignore the future virtual device that will be used to send the clicks currently "stolen" under Wayland.

So, I hope to quickly fix the 3 main bugs with this: MA crash, cursor freeze and the stolen click.

[pallaswept]

Thanks for the update and keeping me in the loop mate. I actually dropped by after my 500th "**** I miss my mouse gestures" moment today so even if there's not a fix here it's really nice to know it's on your to-do list and very thoughtful of you to think of me and let me know. And I think you've got your priorities in order. I have not experienced the problem with adding devices (I do it all the time I have 3 mice) but if that's a thing for some people it's certainly higher priority than getting things working for wayland.

[mingti]

Is there any new progress? This may be the only mouse gesture solution on plasma6 at present.