Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accented characters don't work with HID keyboard in X11, but do work in Wayland #5683

Open
sunarowicz opened this issue Dec 18, 2024 · 17 comments

Comments

@sunarowicz
Copy link

Hello,

up to now I used scrcpy with simulating the physical HID keyboard for typing accented characters in Wayland without any problem. Recently I had to switch from Wayland to X11 and realized that I cannot type accented characters anymore. I'm running scrcpy with the same parameters as earlier and din't changed anything in my my Ubuntu 24.04 desktop, nor in phone. When I log back into Wayland session it works again, if I log into X11 session it still doesn't.

I realized this issue when I was using scrcpy 2.7. Unfortunately upgrade to 3.1 didn't helped here.

I did two same tests, one in Wayland, second in X11, see logs below:

Wayland log:

scrcpy --tcpip=192.168.1.110 -K --no-audio --window-height=1080  -Vverbose --no-audio --no-mouse-hover
scrcpy 3.1 <https://github.com/Genymobile/scrcpy>
INFO: Connecting to 192.168.1.110:5555...
INFO: Connected to 192.168.1.110:5555
DEBUG: Device serial: 192.168.1.110:5555
DEBUG: Using server: /usr/local/share/scrcpy/scrcpy-server
/usr/local/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 47.8 MB/s (90640 bytes in 0.002s)
[server] INFO: Device: [Xiaomi] POCO M2012K11AG (Android 13)
DEBUG: Server connected
VERBOSE: input: UHID create [1] 0000:0000 name=(null) report_desc_size=63
DEBUG: Starting controller thread
DEBUG: Starting receiver thread
DEBUG: Using icon: /usr/local/share/icons/hicolor/256x256/apps/scrcpy.png
VERBOSE: UHID output [1] 01
[server] DEBUG: Using video encoder: 'OMX.qcom.video.encoder.avc'
[server] DEBUG: Display: using SurfaceControl API
INFO: Renderer: opengl
INFO: OpenGL version: 3.3 (Compatibility Profile) Mesa 23.0.0-devel
INFO: Trilinear filtering enabled
DEBUG: Demuxer 'video': starting thread
INFO: Texture: 1080x2400
VERBOSE: input: touch [id=mouse] down position=697,1955 pressure=1.000000 action_button=000001 buttons=000001
VERBOSE: input: touch [id=mouse] up   position=697,1955 pressure=0.000000 action_button=000001 buttons=000000
VERBOSE: keys[1f] = true
VERBOSE: hid keyboard: key down scancode=1f (31) mod=00
VERBOSE: input: UHID input [1] 00 00 1F 00 00 00 00 00
VERBOSE: keys[1f] = false
VERBOSE: hid keyboard: key up   scancode=1f (31) mod=00
VERBOSE: input: UHID input [1] 00 00 00 00 00 00 00 00

X11 log:

scrcpy --tcpip=192.168.1.110 -K --no-audio --window-height=1080  -Vverbose --no-audio --no-mouse-hover
scrcpy 3.1 <https://github.com/Genymobile/scrcpy>
* daemon not running; starting now at tcp:5037
* daemon started successfully
INFO: Connecting to 192.168.1.110:5555...
INFO: Connected to 192.168.1.110:5555
DEBUG: Device serial: 192.168.1.110:5555
DEBUG: Using server: /usr/local/share/scrcpy/scrcpy-server
/usr/local/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 52.5 MB/s (90640 bytes in 0.002s)
[server] INFO: Device: [Xiaomi] POCO M2012K11AG (Android 13)
DEBUG: Server connected
VERBOSE: input: UHID create [1] 0000:0000 name=(null) report_desc_size=63
DEBUG: Starting controller thread
DEBUG: Starting receiver thread
DEBUG: Using icon: /usr/local/share/icons/hicolor/256x256/apps/scrcpy.png
[server] DEBUG: Using video encoder: 'OMX.qcom.video.encoder.avc'
VERBOSE: UHID output [1] 01 
[server] DEBUG: Display: using SurfaceControl API
INFO: Renderer: opengl
INFO: OpenGL version: 3.3 (Compatibility Profile) Mesa 23.0.0-devel
INFO: Trilinear filtering enabled
DEBUG: Demuxer 'video': starting thread
INFO: Texture: 1080x2400
VERBOSE: input: touch [id=mouse] down position=602,1931 pressure=1.000000 action_button=000001 buttons=000001
VERBOSE: input: touch [id=mouse] up   position=602,1931 pressure=0.000000 action_button=000001 buttons=000000

Note: I typed the same accented character at the end of the X11 test above, but nothing appeared in the log. The same do apply for any other accented characters. No log output for them at all.

@rom1v
Copy link
Collaborator

rom1v commented Dec 18, 2024

On X11, I can correctly inject é with UHID keyboard.

Can you please add these logs:

diff --git app/src/input_manager.c app/src/input_manager.c
index 2e4337db1..52ec3bd86 100644
--- app/src/input_manager.c
+++ app/src/input_manager.c
@@ -587,11 +587,13 @@ sc_input_manager_process_key(struct sc_input_manager *im,
     }
 
     enum sc_keycode keycode = sc_keycode_from_sdl(sdl_keycode);
+    LOGI("=== sc_keycode=%d sdl_keycode=%d", keycode, sdl_keycode);
     if (keycode == SC_KEYCODE_UNKNOWN) {
         return;
     }
 
     enum sc_scancode scancode = sc_scancode_from_sdl(event->keysym.scancode);
+    LOGI("=== sc_scancode=%d sdl_scancode=%d", keycode, sdl_keycode);
     if (scancode == SC_SCANCODE_UNKNOWN) {
         return;
     }

@sunarowicz
Copy link
Author

Thank you for your quick response!

After applying your patch and reinstalling there is no change unfortunately. Still no output in log with enabled UHID keyboard in Plasma/X11.

If I log into Plasma/Wayland session (where it works all right) I get the following (just for confirmation the patch is in place):

INFO: === sc_keycode=55 sdl_keycode=55
INFO: === sc_scancode=55 sdl_scancode=55
VERBOSE: keys[24] = true
VERBOSE: hid keyboard: key down scancode=24 (36) mod=00
VERBOSE: input: UHID input [1] 00 00 24 00 00 00 00 00 
INFO: === sc_keycode=55 sdl_keycode=55
INFO: === sc_scancode=55 sdl_scancode=55
VERBOSE: keys[24] = false
VERBOSE: hid keyboard: key up   scancode=24 (36) mod=00
VERBOSE: input: UHID input [1] 00 00 00 00 00 00 00 00

As I was curious I did more testing in GNOME session too. I discovered that the accented characters don't work both in X11 and Wayland sessions there. Here is the complete overview of all test results I did:

  • Plasma/Wayland: works
  • Plasma/X11: doesn't work & no log output
  • GNOME/Wayland: doesn't work & no log output
  • GNOME/X11: doesn't work & no log output

All tests were done with the same command line:
scrcpy --tcpip=192.168.1.110 -K --no-audio --window-height=1080 -Vverbose --no-audio --no-mouse-hover

@rom1v
Copy link
Collaborator

rom1v commented Dec 18, 2024

It looks like an issue with your SDL then.

Does it happen both with the static binary release (which includes SDL statically) and your own build (using SDL from your system)?

@sunarowicz
Copy link
Author

sunarowicz commented Dec 18, 2024

I'm sorry, I have no idea how to get the static binary. I'm on the arm64 architecture and official Ubuntu repository include the ancient 1.25 version only which I believe does not have much sense to test. If you don!t think so, I can install it and test (if it already has the UHID keyboard functionality).

@rom1v
Copy link
Collaborator

rom1v commented Dec 18, 2024

I have no idea how to get the static binary.

https://github.com/Genymobile/scrcpy/blob/master/doc/linux.md#from-the-official-release

@sunarowicz
Copy link
Author

Sure, I know that, but this is not for my arm64 architecture.:-(

@rom1v
Copy link
Collaborator

rom1v commented Dec 18, 2024

Oh sorry, I just read "64" 😄

@sunarowicz
Copy link
Author

Oh sorry, I just read "64" 😄

:-) No problem at all, happens to me often too...:o)

So, no other idea here? Should I check or test something? I would like to help in sorting this out.

It makes no sense to me that it is OK only in the Plasma/Wayland scenario and in all other Plasma/GNOME/X11/Wayland scenarios it fails...

@rom1v
Copy link
Collaborator

rom1v commented Dec 19, 2024

You can build a static binary from your machine (with dependencies built from sources):

release/build_linux.sh arm64  # it builds natively, the explicit arch name is just for filenames
release/package_client.sh linux-arm64 tar.gz

You should get a static arm64 release in release/output/.

@sunarowicz
Copy link
Author

Thank you for the direction. I followed your path.

I found that I have to build server too because of this line in package_client.sh:
cp "$WORK_DIR/build-server/server/scrcpy-server" "$ARCHIVE_DIR/$TARGET_DIRNAME/"
otherwise package_client.sh fails. But when running build_server.sh I get the following error:

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':server:lintVitalReportRelease'.
> SDK location not found. Define a valid SDK location with an ANDROID_HOME environment variable or by setting the sdk.dir path in your project's local properties file at '/home/tux/bin/scrcpy/scrcpy/local.properties'.

So I need to have Android SDK? If so, please advice me how to get it. Or do I think wrong?

@rom1v
Copy link
Collaborator

rom1v commented Dec 20, 2024

Instead of calling build_server.sh, just copy the prebuilt binary to release/work/build-server/server/scrcpy-server.

@sunarowicz
Copy link
Author

Thank you a lot, this helped. After that package_client.sh passed ok.

Next I replaced /usr/local/bin/scrcpy with the statically built binary from the output tarball and copied adb from the same source next to it into the /usr/local/bin/. Also replaced the /usr/local/share/man/man1/scrcpy.1 with the new built version.

Running scrcpy -v gives me the following:

scrcpy 3.1 <https://github.com/Genymobile/scrcpy>

Dependencies (compiled / linked):
 - SDL: 2.30.10 / 2.30.10
 - libavcodec: 61.19.100 / 61.19.100
 - libavformat: 61.7.100 / 61.7.100
 - libavutil: 59.39.100 / 59.39.100
 - libavdevice: 61.3.100 / 61.3.100
 - libusb: - / 1.0.27

But when I run scrcpy --tcpip=192.168.1.110 -K --no-audio --window-height=1080 I get the following:

scrcpy 3.1 <https://github.com/Genymobile/scrcpy>                                           
/usr/local/bin/adb: 1: cannot create ��▒@xKy@8
                                              @%@@@������▒��▒��▒��▒��▒��I��I��d��d��d��@��wf��f��fx�P       ��d��d��d@H@hjfh�fh�R�td��d��d��d��@�P�tdP�P�P��~�~Q�td���  /lib64/ld-linux-7<FPZ`gl{�����������������",8BRXair{����������������
                                                    !.4: Directory nonexistent
/usr/local/bin/adb: 1: @ERY_djr~��������������▒: not found
/usr/local/bin/adb: 1: jw������������,AI]qy�������������: not found
/usr/local/bin/adb: 1: -3A: not found
/usr/local/bin/adb: 19: Syntax error: "(" unexpected
ERROR: "adb start-server" returned with value 2
ERROR: Could not start adb server
ERROR: Server connection failed

/usr/local/bin/ contains the following (adb and scrcpy come from the built output tarball):

drwxr-xr-x  2 root root     4096 Dec 20 11:25 ./
drwxr-xr-x 10 root root     4096 May 12  2024 ../
-rwxr-xr-x  1 root root  7951672 Dec 20 11:25 adb*
-rwxr-xr-x  1 root root   136720 Oct 10 00:21 rtop*
-rwxr-xr-x  1 root root       42 Oct 10 00:21 rtop-wrapper*
-rwxr-xr-x  1 root root 21127680 Dec 20 11:24 scrcpy*

And one more check:

which scrcpy
/usr/local/bin/scrcpy

Could you please help me with this one more?

@rom1v
Copy link
Collaborator

rom1v commented Dec 20, 2024

What is the result of /usr/local/bin/adb devices?

@sunarowicz
Copy link
Author

/usr/local/bin/adb devices
bash: /usr/local/bin/adb: cannot execute binary file: Exec format error

@rom1v
Copy link
Collaborator

rom1v commented Dec 20, 2024

OK, it's the prebuilt binary from google which is not for your arch.

If you have adb installed on your system, just set ADB=adb:

ADB=adb scrcpy

@sunarowicz
Copy link
Author

Yeah, you are right. Seems like different architecture binary.

Finally got it working either by adding ADB=adb or by symlinking /usr/bin/adb to /usr/local/bin/adb. I had to symlinked /usr/local/share/scrcpy/scrcpy-server to /usr/local/bin/scrcpy-server too, otherwise it complained on its non existence.

Unfortunately this exercise did not helped with the accented characters issue. I do not see any change. It still works the same, i.e. all right in Plasma/Wayland, but not in Plasma/X11. There is still no log output on typing any of those characters even with the modified app/src/input_manager.c in X11.

If you have any other idea what could be checked and tested I will be happy to do that. Otherwise I have to start thinking of returning to Wayland and trying to solve other issues that forced me to go back to X11.

In any case thank you a lot for your support, I appreciate it much!

@rom1v
Copy link
Collaborator

rom1v commented Dec 20, 2024

If you have some programming experience, you can write a simple sample app which listens to SDL key/text events and prints them into the console. (You can probably even ask some LLM like ChatGPT to do that for you)

If you receive events on Wayland but not on X11 in your minimal sample, you should report the issue to SDL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants