Sudden death by X_ShmAttach when running over ssh

[martinwguy]

I'm developing a new program on GNU/Linux using SDL2 for graphics, but when I run it over ssh -X the program suddenly dies on startup with exit status 1 saying:

X Error of failed request:  BadRequest (invalid request code or no such operation)
  Major opcode of failed request:  130 (MIT-SHM)
  Minor opcode of failed request:  1 (X_ShmAttach)
  Serial number of failed request:  214
  Current serial number in output stream:  215

It turns out that it's dying inside SDL_GetWindowSurface() the first time that is called, and exiting with status 1, rather than returning a failure code and letting the program fail more gracefully.
At present my program does pixel access by getting the surface's memory and poking into it inside a SDL_Lock/UnlockSurface() pair. I assume that's impossible when the display and program are on different machines and was hoping to use a different SDL primitive in that scenario, although the EFL port of the same program works fine in those conditions.
Either way, it would be nicer if SDL_GetWindowSurface() were to fail rather than bombing out.

[allelomorph]

I do all my development in VMs and containers, and quickly ran into the same issue with X11 trying to use memory sharing (via protocol extension MIT-SHM) over ssh X11 forwarding.

Please see my PR #13747 for an analysis of the problem. The call path in @martinwguy's case is likely SDL_GetWindowSurface > SDL_CreateWindowFramebuffer > X11_CreateWindowFramebuffer > X11_XShmAttach.

After some research, I found the following solutions:

• use xtrace as an intermediary with the --denyextensions option to limit the client to the core X11 protocol
  • advantage: completely non-invasive to SDL or Xlib
  • disadvantage: a bit wasteful to use a protocol logger that is essentially a proxy server just to spoof replies to a few X11 QueryExtension requests
• compile SDL with NO_SHARED_MEMORY defined for preprocessor
  • advantage: non-invasive to SDL source
  • disadvantages:
    • completely removes support for shared memory, even when it is appropriate
    • complicates using SDL as a dependency with FetchContent in a cmake build
• patch SDL source to have runtime detection of remote vs local X server
  • advantage: toggle shared memory use according to use case
  • disadvantage: requires download/patching/building every time SDL is used as a cmake dependency, unless PR is approved 😄