Implement scaled fullscreen

[slipher]

Rendering at a resolution lower than the screen resolution can be desirable if your hardware is too feeble to cope with so many pixels. With SDL2, under certain circumstances, you could pick a lower resolution, and have it automatically stretched to cover the whole screen. But with the SDL2 to SDL3 migration, we lost the ability to use true full-screen modes: SDL3 defaults to always using a full-screen window instead of true fullscreen. (True fullscreen is still supposed to be possible, but you have to use a different API which we don't do.) If you select a resolution lower than the display resolution in SDL3, no scaling is done, and the renderable area is anchored to the bottom left, which is just unusable. See Unvanquished/Unvanquished#2870 for some discussion of positioning the renderable area better, regardless of scaling.

That's not to say that the situation was great in SDL2. Only certain video backends and particular resolutions would produce a reasonable result. The resolution list that we show in the UI is ostensibly a list of fullscreen modes, but most of them produced bad results. Even some of the ones with an aspect matching the display still didn't work well. But with some resolutions having a matching aspect you could get a nice scaled result... if you were using Windows or an x11 compositor. If the underlying compositor was Wayland, it never worked regardless of x11 or Wayland protocol. For unsupported compositors or resolutions, a lot of times you just got the bottom-left anchoring, same as SDL3. Windows did some combination of scaling, letterboxing, and bottom-left anchoring. Hilariously, with certain resolutions you could get all 3 at once: some black bars were cleared on the left and right sides, and the output was scaled up somewhat, but it was still smaller than the remaining rectangle; the left side of the viewport was anchored to the left side's black bar.

Idea 1: reimplement full-screen modes

We could use SDL3's APIs to try to configure true fullscreen modes. A supposed advantage of this is improving performance by bypassing layers of compositing on certain systems. But this heavily depends on whether the fullscreen modes actually work. If support is as patchy as it was in SDL2, I consider this a waste of time.

Idea 2: 3D rendering resolution != final resolution

Make a cvar, say r_3dRenderingScale, to configure a scale factor for the 3D rendering FBOs relative to the final resolution. If you have 1920x1080 resolution and set r_3dRenderingScale 0.5 then 3D rendering would be done at 960x540. The cameraEffects shader would scale up the result as it is copied into the final framebuffer. Then 2D rendering would be done at the final resolution. Something cool with this is maybe we could use r_3dRenderingScale greater than 1 as a form of full screen supersampling antialiasing.

Other ideas:

• Do all rendering including 2D at a reduced resolution, then scale it up as the very last step. But it's not appealing to have to copy everything yet again, since the point is to improve performance.
• Try to get the OS's compositor to scale it up for us. Clearly this is a function they all have in some form, since each OS has some sort of "high DPI" upscaling mechanism. But I don't know whether there are APIs to control this.

[slipher]

Before this is implemented, users might be able to use the OS's DPI scaling as a workaround. For example set desktop scaling to 200%; then, for Wayland, run with SDL_VIDEO_WAYLAND_SCALE_TO_DISPLAY=0 (and also SDL_VIDEODRIVER=wayland I guess)

[illwieckz]

For information I don't remember having seen a performance improvement in using SDL2 fullscreen on Xorg, even on anemic hardware. Actually I do remember I have been surprised to see no actual difference on very low-end hardware, that's my only memory. Here by performance I mean framerate, I never measured the actual delay between the render and the presentation (I don't know how to do that).

[illwieckz]

Other ideas:

• Do all rendering including 2D at a reduced resolution
• Try to get the OS's compositor to scale it up for us.

Both would make the UI ugly. Since both would be ugly, I would avoid relying on system-specific features and favor a generic implementation (i.e. in dæmon), and since it would be ugly, that would be an option for some low preset, as also scaling the UI likely only makes sense when VRAM is low.

[illwieckz]

Also I believe that if we implement scaling in engine, offering the ability to actually display a fullscreen game with a fullscreen window not using a native resolution (what we currently do) would only be an option meaningful for low presets (to reduce VRAM usage). As soon as the final framebuffer can be large enough to avoid using a non-native screen resolution, we should better use the native screen resolution for it, even if some or all other framebuffers are smaller for performance purpose.

[illwieckz]

So to me a proper implementation would be that “Idea 2”, what is described about r_3dRenderingScale (I dislike that name), and possibly another option like r_scale2DAsWell (I dislike that name too) for also scaling down the UI for low end hardware. And then a possible r_ActuallyChangeTheScreenResolution (yet another bad name) to get what we currently do, for when you really want to reduce the whole display server resolution.

[slipher]

possibly another option like r_scale2DAsWell (I dislike that name too) for also scaling down the UI for low end hardware.

You said the main motivation for that would be to decrease VRAM usage. But it seems this would actually increase VRAM usage (by requiring an extra buffer), unless we can get the compositor to do it in which case it would just be the same. So probably there's no use for this.

And then a possible r_ActuallyChangeTheScreenResolution (yet another bad name) to get what we currently do, for when you really want to reduce the whole display server resolution.

Are you talking about using a true fullscreen mode as we can sometimes, inconsistently do in SDL2?

[illwieckz]

possibly another option like r_scale2DAsWell (I dislike that name too) for also scaling down the UI for low end hardware.

You said the main motivation for that would be to decrease VRAM usage. But it seems this would actually increase VRAM usage (by requiring an extra buffer), unless we can get the compositor to do it in which case it would just be the same. So probably there's no use for this.

Great, less things to care about then.

And then a possible r_ActuallyChangeTheScreenResolution (yet another bad name) to get what we currently do, for when you really want to reduce the whole display server resolution.

Are you talking about using a true fullscreen mode as we can sometimes, inconsistently do in SDL2?

Yes. At least the behavior from the end-user point of view. For example, when setting 640×480 in fullscreen game the desktop itself would be reduced to that size (meaning alt-tabbing changes screen resolution back, etc.). Even if the game is actually using a fullscreen borderless window, I don't actually care about being a true fullscreen mode bypassing the compositor or not, but I care about the compositor being forced to switch to a lower screen size. Basically, it's like if in Xorg one would do xrandr -s 640×480 before running the game with r_mode -2.

[illwieckz]

That brings another topic which is listing the fullscreen resolutions supported by the display only makes sense for that possible “also reduce the desktop size” option (what we currently do).

Once we get scaling working, the window size can be fully arbitrary, because switching between windowed/fullscreen would never change the desktop resolution.

[slipher]

Are you talking about using a true fullscreen mode as we can sometimes, inconsistently do in SDL2?

Yes. At least the behavior from the end-user point of view. For example, when setting 640×480 in fullscreen game the desktop itself would be reduced to that size (meaning alt-tabbing changes screen resolution back, etc.). Even if the game is actually using a fullscreen borderless window, I don't actually care about being a true fullscreen mode bypassing the compositor or not, but I care about the compositor being forced to switch to a lower screen size. Basically, it's like if in Xorg one would do xrandr -s 640×480 before running the game with r_mode -2.

Oh I see. So probably the behavior with SDL 2 and an x11 compositor is not a truly true full-screen mode, but rather an instance of getting the compositor to do the scaling for us. Windows has been cited as the example of a platform where you can really bypass the compositor with a full-screen mode.

[illwieckz]

I've seen SDL3 requires libxrandr-dev to build the X11 support, so I guess they just do that on X11: displaying a fullscreen borderless windows atop of everything and resizing the desktop with xrandr…

[VReaperV]

We could use SDL3's APIs to try to configure true fullscreen modes. A supposed advantage of this is improving performance by bypassing layers of compositing on certain systems. But this heavily depends on whether the fullscreen modes actually work. If support is as patchy as it was in SDL2, I consider this a waste of time.

If a borderless window covers the whole screen it should by-pass the compositor, although it's not guaranteed.