Skip to content

Conversation

@X-Ryl669
Copy link

This adds 2 features to mufbset:

Add screenshot feature with saving to PNG

This allow to capture a screenshot (when run like this: mufbset -s dump.png) to a file, by compressing the framebuffer's active area to a PNG file.

Add overlay feature

This allow to display an overlay picture on top of the currently drawn framebuffer. This is useful for display, for example, the volume bar when a core is running (this part will be a PR in internal repository), or a 5mn left message.
It's possible to specify where to overlay the picture, and for how long and if the drawing process should be paused or not.

Currently, overlaying while the drawing process isn't paused is slow because I'm doing it 60 times per second not to miss any frame. I think it's better to compute a fingerprint of the current framebuffer content (for the overlaid area only) and only redraw the overlay if it's changed. I'm currently doing this and will update this PR if it proves better in terms of performance.

@xonglebongle
Copy link
Member

The screenshot feature is a nice addition to mufbset however I feel that the overlay system is not working as I had imagined it to be. If you load up any fast moving content you get a number of glitches as you are copying the frame buffer memory and replacing it. This is not due to yours or anyone's fault, it is a limitation.

@X-Ryl669
Copy link
Author

X-Ryl669 commented May 23, 2025

Yes, unless there's a new framebuffer driver for sunxi that supports layers, it's not physically possible to overlay over a live process without some sort of hacking/glitching.

I've tried many alternatives with more or less success:

  1. Ranging from dumbly alpha blending as fast as possible, it works, but the overlay is 50% transparent as expected (due to our vision persistence)
  2. Detecting framebuffer corruption of what we've written (with a single pixel or/and a complete area CRC) and only drawing if it's changed. This doesn't work because some process hacks the framebuffer vertical offset to do double buffering (like LVGL at some very large period) or retroarch (at 30fps). So you can't detect the frame is modified because the other process is drawing on the other buffer (and you don't want to write to both buffer since you'll be corrupting the only "safe" view)
  3. Sending SIGSTOP/SIGCONT continuously to avoid 2 writer at the same time. This doesn't work since the kernel is very slow to resume paused process, can't be done in realtime (only got ~4fps in that case... disgusting)
  4. ptracing the other process and installing a hardware breakpoint on the framebuffer's memory address writing. This works for an example process I've made, but not for unknown process or I've failed to do so. In all cases, it can't work in realtime either since messages between the debuggee and the debugger are done through signals and signals are so slow. Without hardware breakpoint, I can mprotect the framebuffer's page in the debuggee, so it receives a SIGFAULT when it writes to the framebuffer, but again it's very slow, so not a acceptable solution.

I think the only way to have the "live" overlay feature, without kernel/driver work, is to patch every core to load a dynamic library containing a function that's called after any framebuffer rendering. We can't just redirect memcpy (mmap yes, but it's useless) and some framework, like LVGL use pwrite to render to the framebuffer. So it needs to be cooperative.
We could inject whatever we want in that dynamic loaded function.

However, the pause feature -k works flawlessly and I think it's worth it for most use cases. There's no possible framebuffer corruption here since the original process is paused while the overlay is displayed and the framebuffer is restored before the process is resumed. It would be useful for the parent control "5mn left" overlay, and likely the "volume +", "volume -" overlay too (and maybe the "M + Start" combo too). Using a 1s delay is unnoticeable for the game and the user, but it makes its effect of having a visual feedback for the user's button press.

If you have time, please try it and if you're ok, I can remove the "live" overlay part since it's far from perfect and a lot more complex than the pause overlay mode.

X-Ryl669 added 2 commits May 29, 2025 17:48
…ed).

It's not perfect but for opaque overlay, it might be enough.
Main issue happens for core misusing framebuffer's voffset for double buffering
It's not possible to know when the core changed the buffer it wrote too.
Thus, it's not possible to backup the framebuffer before drawing since the core
can write to anypart of the buffer and move stuff around (instead of overwriting).
If we draw to both buffers (but for core moving stuff around, this leads to visual
corruption) it's slow because the ioctl to know what is the current offset chosen
takes more time than a frame duration.
@X-Ryl669
Copy link
Author

Ok, simplified the code a lot to remove live overlay feature and only support paused overlay feature that works flawlessly. I think it should be possible to update muxsnapshot to use the new options too, but it's in another future PR. Should be ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants