GUI does not update contents when resizing application window

[apetoyan]

Hello @pthom, firstly thank you so much for your hard work in providing us this wonderful library!

Unfortunately, I've been struggling to resolve this issue of the GUI not updating when resizing the application in bordered mode. I noticed this when running the hello_imgui_demodocking app but with runnerParams.appWindowParams.borderless = false. Any time I try to resize, the GUI contents stop updating properly until I let go of the mouse click.

This issue occurs for me on all the demo hello_imgui and imgui_bundle apps and on every backend I've tried. Disabling idling doesn't help. I'm also running on macOS on the M1 chip.

Please let me know if there is something I might be missing. If you like, I can provide a GIF of the issue. Any help is greatly appreciated!

[pthom]

I'm also using a Mac M1.
What I see is that
1/ when the window is borderless everything is repainted when you resize the window.
2/ When it is not borderless, the content is stretched and repainted only after you finished resizing.

The second behavior is something that is buried deep in the ImGui framework, or maybe inside SDL/Glfw or the OS. I'm not sure there is a practical way to change that.

Is it what you're experiencing?

d.mp4

You could try a different combination of backend (renderer and/or platform):

    // Compile with -DHELLOIMGUI_HAS_METAL=ON -DHELLOIMGUI_HAS_VULKAN=ON to enable Metal and Vulkan
    // Compile with -DHELLOIMGUI_USE_SDL2=ON or -DHELLOIMGUI_USE_GLFW3=ON to use SDL2 or GLFW3
    runnerParams.rendererBackendType = HelloImGui::RendererBackendType::Metal;
    runnerParams.platformBackendType = HelloImGui::PlatformBackendType::Glfw;

[apetoyan]

Hello, thank you for the quick reply! Yes, this is exactly what I'm experiencing. I also typically use GLFW+Metal, but I experienced the same thing with SDL and OpenGL. I suppose I can try Vulkan as well just in case.

The only reason I brought it up is because I don't experience this issue with a vanilla Dear ImGui application. So I thought maybe it was some extra feature in hello_imgui, like DPI scaling or idling that is contributing to it.

I know it's not a big deal for many use cases, but for displaying realtime plots it can be a bit distracting. Regardless, I really appreciate you taking the time to respond. I'll keep trying different settings, and if you come across a fix please let me know!

[apetoyan]

Hello, I apologize, I believe you're correct that is an issue with the SDL and GLFW backends. When I said I wasn't experiencing this with a regular ImGui application, it was because I had used the example_apple_metal which has OSX/Cocoa as the platform backend. This actually performs as expected when resizing the application window on macOS.

Unfortunately, Im not great with CMake/C++ and so I'm not sure how to integrate imgui_impl_osx with hello_imgui. If there is a relatively non-trivial way to go about it, please let me know. Otherwise, I know you have more pressing things to work on so I'll be happy to close this issue. Thanks!

[pthom]

https://stackoverflow.com/questions/45880238/how-to-draw-while-resizing-glfw-window

[apetoyan]

Thank you! I was able to get this to work on one of the ImGui examples.

[pthom]

Hello,

I did a change for this in the repo. It was not that easy, since Glfw and SDL handle this in different ways, and this is far
from the classic event loop.
See
ec0d86d

Anyhow, it should work now

[apetoyan]

Wow, thank you so much! I was honestly not expecting a repo change so quickly and I'm very grateful for your hard work.

Everything works great with OpenGL and Vulkan as far as I've tested, however when using the Metal renderer the program crashes right away upon resizing. Is this something you tested/experienced? All my testing is for MacOS so I'm also not sure if the changes had any effects on Windows or DirectX, though if I have time tomorrow I can try it on my PC.

[pthom]

Hmm, thanks for letting me know ! This is a serious issue and it also fails, under Linux.

I had to disable it for the moment: It failed hard under Linux. This is due to the fact that the way this event is handled may cause re-entrance in the rendering loop.

See disable commit:
7866ddf

Testing this library is hard: There are six different platforms (macOS, Linux, Windows, iOS, Android, emscripten), 5 different renderers (OpenGl, DirectX11, DirectX12, Vulkan, Metal) and 2 platforms renderers (Sdl, Glfw). This makes for a grand total of 60 combinations.

[apetoyan]

I see. After doing some digging, seems like this has been discussed a number of times on the imgui Github page, and Omar goes into some detail about it here: ocornut/imgui#3672

Unfortunately there doesn't seem to be a good way to handle this at the moment. It requires a non-standard event loop (as you mentioned) and/or multit-threading to properly deal with, plus different implementations for different platforms, backends, and OS's.

I apologize for not looking into this in more detail before assuming it was something in hello_imgui's implementation. I'm fairly new to imgui and found your repos (imgui_bundle too) to be very helpful in seeing what's possible. I didn't mean to send you down a rabbit hole in trying to fix something that's external. I still appreciate you taking the time to help me and for taking a shot at this complicated issue.

[apetoyan]

Also, if you ever need someone to help in testing out updates for hello_imgui or imgui_bundle, please let me know! I'd be happy to help.

[pthom]

First, thank you for your offer. I will remember it.

Actually there is no issue under Linux. It seems that it could work under Glfw. However, it feels hard with the combination macOS / SDL / Metal. And the way I had to circumvent this issue is very tricky. I think I will give up for the moment.

But I will use this issue on github as a way to remember my investigation, so I will copy-paste some status.

When resizing a standard non-borderless window under macOS / SDL / Metal,

1. the callback resizingEventWatcher is called

sdl_window_helper.cpp:

    static int resizingEventWatcher(void* data, SDL_Event* event)
    {
        //return 0; // Re-entrance into CreateFramesAndRender may break!

        if (event->type == SDL_WINDOWEVENT &&
            event->window.event == SDL_WINDOWEVENT_RESIZED) {
            SDL_Window* win = SDL_GetWindowFromID(event->window.windowID);
            if (win == (SDL_Window*)data) {
                if (gRenderCallbackDuringResize_Sdl)
                    gRenderCallbackDuringResize_Sdl();
            }
        }
        return 0;
    }

With the following call stack:

i.e. we are inside a call to PollEvents

2. The call to gRenderCallbackDuringResize_Sdl(), will actually call AbstractRunner::CreateFramesAndRender(insideReentrantCall=true).
   This is a reentrant call! This call succeeds
   However, it has to call SDL_NewFrame (and others)

3. Then, RunnerSdl2::Impl_PollEvents() continues its loop
   while (SDL_PollEvent(&event))

    However we are unlucky, because in the middle of the SDL_PollEvent(), we get a call to 
   * SDL_PushEvent()
   * which is triggered SDL_SendWindowEvent
   * with is triggered by  (void)windowDidResize:(NSNotification *)aNotification. 
     (in SDL_cocoawindow.m), which also ends by
   

windowDidResize ends with this:

And the program will fail directly at the exit of this function.

The reason is perhaps understandable: we are still inside this loop;

void RunnerSdl2::Impl_PollEvents()
{
    SDL_Event event;
    while (SDL_PollEvent(&event)) // <- here
    {
        if (params.callbacks.AnyBackendEventCallback)
        {
            if (params.callbacks.AnyBackendEventCallback(&event))
                continue;
        }

.. but an event was pushed right in the middle of SDL_PollEvent(), with a strange call stack.

[pthom]

And as a consequence, it seems not reasonable to do this for all platforms / backends / renders. It has too many implications, and should be perhaps be governed by a preference for advanced users who are ready to deal with the possible consequences.

[pthom]

See f92af95

I disabled this by default. But you have an advanced reference which you can set.

If you have time to test this on Windows, I'd be grateful. I'm quite overwhelmed by other projects at the moment.

[pthom]

Combinations OS + Platform Backend + Renderer Backend results with the preference appWindowParams.repaintDuringResize_GotchaReentrantRepaint:

• macOS + Vulkan or OpenGL + SDL or Glfw: windows repaint works with the pref set
• macOS + Metal + SDL or Glfw: bug when resizing the window if the pref is set (works if not set)
• linux + OpenGL + Glfw: the pref is unecessary, the window is repainted during resize

etc etc

[apetoyan]

Hello, sorry for the late response. I did some brief testing after I got home from work today.

• Windows + OpenGL + SDL or Glfw: windows repaint works with the pref set

However, I ran into issues using DirectX, though they may not be related to the pref. Even when building with the 1.4.2 release of hello_imgui, DirectX 11 experienced a lot of scaling/resolution issues, especially when resizing. Using the latest update and setting the preference, the repainting would work but possibly with some additional bugs/artifacts. Hard to say exactly what since the original implementation had many issues. Also, DirectX 12 would not work at all on either version.

I'm not sure if maybe it has to do with my drivers or perhaps I'm missing something during the build process. I'll try to do more thorough testing and get back to you tomorrow. I'll also see if I can squeeze in some Vulkan testing as well.

[apetoyan]

Actually, I didn't catch it at first but for DirectX 11 it throws an assertion upon resizing: "The current implementation of Dx11 backend does not support changing the window size!" This would explain all the bugs. Sorry for the confusion.

[pthom]

Hello, sorry for the late response

Nothing to be sorry about really! This was fast and very informative.

DirectX 11 it throws an assertion upon resizing: "The current implementation of Dx11 backend does not support changing the window size!
Also, DirectX 12 would not work at all on either version

Yes, the DirectX support needs to be improved. I'm by no means a specialist of Windows, So I hope someone someday will help on this aspect.

Cheers

[apetoyan]

Some good news

• Windows + DX11 + SDL: windows repaint works with the pref set
• Windows + DX12 + SDL: windows repaint works with the pref set

Sorry! I had made a mistake when building and unknowingly rebuilt with OpenGL instead of DirectX. I now tested with DX11 and DX12.

Unfortunately, the DirectX implementation is still very broken on Windows with lots of scaling issues. However, the repaint feature magically works despite this allowing you to see the artifacts in real time :) This is on DX12 btw (DX11 doesn't support resizing so I can't test the preference).

And

• Windows + Vulkan + SDL or GLFW: windows repaint works with the pref set

Interestingly, seems like Vulkan sometimes loses V-sync when resizing (I noticed this on both Windows and MacOS, moreso using SDL) though this is not a big deal imo.

Anyways, hope this helped with your investigation! Thanks again for adding this preference option, I'll be sure to use it safely. If you need help with anything else please let me know!

[pthom]

I reopened this issue to let it appear as a faq.

Thank you!

[apetoyan]

Hello again! So I noticed there was still a bit of jitter for GLFW when resizing. To fix this, you can simply use WindowRefreshCallback instead of the WindowSizeCallback!

• Change glfwSetWindowSizeCallback(window, WindowSizeCallback) to glfwSetWindowRefreshCallback(window, WindowRefreshCallback)
• Change WindowSizeCallback(GLFWwindow* window, int width, int height) to WindowRefreshCallback(GLFWwindow*)

Doing this makes the experience extremely smooth when resizing. Unfortunately, glfwPollEvents() still blocks when simply holding down the resize button, but as far as I can tell this can only be solved with multithreading which is beyond my capabilities.

Anyways, hope this helps!