Wrong window painting behaviour in frameless Qt window (win32) - c++

Question

I have an QML application (also tested it with QWidgets, same problem) and to make it borderless (but still support the native WM features like aero snap, etc) I followed this by implementing an QAbstractNativeEventFilter and responding to the WM_NCCALSIZE signal with zero:
switch(msg->message) {
case WM_NCCALCSIZE:
*r = 0;
return 1;
...
}
I also set some window flags which are not in the Qt Namespace with
SetWindowLong(hwnd, GWL_STYLE, WS_OVERLAPPEDWINDOW | WS_VISIBLE | WS_CLIPSIBLINGS | WS_BORDER | WS_SYSMENU | WS_MINIMIZEBOX | WS_MAXIMIZEBOX | WS_THICKFRAME);
This works fine until I move or resize the window which causes Qt to rerender and a unpainted area width the width of the title and the borders appears:
Before moving/resizing
After moving/resizing
I also found a workaround for this by adding the FramelessWindowHint flag in Qt:
window->setFlags(Qt::FramelessWindowHint | Qt::WindowSystemMenuHint | Qt::Dialog);
But now this margins occurs again when the window state changes (maximizing, minimizing, ...). By blocking the WM_SIZE event for example when SIZE_MAXIMIZED is the parameter the margin doesn't appear but then I also am not able to maximize the window from Qt. This means it is a Qt side problem.
I also noticed by inspecting the window style with winspector, that after I have maximized it a new property atom appears:
Can you help me fixing this?

Answer 1

I think using SetWindowLong on your window handle and using the Qt Window Flags/Qt Widget Attributes is asking for trouble. You can go and look at the Qt source to see what happens when processing those window flags.
When I've created frameless windows, I've usually done it to prevent moving and resizing, because I am managing all that separately.
One problem that I had related to it, was when the On Screen Keyboard came up and docked, it would resize my windows. So besides calling resize() I had to also use setFixedSize to prevent my widget from getting manipulated when the operating system attempted to change the size of the window.
So in other words, I would add an application wide QShortcut, listening for the snapping keyboard shortcuts and resize your window the way you want it to when it happens, if you are managing a frameless window.
Hope that helps.

Answer 2

I wonder if this was a flaw of Qt message relay, coz I met with a similar problem which occurs on nested windows. If you press SIZE_MAXIMIZE or SIZE_MINIMIZE button of the parent window, the child window sometimes is not able to receive WM_SIZE message. I suppose there are roughly two solutions: 1. Fix Qt, 2. Work it around.
Here I have a OGL rendering child window, sometimes even the WM_SIZE message is not correctly passed. That is, if you resize parent window, you get part of the client area black.
I just use a simple workaround to fix this, which check the current size with the cached one, and make a sort of manually resize myself.