MFC - Strange access violation while sending WM_IDLEUPDATECMDUI message - c++

I'm currently revising the code on an old MFC application, which I updated to be compiled with Visual Studio 2017. This application contains several MDI frames.
However there is a strange access violation happening randomly when I close these frames, which I cannot figure out, in the main form following function:
BOOL PSS_App::OnIdle(LONG count)
{
if (!count)
if (m_pMainWnd)
{
// look for any top-level windows owned by this class. NOTE handlers are used to
// avoid generation of too many temporary CWnds
for (HWND hWnd = ::GetWindow(m_pMainWnd->m_hWnd, GW_HWNDFIRST); hWnd; hWnd = ::GetNextWindow(hWnd, GW_HWNDNEXT))
if (::GetParent(hWnd) == m_pMainWnd->m_hWnd)
{
// if owned window is active, move the activation to the application window
if (GetActiveWindow() == hWnd && !::GetCapture())
m_pMainWnd->SetActiveWindow();
// update the buttons for the top-level window
::SendMessage(hWnd, WM_IDLEUPDATECMDUI, WPARAM(TRUE), 0L);
}
}
return CWinApp::OnIdle(count);
}
The issue happens when the ::SendMessage(hWnd, WM_IDLEUPDATECMDUI, WPARAM(TRUE), 0L); function is called. I tried to get the hWnd class name using the following code:
char lpClassName[256];
::GetClassName(hWnd, lpClassName, 256);
I get tooltips_class32 as result, but I don't know which class it refers to. If I try to modify the code in the following manner:
// update the buttons for the top-level window
if (std::strcmp(lpClassName, "tooltips_class32") != 0)
::SendMessage(hWnd, WM_IDLEUPDATECMDUI, WPARAM(TRUE), 0L);
The issue no longer happen in this function, and I can close my child MDI frames without problems, however another access violation happens randomly when the main form looses the focus (e.g when I click outside the app).
Can someone explain me what is happening here, or at least provide me info about what may happen? What the tooltips_class32 is exactly, and which kind of class may inherits it in MFC? Or is an ActiveX related control?
NOTE I don't know if that may have a link, but the main application inherits from CWinApp, which inherits from CWinThread. The application isn't using the multi-threading, except for the splash screen. However the issue remains the same even if I completely deactivate the splash screen code.
NOTE also that the application is a huge application containing around 30 attached DLLs. I cannot run it in Debug for now, because another strange access violation happens in one of the modules when it is loaded, which happen in Debug but never in Release. So the application is currently running in release.

Related

Restore a minimized window of another application (C++ WinAPI)

I'm working on a C++ program that which launches a .NET Winforms application.
If the app is already running, I want to restore the window instead. I grab the .NET app's window handle and use SetForegroundWindow() to bring to to the front.
This works except when the application is minimized.
I've tried combinations of the following:
ShowWindow(windowHandle, SW_SHOW);
ShowWindow(windowHandle, SW_RESTORE);
and
SendMessage(windowHandle, WM_SYSCOMMAND, SC_RESTORE, 0);
However, when this code is executed, the event becomes stuck. In the tray I see ".NET-BroadcastEventWindow.2.0.0.0.378734a.0" come up as a second window in the tray along with my .NET app, and the app is never restored properly.
This only happens when I try to send a SW/SC_RESTORE or SHOW event. SetForegroundWindow() has no issues.
Has anyone else seen this problem before?
I've had similar problems in the past that I've solved using Get/SetWindowPlacement():
// Ensure that the given window is not minimized.
// If it is minimized, restore it to its normal state.
void EnsureNotMinimized(HWND hWnd)
{
WINDOWPLACEMENT placement;
placement.length = sizeof(placement);
if(!GetWindowPlacement(hWnd, &placement))
return;
BOOL minimized = (placement.showCmd & SW_SHOWMINIMIZED) != 0;
if(!minimized)
return;
placement.showCmd = SW_SHOWNORMAL;
SetWindowPlacement(hWnd, &placement);
}
However, I've only used this for windows that belong to my own application. I don't know if security would allow it to be used on outsiders.

Why CreateDialog failed with error code 5 in BHO?

I use CreateDialog to create a modeless dialog in SetSite method after get the IWebBrowser2 interface. The dialog resource is in the BHO dll. When create a new instance (I mean doble click the IE shortcut) of IE the creation is successful but when I create a new tab the creation is failed (but in other computer it is successful). there is also something strange is that sometimes create a new tab will also create a new IE process but sometimes will not.
This is the code for dialog creation:
bool MyDialog::Create(MyContext *ls)
{
extern HINSTANCE hInstance; // handle of BHO dll
m_hDialog = CreateDialog(hInstance, MAKEINTRESOURCE(IDD_MYDLG),ls->GetBrowserMainWnd(), MyDlgProc);
if (m_hDialog) {
SetWindowLong(m_hDialog, GWL_USERDATA, (LONG)ls);
SetTimer(m_hDialog, 1, 1000, NULL);
return true;
}
return false;
}
I think this has something to do with dialog creation in different UI thread but not sure about this. Hope somebody can help me with this problem. Thanks a lot!
Update 2014-03-31:
The GetBrowserMainWnd method call IWebBrowser2->get_HWND to get the main window handle. But for IE7 and later the introduced tabbed window makes things complex as MSDN's description:
"Internet Explorer 7. With the introduction of tabbed browsing, the return value of this method can be ambiguous. To alleviate confusion and maintain the highest level of compatibility with existing applications, this method returns a handle to the top-level window frame, not the currently selected tab."
So, I use the example code (refer to http://msdn.microsoft.com/en-us/library/aa752126(v=vs.85).aspx) solved this problem.
It seems that the root cause is the third parameter hWndParent. When I set it to NULL this problem disappear. I think the new process of an IE tab can't access the IE main window handle so failed with error code 5.

Creating multiple dialogs in an MFC app with no main Window, they become children of each other

(title updated)
Following on from this question, now I have a clearer picture what's going on...
I have a MFC application with no main window, which exposes an API to create dialogs. When I call some of these methods repeatedly, the dialogs created are parented to each other instead of all being parented to the desktop... I have no idea why.
But anyway even after creation, I am unable to change the parent back to NULL or CWnd::GetDesktopWindow()... if I call SetParent followed by GetParent, nothing has changed.
So apart from the really weird question of why Windows is magically parenting each dialog to the last one created, is there anything I'm missing to be able to set these windows as children of the desktop?
UPDATED: I have found the reason for all this, but not the solution. From my dialog constructor, we end up in:
BOOL CDialog::CreateIndirect(LPCDLGTEMPLATE lpDialogTemplate, CWnd* pParentWnd,
void* lpDialogInit, HINSTANCE hInst)
{
ASSERT(lpDialogTemplate != NULL);
if (pParentWnd == NULL)
pParentWnd = AfxGetMainWnd();
m_lpDialogInit = lpDialogInit;
return CreateDlgIndirect(lpDialogTemplate, pParentWnd, hInst);
}
Note: if (pParentWnd == NULL)pParentWnd = AfxGetMainWnd();
The call-stack from my dialog constructor looks like this:
mfc80d.dll!CDialog::CreateIndirect(const DLGTEMPLATE * lpDialogTemplate=0x005931a8, CWnd * pParentWnd=0x00000000, void * lpDialogInit=0x00000000, HINSTANCE__ * hInst=0x00400000)
mfc80d.dll!CDialog::CreateIndirect(void * hDialogTemplate=0x005931a8, CWnd * pParentWnd=0x00000000, HINSTANCE__ * hInst=0x00400000)
mfc80d.dll!CDialog::Create(const char * lpszTemplateName=0x0000009d, CWnd * pParentWnd=0x00000000)
mfc80d.dll!CDialog::Create(unsigned int nIDTemplate=157, CWnd * pParentWnd=0x00000000)
MyApp.exe!CMyDlg::CMyDlg(CWnd * pParent=0x00000000)
Running in the debugger, if I manually change pParentWnd back to 0 in CDialog::CreateIndirect, everything works fine... but how do I stop it happening in the first place?
Some thoughts:
First, you are passing NULL for the parent window the whole way through the chain. Its becomming non NULL when MFC tries to find your applications main window.
As I see it you have two mitigations:
Create a CWnd from the desktop window. CWnd::GetDesktopWindow will give you a non NULL window to use as a parent window that will inhibit the AfxGetMainWnd call.
Or trace into AfxGetMainWnd, find out where it is reading the main window from, and see if there is some setting to prevent it finding your dialog window.
On a final note. The MFC terminology is unfortunate :- On Windows, only child windows have parent windows. Popup or desktop windows have owner windows. CreateWindow takes a single parameter that accepts the owner, or parent of the window being created. The distinction is important because while a parent window can be changed, an owner cannot. SetParent will NOT change the owner window for a popup or overlapped window.
OK, found it!
There were actually two problems. I was passing NULL as the parent/owner... but trying to pass CWnd::GetDesktopWindow() was not helping so I'd given up on the idea until finding the behaviour of CDialog::CreateIndirect. That made me take a closer look at my code, and I finally spotted that MyDialog::MyDialog(CWnd *pParent) was calling super::Create(NULL), not super::Create(pParent)... because we'd always passed it NULL before anyway the bug had never been apparent.
So yet again, the hard problem turned out to be only one step away from a typo!
MFC can only create one window at a time IIRC. In the dark and distant past at least, when MFC creates a Win32 Window it needs to associate the MFC CWnd instance with the window. Because the first message a window receives is NOT the WM_CREATE message with the LPVUSERDATA parameter MFC stored the CWnd instance in a thread local variable - in the not unreasonable expectation that the next call to CWnd::WindowProc would be for the window it started trying to create.
I have no idea how one could actually write code to subvert this process. It all hinges on how you could be creating windows at differing velocities.

Creating multiple MFC dialogs through COM, strange behaviour

Updated: please see this other thread instead, all this COM stuff is not part of the problem.
One of our apps has a COM interface which will launch a dialog, e.g:
STDMETHODIMP CSomeClass::LaunchDialog(BSTR TextToDisplay)
{
CDialog *pDlg = new CSomeDialog(TextToDisplay);
pDlg->BringWindowToTop();
}
For some reason when the COM method is called several times at once by the server, we get odd behaviour:
We get multiple dialogs, but only one entry in the taskbar
Dialog Z-order is based on order created and can't be changed... the first dialog created is always shown under the 2nd one, 2nd under 3rd, etc, even when you drag them around
if N dialogs were created, closing one of them closes it and all the others created afterwards. e.g if 5 dialogsa re created and you close the 3rd one, #3,#4,#5 all get closed.
It's somehow like the dialogs are siblings but I don't see anything weird going on. Is it perhaps due to COM, or is this a weird MFC/Win32 issue?
EDIT: If the interface method is called several times separately, it works as expected. Only when the server component sends several through at once does it seem to mess up. Could threading/timings be to blame?
EDIT2:
I put this logging in:
std::stringstream ss;
HWND self = dlg->m_hWnd;
HWND parent = dlg->GetParent() ? dlg->GetParent()->m_hWnd : 0;
ss<<"Dlg created'. HWND = "<<self<<", Parent = "<<parent<<std::endl;
OutputDebugString(ss.str().c_str());
It gave:
Dlg created. HWND = 0013014A, Parent = 00000000
Dlg created. HWND = 001B0390, Parent = 0013014A
Dlg created. HWND = 000B03B0, Parent = 001B0390
So clearly the problem is the dialogs are being made children of each other. But the question is, WHY?! It seems Windows is doing this automatically...
This question seems to be slightly away from the main issue of parenting, so I have tried to separate out the main issue into a new question.
It sounds like the first dialog has been set as the owner of the second, and the second as the owner of the third. Can you change the dialog initialization to explicitly specify the owner window? Is there a window that makes sense to assign? Perhaps the Desktop window, if they're all intended to be top-level?
If you want to be able to access all three (or more), then they would need to be modeless. Try using Create(CSomeClass::IDD, CWnd::GetDesktopWindow()), and you ought to see sibling dialogs, all of which show up on the taskbar.

Managing Window Z-Order Like Photoshop CS

So I've got an application whose window behavior I would like to behave more like Photoshop CS. In Photoshop CS, the document windows always stay behind the tool windows, but are still top level windows. For MDI child windows, since the document window is actually a child, you can't move it outside of the main window. In CS, though, you can move your image to a different monitor fine, which is a big advantage over docked applications like Visual Studio, and over regular MDI applications.
Anyway, here's my research so far. I've tried to intercept the WM_MOUSEACTIVATE message, and use the DeferWindowPos commands to do my own ordering of the window, and then return MA_NOACTIVATEANDEAT, but that causes the window to not be activated properly, and I believe there are other commands that "activate" a window without calling WM_MOUSEACTIVATE (like SetFocus() I think), so that method probably won't work anyway.
I believe Windows' procedure for "activating" a window is
1. notify the unactivated window with the WM_NCACTIVATE and WM_ACTIVATE messages
2. move the window to the top of the z-order (sending WM_POSCHANGING, WM_POSCHANGED and repaint messages)
3. notify the newly activated window with WM_NCACTIVATE and WM_ACTIVATE messages.
It seems the cleanest way to do it would be to intercept the first WM_ACTIVATE message, and somehow notify Windows that you're going to override their way of doing the z-ordering, and then use the DeferWindowPos commands, but I can't figure out how to do it that way. It seems once Windows sends the WM_ACTIVATE message, it's already going to do the reordering its own way, so any DeferWindowPos commands I use are overridden.
Right now I've got a basic implementation quasy-working that makes the tool windows topmost when the app is activated, but then makes them non-topmost when it's not, but it's very quirky (it sometimes gets on top of other windows like the task manager, whereas Photoshop CS doesn't do that, so I think Photoshop somehow does it differently) and it just seems like there would be a more intuitive way of doing it.
Anyway, does anyone know how Photoshop CS does it, or a better way than using topmost?
I havn't seen anything remarkable about Photoshop CS that requries anything close to this level of hacking that can't instead be done simply by specifying the correct owner window relationships when creating windows. i.e. any window that must be shown above some other window specifies that window as its owner when being created - if you have multiple document windows, each one gets its own set of owned child windows that you can dynamically show and hide as the document window gains and looses activation.
You can try handle WM_WINDOWPOSCHANGING event to prevent overlaping another windows (with pseudo-topmost flag). So you are avoiding all problems with setting/clearing TopMost flag.
public class BaseForm : Form
{
public virtual int TopMostLevel
{
get { return 0; }
}
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool EnumThreadWindows(uint dwThreadId, Win32Callback lpEnumFunc, IntPtr lParam);
/// <summary>
/// Get process window handles sorted by z order from top to bottom.
/// </summary>
public static IEnumerable<IntPtr> GetWindowsSortedByZOrder()
{
List<IntPtr> handles = new List<IntPtr>();
EnumThreadWindows(GetCurrentThreadId(),
(hWnd, lparam) =>
{
handles.Add(hWnd);
return true;
}, IntPtr.Zero);
return handles;
}
protected override void WndProc(ref Message m)
{
if (m.Msg == (int)WindowsMessages.WM_WINDOWPOSCHANGING)
{
//Looking for Window at the bottom of Z-order, but with TopMostLevel > this.TopMostLevel
foreach (IntPtr handle in GetWindowsSortedByZOrder().Reverse())
{
var window = FromHandle(handle) as BaseForm;
if (window != null && this.TopMostLevel < window.TopMostLevel)
{
//changing hwndInsertAfter field in WindowPos structure
if (IntPtr.Size == 4)
{
Marshal.WriteInt32(m.LParam, IntPtr.Size, window.Handle.ToInt32());
}
else if (IntPtr.Size == 8)
{
Marshal.WriteInt64(m.LParam, IntPtr.Size, window.Handle.ToInt64());
}
break;
}
}
}
base.WndProc(ref m);
}
}
public class FormWithLevel1 : BaseForm
{
public override int TopMostLevel
{
get { return 1; }
}
}
So, FormWithLevel1 will be always over any BaseForm. You can add any number of Z-order Levels. Windows on the same level behave as usual, but will be always under Windows with level Current+1 and over Windows with level Current-1.
Not being familiar to Photoshop CS it is bit hard to know exactly what look and feel you are trying to achieve.
But I would have thought if you created a modeless dialog window as you tool window and you made sure it had the WS_POPUP style then the resulting tool window would not be clipped to the main parent window and Windows would automatically manage the z-Order making sure that the tool window stayed on top of the parent window.
And as the tool window dialog is modeless it would not interfere with the main window.
Managing Window Z-Order Like Photoshop CS
You should create the toolwindow with the image as the parent so that windows manage the zorder. There is no need to set WS_POPUP or WS_EX_TOOLWINDOW. Those flags only control the rendering of the window.
Call CreateWindowEx with the hwnd of the image window as the parent.
In reply to Chris and Emmanuel, the problem with using the owner window feature is that a window can only be owned by one other window, and you can't change who owns a window. So if tool windows A and B always need to be on top of document windows C and D, then when doc window C is active, I want it to own windows A and B so that A and B will always be on top of it. But when I activate doc window D, I would have to change ownership of tool windows A and B to D, or else they will go behind window D (since they're owned by window C). However, Windows doesn't allow you to change ownership of a window, so that option isn't available.
For now I've got it working with the topmost feature, but it is a hack at best. I do get some consolation in the fact that GIMP has tried themselves to emulate Photoshop with their version 2.6, but even their implementation occasionally acts quirky, which leads me to believe that their implementation was a hack as well.
Have you tried to make the tool windows topmost when the main window receives focus, and non-topmost when it loses focus? It sounds like you've already started to look at this sort of solution... but much more elaborate.
As a note, it seems to be quite well documented that tool windows exhibit unexpected behavior when it comes to z-ordering. I haven't found anything on MSDN to confirm it, but it could be that Windows manages them specially.
I would imagine they've, since they're not using .NET, rolled their own windowing code over the many years of its existence and it is now, like Amazon's original OBIDOS, so custom to their product that off-the-shelf (aka .NET's MDI support) just aren't going to come close.
I don't like answering without a real answer, but likely you'd have to spend a lot of time and effort to get something similar if Photoshop-like is truly your goal. Is it worth your time? Just remember many programmers over many years and versions have come together to get Photoshop's simple-seeming windowing behavior to work just right and to feel natural to you.
It looks like you're already having to delve pretty deep into Win32 API functions and values to even glimpse at a "solution" and that should be your first red flag. Is it eventually possible? Probably. But depending on your needs and your time and a lot of other factors only you could decide, it may not be practical.