ExitInstance not called in MFC app - mfc

Until now, I never really needed the Winapp ExitInstance() of a large MFC (Single Document Interface if it matters) app I'm working on. But now I do, mainly to cleanup memory alocations, unload some DLLs, etc. Well I soon learned by the obvious memory leaks and such that ExitInstance was not being called. Have I missed something obvious? Do I need to manually add something to the message map to make sure my ExitInstance override is called?
I guess i can do my cleanup elsewhere, but it's the best place if I can get it to run. Interestingly, I found quite a few instances of this by typing strings like "ExitInstance never called" and such into Google, and in no case were any real answers offered. The app normally closes when someone clicks the close box or the "Exit" from the File menu, and the OnClose() of the mainframe window certainly does always get called. I even tried forcing things by putting AfxGetMainWnd()->DestroyWindow(); in that mainframe OnClose() event, but still I can't get the ExitInstance() to actually run. Maybe it's just a big dummy function? Or maybe I'M just a big dummy? :-)

I had a similar problem to you...mine was caused by mixing Unicode and MBCS built code....maybe that was your underlying cause?
I had to convert an MBCS application to Unicode, but it was impossible to convert the whole project, so I had to mix Unicode compiled (the application) and MBCS compiled code (the DLLs).
Some of the MBCS DLLs were MFC extension DLLs, others were regular DLLs.
One of the MFC extension DLLs contained resources (bitmap image list, and common dialogs).
I didn't convert the DLL to UNICODE because it had a lot of dependent DLLs which would also have had to be converted, and in addition I didn't need the controls in the common dialogs to support Unicode text.
So I kept the DLL as MBCS, and used AfxSetResourceHandle prior to using any class in the MBCS DLL that used resources.....this was to make the resources get pulled from the DLL directly, instead of through the MFC resource chain, because MFC couldn't find the non-unicode resources otherwise.
I guess MFC doesn't like it when you have a mix of Unicode and non-unicode compiled code containing resources.....lookups in the resource chain fail (I guess has something to do with the conversion of resource IDs to an ID string i.e. via MAKEINTRESOURCE).
I made the main application UNICODE, and made sure the C++ headers of classes in the MBCS DLLs used CStringA in the function prototypes, or accepted wide strings and did the conversion internally.
What I found was my application wouldn't exit properly...it would stay in the MFC CWinThread::PumpMessage/AfxInternalPumpMessage() call, and ExitInstance would never be called.
To solve it, in my CMainFrame::OnDestroy() I put the following as the last 2 statements:
void CMainFrame::OnDestroy()
{
....
CFrameWnd::OnDestroy();
AfxPostQuitMessage(0);
}

I had the same problem. Turns out it was caused by some messages being pretranslated by the CWinApp object after the mainframe was destroyed. It would hit some validty check during that processing and just bail out instead of exiting through the normal exitinstance.
Overriding the apps PreTranslate message and returning immediately when there was no main frame resolved the issue.
BOOL CYourAppClass::PreTranslateMessage( MSG* pMsg )
{
// If the main window has gone away there is no need to do pre-translation.
// If the pre-translation were allowed to proceed the
// CWinAppEx::PreTranslateMessage may bail out without calling
/// the app's ExitInstance nor destructing the app object.
if ( !m_pMainWnd )
return FALSE;
return CWinAppEx::PreTranslateMessage( pMsg );
}

You say that "the OnClose() of the mainframe window certainly does always get called". I had a problem with one of my MFC apps; I had put some thread-cleanup code into my window's OnClose() event handler, and it wasn't getting called. After some pain, I put the thread-cleanup code into an OnDestroy() event-handler, and now it always gets called. So you may want to check whether or not your OnClose() event-handler always gets called.

I had exactly the same problem.
In my case, the issue was solved by overriding PostNcDestroy in CMainFrame. It seems that treating this last message in the main frame window makes it all work:
virtual void PostNcDestroy();
void CMainFrame::PostNcDestroy()
{
}

Another possibility is that there is an assertion failure happening in the middle of the shut down process. For example while destroying a window.
I found that when this happens, ExitInstance also gets skipped.
So, just before closing your application, clear the output/debug log window and see if such a message is printed.
If that's the case, solve the problem and see if ExitInstance gets called again.

Related

Proper Cleanup when using chained WndProcs

When building a chain of WndProcs using SetWindowLongPtr, we store the parent WndProc (the one that has less importance in the chain), so we are able to restore it and call it, like so:
LONG_PTR oldProc = SetWindowLongPtr(hWnd, GWL_WNDPROC, &myWndProc);
WndProc myWndProc(HANDLE hWnd, ...) {
return CallWndProc(oldProc);
}
This is just a rough pseudocode from the top of my head, so please don't focus on that.
The problem with this, however, is, that when trying to remove myWndProc from the chain, like so:
SetWindowLongPtr(hWnd, GWL_WNDPROC, oldProc);
We effectively set the proc, that was in place when first hooking, as the topmost WndProc.
This means, that all WndProcs, that have been added after myWndProc (e.g. all kinds of Overlays), suddenly won't be called anymore.
Furthermore all hooks that are added afterwards point to invalid memory with their oldProc.
This means, hooks done that way can't be properly unloaded without messing something up.
While this would be fixable if all WndProcs belong to the same (our) codebase, this is not fixable with external code.
I haven't found a WM that would be called when a new WndProc is set, so it's also not possible to keep track of which WndProcs have been registered after the current handler, so one could re-build the chain. That would still not solve the issue with invalid oldProcs.
Is there something else where the WinAPI can help fixing this problem?
Afaict using SetWindowLongPtr in that way does something like creating a child window(?) that way one could maybe query something?
Or is the bottom line just, that a WndProc cannot safely be unloaded anymore?
Is there something else where the WinAPI can help fixing this problem?
Yes, actually. This is exactly the kind of situation that SetWindowSubclass() was introduced to address. Let it handle the chaining and unchaining for you. Don't use SetWindowLongPtr(GWLP_WNDPROC) at all anymore.
See Safer subclassing and Subclassing Controls Using ComCtl32.dll version 6 for more information.
That being said, depending on why you are subclassing, SetWindowsHookEx() or SetWinEventHook() may provide an alternative hook you can use for your intended purposes.
We effectively set the proc, that was in place when first hooking, as the topmost WndProc. This means, that all WndProcs, that have been added after myWndProc (e.g. all kinds of Overlays), suddenly won't be called anymore. Furthermore all hooks that are added afterwards point to invalid memory with their oldProc.
This means, hooks done that way can't be properly unloaded without messing something up. While this would be fixable if all WndProcs belong to the same (our) codebase, this is not fixable with external code.
But you forget that hwnd is your window, created by your process, so it is under your control. It means you must design your module to properly restore the chain of window procs. There is no "external code" that can set your window proc to something else.
The emphasis is on "your window."
EDIT
Putting back the WndProc pointer will never affect the chain of window procedures. It is just that after putting it back, your procedure will not be called anymore, but the next one is called directly, which is exactly what you want.

How to disable MFC writing workspace settings to registry?

By default, a basic MFC C++ project in Visual Studio 2010 will store all its workspace settings in the HKCU registry hive under a user-configurable key name. This includes last window size/position, ribbon settings, status bar, etc.
How can you disable this feature completely so as to not write to the registry at all?
I tried not setting SetRegistryKey(), which gave me a debug assertion from the framework on first read/write to registry. SetRegistryKey((LPCTSTR)NULL) gave the same results. SetRegistryBase() seems to have no effect. No other methods in CWinApp/CWinAppEx seem to help.
Edit: My original answer was wrong. I have edited the answer.
You can tell MFC to store settings in an .ini file instead of the registry. See this previous answer. (Update: That only works if you are not using CWinAppEx.)
If you want to prevent MFC from saving some of the state of the menus and toolbars altogether, add the following to your app’s constructor:
m_bSaveState = FALSE;
The m_bSaveState member is only defined if your app is derived from CWinAppEx.
Alternatively, you can override CWinAppEx::SaveState and CWinAppEx::LoadState.
To eliminate the WindowPlacement registry key, override CWinAppEx::StoreWindowPlacement.
You may still get other registry entries being written. A complete solution would involve subclassing CSettingsStore and then, in your application, calling CSettingsStoreSP::SetRuntimeClass. (See this for more information.) This is fairly difficult because there are a whole bunch of virtual functions you will have to override in your custom CSettingsStore class.
I know this is late for many but I found that CFrameWndEx -- it will be in your CMainFrame class -- uses WM_CLOSE window to SAVE your applications default location. I do not know what loads the location. However, if you override WM_CLOSE, then that window state is never saved when you exit the program. It will attempt to re-load your last window state, but since none was ever save to begin with, there's nothing to worry about.
A GOTCHA SITUATION:
Since the framework DOES still call some sort of WM_INIT function to LOAD the last position, if you compiled your code as normal, maximize the application window, closed the program with the X, WM_CLOSE would have saved the application state as MAXIMIZED. If you recompile the app by overriding WM_CLOSE as mentioned above, re-launch the application, you'll notice it starts out maximized! Obvious not what you intended. Thus, you must re-activate (comment out WM_CLOSE), let the program save the application state by re-launch the program, when closing as a normal window. Allow the overridden WM_CLOSE to work again and you'll see that normal window reappears.
CODE:
In your CMainFrame.h
public:
afx_msg void OnClose();
In your CMainFrame.cpp ... Extend your MESSAGE MAP
BEGIN_MESSAGE_MAP(CMainFrame, CFrameWndEx)
*ON_WM_CLOSE() // <<< I ADDED THIS*
....
END_MESSAGE_MAP()
void CMainFrame::OnClose()
{
PostQuitMessage(0);
//CFrameWndEx::OnClose(); << WE DO NOT WANT TO HAVE THIS CALLED!
}

Closing a MessageBox automatically

I have a third party encryption library, which may create a MessageBox if key creation fails. The failure can be caused by bad random number generation or other rarities, and in most cases, trying again will result in success. My code will attempt key creation up to three times before deciding it failed.
Now, the issue is that the program may be used with automation. If a MessageBox is created during automation, it will block the process forever, because there's nobody to click the 'OK' button.
Does anyone know of a way to catch when this message box is created and automatically close it?
Anything is fair game, as long as it's not something that will make security suites angry. This means no hooking or code tunneling.
In summary, I need to catch when a MessageBox is created and close it. The MessageBox's creation is outside of my control. Modifying the code at runtime is not acceptable.
Also, I've noticed there are some other similar questions, but they don't have the same requirements.
EDIT: Additional note, I can find the message box via searching through all windows until I find one with a matching title and then send it a WM_CLOSE message, but I don't think this is a great solution. I also have no guarantee that the message box has been/will be displayed, or how long after my call it will be displayed. It could display instantly, it could display 1200 ms later, or it could not display at all.
Just before you begin the encryption process, install a WH_CBT hook, and in its callback watch for an nCode of HCBT_CREATEWND. If you get a matching class name ('#32770 (Dialog)' ?) and a matching title either return a nonzero value from the callback, or if that doesn't work post a WM_CLOSE (or a BM_CLICK to a relevant button if selecting an option is necessary). Uninstall the hook after the process for not messing with every possible dialog your application pops up.
That sounds like bad design on the part of that library. Generally any sort of utility library (like encryption) has no business invoking any kind of GUI (unless you explicitly ask it to).
Is there possibly some configuration or setting in this library that could disable its use of message boxes?
If not, I'd suggest that you might want to investigate using a different library. After all, if the designers of this library have already made this kind of poor design decision once, then there may be other unfortunate surprises lurking in there.
You can hope that it will be found by GetForegroundWindow, but this may catch other applications. The more brute force way is to iterate over all windows with EnumWindows looking for something that has a caption or text equal to this shown by the library.
I have once "remote controlled" an application by sending mouse click events to some controls. I guess you would have to do this in a separate thread that is watching for Events if a window is opened. Pretty ugly but working...
Create a new thread. If your function fails and a Message Box is opened, obtain a handle to the message box by looping through the windows (GetTopWindow, GetNextWindow) and comparing the window's process id to the one returned from GetCurrentProcessId().
Or, you can avoid all the hard work and just hook the MessageBox API with detours. It's not very hard, and if you don't want to pay for detours, you can do it manually.
Call VirtualProtect and set the memory protection at MessageBox at PAGE_EXECUTE_READWRITE
Create a naked function, and use it as a trampoline.
Create a function identical in parameters to MessageBox (this will be your hook)
Create a jump from MessageBox to your hook function.

Handling Messages in Console Apps/DLLs in C++ Win32

I would like to have the ability to process Win32 messages in a console app and/or inside a standalone DLL.
I have been able to do it in .NET with the following article and it works great in C# inside a console app and standalone DLL
http://msdn.microsoft.com/en-us/magazine/cc163417.aspx
Is there a way to do the equivalent with C/C++ Win32 APIs? I have tried doing RegisterClassEx(...) and CreateWindow(...) even passing in HWND_MESSAGE to hWndParent but the trouble is that after the "invisible" window is created messages are not being processed probably due to the lack of a message pump.
Where would the message pump go if you had a DLL entry point? I have tried creating another thread in a DLL and put while(GetMesage(..)) there but that did not work either.
Any ideas?
You need a message pump yes. The window also has thread affinity so it needs to be created on the same thread that you're running the message pump on. The basic approach is sound, if you include more code it may become clear what the problem is.
In addition to what Logan Capaldo said, you also have the problem that, as a DLL, you don't know at compile time what kind of process is going to be loading you at runtime.
If you are being loaded by a console application (/SUBSYSTEM:CONSOLE), then creating a hidden window of your own and setting up a message pump on that same thread will work fine (as long as you are the first window created).
If you are being loaded by a windows app (/SUBSYSTEM:WINDOWS) then you might run into problems getting messages. They will be sent to the top-level window in the hierarchy, which you didn't create. You'll need to get the hWnd of the main process and subclass it (if you aren't already).
If you are being loaded by a service, then you aren't going to get window messages at all. You instead need to use the RegisterServiceCtrlHandlerEx Function

Invalidate() debug assertion failed message (MFC, VC++)

I've made a custom control, and when I want it to repaint on the screen I call Invalidate(), and afterwards UpdateWindow(), but i get message:
debug assertion failed for a file afxwin2.inl in line 150 which is:
AFXWIN_INLINE void CWnd::Invalidate(BOOL bErase)
{ ASSERT(::IsWindow(m_hWnd)); ::InvalidateRect(m_hWnd, NULL, bErase); }
The thing is that when I run the same app in release mode, it doesn't report any message! So this clue makes me think it's about some environment configuration I should change.
What do you think?
Thanks.
Well,
ASSERT(::IsWindow(m_hWnd));
is an assertion. Assertions are statements which verify that something is true and kill your program if it's not. They're intended to be used for debugging and development rather than for being in the program once it has been released, so they are normally only compiled in in debug builds. So, it wouldn't be there in a release build, and you wouldn't get the error message. That does not mean that there isn't a problem in the release build. It just means that that it's not running the statement to check whether there's a problem.
I don't know a lot about the error in question, but looking at it,
::IsWindow(m_hWnd)
is obviously false (hence the error message). The documentation for IsWindow() would appear to indicate that the problem is that the window handle in question is not a handle for a valid window. Perhaps it hasn't been created properly, or it has already been destroyed. You'll have to figure out why your window handle is invalid.
A quick google search for "mfc iswindow" brings up this thread on msdn which might be of help to you.
You call Invalidate before window is created or after window is destroyed. Quick fix is to test for ::IsWindow(m_hWnd) before Invalidate call. To really fix this bug, find why Invalidate is called when window doesn't exist. For example, attempt to invalidate window from its constructor causes this assertion.
You have called Invalidate() on an CWnd-derived class, but that window's m_hWnd member has not been built yet. You should call the Create (or CreateEx) method first, in order to build it (or use a method that does all that for you, like DoModal() ).