Invalidate() debug assertion failed message (MFC, VC++) - c++

I've made a custom control, and when I want it to repaint on the screen I call Invalidate(), and afterwards UpdateWindow(), but i get message:
debug assertion failed for a file afxwin2.inl in line 150 which is:
AFXWIN_INLINE void CWnd::Invalidate(BOOL bErase)
{ ASSERT(::IsWindow(m_hWnd)); ::InvalidateRect(m_hWnd, NULL, bErase); }
The thing is that when I run the same app in release mode, it doesn't report any message! So this clue makes me think it's about some environment configuration I should change.
What do you think?
Thanks.

Well,
ASSERT(::IsWindow(m_hWnd));
is an assertion. Assertions are statements which verify that something is true and kill your program if it's not. They're intended to be used for debugging and development rather than for being in the program once it has been released, so they are normally only compiled in in debug builds. So, it wouldn't be there in a release build, and you wouldn't get the error message. That does not mean that there isn't a problem in the release build. It just means that that it's not running the statement to check whether there's a problem.
I don't know a lot about the error in question, but looking at it,
::IsWindow(m_hWnd)
is obviously false (hence the error message). The documentation for IsWindow() would appear to indicate that the problem is that the window handle in question is not a handle for a valid window. Perhaps it hasn't been created properly, or it has already been destroyed. You'll have to figure out why your window handle is invalid.
A quick google search for "mfc iswindow" brings up this thread on msdn which might be of help to you.

You call Invalidate before window is created or after window is destroyed. Quick fix is to test for ::IsWindow(m_hWnd) before Invalidate call. To really fix this bug, find why Invalidate is called when window doesn't exist. For example, attempt to invalidate window from its constructor causes this assertion.

You have called Invalidate() on an CWnd-derived class, but that window's m_hWnd member has not been built yet. You should call the Create (or CreateEx) method first, in order to build it (or use a method that does all that for you, like DoModal() ).

Related

What function is called when Alt-Enter is pressed?

I have a game app that has the ability to go fullscreen and back to windowed when Alt-Enter is pressed. However, when it goes fullscreen, I get the following warning from DirectX:
DXGI Warning: IDXGISwapChain::Present: Fullscreen presentation inefficiencies incurred due to application not using IDXGISwapChain::ResizeBuffers appropriately, specifying a DXGI_MODE_DESC not available in IDXGIOutput::GetDisplayModeList, or not using DXGI_SWAP_CHAIN_FLAG_ALLOW_MODE_SWITCH.
I've already ruled out the second two possibilities through testing, so I know the only reasons left for the warning to pop up are either IDXGISwapChain::ResizeBuffers isn't being used right, or Windows is just bugged. Since I can't debug the 2nd possibility, I'm sticking with the ResizeBuffers problem. To debug this, I want to look at what happens when Alt-Enter is pressed going from windowed to fullscreen. However, the app does not seem to be calling my ResizeDXGIBuffers method; in fact, it seems that Alt-Enter is embedded into windows or DirectX somewhere, and I don't know how to find the chain of function calls that go off when it is pressed. EDIT: When my method is put in the WM_ACTIVATEAPP handler, it is called, but this is not what i meant. If i take it out of that message handler, the window STILL goes to fullscreen, even though I am not calling any functions to make the window fullscreen myself. So Alt+Enter must be automatically calling some internal function to do this.
So that is my question: Does anyone know what function is called by windows and/or DirectX 11 when Alt-Enter is pressed?
EDIT: As the tags for this question say, I am using DirectX 11 on a Windows machine. Specifically, Windows 7 64-bit.
EDIT 2: I now completely eat the Alt+Enter keystroke and manually store the state of Alt+Enter being pressed so that I know for certain only my code is being called. The warning I spoke of above persists, however. I am following the MSDN best practices as well, so I don't know where to go from here.
Try handling the WM_ACTIVATEAPP message.
I do not know which framework you use to create your windows, so I can't tell how to concretely handle this message.
After looking at the MSDN best practices page and re-working my code to reflect all of the practices described, the warning has disappeared. I hope this helps anyone else that has the same problem.
Also, thanks to Hans Passant for the link. I already fixed it by the time you posted it, but thanks anyways.

C++ wxWidgets Gui-App stays in memory after closing it

The subject says it all. After i closed my app, it stays in the list of processes with some memory. I tried google perf tools and hours of debugging to find the leak.
Are there other tools to test it and find the problem?
Thank you.
My guess is that you have closed the top level window, and thus all its child windows, but you have not closed the app itself.
This does not happen if your program is arranged in a 'normal' way, but if you have, deliberately or by accident, used an unusual arrangement this can happen.
Fixing it, of course, depends on how exactly you have arranged your code. However, here is a suggestion to begin.
The usual way to close the app is to call wxApp::OnExit() which normally occurs when the top level window closes.
Do you have your own class, derived from wxApp? Do you have an override of OnExit()? If not, then make it so and use the debugger to check whether or not it is being called. If it is not being called, work out how to ensure that it is called.
Another idea: use the following to check that your top level window will close the app
bool wxApp::GetExitOnFrameDelete() const
Returns true if the application will exit when the top-level window is
deleted, false otherwise.
If this return false, use the corresponding set to make it so.
A 3rd idea: The application will not exit while there are any top level windows open. So perhaps you have another top level window that is minimized or invisible but has not been closed? Any wxDialog or WxFrame or window derived from these is a top level window and will prevent the application from closing.
A 4th idea: Do you have any globals or attributes of the application object, whose destructors could enter an endless loop? These destructors are called after the windows are destroyed and if one of them does not return you would see the behaviour you describe.
You may try to look at wxWidget's sample folder. You'll find lots of small but complete applications that contain the full init/exit application cycle.
Inspect some samples and compare with your app's workflow.
Yes...problem solved. A TopLevelWindow that was not destroyed. A Memory Leak....stupid mistake.

Closing a MessageBox automatically

I have a third party encryption library, which may create a MessageBox if key creation fails. The failure can be caused by bad random number generation or other rarities, and in most cases, trying again will result in success. My code will attempt key creation up to three times before deciding it failed.
Now, the issue is that the program may be used with automation. If a MessageBox is created during automation, it will block the process forever, because there's nobody to click the 'OK' button.
Does anyone know of a way to catch when this message box is created and automatically close it?
Anything is fair game, as long as it's not something that will make security suites angry. This means no hooking or code tunneling.
In summary, I need to catch when a MessageBox is created and close it. The MessageBox's creation is outside of my control. Modifying the code at runtime is not acceptable.
Also, I've noticed there are some other similar questions, but they don't have the same requirements.
EDIT: Additional note, I can find the message box via searching through all windows until I find one with a matching title and then send it a WM_CLOSE message, but I don't think this is a great solution. I also have no guarantee that the message box has been/will be displayed, or how long after my call it will be displayed. It could display instantly, it could display 1200 ms later, or it could not display at all.
Just before you begin the encryption process, install a WH_CBT hook, and in its callback watch for an nCode of HCBT_CREATEWND. If you get a matching class name ('#32770 (Dialog)' ?) and a matching title either return a nonzero value from the callback, or if that doesn't work post a WM_CLOSE (or a BM_CLICK to a relevant button if selecting an option is necessary). Uninstall the hook after the process for not messing with every possible dialog your application pops up.
That sounds like bad design on the part of that library. Generally any sort of utility library (like encryption) has no business invoking any kind of GUI (unless you explicitly ask it to).
Is there possibly some configuration or setting in this library that could disable its use of message boxes?
If not, I'd suggest that you might want to investigate using a different library. After all, if the designers of this library have already made this kind of poor design decision once, then there may be other unfortunate surprises lurking in there.
You can hope that it will be found by GetForegroundWindow, but this may catch other applications. The more brute force way is to iterate over all windows with EnumWindows looking for something that has a caption or text equal to this shown by the library.
I have once "remote controlled" an application by sending mouse click events to some controls. I guess you would have to do this in a separate thread that is watching for Events if a window is opened. Pretty ugly but working...
Create a new thread. If your function fails and a Message Box is opened, obtain a handle to the message box by looping through the windows (GetTopWindow, GetNextWindow) and comparing the window's process id to the one returned from GetCurrentProcessId().
Or, you can avoid all the hard work and just hook the MessageBox API with detours. It's not very hard, and if you don't want to pay for detours, you can do it manually.
Call VirtualProtect and set the memory protection at MessageBox at PAGE_EXECUTE_READWRITE
Create a naked function, and use it as a trampoline.
Create a function identical in parameters to MessageBox (this will be your hook)
Create a jump from MessageBox to your hook function.

ExitInstance not called in MFC app

Until now, I never really needed the Winapp ExitInstance() of a large MFC (Single Document Interface if it matters) app I'm working on. But now I do, mainly to cleanup memory alocations, unload some DLLs, etc. Well I soon learned by the obvious memory leaks and such that ExitInstance was not being called. Have I missed something obvious? Do I need to manually add something to the message map to make sure my ExitInstance override is called?
I guess i can do my cleanup elsewhere, but it's the best place if I can get it to run. Interestingly, I found quite a few instances of this by typing strings like "ExitInstance never called" and such into Google, and in no case were any real answers offered. The app normally closes when someone clicks the close box or the "Exit" from the File menu, and the OnClose() of the mainframe window certainly does always get called. I even tried forcing things by putting AfxGetMainWnd()->DestroyWindow(); in that mainframe OnClose() event, but still I can't get the ExitInstance() to actually run. Maybe it's just a big dummy function? Or maybe I'M just a big dummy? :-)
I had a similar problem to you...mine was caused by mixing Unicode and MBCS built code....maybe that was your underlying cause?
I had to convert an MBCS application to Unicode, but it was impossible to convert the whole project, so I had to mix Unicode compiled (the application) and MBCS compiled code (the DLLs).
Some of the MBCS DLLs were MFC extension DLLs, others were regular DLLs.
One of the MFC extension DLLs contained resources (bitmap image list, and common dialogs).
I didn't convert the DLL to UNICODE because it had a lot of dependent DLLs which would also have had to be converted, and in addition I didn't need the controls in the common dialogs to support Unicode text.
So I kept the DLL as MBCS, and used AfxSetResourceHandle prior to using any class in the MBCS DLL that used resources.....this was to make the resources get pulled from the DLL directly, instead of through the MFC resource chain, because MFC couldn't find the non-unicode resources otherwise.
I guess MFC doesn't like it when you have a mix of Unicode and non-unicode compiled code containing resources.....lookups in the resource chain fail (I guess has something to do with the conversion of resource IDs to an ID string i.e. via MAKEINTRESOURCE).
I made the main application UNICODE, and made sure the C++ headers of classes in the MBCS DLLs used CStringA in the function prototypes, or accepted wide strings and did the conversion internally.
What I found was my application wouldn't exit properly...it would stay in the MFC CWinThread::PumpMessage/AfxInternalPumpMessage() call, and ExitInstance would never be called.
To solve it, in my CMainFrame::OnDestroy() I put the following as the last 2 statements:
void CMainFrame::OnDestroy()
{
....
CFrameWnd::OnDestroy();
AfxPostQuitMessage(0);
}
I had the same problem. Turns out it was caused by some messages being pretranslated by the CWinApp object after the mainframe was destroyed. It would hit some validty check during that processing and just bail out instead of exiting through the normal exitinstance.
Overriding the apps PreTranslate message and returning immediately when there was no main frame resolved the issue.
BOOL CYourAppClass::PreTranslateMessage( MSG* pMsg )
{
// If the main window has gone away there is no need to do pre-translation.
// If the pre-translation were allowed to proceed the
// CWinAppEx::PreTranslateMessage may bail out without calling
/// the app's ExitInstance nor destructing the app object.
if ( !m_pMainWnd )
return FALSE;
return CWinAppEx::PreTranslateMessage( pMsg );
}
You say that "the OnClose() of the mainframe window certainly does always get called". I had a problem with one of my MFC apps; I had put some thread-cleanup code into my window's OnClose() event handler, and it wasn't getting called. After some pain, I put the thread-cleanup code into an OnDestroy() event-handler, and now it always gets called. So you may want to check whether or not your OnClose() event-handler always gets called.
I had exactly the same problem.
In my case, the issue was solved by overriding PostNcDestroy in CMainFrame. It seems that treating this last message in the main frame window makes it all work:
virtual void PostNcDestroy();
void CMainFrame::PostNcDestroy()
{
}
Another possibility is that there is an assertion failure happening in the middle of the shut down process. For example while destroying a window.
I found that when this happens, ExitInstance also gets skipped.
So, just before closing your application, clear the output/debug log window and see if such a message is printed.
If that's the case, solve the problem and see if ExitInstance gets called again.

MFC CDialog::Create fails

I'm having problems with some code to create a CDialog based window. The code was working fine last week. The only changes I made was replacing a C++ deque with a hash array. I had commented out the line of code with the Create method being called to allow me to skip loading the window. Now the code doesn't create the window at all anymore.
The Create function returns false and the GetLastError function returns 0. I don't use any custome controls inside the window - just a checkbox and a list control. As far as I can tell (I can't hook a debugger up at this point) the OnCreate and OnInitDialog functions are not being called at all.
I've pasted the code below that I've been using to test the Create function's return and GetLastError
BOOL result = ORDER_HANDLER_GUI.Create(OrderHandlerGUI::IDD, AfxGetMainWnd());
int error = ::GetLastError();
if(result)
AfxMessageBox("Created GUI");
else
{
CString msg;
msg.Format("%d", error);
AfxMessageBox("Could not create GUI");
AfxMessageBox(msg);
}
Update:
I finally managed to get the debugger to attach (this is a plugin loaded in a 3rd party application that didn't like the debugger for some reason). After stepping through the code it seems that AfxGetMainWnd() is returning NULL. I'm doing more testing on this now.
The problems seems to have been with the call to CDynLinkLibrary().
I had commented this out at the request of the company that writes the software that is loading my plugin. Adding this line back in caused some values to still be null, but the window is now created properly.
I'm going to do a bit of research on this and will update if I find anything. If anyone knows more about this than me (not hard to do) feel free to leave comments.
Does the dialog use any controls that might be causing the problem? A richedit for example?