MFC CDialog::Create fails - c++

I'm having problems with some code to create a CDialog based window. The code was working fine last week. The only changes I made was replacing a C++ deque with a hash array. I had commented out the line of code with the Create method being called to allow me to skip loading the window. Now the code doesn't create the window at all anymore.
The Create function returns false and the GetLastError function returns 0. I don't use any custome controls inside the window - just a checkbox and a list control. As far as I can tell (I can't hook a debugger up at this point) the OnCreate and OnInitDialog functions are not being called at all.
I've pasted the code below that I've been using to test the Create function's return and GetLastError
BOOL result = ORDER_HANDLER_GUI.Create(OrderHandlerGUI::IDD, AfxGetMainWnd());
int error = ::GetLastError();
if(result)
AfxMessageBox("Created GUI");
else
{
CString msg;
msg.Format("%d", error);
AfxMessageBox("Could not create GUI");
AfxMessageBox(msg);
}
Update:
I finally managed to get the debugger to attach (this is a plugin loaded in a 3rd party application that didn't like the debugger for some reason). After stepping through the code it seems that AfxGetMainWnd() is returning NULL. I'm doing more testing on this now.

The problems seems to have been with the call to CDynLinkLibrary().
I had commented this out at the request of the company that writes the software that is loading my plugin. Adding this line back in caused some values to still be null, but the window is now created properly.
I'm going to do a bit of research on this and will update if I find anything. If anyone knows more about this than me (not hard to do) feel free to leave comments.

Does the dialog use any controls that might be causing the problem? A richedit for example?

Related

Why CreateDialog failed with error code 5 in BHO?

I use CreateDialog to create a modeless dialog in SetSite method after get the IWebBrowser2 interface. The dialog resource is in the BHO dll. When create a new instance (I mean doble click the IE shortcut) of IE the creation is successful but when I create a new tab the creation is failed (but in other computer it is successful). there is also something strange is that sometimes create a new tab will also create a new IE process but sometimes will not.
This is the code for dialog creation:
bool MyDialog::Create(MyContext *ls)
{
extern HINSTANCE hInstance; // handle of BHO dll
m_hDialog = CreateDialog(hInstance, MAKEINTRESOURCE(IDD_MYDLG),ls->GetBrowserMainWnd(), MyDlgProc);
if (m_hDialog) {
SetWindowLong(m_hDialog, GWL_USERDATA, (LONG)ls);
SetTimer(m_hDialog, 1, 1000, NULL);
return true;
}
return false;
}
I think this has something to do with dialog creation in different UI thread but not sure about this. Hope somebody can help me with this problem. Thanks a lot!
Update 2014-03-31:
The GetBrowserMainWnd method call IWebBrowser2->get_HWND to get the main window handle. But for IE7 and later the introduced tabbed window makes things complex as MSDN's description:
"Internet Explorer 7. With the introduction of tabbed browsing, the return value of this method can be ambiguous. To alleviate confusion and maintain the highest level of compatibility with existing applications, this method returns a handle to the top-level window frame, not the currently selected tab."
So, I use the example code (refer to http://msdn.microsoft.com/en-us/library/aa752126(v=vs.85).aspx) solved this problem.
It seems that the root cause is the third parameter hWndParent. When I set it to NULL this problem disappear. I think the new process of an IE tab can't access the IE main window handle so failed with error code 5.

How to disable MFC writing workspace settings to registry?

By default, a basic MFC C++ project in Visual Studio 2010 will store all its workspace settings in the HKCU registry hive under a user-configurable key name. This includes last window size/position, ribbon settings, status bar, etc.
How can you disable this feature completely so as to not write to the registry at all?
I tried not setting SetRegistryKey(), which gave me a debug assertion from the framework on first read/write to registry. SetRegistryKey((LPCTSTR)NULL) gave the same results. SetRegistryBase() seems to have no effect. No other methods in CWinApp/CWinAppEx seem to help.
Edit: My original answer was wrong. I have edited the answer.
You can tell MFC to store settings in an .ini file instead of the registry. See this previous answer. (Update: That only works if you are not using CWinAppEx.)
If you want to prevent MFC from saving some of the state of the menus and toolbars altogether, add the following to your app’s constructor:
m_bSaveState = FALSE;
The m_bSaveState member is only defined if your app is derived from CWinAppEx.
Alternatively, you can override CWinAppEx::SaveState and CWinAppEx::LoadState.
To eliminate the WindowPlacement registry key, override CWinAppEx::StoreWindowPlacement.
You may still get other registry entries being written. A complete solution would involve subclassing CSettingsStore and then, in your application, calling CSettingsStoreSP::SetRuntimeClass. (See this for more information.) This is fairly difficult because there are a whole bunch of virtual functions you will have to override in your custom CSettingsStore class.
I know this is late for many but I found that CFrameWndEx -- it will be in your CMainFrame class -- uses WM_CLOSE window to SAVE your applications default location. I do not know what loads the location. However, if you override WM_CLOSE, then that window state is never saved when you exit the program. It will attempt to re-load your last window state, but since none was ever save to begin with, there's nothing to worry about.
A GOTCHA SITUATION:
Since the framework DOES still call some sort of WM_INIT function to LOAD the last position, if you compiled your code as normal, maximize the application window, closed the program with the X, WM_CLOSE would have saved the application state as MAXIMIZED. If you recompile the app by overriding WM_CLOSE as mentioned above, re-launch the application, you'll notice it starts out maximized! Obvious not what you intended. Thus, you must re-activate (comment out WM_CLOSE), let the program save the application state by re-launch the program, when closing as a normal window. Allow the overridden WM_CLOSE to work again and you'll see that normal window reappears.
CODE:
In your CMainFrame.h
public:
afx_msg void OnClose();
In your CMainFrame.cpp ... Extend your MESSAGE MAP
BEGIN_MESSAGE_MAP(CMainFrame, CFrameWndEx)
*ON_WM_CLOSE() // <<< I ADDED THIS*
....
END_MESSAGE_MAP()
void CMainFrame::OnClose()
{
PostQuitMessage(0);
//CFrameWndEx::OnClose(); << WE DO NOT WANT TO HAVE THIS CALLED!
}

Closing a MessageBox automatically

I have a third party encryption library, which may create a MessageBox if key creation fails. The failure can be caused by bad random number generation or other rarities, and in most cases, trying again will result in success. My code will attempt key creation up to three times before deciding it failed.
Now, the issue is that the program may be used with automation. If a MessageBox is created during automation, it will block the process forever, because there's nobody to click the 'OK' button.
Does anyone know of a way to catch when this message box is created and automatically close it?
Anything is fair game, as long as it's not something that will make security suites angry. This means no hooking or code tunneling.
In summary, I need to catch when a MessageBox is created and close it. The MessageBox's creation is outside of my control. Modifying the code at runtime is not acceptable.
Also, I've noticed there are some other similar questions, but they don't have the same requirements.
EDIT: Additional note, I can find the message box via searching through all windows until I find one with a matching title and then send it a WM_CLOSE message, but I don't think this is a great solution. I also have no guarantee that the message box has been/will be displayed, or how long after my call it will be displayed. It could display instantly, it could display 1200 ms later, or it could not display at all.
Just before you begin the encryption process, install a WH_CBT hook, and in its callback watch for an nCode of HCBT_CREATEWND. If you get a matching class name ('#32770 (Dialog)' ?) and a matching title either return a nonzero value from the callback, or if that doesn't work post a WM_CLOSE (or a BM_CLICK to a relevant button if selecting an option is necessary). Uninstall the hook after the process for not messing with every possible dialog your application pops up.
That sounds like bad design on the part of that library. Generally any sort of utility library (like encryption) has no business invoking any kind of GUI (unless you explicitly ask it to).
Is there possibly some configuration or setting in this library that could disable its use of message boxes?
If not, I'd suggest that you might want to investigate using a different library. After all, if the designers of this library have already made this kind of poor design decision once, then there may be other unfortunate surprises lurking in there.
You can hope that it will be found by GetForegroundWindow, but this may catch other applications. The more brute force way is to iterate over all windows with EnumWindows looking for something that has a caption or text equal to this shown by the library.
I have once "remote controlled" an application by sending mouse click events to some controls. I guess you would have to do this in a separate thread that is watching for Events if a window is opened. Pretty ugly but working...
Create a new thread. If your function fails and a Message Box is opened, obtain a handle to the message box by looping through the windows (GetTopWindow, GetNextWindow) and comparing the window's process id to the one returned from GetCurrentProcessId().
Or, you can avoid all the hard work and just hook the MessageBox API with detours. It's not very hard, and if you don't want to pay for detours, you can do it manually.
Call VirtualProtect and set the memory protection at MessageBox at PAGE_EXECUTE_READWRITE
Create a naked function, and use it as a trampoline.
Create a function identical in parameters to MessageBox (this will be your hook)
Create a jump from MessageBox to your hook function.

ExitInstance not called in MFC app

Until now, I never really needed the Winapp ExitInstance() of a large MFC (Single Document Interface if it matters) app I'm working on. But now I do, mainly to cleanup memory alocations, unload some DLLs, etc. Well I soon learned by the obvious memory leaks and such that ExitInstance was not being called. Have I missed something obvious? Do I need to manually add something to the message map to make sure my ExitInstance override is called?
I guess i can do my cleanup elsewhere, but it's the best place if I can get it to run. Interestingly, I found quite a few instances of this by typing strings like "ExitInstance never called" and such into Google, and in no case were any real answers offered. The app normally closes when someone clicks the close box or the "Exit" from the File menu, and the OnClose() of the mainframe window certainly does always get called. I even tried forcing things by putting AfxGetMainWnd()->DestroyWindow(); in that mainframe OnClose() event, but still I can't get the ExitInstance() to actually run. Maybe it's just a big dummy function? Or maybe I'M just a big dummy? :-)
I had a similar problem to you...mine was caused by mixing Unicode and MBCS built code....maybe that was your underlying cause?
I had to convert an MBCS application to Unicode, but it was impossible to convert the whole project, so I had to mix Unicode compiled (the application) and MBCS compiled code (the DLLs).
Some of the MBCS DLLs were MFC extension DLLs, others were regular DLLs.
One of the MFC extension DLLs contained resources (bitmap image list, and common dialogs).
I didn't convert the DLL to UNICODE because it had a lot of dependent DLLs which would also have had to be converted, and in addition I didn't need the controls in the common dialogs to support Unicode text.
So I kept the DLL as MBCS, and used AfxSetResourceHandle prior to using any class in the MBCS DLL that used resources.....this was to make the resources get pulled from the DLL directly, instead of through the MFC resource chain, because MFC couldn't find the non-unicode resources otherwise.
I guess MFC doesn't like it when you have a mix of Unicode and non-unicode compiled code containing resources.....lookups in the resource chain fail (I guess has something to do with the conversion of resource IDs to an ID string i.e. via MAKEINTRESOURCE).
I made the main application UNICODE, and made sure the C++ headers of classes in the MBCS DLLs used CStringA in the function prototypes, or accepted wide strings and did the conversion internally.
What I found was my application wouldn't exit properly...it would stay in the MFC CWinThread::PumpMessage/AfxInternalPumpMessage() call, and ExitInstance would never be called.
To solve it, in my CMainFrame::OnDestroy() I put the following as the last 2 statements:
void CMainFrame::OnDestroy()
{
....
CFrameWnd::OnDestroy();
AfxPostQuitMessage(0);
}
I had the same problem. Turns out it was caused by some messages being pretranslated by the CWinApp object after the mainframe was destroyed. It would hit some validty check during that processing and just bail out instead of exiting through the normal exitinstance.
Overriding the apps PreTranslate message and returning immediately when there was no main frame resolved the issue.
BOOL CYourAppClass::PreTranslateMessage( MSG* pMsg )
{
// If the main window has gone away there is no need to do pre-translation.
// If the pre-translation were allowed to proceed the
// CWinAppEx::PreTranslateMessage may bail out without calling
/// the app's ExitInstance nor destructing the app object.
if ( !m_pMainWnd )
return FALSE;
return CWinAppEx::PreTranslateMessage( pMsg );
}
You say that "the OnClose() of the mainframe window certainly does always get called". I had a problem with one of my MFC apps; I had put some thread-cleanup code into my window's OnClose() event handler, and it wasn't getting called. After some pain, I put the thread-cleanup code into an OnDestroy() event-handler, and now it always gets called. So you may want to check whether or not your OnClose() event-handler always gets called.
I had exactly the same problem.
In my case, the issue was solved by overriding PostNcDestroy in CMainFrame. It seems that treating this last message in the main frame window makes it all work:
virtual void PostNcDestroy();
void CMainFrame::PostNcDestroy()
{
}
Another possibility is that there is an assertion failure happening in the middle of the shut down process. For example while destroying a window.
I found that when this happens, ExitInstance also gets skipped.
So, just before closing your application, clear the output/debug log window and see if such a message is printed.
If that's the case, solve the problem and see if ExitInstance gets called again.

Invalidate() debug assertion failed message (MFC, VC++)

I've made a custom control, and when I want it to repaint on the screen I call Invalidate(), and afterwards UpdateWindow(), but i get message:
debug assertion failed for a file afxwin2.inl in line 150 which is:
AFXWIN_INLINE void CWnd::Invalidate(BOOL bErase)
{ ASSERT(::IsWindow(m_hWnd)); ::InvalidateRect(m_hWnd, NULL, bErase); }
The thing is that when I run the same app in release mode, it doesn't report any message! So this clue makes me think it's about some environment configuration I should change.
What do you think?
Thanks.
Well,
ASSERT(::IsWindow(m_hWnd));
is an assertion. Assertions are statements which verify that something is true and kill your program if it's not. They're intended to be used for debugging and development rather than for being in the program once it has been released, so they are normally only compiled in in debug builds. So, it wouldn't be there in a release build, and you wouldn't get the error message. That does not mean that there isn't a problem in the release build. It just means that that it's not running the statement to check whether there's a problem.
I don't know a lot about the error in question, but looking at it,
::IsWindow(m_hWnd)
is obviously false (hence the error message). The documentation for IsWindow() would appear to indicate that the problem is that the window handle in question is not a handle for a valid window. Perhaps it hasn't been created properly, or it has already been destroyed. You'll have to figure out why your window handle is invalid.
A quick google search for "mfc iswindow" brings up this thread on msdn which might be of help to you.
You call Invalidate before window is created or after window is destroyed. Quick fix is to test for ::IsWindow(m_hWnd) before Invalidate call. To really fix this bug, find why Invalidate is called when window doesn't exist. For example, attempt to invalidate window from its constructor causes this assertion.
You have called Invalidate() on an CWnd-derived class, but that window's m_hWnd member has not been built yet. You should call the Create (or CreateEx) method first, in order to build it (or use a method that does all that for you, like DoModal() ).