VC++ 2010: Show open dialog hangs [closed] - c++

Question

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
Hi I've converted VC++6 project to VC++ 2010 and I keep encountering error while displaying modal Open File dialog.
It shows bottom part of dialog but top is missing, also filter also is not populated.
header has public members for storing file information:
CString m_strFilePathName;
CString m_strFileExtName;
My code to show dialog:
static TCHAR BASED_CODE szFilter[] = _T("Open Bin File(*.abs)|")
_T("Default DataBase File(*.ddf)|")
_T("SatCodex File(*.sdx)|")
_T("Format Text File(*.txt)");
// TODO: Add your command handler code here
CFileDlg fd(TRUE, NULL, "", OFN_HIDEREADONLY | OFN_OVERWRITEPROMPT, szFilter);
//"Open Bin File(*.abs)|Default DataBase File(*.ddf)|SatCodex File(*.sdx)|Format Text File(*.txt)");
const int c_cMaxFiles = 1;
const int c_cbBuffSize = (c_cMaxFiles * (MAX_PATH + 1)) + 1;
fd.GetOFN().lpstrFile = m_strFilePathName.GetBuffer(c_cbBuffSize);
fd.GetOFN().nMaxFile = c_cbBuffSize;
// Dialog hangs on this line:
if(fd.DoModal() != IDOK) return;
Diagnostic returns: First-chance exception at 0x006c69cc in AliEditor.exe: 0xC0000005: Access violation reading location 0x00000020.
Breaking while in hanging mode call stack returns this information:
user32.dll!_NtUserWaitMessage#0() + 0x15 bytes <-STOPPED HERE
user32.dll!_NtUserWaitMessage#0() + 0x15 bytes
user32.dll!_DialogBox2#16() + 0x109 bytes
user32.dll!_InternalDialogBox#24() + 0xc9 bytes
user32.dll!_DialogBoxIndirectParamAorW#24() + 0x36 bytes
user32.dll!_DialogBoxIndirectParamW#20() + 0x1b bytes
comdlg32.dll!CFileOpenSave::Show() + 0x146 bytes
AliEditor.exe!CFileDialog::DoModal() Line 748 + 0x26 bytes C++
AliEditor.exe!CMainFrame::OnFileOpen() Line 195 + 0xb bytes C++
AliEditor.exe!_AfxDispatchCmdMsg(CCmdTarget * pTarget, unsigned int nID, int nCode, void (void)* pfn, void * pExtra, unsigned int nSig, AFX_CMDHANDLERINFO * pHandlerInfo) Line 82 C++
AliEditor.exe!CCmdTarget::OnCmdMsg(unsigned int nID, int nCode, void * pExtra, AFX_CMDHANDLERINFO * pHandlerInfo) Line 381 + 0x27 bytes C++
AliEditor.exe!CFrameWnd::OnCmdMsg(unsigned int nID, int nCode, void * pExtra, AFX_CMDHANDLERINFO * pHandlerInfo) Line 973 + 0x18 bytes C++
AliEditor.exe!CWnd::OnCommand(unsigned int wParam, long lParam) Line 2729 C++
AliEditor.exe!CFrameWnd::OnCommand(unsigned int wParam, long lParam) Line 371 C++
AliEditor.exe!CWnd::OnWndMsg(unsigned int message, unsigned int wParam, long lParam, long * pResult) Line 2101 + 0x1e bytes C++
AliEditor.exe!CWnd::WindowProc(unsigned int message, unsigned int wParam, long lParam) Line 2087 + 0x20 bytes C++
AliEditor.exe!AfxCallWndProc(CWnd * pWnd, HWND__ * hWnd, unsigned int nMsg, unsigned int wParam, long lParam) Line 257 + 0x1c bytes C++
AliEditor.exe!AfxWndProc(HWND__ * hWnd, unsigned int nMsg, unsigned int wParam, long lParam) Line 420 C++
Declarations of I guess (I am not C++ developer) overriding (fileDlg.cpp):
#include "stdafx.h"
#include "AliEditor.h"
#include "FileDlg.h"
#ifdef _DEBUG
#define new DEBUG_NEW
#undef THIS_FILE
static char THIS_FILE[] = __FILE__;
#endif
/////////////////////////////////////////////////////////////////////////////
// CFileDlg
IMPLEMENT_DYNAMIC(CFileDlg, CFileDialog)
CFileDlg::CFileDlg(BOOL bOpenFileDialog, LPCTSTR lpszDefExt, LPCTSTR lpszFileName,
DWORD dwFlags, LPCTSTR lpszFilter, CWnd* pParentWnd) :
CFileExportDialog(bOpenFileDialog, lpszDefExt, lpszFileName, dwFlags, lpszFilter, pParentWnd)
{
}
BEGIN_MESSAGE_MAP(CFileDlg, CFileExportDialog)
//{{AFX_MSG_MAP(CFileDlg)
ON_WM_CLOSE()
//}}AFX_MSG_MAP
END_MESSAGE_MAP()
File "dlgfile.cpp" is very big, but this just the top part:
// This is a part of the Microsoft Foundation Classes C++ library.
// Copyright (C) Microsoft Corporation
// All rights reserved.
//
// This source code is only intended as a supplement to the
// Microsoft Foundation Classes Reference and related
// electronic documentation provided with the library.
// See these sources for detailed information regarding the
// Microsoft Foundation Classes product.
#include "stdafx.h"
#include <dlgs.h> // for standard control IDs for commdlg
#include "afxglobals.h"
#define new DEBUG_NEW
////////////////////////////////////////////////////////////////////////////
// FileOpen/FileSaveAs common dialog helper
CFileDialog::CFileDialog(BOOL bOpenFileDialog,
LPCTSTR lpszDefExt, LPCTSTR lpszFileName, DWORD dwFlags,
LPCTSTR lpszFilter, CWnd* pParentWnd, DWORD dwSize, BOOL bVistaStyle)
: CCommonDialog(pParentWnd)
{
OSVERSIONINFO vi;
ZeroMemory(&vi, sizeof(OSVERSIONINFO));
vi.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
::GetVersionEx(&vi);
// if running under Vista
if (vi.dwMajorVersion >= 6)
{
m_bVistaStyle = bVistaStyle;
}
else
{
m_bVistaStyle = FALSE;
}
m_bPickFoldersMode = FALSE;
// determine size of OPENFILENAME struct if dwSize is zero
if (dwSize == 0)
{
dwSize = sizeof(OPENFILENAME);
}
// size of OPENFILENAME must be at least version 5
ASSERT(dwSize >= sizeof(OPENFILENAME));
// allocate memory for OPENFILENAME struct based on size passed in
m_pOFN = static_cast<LPOPENFILENAME>(malloc(dwSize));
ASSERT(m_pOFN != NULL);
if (m_pOFN == NULL)
AfxThrowMemoryException();
memset(&m_ofn, 0, dwSize); // initialize structure to 0/NULL
m_szFileName[0] = '\0';
m_szFileTitle[0] = '\0';
m_pofnTemp = NULL;
m_bOpenFileDialog = bOpenFileDialog;
m_nIDHelp = bOpenFileDialog ? AFX_IDD_FILEOPEN : AFX_IDD_FILESAVE;
m_ofn.lStructSize = dwSize;
m_ofn.lpstrFile = m_szFileName;
m_ofn.nMaxFile = _countof(m_szFileName);
m_ofn.lpstrDefExt = lpszDefExt;
m_ofn.lpstrFileTitle = (LPTSTR)m_szFileTitle;
m_ofn.nMaxFileTitle = _countof(m_szFileTitle);
m_ofn.Flags |= dwFlags | OFN_ENABLEHOOK | OFN_EXPLORER;
if(dwFlags & OFN_ENABLETEMPLATE)
m_ofn.Flags &= ~OFN_ENABLESIZING;
m_ofn.hInstance = AfxGetResourceHandle();
m_ofn.lpfnHook = (COMMDLGPROC)_AfxCommDlgProc;
// setup initial file name
if (lpszFileName != NULL)
Checked::tcsncpy_s(m_szFileName, _countof(m_szFileName), lpszFileName, _TRUNCATE);
// Translate filter into commdlg format (lots of \0)
if (lpszFilter != NULL)
{
m_strFilter = lpszFilter;
LPTSTR pch = m_strFilter.GetBuffer(0); // modify the buffer in place
// MFC delimits with '|' not '\0'
while ((pch = _tcschr(pch, '|')) != NULL)
*pch++ = '\0';
m_ofn.lpstrFilter = m_strFilter;
// do not call ReleaseBuffer() since the string contains '\0' characters
}
if (m_bVistaStyle == TRUE)
{
if (SUCCEEDED(CoInitializeEx(NULL, COINIT_APARTMENTTHREADED)))
{ // multi-threaded is not supported
IFileDialog* pIFileDialog;
IFileDialogCustomize* pIFileDialogCustomize;
HRESULT hr;
USE_INTERFACE_PART_STD(FileDialogEvents);
USE_INTERFACE_PART_STD(FileDialogControlEvents);
if (m_bOpenFileDialog)
{
hr = CoCreateInstance(CLSID_FileOpenDialog, NULL, CLSCTX_INPROC_SERVER,
IID_PPV_ARGS(&pIFileDialog));
}
else
{
hr = CoCreateInstance(CLSID_FileSaveDialog, NULL, CLSCTX_INPROC_SERVER,
IID_PPV_ARGS(&pIFileDialog));
}
if (FAILED(hr))
{
m_bVistaStyle = FALSE;
return;
}
hr = pIFileDialog->QueryInterface(IID_PPV_ARGS(&pIFileDialogCustomize));
ENSURE(SUCCEEDED(hr));
hr = pIFileDialog->Advise(reinterpret_cast<IFileDialogEvents*>(&m_xFileDialogEvents), &m_dwCookie);
ENSURE(SUCCEEDED(hr));
m_pIFileDialog = static_cast<void*>(pIFileDialog);
m_pIFileDialogCustomize = static_cast<void*>(pIFileDialogCustomize);
}
else
{
m_bVistaStyle = FALSE;
}
}
}
CFileDialog::~CFileDialog()
{
free(m_pOFN);
if (m_bVistaStyle == TRUE)
{
HRESULT hr;
hr = (static_cast<IFileDialog*>(m_pIFileDialog))->Unadvise(m_dwCookie);
ENSURE(SUCCEEDED(hr));
(static_cast<IFileDialogCustomize*>(m_pIFileDialogCustomize))->Release();
(static_cast<IFileDialog*>(m_pIFileDialog))->Release();
CoUninitialize();
}
}
const OPENFILENAME& CFileDialog::GetOFN() const
{
return *m_pOFN;
}
OPENFILENAME& CFileDialog::GetOFN()
{
return *m_pOFN;
}
Going step by step I get to this point in same ("dlgfile.cpp") file:
INT_PTR CFileDialog::DoModal()
{
ASSERT_VALID(this);
ASSERT(m_ofn.Flags & OFN_ENABLEHOOK);
ASSERT(m_ofn.lpfnHook != NULL); // can still be a user hook
// zero out the file buffer for consistent parsing later
ASSERT(AfxIsValidAddress(m_ofn.lpstrFile, m_ofn.nMaxFile));
DWORD nOffset = lstrlen(m_ofn.lpstrFile)+1;
ASSERT(nOffset <= m_ofn.nMaxFile);
memset(m_ofn.lpstrFile+nOffset, 0, (m_ofn.nMaxFile-nOffset)*sizeof(TCHAR));
// This is a special case for the file open/save dialog,
// which sometimes pumps while it is coming up but before it has
// disabled the main window.
HWND hWndFocus = ::GetFocus();
BOOL bEnableParent = FALSE;
m_ofn.hwndOwner = PreModal();
AfxUnhookWindowCreate();
if (m_ofn.hwndOwner != NULL && ::IsWindowEnabled(m_ofn.hwndOwner))
{
bEnableParent = TRUE;
::EnableWindow(m_ofn.hwndOwner, FALSE);
}
_AFX_THREAD_STATE* pThreadState = AfxGetThreadState();
ASSERT(pThreadState->m_pAlternateWndInit == NULL);
if (m_bVistaStyle == TRUE)
{
AfxHookWindowCreate(this);
}
else if (m_ofn.Flags & OFN_EXPLORER)
pThreadState->m_pAlternateWndInit = this;
else
AfxHookWindowCreate(this);
INT_PTR nResult = 0;
if (m_bVistaStyle == TRUE)
{
ApplyOFNToShellDialog();
// HERE CALLS **OPENFILENAME& CFileDialog::GetOFN()** method and then hangs
HRESULT hr = (static_cast<IFileDialog*>(m_pIFileDialog))->Show(m_ofn.hwndOwner);
nResult = (hr == S_OK) ? IDOK : IDCANCEL;
}
else if (m_bOpenFileDialog)
nResult = ::AfxCtxGetOpenFileName(&m_ofn);
else
nResult = ::AfxCtxGetSaveFileName(&m_ofn);
if (nResult)
ASSERT(pThreadState->m_pAlternateWndInit == NULL);
pThreadState->m_pAlternateWndInit = NULL;
// Second part of special case for file open/save dialog.
if (bEnableParent)
::EnableWindow(m_ofn.hwndOwner, TRUE);
if (::IsWindow(hWndFocus))
::SetFocus(hWndFocus);
PostModal();
return nResult ? nResult : IDCANCEL;
}
Could anyone help me?
[Update]
As no one can help me at this stage, I've uploaded project to public place here. Can anyone download and tell me what's wrong when I click open file?

Answer 1

The only problem that I can see is that your filter string looks incorrect. It looks like your code is based on one of the MSDN samples. And according to that sample your szFilter should look like this:
static TCHAR BASED_CODE szFilter[] = _T("Open Bin File(*.abs)|*.abs|")
_T("Default DataBase File(*.ddf)|*.ddf|")
_T("SatCodex File(*.sdx)|*.sdx|")
_T("Format Text File(*.txt)|*.txt||");
FYI I used http://msdn.microsoft.com/en-US/library/wh5hz49d(v=vs.100).aspx as a reference.
[update]
I created a small MFC app and pasted in your code. There is a typo in your code sample. The class is CFileDialog, not CFileDlg. I fixed it, then applied the change described above to fix the szFilter String. It works for me. The file dialog displays, I enter some data and dismiss the file dialog. No access violation, no assert. I think you need to look elsewhere in your code to track down the problem.
[update 2]
Ignore everything I previously said. Your original access violation is around line 174 of FileExportDialog.cpp
void CFileExportDialog::OnTypeChange()
{
// get current filename
CWnd *fileNameBox = GetParent()->GetDlgItem(edt1);
ASSERT_VALID(fileNameBox);
CString fileName; fileNameBox->GetWindowText(fileName);
// get current extension
CWnd *typeNameBox = GetParent()->GetDlgItem(cmb1);
ASSERT_VALID(typeNameBox);
CString typeName; typeNameBox->GetWindowText(typeName);
...
The real call stack is:
AliEditor.exe!CWnd::GetDlgItem(int nID) Line 92 + 0x3 bytes C++
AliEditor.exe!CFileExportDialog::OnTypeChange() Line 177 + 0x14 bytes C++
AliEditor.exe!CFileDialog::XFileDialogEvents::OnTypeChange(IFileDialog * __formal) Line 619 C++
comdlg32.dll!76138057()
[Frames below may be incorrect and/or missing, no symbols loaded for comdlg32.dll]
comctl32.dll!731c7613()
comdlg32.dll!7612bdda()
comdlg32.dll!7612c5a0()
KernelBase.dll!76096055()
comctl32.dll!7458b575()
user32.dll!767281c8()
user32.dll!76728326()
user32.dll!76728347()
user32.dll!76730d27()
user32.dll!7673794a()
AliEditor.exe!_AfxActivationWndProc(HWND__ * hWnd, unsigned int nMsg, unsigned int wParam, long lParam) Line 456 + 0x1a bytes C++
user32.dll!767262fa()
user32.dll!76726d3a()
user32.dll!76726ce9()
user32.dll!7672965e()
user32.dll!7675206f()
user32.dll!7674cf4b()
user32.dll!7674ce8a()
user32.dll!7674cc0e()
comdlg32.dll!7612597b()
AliEditor.exe!CFileDialog::DoModal() Line 748 + 0x26 bytes C++
...
The program is crashing at CWnd::GetDlgItem because the CWnd "this" pointer is 0. This is because the first line of CFileExportDialog::OnTypeChange is GetParent() which is NULL because this dialog has no parent.
The class CFileExportDialog is not part of Visual Studio it appears to be based on some public domain code written in 1999 from http://www.codeguru.com/cpp/w-d/dislog/commondialogs/article.php/c1863/CFileExportDialog-Class.htm. And it looks like the implementation of OnTypeChange is based on the original developer reverse engineering the details of the Microsoft CFileDialog class as it was implemented in 1999. I'm guessing that the internals have changed in VS 2010 so that class is not longer useful.
Next time you run the program, set a breakpoint at line 177 of FileExportDialog.cpp. When you hit the breakpoint, single-step through the code and watch the crashes and asserts as the obsolete code attempts to do things it should never have been doing in the first place.
Recommend you dump the CFileExportDialog class and rewrite your program using the standard MFC CFileDialog class.