Crash when calling virtual function - c++

Question

Ok this is a really weird problem. I wanna start off by saying that I'm not a beginner in c++ and I'm certainly not advanced. I'm somewhere in the middle. What I'm trying to do is make a C++ OOP wrapper library (dll) of the Win32 API. Here are the classes of my library. I compiled it with Mingw using the command:
g++ -shared -o bin\win32oop.dll src\Application.cpp src\Form\Form.cpp -Wall
src\Application.h:
#ifndef WOOP_APPLICATION_H_
#define WOOP_APPLICATION_H_
namespace Woop
{
class Application
{
public:
bool Init(void);
virtual bool OnInit(void);
};
}
#endif // WOOP_APPLICATION_H_
src\Application.cpp
#include <windows.h>
#include "Application.h"
#include "Form\Form.h"
LRESULT CALLBACK WndProc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam);
namespace Woop
{
bool Application::Init(void)
{
WNDCLASSEX wc;
wc.cbSize = sizeof(WNDCLASSEX);
wc.style = 0;
wc.lpfnWndProc = WndProc;
wc.cbClsExtra = 0;
wc.cbWndExtra = 0;
wc.hInstance = GetModuleHandle(NULL);
wc.hIcon = LoadIcon(NULL, IDI_APPLICATION);
wc.hCursor = LoadCursor(NULL, IDC_ARROW);
wc.hbrBackground = (HBRUSH)(COLOR_WINDOW + 1);
wc.lpszMenuName = NULL;
wc.lpszClassName = "woop";
wc.hIconSm = LoadIcon(NULL, IDI_APPLICATION);
if(RegisterClassEx(&wc) == 0)
{
MessageBox(NULL, "Window Registration Failed!", "Error!", MB_ICONEXCLAMATION | MB_OK);
return false;
}
this->OnInit();
return true;
}
bool Application::OnInit(void)
{
return true;
}
}
LRESULT CALLBACK WndProc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
Woop::Form *wnd = 0;
if (uMsg == WM_NCCREATE)
{
SetWindowLong (hwnd, GWL_USERDATA, long((LPCREATESTRUCT(lParam))->lpCreateParams));
}
wnd = (Woop::Form *)(GetWindowLong (hwnd, GWL_USERDATA));
if (wnd) return wnd->WndProc(hwnd, uMsg, wParam, lParam);
return ::DefWindowProc (hwnd, uMsg, wParam, lParam);
}
src\Form\Form.h
#ifndef WOOP_FORM_FORM_H_
#define WOOP_FORM_FORM_H_
namespace Woop
{
class Form
{
public:
bool Show(void);
virtual LRESULT WndProc(HWND, UINT, WPARAM, LPARAM);
protected:
HWND _handle;
};
}
#endif // WOOP_FORM_FORM_H_
src\Form\Form.cpp
#include <windows.h>
#include "Form.h"
namespace Woop
{
bool Form::Show(void)
{
_handle = CreateWindowEx(WS_EX_CLIENTEDGE, "woop", "", WS_OVERLAPPEDWINDOW, CW_USEDEFAULT, CW_USEDEFAULT, 240, 120, NULL, NULL, GetModuleHandle(NULL), this);
if(_handle == NULL)
{
MessageBox(NULL, "Window Creation Failed!", "Error!", MB_ICONEXCLAMATION | MB_OK);
return false;
}
ShowWindow(_handle, SW_SHOWNORMAL);
return true;
}
LRESULT Form::WndProc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
switch(uMsg)
{
case WM_DESTROY:
PostQuitMessage(0);
break;
}
return DefWindowProc(hwnd, uMsg, wParam, lParam);
}
}
Here is a the program that I'm testing the library with:
class SampleApp : public Woop::Application
{
bool OnInit(void)
{
Form form;
form.Show();
return true;
}
};
INT APIENTRY WinMain(HINSTANCE, HINSTANCE, LPSTR, INT)
{
SampleApp application;
if(application.Init() == false) return 0;
MSG Msg;
while(GetMessage(&Msg, NULL, 0, 0) > 0)
{
TranslateMessage(&Msg);
DispatchMessage(&Msg);
}
return 0;
}
Alright, now the problem. Do you see that virtual Window Procedure in the Form class? If I remove the virtual from the declaration, the program compiles and runs fine. But when I add it back, it crashes. The infamous "Don't Send" dialog comes up. I'm not sure when it crashes, i'll try to figure that out using MessageBox() (lol, it's what I get for not learning how to debug with gdb). I'm trying to make it so that I can make a class such as LoginForm and derive from Form and override the Window Procedure. I hope I explained the problem well enough :D. This might be a compiler bug or my stupidity :P. Anyways, thanks in advance.

Answer 1

The problem is here:
bool OnInit(void)
{
Form form;
form.Show();
return true;
}
The form object is destroyed when this method is returned.
So the this pointer that you stored when you call Show() is no longer valid.
_handle = CreateWindowEx(WS_EX_CLIENTEDGE, "woop", "", WS_OVERLAPPEDWINDOW,
CW_USEDEFAULT, CW_USEDEFAULT, 240, 120, NULL, NULL,
GetModuleHandle(NULL),
/* Here ----> */ this
);
When you try and do the dispatch it is getting reallyed screwed up because it is using the this pointer to work out the address of the virtual function to call.
The reason that it crasshes with virtual and not when you take virtual away is that virtual method address is calculated at runtime while the normal method address is planted at compile time.
When calculating the address of the virtual method the this pointer is dereferenced in some way (which in this case leads to UB) but because the object is been destroyed the data at that address has proably been re-used so the address you get for the function is some random junk and calling this will never be good.
A simple solution is to make the form part of the application object.
Thus its lifetime is the same as the application:
class SampleApp : public Woop::Application
{
Form form;
bool OnInit(void)
{
form.Show();
return true;
}
};

Answer 2

wc.lpfnWndProc = WndProc;
That cannot work in the general case, although it is not obvious where that WndProc is located. Windows is not going to supply the "this" pointer that an instance method needs when it makes the callback. You are getting away with it right now because you don't access any members of the Form class in your Form::WndProc() method. It works by accident without the virtual keyword but that luck will quickly run out once you start touching members. That will bomb with an AccessViolation exception.
You need to make the Form::WndProc() method a static method.
Making it virtual is going to require you writing code that maps the HWND to a Form instance. This is a pretty standard feature of any class library that wraps the Win32 API. There's lots of value in not having to re-invent that wheel.

Answer 3

I don't know if it is the problem here, but a virtual function is always called indirectly. This means the object is accessed to read the virtual function table, before the virtual function is called. That means that if the object has been overwritten (already deleted, buffer overflow on heap or stack etc..), virtual methods are more likely to crash than others.
This is especially true because most of the member variables are not causing a crash when they are corrupted, so sometimes you can oversee a heap / stack corruption problem easily.
BTW: Starting debugging with gdb is quite simple sometimes: Just load it in gdb (gdb myprogram) and enter run. When the programm crashes get the backtrace with "bt" and you should see if the crash occured inside the virtual method or when calling it.