There is a large legacy project I have to maintain, which I recently upgraded from Visual Studio 2008 to Visual Studio 2012. As it is a COM server and a OCX control, creating all the typelib stuff etc. resulted in some problems that I managed to solve. However, when I run the Release build now I frequently get crashes.
I followed some advise I found here on SO and was able to track down the crash to the following piece of code:
int Phx2Preview::ClearOvlElementList() {
for (int i = 0; i < (int)m_vOvlElements.size(); i++) {
P_SAFE_DELETE(m_vOvlElements[i].pPolyOrig); // <- code crashes here
P_SAFE_DELETE(m_vOvlElements[i].pPolyDispl);
}
m_vOvlElements.clear();
m_vRefElemList.clear();
m_pRefElemSelected = NULL;
return PHXE_NO_ERROR;
}
P_SAFE_DELETE is a macro that checks if the pointer is null, and in case it's not deletes and sets it to null. The actual vector elements are created like this:
if (v1) {
tNew.pPolyOrig = new CInPolygon();
tNew.pPolyDispl = new CInPolygon();
tNew.pPolyOrig->FromSafeArray(v1);
tNew.pPolyOrig->Rotate(NULLPOINT, m_nTurnAngle*__pi/180.);
tNew.eType = (overlayET)type;
tNew.nImagenr = nImageNr;
m_vOvlElements.push_back(tNew);
}
Now, the thing is that CInPolygon is a class from an external library which is created with Visual C++ 7.1. The P_SAFE_DELETE is also defined in a header from that library. From here I know that mixing different runtime versions is bad, and this question lets me suspect that this mixing may be responsible from the crash.
My question is: Why does it happen? After all, since both new and delete are called from the same place, no actual objects are passed between the different CRTs. Also, when the OCX is compiled using Visual Studio 2008, no problems occur. Is this due to pure luck? I guess the basic issue is existing in that setting, too. And, well, what can I do to solve to problem? Switch back to VS2008?
Edit:
As asked: The destructor of CInPolygon is just
CInPolygon::~CInPolygon(void) {
m_vPoints.clear();
}
here the m_vPoints is a std::vector<..> defined in the class. Maybe I should mention that CInPolygon inherits from that:
interface IRoi {
virtual ~IRoi() {
return;
}
public:
// other stuff
};
(Didn't even know that interface was a valid keyword in plain C++...) Could it be that the fact that the base class destructor is defined in the header is causing the problem? After all, that header is also known to the host programm..
tNew.pPolyOrig = new CInPolygon();
Yes, this is guaranteed to fail. Short from having different allocators in your program, your host program cannot possibly compute the size of the CInPolygon object correctly. It uses an entirely different implementation of std::vector. It was significantly rewritten in VS2012 to take advantage of C++11. Inevitably, the code in the library using the old version of vector will corrupt the heap.
You must rebuild the library as well, using the exact same version of the compiler with the exact same settings.
Related
I have a C++ client to a C++/CLI DLL, which initializes a series of C# dlls.
This used to work. The code that is failing has not changed. The code that has changed is not called before the exception is thrown. My compile environment has changed, but recompiling on a machine with an environment similar to my old one still failed. (EDIT: as we see in the answer this is not entirely true, I was only recompiling the library in the old environment, not the library and client together. The client projects had been upgraded and couldn't easily go back.)
Someone besides me recompiled the library, and we started getting memory management issues. The pointer passed in as a String must not be in the bottom 64K of the process's address space. I recompiled it, and all worked well with no code changes. (Alarm #1) Recently it was recompiled, and memory management issues with strings re-appeared, and this time they're not going away. The new error is Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
I'm pretty sure the problem is not located where I see the exception, the code didn't change between the successful and failing builds, but we should review that to be complete. Ignore the names of things, I don't have much control over the design of what it's doing with these strings. And sorry for the confusion, but note that _bridge and bridge are different things. Lots of lines of code missing because this question is already too long.
Defined in library:
struct Config
{
std::string aye;
std::string bee;
std::string sea;
};
extern "C" __declspec(dllexport) BridgeBase_I* __stdcall Bridge_GetConfiguredDefaultsImplementationPointer(
const std::vector<Config> & newConfigs, /**< new configurations to apply **/
std::string configFolderPath, /**< folder to write config files in **/
std::string defaultConfigFolderPath, /**< folder to find default config files in **/
std::string & status /**< output status of config parse **/
);
In client function:
GatewayWrapper::Config bridge;
std::string configPath("./config");
std::string defaultPath("./config/default");
GatewayWrapper::Config gwtransport;
bridge.aye = "bridged.dll";
bridge.bee = "1.0";
bridge.sea = "";
configs.push_back(bridge);
_bridge = GatewayWrapper::Bridge_GetConfiguredDefaultsImplementationPointer(configs, configPath, defaultPath, status);
Note that call to library that is crashing is in the same scope as the vector declaration, struct declaration, string assignment and vector push-back
There are no threading calls in this section of code, but there are other threads running doing other things. There is no pointer math here, there are no heap allocations in the area except perhaps inside the standard library.
I can run the code up to the Bridge_GetConfiguredDefaultsImplementationPointer call in the debugger, and the contents of the configs vector look correct in the debugger.
Back in the library, in the first sub function, where the debugger don't shine, I've broken down the failing statement into several console prints.
System::String^ temp
List<CConfig^>^ configs = gcnew List<CConfig ^>((INT32)newConfigs.size());
for( int i = 0; i< newConfigs.size(); i++)
{
std::cout << newConfigs[i].aye<< std::flush; // prints
std::cout << newConfigs[i].aye.c_str() << std::flush; // prints
temp = gcnew System::String(newConfigs[i].aye.c_str());
System::Console::WriteLine(temp); // prints
std::cout << "Testing string creation" << std::endl; // prints
std::cout << newConfigs[i].bee << std::flush; // crashes here
}
I get the same exception on access of bee if I move the newConfigs[i].bee out above the assignment of temp or comment out the list declaration/assignment.
Just for reference that the std::string in a struct in a vector should have arrived at it's destination ok
Is std::vector copying the objects with a push_back?
std::string in struct - Copy/assignment issues?
http://www.cplusplus.com/reference/vector/vector/operator=/
Assign one struct to another in C
Why this exception is not caught by my try/catch
https://stackoverflow.com/a/918891/2091951
Generic AccessViolationException related questions
How to handle AccessViolationException
Programs randomly getting System.AccessViolationException
https://connect.microsoft.com/VisualStudio/feedback/details/819552/visual-studio-debugger-throws-accessviolationexception
finding the cause of System.AccessViolationException
https://msdn.microsoft.com/en-us/library/ms164911.aspx
Catching access violation exceptions?
AccessViolationException when using C++ DLL from C#
Suggestions in above questions
Change to .net 3.5, change target platform - these solutions could have serious issues with a large mult-project solution.
HandleProcessCorruptedStateExceptions - does not work in C++, this decoration is for C#, catching this error could be a very bad idea anyway
Change legacyCorruptedStateExceptionsPolicy - this is about catching the error, not preventing it
Install .NET 4.5.2 - can't, already have 4.6.1. Installing 4.6.2 did not help. Recompiling on a different machine that didn't have 4.5 or 4.6 installed did not help. (Despite this used to compile and run on my machine before installing Visual Studio 2013, which strongly suggests the .NET library is an issue?)
VSDebug_DisableManagedReturnValue - I only see this mentioned in relation to a specific crash in the debugger, and the help from Microsoft says that other AccessViolationException issues are likely unrelated. (http://connect.microsoft.com/VisualStudio/feedbackdetail/view/819552/visual-studio-debugger-throws-accessviolationexception)
Change Comodo Firewall settings - I don't use this software
Change all the code to managed memory - Not an option. The overall design of calling C# from C++ through C++/CLI is resistant to change. I was specifically asked to design it this way to leverage existing C# code from existing C++ code.
Make sure memory is allocated - memory should be allocated on the stack in the C++ client. I've attempted to make the vector be not a reference parameter, to force a vector copy into explicitly library controlled memory space, did not help.
"Access violations in unmanaged code that bubble up to managed code are always wrapped in an AccessViolationException." - Fact, not a solution.
but it was the mismatch, not the specific version that was the problem
Yes, that's black letter law in VS. You unfortunately just missed the counter-measures that were built into VS2012 to turn this mistake into a diagnosable linker error. Previously (and in VS2010), the CRT would allocate its own heap with HeapAlloc(). Now (in VS2013), it uses the default process heap, the one returned by the GetProcessHeap().
Which is in itself enough to trigger an AVE when you run your app on Vista or higher, allocating memory from one heap and releasing it from another triggers an AVE at runtime, a debugger break when you debug with the Debug Heap enabled.
This is not where it ends, another significant issue is that the std::string object layout is not the same between the versions. Something you can discover with a little test program:
#include <string>
#include <iostream>
int main()
{
std::cout << sizeof(std::string) << std::endl;
return 0;
}
VS2010 Debug : 32
VS2010 Release : 28
VS2013 Debug : 28
VS2013 Release : 24
I have a vague memory of Stephen Lavavej mentioning the std::string object size reduction, very much presented as a feature, but I can't find it back. The extra 4 bytes in the Debug build is caused by the iterator debugging feature, it can be disabled with _HAS_ITERATOR_DEBUGGING=0 in the Preprocessor Definitions. Not a feature you'd quickly want to throw away but it makes mixing Debug and Release builds of the EXE and its DLLs quite lethal.
Needless to say, the different object sizes seriously bytes when the Config object is created in a DLL built with one version of the standard C++ library and used in another. Many mishaps, the most basic one is that the code will simply read the Config::bee member from the wrong offset. An AVE is (almost) guaranteed. Lots more misery when code allocates the small flavor of the Config object but writes the large flavor of std::string, that randomly corrupts the heap or the stack frame.
Don't mix.
I believe 2013 introduced a lot of changes in the internal data formats of STL containers, as part of a push to reduce memory usage and improve perf. I know vector became smaller, and string is basically a glorified vector<char>.
Microsoft acknowledges the incompatibility:
"To enable new optimizations and debugging checks, the Visual Studio
implementation of the C++ Standard Library intentionally breaks binary
compatibility from one version to the next. Therefore, when the C++
Standard Library is used, object files and static libraries that are
compiled by using different versions can't be mixed in one binary (EXE
or DLL), and C++ Standard Library objects can't be passed between
binaries that are compiled by using different versions."
If you're going to pass std::* objects between executables and/or DLLs, you absolutely must ensure that they are using the same version of the compiler. It would be well-advised to have your client and its DLLs negotiate in some way at startup, comparing any available versions (e.g. compiler version + flags, boost version, directx version, etc.) so that you catch errors like this quickly. Think of it as a cross-module assert.
If you want to confirm that this is the issue, you could pick a few of the data structures you're passing back and forth and check their sizes in the client vs. the DLLs. I suspect your Config class above would register differently in one of the fail cases.
I'd also like to mention that it is probably a bad idea in the first place to use smart containers in DLL calls. Unless you can guarantee that the app and DLL won't try to free or reallocate the internal buffers of the other's containers, you could easily run into heap corruption issues, since the app and DLL each have their own internal C++ heap. I think that behavior is considered undefined at best. Even passing const& arguments could still result in reallocation in rare cases, since const doesn't prevent a compiler from diddling with mutable internals.
You seem to have memory corruption. Microsoft Application Verifier is invaluable in finding corruption. Use it to find your bug:
Install it to your dev machine.
Add your exe to it.
Only select Basics\Heaps.
Press Save. It doesn't matter if you keep application verifier open.
Run your program a few times.
If it crashes, debug it and this time, the crash will point to your problem, not just some random location in your program.
PS: It's a great idea to have Application Verifier enabled at all times for your development project.
I'm seeing something weird happening with Visual Studio 2015 Community. Code that worked perfectly in VS2012 crashes at startup when ported to VS2015, before main is invoked: the classic symptoms of some static initialization mess. I have a few static variables, but used properly, with the standard "Construct On First Use" pattern, e.g.:
const int& i()
{
static int *v = new int(1);
return *v;
}
The code above causes an assert from the runtime while initializing the program (see image http://i.stack.imgur.com/nsQGS.png). Pressing retry simply quits the program: no call stack, no information whatsoever.
The code below however works flawlessly:
const int& i()
{
static int *v = nullptr;
if (v == nullptr)
v = new int(1);
return *v;
}
It seems to me that VS2015 is doing what looks like some (illegal) optimization (even in a debug build), by executing the static variable initialization code during program initialization, and not the first time that the function is called, as the C++ standard requires.
I tried several variations of the code above: class methods, free functions, different objects (std::vector, cv::Mat), always with the same result: the static pointer has to be initialized to null or the program doesn't start.
So... am I missing something or is VS2015 actually messing up badly?
UPDATE:
I spent some time trying to find the minimal setup that shows the problem, and this is what I got:
create a project using the "CLR empty project" template.
add a cpp file, containing just the offending function and main(). Main can be empty, it doesn't matter: the bug occurs BEFORE it is even reached.
add 'main' as entry point (or you get a link error).
The x86 version works, but the x64 doesn't.
For comparison, a project with the identical code but created as "Win32 console application" doesn't have the problem, even after adding the /CLR option. I can see differences in the vcxproj files, but none justifies the error, although one or more of them clearly IS the cause.
Is there a way to upload a zip with the project?
Well, #bogdan got it right. My project had a mix of settings that was neither /SUBSYSTEM:CONSOLE nor /SUBSYSTEM:WINDOWS. Once I fixed that, everything started to work as expected. My test projects had the same problem, I blame Microsoft for not having a clear "CLR windows app" C++ template any more in VS2015 (they want to push you to use C# for that, which makes sense most of the times, but not always).
I found this page particularly useful in explaining all the different settings that have to be consistent (it's not just /SUBSYSTEM...).
I wish I could mark #bogdan's comment as answer, but I can't see anything to do that.
Thanks Bogdan!
C++ class constructor can be inlined or not be inlined. However, I found a strange situation where only inline class constructor can avoid Visual Studio memory crash. The example is as follows:
dll.h
class _declspec(dllexport) Image
{
public:
Image();
virtual ~Image();
};
class _declspec(dllexport) Testimage:public Image
{
public:
Testimage();
virtual ~Testimage();
};
typedef std::auto_ptr<Testimage> TestimagePtr;
dll.cpp
#include "dll.h"
#include <assert.h>
Image::~Image()
{
std::cout<<"Image is being deleted."<<std::endl;
}
Image::Image()
{
}
Testimage::Testimage()
{
}
Testimage::~Testimage()
{
std::cout<<"Geoimage is being deleted."<<std::endl;
}
The dll library is compiled as a dynamic library, and it is statically linked to the C++ runtime library (Multi-threaded Debug (/MTd)). The executable program that runs the library is as follows:
int main()
{
TestimagePtr my_img(new Testimage());
return 0;
}
The executable program will invoke the dll library and it also statically links the runtime library. The problem I have is that when running the executable program the following error message appears:
However, when the class constructor in dll is inlined as the following codes show:
class _declspec(dllexport) Image
{
public:
Image();
virtual ~Image();
};
class _declspec(dllexport) Testimage:public Image
{
public:
Testimage()
{
}
virtual ~Testimage();
};
The crash will disappear. Could someone explain the reason behind? Thanks! By the way, I am using VC2010.
EDIT: The following situation also trigger the same crash
.
Situation 1
int main()
{
//TestimagePtr my_img(new Testimage());
Testimage *p_img;
p_img = new Testimage();
delete p_img;
return 0;
}
it is statically linked to the C++ runtime library (Multi-threaded Debug (/MTd)
This is a very problematic scenario in versions of Visual Studio prior to VS2012. The issue is that you have more than one version of the CRT loaded in your process. One used by your EXE, another used by the DLL. This can cause many subtle problems, and not so subtle problems like this crash.
The CRT has global state, stuff like errno and strtok() cannot work properly when that global state is updated by one copy of the CRT and read back by another copy. Relevant to your crash, a hidden global state variable is the heap that the CRT uses to allocate memory from. Functions like malloc() and ::operator new use that heap.
This goes wrong when objects are allocated by one copy of the CRT and released by another. The pointer that's passed to free() or ::operator delete belongs to the wrong heap. What happens next depends on your operating system. A silent memory leak in XP. In Vista and up, you program runs with the debug version of the memory manager enabled. Which triggers a breakpoint when you have a debugger attached to your process to tell you that there's a problem with the pointer. The dialog in your screenshot is the result. It isn't otherwise very clear to me how inlining the constructor could make a difference, the fundamental issue however is that your code invokes undefined behavior. Which has a knack for producing random outcomes.
There are two approaches available to solve this problem. The first one is the simple one, just build both your EXE and your DLL project with the /MD compile option instead. This selects the DLL version of the CRT. It is now shared by both modules and you'll only have a single copy of the CRT in your process. So there is no longer a problem with having one module allocating and another module releasing memory, the same heap is used.
This will work fine to solve your problem but can still become an issue later. A DLL tends to live a life of its own and may some day be used by another EXE that was built with a different version of the CRT. The CRT will now again not be shared since they'll use different versions of the DLL, invoking the exact same failure mode you are seeing today.
The only way to guarantee that this cannot happen is to design your DLL interface carefully. And ensure that there will never be a case where the DLL allocates memory that the client code needs to release. That requires giving up on a lot of C++ goodies. You for example can never write a function that returns a C++ object, like std::string. And you can never allow an exception to cross the module boundary. You are basically down to a C-style interface. Note how COM addresses this problem by using interface-based programming techniques and a class factory plus reference counting to solve the memory management problem.
VS2012 has a counter-measure against this problem, it has a CRT version that allocates from the default process heap. Which solves this particular problem, not otherwise a workaround for the global state issue for other runtime functions. And adds some new problems, a DLL compiled with /MT that gets unloaded that doesn't release all of its allocations now causes an unpluggable leak for example.
This is an ugly problem in C++, the language fundamentally misses an ABI specification that addresses problems like this. The notion of modules is entirely missing from the language specification. Being worked on today but not yet completed. Not simple to do, it is solved in other languages like Java and the .NET languages by specifying a virtual machine, providing a runtime environment where memory management is centralized. Not the kind of runtime environment that excites C++ programmers.
I tried to reproduce your problem in VC2010 and it doesn't crash. It works with a constructor inline or not. Your problem is probably not in what you write here.
Your project is too hard to open as it seams to have its file pathes set in absolute, probably because generated with CMake. (So the files are not found by the compiler).
The problem I see in your code is that you declare the exported classes with _declspec(dllexport) directly written.
You should have a #Define to do this, and the value should be _declspec(dllimport) when read from the exe compilation. Maybe the problem comes from that.
I am trying to migrate some static C++ libraries into DLLs with a C interface so I don't need to build a separate version of the library for every version of Visual Studio (i.e. CRT) we want to support. However, I do like the convenience of using STL objects for some of the function calls. I came up with something that seems to work, but was wondering if there may be some hidden things that I'm just not thinking of.
Here is what I came up with to get STL versions of functions while still maintaining Visual Studio independence.
Original library function:
//library.h
...
std::wstring GetSomeString();
...
StringGenerator* mStrGen; //assume forward declared for pimpl implementation
//library.cpp
std::wstring library::GetSomeString()
{
return mStrGen->GetString(); //returns a wstring;
}
First, I created a private function that would provide the C interface
//library.h
__declspec(dllexport) void GetSomeStringInternal(wchar_t* pSomeString);
//library.cpp
void library::GetSomeString(wchar_t*& pSomeString)
{
if(pSomeString!= nullptr) {
delete [] pSomeString; //assumes allocated by the DLL
}
std::wstring tmpString(mStrGen->GetString());
size_t stringLength(tmpString.size());
stringToReturn = new wchar_t[stringLength + 1];
wcscpy_s(pSomeString, stringLength + 1, tmpString.c_str());
}
Next, I added a private function that deallocates memory allocated by the DLL
//library.h
__declspec(dllexport) void FreeArray(void* arrayPtr);
//library.cpp
void library::FreeArray(void* arrayPtr)
{
if(arrayPtr) {
delete [] arrayPtr;
}
}
Finally, I converted the original C++ function returning a string into a function that calls the internal C interface function
//library.h
std::wstring GetSomeString()
{
std::wstring someString(L"");
wchar_t* pSomeString= NULL;
GetSomeStringInternal(pSomeString);
someString = pSomeString;
FreeArray(pSomeString);
return someString;
}
//library.cpp
//removed GetSomeString from cpp since it is defined in header
My thinking is that since the header will be compiled every time it is included, an application that uses a different version of the CRT will compile the function using its implementation of the CRT. All data passed into and out of the library uses a C interface to preserve compatibility and memory is allocated and freed by the library so you don't run into one version of the CRT trying to free memory from a different version.
It seems to perform as I intend:
The library can be used by programs compiled with multiple versions of Visual Studio.
There are no memory leaks or access violations
If I modify the code so that memory is allocated in the GetSomeString function in the header, I do get a memory access error when trying to free that memory.
The GetSomeString function is compiled by the library and included in the DLL, but it is never called since it is 1) not exported and 2) the compiling program will always choose its version since it is inlined.
Is there anything I am missing, or is this a valid way of going about providing a C++ interface to a library that is Visual Studio version independent?
Side note: I have run into some deletion issues if I have a program that uses a std::shared_ptr<library>, but haven't researched that issue enough and will probably have a follow up question on that problem.
One thing I could see becoming a problem is if you actually needed to pass large objects around by reference for performance reasons. You're dealing with the binary compatibility issues by copying all the data to and from a compatible format, which is fine until it becomes a performance issue.
We will convert a number of programs written in C++ and MFC in Visual Studio.NET 2002 to Visual Studio 2010. What problems can we expect to encounter? What changes are there in the libraries that are worth knowing?
MFC has had a number of breaking changes over those releases. All the changes are documented on MSDN, and usually they're pretty straightforward - function signature changes and the like (which can often be fixed simply by inspecting the compiler error message and working out what it wants instead).
I've been through this moving a project to VS 2008 and the two big ones were the "secure CRT" functions and the for loop scope change. (I can't remember precisely when that happened, but it might affect you.) Basically the compiler is your friend ... build the whole thing and it will find the problems, which you can then fix. You can suppress the secure CRT warnings but you might as well get them taken care of.
I'm not aware of any "I'm happy to compile but I won't do quite what I used to do at runtime, thus ruining your world" breaking changes in MFC or C++ over the last decade. So once you make the compiler happy you should be confident your app still works.
dynamic_cast will behave differently at runtime
class A
{
}
class B : public A
{
}
class C : public A
{
}
//...
C* c = new C();
//This used to work, i.e. didn't return NULL, with 2002
B* b = dynamic_cast<B*>(c); //... won't work any more --> returns NULL.