When building a dynamic library (in C++), we can select Multi-threaded Debug (/MTd) or Multi-threaded Debug DLL (/MDd) run time library for Windows. If we select Multi-threaded Debug, then the created dynamic library will be in charge of memory allocation for all the variables that are used in the library. Therefore, the following example will show a case where /MDd setting will work while /MTd setting will fail:
my_dll.h
class MY_EXPORT DllClass
{
public:
std::vector<int> abcWorkable;
void create_new_input_workable();
};
my_dll.cpp
void DllClass::create_new_input_workable()
{
abcWorkable.push_back(3);
abcWorkable.push_back(4);
}
main.cpp
int main(void)
{
DllClass mine;
//mine.abcWorkable.reserve(20);
mine.create_new_input_workable();
return 0;
}
There are two work-around solutions to this problem: one is using static library rather than the dynamic one and the other is to allocate memory only in the dynamic library or the executable, for example if we change main.cpp:
int main(void)
{
DllClass mine;
mine.abcWorkable.reserve(20);
mine.create_new_input_workable();
return 0;
}
It will work as this time the variable std::vector<int> abcWorkable is allocated memory in the executable program. However, if the variable insider the class (in the dll) is difficult to allocate memory in the executable, then this solution may fail. I also take another example:
my_dll.h
class MY_EXPORT DllClass
{
public:
std::list<std::vector<int> > myContainer;
void create_new_input();
}
my_dll.cpp
void DllClass::create_new_input()
{
std::vector<int> abc;
abc.push_back(2);
abc.push_back(3);
myContainer.push_back(abc);
}
main.cpp
int main()
{
DllClass mine;
mine.create_new_input();
std::list<std::vector<int> >::iterator it = mine.myContainer.begin();
std::list<std::vector<int> >::iterator itEnd = mine.myContainer.end();
while(it != itEnd)
{
for(int i=0; i<(*it).size(); i++)
std::cout<<(*it)[i]<<std::endl;
it++;
}
return 0;
}
It is impossible to allocate memory before hand for the variable std::list<std::vector<int> > myContainer, then my question is how I can deal with this situation? (Except using static library) Thanks.
When you build with /MT then every module will have its own copy of the CRT. With its own global variables like errno and its own memory allocator that uses its own heap. The final program will run with multiple CRT copies.
This is very troublesome, particularly with your code. Because you return an std::string by value from the DLL. Which was allocated on the heap that the CRT copy that the DLL uses. This object needs to be released again by the caller. But it uses a different heap and cannot possibly release the object. Kaboom.
Using /MD is a hard requirement here. Which ensures that every module uses the same CRT copy and only a single implementation is present in the final program. As long as they were built with the same CRT version anyway.
Related
UPD (SOLVED): This issue has been solved by this - In short, the case CMAKE will produce visiable UNIQUE objects, and either -fvisibility=hidden (then add attribute visibility=default manually) or --no-gnu-unique can avoid the problem.
I found this when importing a large project into myselfs, and the following codes should be a minimal demo to illustrate it:
/* ---- header.h ---- */
struct A {
typedef (void (*func_t)();
static void empty() {}
static constexpr func_t funcs[1] = {empty};
const func_t *func_list;
A() { func_list = funcs; }
};
struct B {
B();
};
struct X {
A a;
B b;
};
/* ----- header.cpp ----- */
#include "header.h"
B::B() {}
/* ----- main.cpp ----- */
#include "header.h"
extern "C" {
void test() {
auto x = new X;
delete x;
// print(version_nubmer);
}
}
The library is built from CMake
add_library(main SHARED main.cpp header.cpp)
target_compile_options(main PRIVATE -fPIC)
Then I use a program that sequentially calls dlopen(); dlsym(test)(); dlclose(); dlopen(); dlsym(test)(); on this library (with RTLD_LAZY | RTLZ_LOCAL), where before the second dlopen(),
I change the version number to a different value, expecting it print the updated one. But, it doesn't. I still see it give the old value, which means dlclose() does not really detach the lib.
If I set CMAKE_INTERPROCEDURAL_OPTIMIZATION to ON, or manually build it from command line (e.g., g++ -o libmain.so -fPIC -shared main.cpp header.cpp) the observation disapears. Can anyone tell the difference here?
UPD: Between the dlclose() and second dlopen(), we insert something to block the program like scanf() or getchar(), and continue the program only after we rebuilt the lib. This will ensure the reproduction to the three cases (CMAKE, CMAKE+lto, cmd).
You can not rely on dlclose() to unload any library.
Per POSIX (bolding mine):
The dlclose() function shall inform the system that the symbol table handle specified by handle is no longer needed by the application.
An application writer may use dlclose() to make a statement of intent on the part of the process, but this statement does not create any requirement upon the implementation. ...
Assuming you're running on Linux:
A successful return from dlclose() does not guarantee that the
symbols associated with handle are removed from the caller's
address space.
I have a huge map that is needed in multiple shared objects ..
//mymap.h
extern const std::unordered_map<int, std::string> gMAP;
std::string getname(int i) { return gMAP[i]; }
//mymap.cpp
extern const std::unordered_map<int, std::string> gMAP = {
{1, "one" },
{2, "two" },
{3, "three" },
...
}
And then I have library1 (lib1.so):
//CMakeLists.txt lib1.so
add_library(lib1 SHARED lib1.cpp mymap.cpp)
//lib1.cpp
#include "mymap.h"
void lib1_dosomething(int i) { std::cout << getname(i) << std::endl; }
And then I have library2 (lib2.so):
//CMakeLists.txt lib2.so
add_library(lib2 SHARED lib2.cpp mymap.cpp)
//lib2.cpp
#include "mymap.h"
void lib1_dosomething(int i) { std::cout << getname(i) << std::endl; }
This is simply pseudo code of our complex and much bigger system. Note that not all the libraries in our system need the map. Only very few (which is why I went for compiling in the project itself).
My understanding of making it this way is as follows:
Each library that compiles "mymap.cpp" has a 'new' instance of the map
There shall be no conflicts when all the libraries are loaded into memory when the system runs. And each library creates the global map when it is loaded and destroys it when it is unloaded from memory.
So my first question is: Is this understanding correct ?
But, I see a problem during the shutdown of our system. I noticed that each library creates an instance as expected, and destroys its instance during unloading. But one library seems to call the destructor of the std::unordered_map twice. And as a result we see a deallocation error (double free or corruption (!prev)), which i cannot explain why and what is causing this. Any ideas ?
I am thinking of compiling the map in a shared library of itself, so that we only have absolutely one instance throughout the system, but i want to understand why I see the "double free or corruption (!prev)" error.
I've been porting some c++ app from Visual Studio 2013 to Visual Studio 2017. Aside from the plethora of new warnings that I had to fix, the compilation and linking went okay.
However, when running the app, it 'stalled' when trying to re-enter the constructor of a singleton (when successive function calls form a loop back to the constructor). It seems that this behaviour was okay in VS2013, but is no longer valid in VS2017. There is no error message.
I'm aware of all the bad things related to singletons, and that there should at least not be loops. The question is not there.
Is there a way to tell the VS2017 compiler that I'd like to shoot myself in the foot, and allow the same behaviour that was there in VS2013?
I don't have access to the code that causes this behaviour because it comes from a third-party library, this is why I can't 'just fix it', unfortunately.
Here is an example which works in VS2013, but doesn't work in VS2017:
main.cpp
#include "Singleton.h";
int
main( void )
{
std::cout << "let's do this!" << std::endl;
int two = Singleton::GetReference().getTwo();
std::cout << "ok" << std::endl;
return 0;
}
Singleton.h
#pragma once
class Stuff;
class Singleton
{
public:
static Singleton& GetReference();
int getTwo() { return 2; }
private:
Singleton();
Stuff* stuff;
};
Singleton.cpp
#include "Singleton.h"
#include "Stuff.h"
Singleton&
Singleton::GetReference() {
static Singleton theInstance;
return theInstance;
}
Singleton::Singleton()
{
stuff = new Stuff();
}
Stuff.h
#pragma once
class Stuff
{
public:
Stuff();
private:
int two;
};
Stuff.cpp
#include "Stuff.h"
#include "Singleton.h"
Stuff::Stuff()
{
two = Singleton::GetReference().getTwo();
}
In the code above, when step-by-step debugging, the first time we get on the line static Singleton theInstance; will work as expected, but the second time, a F11 will go to the file thread_safe_statics.cpp, into the method extern "C" void __cdecl _Init_thread_header(int* const pOnce). A Shift+F11 will exit the method and the program will wait indefinitely at the line specified (observed when pausing the program from the debugger).
PS
This issue probably occurs in Visual Studio 2015 too, as the documentation linked from the accepted answer mentions VS2015.
/Zc:threadSafeInit-
The general "Conformance" page is MSDN: Conformance, which details which new features you can disable.
I needed the code for sizedDealloc, where my new compiler was creating a sized new operator for a library which broke older compiled expectations.
As this is a compile flag, at least some of the code would be in your control, and you should be able to unravel the beast.
The constructor Stuff::Stuff is calling a function on an incompletely constructed object.
That would create "Undefined behavior". If the value "2" is not set till the end of the constructor (for example).
Probably the Singleton needs to be split into 2, one which delivers the early static data (e.g. 2).
The second which delivers the held object Stuff. Stuff would only rely on the first, which would break the deadlock.
Alternatively, a second constructor to Stuff which told it which object to use, and was called from the Singleton::Singleton
The MSDN article to disable "Magic Statics" MSDN : disable threadsafe static initialization
I am now using Boost Unit Test to perform unit test for my project. Every time I run the unit test, I got a memory stack problem. I debug into the source code of BOOST library, and I find that the problem comes from invoking the following codes in unit_test_suite.hpp file:
void
traverse_test_tree( test_unit_id id, test_tree_visitor& V )
{
global_i = global_i + 1;
std::cout<<global_i<<std::endl;
if( ut_detail::test_id_2_unit_type( id ) == tut_case )
traverse_test_tree( framework::get<test_case>( id ), V );
else
traverse_test_tree( framework::get<test_suite>( id ), V );
}
The error information I have obtained from VC10 is:
Unhandled exception at 0x779815de in TestApplication.exe: 0xC00000FD: Stack overflow.
I was wondering what's wrong with the test program. Thanks!
EDIT Based on the suggestions I looked through my codes, and very strange things happen: if the test suite is defined in same program with main(), it works; however, if the test suite is from a .dll, the error will occur. I list the following codes to illustrate my problem:
boost::unit_test::test_suite* main_global_test_suite;
void Hellotestdll()
{
int i= 1;
int j= 2;
BOOST_CHECK(i == j);
}
boost::unit_test::test_suite* get_abc_test_suite()
{
test_suite* ts = BOOST_TEST_SUITE( "unit_geometric" );
ts->add( BOOST_TEST_CASE( &Hellotestdll ) );
return ts;
}
int main( int argc, char* argv[] )
{
try
{
/**
* Step 1. obtain options
*/
char* optionLine[1024];
int len;
len = obtain_options(optionLine, argc, argv);
/**
* Step 2. perform unit test based on user's options
*/
int test_status=0;
main_global_test_suite = get_abc_test_suite();
test_status = unit_test_main(run_global_test_suite, len, optionLine);
return test_status;
}
catch(std::exception& e)
{
std::cout << e.what() << std::endl;
return 1;
}
catch (const std::string& s)
{
std::cout << s << std::endl;
return 1;
}
catch (...)
{
return 1;
}
}
The above codes work very well. But if the test suite is from a .dll, for example:
// dll_header.h
namespace abc
{
ABC_EXPORT boost::unit_test::test_suite* get_geometric_test_suite();
}
// dll_header.cpp
namespace abc
{
using namespace boost;
using namespace boost::unit_test;
void Hellotestdllabc()
{
int i= 1;
int j= 2;
BOOST_CHECK(i == j);
}
boost::unit_test::test_suite* get_abc_test_suite()
{
test_suite* ts = BOOST_TEST_SUITE( "unit_abc" );
ts->add( BOOST_TEST_CASE( &Hellotestdllabc ) );
return ts;
}
}
Then if I invoke this test suite, with the following codes:
int main( int argc, char* argv[] )
{
............
/**
* Step 2. perform unit test based on user's options
*/
int test_status=0;
main_global_test_suite = abc::get_abc_test_suite();
test_status = unit_test_main(run_global_test_suite, len, optionLine);
return test_status;
}
The annoying stack overflow error will happen.
Summery of the Problems
(1) boost dll with MDd (Succeed)
If I link the boost unit test library (with the definition -DBOOST_ALL_DYN_LINK -DBOOST_TEST_NO_MAIN -DBOOST_TEST_DYN_LINK -DBOOST_ALL_NO_LIB)and the running executable program with the same dynamic run-time library (Multi-thread Debug Dll (MDd)), it will work.
(2) boost dll with MTd (Failed)
If the boost unit test library (with the definition -DBOOST_ALL_DYN_LINK -DBOOST_TEST_NO_MAIN -DBOOST_TEST_DYN_LINK -DBOOST_ALL_NO_LIB)and the running executable program are compiled and link with the same static run-time libray (Multi-thred Debu (MTd)), I will have a crash, but the crash is different from the one I reported above:
(3) boost static lib with MDd (Failed)
If the boost is built as a static library (with the definition of -DBOOST_TEST_NO_MAIN -DBOOST_ALL_NO_LIB), and both the boost library and the executable program are built with the same dynamic run-time library (MDd). The following crash will happen:
(4) boost static lib with MTd (Failed)
If the boost is built as a static library (with the definition of -DBOOST_TEST_NO_MAIN -DBOOST_ALL_NO_LIB), and both the boost library and the executable program are built with the same static run-time library (MTd). The following crash will happen:
ABC_EXPORT boost::unit_test::test_suite* get_geometric_test_suite();
The point of unit tests is to find problems in code early. That worked, it is just that you found the problem very early. Too early to even allow the unit test to run properly.
Functions in a DLL that returns pointers to C++ objects are a problem in general. It will only come to a good end when the layout of the C++ object exactly matches the assumptions made by the compiler when it compiled both your DLL and your EXE. And that the object lives on a heap that both modules have access to, required since the DLL creates the object and your EXE needs to delete it.
To allow the object to be properly deleted, both the DLL and the EXE must share the same CRT version. You will get into trouble when you build your program with /MT, asking for the static version of the CRT. The relevant compiler setting is C/C++, Code Generation, Runtime library setting. Your Debug configuration must use /MDd, your Release configuration must use /MD. For both the EXE and the DLL project, as well as the Boost library when it was compiled. If it is /MTd and /MT then the DLL will have its own copy of the CRT linked into it and will use its own heap to allocate from. The EXE cannot properly delete the object since it uses another heap. Doing so anyway will produce undefined behavior. Anything can happen, you tend to only get lucky when you run your program on a version of Windows that's newer than XP. Vista and up will use a debug heap when you run your unit test with a debugger attached, it invokes a breakpoint when it notices that the pointer passed to ::operator delete is invalid. Be sure to allow the linker to automatically find the correct Boost .lib to link to, don't force it yourself.
Object layout is more likely your problem, unfortunately much harder to diagnose. You stay out of trouble by building your EXE and DLL with the exact same compiler settings. With the additional requirement that they must match the settings that were used to build the Boost libraries. Which certainly is the difficult part, that requires a time machine. Particularly the _HAS_ITERATOR_DEBUGGING macro is a troublemaker, basic STL classes like std::vector will have a different layout that depends on the value of that macro.
I realize this is very vague, but there isn't enough info in the question to truly diagnose this issue. A very basic check you can do is putting the returned boost::unit_test::test_suite pointer in a watch expression. If you suddenly see the members of that object change when you step into Boost code then you know you have an object layout problem. What happens next is highly unpredictable, a stack overflow is certainly possible. Yet another diagnostic is to use the Debug + Windows + Registers window. Ensure that the ESP register value is stable when you step over functions.
I'm running into a strange crash. I am trying to separate out various modules of my Application (MFC based, developed in VS2005) into DLLs. Following is the skeletal code of how I'm trying to achieve it:
In a common Header file(say base.h):
class Base {
vector<message> messages;
...
...
};
In a header file in DLL source code(say class.h):
class Derived : public Base {
private:
int hoo();
...
public:
void foo();
int goo();
...
};
extern "C" __declspec (dllexport) Derived* CreateDerived();
In class.cpp
Derived* CreateDerived()
{
return new Derived;
}
In a file in main Application code:
#include "base.h"
#include "class.h"
typedef Derived* (*DerivedCreator)();
...
...
void LoadDll()
{
//DLL Load Code...
...
...
DerivedCreator creator = reinterpret_cast<DerivedCreator>(::GetProcAddress(dllHandle, "CreateDerived"));
Derived* pDerived = creator();
pDerived->messages.push_back(message("xyz"));//Crashes here...
}
The problem is the code craches the moment I try to access the vector member of the Base class. This only happens in Release mode. It works fine in Debug mode. The error message that i get when I execute it from Visual Studio in Release mode is:
"Microsoft Visual Studio C Runtime Library has detected a fatal error in Samsung SSD Magician.exe.
Press Break to debug the program or Continue to terminate the program."
But when I execute the release binary directly and attach the debugger to it, I get an Access Violation. At this point if I check the vector in debugger, it shows 6-digit entries in it, none of them readable. I'm able to see correct values of rest of the members of Base class in the Derived pointer.
Any help would be much appreciated.
It's dangerous to pass stl containers across a DLL boundary.
The reason here is that each module (the main application and the DLL) has it's own instance of the heap. If you allocate dynamic memory in the context of DLL, then pass the pointer to the application and release that memory in the context of the application, that causes heap corruption.
That is exactly what happens in your example.
Derived* pDerived = creator();
CreateDerived is called.
Derived* CreateDerived()
{
return new Derived;
}
new Derived allocates memory in DLL heap.
pDerived->messages.push_back(message("xyz"));
Inside push_back, an additional memory is allocated for Base::messages, and that allocation is done on the application heap. Crash!
A conclusion is that you need to rethink the DLL interface in order to perform all operation on the vector only inside the DLL.