I'm using some 3rd party. I'm using it's shared library version, since the library is big (~60MB) and is used by several applications.
Is there a way at application startup to find out that release/debug version of library is used respectively for release/debug version of my application?
Longer description
The library which exposes C++ interface. One of API methods return std::vector<std::string>.
The problem when I compile my application in debug mode, debug version of the library should be used. Same for release. If incorrect version of the library is used application is crashed.
According to gcc (see http://gcc.gnu.org/onlinedocs/libstdc++/manual/bk01pt03ch17s04.html)
but with a mixed mode standard library
that could be using either debug-mode
or release-mode basic_string objects,
things get more complicated
P.S. 1
It looks like proposal of Timbo is a possible solution - use different soname for debug and release libraries. So, what should be passed to ./configure script to change library soname?
P.S. 2
My problem is not at link time, but rather at run time.
P.S. 3
Here is question demonstrating problem I is facing with.
The debug mode referenced here has nothing to do with debug or release build of your application. The STL debug mode is activated with -D_GLIBCXX_DEBUG and is a special checking mode.
It is very unlikely that the 3rd party library was in fact compiled with STL checking mode, but if it was, it would likely very promptly mention that your code should also be compiled with -D_GLIBCXX_DEBUG.
If the 3rd party library was not built with checking STL, then it is compatible with your code regardless of whether you are doing optimized or debug build.
Since you state that debug build of your code linked with optimized build of 3rd party library causes a crash, that crash is most likely caused by a bug in your code (or possibly by a bug in 3rd party library).
Valgrind and GDB are your friends.
I believe that you have misread the documentation at the link you provide. In particular, you've misunderstood its purpose -- that section is entitled "Goals", and describes a number of hypothetical designs for a C++ debug library and the consequences of those designs in order to explain the actual design choices that were made. The bits of text that follow the lines you quoted are describing the chaos that would result from a hypothetical implementation that had separate designs for release-mode and debug-mode strings. It goes on to say:
For this reason we cannot easily provide safe iterators for the std::basic_string class template, as it is present throughout the C++ standard library.
(Or, rephrasing that, providing a special "debug" version of string iterators is impossible.)
...
With the design of libstdc++ debug mode, we cannot effectively hide the differences between debug and release-mode strings from the user. Failure to hide the differences may result in unpredictable behavior, and for this reason we have opted to only perform basic_string changes that do not require ABI changes. The effect on users is expected to be minimal, as there are simple alternatives (e.g., __gnu_debug::basic_string), and the usability benefit we gain from the ability to mix debug- and release-compiled translation units is enormous.
In other words, the design of the debug and release modes in GCC's libstdc++ has rejected this hypothetical implementation with separate designs for the strings, specifically in order to allow cross-mode linking of the sort that you are worrying about how to avoid.
Thus, you should not have problems with compiling your library once, without -D_GLIBCXX_DEBUG (or with it, if for some reason you prefer), and linking it with either mode of your application. If you do have problems, it is due to a bug somewhere. [But see edit below! This is specific to std::string, not other containers!]
Edit: After this answer was accepted, I followed up in answering the follow-up question at std::vector<std::string> crash, and realized that the conclusion of this answer is incorrect. GCC's libstdc++ does clever things with strings to support "Per-use recompilation" (in which all uses of a given container object must be compiled with the same flags, but uses of the same container class within a program need not be compiled with the same flags), but that is not the same thing as complete "Per-unit compilation" that would provide the cross-linking ability you need. In particular, the documentation says of that cross-linking ability,
We believe that this level of recompilation is in fact not possible if we intend to supply safe iterators, leave the program semantics unchanged, and not regress in performance under release mode....
Thus, if you're passing containers across your library interface, you will need two separate libraries. Honestly, for this situation I've found that the easiest solution is just to install the two libraries into different directories (one for each variant -- and you'll want both to be separate from your main library directory). Alternately, you can rename the debug library file and then install it manually.
As a further suggestion -- you're presumably not running this in debug mode very often. It may be worth only compiling and linking the debug version statically into your application, so you don't have to worry about installing multiple dynamic libraries and keeping them straight at runtime.
Give the debug and release versions of the DLL different names and link the correct one through library dependency. Your application then wont start unless it finds the correct DLL.
This is the sort of check you should be doing in your build system. In your build script,
if you're building for release then link against the release library.
if you're building for debug then link against the debug library.
For instance, if you're using make:
release: $(OBJ)
$(CC) $(CXXFLAGS_RELEASE) $(foreach LIB,$(LIBS_RELEASE),-l$(LIB))
debug: $(OBJ)
$(CC) $(CXXFLAGS_DEBUG) $(foreach LIB,$(LIBS_DEBUG),-l$(LIB))
Related
Brief introduction:
I am refactoring win32 application, I use VS 2008.
The application consists of both of my own dll and 3-rd party dll`s.
Issue:
When I run the application in the debug mode and execute some action the error is raised: the application programm has triggered a breakpoint, heap is corrupted.
Actions undertaken:
I have searched the internet and found that this error may be because of different build flags (multi-threaded debug /MD and multi-threaded debug dll /MDd) were used for dlls within the project(which results that they use different c runtime libraries, and for each library own list for work with memory is maintained this therfore can lead to the heap corruption).
I have checked my dlls - they all have the same flag: debug multithreaded dll. So I think that one of the 3-rd party DLL maybe was built with multi-threaded debug flag.
Questions:
Is it possible to find out with what flag 3-rd party library was
built, if so how can I do this.
How can I sort you my issue of
different build flags?
Is my guess about that error is due to
different build flags is correct?
Is it possible to find out with what flag 3-rd party library was built, if so how can I do this
Yes. C or C++ DLLs built with Visual Studio versions 2005 and 2008 require a manifest that states what version of the C runtime DLL they need. You can see it with VS, File + Open + File, select the DLL and open the node labeled "RT_MANIFEST". Resource ID 2 is the manifest that states the type and version of the CRT. It should match yours. Export it to make it easier to read. If it is missing then it either wasn't built with /MD or used a completely different version of VS, which in itself is bad news.
How can I sort you my issue of different build flags?
You can't. You'll need to contact the 3rd party and request a build that's compatible with yours.
Is my guess about that error is due to different build flags is correct?
It is possible but not terribly likely. Having a mismatch does not automatically cause a crash, a programmer can certainly design the DLL interface so that's never an issue. You can typically tell from the function signature and documentation. The problem case is where the DLL allocates an object and you are supposed to release it. It will be obvious when the function returns a pointer. Or a standard C++ class library object like std::string. Less obvious is when it throws an exception. Such a problem is also highly repeatable, it will bomb consistently, not occasionally.
The biggest mistake you are making is asking this question here. You should be talking to a programmer employed by that 3rd party that has worked on this DLL. He'll know the exact answer to your questions and can very easily solve your problem. If you cannot get this kind of support then you should not be using these DLLs, they'll make your life miserable for a long time to come.
I know next to nothing about the linking process, and it almost always gets in the way when I am trying to start a new project or add a new library. Whenever I search for fixes to these type of errors, I will find people with a similar problem but rarely any sort of fix.
Is there any generalized way of going about finding what the problem is, and fixing it?
I'm using visual studio 2010, and am statically linking my libraries into my program. My problems always seem to stem from conflicts with LIBCMT(D).lib, MSVCRT(D).lib, and a few other libraries doublely defining certain functions. If it matters at all, my intent is to avoid using "managed" C++.
If your error is related to LIBCMT(D).lib and the like, usually that depends from the fact that you are linking against a library that uses a different CRT version than yours. The only real fix is to either use the library compiled for the same version of the CRT you use (often there is the "debug" and "release" version also for this reason), either (if you are desperate) change the CRT version you use to match the one of the library.
What is happening behind the scenes is that both your program and your library need the CRT functions to work correctly, and each one already links against it. If they are linking against the same version of it nothing bad happens (the linker sees that it's the same and doesn't complain), otherwise there are multiple conflicting implementations of the same functions, so the linker doesn't know which are right for which object modules (and also, since they are probably not binary compatible, internal data structures of the two CRTs will be incompatible).
The specific link errors you mentioned (with LIBCMT(D).lib, MSVCRT(D).lib libraries) are related to conflicts in code generation options between modules/libraries in your program.
When you compile a module, the compiler automatically inserts in the resulting .obj some references to the runtime libraries (LIBCMT&MSVCRT). Now, there is one version of these libraries for each code generation mode (I'm referring to the option at Configuration properties -> C/C++ -> Code Generation -> Runtime Library). So if you have two modules compiled with a different mode, each of them will reference a different version of the library, the linker will try to include both, and of course there'll be duplicated symbols, since essentially all the symbols are the same in these libraries, only their implementations differ.
The solution comes in three parts. First, make sure all the modules in a project use the same mode. Second, if you have dependencies between projects, all of them have to use the same mode. Third, if you use third-party libraries, you have to either know which mode they use (and adopt it) or be able to recompile them with the desired mode.
The last one is the most difficult. Sometimes, libraries come pre-compiled, and not always the provider gives information about the mode used. Worse, if you're using more than one third-party library, they may have conflicting modes. In those cases, you have no better option than trial-and-error.
Also notice that each Visual Studio version has its own set of runtime libraries, so when using third-party libraries you have to use those compiled with the same version of Visual Studio you're using. If the provider doesn't offer it, your only choice is to recompile yourself.
I have this job of implementing a library that provides a file-sharing feature.
This has already happened twice:
First, in a string in an if-else path, only the if path is being executed, but when i change a spelling in the else path, the software after a few minutes crashes in an std library. I verified with a debug attached that the lined changed was never being touched. When i reversed the change, it works nicely again.
Second, my software crashes on a std library again with the out-of-array check into a standard basic_string destructor.
I did everything, all library matched the _HAS_ITERATOR_DEBUGGING.
After 4 hours I discovered that the problematic file is TorrentFile.cpp/h.
If i add a function ( even though it is never called ), the program crashes at the end of that file, but if its not there, there's no bug. The code causing the problem:
std::vector<TorrentFileListPacket> TorrentFile::GetFileMap()
{
std::vector<TorrentFileListPacket> vFiles;
return vFiles;
};
If i comment this code out, the crash is gone.
This is really driving me crazy!
I've been a developer for 8 years, and I've never seen something like this before!
Additional Information
My memory is OK, I'm using Visual Studio 2010 with SP1 in Windows 7. The library is libTorrent from RasterBar and it links to boost. The software is using MFC.
This smells strongly of memory corruption in a totally different location from where you would expect from the crashes. Most likely adding and removing functions is changing the memory layout in such a way that causes the memory corruption's effects to be immediately visible or not.
Your best hope is something like Purify or Valgrind to hunt it down.
You probably want to make sure that all your object files and libraries are ABI compatible with each other.
Numerous compiler settings will change the ABI. Especially debug and release builds and iterator debugging. The struct layout for standard containers typically change when you enable iterator debugging (which I believe is on by default for all debug builds in msvc, and off for release builds).
So, if a single object file, static library or DLL that you link against is built with an incompatible configuration, you typically see very odd behaviors. With libtorrent you need to make sure you build the library with the same configuration as you link against it with. Many of the TORRENT_* defines will actually change some aspect of some struct layout or function call. Make sure you define the exact same set of those in your client as when building the library. One simple way of dealing with this problem is to simply pull all source files into your project and build everything together.
If you are using libtorrent as a DLL (or boost for that matter), are they compiled against the same C Run-Time?
Often when I run into this type of issue it is because I make a call into a library that was compiled with MinGW (which uses the CRT from VS6.0) or an older version of Visual Studio. If memory is allocated by the library and then free'd by your application, you will often get these types of errors in the destructor.
If you aren't sure, you can open the DLL in question in a tool like the Dependency Walker. Look for the dependency MSVCRT.DLL, MSVCR100.DLL, etc.
I am writing an C++ application that could be compiled under Linux (gcc 4.3) and Windows (MS VS08 Express).
My application uses third-party libraries,
On Linux , they are compiled as shared libraries, while
on Windows, there are two versions "Debug" and "Release".
I know that debug version provides extra support for debugging ( just like using -ggdb option in linux gcc, right? )
But I found that if my application is in debug version , the libraries must also be in debug version, otherwise the application will crash.
Why is there such a limit? It seems that there are no such limits in the linux world
Thank you very much!
The Debug configuration of your
program is compiled with full symbolic
debug information and no optimization.
Optimization complicates debugging,
because the relationship between
source code and generated instructions
is more complex.
The Release configuration of your
program contains no symbolic debug
information and is fully optimized.
Debug information can be generated in
PDB Files (C++) depending on the
compiler options used. Creating PDB
files can be very useful if you later
need to debug your release version.
Debug vs Release
It is also possible to debug your release build with the compiler flags.
Debugging Release Builds
Heap Layout - Guard Bytes to prevent overwriting
Compilation - Removing Assert/Debug Info
Pointer Support - Buffers around pointers to prevent seg faults
Optimization - Inline Functions
Common Problems When Creating Release Builds
To elaborate on Martin Tornwall. The various libraries linked when in Debug or Release
LIBCPMT.LIB, Multithreaded, static link, /MT, _MT
LIBCPMTD.LIB, Multithreaded, static link, /MTd, _DEBUG, _MT
CRT Libraries
Most likely, the Release and Debug versions are linked against different versions of the C++ Runtime library. Usually, a Debug build links against the "Multithreaded Debug DLL" runtime, whereas a Release build will generally link against "Multithreaded DLL". Loading DLLs whose runtime libraries are mismatched with that of the application will often lead to mysterious crashes.
You could try verifying that all your DLLs build against the same runtime library as your application, regardless of which configuration (Debug or Release) is active. Whether or not this is desirable is another question entirely.
The ability to choose which runtime library to link with enables the application developer to select the best feature set given her application's requirements. For example, a single-threaded application might incur performance penalties as a result of unnecessary thread synchronization if it is linked with a runtime library that is designed with thread safety in mind. Likewise, the consequences of linking a multi-threaded application against a single-threaded runtime could potentially be disastrous.
While I never intentionally link libraries that were built with different compiler settings, there isn't much point in doing so, I only know of the STL classes in the Dinkumware implementation (the one MSFT uses) to cause this problem.
They support a feature called 'iterator debugging' which is turned on by default in the Debug configuration. This adds members to the classes to aid the diagnostic code. Making them larger. This goes bad when you create the object in a chunk of code that was compiled with one setting and pass it to code that was compiled with the opposite setting. You can turn this off by setting the _HAS_ITERATOR_DEBUGGING macro to 0. Which is rather a major loss, the feature is excellent to diagnose mistakes in the way you use STL classes.
Passing objects or pointers between different libraries is always a problem if you don't carefully control the compile settings. Mixing and matching the CRT version and flavor gets you in trouble when you do so. This normally generates a warning from the linker, not sure what you did to not see it. There won't be one if the code lives in a DLL.
I have an annoying problem which I might be able to somehow circumvent, but on the other hand would much rather be on top of it and understand what exactly is going on, since it looks like this stuff is really here to stay.
Here's the story: I have a simple OpenGL app which works fine: never a major problem in compiling, linking, or running it. Now I decided to try to move some of the more intensive calculations into a worker thread, in order to possibly make the GUI even more responsive — using Boost.Thread, of course.
In short, if I add the following fragment in the beginning of my .cpp file:
#include <boost/thread/thread.hpp>
void dummyThreadFun() { while (1); }
boost::thread p(dummyThreadFun);
, then I start getting "This application has failed to start because MSVCP90.dll was not found" when trying to launch the Debug build. (Release mode works ok.)
Now looking at the executable using the Dependency Walker, who also does not find this DLL (which is expected I guess), I could see that we are looking for it in order to be able to call the following functions:
?max#?$numeric_limits#K#std##SAKXZ
?max#?$numeric_limits#_J#std##SA_JXZ
?min#?$numeric_limits#K#std##SAKXZ
?min#?$numeric_limits#_J#std##SA_JXZ
Next, I tried to convert every instance of min and max to use macros instead, but probably couldn't find all references to them, as this did not help. (I'm using some external libraries for which I don't have the source code available. But even if I could do this — I don't think it's the right way really.)
So, my questions — I guess — are:
Why do we look for a non-debug DLL even though working with the debug build?
What is the correct way to fix the problem? Or even a quick-and-dirty one?
I had this first in a pretty much vanilla installation of Visual Studio 2008. Then tried installing the Feature Pack and SP1, but they didn't help either. Of course also tried to Rebuild several times.
I am using prebuilt binaries for Boost (v1.36.0). This is not the first time I use Boost in this project, but it may be the first time that I use a part that is based on a separate source.
Disabling incremental linking doesn't help. The fact that the program is OpenGL doesn't seem to be relevant either — I got a similar issue when adding the same three lines of code into a simple console program (but there it was complaining about MSVCR90.dll and _mkdir, and when I replaced the latter with boost::create_directory, the problem went away!!). And it's really just removing or adding those three lines that makes the program run ok, or not run at all, respectively.
I can't say I understand Side-by-Side (don't even know if this is related but that's what I assume for now), and to be honest, I am not super-interested either — as long as I can just build, debug and deploy my app...
Edit 1: While trying to build a stripped-down example that anyway reproduces the problem, I have discovered that the issue has to do with the Spread Toolkit, the use of which is a factor common to all my programs having this problem. (However, I never had this before starting to link in the Boost stuff.)
I have now come up with a minimal program that lets me reproduce the issue. It consists of two compilation units, A.cpp and B.cpp.
A.cpp:
#include "sp.h"
int main(int argc, char* argv[])
{
mailbox mbox = -1;
SP_join(mbox, "foo");
return 0;
}
B.cpp:
#include <boost/filesystem.hpp>
Some observations:
If I comment out the line SP_join of A.cpp, the problem goes away.
If I comment out the single line of B.cpp, the problem goes away.
If I move or copy B.cpp's single line to the beginning or end of A.cpp, the problem goes away.
(In scenarios 2 and 3, the program crashes when calling SP_join, but that's just because the mailbox is not valid... this has nothing to do with the issue at hand.)
In addition, Spread's core library is linked in, and that's surely part of the answer to my question #1, since there's no debug build of that lib in my system.
Currently, I'm trying to come up with something that'd make it possible to reproduce the issue in another environment. (Even though I will be quite surprised if it actually can be repeated outside my premises...)
Edit 2: Ok, so here we now have a package using which I was able to reproduce the issue on an almost vanilla installation of WinXP32 + VS2008 + Boost 1.36.0 (still pre-built binaries from BoostPro Computing).
The culprit is surely the Spread lib, my build of which somehow requires a rather archaic version of STLPort for MSVC 6! Nevertheless, I still find the symptoms relatively amusing. Also, it would be nice to hear if you can actually reproduce the issue — including scenarios 1-3 above. The package is quite small, and it should contain all the necessary pieces.
As it turns out, the issue did not really have anything to do with Boost.Thread specifically, as this example now uses the Boost Filesystem library. Additionally, it now complains about MSVCR90.dll, not P as previously.
Boost.Thread has quite a few possible build combinations in order to try and cater for all the differences in linking scenarios possible with MSVC. Firstly, you can either link statically to Boost.Thread, or link to Boost.Thread in a separate DLL. You can then link to the DLL version of the MSVC runtime, or the static library runtime. Finally, you can link to the debug runtime or the release runtime.
The Boost.Thread headers try and auto-detect the build scenario using the predefined macros that the compiler generates. In order to link against the version that uses the debug runtime you need to have _DEBUG defined. This is automatically defined by the /MD and /MDd compiler switches, so it should be OK, but your problem description suggests otherwise.
Where did you get the pre-built binaries from? Are you explicitly selecting a library in your project settings, or are you letting the auto-link mechanism select the appropriate .lib file?
I believe I have had this same problem with Boost in the past. From my understanding it happens because the Boost headers use a preprocessor instruction to link against the proper lib. If your debug and release libraries are in the same folder and have different names the "auto-link" feature will not work properly.
What I have done is define BOOST_ALL_NO_LIB for my project(which prevents the headers from "auto linking") and then use the VC project settings to link against the correct libraries.
Looks like other people have answered the Boost side of the issue. Here's a bit of background info on the MSVC side of things, that may save further headache.
There are 4 versions of the C (and C++) runtimes possible:
/MT: libcmt.lib (C), libcpmt.lib (C++)
/MTd: libcmtd.lib, libcpmtd.lib
/MD: msvcrt.lib, msvcprt.lib
/MDd: msvcrtd.lib, msvcprtd.lib
The DLL versions still require linking to that static lib (which somehow does all of the setup to link to the DLL at runtime - I don't know the details). Notice in all cases debug version has the d suffix. The C runtime uses the c infix, and the C++ runtime uses the cp infix. See the pattern? In any application, you should only ever link to the libraries in one of those rows.
Sometimes (as in your case), you find yourself linking to someone else's static library that is configured to use the wrong version of the C or C++ runtimes (via the awfully annoying #pragma comment(lib)). You can detect this by turning your linker verbosity way up, but it's a real PITA to hunt for. The "kill a rodent with a bazooka" solution is to use the /nodefaultlib:... linker setting to rule out the 6 C and C++ libraries that you know you don't need. I've used this in the past without problem, but I'm not positive it'll always work... maybe someone will come out of the woodwork telling me how this "solution" may cause your program to eat babies on Tuesday afternoons.
This is a classic link error. It looks like you're linking to a Boost DLL that itself links to the wrong C++ runtime (there's also this page, do a text search for "threads"). It also looks like the boost::posix::time library links to the correct DLL.
Unfortunately, I'm not finding the page that discusses how to pick the correctly-built Boost DLL (although I did find a three-year-old email that seems to point to BOOST_THREAD_USE_DLL and BOOST_THREAD_USE_LIB).
Looking at your answer again, it appears you're using pre-built binaries. The DLL you're not able to link to is part of the TR1 feature pack (second question on that page). That feature pack is available on Microsoft's website. Or you'll need a different binary to link against. Apparently the boost::posix::time library links against the unpatched C++ runtime.
Since you've already applied the feature pack, I think the next step I would take would be to build Boost by hand. That's the path I've always taken, and it's very simple: download the BJam binary, and run the Boost Build script in the library source. That's it.
Now this got even a bit more interesting... If I just add this somewhere in the source:
boost::posix_time::ptime pt = boost::posix_time::microsec_clock::universal_time();
(together with the corresponding #include stuff), then it again works ok. So this is one quick and not even too dirty solution, but hey — what's going on here, really?
From memory various parts of the boost libraries need you to define some preprocessor flags in order to be able to compile correctly. Stuff like BOOST_THREAD_USE_DLL and so on.
The BOOST_THREAD_USE_DLL won't be what's causing this particular error, but it may be expecting you to define _DEBUG or something like that. I remember a few years ago in our boost C++ projects we had quite a few extra BOOST_XYZ preprocessor definitions declared in the visual studio compiler options (or makefile)
Check the config.hpp file in the boost thread directory. When you pull in the ptime stuff it's possibly including a different config.hpp file, which may then define those preprocessor things differently.