I'm trying to build a very complex open-source project with VC++. The project consists of dozens of libraries and one executable depending on those libraries.
For some reasons VC++ linker doesn't want to see about 40 functions implemented in one of those libraries and reports "unresolved external reference" on each, so I can't link. I don't want to waste time resolving the problem - those functions are likely never called.
I'd like to just ask the linker to link what it sees and insert some reasonable error handling (like reporting an error and terminating the program) instead of missing functions. How can I do that?
You can use the /FORCE:UNRESOLVED linker option.
The documentation for that contains the rather understated warning:
A file created with this option may
not run as expected.
In pratice, there'll be no error handling - just a crash.
If the functions are truly never called, then create actual libraries (.lib files) for the libraries. Then the linker will only extract from the libraries what's needed.
The linker's job is to resolve all references, so I don't think you're going to get it to insert error handling code.
P.S. The first thing I'd check is to see if C functions got compiled as C++, leading to the missing symbols.
If they're never called, remove the references from your project. If they are called, then fix the damn problem. There's no real other option here.
There are some notable exceptions, but most OpenSource projects are not designed to be built under VisualStudio.
Generally for a Windows port you are better off using either cygwin or the mingw system. My advice is usually for mingw, unless the program uses a lot of Unix OSey calls like pipes and signals.
Related
I've used this as a reference, but it doesn't appear to be complete. When I set /NODEFAULTLIB, I get linker errors about missing __except_list, __load_config_used, and a few other things apparently related to SEH. It's easy enough to get _CxxThrowException and friends linked correctly, but I have no idea what __except_list is supposed to be and I don't know where to start looking.
I'm using both MSC and IC. When using IC, the linker reports __except_list as missing, though this is not the case with MSC.
For those of you who's favorite answer is "why?": It's because I want to know how this works, and I'm tired of do-nothing test applications starting life with 300kb committed for absolutely no reason.
If you want to get rid of CRT dependency, then pass /Zl option to the compiler. Linker does not need any special options (compiler won't insert
/DEFAULTLIB:... into .drectve section, so linker won't bind your EXE to any CRT).
Of course you shouldn't be using C++ exceptions or heap in your code; or you need to provide your own version of library functions written in assembly language.
linker option /safeseh:no (look at IMAGE_LOAD_CONFIG_DIRECTORY in PE image info).
I know next to nothing about the linking process, and it almost always gets in the way when I am trying to start a new project or add a new library. Whenever I search for fixes to these type of errors, I will find people with a similar problem but rarely any sort of fix.
Is there any generalized way of going about finding what the problem is, and fixing it?
I'm using visual studio 2010, and am statically linking my libraries into my program. My problems always seem to stem from conflicts with LIBCMT(D).lib, MSVCRT(D).lib, and a few other libraries doublely defining certain functions. If it matters at all, my intent is to avoid using "managed" C++.
If your error is related to LIBCMT(D).lib and the like, usually that depends from the fact that you are linking against a library that uses a different CRT version than yours. The only real fix is to either use the library compiled for the same version of the CRT you use (often there is the "debug" and "release" version also for this reason), either (if you are desperate) change the CRT version you use to match the one of the library.
What is happening behind the scenes is that both your program and your library need the CRT functions to work correctly, and each one already links against it. If they are linking against the same version of it nothing bad happens (the linker sees that it's the same and doesn't complain), otherwise there are multiple conflicting implementations of the same functions, so the linker doesn't know which are right for which object modules (and also, since they are probably not binary compatible, internal data structures of the two CRTs will be incompatible).
The specific link errors you mentioned (with LIBCMT(D).lib, MSVCRT(D).lib libraries) are related to conflicts in code generation options between modules/libraries in your program.
When you compile a module, the compiler automatically inserts in the resulting .obj some references to the runtime libraries (LIBCMT&MSVCRT). Now, there is one version of these libraries for each code generation mode (I'm referring to the option at Configuration properties -> C/C++ -> Code Generation -> Runtime Library). So if you have two modules compiled with a different mode, each of them will reference a different version of the library, the linker will try to include both, and of course there'll be duplicated symbols, since essentially all the symbols are the same in these libraries, only their implementations differ.
The solution comes in three parts. First, make sure all the modules in a project use the same mode. Second, if you have dependencies between projects, all of them have to use the same mode. Third, if you use third-party libraries, you have to either know which mode they use (and adopt it) or be able to recompile them with the desired mode.
The last one is the most difficult. Sometimes, libraries come pre-compiled, and not always the provider gives information about the mode used. Worse, if you're using more than one third-party library, they may have conflicting modes. In those cases, you have no better option than trial-and-error.
Also notice that each Visual Studio version has its own set of runtime libraries, so when using third-party libraries you have to use those compiled with the same version of Visual Studio you're using. If the provider doesn't offer it, your only choice is to recompile yourself.
I have a project which I can exclude some of libraries from linker and still builds ?
Is it any better to exclude them in terms of the performance and memory of final product ?
A good c++ linker would not include any calls from any libs that are not used in the code (the so-called ¨dead-code stripping¨).
So, I would say it depends what kind of C++ linker you are using to emit the final release. Maybe you should refer to your linker documentation to get information on dead-code stripping. If it is not doing that, then it would definitely help reducing the final memory footprint of the program.
Cheers, and hope that info helps !
Excluding some unused libraries from the final executable might make startup a bit faster and save a tiny amount of memory - chances are only the header and library startup code will actually end up being loaded, and these can be paged out after startup.
However, don't do it manually. If you were told to add the library there's probably a reason for it - perhaps some function call you're not using yet requires it, and later on if you use that function call you may have forgotten about it.
Most linkers have an option to exclude unused libraries automatically, so you may want to just enable that option to have it take care of things for you.
Note: In some rare cases, the library's startup code might have some important effect, in which case you should not exclude it. This is something that is best determined by checking the library's documentation; things like this should (hopefully!) be clearly documented.
It should not make any difference.
Any linker of any worth will not include anything from libraries that are not (directly or indirectly) referenced by the application, even if those libraries are specified on the command line.
The only reasons to include (a part of) a library are:
- The application uses a function or global object from the library
- A part of a library that was included to resolve some references has a reference to a function or global object of this library.
A linker does not just blindly put all the things you provide together in an application, but it makes a distinction between object files (for the application) and libraries.
The linker first collects all the object files and resolves as many references that are made between the files.
After that, the linker goes through the specified libraries and takes from each library those parts that are needed to resolve the (known) unresolved references. This may create new unresolved references due to dependencies between the libraries. Most linkers will make only a single pass over the libraries, but some may perform multiple passes to resolve all the references.
Parts of libraries that are not needed to resolve a reference are not included in the executable.
Yes, it's always better to exclude the unneccessary libraries.
Problem:
I'm trying to use a library named DCMTK which used some other external libraries ( zlib, libtiff, libpng, libxml2, libiconv ). I've downloaded these external libraries (*.LIB & *.h files ) from the same website. Now, when I compile the DCMTK library I'm getting link errors (793 errors) like this:
Error 2 error LNK2005: __encode_pointer already defined in MSVCRTD.lib(MSVCR90D.dll) LIBCMTD.lib dcmmkdir
Error 3 error LNK2005: __decode_pointer already defined in MSVCRTD.lib(MSVCR90D.dll) LIBCMTD.lib dcmmkdir
Error 4 error LNK2005: __CrtSetCheckCount already defined in MSVCRTD.lib(MSVCR90D.dll) LIBCMTD.lib dcmmkdir
Error 5 error LNK2005: __invoke_watson already defined in MSVCRTD.lib(MSVCR90D.dll) LIBCMTD.lib dcmmkdir
Error 6 error LNK2005: __errno already defined in MSVCRTD.lib(MSVCR90D.dll) LIBCMTD.lib dcmmkdir
Error 7 error LNK2005: __configthreadlocale already defined in MSVCRTD.lib(MSVCR90D.dll) LIBCMTD.lib dcmmkdir
Error 8 error LNK2005: _exit already defined in MSVCRTD.lib(MSVCR90D.dll) LIBCMTD.lib dcmmkdir
Documentation:
This seems to be a popular error for this library so, they do have a FAQ entry addressing this issue which ( http://forum.dcmtk.org/viewtopic.php?t=35 ) says:
The problem is that the linker tries to combine different,
incompatible versions of the Visual
C++ runtime library into a single
binary.
This happens when not all parts of your project and the libraries you
link against are generated with the
same code generation options in Visual
C++.
Do not use the /NODEFAULTLIB workaround, because strange software
crashes may follow. Fix the problem!
DCMTK is by default compiled with the "Multithreaded" or "Multithreaded
Debug" code generation option (the
latter for Debug mode).
Either change the project settings of all of your code to use these code
generation options,
or change the code generation for all DCMTK modules and re-compile.
MFC users beware: DCMTK should be compiled with "Multithreaded DLL" or
"Multithreaded DLL Debug" settings if
you want to link the libraries into an
MFC application.
Solution to same problem for others:
Huge Amount of Linker Issues with Release Build Only says:
It seems that your release build is
trying to link to something that was
built debug. You probably have a
broken dependency in your build, (or
you missed rebuilding something to
release by hand if your project is
normally built in pieces).
More technically, you seem to be
linking projects built with different
C Run Time library settings, one
with "Multi-Threaded", another one
with "Multi-Threaded Debug". Adjust
the settings for all the projects to
use the very same flavour of the
library and the issue should go away
Questions:
Till now I used to think that Name mangling is the only problem that may cause linking failures if its not been standardized. Just now I knew there are other things also which can cause same effect.
Whats up with the "Debug Mode" (Multi-Threaded Debug) and "Release Mode" (Multi-Threaded)? What exactly is happening under the hood? Why exactly this thing is causing linking error?
I wonder if there is something called "Single-Threaded Debug" and "Single-Threaded" which again causes the same thing.
Documentation talks something about "Code Generation Options". What Code Generation Options? WTH are they?
Documentation specifically warns us not to use /NODEFAULTLIB workaround. (example /NODEFAULTLIB :msvcrt ). Why? How would I cause troubles? what exactly is it?
Please explain the last point in the documentation for MFC users. Because I'm going to use MFC later in this project. Explain Why should we do it? What troubles would it cause if I don't.
Anything more you'd like to mention? I mean regarding similar errors. I'm very interested in Linker & its problems. So, if there are any similar things you can mentions them or some keywords atleast.
Whats up with the "Debug Mode"
(Multi-Threaded Debug) and "Release
Mode" (Multi-Threaded)? What exactly
is happening under the hood? Why
exactly this thing is causing linking
error?
The linker drags in libraries for several different reasons. The simplest is that a library is listed on the linker command line, or in the linker answer file on the linker command line. But any object files, whether compiled in your project or packed into a library, can also contain linker options including requesting particular libraries be linked in. In fact, the Visual C++ compiler automatically embeds such linker options matching the project options you use when compiling.
At link time, all the linker options from all object files and objects in static library files get combined. If more than one CRT library filename is requested, the linker reads in all of them and them you get naming conflicts, where the linker doesn't know which one to use.
I wonder if there is something called
"Single-Threaded Debug" and
"Single-Threaded" which again causes
the same thing.
There used to be, but the last few versions of Visual C++ have only shipped multi-thread compatible libraries.
Documentation talks something about
"Code Generation Options". What Code
Generation Options? WTH are they?
Look inside your project options.
Documentation specifically warns us
not to use /NODEFAULTLIB workaround.
(example /NODEFAULTLIB :msvcrt ). Why?
How would I cause troubles? what
exactly is it?
If you use /NODEFAULTLIB, all the linker settings stored within object files and objects in libraries get ignored. You'll end up with no runtime library and maybe missing other libraries. You can add them back in by hand, but it's still a big mess.
Please explain the last point in the
documentation for MFC users. Because
I'm going to use MFC later in this
project. Explain Why should we do it?
What troubles would it cause if I
don't. Anything more you'd like to
mention? I mean regarding similar
errors. I'm very interested in Linker
& its problems. So, if there are any
similar things you can mentions them
or some keywords atleast.
MFC applications and the MFC library have to use the same memory management functions, so that memory allocated by MFC can be freed by the application and vice-versa. FILE handles and other resources are also shared. The MFC DLLs are already compiled to use the CRT in a DLL, and in order to be able to share resources you need to use the same CRT, which means using a DLL too.
You need to configure project properties so that your debug build links with DCMTK's debug build and your release build links with DCMTK's release build.
The above is what you need to do. Below are explanations of some other random things you asked about.
Older versions of Visual Studio used to have single threaded libraries (release and debug versions) besides multithreaded libraries (release and debug versions). For your project you can pretend single threaded libraries never existed.
If you experiment with random ways to trick the linker into shutting up and leaving problems to be found by your customers instead of by yourself, you might find that the /NODEFAULTLIB option will do that. The makers of the DCMTK library are warning you not to do that because some other people did the same dumb thing in the past.
Whats up with the "Debug Mode" (Multi-Threaded Debug) and "Release Mode" (Multi-Threaded)? What exactly is happening under the hood? Why exactly this thing is causing linking error?
They are different versions of the C runtime library. You can statically link to the runtime library in debug and release mode. In the Code Generation Options (mentioned below), those would be "Multi-Threaded Debug" and "Multi-Threaded". The options "Multi-Threaded Debug DLL" and "Multi-Threaded DLL" dynamically link to the C runtime. By dynamically linking to the runtime, you'll also have to ship your installer configured to install the VC redistributable package that contains the proper runtime dlls for your version of Visual C++.
Statically linking to the C runtime is generally frowned upon, even by Microsoft:
In addition to all the methods
described above of distributing the
Visual C++ libraries DLLs, there is
one last option for building your
application which does not require you
to distribute the DLLs. However, this
option only works for native-only code
(it is not supported with /clr) and
leaves your customers seriously
vulnerable to any security holes as
well as adds a significant burden upon
yourself to patch all customer systems
should a vulnerability be found in any
of the libraries. This option is to
statically link in the libraries as
.lib files instead of dynamically
loading them as DLLs. You do this by
using the /MT flag on the cl.exe
command line (vs /MD), or selecting
the appropriate option in your project
properties through Visual Studio. You
may wish to use this option when
testing early debug builds of your
application on test machines before
you start working on setup. [See
footnote 3]
However, I can think of no scenarios
in which this is actually the right
thing to do when shipping your product
to customers. Basically, what this
approach does is pulls in the binary
code needed from .LIB files at compile
time, making it a part of your .exe or
.dll files. It increases the size of
your application, and there is no way
to update the libraries apart from
recompiling your application with new
.LIBs and redistributing your
application all over again. What this
means is that unless you go touch
every single machine which has
installed your application every time
there is a security vulnerability
found in the Visual C++ libraries and
completely reinstall your updated
binaries, you will be leaving your
customers vulnerable to attack. If
instead you use the DLLs, every time
there is a security vulnerability
found in the Visual C++ libraries,
Microsoft will install the update
centrally into the WinSxS folder via
Windows Update and all requests for
the DLLs will be redirected to the
updated version. This removes all
servicing burden on your side and also
allows the user to install one small
update which will touch all their
applications instead of replacing
every installed exe and DLL on their
system. Please, do not distribute an
application built by linking
statically against the Visual C++
libraries unless you have a system in
place for updating every customer
machine and also have a very good
reason to do so. At this time, I can
think of no circumstance under which
this would be the right thing to do
for a shipping application.
I wonder if there is something called
"Single-Threaded Debug" and
"Single-Threaded" which again causes
the same thing.
No such thing, see above.
Documentation talks something about "Code Generation Options". What Code Generation Options? WTH are they?
Right click on your Visual C++ project (from within Visual Studio) and select Properties. Under Configuration Properties->C/C++->Code Generation
Documentation specifically warns us not to use /NODEFAULTLIB workaround. (example /NODEFAULTLIB :msvcrt ). Why? How would I cause troubles? what exactly is it?
Take their advice and don't do it.
Please explain the last point in the documentation for MFC users. Because I'm going to use MFC later in this project. Explain Why should we do it? What troubles would it cause if I don't.
Because MFC is dynamically linked to the C runtime, using libraries that are statically linked to the C runtime will cause the linker errors you listed first in your post.
Anything more you'd like to mention? I mean regarding similar errors. I'm very interested in Linker & its problems. So, if there are any similar things you can mentions them or some keywords atleast.
From my experience, always dynamically link to the C runtime. It generally saves you a lot of headaches like the one you're experiencing right now.
I'm working in a Linux environment with C++, using the GCC compiler.
I'm currently working on modifying and upgrading a large pre-existing body of code. As part of this, it has been necessary to add quite a large number of small references throughout the code in a variety of places to link things together, and also to add in several new external code libraries. There is also quite a large and complex structure of Makefiles linked to a configure.ac file to handle the build process.
Upon starting the build process everything compiles without a problem, but comes back with the dreaded linker error when trying to use a newly added custom code library we've created. We have now been through a vast amount of code with a fine tooth comb looking for spelling mismatches, checking the order that all the libraries are included in the build process, and checked that the .o files created contain what we need using dumps, and all are as and where they should be. We've also tested the library separately and the problem definitely doesn't lie there.
In short, we've tried most things that you should normally do in these scenarios.
Is there a tool for C++ that can detect linker errors automatically, in a similar vein to cppcheck or splint (both of which we have run to no avail) that could help here?
Don't know your platform, but I spent sometime with linker problems in gcc till I realized that the static library (.a) linking requires specific ordering, its not the same to link gcc object.o first.a second.a than gcc object.o second.a first.a.
FWIW (not much) I try to tackle this sort of issue by using another linker as I have access to a couple of different platforms. If you can use another linker you will find either:
a) the program links, which transforms your problem from 'why doesn't it link ?' to 'what are the differences between linkers and linking ?' which isn't exactly a step forward, but sometimes a step to one side gives you a different perspective from which you can see a solution;
OR
b) it fails to link, in which case the other linker might give more useful information about why it fails.