I'm having some strange issues building and running a project on another computer. It's a side-by-side error. Usually the cause is that c++ redistributable is not installed on the machine etc. However in this case the project is compiled on that machine. MSVC++ 2005 is installed, the runtimes should be there (I installed the runtime again for good measure anyway). Why is the linker referencing a runtime library that isn't available on the machine?
I'm dynamically linking to runtime library.
Any ideas on how to debug this issue?
Thanks.
EDIT
I didn't want to start another post because it's related. Because of this DLL version mess, is this a good reason to statically link to runtime? Will I avoid all these problems? I don't see any advantages to dynamically link to runtime any more. I was under the impression that with DLL runtime you get the benefit of updates/bug fixes with new DLLs. However because of the SxS and manifests it ensures that it loads the specific version (old version) of the DLL anyway? So what's the point of dynamic runtime at all? Maybe a few kb of space saved because you're not embedding the re-used functions in all the dependent libraries. But compare this to the cost of your app won't run because some ancient runtime version is removed from the machine, isn't it worth it?
Thanks again. Still tracing the original problems and will probably have to recompile every single library I'm using.
sxstrace will tell you exactly what is going on with respect to SxS. It will show what dlls are searched and how they are mapped to actual versions.
Now, which runtime is loaded is coming from the manifest file that gets included in your project. Looking at the one you mention, it looks like the one from Visual2005, with no service pack. SP1 changed the crt to 8.0.50727.762
Some details on sxstrace on vista and XP
Well, since you added a question to your question, let me add an answer to my answer:
SxS will not necessarily load the version you specify inside your manifest. The SxS system keeps track of security fixes made to specific versions, e.g. and will change which version it loads even when you ask for a specific version.
That said, if your program uses DLLs, and you want to share C objects (e.g. malloc'ed memory) between them, then your only option is the CRT DLL. It really depends what your constraints are.
It may happen when you compile along a third party library, or object files, that you compiled on another machine and the copied on the machine where the issue occurs.
Try to find such binary files on your machine, and recompile them on that machine.
Not an answer to the problem, but an answer to this question:
Why is the linker referencing a runtime library that isn't available on the machine?
The linker doesn't need the actual runtime library to link. It just needs (typically), the .lib file at link time. The .lib file tells the linker what the runtime library will provide (as in exported symbols) when the OS locates the dll at runtime.
Dependency Walker can be helpful in cases like this to help debug the problem.
EDIT: followup to the new question. Static linking does resolve these issues, however it also introduces some new issues. You can share dynamically allocated objects between dlls - however, whichever dll allocated the object must be the one to deallocate it. Any methods on the object that allocate/reallocate/deallocate member data/objects similarly must be governed in order to avoid heap corruption. Non-inline refcounting/shared pointers will be helpful. Alternatively, shared mem allocators can be helpful too.
Here's a possibly related forum post. Not sure if this is the problem, but it seems worth checking.
The summary is that MS updated the ATL, CRT, MFC, and a few other libraries on VS 2005 developer machines via an automatic update.
On machines without VS2005 installed, they only updated the ATL via an automatic update, causing SxS errors.
You can either uninstall the update on the dev machine, or upgrade the runtimes manually on the machine you're trying to run on. Details on the post.
Related
I have this job of implementing a library that provides a file-sharing feature.
This has already happened twice:
First, in a string in an if-else path, only the if path is being executed, but when i change a spelling in the else path, the software after a few minutes crashes in an std library. I verified with a debug attached that the lined changed was never being touched. When i reversed the change, it works nicely again.
Second, my software crashes on a std library again with the out-of-array check into a standard basic_string destructor.
I did everything, all library matched the _HAS_ITERATOR_DEBUGGING.
After 4 hours I discovered that the problematic file is TorrentFile.cpp/h.
If i add a function ( even though it is never called ), the program crashes at the end of that file, but if its not there, there's no bug. The code causing the problem:
std::vector<TorrentFileListPacket> TorrentFile::GetFileMap()
{
std::vector<TorrentFileListPacket> vFiles;
return vFiles;
};
If i comment this code out, the crash is gone.
This is really driving me crazy!
I've been a developer for 8 years, and I've never seen something like this before!
Additional Information
My memory is OK, I'm using Visual Studio 2010 with SP1 in Windows 7. The library is libTorrent from RasterBar and it links to boost. The software is using MFC.
This smells strongly of memory corruption in a totally different location from where you would expect from the crashes. Most likely adding and removing functions is changing the memory layout in such a way that causes the memory corruption's effects to be immediately visible or not.
Your best hope is something like Purify or Valgrind to hunt it down.
You probably want to make sure that all your object files and libraries are ABI compatible with each other.
Numerous compiler settings will change the ABI. Especially debug and release builds and iterator debugging. The struct layout for standard containers typically change when you enable iterator debugging (which I believe is on by default for all debug builds in msvc, and off for release builds).
So, if a single object file, static library or DLL that you link against is built with an incompatible configuration, you typically see very odd behaviors. With libtorrent you need to make sure you build the library with the same configuration as you link against it with. Many of the TORRENT_* defines will actually change some aspect of some struct layout or function call. Make sure you define the exact same set of those in your client as when building the library. One simple way of dealing with this problem is to simply pull all source files into your project and build everything together.
If you are using libtorrent as a DLL (or boost for that matter), are they compiled against the same C Run-Time?
Often when I run into this type of issue it is because I make a call into a library that was compiled with MinGW (which uses the CRT from VS6.0) or an older version of Visual Studio. If memory is allocated by the library and then free'd by your application, you will often get these types of errors in the destructor.
If you aren't sure, you can open the DLL in question in a tool like the Dependency Walker. Look for the dependency MSVCRT.DLL, MSVCR100.DLL, etc.
it's been a long while since I've used VS 2010 and C++, and as I'm getting back to using it, I'm running into the same problems that plagued me last year: the exe's that I compile do not run well on older machines that do not have the correct C++ runtimes. I do not even know what link to give them, and I told them to install this after they had an error that said "The program can't start because MSVCR100.dll is missing from your computer. Try Reinstalling the program to fix this problem. Click OK to close the application."). So I went in and set the code generation to /MT and disabled quite a few options, and tried messing around with lots of options, but still the same result.
My question is: Is there a list of complete VS 2010 C++ distributables that I can just give and tell them to install so all of the C++ programs I compile on my VS 2010 will work on Windows XP, or even better, a way to general a standalone exe that contains all it needs to work, and does not rely on DLLs? I'm thinking like linking to a library that has everything the exe references. If it helps, I'm building for both x64 and x86.
P.S. What's up with the manifest file, should I include it or not?
The easiest thing to do is to just install the VC++ Redistributable Package. It has both x86 and x64 versions.
Firstly, before I actually give you the detail:
Warning
If you do this, things will be bad for two reasons:
If there are security or other bugs in the MSVC runtimes, and you take this approach, they're baked into your app which means you need to re-distribute. DLLs are preferred because theoretically people use system update which means any errors get fixed.
Everything else you compile into your exe also needs to do this. If you don't, you end up with two versions of the code and whatever you're using won't link.
One possible solution is to bake the MSVC runtime into your application, by using the cl.exe option (C/C++ compiler settings) /MT which means multi-threaded version of the C/C++ runtime linked statically. As I said, if you try to link against something that is linked itself dynamically to the runtime, you're going to end up in a mess. Also, as I said, this represents an additional security risk factor, so bear that in mind.
The other options are to write an installer that can either download the appropriate runtime, or include the DLL needed.
If you're using some feature of the runtime that exceeds a certain version of Windows (generic statement, but it does happen) then you should be able to use the Windows SDK to target various versions of Windows using appropriate C runtimes.
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=a7b7a05e-6de6-4d3a-a423-37bf0912db84
google text: visual studio c++ redist
Do not statically link to the runtime; specifically don't do so if you're using any kind of dll for other purposes. It introduces all kinds of bogus problems wrt heap management that you probably don't want to mess with.
Open the properties dialog for your project and select Configuration Properties | C/C++ | Code Generation. The default setting is Multi-threaded DLL. Change that to Multi-threaded and you'll be building and .EXE with the run-time statically linked in. Don't forget to do the same for the debug version.
If you're using MFC or ATL, you will need to navigate to Configuration Properties | General and set "Use of MFC" or "Use of ATL" to link statically as well.
NB: If you link the runtime statically, you must make sure that any other library you're linking in also links it in statically. Otherwise you'll wind up with two copies of the runtime in memory, each with its own heap and bad things will happen when code using one runtime tries to free an object allocated by the other runtime.
This previous answer should hold true for VS2010. I still build with VS2005, but all my apps use the static CRT for the sole reason of being able to run across old and newer machines alike.
I'm not using any features of the MSVCRT100.dll (I don't even know if there are new features).
Well, you should be able to statically link it. Your .dll will be way bigger, but would not require msvcrt. This is controlled by Code Generation->Runtime library (choose /MT).
Most applications use the C/C++ Runtimes. You may be using the runtime in a fashion you don't know of yet ... call fopen() somewhere? Then you use it.
However, as it pointed out by BarsMonster, you can link statically to the runtime. Your binary size grows, but you have no external dependencies. In fact this is the method you would choose if don't want to use installer software to deploy your application.
It's almost certainly the best choice for stuff like external libraries that are not bound to a particular application and could be reused several times. If you release your DLL to somebody in a SDK, i'd recommend providing lib's and dll's for both static and dynamic linkage to the runtime.
Keep in mind, however, that static linkage has one serious disadvantage: heap memory is not shared across DLL boundaries then. A memory block must be freed by the module (DLL) which allocated it i the first place. If you can't fulfil this requirement, do not use static linkage. Deploying with the runtime can't be avoided then.
You can use VS2010 and still target older versions of the runtime. This can be configured in your project properties, here's an image:
pic from VC++ blog http://blogs.msdn.com/photos/vcblog/images/9934271/original.aspx
Obviously you still need the toolset you're targeting installed.
For more info you can look at this VC++ team blog post.
Unfortunately, yes. You'll need the VC10 runtime for your platform (x86) or (x64) -- keep in mind though the runtime may change, though it is highly unlikely since VStudio has been in it's final phases for a while now.
It is the core runtime library, you can find out more of your dependencies using DependencyWalker (http://www.dependencywalker.com)
Or alternatively, try it :-)
I would like some information on the runtime libraries for Visual Studio 2008. Most specifically when should I consider the DLL versions and when should I consider the Static versions.
The Visual Studio documentation delineates the technical differences in terms of DLL dependencies and linked libraries. But I'm left wondering why I should want to use one over the other. More important, why should I want to use the multi-threaded DLL runtime when this obviously forces my application into a DLL dependency, whereas the static runtime has no such requirement on my application user machine.
Linking dynamically to the runtime libraries complicates deployment slightly due to the DLL dependency, but also allows your application to take advantage of updates (bug fixes or more likely performance improvements) to the MS runtime libraries without being recompiled.
Statically linking simplifies deployment, but means that your application must be recompiled against newer versions of the runtime in order to use them.
Larry Osterman feels that you should always use the multi-threaded DLL for application programming. To summarize:
Your app will be smaller
Your app will load faster
Your app will support multiple threads without changing the library dependency
Your app can be split into multiple DLLs more easily (since there will only be one instance of the runtime library loaded)
Your app will automagically stay up to date with security fixes shipped by Microsoft
Please read his whole blog post for full details.
On the downside, you need to redistribute the runtime library, but that's commonly done and you can find documentation on how to include it in your installer.
Dynamically linking the runtime library can give you faster program start up times and smaller system memory usage since the dll can be shared between processes and won't need to be loaded again if it's already used by another process.
I think that main difference is how exceptions will be processed. Microsoft doesn't recommend to link statically to the CRT in a DLL unless the consequences of this are specifically desired and understood:
For example, if you call _set_se_translator in an executable that loads the DLL linked to its own static CRT, any hardware exceptions generated by the code in the DLL will not be caught by the translator, but hardware exceptions generated by code in the main executable will be caught.
I have written some code that makes use of an open source library to do some of the heavy lifting. This work was done in linux, with unit tests and cmake to help with porting it to windows. There is a requirement to have it run on both platforms.
I like Linux and I like cmake and I like that I can get visual studios files automatically generated. As it is now, on windows everything will compile and it will link and it will generate the test executables.
However, to get to this point I had to fight with windows for several days, learning all about manifest files and redistributable packages.
As far as my understanding goes:
With VS 2005, Microsoft created Side By Side dlls. The motivation for this is that before, multiple applications would install different versions of the same dll, causing previously installed and working applications to crash (ie "Dll Hell"). Side by Side dlls fix this, as there is now a "manifest file" appended to each executable/dll that specifies which version should be executed.
This is all well and good. Applications should no longer crash mysteriously. However...
Microsoft seems to release a new set of system dlls with every release of Visual Studios. Also, as I mentioned earlier, I am a developer trying to link to a third party library. Often, these things come distributed as a "precompiled dll". Now, what happens when a precompiled dll compiled with one version of visual studios is linked to an application using another version of visual studios?
From what I have read on the internet, bad stuff happens. Luckily, I never got that far - I kept running into the "MSVCR80.dll not found" problem when running the executable and thus began my foray into this whole manifest issue.
I finally came to the conclusion that the only way to get this to work (besides statically linking everything) is that all third party libraries must be compiled using the same version of Visual Studios - ie don't use precompiled dlls - download the source, build a new dll and use that instead.
Is this in fact true? Did I miss something?
Furthermore, if this seems to be the case, then I can't help but think that Microsoft did this on purpose for nefarious reasons.
Not only does it break all precompiled binaries making it unnecessarily difficult to use precompiled binaries, if you happen to work for a software company that makes use of third party proprietary libraries, then whenever they upgrade to the latest version of visual studios - your company must now do the same thing or the code will no longer run.
As an aside, how does linux avoid this? Although I said I preferred developing on it and I understand the mechanics of linking, I haven't maintained any application long enough to run into this sort of low level shared libraries versioning problem.
Finally, to sum up: Is it possible to use precompiled binaries with this new manifest scheme? If it is, what was my mistake? If it isn't, does Microsoft honestly think this makes application development easier?
Update - A more concise question: How does Linux avoid the use of Manifest files?
All components in your application must share the same runtime. When this is not the case, you run into strange problems like asserting on delete statements.
This is the same on all platforms. It is not something Microsoft invented.
You may get around this 'only one runtime' problem by being aware where the runtimes may bite back.
This is mostly in cases where you allocate memory in one module, and free it in another.
a.dll
dllexport void* createBla() { return malloc( 100 ); }
b.dll
void consumeBla() { void* p = createBla(); free( p ); }
When a.dll and b.dll are linked to different rumtimes, this crashes, because the runtime functions implement their own heap.
You can easily avoid this problem by providing a destroyBla function which must be called to free the memory.
There are several points where you may run into problems with the runtime, but most can be avoided by wrapping these constructs.
For reference :
don't allocate/free memory/objects across module boundaries
don't use complex objects in your dll interface. (e.g. std::string, ...)
don't use elaborate C++ mechanisms across dll boundaries. (typeinfo, C++ exceptions, ...)
...
But this is not a problem with manifests.
A manifest contains the version info of the runtime used by the module and gets embedded into the binary (exe/dll) by the linker. When an application is loaded and its dependencies are to be resolved, the loader looks at the manifest information embedded in the exe file and uses the according version of the runtime dlls from the WinSxS folder. You cannot just copy the runtime or other modules to the WinSxS folder. You have to install the runtime offered by Microsoft. There are MSI packages supplied by Microsoft which can be executed when you install your software on a test/end-user machine.
So install your runtime before using your application, and you won't get a 'missing dependency' error.
(Updated to the "How does Linux avoid the use of Manifest files" question)
What is a manifest file?
Manifest files were introduced to place disambiguation information next to an existing executable/dynamic link library or directly embedded into this file.
This is done by specifying the specific version of dlls which are to be loaded when starting the app/loading dependencies.
(There are several other things you can do with manifest files, e.g. some meta-data may be put here)
Why is this done?
The version is not part of the dll name due to historic reasons. So "comctl32.dll" is named this way in all versions of it. (So the comctl32 under Win2k is different from the one in XP or Vista). To specify which version you really want (and have tested against), you place the version information in the "appname.exe.manifest" file (or embed this file/information).
Why was it done this way?
Many programs installed their dlls into the system32 directory on the systemrootdir. This was done to allow bugfixes to shared libraries to be deployed easily for all dependent applications. And in the days of limited memory, shared libraries reduced the memory footprint when several applications used the same libraries.
This concept was abused by many programmers, when they installed all their dlls into this directory; sometimes overwriting newer versions of shared libraries with older ones. Sometimes libraries changed silently in their behaviour, so that dependent applications crashed.
This lead to the approach of "Distribute all dlls in the application directory".
Why was this bad?
When bugs appeared, all dlls scattered in several directories had to be updated. (gdiplus.dll) In other cases this was not even possible (windows components)
The manifest approach
This approach solves all problems above. You can install the dlls in a central place, where the programmer may not interfere. Here the dlls can be updated (by updating the dll in the WinSxS folder) and the loader loads the 'right' dll. (version matching is done by the dll-loader).
Why doesn't Linux have this mechanic?
I have several guesses. (This is really just guessing ...)
Most things are open-source, so recompiling for a bugfix is a non-issue for the target audience
Because there is only one 'runtime' (the gcc runtime), the problem with runtime sharing/library boundaries does not occur so often
Many components use C at the interface level, where these problems just don't occur if done right
The version of libraries are in most cases embedded in the name of its file.
Most applications are statically bound to their libraries, so no dll-hell may occur.
The GCC runtime was kept very ABI stable so that these problems could not occur.
If a third party DLL will allocate memory and you need to free it, you need the same run-time libraries. If the DLL has allocate and deallocate functions, it can be ok.
It the third party DLL uses std containers, such as vector, etc. you could have issues as the layout of the objects may be completely different.
It is possible to get things to work, but there are some limitations. I've run into both of the problems I've listed above.
If a third party DLL allocates memory that you need to free, then the DLL has broken one of the major rules of shipping precompiled DLL's. Exactly for this reason.
If a DLL ships in binary form only, then it should also ship all of the redistributable components that it is linked against and its entry points should isolate the caller from any potential runtime library version issues, such as different allocators. If they follow those rules then you shouldn't suffer. If they don't then you are either going to have pain and suffering or you need to complain to the third party authors.
I finally came to the conclusion that the only way to get this to work (besides statically linking everything) is that all third party libraries must be compiled using the same version of Visual Studios - ie don't use precompiled dlls - download the source, build a new dll and use that instead.
Alternatively (and the solution we have to use where I work) is that if the third-party libraries that you need to use all are built (or available as built) with the same compiler version, you can "just" use that version. It can be a drag to "have to" use VC6, for example, but if there's a library you must use and its source is not available and that's how it comes, your options are sadly limited otherwise.
...as I understand it. :)
(My line of work is not in Windows although we do battle with DLLs on Windows from a user perspective from time to time, however we do have to use specific versions of compilers and get versions of 3rd-party software that are all built with the same compiler. Thankfully all of the vendors tend to stay fairly up-to-date, since they've been doing this sort of support for many years.)