I've got an SDK I'm working on and the previous developer just dropped the DLLs in System32 (Apparently a serious offense: see here)
So assuming I move them out into \Program Files\\SDK (or whatever), how do I make sure that all the apps that needs those DLLs can access them? And to clarify, all apps that access these are doing early (static) binding to the DLLs at compile time so I can't pass the full path to them or anything. They need to be able to find it just given the DLL filename only.
Along the same lines, what about including a particular version of MSVCR80.dll? They all depend on this but I need to make sure they get a specific version (the one I include).
Any ideas?
An SDK is by definition a development kit. It's not a deployment patch...
What this means is that the applications that depend on those assemblies should ship with them and install them into their local \program files.. directories.
The reason for this is let's say you decide to do a breaking change by eliminating an entry point for example. By installing your "SDK", it has the potential to stop older programs from functioning.
You could take a play from the Java handbook and update the PATH environment variable. Whenever a program makes a call to an external assembly it searches along that environment variable until it finds it.
Of course, this could still result in the problem showing up. So your best bet is to just install the SDK into Program Files and let the developers of the products that depend on your toolkit decide whether they want to update their versions or not.
UPDATE
As I'm thinking about this, one last possibility is to GAC your assemblies. In the event you do so, bear in mind that they should be strongly named and properly versioned so as not to step on each other. I don't recommend this route because it hides the actual locations of the assemblies and makes uninstalling a little more difficult then simply hitting delete on your directory.
I can't tell you about your own DLLs, but you should never redistribute Microsoft DLLs alone.
You always have to use Microsoft Redistributable Package.
For example, if your application depends on dll from Dev Studio 2005 SP1, you should redistribute your application with Microsoft Visual Studio 2005 SP1 redistributable. The same applies to 2008. MS provide MSI based installer and Merge Module to include in your own product installer.
You are asking about "DLL Hell", something I had thought every Windows developer was familiar with. The order of search for DLLs is:
the directory the exex that calls them was loaded from
the current directory
various Windows directories (as discussed in your previous question)
directories in the PATH variable
As the Windows directories should be ruled out, that leaves you with three options.
You can put your install path in the search path, which will allow the applications to find them.
Alternatively, you can deploy the DLL's into the same directory as the application that depends on them.
I believe the first is better from an SDK perspective - it'll make development easier. But I think the second is better for when the application gets deployed to end-users, unless you expect there may be many consumers on a single system so the disk and memory footprint of having copies of the DLL are prohibitive.
If you can't install the dlls into the same directory as the exe using them you could append your directory to the PATH environment variable.
You don't say which version of Windows you're using, as the details are slightly different from what I remember.
You could also put your version of MSVCR80.dll in the same folder. However, you'd have to ensure that your folder was before the system one on the path otherwise the linker would pick up the "standard" one first. However, if you adopted the "local" dlls approach then you wouldn't have this problem as Windows searches the local directory first and so will pick up your version of MSVCR80.dll.
Is your version the latest or a previous version? You might be better off getting your app to work with that version or later and then allow the users to update their machines as required. This also illustrates why you should never mess with the dlls in \Windows or \Windows\system32 as, as others have pointed out, you could break other applications by changing the version of this dll.
Related
Today I build my application and packaged the installer with QtIF. It worked nice on my computer but complained about missing msvcp140_1.dll in another computer.
Then I run find . -iname "msvcp140_1.dll" and found more than five different ones on my computer, I checked the md5sum.
Then I spend the time to try all of them on the other computer and all seem to work fine. No more complains about missing DLL.
How should I choose between the DLLs? Just pick any seems too easy.
Is there someway to inspect the DLLs, to check for a version?
I believe that DLL was a dependency from another binary included in my application, was not that the QtIF failed to include it.
Call the MS Redist Installer from your Installer. This can be done quietly, so that the end user does not notice it.
Find the vcredist_x64.exe file (or vcredist_x32 for 32 Bit applications), add it to your installer,
let it extract to the "TEMP" folder
and then call vcredist_x64.exe /quiet at the end of your install.
This has several advantages:
You will definitely copy all required files to the users computer.
Should new versions of the runtime library be released by Microsoft and should they already be on your users computer, your code will use the newer versions, which may include bugfixes.
Windows Update may also update the libraries.
That said, it is possible to copy the DLLs themselves, but you should make sure
a) you choose the right ones
b) they come from a trustworthy place, i.e. your VS installation folder
c) reside in the same directory as the executable - otherwise you will run into trouble with manifests.
The reason you might want to include the DLLs directly is if you want to reduce the overall size of your installer.
We did this a couple of years with our products, but finally gave in and simply used the vcredist_x64.exe, even if that increased the installer binary another couple of MB in size. But in the long run that's the easiest way.
I think (not sure), msvcp140_1.dll is an additional DLL for the VS 2019 runtime. VS 2017 runtime does not need this, but the new one does.
The non-redistributable method is to ship the DLL's that come with your compiler, i.e. the compiler that built your executable. After all, the DLL will be loaded into the same process as your EXE. It's only logical that these should match.
You'll find those DLL's in \Program Files*\Microsoft Visual Studio *\VC\Redist
I have a lot of basic questions about DLLs and how to bundle third-party libraries with your software, but most everything I find when searching is specific to C# (such as this question) and I'm looking for more general information about how DLLs work and what their purpose is.
I have, for example, a CMake-managed C++ application that links to one or more third-party DLLs.
Their DLLs are installed to C:/Program Files/Intel/SomeLibrary/bin/somelibrary.dll. I can build my app fine by having an environment variable take care of finding their headers.
Obviously, I can't actually run my executables out of the box, because somelibrary.dll can't be found. I have to either copy it into the same folder as the executable, or add Intel/SomeLibrary/bin to my Path.
Questions
How do other programs link to their required DLLs? Do they just package copies of all the third-party DLLs they use in with their own software?
Is it a different process for some third-party libraries -- like maybe Intel libraries you'd bundle with your software, but for Microsoft's VS Redistributables you'd require the user to install them? It looks like your installer (WiX example) handles installing them if they don't exist (and then does it install a new copy for every piece of software that needs them?).
Could you potentially have many copies on your computer of the same DLL because a bunch of separate programs needed it but had no way of knowing if you had it somewhere already?
Put another way, could two programs, written by different people, ever link to the same DLL file? How do they find that DLL, or install it if it doesn't exist?
I apologize in advance for whatever fundamental misunderstandings I've made in asking these -- I'm pretty new to the idea of packaging already-built software.
The short answer is that the dll loader will look for dlls in the local application folder and then searching a bunch of system locations. There are complexities with manifests, the SafeDllSearchMode feature (affects directory search order) and developers loading dlls from custom locations by override (LoadLibraryEx) and several others (see below).
Dependencty Types: There are many types of dll dependencies: Module / dll dependencies mind map.
Dynamic-Link Library Search Order: Here is an extract from Microsoft Docs on the topic: "A system can contain multiple versions of the same dynamic-link library (DLL). Applications can control the location from which a DLL is loaded by specifying a full path or using another mechanism such as a manifest."
In other words you can 1) load a dll from a specific location or 2) you can let the OS find it for you via standard mechanisms. The "SafeDllSearchMode" registry setting affects when the users current directory is searched. It now defaults to on, but can be disabled. See the link above for the authoritative details. Beware of OS changes and security fixes - all can affect this order in the future.
Q1: Always use the runtime provided for your dependent dll if available. Don't just include a few dlls if you are not supposed to.
Q2: It is correct that runtimes are often very different. Sometimes they are just a single or a few dlls that you can just install in your local application folder. Other times you install a runtime setup which installs files to appropriate folders for shared use. Some runtimes come as "merge modules" that can merge files into your own MSI installer, and some come as separate MSI installers or setup.exe installers.
Q3: The whole idea of using dlls is essentially to be able to load the same binary component into many processes in order to be able to update a single dll to fix problems in all "consumers" of the dll and also to share them in memory (important back in the day with little memory). Dlls can also be "delay loaded" - which means they won't be loaded until they are necessary - avoiding memory bloat. That is the idea anyway.
Windows OS Version: The search order (where to search on disk) for the right dlls to load is a pretty complicated topic. How the OS loader mechanism locates DLLs can be affected by a number of things, for example 1) the OS version, 2) manifests, 3) hard coded altered load paths (LoadLibraryEx), or 4) dll redirects, and 5) previously loaded dlls in memory (here be dragons), etc...
UWP & .NET: There are also new development with regards to UWP applications. For .NET assemblies the resolution is different again (which you state), please see this article: How the Runtime Locates Assemblies.
WinSxS: The manifest is a newer concept in Windows - in the olden days there would be no such thing and the search order would be simpler, but less flexible. With the advent of manifests and side-by-side Win32 assemblies (several versions of the same dll installed in different versions) the manifest is embedded in the executable (or it is added externally) and it tells the binary where to look for the runtime dlls - in the WinSxS folder hierarchy of Windows (Normally: C:\Windows\WinSxS) where multiple versions of the same dll can reside (in different sub folders).
Most Microsoft runtimes - basically all modern ones - reside here and hence require a manifest to instruct the binary what version to look for and load. There are ways to redirect such manifests too... "Publisher Configuration Files" and 2. Sigh... This can be used to enforce a certain new runtime version when the old one is undesired for use. Unused runtimes can also be "garbage collected" via ways to shrink the WinSxS folders (See this answer - section "Scavenging / Putting WinSxS on a Diet")... Hmmm... Never hard code paths to WinSxS.
Dll Search Order: The old and outdated tool Dependency Walker has an outdated, but very good description of the DLL Search Order in its help file.
Tip: The updated documentation for this is: Microsoft Docs: Module Search Order for Desktop Applications. However, to get a rough idea of search order, you can just launch Dependency Walker and select: Options => Configure Module Search Order.... Then make sure the Expand button is deselected. What you see - outdated as it might be - can help to recollect how module search order works:
Links:
DLL search on windows (old Stackoverflow answer on the same topic)
The path-searching algorithm is not a backtracking algorithm
The case of the DLL that refuses to load
How do I deal with this situation where the user's machine may not have dll like msvcp100.dll, msvcp100.dll? I don't want to my software not install on user's machine because this kind of error. I've been thinking in either find a tool and copy every single needed dll to the executable file run or try build a static version of Qt (may have sort of same result, in the end). I've seen applications that doesn't provide those dlls and it's up to user to get them, install etc. It may be a issue for those users, finding the right dll matching version and all. I'd like to avoid it.
How is this usually solved in a real world application?
You redistribute them in your installer.
Deployment of Qt apps is an uneasy issue. One should understand that you need to redistribute a compiler's dlls, as well as Qt dlls. For the msvc 2010 compiler you may just put the msvcp100.dll and msvcr100.dll files near your executable file.
As for Qt, the easiest way for windows deployment is to use the windeployqt command prompt util.
Update. How is this usually solved in real applications?
There is no fully automatic tools, which could determine all the dependencies, because the deployment of applications is a complex task. For example, you application can depend on many things such as libraries dlls, compiler's dlls, registry keys, drivers, environment variables, computer reboot requirement, ActiveX/COM components, other installed applications, etc...
But there are tools that can help you in this matter. They can be configured once for target project and then work all the time. This is called Build Automation, another similar concept is a Continuous Integration. As for the creation installation packages for end users, there is a lot of tools such as Nullsoft Scriptable Install System, Inno Setup, Qt Installer Framework, WiX, etc...List of installation software.
the installer script, put a command to download the dll that the application needs to run, and in the application source code, put it to use the dll's ONLY in the folder where it is running
If you are using the NSI installer, use this example
I have read these two SO questions: Which runtime libraries to ship? and License of runtime libraries included in GCC? - both were very helpful but not quite what I was looking for.
I have always just written programs for use on my own machine, which has never caused me any problems, but now I want to start running software on other machines and I'm wary of the runtime requirements.
EDIT: See below example instead, this was misleading.
Specifically, if I write a C++ program on a Windows machine, compiled with gcc through MinGW, and want to run it on another machine:
Do I have to send the libstdc++.dll with my program?
Is this single file (I assume placed in the executable's directory) sufficient to allow the program to run?
Also, an identical example, except this time it is an Objective-C program. Is sending the libobjc.dll file to the other machine sufficient to allow the program to execute properly?
I am used to running programs on machines which have developer tools, etc, installed, but now I'm looking to run them on general purpose machines (friends', colleagues' etc), and I'm not quite sure what to do!
EDIT: In response to edifice's answer, I feel I should clarify what it is I'm looking for. I know how to identify the necessary DLL(s) (/dylibs, etc) that my programs use, (although I am accustomed to doing that work manually; I had not heard of any of the tools). My question was more "What do I do now?"
A more general example is probably needed:
Let's say I have written a program which has object files derived from C++, C and/or Objective-C(2) code. I have used some Windows API code which compiled successfully using MinGW's gcc. I also have a custom DLL I wrote in Visual Studio (C++).
I have identified which DLL's my program will use at runtime (one of which may be GCC's libobjc.dll, I'm not sure if this would/should make a difference on a Windows machine, but I want to make this as general as possible) - The "prerequisite DLLs".
I would like to run it on my colleagues' computers, most of which run Windows 7, but some now run Windows 8. Starting at the very start for the sake of completeness:
Do I need to transfer the prerequisite DLLs to my colleagues' computers?
What directory should I place them in? (exe directory / a system directory?)
Once in place, will the presence of these DLLs allow the program to execute correctly? (Assuming it knows where to find them)
Are there any other files that should be transferred with the DLLs?
Basically I'm trying to determine the entire thought-process for developing and running an application on another machine in terms of system runtime requirements.
When loading DLLs, the first place Windows looks is the directory that the exe is in. So it will probably work just fine to put the DLLs there.
For the Microsoft DLLs though, I think it makes more sense to ask your colleague to install the Visual C++ runtime, which is a redistributable package from Microsoft. Ideally you would make an installer using something like WiX and it would install that prerequisite for you, but it is OK to just tell your colleague to do it.
Be sure to include a license file with your software if you include DLLs from gcc, because the GPL requires it.
libstdc++ isn't necessarily sufficient. You almost certainly need libgcc too, but actual dependencies are liable to vary with your particular application.
The best way to determine what you need to ship with your application is to load your EXE into a program like Dependency Walker.
Just as an example, I've compiled a test C++ program which simply prints a std::string. As you can see, it depends directly on two modules other than those that come with Windows; libgcc_s_dw2-1.dll in addition to libstdc++-6.dll.
You should remember to expand the tree under each DLL to make sure that it itself doesn't have any other dependencies (if A depends on B, B might depend on C even if A doesn't directly depend on C).
If you're worried and want the strongest assurances, you could install Windows into a virtual machine (VirtualBox is free) and test your application inside it. If you use Microsoft APIs, you may wish to check the MSDN documentation to see with what version of Windows they were introduced and ensure that it aligns with your target minimum Windows version.
Update: As xtofl points out this won't cover libraries loaded dynamically using LoadLibrary. If you want to cover this base, use Process Monitor to examine what DLL files are touched when you run the application. (Add an 'Image Path' criterion with the path to your EXE in order not to get flooded.) This has the added advantage that it covers all files, registry entries, etc. that your application depends on, not just DLLs.
Much related to this question, we have a scenario on my team where we need to copy the contents of a folder for a suite of libraries and configuration files for said libraries to our folder where our test code is running from, as part of the test's deployment step.
Due to the installation size, and other factors, checking in this install folder into source control for sharing between team members just isn't viable.
The install path for the folder is either /Program Files/InternalTool/ or /Program Files (x86)/InternalTool/ depending on the installed environment. I want to setup my .testrunconfig file such that when a person gets the latest version of the solution, they don't have to worry about fixups for the path to the shared internal library suite.
Is there a way to make this seamless for all members involved, and if so, how could one accomplish this?
Restrictions are as follows:
can't check in shared suite
shared suite has no override for installation path
Is this possible, or am I asking for too much?
We handle this sort of issue (our issues are not the same but are similar) by having different config files with different names and copying the correct one over when it is needed.
In some cases we automate this within the batch job that gets the latest version.
This was actually way, way easier than I expected.
While the UI doesn't support many things with the local test run config file, I was able to set the path using the standard %ProgramFiles%.
On x86 systems, this resolves, on most systems, to C:\Program Files\.
On x64 systems, this resolves, on most systems, to C:\Program Files\.
But! If the caller is 32-bit, and not 64-bit or set to MSIL, %ProgramFiles% will resolve to C:\Program Files(x86)\. Since there is no 64-bit mstest, the resolution should happen seamlessly. For an example, this is ripped from my LocalTestRun.testrunconfig file, and then properly sanitized:
<Deployment>
<DeploymentItem filename="%ProgramFiles%\InternalSuite\" />
</Deployment>
While I haven't had the chance to fully test this yet, this should resolve our issue just fine. I have tested this on my 32-bit system, and have found that it resolves right as rain.
Hope this helps someone else!