As a background: my application requires:
admin privileges
access to WinAPI DLLs
be able to run on all OSs: Win7-Win10
Normally, to use API, I can just link required *.lib files. However it uses default search order, that means (according to https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order) it firstly loads DLLs from "The directory from which the application loaded."
As a result, if the DLL exists in the same directory, running my app by double-click loads also that DLL.
I want to look for DLLs only in system directories (similarly to https://stackoverflow.com/a/46182665/9015013 ).
I know I can try to create some kind of proxy, like
BOOL WinAPIFunction(WinAPIType param) {
return reinterpret_cast<decltype(&WinAPIFunction)>(
reinterpret_cast<void*>(GetProcAddress(manually_loaded_module, "WinAPIFunction")))(param);
}
But it is hard to maintain all these functions. Is there any better method to force windows to look only in system32? I thought about manifest file but it requires version for each DLL that can break "capability" requirement (DLLs have different version for Win7 and Win10)
The solution is posted by #Eryk Sun in comment above.
It is sufficient to add all DLLs not listed in known dlls to delayed loaded libraries and call SetDefaultDllDirectories(LOAD_LIBRARY_SEARCH_SYSTEM32); at the beginning of the WinMain.
Thanks
You can try to use "Known DLLs" feature: If Windows "knows" that DLL, Windows doesn't search dll file. Known feature is described into link, you written in question.
Related
I have a lot of basic questions about DLLs and how to bundle third-party libraries with your software, but most everything I find when searching is specific to C# (such as this question) and I'm looking for more general information about how DLLs work and what their purpose is.
I have, for example, a CMake-managed C++ application that links to one or more third-party DLLs.
Their DLLs are installed to C:/Program Files/Intel/SomeLibrary/bin/somelibrary.dll. I can build my app fine by having an environment variable take care of finding their headers.
Obviously, I can't actually run my executables out of the box, because somelibrary.dll can't be found. I have to either copy it into the same folder as the executable, or add Intel/SomeLibrary/bin to my Path.
Questions
How do other programs link to their required DLLs? Do they just package copies of all the third-party DLLs they use in with their own software?
Is it a different process for some third-party libraries -- like maybe Intel libraries you'd bundle with your software, but for Microsoft's VS Redistributables you'd require the user to install them? It looks like your installer (WiX example) handles installing them if they don't exist (and then does it install a new copy for every piece of software that needs them?).
Could you potentially have many copies on your computer of the same DLL because a bunch of separate programs needed it but had no way of knowing if you had it somewhere already?
Put another way, could two programs, written by different people, ever link to the same DLL file? How do they find that DLL, or install it if it doesn't exist?
I apologize in advance for whatever fundamental misunderstandings I've made in asking these -- I'm pretty new to the idea of packaging already-built software.
The short answer is that the dll loader will look for dlls in the local application folder and then searching a bunch of system locations. There are complexities with manifests, the SafeDllSearchMode feature (affects directory search order) and developers loading dlls from custom locations by override (LoadLibraryEx) and several others (see below).
Dependencty Types: There are many types of dll dependencies: Module / dll dependencies mind map.
Dynamic-Link Library Search Order: Here is an extract from Microsoft Docs on the topic: "A system can contain multiple versions of the same dynamic-link library (DLL). Applications can control the location from which a DLL is loaded by specifying a full path or using another mechanism such as a manifest."
In other words you can 1) load a dll from a specific location or 2) you can let the OS find it for you via standard mechanisms. The "SafeDllSearchMode" registry setting affects when the users current directory is searched. It now defaults to on, but can be disabled. See the link above for the authoritative details. Beware of OS changes and security fixes - all can affect this order in the future.
Q1: Always use the runtime provided for your dependent dll if available. Don't just include a few dlls if you are not supposed to.
Q2: It is correct that runtimes are often very different. Sometimes they are just a single or a few dlls that you can just install in your local application folder. Other times you install a runtime setup which installs files to appropriate folders for shared use. Some runtimes come as "merge modules" that can merge files into your own MSI installer, and some come as separate MSI installers or setup.exe installers.
Q3: The whole idea of using dlls is essentially to be able to load the same binary component into many processes in order to be able to update a single dll to fix problems in all "consumers" of the dll and also to share them in memory (important back in the day with little memory). Dlls can also be "delay loaded" - which means they won't be loaded until they are necessary - avoiding memory bloat. That is the idea anyway.
Windows OS Version: The search order (where to search on disk) for the right dlls to load is a pretty complicated topic. How the OS loader mechanism locates DLLs can be affected by a number of things, for example 1) the OS version, 2) manifests, 3) hard coded altered load paths (LoadLibraryEx), or 4) dll redirects, and 5) previously loaded dlls in memory (here be dragons), etc...
UWP & .NET: There are also new development with regards to UWP applications. For .NET assemblies the resolution is different again (which you state), please see this article: How the Runtime Locates Assemblies.
WinSxS: The manifest is a newer concept in Windows - in the olden days there would be no such thing and the search order would be simpler, but less flexible. With the advent of manifests and side-by-side Win32 assemblies (several versions of the same dll installed in different versions) the manifest is embedded in the executable (or it is added externally) and it tells the binary where to look for the runtime dlls - in the WinSxS folder hierarchy of Windows (Normally: C:\Windows\WinSxS) where multiple versions of the same dll can reside (in different sub folders).
Most Microsoft runtimes - basically all modern ones - reside here and hence require a manifest to instruct the binary what version to look for and load. There are ways to redirect such manifests too... "Publisher Configuration Files" and 2. Sigh... This can be used to enforce a certain new runtime version when the old one is undesired for use. Unused runtimes can also be "garbage collected" via ways to shrink the WinSxS folders (See this answer - section "Scavenging / Putting WinSxS on a Diet")... Hmmm... Never hard code paths to WinSxS.
Dll Search Order: The old and outdated tool Dependency Walker has an outdated, but very good description of the DLL Search Order in its help file.
Tip: The updated documentation for this is: Microsoft Docs: Module Search Order for Desktop Applications. However, to get a rough idea of search order, you can just launch Dependency Walker and select: Options => Configure Module Search Order.... Then make sure the Expand button is deselected. What you see - outdated as it might be - can help to recollect how module search order works:
Links:
DLL search on windows (old Stackoverflow answer on the same topic)
The path-searching algorithm is not a backtracking algorithm
The case of the DLL that refuses to load
My basic issue is this: my program (MyProgram.exe) has a dependency on a DLL from another program (OtherProgram), and I'm trying to avoid repackaging a new DLL every time OtherProgram updates. I'd like to have MyProgram.exe link in OtherProgram's DLL when it launches, but I'm not completely sure that Windows allows for this. So if there is some kind of workaround that would also be acceptable.
And just for some background, the platform is Windows 7 x64, and MyProgram.exe runs fine when I create a symlink in the MyProgram.exe project directory to the DLL in OtherProgram's install directory. When I try to run it without the symlink, I get the "program can't start because OtherProgramDLL.dll is missing from your computer" error.
Any advice or links to relevant info is greatly appreciated!
EDIT: Clarification: the DLL is not linked at compile-time, this issue crops up at runtime
There are two types of dynamic linking in the Windows world:
Load-Time linking is when a DLL is loaded automatically when your program starts up. Windows finds this DLL using a specific algorithm I'll discuss below.
Run-Time linking is when you specifically load a DLL by calling LoadLibrary in your code. Similar rules apply as to how the library is found, but you can specify a fully-qualified or relatively-qualified path to control the search.
In the case of Load-Time linking, MS recommends that your program's DLLs are stored in and loaded from the same directory where your application is loaded from. If this is at all workable, this is probably your best option.
If that doesn't work, there are several other options, outlined here. One is to leverage the search order by putting the DLL in either the working directory or the directory where the application was loaded from.
You can change the working directory of an application by:
Create a shortcut to your application.
Bring up the shortcut's properties
Edit the "Start in" property with the directory where the DLL is located.
When you launch your application using the shortcut, it will load the right DLL.
Other options for load-time linking include:
Adding a manifest to your application which specifies where your dependent assemblies are, or,
Setting the PATH.
You could use LoadLibrary, but you would need a way to guarantee the DLL's location. This Wikipedia article provides good example on how to use the DLL after it has been loaded.
You can add the directory where the dll is located to the PATH environment variable.
I have struggled with the same problem and also found a dead end with the suggested methods like LoadLibrary, SetDllDirectory, Qt's addLibraryPath and others. Regardless of what I tried, the problem still remained that the application checked the libraries (and didn't find them) before actually running the code, so any code solution was bound to fail.
I almost got desperate, but then discovered an extremely easy approach which might also be helpful in cases like yours: Use a batch file! (or a similar loader before the actual application)
A Windows batch file for such a purpose could look like this:
#echo off
PATH=%PATH%;<PATH_TO_YOUR_LIB>
<PATH_TO_YOUR_APP_EXE>
/edit: Just saw #SirDarius comment in Luchian's answer which describes that way, so just take my batch code bit as a reference and all credits go to him.
I have the same problem with one application I am working on.
I do not want to use runtime loading because there are tens of functions I would need to manually create function pointer for.
Mr Dibling's mention of manifest file opened a new door for me but I sadly found out that the oldest version of windows that supports the feature is Windows 7. It won't even work on Vista.
Long story short, a friend familiar with Windows Application development told me to look up Delay-Loaded DLL, which turns out to solve the problem perfectly with minimal effort. It delays the loading of DLL library to either the point you manually do, or the first time its function is called. So you just need to add your DLL path to the search path before that happens, where SetDllDirectory helps.
Here is the steps to make it work:
1) Specify the DLL to be delay-loaded to linker, either through your makefile, cmake or VS property page (Linker->Input of VS2015)
2) Call SetDllDirectory at the beginning of your program, before any call to the DLL is made.
Delay-loaded DLL is supported all the way back to VC6.
SetDllDirectory is supported after XP SP1.
Use Symbolic Links to the 3rd Party Executables
I found the approach advocated by Aaron Margosis useful. See:
Using NTFS Junctions to Fix Application Compatibility Issues on 64-bit Editions of Windows
Essentially, create symbolic links to each of the dependent 3rd Party executables. Place these symbolic link files in and amongst your own dependent executable files. Except for filename changes to the targets, the 'soft' symbolic links will resolve the load-time dependencies even as the target of the links are changed by future updates.
I have successfully divided a large MFC project into a couple of smaller DLL projects. Now I want to have a separate folder called "DLL" in my application's folder, where all the all the DLLs from the subprojects are placed.
Can anybody give me guidance in how to achieve this?
If you use LoadLibrary, you simply have to explicitly specify the full path of the DLLs you load.
If the DLLs are implicitly linked, you can do this in two ways.
Have the installer modify the PATH variable. This is intrusive and "bad form"
Write a "loader" application that locally modifies the path variable, then executes the real executable.
The best solution would be to simply put the DLLs in the same directory as the executable.
DLL redirection is a fairly new feature (Windows 2000 IIRC). Name your DLL directory <myapp>.exe.local, and Windows will check it first for anything loaded via LoadLibrary(Ex). This includes delay-loaded DLLs.
EDIT: As pointed out by Eric this doesn't work. Sorry.
See Dynamic-Link Library Search Order. In short you can do so using registry keys under the "HKEY_LOCAL_MACHINE\SORTWARE\Microsoft\Windows\CurrentVersion\App Paths" key.
A reg file like the following shows how:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\MyApp.exe]
#="C:\\Program Files\\MyCompany\\MyApp\\MyApp.exe"
"Path"="C:\\Program Files\\MyCompany\\MyApp\\MyDLLs"
My application uses one version of library (a.dll), I am using another DLL(b.dll) which in-turn uses older version of the same library (a.dll) that i use. I am building the application by embedding a manifest file. The DLL i use is also using a embedded manifest file. I am having both the versions of the library in my WinSXS folder. My application is not able to load the appropriate versions of DLLs.
Will having a separate manifest file (not embedding into DLL) help resolving the problem? What is the work around?
Your situation is exactly the situation WinSxS is supposed to solve. It should be working.
Either: The manifest files are pointing to the same version, or one of the manifest files is not embedded properly, or
The shared assembly in WinSxS was installed with a configuration policy that automatically redirects requests for v1.0 to v1.1
Some clarifications are needed: App.exe and b.dll are implicitly linked against a.dll? Or do they load it via LoadLibrary.
If B.DLL loads A.DLL explicitly using LoadLibrary then you need to add ISOLATION_AWARE_ENABLED to your pre-processor definitions to ensure that LoadLibrary calls made by B.DLL look in the correct activation context. Otherwise they will be made in the context of the default activation context which was created by the EXE's manifest.
It will depend on what the duplicated DLLs do and if their versions are compatible. (e.g. Do they both access shared objects in memory? If so, there's a good chance something will blow up.)
It will also depend on how those two same-named DLLs are loaded. If it's anything other than an explicit LoadLibrary with a full path then things probably will not work. There's an ongoing discussion of this here: Determine the loaded path for DLLs
In general, it might work if you're lucky. The fact it may go catastrophically wrong is a good reason to avoid the issue entirely, if you can. (In the worst case, you could host one of the modules in another process and proxy all the calls to it. Ideally, just be able to use the same DLL version in both modules.)
I am writing a DLL to plug into another (3rd party) application. The DLL will need to depend on another set of DLLs (for license reasons I cannot link statically).
I would like my DLL to be "xcopy-deployable" to any directory. I would also like not to require adding this directory to the path.
If I just build the DLL the usual way, Windows will refuse to load the DLL, since it cannot find the DLLs next to the current process.
Are there any good options for helping Windows locate the DLL?
To answer some questions:
The DLL is written in C++.
The extra DLLs are QT-dlls.
I would like to place the extra DLLs in the same folder as my plugin DLL. I can get the name of that folder from GetModuleFileName.
The application is Firefox, the DLL is a PKCS#11 security module.
The application loads the DLL using the full path to the DLL (the user supplies it when installing the plugin).
Requiring that the DLLs be placed in System32 or next to the application would work, but it is a bit messy and could cause problems with uninstallers.
LoadLibrary and GetProcAddress would of course work, but is not really feasible in my case. I am using hundreds, if not thousands, of methods in the other DLLs. I really need to use the import-libraries.
I had thought about using delay-loaded dlls combined with SetDllDirectory in DllMain. Have anyone tried anything like this?
I can think of 3 ways.
put the dlls in the same folder as your application (you cannot do this?)
Use runtime linking. LoadLibrary() and GetProcAddress()
Use a manifest http://msdn.microsoft.com/en-us/library/aa374182(VS.85).aspx
But if the dll isn't in the same folder as the .exe, how are you going to know where it is? forget Windows not knowing, how do you know?
you can specify the path of dll as the parameter of LoadLibrary().
Another option is to modify the PATH variable. Have a batch file for launching the main app, and set the PATH=%PATH%;%~dp0. This ensures a minimal footprint, with no additional traces left in the system after running.