I'm trying to prevent dll-hijacking vulnerability in my executable, which has only one explicit dynamic dependence - winhttp.dll.
My attempt is to use linker /delayload:winhttp.dll option and call SetDefaultDllDirectories in the very beginning of the program. Seems like it is working for some modules AND for winhttp itself, but not for it's dependencies - binary is still loading mpr.dll and a few other modules before main (and it is first trying to load it from my executable folder).
It is not helping to add mpr.dll to /delayload list - I'm getting LINK : warning LNK4199: /DELAYLOAD:mpr.dll ignored; no imports found from mpr.dll.
Dependency Walker is showing me this dependence chain: winhttp -> oleaut32 -> combase -> ole32 -> mpr.
Why is this module loading before any of winhttp functions usage?
P.S. I really want to avoid using LoadLibrary + GetProcAddress manually.
Win 8.1, MSVS 2013 Update 5
Procmon:
Dep. walker:
It is crazy, but the true reason for this behaviour appears to be my executable name. It does contain a word "launcher", which seems to be the one, triggering Windows AppCompat layer.
The same executable renamed to "test1.exe" works all well.
Related
Looking for some advice as to what can cause a crash of this type in Windows when loading a dll.
I'm writing a Windows VST3 .dll plugin in C++, and experiencing a crash on startup when my .dll is loaded. When running the host application and the plugin via Visual Studio 2019 debugger I'm getting the Access violation executing location ... dialog shown above, and the call stack is empty, making me think memory is completely corrupted.
Maybe this is caused by linking options I need to tweak when building the .dll?
Here are some facts:
The plugin itself works fine. I can get it to load and unload without issues when all 3rd party API calls are commented out.
If I link some 3rd party libraries into my plugin, then again everything works fine.
Once I make a call to any of those 3rd party libraries, then that causes the segfault on startup.
Plugin is built with /MD, but I also tried /MT and saw the same behaviour.
I'm using JUCE C++ to get the VST3 framework and GUI components.
An example of one of the 3rd party library I'm linking against and attempting to call into is the SuperpoweredSDK C++ library. But I've tried other libraries, and I'm seeing the same behaviour with each one.
Unfortunately, getting an entire blank VST3 plugin together is too much code to post in a StackOverflow question. But since I can get the plugin to work if I comment out all calls to 3rd party libraries, I don't think the code itself is the issue, I think the way the dll (.vst3) file is built is the problem.
The critical portion of the CMakeLists.txt file that builds the .vst3 dll file looks like this:
FIND_LIBRARY ( SuperpoweredSDK NAMES SuperpoweredWin141_Debug_MD_x64.lib PATHS ${SUPERPOWERED_DIR}/libWindows )
SET ( TEST_EXTERNAL_DEPS PRIVATE Threads::Threads ${SuperpoweredSDK} )
FILE ( GLOB JUCE_SOURCE ${CMAKE_HOME_DIRECTORY}/JuceLibraryCode/*.cpp )
LIST ( FILTER JUCE_SOURCE EXCLUDE REGEX ".+/JuceLibraryCode/include_juce_audio_plugin_client_[^u].+" )
FILE ( GLOB PLUGIN_SOURCE *.cpp )
ADD_LIBRARY ( TestVST3 SHARED ${JUCE_SOURCE} ${PLUGIN_SOURCE} ${CMAKE_HOME_DIRECTORY}/JuceLibraryCode/include_juce_audio_plugin_client_VST3.cpp )
SET_TARGET_PROPERTIES ( TestVST3 PROPERTIES OUTPUT_NAME "Test" SUFFIX ".vst3" )
TARGET_LINK_LIBRARIES ( TestVST3 PUBLIC ${TEST_EXTERNAL_DEPS} )
what can cause a crash of this type in Windows when loading a dll
Typical reason is initializer of some global variable. Example code that will crash when loading the DLL:
class C
{
public:
C() { __debugbreak(); } // Compiles into `int 3` interrupt, will definitely crash
};
static C g_c;
When you link an external library but don’t call it, linker’s dead code elimination feature often drops complete .obj file which are linked but unused. This also drops variables there, along with their constructors.
One way to debug, set breakpoint on DllMain. But not the DllMain in your code, another one, that’s actually exported. In VC++ 2017 CRT that one is called dllmain_dispatch, the source file for that is in VC++ runtime, crt\src\vcruntime\dll_dllmain.cpp. That dllmain_dispatch calls a couple of functions before calling your DllMain, one of them, dllmain_crt_process_attach, calls _initterm and that one actually calls constructors of all the global stuff. At least one of them fails in your case.
P.S. A possible reason of a crash like this, your third-party library is trying to use COM, and the thread which called LoadLibrary never called CoInitialize().
We eventually did find the cause of this crash. With VST3 files, the DAW loads each .dll (or .vst3), queries the capabilities, and immediately unloads the dll again. It then moves on to the next file, and the process starts all over until all plugins have been queried.
In this case, one of the 3rd party libs I was using (SuperpoweredSDK) starts a background thread to verify the license against a web server somewhere. This was not documented. With the background thread started, the DAW unknowingly proceeds to unload the DLL, leaving the thread hanging there running code that no longer exists! And thus, the crash without any call stack.
Several possible solutions:
Move the SuperpoweredSDK initialization code until later when it really is needed.
In a future not-yet-released version of SuperpoweredSDK (current version is 2.0.1), they're apparently adding an API call to shutdown Superpowered, and that shutdown call will wait until all background threads have finished running. Make sure you call this new API.
How to get from C/C++ extension source code to a pyd file for windows (or other item that I could import to Python)?
edit: The specific library that I wanted to use (BRISK) was included in OpenCV 2.4.3 so my need for this skill went away for the time being. In case you came here looking for BRISK, here is a simple BRISK in Python demo that I posted.
I have the Brisk source code (download) that I would like to build and use in my python application. I got as far as generating a brisk.pyd file... but it was 0 bytes. If there is a better / alternative way to aiming for a brisk.pyd file, then of course I am open to that as well.
edit: Please ignore all the attempts in my original question below and see my answer which was made possible by obmarg's detailed walkthrough
Where am I going wrong?
Distutils without library path: First I tried to build the source as is with distutils and the following setup.py (I have just started learning distutils so this is a shot in the dark). The structure of the BRISK source code is at the bottom of this question for reference.
from distutils.core import setup, Extension
module1 = Extension('brisk',
include_dirs = ['include', 'C:/opencv2.4/build/include', 'C:/brisk/thirdparty/agast/include'],
#libraries = ['agast_static', 'brisk_static'],
#library_dirs = ['win32/lib'],
sources = ['src/brisk.cpp'])
setup (name = 'BriskPackage',
ext_modules = [module1])
That instantly gave me the following lines and a 0 byte brisk.pyd somewhere in the build folder. So close?
running build
running build_ext
Distutils with library path: Scratch that attempt. So I added the two library lines that are commented out in the above setup.py. That seemed to go ok until I got this linking error:
creating build\lib.win32-2.7
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:win32/lib /LIB
PATH:C:\Python27_32bit\libs /LIBPATH:C:\Python27_32bit\PCbuild agast_static.lib brisk_static.lib /EXPORT:initbrisk build
\temp.win32-2.7\Release\src/brisk.obj /OUT:build\lib.win32-2.7\brisk.pyd /IMPLIB:build\temp.win32-2.7\Release\src\brisk.
lib /MANIFESTFILE:build\temp.win32-2.7\Release\src\brisk.pyd.manifest
LINK : error LNK2001: unresolved external symbol initbrisk
build\temp.win32-2.7\Release\src\brisk.lib : fatal error LNK1120: 1 unresolved externals
error: command '"C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\link.exe"' failed with exit status 1120
Uncontrolled flailing: I thought maybe the libraries needed to be built, so I did a crash course (lots of crashing) with cmake + mingw - mingw + vc++ express 2010 as follows:
cmake gui: source: c:/brisk, build: c:/brisk/build
cmake gui: configure for Visual Studio 10
cmake gui: use default options and generate (CMAKE_BACKWARDS_COMPATIBILITY, CMAKE_INSTALL_PREFIX, EXECUTABLE_OUTPUT_PATH, LIBRARY_OUTPUT_PATH)
VC++ Express 10: Change to Release and build the solution generated by cmake and get about 20 pages of what look like non-critical warnings followed by all succeeded. Note - no dlls are generated by this. It does generate the following libraries of similar size to the ones included with the download:
win32/lib/Release/
agast_static.lib
brisk_static.lib
Further flailing.
Relevant BRISK source file structure for reference:
build/ (empty)
include/brisk/
brisk.h
hammingsse.hpp
src
brisk.cpp
demo.cpp
thirdparty/agast/
include/agast/
agast5_8.h ....
cvWrapper.h
src/
agast5_8.cc ...
CMakeLists.txt
win32/
bin/
brisk.mexw32
opencv_calib3d220.dll ...
lib/
agast_static.lib
brisk_static.lib
CMakeLists.txt
FindOpenCV.cmake
Makefile
Are you sure that this brisk library even exports python bindings? I can't see any reference to it in the source code - it doesn't even seem to import python header files. This would certainly explain why you've not had much success so far - you can't just compile plain C++ code and expect python to interface with it.
I think your second distutils example is closest to correct - it's obviously compiling things and getting to the linker stage, but then you encounter this error. That error just means it can't find a function named initbrisk which I'm guessing would be the top level init function for the module. Again this suggests that you're trying to compile a python module from code that isn't meant for it.
If you want to wrap the C++ code in a python wrapper yourself you could have a look at the official documentation on writing c/c++ extensions. Alternatively you could have a look into boost::python, SIP or shiboken which try to somewhat (or completely) automate the process of making python extensions from C++ code.
EDIT: Since you seem to have made a decent amount of effort to solve the problem yourself and have posted a good question, I've decided to give a more detailed response on how to go about doing this.
Quick Tutorial On Wrapping C++ Libraries Using boost::python
Personally I've only ever used boost::python for stuff like this, so I'll try and give you a good summary of how to go about doing that. I'm going to assume that you're using Visual C++ 2010. I'm also going to assume that you've got a 32bit version of python installed, as I believe the boost pro libraries only provide 32bit binaries.
Installing boost
First you'll need to grab a copy of the boost library. The easiest way to do this is to download an installer from the boost pro website. These should install all the header files and binary files that are required for using the boost c++ library on windows. Take note of where you install these files to, as you'll need them later on - it might be best to install to a path without a space in it. For easyness I'm going to assume you put these files in C:\boost but you can substitute that for the path you actually used.
Alternatively, you can follow these instructions to build boost from source. I'm not 100% sure, but it might be the case that you need to do this in order to get a version of boost::python that is compatible with the version of python you have installed.
Setting up a visual studio project
Next, you'll want to setup a visual studio project for brisk.pyd. If you open visual studio, go to New -> Project then find the option for Win32 Project. Set up your location etc. and click ok. In the wizard that appears select a DLL project type, and then tick the empty project checkbox.
Now that you've created your project, you'll need to set up the include & library paths to allow you to use python, boost::python and the brisk.lib file.
In Visual Studios solution explorer, right click on your project, and select properties from the menu that appears. This should open up the property pages for your project. Go to the Linker -> General section and look for the Additional Library Directories section. You'll need to fill this in with the paths to the .lib files for boost, python and your brisk_static.lib. Generally these can be found in lib (or libs) subdirectories of
wherever you've installed the libraries. Paths are seperated with semicolons. I've attached a screenshot of my settings below:
Next, you'll need to get visual studio to link to the .lib files. These sections can be found in the Additional Dependencies field of the Linker -> Input section of the properties. Again it's a semicolon delimited list. You should need to add in libraries for python (in my case this is python27.lib but this will vary by version) and brisk_static.lib. These do not require the full path as you added that in the previous stage. Again, here's a screenshot:
You may also need to add the boost_python library file but I think boost uses some header file magic to save you the trouble. If I'm incorrect then have a look in you boost library path for a file named similar to boost_python-vc100-mt.lib and add that in.
Finally, you'll need to setup the include paths to allow your project to include the relevant C++ header files. To get the relevant settings to appear in project properties, you'll need to add a .cpp file to your project. Right click the source files folder in your solution explorer, and then go to add new item. Select a C++ File (.cpp) and name it main.cpp (or whatever else you want).
Next, go back to your project properties and go to C/C++ -> General. Under the additional libraries directory you need to add the include paths for brisk, python and boost. Again, semicolons for seperators, and again here's a screenshot:
I suspect that you might need to update these settings to include the opencv2 & agast libraries as well but I'll leave that as a task for you to figure out - it should be much the same process.
Wrapping existing c++ classes with boost::python.
Now comes the slightly trickier bit - actually writing C++ to wrap your brisk library in boost python. You can find a tutorial for this here but i'll try and go over it a bit as well.
This will be taking place in the main.cpp file you created earlier. First, add the relevant include statements you'll need at the top of the file:
#include <brisk/brisk.h>
#include <Python.h>
#include <boost/python.hpp>
Next, you'll need to declare your python module. I'm assuming you'd want this to be called brisk, so you do something like this:
BOOST_PYTHON_MODULE(brisk)
{
}
This should tell boost::python to create a python module named brisk.
Next it's just a case of going through all the classes & structs that you want to wrap and declaring boost python classes with them. The declerations of the classes should all be contained in brisk.h. You should only wrap the public members of a class, not any protected or private members. As a quick example, I've done a couple of the structs here:
BOOST_PYTHON_MODULE(brisk)
{
using namespace boost::python;
class_< cv::BriskPatternPoint >( "BriskPatternPoint" )
.def_readwrite("x", &cv::BriskPatternPoint::x)
.def_readwrite("y", &cv::BriskPatternPoint::y)
.def_readwrite("sigma", &cv::BriskPatternPoint::sigma);
class< cv::BriskScaleSpace >( "BriskScaleSpace", init< uint8_t >() )
.def( "constructPyramid", &cv::BriskScaleSpace::constructPyramid );
}
Here I have wrapped the cv::BriskPatternPoint structure and the cv::BriskScaleSpace class. Some quick explanations:
class_< cv::BriskPatternPoint >( "BriskPatternPoint" ) tells boost::python to declare a class, using the cv::BriskPatternPoint C++ class, and expose it as BriskPatternPoint in python.
.def_readwrite("y", &cv::BriskPatternPoint::y) adds a readable & writeable property to the BriskPatternPoint class. The property is named y, and will map to the BriskPatternPoint::y c++ field.
class< cv::BriskScaleSpace >( "BriskScaleSpace", init< uint8_t >() ) declares another class, this time BriskScaleSpace but also provides a constructor that accepts a uint8_t (an unsigned byte - which should just map to an integer in python, but I'd be careful to not pass in one greater than 255 bytes - I don't know what would happen in that situation)
The following .def line just declares a function - boost::python should (I think) be able to determine the argument types of functions automatically, so you don't need to provide them.
It's probably worth noting that I haven't actually compiled any of these examples - they might well not work at all.
Anyway, to get this fully working in python it should just be a case of doing similar for every structure, class, property & function that you want accessible from python - which is potentially quite a time consuming task!
If you want to see another example of this in action, I did this here to wrap up this class
Building & using the extension
Visual studio should take care of building the extension - then using it is just a case of taking the .DLL and renaming it to .pyd (you can get VS to do this for you, but I'll leave that up to you).
Then you just need to copy your python file to somewhere on your python path (site-packages for example), import it and use it!
import brisk
patternPoint = brisk.BriskPatternPoint()
....
Anyway, I have spent a good hour or so writing this out - so I'm going to stop here. Apologies if I've left anything out or if anything isn't clear, but I'm doing this mostly from memory. Hopefully it's been of some help to you. If you need anything clarified please just leave a comment, or ask another question.
In case someone needs it, this what I have so far. Basically a BriskFeatureDetector that can be created in Python and then have detect called. Most of this is just confirming/copying what obmarg showed me, but I have added the details that get all the way to the pyd library.
The detect method is still incomplete for me though since it does not convert data types. Anyone who knows a good way to improve this, please do! I did find, for example, this library which seems to convert a numpy ndarray to a cv::Mat, but I don't have the time to figure out how to integrate it now. There are also other data types that need to be converted.
Install OpenCV 2.2
for the setup below, I installed to C:\opencv2.2
Something about the API or implementation has changed by version 2.4 that gave me problems (maybe the new Algorithm object?) so I stuck with 2.2 which BRISK was developed with.
Install Boost with Boost Python
for the setup below, I installed to C:\boost\boost_1_47
Create a Visual Studio 10 Project:
new project --> win32
for the setup below, I named it brisk
next --> DLL application type; empty project --> finished
at the top, change from Debug Win32 to Release Win32
Create main.cpp in Source Files
Do this before the project settings so the C++ options become available in the project settings
#include <boost/python.hpp>
#include <opencv2/opencv.hpp>
#include <brisk/brisk.h>
BOOST_PYTHON_MODULE(brisk)
{
using namespace boost::python;
//this long mess is the only way I could get the overloaded signatures to be accepted
void (cv::BriskFeatureDetector::*detect_1)(const cv::Mat&,
std::vector<cv::KeyPoint, std::allocator<cv::KeyPoint>>&,
const cv::Mat&) const
= &cv::BriskFeatureDetector::detect;
void (cv::BriskFeatureDetector::*detect_vector)(const std::vector<cv::Mat, std::allocator<cv::Mat>>&,
std::vector< std::vector< cv::KeyPoint, std::allocator<cv::KeyPoint>>, std::allocator< std::vector<cv::KeyPoint, std::allocator<cv::KeyPoint>>>>&,
const std::vector<cv::Mat, std::allocator<cv::Mat>>&) const
= &cv::BriskFeatureDetector::detect;
class_< cv::BriskFeatureDetector >( "BriskFeatureDetector", init<int, int>())
.def( "detect", detect_1)
;
}
Project Settings (right-click on the project --> properties):
Includes / Headers
Configuration Properties --> C/C++ --> General
add to Additional Include Directories (adjust to your own python / brisk / etc. base paths):
C:\opencv2.2\include;
C:\boost\boost_1_47;
C:\brisk\include;C:\brisk\thirdparty\agast\include;
C:\python27\include;
Libraries (linker)
Configuration Properties --> Linker --> General
add to Additional Library Directories (adjust to your own python / brisk / etc. base paths):
C:\opencv2.2\lib;
C:\boost\boost_1_47\lib;
C:\brisk\win32\lib;
C:\python27\Libs;
Configuration Properties --> Linker --> Input
add to Additional Dependencies (adjust to your own python / brisk / etc. base paths):
opencv_imgproc220.lib;opencv_core220.lib;opencv_features2d220.lib;
agast_static.lib; brisk_static.lib;
python27.lib;
.pyd output instead of .dll
Configuration Properties --> General
change Target Extension to .pyd
Build and rename if necessary
Right-click on the solution and build/rebuild
you may need to rename the output from "Brisk.pyd" to "brisk.pyd" or else python will give you errors about not being able to load the DLL
Make brisk.pyd available to python by putting it in site packages or by putting a .pth file that links to its path
Update Path environment variable
In windows settings, make sure the following are included in your path (again, adjust to your paths):
`C:\boost\boost_1_47\lib;C:\brisk\win32\bin`
I am developing a firewall application which is complete for Windows XP.
To accomplish this, I am creating a dll file, which can be used in C# file.
For Vista or later, I used this code, but it's not compiling. I have win sdk 2008 and it is showing this error:
Error 19 error LNK2019: unresolved external symbol _FwpmEngineOpen0#20 referenced in function "private: unsigned long __thiscall PacketFilter::CreateDeleteInterface(bool)" (?CreateDeleteInterface#PacketFilter##AAEK_N#Z) PacketFilter.obj FirewallVista ".
It must be some project property setting problem because I know all lib files are imported correctly.
It seems like you're trying to compile the simple WFP project from codeproject. The guys above probably didn't understand the questions but the answer was:
find the first function linker fails at and find it on MSDN, eg. FwpmEngineOpen0 and find which library it needs to link to and add it to the project properties / Linker / Input / Additional Dependencies
Actually in the sample, there was another function failing to link UuidCreate
Long story short:
Create a plain Win32 console application (I did it with VS2012) and build it - make sure it works
Add the header file to the project (PacketFilter.h) to the project
Replace the main source file with (PacketFilter.cpp) contents, but don't delete the include for "stdafx.h" on the top of the file
Add the following libraries into the linker section (as per above)
Fwpuclnt.lib
Rpcrt4.lib
Build and run the application (you might need to change the character set to multi-byte too).
You'll have to modify the IP address in the source to make the sample work. I picked a local IP address and did
ping -n 100 192.168.100.200
See the ping returning, start the exe, and see ping failing to reach destination (timing out) - eg. firewall is working. Then press any key to stop firewall and see ping reaching destination again.
I would've posted the link to the codeproject page, but as I have less than 10 credit - I can't :-P
Google for: "Firewall using Vista's Windows Filtering Platform APIs"
Hope this helps anyone who tries to build this sample.
You are missing a library reference. Check that you have imported the .lib that your code is expecting.
EDIT: The missing import is from the library you're talking about itself. That probably indicates you forgot to import the .lib of that actual library, or you need to attach the .cpp files from that project into your own project. In other words, you need more than the headers.
Also, if you're working from .NET, consider using the plain Vista Firewall API because it is already exposed as a COM coclass, which .NET can talk with natively.
Right click on your project name in Solution explorer, select Properties. Then open Linker -> Input. Open the dropdown next to Additional Dependencies select <Edit>. Then type the following in the top textbox:
%(AdditionalDependencies)
$(DDK_LIB_PATH)\NTOSKrnl.lib
$(DDK_LIB_PATH)\FwpKClnt.lib
$(DDK_LIB_PATH)\NetIO.lib
$(DDK_LIB_PATH)\NDIS.lib
$(DDK_LIB_PATH)\WDMSec.lib
$(SDK_LIB_PATH)\UUID.lib
I'm writing a wrapper program that loads Winamp input plugins. I've got it working well so far with quite a few plugins, but for some others, I get an error message at runtime when I attempt to call LoadLibrary on the plugin's DLL. (It seems to happen mostly with plugins that were included with Winamp.) A dialog appears and gives me the error code and message above. This happens, for example, with the in_flac.dll and in_mp3.dll plugins (which come with Winamp). Any ideas on how I can remedy this situation?
EDIT:
This basically iterates through the plugins in a directory and attempts to load and then free each one. Some plugins produce the error I mentioned above, while others do not.
wstring path = GetSearchPath();
FileEnumerator e(path + L"in_*.dll");
while(e.MoveNext()) {
wstring pluginPath = path + e.GetCurrent().cFileName;
MessageBoxW(NULL, pluginPath.c_str(), L"Message", MB_OK);
HINSTANCE dll = LoadLibraryW(pluginPath.c_str());
if(!dll) {
pluginPath = wstring(L"There was an error loading \"") + wstring(e.GetCurrent().cFileName) + L"\":\n" + LastErrorToString();
MessageBoxW(NULL, pluginPath.c_str(), L"Error", MB_OK);
continue;
}
FreeLibrary(dll);
}
Starting from Visual Studio 2005, the C/C++ runtime MUST be put in the Windows side-by-side cache (C:\windows\WinSxS), so putting the CRT DLL's next to your exe doesn't work anymore (with one exception, see later).
You MUST also refer to the CRT DLL's via a manifest file. This manifest file is generated by the linker and will have a name like myexe.exe.manifest or mydll.dll.manifest. Distribiute this manifest with your application/DLL or link it in the exe/dll using the mt command.
The side-by-side cache and the manifest file system were introduced in Windows XP and are mainly intended to solve the DLL hell and to increase security.
Not referring to the CRT using a manifest or not putting the CRT in the side-by-side cache will generate error 6034.
If you still want to put the CRT DLL's next to your application, you could also use private assemblies, which means creating a kind of mini-side-by-side cache in the folder of your application. You can find additional information on MSDN.
there can be many reason... put your code here for clarification... one of the many solution might be is
Rebuild your application with a manifest. Building an application with Visual Studio automatically puts the manifest into the resulting EXE or DLL file. If you are building at the command line, use the mt.exe tool to add the manifest as a resource. Use resource ID 1 if building an EXE, 2 if building a DLL.
Also, you may try adding this into the code:
#pragma comment(linker,"/manifestdependency:\"type='win32' name='Microsoft.VC90.CRT' version='9.0.21022.8' processorArchitecture='x86' publicKeyToken='1fc8b3b9a1e18e3b'\"")
I'm including python.h in my Visual C++ DLL file project which causes an implicit linking with python25.dll. However, I want to load a specific python25.dll (several can be present on the computer), so I created a very simple manifest file named test.manifest:
<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0'>
<file name="python25.dll" />
</assembly>
And I'm merging it with the automatically embedded manifest file generated by Visual Studio thanks to:
Configuration Properties -> Manifest Tool -> Input and Output -> Additional Manifest Files
-->$(ProjectDir)\src\test.manifest
python25.dll is now loaded twice: the one requested by the manifest, and the one that Windows should find through its search order.
Screendump of Process Explorer http://dl.dropbox.com/u/3545118/python25_dll.png
Why is that happening and how can I just load the DLL file pointed by the manifest?
After exhaustive battle with WinSxS and DLL redirection, here's my advice for you:
Some background
Various things can cause a DLL file to be loaded under Windows:
Explicit linking (LoadLibrary) -- the loader uses the current activation context of the running EXE file. This is intuitive.
Implicit linking ("load time linkage", the "auto" ones) -- the loader uses the default activation context of the depending DLL file. If A.exe depends on B.dll depends on C.dll (all implicit linkage), the loader will use B.dll's activation context when loading C.dll. IIRC, it means if B's DllMain loads C.dll, it can be using B.dll's activation context -- most of the time it means the system-wide default activation context. So you get your Python DLL from %SystemRoot%.
COM (CoCreateInstance) -- this is the nasty one. Extremely subtle. It turns out the loader can look up the full path of a DLL file from the registry using COM (under HKCR\CLSID). LoadLibrary will not do any searching if the user gives it a full path, so the activation context can't affect the DLL file resolution. Those can be redirected with the comClass element and friends, see [reference][msdn_assembly_ref].
Even though you have the correct manifest, sometimes someone can still change the activation context at run time using the Activation Context API. If this is the case, there is usually not much you can do about it (see the ultimate solution below); this is just here for completeness. If you want to find out who is messing with the activation context, WinDbg bp kernel32!ActivateActCtx.
Now on to finding the culprit
The easiest way to find out what causes a DLL file to load is to use Process Monitor. You can watch for "Path containing python25.dll" or "Detail containing python25.dll" (for COM lookups). Double clicking an entry will actually show you a stack trace (you need to set the symbol search paths first, and also set Microsoft's PDB server). This should be enough for most of your needs.
Sometimes the stack trace obtained from above could be spawned from a new thread. For this purpose you need WinDbg. That can be another topic, but suffice to say you can sxe ld python25 and look at what other threads are doing (!findstack MyExeModuleName or ~*k) that causes a DLL file to load.
Real world solution
Instead of fiddling with this WinSxS thing, try hooking LoadLibraryW using Mhook or EasyHook. You can just totally replace that call with your custom logic. You can finish this before lunch and find the meaning of life again.
[msdn_assembly_ref]: Assembly Manifests
I made some progress for the understanding of the issue.
First let me clarify the scenario:
I'm building a DLL file that both embeds and extends Python, using the Python C API and Boost.Python.
Thus, I'm providing a python25.dll in the same folder as my DLL file, as well as a boost_python-vc90-mt-1_39.dll.
Then I have an EXE file which is a demo to show how to link to my DLL file: this EXE file doesn't have to be in the same folder as my DLL file, as long as the DLL file can be found in the PATH (I'm assuming that the end user may or may not put it in the same folder).
Then, when running the EXE file, the current directory is not the one containing python25.dll, and that's why the search order is used and some other python25.dll can be found before mine.
Now I figured out that the manifest technique was the good approach: I managed to redirect the loading to "my" python25.dll.
The problem is that this is the Boost DLL file boost_python-vc90-mt-1_39.dll that's responsible for the "double" loading!
If I don't load this one, then python25.dll is correctly redirected. Now I somehow have to figure out how to tell the Boost DLL file not to load another python25.dll...
Dependency Walker is usually the best tool for resolving this kind of problem. I'm not too sure how well it handles manifests though...
Where in this entangled mess is the actual process executable file?
Two possibilities come to mind:
You are writing a Python extension DLL file. So the Python process is loading your DLL file, and it would already have its own python25.dll dependency.
The EXE file loading your DLL file is being built with header files and libraries provided by the DLL file project. So it is inheriting the #pragma comment(lib,"python25.lib") from your header file and as a result is loading the DLL file itself.
My problem with the second scenario is, I'd expect the EXE file, and your DLL file, to be in the same folder in the case that the EXE file is implicitly loading your DLL file. In which case the EXE file, your DLL file and the python25.dll are all already in the same folder. Why then would the system32 version ever be loaded? The search order for implicitly loaded DLL files is always in the application EXE file's folder.
So the actual interesting question implicit in your query is: How is the system32 python26.dll being loaded at all?
Recently, I hit a very similar problem:
My application embedding Python loads the python32.dll from a known location, that is a side-by-side assembly (WinSxS) with Python.manifest
Attempt to import tkinter inside the embedded Python interpreter caused second loading of the very same python32.dll, but under a different non-default address.
The initialisation function of tkinter module (specifically, _tkinter.pyd) was failing because to invalid Python interpreter thread state (_PyThreadState_Current == NULL). Obviously, Py_Initialize() was never called for the second Python interpreter loaded from the duplicate python32.dll.
Why was the python32.dll loaded twice? As I explained in my post on python-capi, this was caused by the fact the application was loading python32.dll from WinSxS, but _tkinter.pyd did not recognise the assembly, so python32.dll was loaded using the regular DLL search path.
The Python.manifest + python32.dll assembly was recognised by the DLL loading machinery as a different module (under different activation context), than the python32.dll requested by _tkinter.pyd.
Removing the reference to Python.manifest from the application embedding Python and allowing the DLL search path to look for the DLLs solved the problem.