When I open a dump file in WinDbg, it looks for pdb files in c:\code\appV1 folder
When I open the exe file directly in WinDbg, it looks for the pdb files in c:\code\appV2 folder.
This is driving me crazy because in both cases, windbg is debugging the same exe which is at c:\MyApp\app.exe
The c:code\appV1 make sense to search in, this is path where pdb files were generated by the exe but why does it search in c:\code\appV2 which has no connection whatsoever with the dump file or the exe?
I did extensive digging into this, created a demo project and simulated crash and tried various combinations. The answer is that dump file seems to have an overriding effect on the executable when it comes to default search paths (pdb).
Here is how you can reproduce this scenario:
Create project called App1 at say c:\code\App1 and build it.
Copy the above project and rename the main folder as App1Clone and build it, say in c:\code\App1Clone
Now first copy App1 to say c:\test folder and run it. It will crash and create dump file.
Now copy the App1 from App1Clone folder to c:\test folder and run it. It will crash and create dump file.
Now both dump files are linked to c:\test\app1.exe but if the wrong app1.exe was present (=search paths didn't match) it will apparently pick the paths supposedly stored in the dump file. I don't know if dump files even store this path but this appears to be the case based on my experimentation.
Related
I try to use a shared .pch file, which is compiled in one project and used in others.
However the .pch file is deleted if a .pdb filename of the PCH project differs from .pdb filenames of the other projects.
This page doesn't answer the question: https://devblogs.microsoft.com/cppblog/shared-pch-usage-sample-in-visual-studio/
I don't want to use a same name for all PDBs.
Questions:
1) Why the .pch file is deleted at the start of other projects compilation, which leads to a C1083 error (.pch not found), if PDB names are not equal, not like in that page?
2) I copy pch.pdb and pch.idb files using COPY command, is there a RENAME comand or something, if the copied pch.pdb should be named just like a dependent project's PDB? And where can I find a complete list of Custom Build Step command?
3) I don't understand the purpose of "Additional dependencies" and "Outputs" in Custom Build Step. Can I enter the .pch filename into the dependency list, so it won't be deleted? Does the output list need to contain the dependent project's PDB name, or the pch.pdb, or both?
For some reason (I did this or not) the generated by compiler .pdb file was not $(PlatformToolsetVersion).pdb, but $(ProjectName).pdb . So the copied into other project folders shared .pdb file was pch.pdb in my case, while other projects were expecting different names. And that was triggering a DELETE task in Microsoft.CppCommon.targets , ("Delete the pch file if the pdb file has been deleted."). Instead of changing the output .pdb name I just looked into XCOPY command and made it to change the copied filename to an expected by a specific project (actually then I just added a custom Target with a renaming Copy task right into the project file instead of using the CustomBuildStep calling a xcopy OS's command, as now I learned more about MSBuild).
Then I also changed the generated by Linker output .pdb, just added "Linked" suffix to the name, so there are no conflicts between Compiler's and Linker's PDBs. Not sure if that is a good idea to change the default settings without a big reason.
I guess it's better just to change the Compiler's output PDB to $(PlatformToolsetVersion).pdb , so all projects will use the same name.
That was the first time I had looked into MSBuild and advanced project settings, now it seems to be obvious, that a project using a shared .pdb wants some familiar .pdb name, not a random pch.pdb
Here is my custom Target imported into project files copying the shared .pdb only if it was rebuilt (.idb is not generated in my case):
<Target Name="CopyFreshPchPdb" BeforeTargets="ClCompile"
Inputs="$(PchDir)\pch.pdb"
Outputs="$(IntDir)\$(ProjectName).pdb">
<Message Importance="High" Text="Copying shared pch.pdb" />
<Copy
SourceFiles="$(PchDir)\pch.pdb"
DestinationFiles="$(IntDir)\$(ProjectName).pdb">
</Copy>
</Target>
Did you use the sample code under the github link.
If so, you should download and then use that sample and if you create your own project, you should check your projects carefully.
Borrowing from this tutorial to your project, I think you need to pay attention to whether you have any additional custom targets in your xxx.vcxproj file to delete the PCH file. Therefore, you need to check each xxx.vcxproj file carefully. In vs, there will be no deletion of certain files due to the different .pdb file names of the PCH project and other projects, so check whether there are additional operations of your own.
1) Why the .pch file is deleted at the start of other projects
compilation, which leads to a C1083 error (.pch not found), if PDB
names are not equal, not like in that page?
First of All, make sure that there are no other option to delete PCH file in your projects.
The PCH project is to create a PCH file and the other two projects are to use such file. And every time when you build the two projects(reference the PCH project), and always execute the build of PCH project and then build the two project. So the PCH is always created and later be used in two projects.
Based on it, you should ensure that all three projects create and use the same address for the file.
SharedPCH project
ConsoleApplication1
ConsoleApplication2
In the sample code,the PCH file exists under SharedPchSample\Outputs\Intermediate\Shared\Win32\Debug.
2) I copy pch.pdb and pch.idb files using COPY command, is there a
RENAME comand or something, if the copied pch.pdb should be named just
like a dependent project's PDB? And where can I find a complete list
of Custom Build Step command?
Custom Build Step is under every project-->Properties-->Custom Build Step-->Command Line, and then you can find it.That custom step is just CMD command. And you can execute CMD in that to do extra opertion.
Besides, I guess you want to make those xxx.pdb and xxx.idb be the same name of the project name in order to distinguish one from another. You can right-click on every project-->Properties-->C/C++-->Output Files-->Program Database File Name-->change it and to use $(IntDir)$(ProjectName).pdb. More about Custom Build Steps, you can refer to this link.
I don't understand the purpose of "Additional dependencies" and
"Outputs" in Custom Build Step. Can I enter the .pch filename into the
dependency list, so it won't be deleted? Does the output list need to
contain the dependent project's PDB name, or the pch.pdb, or both?
Additional dependencies is set to use the PCH file's content in the project 1 and 2 which is similar to configuring the address of a reference class library in a c++ project. And I think it might be redundant and since the author has add it which implies that it is well-founded.
And Outputs is the author customized output path, the author changed the output address of the project, and started a new custom output path and a temporary output path.
Actually, the xxx.pch and its pdb and idb file will not be copied into outputpath. So the custom build step is to copied the files into temporary output path. And if you want to copied them into the final outputpath, you can also use these in CustomBuildStep.targets file:
<CustomBuildStep>
<Command>
if EXIST "$(SharedPdb)" xcopy /Y /F "$(SharedPdb)" "$(IntDir)"
if EXIST "$(SharedIdb)" xcopy /Y /F "$(SharedIdb)" "$(IntDir)"
if EXIST "$(SharedPdb)" xcopy /Y /F "$(SharedPdb)" "$(OutDir)"
if EXIST "$(SharedIdb)" xcopy /Y /F "$(SharedIdb)" "$(OutDir)"
</Command>
<Outputs>$(IntDir)vc$(PlatformToolsetVersion).pdb;</Outputs>
<Inputs>$(SharedPdb)</Inputs>
</CustomBuildStep>
And in fact, one project references another project, and the output files of the referenced project are automatically copied to the main project. Perhaps this is because the author's SharePCH project does not generate the pdb and idb files, so those files of the dependent project will not be found in the main project.
I'd like to add that I had a similar situation where a shared .pch file was being deleted but for a different reason related to the .pdb file.
The reason was that the pdb formats were different across the exe and the PCH.lib. The PCH library project has its 'Project Properties -> C/C++ -> General -> Debug Information Format' set to 'C7 compatible (/Z7)'.
When I added a new project exe that depended on the PCH library I had forgotten that new projects default to using 'Program Database (/Zi)' for its 'Debug Information Format'.
So now when the main project is being built and linked with the PCH it would delete the PCH.pch file and complain that the .PCH is missing.
Having all projects with the same matching Debug Information Format was the fix to prevent the PCH from being deleted.
I am using VS2015 debugger on my C++ app. When I start the app, debugger gives the message
Loaded 'C:\MyDir\Working\x64\Debug\MyApp.exe'. Cannot find or open the PDB file
As a consequence, I cannot set breakpoints.
There is a .pdb file in the same directory as the .exe, but it doesn't match, according to VS debugger, and also according to WidDBG Symchk. Symchk does not provide the reason for the mismatch, even with /v option.
Complete rebuild does not make this problem go away. It is only occurring for debug build, and it just started today. Before today, there was no problem with mismatched pdb's, either for debug or release builds.
The VS options I am using are:
C++: Debug Information Format=Program Database (/Zi), Program Database File Name=$(IntDir)%(Filename).pdb;
Linker: Generate Debug Info=Optimize for debugging (/DEBUG), Generate Program Database File=$(OutDir)MyApp.pdb, Generate Full Program Database File=Yes.
The pdb files for the individual objects appear in the intermediate directory, and MyApp.pdb appears in the output directory, along with MyApp.exe.
Now, here's the weird part: when delete the existing MyApp.pdb and then relink, a new .pdb file appears in the output directory with a current mod time. While the linker is running, the pdb file grows to be large (~70 MB), but as the linker completes, the pdb file suddenly becomes small (~4 MB), and the mod time changes to a few hours earlier today. This is very suspicious, and probably accounts for the pdb mismatch.
The linker's final output lines are
Finished searching libraries
MyApp.vcxproj -> C:\MyDir\Working\x64\Debug\MyApp.exe
MyApp.vcxproj -> C:\MyDir\Working\x64\Debug\\MyApp.pdb (Full PDB)
How can I force VS to produce a matched and correct pdb file for the debug build?
UPDATE: The problem was that there is a pdb file MyApp.pdb created in the intermediate directory (it is the pdb file created by the compiler for MyApp.cpp). For some reason, the linker replaces the "real" pdb file with this one at the end. Since they have the same name, MyApp.pdb, Symchk doesn't show a name mismatch (although there may be a timestamp mismatch that isn't evident).
It is not obvious how the debugging info for MyApp.cpp can be included in the final MyApp.pdb.
I am working on a project which requires me to open an HTML file and use its contents. I added it to Resource files but when I try to open it lie this:
std::ifstream templateFile;
templateFile.open("filename.html", std::ifstream::in);
The operation fails. I checked it by using templateFile.fail().
The above operation works when I provide the full path. The file lies in the project folder along with other files. I tried setting build action to content but still it doesnt work. Please Help.
Output directory, where your executable is compiled and put into differs from the source directory, where you create all your .cpp/.hpp files (I assume there is filename.html file). Local path filename.html is supposed to be local for your executable file, not the source file.
Read more about changing the output directory here: https://msdn.microsoft.com/en-us/library/ms165410.aspx
Under Configuration Properties / Debugging, see what your Working Directory is using the macros dialog box. Move your file into this folder.
Click the button shown in the figure. There, click either Edit or Browse. Browse will take you to the working directory. Edit will expose the link to open the macros box
I have written a c++ program and deployed it in say c:\my_app, and my executable's path is c:\my_app\my_app.exe. Say, my_app needs many files such as the_file.txt, which is located in c:\my_app\the_file.txt.
In my executable, I open the txt file as, xx.open("the_file.txt");
Moreover, I have associated my program with let's say .myp extension.
When I'm on Desktop, and want to open a file named example.myp, my program can not see the_file.txt. Because, it (somehow) assumes that it's currently working on Desktop.
Is there any easy way to handle this problem by changing shell command for open in HKEY_CLASSES_ROOT? The naive solution would be to change all file open operations with something like %my_app_location/the_file.txt". I don't want to do that.
Always use a full path name to open a file. In other words, don't open "foo.txt", open "c:\bar\foo.txt". To find the install directory of your EXE use GetModuleFileName(), passing NULL for the module handle.
These days you shouldn't add files to c:\my_app....
Instead use the ProgramData Folder and full paths.
Use SHGetSpecialFolderPathA with CSIDL_COMMON_APPDATA to get the ProgramData folder and the create your program directory and add your files.
You should set current directory for your app's folder with SetCurrentDirectory function. After that you can open file by name without full path
I'm including python.h in my Visual C++ DLL file project which causes an implicit linking with python25.dll. However, I want to load a specific python25.dll (several can be present on the computer), so I created a very simple manifest file named test.manifest:
<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0'>
<file name="python25.dll" />
</assembly>
And I'm merging it with the automatically embedded manifest file generated by Visual Studio thanks to:
Configuration Properties -> Manifest Tool -> Input and Output -> Additional Manifest Files
-->$(ProjectDir)\src\test.manifest
python25.dll is now loaded twice: the one requested by the manifest, and the one that Windows should find through its search order.
Screendump of Process Explorer http://dl.dropbox.com/u/3545118/python25_dll.png
Why is that happening and how can I just load the DLL file pointed by the manifest?
After exhaustive battle with WinSxS and DLL redirection, here's my advice for you:
Some background
Various things can cause a DLL file to be loaded under Windows:
Explicit linking (LoadLibrary) -- the loader uses the current activation context of the running EXE file. This is intuitive.
Implicit linking ("load time linkage", the "auto" ones) -- the loader uses the default activation context of the depending DLL file. If A.exe depends on B.dll depends on C.dll (all implicit linkage), the loader will use B.dll's activation context when loading C.dll. IIRC, it means if B's DllMain loads C.dll, it can be using B.dll's activation context -- most of the time it means the system-wide default activation context. So you get your Python DLL from %SystemRoot%.
COM (CoCreateInstance) -- this is the nasty one. Extremely subtle. It turns out the loader can look up the full path of a DLL file from the registry using COM (under HKCR\CLSID). LoadLibrary will not do any searching if the user gives it a full path, so the activation context can't affect the DLL file resolution. Those can be redirected with the comClass element and friends, see [reference][msdn_assembly_ref].
Even though you have the correct manifest, sometimes someone can still change the activation context at run time using the Activation Context API. If this is the case, there is usually not much you can do about it (see the ultimate solution below); this is just here for completeness. If you want to find out who is messing with the activation context, WinDbg bp kernel32!ActivateActCtx.
Now on to finding the culprit
The easiest way to find out what causes a DLL file to load is to use Process Monitor. You can watch for "Path containing python25.dll" or "Detail containing python25.dll" (for COM lookups). Double clicking an entry will actually show you a stack trace (you need to set the symbol search paths first, and also set Microsoft's PDB server). This should be enough for most of your needs.
Sometimes the stack trace obtained from above could be spawned from a new thread. For this purpose you need WinDbg. That can be another topic, but suffice to say you can sxe ld python25 and look at what other threads are doing (!findstack MyExeModuleName or ~*k) that causes a DLL file to load.
Real world solution
Instead of fiddling with this WinSxS thing, try hooking LoadLibraryW using Mhook or EasyHook. You can just totally replace that call with your custom logic. You can finish this before lunch and find the meaning of life again.
[msdn_assembly_ref]: Assembly Manifests
I made some progress for the understanding of the issue.
First let me clarify the scenario:
I'm building a DLL file that both embeds and extends Python, using the Python C API and Boost.Python.
Thus, I'm providing a python25.dll in the same folder as my DLL file, as well as a boost_python-vc90-mt-1_39.dll.
Then I have an EXE file which is a demo to show how to link to my DLL file: this EXE file doesn't have to be in the same folder as my DLL file, as long as the DLL file can be found in the PATH (I'm assuming that the end user may or may not put it in the same folder).
Then, when running the EXE file, the current directory is not the one containing python25.dll, and that's why the search order is used and some other python25.dll can be found before mine.
Now I figured out that the manifest technique was the good approach: I managed to redirect the loading to "my" python25.dll.
The problem is that this is the Boost DLL file boost_python-vc90-mt-1_39.dll that's responsible for the "double" loading!
If I don't load this one, then python25.dll is correctly redirected. Now I somehow have to figure out how to tell the Boost DLL file not to load another python25.dll...
Dependency Walker is usually the best tool for resolving this kind of problem. I'm not too sure how well it handles manifests though...
Where in this entangled mess is the actual process executable file?
Two possibilities come to mind:
You are writing a Python extension DLL file. So the Python process is loading your DLL file, and it would already have its own python25.dll dependency.
The EXE file loading your DLL file is being built with header files and libraries provided by the DLL file project. So it is inheriting the #pragma comment(lib,"python25.lib") from your header file and as a result is loading the DLL file itself.
My problem with the second scenario is, I'd expect the EXE file, and your DLL file, to be in the same folder in the case that the EXE file is implicitly loading your DLL file. In which case the EXE file, your DLL file and the python25.dll are all already in the same folder. Why then would the system32 version ever be loaded? The search order for implicitly loaded DLL files is always in the application EXE file's folder.
So the actual interesting question implicit in your query is: How is the system32 python26.dll being loaded at all?
Recently, I hit a very similar problem:
My application embedding Python loads the python32.dll from a known location, that is a side-by-side assembly (WinSxS) with Python.manifest
Attempt to import tkinter inside the embedded Python interpreter caused second loading of the very same python32.dll, but under a different non-default address.
The initialisation function of tkinter module (specifically, _tkinter.pyd) was failing because to invalid Python interpreter thread state (_PyThreadState_Current == NULL). Obviously, Py_Initialize() was never called for the second Python interpreter loaded from the duplicate python32.dll.
Why was the python32.dll loaded twice? As I explained in my post on python-capi, this was caused by the fact the application was loading python32.dll from WinSxS, but _tkinter.pyd did not recognise the assembly, so python32.dll was loaded using the regular DLL search path.
The Python.manifest + python32.dll assembly was recognised by the DLL loading machinery as a different module (under different activation context), than the python32.dll requested by _tkinter.pyd.
Removing the reference to Python.manifest from the application embedding Python and allowing the DLL search path to look for the DLLs solved the problem.