Suppose that I compiled a dynamic library (Windows DLL and/or Linux shared object file, .so) in debug mode for use by a client application that links to it dynamically. My source code is available to the client application developer.
I need some clarification regarding the following debugging scenario. I've always understood/assumed that in order for the client application to debug into my library
(for e.g. in order for a client application developer to step into my source code while debugging, say using F10 in MS VC++), that they would have to have actually built a local copy of my libraries themselves (with access to my source code), or atleast have local access to my source code without having built it (not sure if that would suffice?).
Am I right on this? In other words, I think it is not merely enough to provide libraries with debugging symbols (PDB files in MS VC++) if the client application is linking dynamically to my application which has itself been built dynamically. Appreciate if anyone can help sort this out for me? How about the situation in Linux? My understanding again is the same as the above. Now if I had compiled a static library (Windows LIB and/or Linux library .a); my understanding is that the they then don't need to have a locally build copy of my source code (I haven't tried this one out yet)?
Is/are my premise(s) correct? If not, can someone kindly provide some detailed explanation preferably with an example? Thanks for your input.
As requested, here's my comment as an answer. Since it only addresses the Windows side of things, anybody who has the Linux (or Mac!) part of the answer is free to edit it in (I've marked this as a community wiki answer).
For VC++, the debug build DLL + matching PDB + matching source is all you need. The hard part is getting them all to match ;-)
Also, it works more smoothly if the source files are at the same path as when the DLL was compiled, but Visual Studio is also perfectly capable of prompting you to browse to the source manually if you have it.
I have more experience with Windows than linux. But I would think the concept is similar.
if the client application is linking dynamically to my application which has itself been built dynamically.
I'm not quite sure if I understand "building dynamically". You might be confused with the dynamic aspect of dll? dll is linked at runtime (not build time) to allow a part of component to be deployed without a full app. For example, an app on Windows that rely on a dll provided by the OS are not impacted when Windows updates that dll as long as the interface is maintained. The only difference between a dll and exe is that dll's entry function is dllmain as opposed to main in exe.
(The only "dynamic build" concept I can think of is building templated classes. But I don't think that's what you mean here.)
Hence, debugging a .dll isn't different from debugging a .exe, it's just that .dll is a separate binary file from the executable. All the source code provide is allowing debugger to align the stepping with lines in source code. When source code is not available, then debugger can still step through assembly code with symbols.
When situation doesn't allow, then developers who are good at reading assembly code can do debugging with only symbols and no source code.
You can usually build a binary with optimized option, then compiler might optimize the assembly code so much that source code alignment in the debugger might not be possible. This usually happens with released code. In those cases when you step through the code, you sometimes see the line or condition jumps that are seemingly different from what you would expect. There is the same on .exe, .exe with libs, or .dll. This is probably why you thought it is always necessary to build your own binary to debug dlls?
Related
I have read these two SO questions: Which runtime libraries to ship? and License of runtime libraries included in GCC? - both were very helpful but not quite what I was looking for.
I have always just written programs for use on my own machine, which has never caused me any problems, but now I want to start running software on other machines and I'm wary of the runtime requirements.
EDIT: See below example instead, this was misleading.
Specifically, if I write a C++ program on a Windows machine, compiled with gcc through MinGW, and want to run it on another machine:
Do I have to send the libstdc++.dll with my program?
Is this single file (I assume placed in the executable's directory) sufficient to allow the program to run?
Also, an identical example, except this time it is an Objective-C program. Is sending the libobjc.dll file to the other machine sufficient to allow the program to execute properly?
I am used to running programs on machines which have developer tools, etc, installed, but now I'm looking to run them on general purpose machines (friends', colleagues' etc), and I'm not quite sure what to do!
EDIT: In response to edifice's answer, I feel I should clarify what it is I'm looking for. I know how to identify the necessary DLL(s) (/dylibs, etc) that my programs use, (although I am accustomed to doing that work manually; I had not heard of any of the tools). My question was more "What do I do now?"
A more general example is probably needed:
Let's say I have written a program which has object files derived from C++, C and/or Objective-C(2) code. I have used some Windows API code which compiled successfully using MinGW's gcc. I also have a custom DLL I wrote in Visual Studio (C++).
I have identified which DLL's my program will use at runtime (one of which may be GCC's libobjc.dll, I'm not sure if this would/should make a difference on a Windows machine, but I want to make this as general as possible) - The "prerequisite DLLs".
I would like to run it on my colleagues' computers, most of which run Windows 7, but some now run Windows 8. Starting at the very start for the sake of completeness:
Do I need to transfer the prerequisite DLLs to my colleagues' computers?
What directory should I place them in? (exe directory / a system directory?)
Once in place, will the presence of these DLLs allow the program to execute correctly? (Assuming it knows where to find them)
Are there any other files that should be transferred with the DLLs?
Basically I'm trying to determine the entire thought-process for developing and running an application on another machine in terms of system runtime requirements.
When loading DLLs, the first place Windows looks is the directory that the exe is in. So it will probably work just fine to put the DLLs there.
For the Microsoft DLLs though, I think it makes more sense to ask your colleague to install the Visual C++ runtime, which is a redistributable package from Microsoft. Ideally you would make an installer using something like WiX and it would install that prerequisite for you, but it is OK to just tell your colleague to do it.
Be sure to include a license file with your software if you include DLLs from gcc, because the GPL requires it.
libstdc++ isn't necessarily sufficient. You almost certainly need libgcc too, but actual dependencies are liable to vary with your particular application.
The best way to determine what you need to ship with your application is to load your EXE into a program like Dependency Walker.
Just as an example, I've compiled a test C++ program which simply prints a std::string. As you can see, it depends directly on two modules other than those that come with Windows; libgcc_s_dw2-1.dll in addition to libstdc++-6.dll.
You should remember to expand the tree under each DLL to make sure that it itself doesn't have any other dependencies (if A depends on B, B might depend on C even if A doesn't directly depend on C).
If you're worried and want the strongest assurances, you could install Windows into a virtual machine (VirtualBox is free) and test your application inside it. If you use Microsoft APIs, you may wish to check the MSDN documentation to see with what version of Windows they were introduced and ensure that it aligns with your target minimum Windows version.
Update: As xtofl points out this won't cover libraries loaded dynamically using LoadLibrary. If you want to cover this base, use Process Monitor to examine what DLL files are touched when you run the application. (Add an 'Image Path' criterion with the path to your EXE in order not to get flooded.) This has the added advantage that it covers all files, registry entries, etc. that your application depends on, not just DLLs.
Assume this scenario: An application (app.exe) is using multiple .dlls. I am debugging a function, bugged_function() from one of the .dlls used by the app: util.dll. While I am debugging bugged_function() from this I realize that something in the code is wrong and changes have to be made.
Steps to perform normally:
1. close app.exe
2. modify code in the function
3. recompile util.dll
4. rerun app.exe
What I want:
Bypass step 1 and 4. To do that I need to unload in some way, if possible, util.dll library so when compiling it can be overridden. The the application must somehow reload the library again.
EDIT 1:
I do not know how bugged_function() is called. Assume that I only have access to the source code of the library util.dll used by app.exe.
EDIT 2:
I am using Visual Studio 2010, and when I debug, I attach to app.exe process.
If the application is using the dll via run-time dynamic linking, it could be unloaded (FreeLibrary or similar), then reloaded (LoadLibrary or similar).
If the application is using the dll via load-time dynamic linking, I think you're out of luck.
Edit: I misread the question slightly. Since you can't modify app.exe, you'll have to rely on built-in functionality of that application for runtime loading and unloading, if it has it. That depends totally on the application.
Visual studio can edit and continue. So if you are at a breakpoint, you can make the changes you need to then continue your debugging. Visual studio will compile and apply the changes while maintaining state.
Edit: fixed edit and continue naming.
My basic issue is this: my program (MyProgram.exe) has a dependency on a DLL from another program (OtherProgram), and I'm trying to avoid repackaging a new DLL every time OtherProgram updates. I'd like to have MyProgram.exe link in OtherProgram's DLL when it launches, but I'm not completely sure that Windows allows for this. So if there is some kind of workaround that would also be acceptable.
And just for some background, the platform is Windows 7 x64, and MyProgram.exe runs fine when I create a symlink in the MyProgram.exe project directory to the DLL in OtherProgram's install directory. When I try to run it without the symlink, I get the "program can't start because OtherProgramDLL.dll is missing from your computer" error.
Any advice or links to relevant info is greatly appreciated!
EDIT: Clarification: the DLL is not linked at compile-time, this issue crops up at runtime
There are two types of dynamic linking in the Windows world:
Load-Time linking is when a DLL is loaded automatically when your program starts up. Windows finds this DLL using a specific algorithm I'll discuss below.
Run-Time linking is when you specifically load a DLL by calling LoadLibrary in your code. Similar rules apply as to how the library is found, but you can specify a fully-qualified or relatively-qualified path to control the search.
In the case of Load-Time linking, MS recommends that your program's DLLs are stored in and loaded from the same directory where your application is loaded from. If this is at all workable, this is probably your best option.
If that doesn't work, there are several other options, outlined here. One is to leverage the search order by putting the DLL in either the working directory or the directory where the application was loaded from.
You can change the working directory of an application by:
Create a shortcut to your application.
Bring up the shortcut's properties
Edit the "Start in" property with the directory where the DLL is located.
When you launch your application using the shortcut, it will load the right DLL.
Other options for load-time linking include:
Adding a manifest to your application which specifies where your dependent assemblies are, or,
Setting the PATH.
You could use LoadLibrary, but you would need a way to guarantee the DLL's location. This Wikipedia article provides good example on how to use the DLL after it has been loaded.
You can add the directory where the dll is located to the PATH environment variable.
I have struggled with the same problem and also found a dead end with the suggested methods like LoadLibrary, SetDllDirectory, Qt's addLibraryPath and others. Regardless of what I tried, the problem still remained that the application checked the libraries (and didn't find them) before actually running the code, so any code solution was bound to fail.
I almost got desperate, but then discovered an extremely easy approach which might also be helpful in cases like yours: Use a batch file! (or a similar loader before the actual application)
A Windows batch file for such a purpose could look like this:
#echo off
PATH=%PATH%;<PATH_TO_YOUR_LIB>
<PATH_TO_YOUR_APP_EXE>
/edit: Just saw #SirDarius comment in Luchian's answer which describes that way, so just take my batch code bit as a reference and all credits go to him.
I have the same problem with one application I am working on.
I do not want to use runtime loading because there are tens of functions I would need to manually create function pointer for.
Mr Dibling's mention of manifest file opened a new door for me but I sadly found out that the oldest version of windows that supports the feature is Windows 7. It won't even work on Vista.
Long story short, a friend familiar with Windows Application development told me to look up Delay-Loaded DLL, which turns out to solve the problem perfectly with minimal effort. It delays the loading of DLL library to either the point you manually do, or the first time its function is called. So you just need to add your DLL path to the search path before that happens, where SetDllDirectory helps.
Here is the steps to make it work:
1) Specify the DLL to be delay-loaded to linker, either through your makefile, cmake or VS property page (Linker->Input of VS2015)
2) Call SetDllDirectory at the beginning of your program, before any call to the DLL is made.
Delay-loaded DLL is supported all the way back to VC6.
SetDllDirectory is supported after XP SP1.
Use Symbolic Links to the 3rd Party Executables
I found the approach advocated by Aaron Margosis useful. See:
Using NTFS Junctions to Fix Application Compatibility Issues on 64-bit Editions of Windows
Essentially, create symbolic links to each of the dependent 3rd Party executables. Place these symbolic link files in and amongst your own dependent executable files. Except for filename changes to the targets, the 'soft' symbolic links will resolve the load-time dependencies even as the target of the links are changed by future updates.
I am starting to learn DX11 and running into trouble with the effects framework. I know it was released as source and I have to build it, but the output from the build is not what I expected.
According to the research I've done on this question, the output from the build should be
D3DX11EffectsD.lib for debug
D3DX11Effects.lib for release
However, when I look into the 'Effects11\Debug' directory after building the project, I only see a file Effects11.lib (well, an Effects11 Object Library file which I assume is a .lib, I'm new to c++), and the exact same file in 'Effects11\Release'. Whats going on here? I've never used VS 2010 for c++ before now but I think I am building the solution correctly.
Is this a matter of renaming the files or have I done something wrong without realizing it? There really isn't much documentation on building and linking libraries in vs 2010 that I could find. Anybody have any suggestions?
Thanks
If you compiled exactly what you got off the web, then I think it would be just a naming convention problem.
You should try compiling it into your end application and see if it yells about debugging symbols missing.
You can also go into the build settings (it has been a while since I have used visual studio for anything other than C# so I don't know exactly where that menu option would be (I assume right clicking on the project should yield some useful results)...I generally do my C++ stuff on linux) and check to see what the built targets are for debug and release. If it turns out that the names are the same for both, but the build targets (i.e. the folder and a few other options, like including debugging symbols) are different then you should be good and it is just a naming problem.
Also, if the files are the exact same size it is likely that they are the same since the debug file should be at least a bit larger than the release one.
If it turns out that they are the same file, try re-downloading or re-extracting the source and just compiling the project again without any changes and see if that gets any results.
I am a Java programmer and have come across a very nasty situation. For POC purposes, I need to write down a small segment of my solution that will run as a standalone application doing something very specific.
I have 2 questions:-
I can write the code, but what I don't know is how do I create an installer and exe out of that C++ code.
Secondly, I need to parse a sqlite db file and show its data in the application. Is there a sqlite windows dll or some C++ library or something that I can use, instead of asking the user to install sqlite (or doing it myself through the installer)? So basically, I don't want an extra program to be pushed in, just a dll or some C++ library..
Please let me know if you have an answer to either or both the issues that I'm facing.
Thanks
Compiling your code will turn it in to an executable. For distribution, you'll want to build it in Release mode. I'm not sure what version of Visual Studio you are using, but you might have a "Setup and Deployment" Project type which will enable you to create an installer. Failing that, you may have to look at InstallShield or a tool like that to ensure that the installer has all necessary files (such as the runtime libraries).
SQLLite is called light for a reason! The source code for it can be incorporated directly in to your project and compiled alongside the rest of the files (see: http://www.sqlite.org/selfcontained.html ). This means no external libraries are necessary to link against, and no extra DLLs need to be redistributed alongside your executable.