I have a 3rd party static library L (pjsip.lib). This static library depends on many static variables and functions.
I created a DLL D using L.
I created an application A that uses both D and L.
Here I have a problem. It looks like there multiple instances of static functions and variables that L uses, and they have different values depending on whether it is accessed by directly A or through L. I am not able to build L as DLL. Is there any way that both A and D can use the same code segment?
I am using Visual C++ (2015).
Place the static lib in its own dll and export all the symbols you need (use a DEF) file it necessary. Now both your exe and other dll can link to it dynamically.
Related
My project builds through few static libraries which should linked to main dll library gain one single dll as a result.
Using __declspec(dllexport) attribute does not lead to appearance of specified functions of static libraries to dll, libraries not linked with dll at all.
Then I tried to build each library as shared for getting proper names of exported functions and created .def file based on them. Using .def file leaded to result.
Should __declspec(dllexport) and .def-file act equally in my case?
Is it possible to generate a .def file from sources? Since I have C++ code I'm not able to write .def file by myself because of mangling and presence classes in API, the approach described above with temporary generated dlls is inconsistent for production.
Update
I'd like to explain in detail about the structure of my project. The solution consists of a few projects (modules).
+
|
+-+ static_lib1
| +
| +--+ src
|
+-+ static_lib2
| +
| +--+ src
|
+-+ dynamic_lib (linked with static_lib1 and static_lib2)
+
+--+ src
Each sub-project weakly depends on others, let's assume they are not connected for clearness. Each module has own public interface. I want to have all modules as single dynamic library so my artifact is dynamic_lib.dll, but in fact static libraries are not linked with it.
Static libraries should not contain any __declspec or __attribute((dll...)) things. They are nothing more than multiple object files (usually *.obj or *.o), composed into one, single file.
All you need to do in order to use such library (either in .exe or .dll) is to include proper headers and link them - with Visual Studio it's pretty easy.
First of all, you need to know 1) where your static libraries are placed and 2) their exact names. Go to project properties and then General. Target name contains name for the output file, while Output directory indicates in which folder your .lib will be placed.
Note: This path may be different for every project! For multi-project solution, I always set this to a common path to avoid configuration problems.
Now, go to properties of project, that will consume this library (link with it). Go to Linker -> Input and then add name of your .lib to Additional dependencies (entries are separated with semicolon):
You need to add all libraries you want have linked in. Also, folder, in which these libraries are placed, must added to Linker -> General -> Additional library directories. If all .libs are placed in the same place - good, otherwise copy them into shared location or add multiple entries to Additional library directories list.
And the last thing - remember, that you also need to include headers with declarations of functions and objects, that you want to use. Basic thing, I know, but has to be mentioned.
UPDATE
unresolved external when trying to use dll library in an external prodjects
Your problem is not related to linking at all. The thing is, that you misunderstood what, linking a static library exactly does.
I am guessing, that functions reported as unresolved are not used by your DLL, right? But you expect them to be inside it, right?
When your DLL refers to an external content (like function or variable), it is resolved at linking time - together with all dependencies. But that's all. If your static library has a function named print_sample_string(), but your DLL does not use it, it won't be attached to DLL image. Think about this carefully - why should it be?
Even more - functions, that are not dllexported explicitly won't be visible anyway. Functions have by default external storage - so basically, they are private DLL's content.
So to answer your question directly - if you need to use functions/variables from static_lib1.lib, attach it to client application - just like you are attaching it now to dynamic_lib. There is no other way. (*)
(*) Truly speaking - there is. You can create intermediate function in DLL, that is exported and call desired function inside:
Somewhere in dynamic_lib:
DLL_EXP_IMP long CallFunctionFromA_Lib()
{
return some_function(); //this function is from static_lib1.lib
}
Somewhere in .exe:
long result = CallFunctionFromA_Lib(); //internally this will call function from static_lib1.lib
I can't imagine, however, why would you want to do this and not simply link A.lib and use it directly.
Here Raymond Chan explains this behavior and the best solution is really just to use def file. As for how to generate it automatically for static a library - this discussion looks like a good starting point.
I made static library A, the library is using another static library B.
And in order to use library A in code C, i added library A and header A but B.
Because i have known that static library is combined in compile time, so library B is already contained within A.
But When i compile code C, it seem to require that i add library B also showing undefined reference error.
And when i add library B also, the error disappear.
Why ?
You'll need to add both A and B.
When static lib A uses static lib B, B is absolutely NOT combined in A. That is impossible since when generating static libraries there is no link step.
This is the difference between shared and static libraries on Linux.
If your library is written well, you may not need to add B headers in code C, but when linking, (executable or shared library or even DLL in windows) you'll need all used static libraries.
I have some statically compiled libraries (.lib) that I use in my project, which is written in C++ and built on both Windows and Linux. At my project's entry-point to these libraries, I use just one or two functions from the 'main' library in the static library suite, really (but I'm sure that these functions call many others in the other libraries in the suite).
I would ideally like to instead have a suite of dynamically linked libraries (DLLs) that wraps around each of the libs in the static lib suite; I've read/heard that the way to do this on Windows (e.g., Visual Studio 2005/2008/2010) is to "create a wrapper DLL" with some exposed functions calling the underlying static library functions. I would very much appreciate if someone can give me some detailed step-by-step including possibly some snippets, of how to go about doing this in MS Visual Studio 2005/2008/2010. I am sure some of you may already be doing this on a day-to-day basis; your experience is very much appreciated.
Edit:
For the benefit of others like myself, I am posting the first 'useful' link I found:
http://tom-shelton.net/index.php/2008/12/11/creating-a-managed-wrapper-for-a-lib-file/
"Convert a library to another library type" seems easy, but it is not. There is no straight-forward step-by-step way to do this because C++ and DLLs do not play well together at all, and your code will need to be adapted to support a DLL interface.
A concise way to describe the problem is this:
A .lib's interface is C++
A .dll's interface is C
Thus, a DLL's interface simply doesn't support C++ and you need to be clever to make it work - this is why the ambiguous existing answers.
One standard way is via COM, which means building an entire COM wrapper for the library, complete with class factory, interfaces, objects, and using BSTR instead of std::string. I would guess is not practical.
Another solution is to create a C interface for your C++ library which is DLL-safe. That means basically creating a winapi-style interface, which again is probably not practical or defeats the purpose of using your library at all. This is what #David Heffernan suggests. But what he doesn't address is how you must change your code to be DLL-compatible.
An important but subtle problem is you cannot pass ANY templated C++ objects across DLL boundaries. This means passing std::string in or out of a DLL function is considered unsafe. Each binary gets its own copy of the std::string code, and there's no guarantee that they will happen to play nicely with each other. Each binary (potentially) also gets its own copy of the CRT and you will mess up internal state of one module by manipulating objects from another.
Edit: You can export C++ objects in MSVC using __declspec(dllexport) and importing them using __declspec(dllimport). But there are a lot of restrictions on this and subtleties that cause problems. Basically this is a shortcut for getting the compiler to create a cheap C-style interface for your exported class or function. The problem is it doesn't warn you about how much unsafe stuff is happening. To reiterate:
If there are ANY templated symbols crossing DLL bounds, it is not safe (std::* for example).
Any objects with CRT-managed state should not cross DLL bounds (FILE* for example).
If you do not care about interface adaptation at all, you can export symbols from a static .lib to a .dll fairly easily. The trick is, you do not use Visual Studio GUI or projects at all, but just the linker (link.exe).
With this method, C symbols will remain C symbols and C++ symbols will remain C++ symbols. If you need to change that, you need to write wrapper code (e.g. extern C interfaces). This method simply presents existing symbols from the .objs in the .lib as official exports from the DLL.
Assume we have a .lib compiled from a source TestLib.c
#include <stdio.h>
void print(char* str)
{
printf("%s\n", str);
}
int add(int a, int b)
{
return a + b;
}
We compiled this into a static lib TestLib.lib. Now we wish to convert TestLib.lib to TestLibDll.dll (the base name should not be the same or you will get issues with the link output since the linker also creates DLL link .lib). To do this, we use link.exe outside Visual Studio GUI. Launch the "x64 Native Tools Command Prompt for Visual Studio xx" to get a cmd with the toolchain in path. (If you need 32 bit version, use x86 Native Tools instead). Change to the folder with TestLib.lib (e.g x64\Release). Then run:
link /DLL /EXPORT:add /EXPORT:print /OUT:TestLibDll.dll TestLib.lib
This should produce TestLibDll.dll. (The linker may complain a bit about there being no .obj, but you can ignore this.) The exports are:
dumpbin /exports TestLibDll.dll
Microsoft (R) COFF/PE Dumper Version 14.29.30040.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file TestLibDll.dll
File Type: DLL
Section contains the following exports for TestLibDll.dll
00000000 characteristics
FFFFFFFF time date stamp
0.00 version
1 ordinal base
2 number of functions
2 number of names
ordinal hint RVA name
1 0 00001080 add
2 1 00001070 print
We have successfully exported the functions.
In the case there are many functions, using /EXPORT is tedious. Instead make a .def file. For our example, here is TestLibDll.def:
LIBRARY TestLibDll
EXPORTS
print #1
add #2
The linker is then run as
link /DLL /DEF:TestLibDll.def /OUT:TestLibDll.dll TestLib.lib
This example uses x64 C symbols, which makes it straightforward. If you have C++ symbols, you need to provide the mangled version of the symbol in the /EXPORT argument or in the def file. For more complex situations than a single static lib, you may need to provide more link libraries on the command line and/or /LIBPATH args to point to link library folders.
Again, this method is only for exporting symbols verbatim from a static library. I personally used it to create a DLL to be loaded in Python with ctypes for a closed source static library. The advantage is you don't need to write any wrapper code or create any additional VS projects at all.
Note: the accepted answer provides a good discussion of pitfalls regarding C++ DLL interface and why C wrappers are a good idea. I did not focus on that here, only on the mechanics of getting the symbols to be exported to the DLL. Using a C interface to DLL if possible remains good advice.
This was a little big to add as a comment to tenfour's response...
If you want to still maintain a C++ API when using the DLL wrapper, you can put C++ to C conversion functions in the header file. This ensures that only C compatible data types ever cross the DLL boundary.
As an example
//MyDLL.h
class MyDLL {
public:
...
int Add2ToValues(std::vector<int>& someValues) {
int* cValues = new int[someValues.size()];
memcpy(cValues, &someValues[0], someValues.size() * sizeof(int));
int retVal = Add2ToValues_Internal(cValues, someValues.size());
someValues.assign(std::begin(cValues), std::end(cValues));
delete [] cValues;
return retVal;
}
private:
int Add2ToValues_Internal(int* valuesOut, const int numValues);
};
//MyDLL.cpp
int MyDLL::Add2ToValues_Internal(int* values, const int numValues)
{
for(int i = 0; i < numValues; ++i) {
values[i] += 2;
}
return 0;
}
One catch I ran into when doing these wrappers is that you must allocate and deallocate any memory within the header file. Since the header file will be compiled by the application that is using your library, it will use the CRT for whatever compiler you are using to build your application. All interaction with the DLL uses C, so you won't run into any runtime mismatches and all memory is allocated and freed either entirely within the DLL or entirely within the application so you don't have any cross DLL memory management issues either. In the example, I both allocated and deallocated in the header. If you need to allocate data in the _Internal function, you'll need to also add a function that allows you to free that memory within the DLL. Once inside of the _Internal functions, you are free to use as much C++ as you want.
I'm having trouble wrapping my head around how to interface with C/C++ libraries, both static (.lib/.a) and dynamic (.dll/.so), in D. From what I understand, it's possible to tell the DMD compiler to link with .lib files, and that you can convert .dll files to .lib with the implib tool that Digital Mars provides. In addition, I've come across this page, which implies being able to call functions in .dlls by converting C header files to D interface files. Are both of these methods equivalent? Would these same methods work for Unix library files? Also, how would one reference functions, enums, etc from these libraries, and how would one tell their D compiler to link with these libs (I'm using VisualD, specifically)? If anyone could provide some examples of referencing .lib, .dll, .a, and .so files from D code, I'd be most grateful.
Note you are dealing with three phases for generating an executable. During compilation you are creating object files (.lib/.a are just archives of object files). Once these files are created you use a Linker to put all the pieces together. When dealing with dynamic libraries (.dll, .so) there is the extra step of loading the library when the program starts/during run-time.
During compilation the compiler only needs to be aware of what you are using, it doesn't care if it is implemented. This is where the D interface files come in and are kind of equivalent to Header Files in this respect. Enumerations are declared in the D interface file and must also be defined because they only exist at compile time. Functions and variables can just be declared with no body.
int myFunction(char* str);
The guide for converting a header file to D is in the page you referenced. These files can then be passed to the compiler or exist in the Include Path.
When the linker runs is when you'll need the .lib/.a file. These files can be passed to the compiler which will forward them to the Linker or you can use pragma(lib, "my.lib"); in your program. In both cases the linker must be able to finding at link time (compilation).
In Linux I don't believe there is a difference for linking dynamic and static. In Windows you don't even need the D interface file. Instead you must obtain the function through system calls. I'm really not that familiar with this area, but I suggest Loading Plugins (DLLs) on-the-fly
Update: I can't help much with VisualD, but there is D for .NET Programmers.
There samples in D distribution of how to do this.
You need to define thunk module like this:
module harmonia.native.win32;
version(build) { pragma(nolink); }
export int DialogBoxParamA(HINSTANCE hInstance, LPCSTR lpTemplateName,
HWND hWndParent, DLGPROC lpDialogFunc, LPARAM dwInitParam);
and include import libs of DLLs where functions like DialogBoxParamA are defined.
I have a static lib A, which also uses static libs B, C and D.
I then have applications X and Y which both use A, but not B, C or D.
Is there some way to make it so X and Y will automatically see that A used B, C and D and link them, so that I don't need to keep track for the entire dependency tree so I can explicitly pass every static lib (quite a lot with things like Windows, Boost, etc)?
Static libraries do not link with other static libraries. Only when building the executable (or shared library/DLL) is linkage performed, and the way to keep track of this is (of course) to use make.
I think, conceptually, you might be able to merge libs together to achieve what you want - they are after simply collections of symbols ready made for the linker. Having said that, I've never seen a tool to do it. The binary format of a lib is a compiler matter, so it would have to be a mingw or gcc specific tool.
In terms of knowing which version of Boost lib A uses, there isn't really much for it but to find the documentation for A.
The easy way is to always offer the particular libraries A, B, C, and D to be linked. For a true library, the linker only retrieves the modules needed.
The key problem with A having some implicit linkage to other libraries is that they aren't necessarily uniquely identifiable. For example, should it use version 2.0 or 3.1? The one in /usr/share/lib or the one in /usr/lib/X11/xdm/share/lib? Etc.