I'm having trouble wrapping my head around how to interface with C/C++ libraries, both static (.lib/.a) and dynamic (.dll/.so), in D. From what I understand, it's possible to tell the DMD compiler to link with .lib files, and that you can convert .dll files to .lib with the implib tool that Digital Mars provides. In addition, I've come across this page, which implies being able to call functions in .dlls by converting C header files to D interface files. Are both of these methods equivalent? Would these same methods work for Unix library files? Also, how would one reference functions, enums, etc from these libraries, and how would one tell their D compiler to link with these libs (I'm using VisualD, specifically)? If anyone could provide some examples of referencing .lib, .dll, .a, and .so files from D code, I'd be most grateful.
Note you are dealing with three phases for generating an executable. During compilation you are creating object files (.lib/.a are just archives of object files). Once these files are created you use a Linker to put all the pieces together. When dealing with dynamic libraries (.dll, .so) there is the extra step of loading the library when the program starts/during run-time.
During compilation the compiler only needs to be aware of what you are using, it doesn't care if it is implemented. This is where the D interface files come in and are kind of equivalent to Header Files in this respect. Enumerations are declared in the D interface file and must also be defined because they only exist at compile time. Functions and variables can just be declared with no body.
int myFunction(char* str);
The guide for converting a header file to D is in the page you referenced. These files can then be passed to the compiler or exist in the Include Path.
When the linker runs is when you'll need the .lib/.a file. These files can be passed to the compiler which will forward them to the Linker or you can use pragma(lib, "my.lib"); in your program. In both cases the linker must be able to finding at link time (compilation).
In Linux I don't believe there is a difference for linking dynamic and static. In Windows you don't even need the D interface file. Instead you must obtain the function through system calls. I'm really not that familiar with this area, but I suggest Loading Plugins (DLLs) on-the-fly
Update: I can't help much with VisualD, but there is D for .NET Programmers.
There samples in D distribution of how to do this.
You need to define thunk module like this:
module harmonia.native.win32;
version(build) { pragma(nolink); }
export int DialogBoxParamA(HINSTANCE hInstance, LPCSTR lpTemplateName,
HWND hWndParent, DLGPROC lpDialogFunc, LPARAM dwInitParam);
and include import libs of DLLs where functions like DialogBoxParamA are defined.
Related
I just started a graphical C++ course and I have problem getting an overview how it is.
we got some starting code, two files; one of type "C++ Source" and another of "C/C++ Header".
its supposed to be a graphical program which fills the screen with color.
also, we are using some custom libraries such as SDL and GLM, in the same folder as those two files there is a folder named gml and loads of subfolders, which I wont get into.
I have downloaded mingw, cmake and Visual Studio 11 beta for c++.
I've tried making a normal Win32 program and also a forms-application for the graphical part, but its always something wrong when compiling.
My question: how are you supposed to handle C++ files? I just got used to java and there its so easy to just open the .java file and paste into your IDE, dealing with C++ makes me really confused.
Hmm... Where to begin...
Somethings that happen behind the scenes in other languages are much more visible in C++. The process of obtaining a binary (say, an executable) from C++ involves first compiling the source code (There are sub-steps of this but the compiler handles them) to obtain object files, then the object files are linked by the linker to generate a binary.
In theory, you could simply #include all the cpp files in a project, and compile them all together and "link" (although there's nothing to link) but that would take a very long time, and more importantly, in complex projects that could deplete the memory available to your compiler.
So, we split our projects into compilation units, and by convention a .cpp file represents a single compilation unit. A compilation unit is the part of your project that gets compiled to generate one object file. Even though compilation units are compiled separately, some code has to be common among them, so that the piece of code in each of them can use the functionalities implemented by the others. .h files conventionally serve this purpose. Things are basically declared (sort of announced) in them, so that each compilation unit knows what to expect when it's a part of a linking process to generate a binary.
There's also the issue with libraries. You can find mainly two kinds of things in libraries;
Already implemented functionality, shipped to you in the form of binary files including CPU instructions that can almost be run (but they've to be inserted in the right place). This form is accompanied by .h files to let your .cpp files know what to expect in the library.
The second type is functionality implemented directly in the .h
files. Yes, this is possible under special cases. There are cases,
where the implementation has to (a weak has to) accompany the
declaration (inlined functions, templated types etc.).
The first type comes in two flavors: A "static library" (.lib in windows, .a in linux), that enters your executable and becomes a part of it during linking, and a "dynamic library", that is exposed to your binary (so it knows about it) but that doesn't become a part of it. So, your executable will be looking for that dynamic library (.dll files in windows and .so files in linux f.x.) while it's run.
So, in order for your .cpp files to be able to receive services from libraries, they have to #include their .h files, to know about what there is in them. Later on, during linking, you have to show the linker where (what path in the file system) to find the binary components of those libraries. Finally, if the library is dynamic, the .dll's (or .so's etc.) must be accessible during run time (keep them in the same folder for instance).
While compiling your compilation units you have to tell the compiler where to find the .h files. Otherwise, all it will see will be #include <something.h> and it won't know where to find that file. with gcc, you tell the compiler with the -I option. Note that, you just tell the folder. Also of importance is that if the include directive looks like #include<somefolder/somefile.h> you shouldn't include somefolder in the path. So the invocation looks like:
g++ mycompilationunit.cpp -IPATH/TO/THE/INCLUDED/FILES -IPATH/TO/OTHER/INCLUDED/FILES -c
The -c option tells the compiler that it shouldn't attempt to make an executable just from this compilation unit, so it creates a .o file, to be linked with others later. Since we don't tell it the output file name, it spits out mycompilationunit.o.
Now we want to generate our binary (you probably want an executable, but you could also want to create a library of yours). So we have to tell the linker everything that goes into the binary. All the object files and all the static and dynamic libraries. So, we say: (Note g++ here also acts as the linker)
g++ objectfile1.o objectfile2.o objectfile3.o -LPATH/TO/LIBRARY/BINARIES -llibrary1 -llibrary2 -o myexecutable
Here, -L option is self explanatory in the example. -l option tells which binaries to look for. The linker will accept both static and dynamic libraries if it finds them on the path, and if it finds both, it'll choose one. Note that what goes after -l is not the full binary name. For instance in linux library names take the form liblibrary.so.0 but they're referred to as -llibrary in the linker command. finally -o tells the compiler what name to give to your executable. You need some other options to f.x. create a dynamic library, but you probably don't need to know about them now.
What is the difference between a .cpp file and a .h file?
Look at this answer. Also a quick google search explains a bit too.
Pretty much .h (header) files are declerations and .cpp (source) files are definitions. It is possible to combine both files into one .cpp file but as projects get bigger and bigger its becomes annoying and almost unreasonable.
Hope that helps.
In C++ there is a notion of a function declaration (the function signature) and a function definition (the actual code).
A header file (*.h) contains the declarations of functions and classes. A source file (*.cpp, *.c++, *.C) contains the definitions.
A header file can be included in a source file using #include directive.
When you define a class in C++, you typically only include the declarations of the member functions (methods in Java lingo), and you put the class definition into a header file. The member function definitions containing the body of each function are typically put outside the class definition and into the source file.
Generally the best thing to do here is to get a book on C++ or C, and to look at some sample code.
Header files (.h) are supposed to contain definitions of classes, methods, and variables. Source file (.cpp) will contain the code. So in your .cpp file you need to include the header file as #include "header-file-name.h".
Then use g++ to compile the .cpp file. Make sure that the path to .h file is correct.
If you are using CodeBlocks or Visual Studio, then just compiling the project and running will do everything for you. You can also add .h or .cpp file from there. You need not worry about anything.
Hope this helps.
I'm going to ask how it's done in c++, but this idea can apply to multiple languages. If you know how to do it in objective-c as well, please provide any similarities between the two
Lets say I want to create an instance of an ofstream like
ofstream myfile;
I'm assuming all I have on my computer is the *.o file (in a library archive) and the *.h file for iostream class. If this part isn't true let me know. I am assuming this when all I have installed is the runtime and the devel packages, not the source files.
How does it connect the header file to the object file, is there a naming scheme. And where does it look and in what order.?
Why this is confusing me is normally when I want to create a class I link my implementation of the class with the program, so where does it now and how does it now to link the files?
One more, does it matter if it loaded statically or dynamically?
Thank you in advance, and sry if this is a silly question.
Computer Science 101:
Broadly speaking (VERY broadly!), there are two kinds of "programs":
a) Interpreted: you read the program source line-by-line every time you execute it
<= *nix shell scripts and DOS .bat files are "interpeted"
b) Compiled: you read the source once (to convert it into a "binary machine code"). You link the machine code "object files" to build an "executable program".
You're talking about "compiled programs"
The "ofstream" part is irrelevant once the program is "compiled"
The binary implementation for "ofstream" can be compiled directly into the executable, or it can be dynamically loaded from a shared library (.dll) at runtime.
A "compiler" users ".h" headers to process the source file.
A "linker" uses ".lib" libraries to match symbols and link static code at link type.
The "Operating System" recognizes dynamic links and loads the needed shared libraries (.dll's) at runtime.
Three different things, all independent of each other: Compiler/source code, Linker/machine object code, OS/executable programs
'Hope that helps .. a bit...
This is not standardized and it's up to the implementation. I don't know about *unix, but I assume it's fairly similar to Windows.
You can assume that .o files are similar to library files .lib.
The header does define the class definition, so that the linker knows what to look for in the library.
Say you have a header:
class A
{
public:
A();
void foo();
};
and a lib file A.lib.
You include that header and call:
A a;
a.foo();
The compiler finds the declarations for bot A() and A::foo(). Now it knows it has to search the library for these functions. Names in the library are decorated, and contain modifiers, but its specific to the compiler so the linker finds the functions if they are exported in the library. It then binds the functions to the specific entry point from the dll.
If by dynamic loading you mean using LoadModule() and GetProcAddress() instead of linking, than the concept is pretty similar.
If you do static linking all symbols with linkage are available in the .obj file. The linker binds the calls of the functions to the entry points of the functions. There is a name mangeling involved in this process so that the symbols can be resolved correctly.
Dynamic linking is a platform dependent issue and not part of the C or C++ standard as far as I know.
I am creating a project that uses a DLL. To build my project, I need to include a header file, and a lib file. Why do I need to include the respective lib file? shouldn't the header file declare all the needed information and then at runtime load any needed library/dll?
Thanks
In many other languages, the equivalent of the header file is all you need. But the common C linkers on Windows have always used import libraries, C++ linkers followed suit, and it's probably too late to change.
As a thought experiment, one could imagine syntax like this:
__declspec(dllimport, "kernel32") void __stdcall Sleep(DWORD dwMilliseconds);
Armed with that information the compiler/linker tool chain could do the rest.
As a further example, in Delphi one would import this function, using implicit linking, like so:
procedure Sleep(dwMilliseconds: DWORD); stdcall; external 'kernel32';
which just goes to show that import libraries are not, a priori, essential for linking to DLLs.
That is a so-called "import library" that contains minimal wiring that will later (at load time) ask the operating system to load the DLL.
DLLs are a Windows (MS/Intel) thing. The (generated) lib contains the code needed to call into the DLL and it exposes 'normal' functions to the rest of your App.
No, the header file isn't necassarily enough. The header file can contain just the declarations of the functions and classes and other things you need, not their implementations.
There is a world of difference between this code:
void Multiply(int x, int y);
and this code:
void Multiply(int x, int y)
{
return x * y;
}
The first is a declaration, and the second is a definition or implementation. Usually the first example is put in header files, and the second one is put in .CPP files (If you are creating libraries). If you included a header with the first and didn't link in anything, how is your application supposed to know how to implement Multiply?
Now if you are using header files that contain code that is ALL inlined, then you do not need to link anything. But if even one method is NOT inlined, but has its implementation in a .CPP file that is compiled to a .lib file, than you need to link in the .lib file.
[EDIT]
With your use of Import Libraries, you are telling the linker to NOT include the implementation details of the imported code into your binary. Instead the OS will then load the import DLL at run-time into your process. This will make your application smaller, but you have to ship another DLL with it. If the implementation of the library changes, you can just reship another DLL to your customers, and not have to reship the entire application.
There is another option where you can just link in a library and you don't need to ship another DLL. That option is where the Linker will include the implementation into your application, making it bigger in size. If you have to change the implementation details in the imported library, then you have to recompile and relink your entire application, and reship the entire thing to your customers.
There are two relevant phases in the building process here:
compilation: from the source code to an object file. During the compilation, the compiler needs to know what external things are available, for that one needs a declaration. Declarations designed to be used in several compilation units are grouped in header. So you need the headers for the library.
linking: For static libraries, you need the compiled version of the library. For dynamic libraries, in Unix you need the library, in windows, you need the "import library".
You could think that a library could also embed the declarations or the header could include the library which needs to be linked. The first is often done in other languages. The second is sometimes available through pragmas in C and C++, but there is no standard way to do this and would be in conflict with common usage (such as choosing a library among several which provide code variant for the same declarations, for instance debug/release single thread/multithreads). And neither choice correspond well with the simple compilation model of C and C++ which has its roots in the 60's.
The header file is consumed by the compiler. It contains all the forward declarations of functions, classes and global variables that will be used. It may also contain some inline function definitions as well.
These are used by the compiler to give it the bare minimum information that it needs to compile your code. It will not contain the implementation details.
However you still need to link in all the function, and variable definitions that you have told the compiler about. Failure to do so will result in a linker error. Often this is contains in other object files which may be joined into a single static library.
In the case of DLLs (or .so files), we still need to tell the linker where in the DLL or shared object the missing symbols are. On windows, this information is contained in a .lib file. This will generate the code to load and link the code at runtime.
On unix the the dll and lib files are combined into a single .so file which you must link against to about linker errors.
You can still use a dll without a .lib file but you will then have to load and link in all the symbols manually using operating system APIs.
from 1000 ft, the lib contains the list of the functions that dll exports and addresses that are needed for the call.
I have some statically compiled libraries (.lib) that I use in my project, which is written in C++ and built on both Windows and Linux. At my project's entry-point to these libraries, I use just one or two functions from the 'main' library in the static library suite, really (but I'm sure that these functions call many others in the other libraries in the suite).
I would ideally like to instead have a suite of dynamically linked libraries (DLLs) that wraps around each of the libs in the static lib suite; I've read/heard that the way to do this on Windows (e.g., Visual Studio 2005/2008/2010) is to "create a wrapper DLL" with some exposed functions calling the underlying static library functions. I would very much appreciate if someone can give me some detailed step-by-step including possibly some snippets, of how to go about doing this in MS Visual Studio 2005/2008/2010. I am sure some of you may already be doing this on a day-to-day basis; your experience is very much appreciated.
Edit:
For the benefit of others like myself, I am posting the first 'useful' link I found:
http://tom-shelton.net/index.php/2008/12/11/creating-a-managed-wrapper-for-a-lib-file/
"Convert a library to another library type" seems easy, but it is not. There is no straight-forward step-by-step way to do this because C++ and DLLs do not play well together at all, and your code will need to be adapted to support a DLL interface.
A concise way to describe the problem is this:
A .lib's interface is C++
A .dll's interface is C
Thus, a DLL's interface simply doesn't support C++ and you need to be clever to make it work - this is why the ambiguous existing answers.
One standard way is via COM, which means building an entire COM wrapper for the library, complete with class factory, interfaces, objects, and using BSTR instead of std::string. I would guess is not practical.
Another solution is to create a C interface for your C++ library which is DLL-safe. That means basically creating a winapi-style interface, which again is probably not practical or defeats the purpose of using your library at all. This is what #David Heffernan suggests. But what he doesn't address is how you must change your code to be DLL-compatible.
An important but subtle problem is you cannot pass ANY templated C++ objects across DLL boundaries. This means passing std::string in or out of a DLL function is considered unsafe. Each binary gets its own copy of the std::string code, and there's no guarantee that they will happen to play nicely with each other. Each binary (potentially) also gets its own copy of the CRT and you will mess up internal state of one module by manipulating objects from another.
Edit: You can export C++ objects in MSVC using __declspec(dllexport) and importing them using __declspec(dllimport). But there are a lot of restrictions on this and subtleties that cause problems. Basically this is a shortcut for getting the compiler to create a cheap C-style interface for your exported class or function. The problem is it doesn't warn you about how much unsafe stuff is happening. To reiterate:
If there are ANY templated symbols crossing DLL bounds, it is not safe (std::* for example).
Any objects with CRT-managed state should not cross DLL bounds (FILE* for example).
If you do not care about interface adaptation at all, you can export symbols from a static .lib to a .dll fairly easily. The trick is, you do not use Visual Studio GUI or projects at all, but just the linker (link.exe).
With this method, C symbols will remain C symbols and C++ symbols will remain C++ symbols. If you need to change that, you need to write wrapper code (e.g. extern C interfaces). This method simply presents existing symbols from the .objs in the .lib as official exports from the DLL.
Assume we have a .lib compiled from a source TestLib.c
#include <stdio.h>
void print(char* str)
{
printf("%s\n", str);
}
int add(int a, int b)
{
return a + b;
}
We compiled this into a static lib TestLib.lib. Now we wish to convert TestLib.lib to TestLibDll.dll (the base name should not be the same or you will get issues with the link output since the linker also creates DLL link .lib). To do this, we use link.exe outside Visual Studio GUI. Launch the "x64 Native Tools Command Prompt for Visual Studio xx" to get a cmd with the toolchain in path. (If you need 32 bit version, use x86 Native Tools instead). Change to the folder with TestLib.lib (e.g x64\Release). Then run:
link /DLL /EXPORT:add /EXPORT:print /OUT:TestLibDll.dll TestLib.lib
This should produce TestLibDll.dll. (The linker may complain a bit about there being no .obj, but you can ignore this.) The exports are:
dumpbin /exports TestLibDll.dll
Microsoft (R) COFF/PE Dumper Version 14.29.30040.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file TestLibDll.dll
File Type: DLL
Section contains the following exports for TestLibDll.dll
00000000 characteristics
FFFFFFFF time date stamp
0.00 version
1 ordinal base
2 number of functions
2 number of names
ordinal hint RVA name
1 0 00001080 add
2 1 00001070 print
We have successfully exported the functions.
In the case there are many functions, using /EXPORT is tedious. Instead make a .def file. For our example, here is TestLibDll.def:
LIBRARY TestLibDll
EXPORTS
print #1
add #2
The linker is then run as
link /DLL /DEF:TestLibDll.def /OUT:TestLibDll.dll TestLib.lib
This example uses x64 C symbols, which makes it straightforward. If you have C++ symbols, you need to provide the mangled version of the symbol in the /EXPORT argument or in the def file. For more complex situations than a single static lib, you may need to provide more link libraries on the command line and/or /LIBPATH args to point to link library folders.
Again, this method is only for exporting symbols verbatim from a static library. I personally used it to create a DLL to be loaded in Python with ctypes for a closed source static library. The advantage is you don't need to write any wrapper code or create any additional VS projects at all.
Note: the accepted answer provides a good discussion of pitfalls regarding C++ DLL interface and why C wrappers are a good idea. I did not focus on that here, only on the mechanics of getting the symbols to be exported to the DLL. Using a C interface to DLL if possible remains good advice.
This was a little big to add as a comment to tenfour's response...
If you want to still maintain a C++ API when using the DLL wrapper, you can put C++ to C conversion functions in the header file. This ensures that only C compatible data types ever cross the DLL boundary.
As an example
//MyDLL.h
class MyDLL {
public:
...
int Add2ToValues(std::vector<int>& someValues) {
int* cValues = new int[someValues.size()];
memcpy(cValues, &someValues[0], someValues.size() * sizeof(int));
int retVal = Add2ToValues_Internal(cValues, someValues.size());
someValues.assign(std::begin(cValues), std::end(cValues));
delete [] cValues;
return retVal;
}
private:
int Add2ToValues_Internal(int* valuesOut, const int numValues);
};
//MyDLL.cpp
int MyDLL::Add2ToValues_Internal(int* values, const int numValues)
{
for(int i = 0; i < numValues; ++i) {
values[i] += 2;
}
return 0;
}
One catch I ran into when doing these wrappers is that you must allocate and deallocate any memory within the header file. Since the header file will be compiled by the application that is using your library, it will use the CRT for whatever compiler you are using to build your application. All interaction with the DLL uses C, so you won't run into any runtime mismatches and all memory is allocated and freed either entirely within the DLL or entirely within the application so you don't have any cross DLL memory management issues either. In the example, I both allocated and deallocated in the header. If you need to allocate data in the _Internal function, you'll need to also add a function that allows you to free that memory within the DLL. Once inside of the _Internal functions, you are free to use as much C++ as you want.
I am making a static library of my own. I have taken my code which works and now put it into a static library for another program to use. In my library I am using another static library which I don't want the people who will be using my API to know. Since, I want to hide that information from them I can't tell them to install the other static library.
Anyway, I used the command line Lib.exe to extract and create a smaller lib file of just the obj's I used. However, I get a bunch of LNK4006 :second definition ignored linker warnings for each obj I use followed by LNK4221 no public symbols found;archive member will be inaccessible.
I am doing this work in vs2008 and I am not sure what I am doing wrong.
I am using the #pragma comment line in my .cpp file
I have also modified the librarian to add my smaller .lib along with its location.
my code simply makes calls to a couple functions which it should be able to get from those Obj file in the smaller lib.
All my functions are implemented in .cpp file and my header just have the includes of the third party header files and come standard c++ header files. nothing fancy. I have actually no function definitions in there atm. I was going to put the API definition in there and implement that in the .cpp for this static lib that i was going to make. However, I just wanted to build my code before I added more to it.
I did read http://support.microsoft.com/default.aspx?scid=kb;EN-US;815773 but it did not provide a solution.
Even if you extract all objects from the other library and put them in your own library, your users will still be able to see what's in your library and thus see all the object names. In many cases the names of the objects will reveal what's actually the other library you are using.
Instead of distributing your library as a static library, consider distributing it as a DLL. In the DLL you can easily hide all the underlying things and only make public what you want to make public.