What is the correct way to convert files like d3d11.lib that are provided in the DirectX SDK to the *.a GCC library format? I've tried the common reimp method for converting *.lib files to *.a files, but it doesn't seem to work.
Step one involves creating a definitions file:
bin\reimp -d d3d11.lib
Let's say I want to use the D3D11CreateDevice function that should be provided in this library. If I open the created definitions file everything seems to be OK:
LIBRARY "d3d11.dll"
EXPORTS
(...)
D3D11CreateDevice
D3D11CreateDeviceAndSwapChain
(...)
Next I try to create the *.a file using the definitions file and the original lib file:
bin\dlltool -v -d d3d11.def -l libd3d11.a
This does in fact produce a valid library (and no error messages when dlltool is set to verbose), but if I try to use the function D3D11CreateDevice that should be implemented in it, I get an error:
undefined reference to `D3D11CreateDevice'
If I ask nm what symbol are present in the library (and filter using grep), I get this:
D:\Tools\LIB2A>bin\nm libd3d11.a | grep D3D11CreateDevice
File STDIN:
00000000 I __imp__D3D11CreateDeviceAndSwapChain
00000000 T _D3D11CreateDeviceAndSwapChain
00000000 I __imp__D3D11CreateDevice
00000000 T _D3D11CreateDevice
The imp function is the function that calls the actual implementation of D3D11CreateDevice inside the DLL. However, that actual implementation is now prefixed by an underscore.
Why is "_D3D11CreateDevice" defined while "D3D11CreateDevice" is not even though it is mentioned in the definitions file?
Just do:
copy d3d11.lib libd3d11.a
Alternatively you use X:\path\to\d3d11.lib on the GCC command line instead of -ld3d11. The GNU utilities on Windows use the same PECOFF archive format that Microsoft's tools use.
An outdated version of dlltool will prepend an underscore to every function when converting d3d11lib. Solved it by using a dlltool.exe from MinGW-w64 4.9.2. This dlltool produces a library with the correct function names.
When using the regular d3d11.lib provided by Microsoft in combination with headers provided by anyone, a SIGSEGV will occur when stepping into the library at runtime. This means that you do have to convert to the *.a format for some reason not investigated.
Related
I'm working with legacy C++ code compiled with g++. The files in question are compiled using a library. My goal is to determine every use of a function or macro from a particular library in each of these files. (In my case, OpenSSL is the library in question, and I'll reference it as such throughout the rest of the post. However, I think my question generically applies to any C library I'd compile against.)
I could conceive of this being easier if OpenSSL were a C++ library using a namespace - I could simply grep on the namespace to find the OpenSSL functions. Since, however, it is a C library, undecorated OpenSSL functions and macros are sprinkled across some the source files and I can't readily tell by scanning the source which functions are from OpenSSL and which are other local functions or functions from other libraries.
Looking through Stack Overflow, I see questions like this for the Windows environment, but I don't see any answers for a Linux environment. Broadening my search, I see references to nm and objdump, but if it's possible to get the details I'm looking for from these tools from an object file, I can't figure out the correct parameters to use.
Thanks in advance for your help!
I don't think there is a simple and quick solution for this, you will have to do some work for this.
There are three ways your software might link with openssl.
Static linking.
Dynamic link with the runtime linker
Manual linking with dlopen.
In all cases, the best solution would be to remove the header files and the openssl library from their location and recompile the code.
If you do not have access to the code you have to use nm or objdump to get the symbols from your executable and cross reference them with the ones in the openssl library. This will not work if you are using dlopen to link the library.
Another option would be to get the openssl library and recompile it with tracing enabled and execute your code with the new library.
The nm tool is used to list all the symbols in an object, regardless if it is a library or an executable. You can make a bash script that cross-references the output of calling nm on the openssl library and on your executable. The way to call this is nm objname. The third column is the one with the symbols.
objdump is a more precise tool that you can use to list all the symbols that are undefined in your. You can use it to list the header of your executable (objdump -h objname), this normally lists all the libraries your executable needs at runtime to run. If openssl is listed here then this means you are linking against it dynamically with the run time linker. You can use objdump -R with openssl to get the symbols in the openssl interface. You can cross-reference this with the symbols listed when calling objdump -r with your executable
A coworker of mine was able to get this information using nm. Here's the procedure we followed:
Get the List of Symbols
As suggested by riodoro1 above, the list of objects from the library used by your code can be obtained by linking without the library (without -lcrypto in my case, for instance). Alternatively, this can be obtained as described below using nm
Run nm on all relevant objects:
find . -name '*.o' -exec nm {} \; > nm.txt
Find undefined symbols referenced by objects and strip symbols:
grep '^ *U' nm.txt > nm2.txt
Remove C++ symbols (mangled names begin with _Z), uniquify those remaining:
grep -v ' _Z' nm2.txt | sort | uniq > nm3.txt
Manually edit nm3.txt, remove symbols not part of openssl, write to nm4.txt.
Use the Preprocessor to Expand Macros
Build the cc files normally, capture output to log file. Isolate the lines that show the commands that compiled lotus source files. Search and replace in the output to produce commands to invoke the preprocessor. Change:
-o .../file.o => -o .../file.i
' -c ' => ' -E '
Run the modified commands to produce preprocessor output.
The preprocessor output contains the full text from all included
header files, followed by the preprocessed C code. Headers are
long and uninteresting so strip them from the output. We'll get
just C code with expanded macros.
bash -c 'for f in `find . -name "*.i"`; do cat "${f}" | perl cat-preproc-without-headers.pl > "${f}"cc; done'
Here's the contents of cat-preproc-without-headers.pl:
#!/usr/bin/perl
# Write lines to stdout if cat != 0
$cat = 0;
while(<>) {
if(/^# [1-9]\d* .*\.cc/) {
$cat = 1;
} elsif(/^# [0-9]/) {
$cat = 0;
} elsif($cat) {
print;
}
}
Conclusion
With the list of symbols and the expanded macros, you now have all the symbols from the library and the places where they are used in the source code.
As per #firebrush suggestion I post my comment as an answer (maybe for posterity).
In order to see where the library functions are used You can remove the library from linking and see what .o files have missing references.
I am trying to build gnu iconv on OSX as a static library. This is not a problem, it builds fine with
./configure --enable-static
make clean && make
but when I run nm on libiconv.a, I get the following results
...
_libiconv
_libiconv_open_
_libiconv_close_
...
This is problematic, because I want to build libxml2 using this library, and it requires the following symbols
iconv
iconv_open
iconv_close
Looking through the header file, it seems like the difference between these two symbol names is whether LIBICONV_PLUG is defined. But when I run make as
make clean && make CPPFLAGS=-DLIBICONV_PLUG
I get errors because several things are not defined, such as ICONV_GET_DISCARD_ILSEQ and ICONV_SET_HOOKS. Looking through the header file again, these are only defined if LIBICONV_PLUG is not defined.
My question is, am I using LIBICONV_PLUG correctly? Is there some other way to get a static library with the symbols I need? Should I go through the undefined symbols and define them myself by hand?
There are three possible ways to build and use GNU libiconv.
In all three cases, the symbols that it defines for use by a C or C++ program (through the header file) are 'iconv_open', 'iconv', 'iconv_close'.
The normal one is that, at object file level, it defines symbols 'libiconv_open' etc., so as not to conflict with the operating system's standard libraries. The mapping between the symbol at C level and at object file level happens in <iconv.h>.
You will get a link error if you use the system's <iconv.h> with the GNU libiconv.{so,dylib,a} or vice versa. This is a feature, because the two have slightly different features (OS X libiconv.dylib supports an encoding named "UTF-8-MAC", whereas GNU libiconv has a number of improvements and fixes) and mismatch can lead to trouble.
If you are a system vendor, you can build GNU libiconv in such a way that it defines 'iconv_open' instead of 'libiconv_open', etc. This is achieved through a simple edit of iconv.h.
Or, specifically on OS X, you can pick the libiconv from https://opensource.apple.com/ and compile it for yourself. But be aware that this version is based on GNU libiconv 1.11, that is, it is quite old.
As explained in GNU libiconv's README, the LIBICONV_PLUG flag creates a library that will override the functionality in the system; this works only on GNU, Solaris, and OSF/1.
In your case, where you only want to have GNU libiconv be used by some free software package (such as libxml2), the simplest way is to take the first approach, and during compilation of that package, use -I and -L options to make it find the GNU libiconv header file and library.
In fact, libxml2 makes this easy: its configure script already has an option --with-iconv=prefix, through which you specify the directory hierarchy in which you have installed GNU libiconv (the header file in prefix/include/ and the object file in prefix/lib/); the configure script will then synthesize the appropriate -I and -L options.
I created a .A and a corresponding .dll file from mingw. This is to be run on windows. Now I am attempting to interface and use the function in the .A library from a different component and I am getting a linker error. I wanted to make sure if my functions were exported properly. I therefore followed this link and decided to do this
> lib.exe /list libSomeLibrary.a
as a result I get something like this
d001861.o
d001862.o
d001863.o
d001864.o
d001865.o
d001866.o
d001867.o
d001868.o
d001869.o
d001870.o
d001871.o
d001872.o
d001873.o
d001874.o
d001875.o
d001876.o
d001877.o
d001878.o
d001879.o
d001880.o
d001881.o
d001882.o
d001883.o
....
....
Is this correct. Is there a way for me to actually get the names of the functions in the lib file.
Since you are using MinGW, you may use the nm command to list the contents of any libfoo.a archive file; used thus:
nm -A libfoo.a | more
(or better still, if you have the less command, e.g. in MinGW's MSYS shell environment):
nm -A libfoo.a | less
it will list all symbols specified within the archive, grouped and qualified by the name of the embedded object file which provides each one.
A .a library file contains a number of individual object files combined together into a library archive. It is the individual .o object files which actually contain the symbols and code.
You should be able to use the dumpbin tool to examine a library on Windows systems. The /exports option should show exported symbols. I don't have any experience with this tool but the page above implies it should be able to operate directly on windows lib files.
You state you are using MingW which comes with GNU Binutils, including nm and objdump, both of which can be used to examine object files. To use these tools you might find it more convenient to extract the .o files from the .a file, which you can do using the ar command. If you don't do that then it will list all of the information in all of the object files within the archive.
Example (shown on Linux system but these tools should work the same in Windows if using MingW):
Extract corefile.o from libbfd.a
$ ar x /usr/lib64/libbfd.a corefile.o
Examine global (external) symbols in corefile.o
$ nm -g corefile.o
0000000000000000 T bfd_core_file_failing_command
0000000000000030 T bfd_core_file_failing_signal
0000000000000060 T bfd_core_file_pid
U bfd_set_error
0000000000000090 T core_file_matches_executable_p
U filename_cmp
00000000000000d0 T generic_core_file_matches_executable_p
U _GLOBAL_OFFSET_TABLE_
U strrchr
I need JPEG handling capabilities in my project so I decided to use jpeg-8d library, after downloading packages I've made usual ./configure; make and make install. make test returns no errors and no problems was reported during the whole process but when I opened example.c in Code Blocks 10.05 compilation failed. I've "googled" some solutions and added
#ifdef __cplusplus
extern "C" {
#endif
...
#ifdef __cplusplus
}
#endif
In jpeglib.h file, recompiled, then added these lines to linker section of code blocks:
/usr/local/lib/libjpeg.a
/usr/local/lib/libjpeg.so
also added according directories into directories section.
but compiler still fails to compile example.c, here are some of the errors:
> /home/raff/CodeBlocks/jpeg-8d/example.c|116|warning: incompatible implicit declaration of built-in function ‘exit’|
> example.c|| undefined reference to 'image_width'
> example.c|| undefined reference to 'image_height'
If anyone successfully installed and used jpeg library please help, I have no problem in switching IDE and/or linux distribution.
Your instalation of the library seems to be fine (otherwise it would complain about missing jpeglib.h during compilation)
The file example.c is supposed to be part of documentation, not a code you can compile and use. Note that there's no main(), for example. You should use it as a reference on how to code methods for compressing and decompressing jpeg files.
The variables image_width and image_height (and image_buffer) are declared as extern, so they are assumed to be defined in some other file. You can write your program defining those variables and filling up the buffer and only after that you can use the provided methods (write_JPEG_file and read_JPEG_file).
Are you linking with static or shared library?
If it's static, do you place it after the object files in linker's command line?
If it's shared, and you're using newer toolchain, this may also be the issue.
If you're linking correctly, what does readelf -s show on the shared library? What does nm show on static library?
I was so sure that example.c was in fact an example of using this library that I did not check for its content. Moreover there was some more problems.
After writing proper program I`ve encountered an error:
error while loading shared libraries libjpeg.so.8: cannot open shared object file: No such file or directory
But this was fixed by this pair of shell commands:
ranlib /usr/local/lib/libjpeg.a
ldconfig /usr/local/lib
Hope this helps.
I don't understand how GCC works under Linux. In a source file, when I do a:
#include <math.h>
Does the compiler extract the appropriate binary code and insert it into the compiled executable OR does the compiler insert a reference to an external binary file (a-la Windows DLL?)
I guess a generic version of this question is: Is there an equivalent concept to Windows DLLs under *nix?
Well. When you include math.h the compiler will read the file that contains declarations of the functions and macros that can be used. If you call a function declared in that file (header), then the compiler inserts a call instruction into that place in your object file that will be made from the file you compile (let's call it test.c and the object file created test.o). It also adds an entry into the relocation table of that object-file:
Relocation section '.rel.text' at offset 0x308 contains 1 entries:
Offset Info Type Sym.Value Sym. Name
0000001c 00000902 R_386_PC32 00000000 bar
This would be a relocation entry for a function bar. An entry in the symbol table will be made noting the function is yet undefined:
9: 00000000 0 NOTYPE GLOBAL DEFAULT UND bar
When you link the test.o object file into a program, you need to link against the math library called libm.so . The so extension is similar to the .dll extension for windows. It means it is a shared object file. The compiler, when linking, will fix-up all the places that appear in the relocation table of test.o, replacing its entries with the proper address of the bar function. Depending on whether you use the shared version of the library or the static one (it's called libm.a then), the compiler will do that fix-up after compiling, or later, at runtime when you actually start your program. When finished, it will inject an entry in the table of shared libraries needed for that program. (can be shown with readelf -d ./test):
Dynamic section at offset 0x498 contains 22 entries:
Tag Type Name/Value
0x00000001 (NEEDED) Shared library: [libm.so.6]
0x00000001 (NEEDED) Shared library: [libc.so.6]
... ... ...
Now, if you start your program, the dynamic linker will lookup that library, and will link that library to your executable image. In Linux, the program doing this is called ld.so. Static libraries don't have a place in the dynamic section, as they are just linked to the other object files and then they are forgotten about; they are part of the executable from then on.
In reality it is actually much more complex and i also don't understand this in detail. That's the rough plan, though.
There are several aspects involved here.
First, header files. The compiler simply includes the content of the file at the location where it was included, nothing more. As far as I know, GCC doesn't even treat standard header files differently (but I might be wrong there).
However, header files might actually not contain the implementation, only its declaration. If the implementation is located somewhere else, you've got to tell the compiler/linker that. By default, you do this by simply passing the appropriate library files to the compiler, or by passing a library name. For example, the following two are equivalent (provided that libcurl.a resides in a directory where it can be found by the linker):
gcc codefile.c -lcurl
gcc codefile.c /path/to/libcurl.a
This tells the link editor (“linker”) to link your code file against the implementation of the static library libcurl.a (the compiler gcc actually ignores these arguments because it doesn't know what to do with them, and simply passes them on to the linker). However, this is called static linking. There's also dynamic linking, which takes place at startup of your program, and which happens with .dlls under Windows (whereas static libraries correspond to .lib files on Windows). Dynamic library files under Linux usually have the file extension .so.
The best way to learn more about these files is to familiarize yourself with the GCC linker, ld, as well as the excellent toolset binutils, with which you can edit/view library files effortlessly (any binary code files, really).
Is there an equivalent concept to Windows DLLs under *nix?
Yes they are called "Shared Objects" or .so files. They are dynamically linked into your binary at runtime. In linux you can use the "ldd" command on your executable to see which shared objects your binary is linked to. You can use ListDLLs from sysinternals to accomplish the same thing in windows.
The compiler is allowed to do whatever it pleases, as long as, in effect, it acts as if you'd included the file. (All the compilers I know of, including GCC, simply include a file called math.h.)
And no, it doesn't usually contain the function definitions itself. That's libm.so, a "shared object", similar to windows .DLLs. It should be on every system, as it is a companion of libc.so, the C runtime.
Edit: And that's why you have to pass -lm to the linker if you use math functions - it instructs it to link against libm.so.
There is. The include does a textual include of the header file (which is standard C/C++ behavior). What you're looking for is the linker . The -l argument to gcc/g++ tells the linker what library(ies) to add in. For math (libm.so), you'd use -lm. The common pattern is:
source file: #include <foo.h>
gcc/g++ command line: -lfoo
shared library: libfoo.so
math.h is a slight variation on this theme.