Lets say my executable file, MyApp, links dynamically against an so file, boost-system.so. I compile and everything works fine for a while; my calls to boost-system funcs do what they are supposed to do.
After a while i realize i need to link to another so file, SomeAPI.so. However, SomeAPI.so have statically linked against boost-system.a, but an older, buggy version, which has functions with exactly the same names as "my" boost-system (ofcourse). Now, ALL calls (both from MyApp and SomeAPI) will go to the version SomeAPI statically linked (or my version of boost-system, depending on link order, both are bad for me).
I would like my calls from MyApp to go to my version of boost-system, and SomeAPI to use its statically linked boost-system.a functions. On Windows this is how it works.
I realize why this is not happening, but is there any way around this except renaming the namespaces in "my" boost-system to something local?
Use LD version scripts to hide function names that you don't want other .so files to see.
Related
Introduction
I am trying to use Toulbar2 as a C++ library in my CMake project, however I am having much trouble linking it to my main executable.
I found many similar questions on this topic, both here and on other similar website, but none of them helped me with my specific issue. I tried literally everything and I did not menage to make it work, I was hoping that some of you may help me with that.
I am running Ubuntu 18.04, CMake version 3.23 and in my project I am using the standard C++11. I am a proficient programmer, but I am just an beginner/intermediate user of both C++ and CMake.
What I've already tried to do
I cannot list all my attempts, so I will only mention those I think are my best ones, to give you an idea of what I may be doing wrong.
1) In my first attempt, I tried to use the same approach I used for any non-standard library I imported, i.e. using find_package() in CMakeLists.txt to then link the found LIBRARIES and include the found INCLUDE_DIRS. However, I soon realised that Toulbar2 provides neither a Find<package>.cmake or <name>Config.cmake file. So, this approach could not work.
2) My second attempt is the one that in my opinion brought me the closest to the solution I hoped for. You can easily compile Toulbar2 as a dynamic library using the command: cmake -DLIBTB2=ON .. in an hypothetical build directory you previously created. After compiling with make you have your .so file in build/lib/Linux. After installation, you can make CMake find this library by itself using the command find_library. So, my CMakeLists.txt ended up looking like this:
[...]
find_library(TB2_LIBRARIES tb2)
if(TB2_LIBRARIES)
set(all_depends ${all_depends} ${TB2_LIBRARIES})
else(TB2_LIBRARIES)
add_compile_definitions("-DNO_TB2")
message("Compiling without Toulbar2, if you want to use it, please install it first")
endif(TB2_LIBRARIES)
[...]
target_link_libraries(main ${all_depends})
[...]
This code works to some extent, meaning that CMake correctly finds the library and runs the linking command, however if I try to #include <toulbar2lib.hpp> the header is not found. So I figured out I should have told CMake where to find that header, so I ended up adding a
include_directories(/path/to/header/file's/directory)
However, I still have another problem. The header is found, but a lot of names used in the header are not found at compilation time. The reason is that in Toulbar2 some variables/types are defined conditionally by using preprocessing directives like #ifdef or #ifndef, and in turn the global variables used in these conditions are defined through CMake at compilation time. If you are interested in an example, I can mention the Cost type that is used in the mentioned header file. I see that there's a piece missing in the puzzle here, but I cannot figure out which one. Since I pre-compiled the library those definitions should exist when I include the header file, because I am correctly linking the correspondent library that contains those definitions.
3) My third attempt is less elegant than the the other two I mentioned, but I was desperately trying to find a solution. So, I copied the whole toulbar2 cloned folder inside my project and I tried to add it as a subdirectory, meaning that my main CMakeLists.txt contains the line:
add_subdirectory(toulbar2)
It provides a CMakeLists.txt too, there should be no problem in doing it. Then I include the src directory of toulbar2, that contains the header file I need, and I should be okay. Right? Wrong. I got the same problem that I had before with (2), i.e. some variables/types conditionally defined were not actually defined when I tried to compile my project, even though the subproject toulbar2 was correctly (no errors) compiled.
I just wanted to mention that any answer is welcome, however if you could help me figure out an elegant solution (see 1 or 2) for this problem it would be way better, as this code is intended to be published soon or later. Thank you in advance for your help.
Solution 2) looks fine. You just need to add the following compilation flags -DNDEBUG -DBOOST -DLONGDOUBLE_PROB -DLONGLONG_COST when compiling your project with toulbar2lib.hpp. See github/toulbar2 README.md how to compile without cmake for those flags (except WCSPFORMATONLY that should not by used in this context).
Suppose I have an application app.exe which does a load-time link to d.dll, which in turn does a load-time link to dangerous.dll. Suppose that dangerous.dll is considered to have various security vulnerabilities. Suppose also that none of the functionality in dangerous.dll is needed by app.exe, and app.exe does not make any calls to d.dll that require it to use any functions in dangerous.dll.
I want to distribute the app without dangerous.dll, and ideally with minimal or no modifications to app.exe or d.dll. If I just remove dangerous.dll, I get a loading error when starting app.exe because it loads d.ll and d.dll tries to load dangerous.dll, which fails.
Option 1
I could make a "no-op" version of dangerous.dll, that provided all the same export function signatures as dangerous.dll, but had no-op code for all the exported functions themselves. That might not be too bad. I wonder if there already exists a tool that can take a .dll file and output another .dll file that has the same export functions, but ones that do nothing? This has the disadvantage that my no-op version of dangerous.dll would still be visible to the user, and it would look like my application still has the vulnerabilities in dangerous.dll.
Option 2
I wonder if there is some way, without building d.dll from source, I could hack d.dll so that it does not attempt a load-time load of dangerous.dll. It would be okay if this caused any calls from d.dll to a dangerous.dll function to fail/crash, since as I say app.exe should not cause d.ll to use any of the functions in dangerous.dll.
If you have the source for d.dll, then just fix it to not load/use dangerous.dll.
If you don't have the source, just build a dangerous.dll with stub implementations for all exported functions.
Case closed. Move on and spend your time on more productive stuff.
I have this static(.a/.lib) library which I wanted to bind in my dynamic(.dll).
Lets say the static library is libColors.a and the dynamic is SWC.dll.
Now I already change the libColors.a project properties to make a static library--build it--and I haven't found any .dll created, only the .o and .a files which is expected. I switched to my SWC.dll project and change its properties to make a dynamic library and check the box to create also a .a file--link the libColors.a--build it--and the .dll file is created. (also note that I put __declspec(import/export) specification on its classes)
Now I want this SWC.dll to use in an executable file. However, when I include only the SWC.h file in the executable, it gets tons of undefined references. Seems there's a problem here since I didn't call any functions on SWC.h. So, I add on my .exe build options the SWC.a, now it builds with no error. But when I run it, it says I needed the Colors.dll? What do I miss?
You can use tools like objdump, CFF Explorer or Dependency Walker to view the import table of the PE-COFF executable in question.
The likely explanation is that the way your Colors project is compiled isn't really a static library. You can check this by inspecting both SWC.dll and your test executable's import table. Chances are that you'll find an import entry from Colors.dll in one of them and what function's it's trying to import.
Check the data and functions declared in Color.h and make sure they're not decorated with __declspec() anywhere. If the functions in Color.h preprocesses into something like __declspec(dllimport) in 'SWC' this can cause the problem you're seeing.
I have access to a large C++ project, full of files and with a very complicated makefile courtesy of automake & friends
Here is an idea of the directory structure.
otherproject/
folder1/
some_headers.h
some_files.cpp
...
folderN/
more_headers.h
more_files.cpp
build/
lots_of things here
objs/
lots_of_stuff.o
an_executable_I_dont_need.exe
my_stuff/
my_program.cpp
I want to use a class from the big project, declared in say, "some_header.h"
/* my_program.cpp */
#include "some_header.h"
int main()
{
ThatClass x;
x.frobnicate();
}
I managed to compile my file by painstakingly passing lots of "-I" options to gcc so that it could find all the header files
g++ my_program.cpp -c -o myprog.o -I../other/folder1 ... -I../other/folderN
When it comes to compiling I have to manually include all his ".o"s, which is probably overkill
g++ -o my_executable myprog.o ../other/build/objs/*.o
However, not only do I have to do things like manually removing his "main.o" from the list, but this isn't even enough since I forgot to also link against all the libraries that he happened to use.
otherproject/build/objs/StreamBuffer.h:50: undefined reference to `gzread'
At this point I am starting to feel I am probably doing something very wrong. How should I proceed? What is the usual and what is the best approach this kind of issue?
I need this to work on Linux in case something platform-specific needs to be done.
Generally the project's .o files should come grouped together into a library (on Linux, .a file if it's a static library, or .so if it's a dynamic library), and you link to the library using the -L option to specify the location and the -l option to specify the library name.
For example, if the library file is at /path/to/big_project/libbig_project.a, you would add the options -L /path/to/big_project -l big_project to your gcc command line.
If the project doesn't have a library file that you can link to (e.g. it's not a library but an executable program and you just want some of the code used by the executable program), you might want to try asking the project's author to create such a library file (if he/she is familiar with "automake and friends" it shouldn't be too much trouble for him), or try doing so yourself.
EDIT Another suggestion: you said the project comes with a makefile. Try makeing it with the makefile, and see what its compiler command line looks like. Does it have many includes and individual object files as well?
Treating an application which was not developed as a library as if it was a library isn't likely to work. As an offhand example, omitting the main might wind up cutting out initialization code that the class you want depends upon.
The responsible thing to do here is to read the code, understand it, and turn the functionality you want into a proper library. Build the "exe you don't need" with debug symbols and set breakpoints in the constructors and methods of the class. Step into them so you get a grasp on the functionality and what parts of the program are relevant and irrelevant to your needs.
Hopefully the code is under some kind of version control system that supports branching (such as Git). If not, make your own repository that does. Edit the files until you've organized them into a library and code that uses the library. Make sure it works properly within the context of the original program. Then turn around and use this library in your own program.
If you've done a good job, you might be able to convince the original authors to accept the separation back into their original codebase. If not, at least version control has your back so you can manage integration of future changes.
So, I have an interesting issue. I am working with a proprietary set of dlls that I ,obviously, don't have the source for. The goal is to write an intermediate dll that groups together a large series of funnction calls from the proprietary dlls. The problem I am having, when compiling with g++, is that I get errors for the original dlls along the lines of:
cannot export libname_NULL_THUNK_DATA. Symbol not found.
If I add a main and just compile to an executable everything works as expected. I'm using mingw for compilation. Thanks for any help.
In response to the first reply: Either I'm confused about what you're saying or I didn't word my question very well. I'm not explicitly trying to export anything from my wrapper I am just calling functions from their dlls. The problem is that I get errors that it can't export these specific symbols from the dll to my wrapper. The issue is that I'm not even entirely sure what these _NULL_THUNK_DATA symbols are for. I did a search and read somewhere that they shouldn't be exported because they're internal symbols that windows uses. I have tried using the --exclude-symbols directive to the linker but it didn't seem to do anything. I apologize if I'm completely misunderstanding what you're trying to say.
So, I think my issue was related to this. When just compiling a standard executable that uses a dll I was able to include the headers and directly call the functions for example:
#include :3rdparty.h
int main(){
dostuff(); // a function in the 3rdparty.dll
}
this would compile and run fine. I just needed to link the libraries in the g++ command.
When linking with the -shared flag I would get these errors (with main removed of course). I think it has something to do with the fact that by default g++ attempts to import all symbols from the dll. What I didn't understand is why this happens in the dll vs in an executable. I will try doing it using GetProcAddress(). Thank you!
it should be as easy as you think it should be.
eg:
your dll code needs:
void doStuff()
{
3rdparty.login();
3rdparty.dostuff();
3rdparty.logoff();
};
so far - so good, you've included the right headers .... (if you have them, if you don't then you need to import the library using LoadLibrary(), then create a function pointer to each exported dll entrypoint using GetProcAddress() and then call that function pointer)
You then link with the 3rd party lib and that's it. Occasionally you will have to wrap the definitions with 'extern "C"' in order to get the linkage name mangling correct.
As you say you're using g++, you can't be getting confused with __declspec(dllimport) which is a MS VC extension.
"Compiling" tells me that you're approaching this from the wrong end. Your DLL should not export its own wrapper functions, but directly refer to exports from other DLLs.
E.g. in a Windows Kernel32.DEF file, the following forward exists:
EXPORTS
...
HeapAlloc = NTDLL.RtlAllocHeap
There's no code for the HeapAlloc function.