I'm currently switching to CMake for my build system (on Windows). I want to build both a shared and a static version of my library.
To include 3rd-party libraries (mostly from github), I run their CMake and install them. I then get a folder /lib/$LibName/cmake, which contains $LibnameConfig.cmake. This can be loaded into my own project using find_package, and is overall quite nice.
Now for the problem: Every last library I'm using overwrites these Config.cmake-files when building shared/static. From what I found online, I need to link static with static and shared with shared; but that is not possible as the Configs are overwriting each other.
Installing in different folders and then using find_package on both also doesn't work, as the package-name is the same for both shared and static (the .lib has a different name, but not the package).
As all the libraries I'm using have this overwrite, I thought it might be that my uneducated approach to this problem is flawed.
So, what is the standard way to use both the shared and static version of a library in a CMake-project; possibly switching between both with BUILD_SHARED_LIBS or a similar flag?
PS: Another thing I tried is pointing find_package to a different folder depending on BUILD_SHARED_LIBS. However, once the package is found, it is not re-searched, so I always and up with the folder associated with the default setting of this switch.
Related
I have an issue where I'm developing on one system where Boost is installed on:
/usr/include
/usr/lib
On a system I will deploy this on, the libboost libraries are at:
/nfs/mount/boost
/nfs/mount/lib
And I can't go about changing every system I deploy on to install libboost in the same place.
Is there a way to either:
include libboost as part of the binary executable such that loading from the system lib paths is not needed.
make the executable search for different directories when trying to load to libboost?
I'm using g++ 8
Sounds like you need a more sophisticated build environment.
I'm not sure what you mean here:
include libboost as part of the binary executable such that linking is not needed
Linking is not something you can skip. If you are trying to avoid distributing .dll/.so files with your executable, you need to avoid using the portions of the boost library that require compilation of the boost binaries (i.e. those not listed here https://www.boost.org/doc/libs/1_62_0/more/getting_started/windows.html#header-only-libraries).
Of course, that is not often the case. So...
make the executable search for different directories when trying to link to libboost?
This will never work reliably as you scale and is a nightmare in the CI world. This is where package managers such as conan (https://conan.io/) come to save the day. Delegating the package management to a third-party is the most reliable way of getting your code to build across multiple environments/platforms.
Also, building your executable and distributing it are separate operations. Any dynamically linked libraries will need to be discoverable on the system path at runtime.
Context: I have built a c++ executable using Qt 5.13.1 on an OpenSUSE platform (some Linux distribution).
I have to install it on a platform that already have an old Qt version installed and I can't get rid of it.
So I have installed the proper version of the needed Qt shared libraries (plus the platforms plugins) at a custom location and set up a qt.conf file that will specify the library path to use (same thing for the plugin path).
My own Qt installation folder contains:
A lib/ directory which contains the libraries to use.
A plugins/ directory which contains the platforms/ plugin folder.
The qt.conf file is installed alongside my executable and is filled as follows:
[Paths]
Prefix = relative_path/from/executable_location/to/install_dir/
I don't need to specify the lib/ and plugins/ directories in the entries Libraries and Plugins because they are defaulted at these values.
But even if I set them explicitly, it changes nothing.
I also tried with absolute paths but it changes nothing too.
Issue: The issue I encounter is that my executable does not load the libraries, neither find the platforms plugins, despite the qt.conf file is successfully loaded.
I have displayed the output of the following instructions:
QLibraryInfo::location(QLibraryInfo::PrefixPath);
QLibraryInfo::location(QLibraryInfo::LibrariesPath);
QLibraryInfo::location(QLibraryInfo::PluginsPath);
And they contains what's I specified in my qt.conf file. So the issue does not come from here.
Question: I have checked that QLibraryInfo field members had been properly initialized but it seems that my QApplication instance just ignore it.
How to make the QLibraryInfo configuration to be applied by the QApplication instance ?
I know that there exist a QCoreApplication::addLibraryPath() member than could be used to load libraries but I can't find anything alike for the Qt plugins as well.
But I'm pretty sure that we don't have to use this function since the information are already loaded by QLibraryInfo.
EDIT:
After some more investigations, I've found that it works fine for loading the plugins.
The problem remains the same for loading the libraries. It seems that I missed something but I cannot figure it out.
For now, I use the LD_PRELOAD environment variable in order to make it work (which is quite ugly).
EDIT 2:
I found two threads on internet about guys having the same issue:
https://forum.qt.io/topic/58499/solved-problems-with-setting-paths-to-libs-and-plugins-in-qt-app-s-executable-file
https://www.qtcentre.org/threads/32236-qt-conf-(again)-on-windows
The second one is explaining why the library load does not work.
Actually, in order to be able to parse the qt.conf file, the application needs to have the Core module loaded (libQt5Core.so.5 in my case) which makes sense (I should have thought of it).
The issue here is that libQt5Core.so.5 is one of the libraries to load (it's a vicious circle), so the executable cannot run at all.
Solution (still not load any library): It seems that I can't avoid to either load libQt5Core.so.5 with LD_PRELOAD or add it alongside the executable location.
Note: We can use LD_LIBRARY_PATH instead of LD_PRELOAD if there is no already existing Qt installation in the system "lib" directories.
I use LD_PRELOAD here because I want my Qt installation to take precedence over the already existing one (and be loaded first/instead of the system installed ones).
The problem with your libraries is that the Linux dynamic linker only searches a few standard locations, and other paths configured in /etc/ld.so.conf. One workaround is the environment variable LD_LIBRARY_PATH with the paths of your libraries. Another is the RPATH/RUNPATH values embedded in your binaries. CMake uses RPATH by default, and you can use QMAKE_RPATH in Qmake projects too. You can assemble by hand a directory with programs and libraries related using RPATH, but my advice is to create AppDir / AppImage packages using linuxdeployqt which is very similar to using the official Qt tools windeployqt and macdeployqt.
I have finally found out what was wrong.
qt.conf is parsed by Qt Core at runtime.
But the executable dependencies are loaded before the QApplication even exists. If they are not found, the program will just crash right at the beginning (dependencies loading step). And so, qt.conf will never be parsed.
In my case, I have an already existing installation of Qt in the system libraries folder. These are the ones that are found by the loader.
In this case, the dependencies are found and the programs runs (assuming there's no incompatibility between the two versions).
But even if the qt.conf is now parsed, the good version of the library will still not be loaded because the loader already has loaded the dependencies from the wrong Qt install (found at loading step).
Therefore the morality is: We cannot load executable dependencies by using qt.conf because they must be known before execution time (when the loader is looking for them).
Possible solutions:
Install the libraries in one of the folder searched by the loader (/usr/lib/ for example). If there is an already installed version of the libraries, remove it if possible, or if not, do ensure/check that you placed the proper version in a folder that will be searched first by the loader.
Use LD_LIBRARY_PATH to tell the loader where to search for libraries (if a wrong version is already installed, use LD_PRELOAD instead because LD_LIBRARY_PATH will not take precedence over what's inside /etc/ld.so.conf).
Set a rpath (which locates the proper installation directory of the libraries) when building the executable.
Add the required shared libraries in the executable directory.
(Not tested) Explicitly call the loader at application startup with the --library-path option (for example: "$ /lib/ld-linux.so.2 --library-path my_lib_path executable").
I may have missed other workarounds, this list is not guaranteed to be exhaustive.
I'm currently trying to build the Apache qpid messaging broker on Windows using CMake and the Boost (C++). I went through the initial steps of setting up Qpid on Windows as specified in this file. I set the variable BOOST_ROOT to "C:/< Home >/Downloads/boost_1_61_0" initially so that CMake could find the Boost Libraries (e.g. random, math, chrono, etc.) but I got this log saying that it could only find some of the libraries, but not others. I thought that maybe the filepath needed to include the directory where the Boost libraries were actually located (i.e. C:/< Home >/Downloads/boost_1_61_0/boost) but I was unable to reset the BOOST_ROOT environment variable despite deleting it and creating another one.
I was advised on this forum to delete the cache file for CMake using the GUI so I could reset the variable, but that didn't seem to work either.
Am I using the correct file path for BOOST_ROOT? If not, how should I go about changing it?
Some of the Boost libraries (e.g. chrono) need to be compiled first. If you have only downloaded Boost library from the website, you probably have not built these libraries. Check Boost website for instructions how to build on Windows. The site also lists which libraries need to be built. Alternatively, you can search for sites where pre-built Boost libraries are hosted. You can download and install from there.
Before, I preferred to use static library when deploying my application. But later I find it more convenient to use shared library since sometimes I can just replace the library instead of rebuilding the whole application.
One problem is that how to set the shared library path in the deployed environment. I know one way is to set $LD_LIBRARY_PATH, but for some reason I am not allowed to do it. Is there any way to do it within cmake? For example, in the deployed environment I put all shared libraries in folder /app/lib. Suppose that I put my executable file under the folder /app/bin. How should I set RPATH in cmake?
I tried
set(CMAKE_INSTALL_RPATH "$ORIGIN/../lib")
But it still cannot find the shared library. Thanks for any help.
In the project I am working on, we provide the possibility to dynamically load additional features. For that we use dlopen.
To find this libraries we have something we call module path. There we have a default path, where the shared libraries are (a lot of them are shipped).
At the moment we have two default paths: we first look in the build directory for the shared library and afterwards in the install directory. This is because it should also be possible to run the application without installing it (so in that case it needs to look first in the build directory).
Now the problem ist, if a user builds the application from source and installs it with make install, the libraries in her build directory are loaded by default. This will result in a crash. So it only works if the user afterwards removes or renames the build directory.
No the question: is there a trick (either by C++ or by the build system) to know whether the application is installed or not. The problem is, that the functionality is implemented in a shared library and the implemented way to search for modules has to work also for other applications that link against our library (so we can not rely on the path of the executable). We use CMake as a build system.
To make the situation even harder, the solution has to work on Windows, Linux and Mac OS X.
EDIT:
I further investigated and the problem is more complicated. This is the situation:
There is a small executable a
Furthermore there is a "main" library main.so
then there is a dynamically loaded library lib.so
lib.so links against main.so
The problem is, that lib.so has the absolute path to main.so in the build directory in its rpath. Thanks to the tip of #MSalters I now was able to make a hack to make sure to load the correct version of lib.so (the one in the install directory) but since it has the build path in the rpath it loads the wrong main.so (so in fact there are two copies of main.so in the memory - this messes things up).
Is there a way to remove this reference to the build path from the library? I tried all options of cmake related to rpath without success
Can't you check where the executable itself is? If it's in the build directories, use build libraries -- if it's in the install, use install?
getcwd() has equivalents on all of those platforms, but it might not be what you want -- it depends on how you run the executable.
To get the process's location is system specific, I think, but it shouldn't be too hard to wrap that.
The installed version should not have the build directory in the rpath.
You might want to do the linking twice (once for the build version and once for the installed version). Usually, on *nix systems, the installed binary has some static path where it tries to find plugins. You might define some environment variable (or command-line argument) to overload it for the build execution (and use a wrapper script to set it in the build environment).
Check how it is solved by some projects (Firefox for example).
I don't know much about windows system but I think the standard way of doing this is to search plugins in the same directory as the executable.