I have downloaded a ROS2 demo from the examples repository.
Specifically, I have used minimal_subscriber and minimal_publisher.
I have ROS2 installed in /opt/ros2 and when I build these two examples with colcon build, it generates an install/ directory with lib/, shared/ and the other usual directory structure.
I can execute and run these demos perfectly fine with my current setup in my machine, but these executables link to libraries present in /opt/ros2, so when I want to execute them in another machine without ROS2 installed or I move my ROS2 installation in my machine, the executables cannot find the shared objects.
I've added a really simple script that adds all dependencies to install/lib when building but the executables don't seem to care, they aren't looking for shared libraries in the generated lib directory, they keep trying to search in /opt/ros2.
I believe this is something I should solve in CMake and it's not ROS2 specific, so, is there any way I can tell my generated executables to search in a diferent directory? (../lib or ./lib in my case)
If you are building them yourself (assumed since you mention CMake), you can set the RPATH in CMake (docs here). Specifically set the CMAKE_INSTALL_RPATH something like:
set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
If you can't rebuild them, you can set LD_LIBRARY_PATH in your environment to where the libraries are located, or you can patch the executables themselves with an updated RPATH by using patchelf.
In order to get a relative RPATH rather than an absolute RPATH, use the $ORIGIN variable in your rpath spec. See "Recommendations" the the link above for more details.
Related
I have an issue where I'm developing on one system where Boost is installed on:
/usr/include
/usr/lib
On a system I will deploy this on, the libboost libraries are at:
/nfs/mount/boost
/nfs/mount/lib
And I can't go about changing every system I deploy on to install libboost in the same place.
Is there a way to either:
include libboost as part of the binary executable such that loading from the system lib paths is not needed.
make the executable search for different directories when trying to load to libboost?
I'm using g++ 8
Sounds like you need a more sophisticated build environment.
I'm not sure what you mean here:
include libboost as part of the binary executable such that linking is not needed
Linking is not something you can skip. If you are trying to avoid distributing .dll/.so files with your executable, you need to avoid using the portions of the boost library that require compilation of the boost binaries (i.e. those not listed here https://www.boost.org/doc/libs/1_62_0/more/getting_started/windows.html#header-only-libraries).
Of course, that is not often the case. So...
make the executable search for different directories when trying to link to libboost?
This will never work reliably as you scale and is a nightmare in the CI world. This is where package managers such as conan (https://conan.io/) come to save the day. Delegating the package management to a third-party is the most reliable way of getting your code to build across multiple environments/platforms.
Also, building your executable and distributing it are separate operations. Any dynamically linked libraries will need to be discoverable on the system path at runtime.
I have a question related to how to install a built executable program with cmake when it relies on some external libraries. Suppose my executable is abc, and it relies on two external libraries: lib1.so and lib2.so. The structure of the codes are as follows:
-.........
|----bin (lib1.so lib2.so)
|----include(lib1.h lib2.h)
|----src(main.cpp)
When the executable program is installed using the following cmake commands:
INSTALL(TARGETS ${Exe_Name}
RUNTIME DESTINATION Path to bin
LIBRARY DESTINATION Path to bin)
I expect that the executable program will be in the same directory with lib1.so and lib2.so. However, when I execute the built program in the installation folder, I met the following error:
error while loading shared libraries: lib1 can not open shared object file No such file or directory
If I use ldd to check the executable, I found lib1.so and lib2.so not found. After searching for possible solutions, I found if I call the executable in this way, then it worked:
LD_LIBRARY_PATH=./ ./my_program_run
Then my question is how I can let my executable program knows the locations of the shared libraries with cmake when it is installed? Thanks.
This is best solved this with the RPATH of the final executable. RPATH is a hardcoded search path for the executable itself, and allows the use of the string $ORIGIN, which expands to the location of the executable at runtime. See this reference: http://man7.org/linux/man-pages/man8/ld.so.8.html
CMake strips the rpath of a binary at installation time, to avoid the binary picking up libraries littered around your development tree. But it also provides a simple way to modify the installation rpath for exactly this reason. Here's the short answer:
IF(UNIX)
SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_RPATH}:\$ORIGIN/../bin:\$ORIGIN")
ENDIF()
This particular example appends to the existing rpath, and adds . and ../bin to the search path, all relative to the location of the binary.
Some developers claim that adjusting the RPATH of the binary is not a good idea. In the ideal world, all the libraries would live in the system library directories. But if you take this to the extreme, you end up with Windows (at least the older ones), where c:\windows\system32 is full of junk that came from who knows where, and may or may not conflict with other software, etc. Using rpath and installing everything in one place seems like a great solution.
If the application is to be cleanly installed to a standard linux distribution, then you should either install the supporting shared libraries into a standard location (/usr/lib), or you should add the libraries location to the ld.so config, by create an /etc/ld.so.conf.d/myprogram.conf file containing the name of the directory the libraries are in.
If the installation is temporary or more ad-hoc, then a script to set the LD_LIBRARY_PATH is suitable.
The libraries are searched in the predefined locations which includes standard library paths configured with ld.so.conf and LD_LIBRARY_PATH. You can also try to compile your app with -rpath, but it is not recommended by some developers. I suggest you to create a wrapper script which will set LD_LIBRARY_PATH and run the real application like that:
"theapp" script:
#!/bin/sh
dir="`dirname \"$0\"`"
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}"$dir"
exec "$dir/theapp.real" # your real application
The application, script and libraries should be in the same location (under bin/).
I have the following situation:
I'm working on an application that depends on a number of third party libs, among them wxwidgets
I build the application for multiple target configurations (x86, arm, Linux, Windows) using Linux as my build host system
Due to the above mentioned multiple target configurations, I have chosen to build those third-party libs from source, using CMake's ExternalProject_Add function.
The third-party libs are built 'on-demand' at a location separate from my application's CMAKE_BINARY_DIR so that I can wipe the build tree for my application without having to rebuild the third-party libs (takes a looooong time).
The location of the third-party libs is different depending on what target configuration I build them for (obviously).
I'm quite new to CMake and the problem I currently face is this:
The source files in my application can't find the wx include files and I need to set the correct linker flags to be able to link my application against wxwidgets.
This seems to be handled by a utility 'wx-config' that provides exactly that info as output when run with either the --cppflags or --libs flag. I can not however, figure out how to catch that output and append it to the include dirs and linked libraries I setup from my CMakeLists.txt files.
So basically what I want is.
Build wxwidgets (if it doesn't exist) for the current target configuration
Run wx-config --cppflags and --libs to find out the correct include dirs and linker flags for the current target configuration
Use the info from step 2 when building targets that are my own application
So far I've tried something like this:
# Set a target-configuration-specific location
set(wxwidgetsTop ${MYPROJECT_EXTERNAL_DIR}/wxwidgets/wxwidgets_${MYPROJECT_CURRENT_TARGET_CFG})
# Build the project
ExternalProject_Add( wxWidgetsExternal
PREFIX ${wxwidgetsTop}
URL ${MYPROJECT_EXTERNAL_DIR}/tarballs/wxWidgets-3.0.2.tar.bz2
SOURCE_DIR ${wxwidgetsTop}/src/wxwidgets
CONFIGURE_COMMAND ${configure_cmdline}
BUILD_COMMAND make -j${MYPROJECT_NCPU}
INSTALL_COMMAND make install
)
# Create a wxwidgets target to be used as a dependency from other code
add_library(wxWidgets IMPORTED STATIC GLOBAL)
add_dependencies(wxWidgets wxWidgetsExternal)
# (non-working) attempt to get the correct include dirs and linker
# flags for wxwidgets
add_custom_command(TARGET wxWidgetsExternal
POST_BUILD
COMMAND ${INSTALL_DIR}/bin/wx-config ARGS --cppflags
COMMENT "Running wx-config"
)
but the above does not provide a way to actually use the result from the custom command to append the cppflags and linker options when building the targets that make up my application.
What is a good way to achieve what I want?
I see three different ways of doing this:
Method 1: use find_package
Use wxWidgets as a standalone requirement for your project, and expect the devs to install it before building your project. In your CMakeLists.txt you will need to call find_package(wxWidgets), like this:
find_package(wxWidgets COMPONENTS net gl core base)
if(wxWidgets_FOUND)
include(${wxWidgets_USE_FILE})
# and for each of your dependent executable/library targets:
target_link_libraries(<YourTarget> ${wxWidgets_LIBRARIES})
endif()
This has the advantage of not rebuilding the lib if you rebuild your project, however it requires some work for your user (they need to handle the installation of wxWidgets by hand) and for you (you need to setup include paths / compile definitions / ... by hand).
Method 2: embed wxWidgets
The second option is to bundle wxWidgets in your repo (svn external or git submodule) and usually (re)write the CMakeLists.txt of this lib to be target-oriented. Then, in your top-most CMakeLists.txt, you can do the following:
# for example, if you just need core and net:
target_link_librairies(my_app PUBLIC wxWidgetsCore wxWidgetsNet)
# No need to manually setup include dirs, etc...
To make a CMakeLists.txt target-oriented, you define include directories and other compilation properties for a target, not a directory. Example:
# When defining wxWidgetsCore, for example
add_library(wxWidgetsCore ...)
target_include_directories(wxWidgetsCore PUBLIC someDir)
target_compile_definitions(wxWidgetsCore PUBLIC -pedantic)
target_link_libraries(wxWidgetsCore PUBLIC someLib)
The drawback of this approach is that rebuilding your project will trigger a rebuild of wxWidgets. However, it is possible to trick this by not using "rebuild" but "clean just my app, then build". Here is some insight on how to achieve this.
Method 3: some sort of hybrid
The big drawback of method 2 leads to the third approach: don't put wxWidgets in your project, but create a CMakeLists.txt that will "import" the lib. The idea: you ask your user for the directory where wxWidgets is installed, then this script will setup everything for your project. First, put the CMakeLists.txt here:
/your-project-root
/thirdparty
/wxWidgets
CMakeLists.txt
/dir-where-wxwidgets-is-installed
...
Now, you define an imported target:
# When defining wxWidgetsCore, for example
set(WX_INCLUDE_DIR ${USER_SPECIFIED_WX_ROOT}/include)
add_library(wxWidgetsCore IMPORTED GLOBAL)
set_property(TARGET wxWidgetsCore APPEND PROPERTY
INTERFACE_INCLUDE_DIRECTORIES ${WX_INCLUDE_DIR})
See INTERFACE_INCLUDE_DIRECTORIES and INTERFACE_LINK_LIBRARIES. You need your user to have build wxWidgets somewhere in his system, but from your point of view you just do target_link_libraries(your_app PUBLIC wxWidgets...), as in method 2. The advantage is that this approach is interchangeable with method 2 transparently, and you don't put the whole dependency in your project.
Setting cppflags and linker flags has to be done at CMake time, but you are trying to run wx-config at build time and you are not capturing its output anyway, so your add_custom_command() isn't doing anything useful other than printing things to the build tool's output.
Ideally, you would use the FindwxWidgets module CMake already provides. It requires wxWidgets to already be built (but see further below). Have a look at the CMake documentation for it and see if that at least sounds like what you are trying to achieve manually by using wx-config. If you can get FindwxWidgets to do the job for you, that would be a much cleaner approach.
Getting something to build at configure time so you can use it later on in your CMakeLists.txt file is a bit more tricky. ExternalProject_Add() downloads and builds things at build time, but you need wxWidgets to be built earlier at configure time. I wrote an article recently for how to do at least the downloading part at configure time and you should be able to adapt it to do the whole build at configure time instead. The article uses Google Test as its example and can be found here:
https://crascit.com/2015/07/25/cmake-gtest/
It would be trivial to make it put the wxWidgets build wherever you like, not just in the CMAKE_BINARY_DIR area. That would allow you to have different wxWidgets builds for each build configuration and to be able to wipe out your application's build tree independently of the wxWidgets builds.
Hope that points you in the right direction.
The solution I use checks for wxWidgets installation in the system using find_package, if it's not found, then the script downloads wxWidgets from github and links the program against downloaded library. The lib is installed in the build directory, so only the first build is slow - subsequent builds do not even check wxWidgets sources timestamps, so the process is as fast as building using preinstalled wxWidgets library.
Here's how my script does it:
It quietly checks for wxWidgets installation using find_package(wxWidgets QUIET),
If it's found, the script adds a dummy library wxWidgets_external,
If it's not, then it creates an ExternalProject named wxWidgets_external which downloads, builds and installs the library in the build dir, setting wxWidgets_ROOT_DIR to point to the wxWidgets installation dir,
Then we add another ExternalProject pointing to a folder with the main program's source files and CMakeLists.txt build script. This external projects depends on wxWidgets_external which is either a dummy library in case wxWidgets is preinstalled in the system, or an external project set up to download the library from github,
In the aforementioned CMakeLists.txt we again call find_package, this time with REQUIRED parameter and use the library the standard way (https://docs.wxwidgets.org/trunk/overview_cmake.html). Because we set up the dependencies and variables correctly, this call will use either preinstalled wxWidgets (if it's available) or the one downloaded from github.
There are more quirks to it, but that's the gist of it. The full sample code (tested on Linux, Windows and Mac) is available on github (https://github.com/lszl84/wx_cmake_template).
Also see full blog post which explains this in more detail: https://justdevtutorials.medium.com/wxwidgets-cmake-multiplatform-superbuild-4ea86c4e6eda
I'm working on multiple projects at the same time (some libraries, and some games that depend on them). They're all on GitHub, in separate repos.
For convenience, I pull every repo in a "workspace folder", like this:
/home/myWorkspace/Library1/
/home/myWorkspace/Library2/
/home/myWorkspace/Library3/
/home/myWorkspace/App1/
/home/myWorkspace/App2/
I have FindLibrary1, FindLibrary2 and FindLibrary3 .cmake files, which, in order, look for the library in ../ (which corresponds to the "myWorkspace" folder), then /usr/lib/.
While on Windows CMake finds the libraries in myWorkspace/, on Linux, no matter what, installed libraries are always found first.
Since I'd like to work in myWorkspace/ folder, then installing the libraries after I'm done, I'd prefer CMake to find and link everything within the myWorkspace/ folder.
I'd also like CMake to search for the libraries in /usr/lib/ and /usr/local/lib/ if there is no myWorkspace/ folder, but if myWorkspace/ exists, it should have the priority.
Examples of CMake files I'm using:
SSVUtils: library with no dependancies
SSVUtils CMakeLists: https://github.com/SuperV1234/SSVUtils/blob/master/CMakeLists.txt
FindSSVUtils.cmake: https://github.com/SuperV1234/SSVUtils/blob/master/cmake/modules/FindSSVUtils.cmake
SSVUtilsJson: library that depends on SSVUtils and SSVJsonCpp
SSVUtilsJson CMakeLists: https://github.com/SuperV1234/SSVUtilsJson/blob/master/CMakeLists.txt
FindSSVUtilsJson.cmake: https://github.com/SuperV1234/SSVUtilsJson/blob/master/cmake/modules/FindSSVUtilsJson.cmake
Any ideas how I can prioritize the myWorkspace/ folder while still having the possibility to find libs in file system paths?
By default, find_path (or find_library, etc.) first checks for files in the system standard locations, before searching in the values provided in PATH. That's why "installed" libraries are always found first on Linux (but not on Windows, that doesn't have standard locations for installed libraries).
You can disable that behavior by using the NO_CMAKE_SYSTEM_PATH option: it will skip detection of files in standard locations.
Now... if you still want to use the installed libraries as a fallback when local versions are not found, you can do it in a two step process:
find_path(... NO_CMAKE_SYSTEM_PATH)
if (nothing_found)
find_path(...)
endif()
In the project I am working on, we provide the possibility to dynamically load additional features. For that we use dlopen.
To find this libraries we have something we call module path. There we have a default path, where the shared libraries are (a lot of them are shipped).
At the moment we have two default paths: we first look in the build directory for the shared library and afterwards in the install directory. This is because it should also be possible to run the application without installing it (so in that case it needs to look first in the build directory).
Now the problem ist, if a user builds the application from source and installs it with make install, the libraries in her build directory are loaded by default. This will result in a crash. So it only works if the user afterwards removes or renames the build directory.
No the question: is there a trick (either by C++ or by the build system) to know whether the application is installed or not. The problem is, that the functionality is implemented in a shared library and the implemented way to search for modules has to work also for other applications that link against our library (so we can not rely on the path of the executable). We use CMake as a build system.
To make the situation even harder, the solution has to work on Windows, Linux and Mac OS X.
EDIT:
I further investigated and the problem is more complicated. This is the situation:
There is a small executable a
Furthermore there is a "main" library main.so
then there is a dynamically loaded library lib.so
lib.so links against main.so
The problem is, that lib.so has the absolute path to main.so in the build directory in its rpath. Thanks to the tip of #MSalters I now was able to make a hack to make sure to load the correct version of lib.so (the one in the install directory) but since it has the build path in the rpath it loads the wrong main.so (so in fact there are two copies of main.so in the memory - this messes things up).
Is there a way to remove this reference to the build path from the library? I tried all options of cmake related to rpath without success
Can't you check where the executable itself is? If it's in the build directories, use build libraries -- if it's in the install, use install?
getcwd() has equivalents on all of those platforms, but it might not be what you want -- it depends on how you run the executable.
To get the process's location is system specific, I think, but it shouldn't be too hard to wrap that.
The installed version should not have the build directory in the rpath.
You might want to do the linking twice (once for the build version and once for the installed version). Usually, on *nix systems, the installed binary has some static path where it tries to find plugins. You might define some environment variable (or command-line argument) to overload it for the build execution (and use a wrapper script to set it in the build environment).
Check how it is solved by some projects (Firefox for example).
I don't know much about windows system but I think the standard way of doing this is to search plugins in the same directory as the executable.