CLion can't find shared library when running an executable - c++

I am working on a project. I've been using a simple editor so far and my own Makefile to build it. I would like to switch to CLion, though.
According to this question you can tell CMake to run your Makefile. So my CMake.txt looks like this:
cmake_minimum_required(VERSION 3.6)
project(rekotrans_testbed_simulator)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
add_custom_target(rekotrans_testbed_simulator COMMAND make -C ${rekotrans_testbed_simulator_SOURCE_DIR} CLION_EXE_DIR=${PROJECT_BINARY_DIR})
It builds fine. I also set the working directory and pointed at the correct executable.
In my project I test using cppunit 1.13. However it can't find the shared library:
/home/kunterbunt/dev/comnets/git-repository/rekotrans-testbed-simulator/rekotrans-testbed-simulator-tests: error while loading shared libraries: libcppunit-1.13.so.0: cannot open shared object file: No such file or directory
LD_LIBRARY_PATH points to
echo $LD_LIBRARY_PATH
/usr/local/lib
and /usr/local/lib contains the library:
ls /usr/local/lib/
libcppunit-1.13.so.0# libcppunit-1.13.so.0.0.2* libcppunit.a libcppunit.la* libcppunit.so# pkgconfig/
ldd shows this:
ldd /home/kunterbunt/dev/comnets/git-repository/rekotrans-testbed-simulator/rekotrans-testbed-simulator-tests
linux-vdso.so.1 (0x00007ffc257e8000)
libboost_thread.so.1.63.0 => /usr/lib/libboost_thread.so.1.63.0 (0x00007f1c73254000)
libboost_system.so.1.63.0 => /usr/lib/libboost_system.so.1.63.0 (0x00007f1c73050000)
libboost_date_time.so.1.63.0 => /usr/lib/libboost_date_time.so.1.63.0 (0x00007f1c72e3f000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f1c72c22000)
libboost_program_options.so.1.63.0 => /usr/lib/libboost_program_options.so.1.63.0 (0x00007f1c729a4000)
libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f1c727a0000)
libcppunit-1.13.so.0 => /usr/local/lib/libcppunit-1.13.so.0 (0x00007f1c72563000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f1c721db000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f1c71ed7000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f1c71cc0000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f1c71922000)
librt.so.1 => /usr/lib/librt.so.1 (0x00007f1c7171a000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1c7347c000)
So why can't CLion find it? Everything works if I run the binary from console.

Have a look at How to set the environmental variable LD_LIBRARY_PATH in linux:
If you add your custom library path to the LD configuration, then CLion will find your libraries automatically and you don't have to add them to the run configurations.
On Ubuntu/Debian, you can configure LD by creating a new .conf file
sudo nano /etc/ld.so.conf.d/myLocalLibs.conf
that simply contains the path to your libraries: /usr/local/lib. Finally, call
sudo ldconfig
to update the LD configuration.
Note that on some systems (Ubuntu/Debian) you cannot set the LD_LIBRARY_PATH in /etc/profile or /etc/environment:
Since Ubuntu 9.04 Jaunty Jackalope, LD_LIBRARY_PATH cannot be set in
$HOME/.profile, /etc/profile, nor /etc/environment files. You must use
/etc/ld.so.conf.d/*.conf configuration files. See Launchpad bug #366728
for more information.
(help.ubuntu.com)

As oLen correctly pointed out, CLion doesn't seem to start as my user when launched via the GUI (Gnome in my case). I don't know as what it starts, but setting LD_LIBRARY_PATH=/usr/local/lib in /etc/profile and rebooting (or resourcing) it makes it work - in a nutshell the variable wasn't set for whatever user is running CLion.
Another way is Run -> Edit Configurations -> (select your application) -> Environment variables. Here you can manually set LD_LIBRARY_PATH to whatever you need, in my case to /usr/local/lib.

As an alternative to the accepted answer you can go to Run -> Edit Configurations -> Templates, choose CMake Application (and/or Google Test) and set Environment variables: to:
LD_LIBRARY_PATH=/usr/local/gcc-latest/lib64
Any new application created from now on will inherit these settings.

Related

Library not linked even though present in link targets

I'm trying to build an application using cmake but the executable is unable to link to the libraries, the ldd of my executable looks like this
linux-vdso.so.1 (0x00007ffd447a0000)
libavcodec.so.58 => /usr/local/ffmpeg/libavcodec.so.58 (0x00007f5673121000)
libavformat.so.58 => /usr/local/ffmpeg/libavformat.so.58 (0x00007f5672cdb000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f5671ffb000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f5671de3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f56719f2000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5671654000)
/lib64/ld-linux-x86-64.so.2 (0x00007f567629b000)
libswresample.so.3 => not found
libavutil.so.56 => not found
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f5671435000)
libavutil.so.56 => not found
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f5671231000)
while in my make --trace the library is present
/usr/bin/c++ -I/home/arunabh-compute/sensor-drivers -isystem
/usr/local/include -isystem /usr/local/GenICam/include -Wall -O3 -
std=gnu++14 -o CMakeFiles/save_cpp.dir/stdafx.cpp.o -c /home/arunabh-
compute/sensor-drivers/stdafx.cpp
CMakeFiles/save_cpp.dir/build.make:178: update target 'save_cpp' due
to: CMakeFiles/save_cpp.dir/link.txt
CMakeFiles/save_cpp.dir/Cpp_Save.cpp.o
CMakeFiles/save_cpp.dir/stdafx.cpp.o
CMakeFiles/save_cpp.dir/build.make /usr/local/ffmpeg/libavcodec.so
/usr/local/ffmpeg/libavcodec.so.58
/usr/local/ffmpeg/libavcodec.so.58.18.100
/usr/local/ffmpeg/libavdevice.so /usr/local/ffmpeg/libavdevice.so.58
/usr/local/ffmpeg/libavdevice.so.58.3.100
/usr/local/ffmpeg/libavfilter.so /usr/local/ffmpeg/libavfilter.so.7
/usr/local/ffmpeg/libavfilter.so.7.16.100
/usr/local/ffmpeg/libavformat.so /usr/local/ffmpeg/libavformat.so.58
/usr/local/ffmpeg/libavformat.so.58.12.100
/usr/local/ffmpeg/libavutil.so /usr/local/ffmpeg/libavutil.so.56
/usr/local/ffmpeg/libavutil.so.56.14.100
/usr/local/ffmpeg/libswresample.so
/usr/local/ffmpeg/libswresample.so.3
/usr/local/ffmpeg/libswresample.so.3.1.100
/usr/local/ffmpeg/libswscale.so /usr/local/ffmpeg/libswscale.so.5
/usr/local/ffmpeg/libswscale.so.5.1.100
even though libswresample.so.3, libavutil.so.56 is clearly present in the make --trace the application doesn't link to it for some reason, additionally if i put the /usr/local/ffmpeg folder in my LD_LIBRARY_PATH everything starts working just fine. Additionally libavcodec.so.58 is found. Trying to understand what's happening and how to fix it. Thanks.
By linking with these libraries, you marked them as needed in your binary, without recording any paths.
By default, your system only looks for dynamic libraries in a handful of locations.
You have a few options:
Add /usr/local/ffmpeg to a file in /etc/ld.so.conf.d/ (don't forget to call ldconfig). This will tell your dynamic linker (ld.so) to look there.
Add /usr/local/ffmpeg to the LD_LIBRARY_PATH environment variable as you did. You can make this permanent by editing /etc/environment on some machines. This has the same effect, but can be overridden by users. Most games on Linux rely on this to find the bundled libraries, for example, but this does necessitate a launcher script.
Link your binary with -rpath=/usr/local/ffmpeg. This informs the dynamic linker that you would like it to look there too. You can do this in CMake by setting the INSTALL_RPATH target property. You probably want to set BUILD_WITH_INSTALL_RPATH as well, so you do not have to set LD_LIBRARY_PATH during development or set a separate BUILD_RPATH.
Finally, you can just move the ffmpeg libraries to a directory in the default search path. However, this may clash with system libraries and requires root access. This is more a last-resort or convenience-over-everything approach.

Qt 5.8 (ubuntu 16.04) application deployment on Linux

I have an application created by using Qt 5.8. The app is developed on Ubuntu 16.04.
After run ldd app_name I can see the libraries the app is using:
linux-vdso.so.1 => (0x00007ffe16bef000)
libmysqlclient.so.20 => /usr/lib/x86_64-linux-gnu/libmysqlclient.so.20 (0x00007f1f455b0000)
libQt5Widgets.so.5 => /home/r0ng/Qt/5.8/gcc_64/lib/libQt5Widgets.so.5 (0x00007f1f44d7f000)
libQt5Gui.so.5 => /home/Qt/5.8/gcc_64/lib/libQt5Gui.so.5 (0x00007f1f445e3000)
libQt5Core.so.5 => /home/Qt/5.8/gcc_64/lib/libQt5Core.so.5 (0x00007f1f43ec3000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1f43b41000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1f4392a000)
... ...
Now, I am trying to deployment my application on other Linux machines.
The first thing I checked is the official doc:
http://doc.qt.io/qt-5/linux-deployment.html
I found I do NOT have the configure script.
find -type f -name "configure*"
./Docs/Qt-5.8/qtgamepad/images/configuregamepadbuttons-example.png
./Docs/Qt-5.8/qtdoc/configure-options.html
./Examples/Qt-5.8/gamepad/configureButtons/configureButtons.pro
./Examples/Qt-5.8/gamepad/configureButtons/doc/images/configuregamepadbuttons-example.png
./5.8/gcc_64/mkspecs/features/configure_base.prf
./5.8/gcc_64/mkspecs/features/configure.prf
./5.8/android_armv7/mkspecs/features/configure_base.prf
./5.8/android_armv7/mkspecs/features/configure.prf
Do I need to install Qt in a specific way to get the configure script? I also try to find the script in Qt Maintenace tool. But nothing there.
Another attempt is based on this thread
Deployment of QT apps on linux. It seems Qt project file (*.pro) allowing developers copy all dynamic libraries to the application's current working directory. But I have no clue to set value ORIGIN. And what does -Wl -rpath do in that case?
Anyone knows how to let the Qt load *.so file from the current working directory?
Thanks
The configure script is present when you build Qt from sources and probably you don't need that. To load shared libraries from the current working directory you need to point the LD_LIBRARY_PATH env variable to the directory where the shared libraries are located. You can find an example of a script that does this in the official Qt documentation for Linux Deployment. For example i use this modified script on my deployments:
#!/bin/sh
dirname=`dirname $0`
tmp="${dirname#?}"
if [ "${dirname%$tmp}" != "/" ]; then
dirname=$PWD/$dirname
fi
LD_LIBRARY_PATH=$dirname
export LD_LIBRARY_PATH
$dirname/myapp "$#"
And this is an example of the directory layout of one of my Qt apps that i currently distribute:
./myapp
./myapp.png
./myapp.sh
./libQt5X11Extras.so.5
./libQt5Core.so.5
./libQt5Network.so.5
./libQt5XcbQpa.so.5
./libicui18n.so.56
./libQt5Gui.so.5
./libicuuc.so.56
./libicudata.so.56
./sqldrivers
./sqldrivers/libqsqlite.so
./libQt5Widgets.so.5
./translations
./translations/qt_es.qm
./translations/qt_en.qm
./platforms
./platforms/libqxcb.so
./libQt5Sql.so.5
./libQt5DBus.so.5
./bearer
./bearer/libqnmbearer.so
./bearer/libqgenericbearer.so
./bearer/libqconnmanbearer.so
./imageformats
./imageformats/libqjpeg.so
Of course, you will need more or less libraries depending of the dependencies of your application. I currently use the official Qt 5.8 installer so the shared libraries comes from there instead of the ones provided by the distro.

How can I programatically determine where my C++ runtime libraries are?

I'm building C++11 with GCC 4.8 on CentOS 6 using "hax", then deploying on arbitrary CentOS 6 targets (on which anything C++-related out of the box will have been built as C++03 with GCC 4.3) (ref).
To make this work, I'm going to ship all of my third-party libs as well as the g++ runtimes, and rpath my executables so the newer libs are assuredly found in the proper place. For the runtimes, by my count I need to ship libstdc++ and libgcc_s. But I need to know where they are on my build system so that I can package them up.
Is there some neat way I can query for their location from within my packaging script?
(If the best approach is too awkward I'll just link them statically, but I'd like to avoid that if I can as my project includes several executables. Also if I were to statically link everything I believe I'd run the risk of GPL-ing my whole project, e.g. by statically linking the MySQL C API via my C++ MySQL wrapper lib. Could do a mixture of both, I suppose, though some sources warn against this…)
For bonus points, do I need to add anything to this list, out of libssl, libcrypto, libm, libpthread, libc, librt, libz and ld-linux-x86-64?
If I understand correctly, you have already built your binaries and just want to get a list of runtime libraries to package them along with binaries? You can try using ldd for this, like:
> ldd /usr/bin/ls
linux-vdso.so.1 (0x00007ffe76dd2000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fc97131f000)
libcap.so.2 => /lib64/libcap.so.2 (0x00007fc97111a000)
libacl.so.1 => /lib64/libacl.so.1 (0x00007fc970f10000)
libc.so.6 => /lib64/libc.so.6 (0x00007fc970b68000)
libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x00007fc970902000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fc9706fd000)
/lib64/ld-linux-x86-64.so.2 (0x000055c4ba4ed000)
libattr.so.1 => /lib64/libattr.so.1 (0x00007fc9704f8000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc9702db000)
This way you will see all libraries needed except, of course, the ones that are used via dlopen().
In my Makefile:
GCC_INSTALL_DIR := $(shell $(CXX) -print-search-dirs | grep install | cut -d' ' -f2)
…then my main build target will perform:
ln -sf $(GCC_INSTALL_DIR)/libstdc++.so $(BIN_DIR)/deps/
…and I can dump everything in $(BIN_DIR)/deps into the right place on install.
I think.

error while loading shared libraries: libQt5Multimedia.so.5: cannot open shared object file: No such file or directory

I have a 32-bit linux Virtual Box which I am compiling my Qt c++ code on.
I am copying the target files to a 64-bit linux box (the 32-bit executable should still run on this PC).
The build is using dynamic linking, so I was going to start copying across the Qt lib files that I need. I planned to just do this manually by running the program and letting it tell me which lib files are missing + I know many of the files that I need so it won't take me long (normally).
So in my 64-bit box target folder I have the files:
Rpe
So I run my executable called "Rpe" and it complains:
./Rpe: error while loading shared libraries: libQt5Multimedia.so.5: cannot open shared object file: No such file or directory
I expected this to happen, then I go and copy the file libQt5Multimedia.so.5 from my 32-bit linux qt lib folder into the target directory on my 64-bit linux box. In my target folder I now have the files:
Rpe
libQt5Multimedia.so.5
I now run the Rpe executable again and I expect it to complain about a different lib file, however it is still complaining about the same libQt5Multimedia.so.5 file not being found.
What am I doing wrong here?
EDIT-1
LDD output:
adadacha#duanedibbley:~/sandbox$ ldd Rpe
linux-gate.so.1 => (0xf7731000)
libQt5Multimedia.so.5 => not found
libQt5Network.so.5 => not found
libQt5Xml.so.5 => not found
libQt5Core.so.5 => not found
libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xf7616000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf75f9000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf744a000)
libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf7404000)
/lib/ld-linux.so.2 (0x5661f000)
adadacha#duanedibbley:~/sandbox$
I would create a directory for your "product" with the executable in a bin sub-directory and the Qt libraries (and any other libraries) in a lib sub-directory.
In the parent directory I would add a wrapper script to set $LD_LIBRARY_PATH and execute the binary (not tested):
#!/bin/sh
dirname="$(dirname "$0")"
exename="$(basename "$0")"
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"$dirname/lib"
export LD_LIBRARY_PATH
exec "$dirname/bin/$exename" $*

CMake: Linking Shared C++ Object from ExternalProject Produces Binaries with Relative Path, not Absolute

Problem: I am building an external project in CMake. The project has a Makefile that ultimately produces a shared object. I want to link against and install this object in my super project, just as if it were one of the libraries in the project. The problem is that the ExternalProject lib is getting linked into my applications and libraries with a relative path, not an absolute path which causes problems when running from any directory besides where CMake puts it.
I have created a sample SSCCE example project to demonstrate my overall setup. Feel free to peruse and compile if needed (git clone https://github.com/calebwherry/cmake-SO-question-main --recursive && cd cmake-SO-question-main && mkdir build && cd build && cmake .. && make && cd src/app/testApp && ldd testApp).
Whenever I run ldd on the executable and libs, I get output like this:
linux-vdso.so.1 => (0x00007fff8b5a2000)
libTestLib.so => /home/jwherry3/repos/cmake-superprj-main-test/build/src/lib/TestLib/libTestLib.so (0x00007f592da57000)
../../lib/libExtLib.so (0x00007f592d855000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f592d539000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f592d2b7000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f592d0a0000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f592cd14000)
/lib64/ld-linux-x86-64.so.2 (0x00007f592dc5a000)
I have tried all kinds of things dealing with RPATHS but can't get the ExtLib to link right. The lib that is local to the project (libTestLib.so) links just fine.
I also tried to set the LD_LIBRARY_PATH to override the relative path when I run the app but even when I do that, it still doesn't find the library. I suppose because it is relative it does not follow the normal linking order? The outcome is that the binary will not run unless I am in the directory where it resides.
I feel like I'm doing something really dumb when creating dependencies with the ExternalProject and that is my issue but I have beat my head for 3 days and haven't come up with anything.
System setup: Debian Wheezy 64-bit, CMake 3.0.2, g++-4.9.2.
Took awhile but I have finally arrived at an answer with help from the CMake Users listserv, specifically #brad-king.
The main issue is that I am not compiling my shared object correctly in my external project. Brad's answer to my question:
Thanks for the complete/simple example. The problem is that
the libExtLib.so file built by the external project does not
have DT_SONAME set by the linker. When a linker is given a
path to a shared library file that has no soname then the path
is copied into the DT_NEEDED field of the consumer since it
has no soname to use for that.
There are two solutions:
Make sure DT_SONAME gets set properly by the external build system.
Tell CMake that the imported library file has no soname:
set_property(TARGET ExtLib PROPERTY IMPORTED_NO_SONAME 1)
CMake will then link it via -lExtLib and the linker will not
store the path to the file in DT_NEEDED but only the file name.
Either of these should resolve the issue. Number 1 is cleaner.
Since I have control over the Make files in the external library, I opted for the cleaner first solution by compiling the shared object like so:
$(SHARED_TARGET): $(OBJECTS)
$(CXX) $(CXXFLAGS) $(OBJECTS) -o $# $(LDFLAGS) -Wl,-soname,$#
I have revised my original example and made it a lot simpler: https://github.com/calebwherry/cmake-SO-question-main. I will leave it up as a reference for anyone stumbling across this post at a later date.
P.S.
There is another fix that is sub-optimal. If I hadn't done any of the above, I could have just given target_link_library the full path to the library I was linking against and not used the imported library target. I have noted this in the CMake file in the repo and commented it out. Not a great solution but does solve the issue another way.