I am trying to do is produce a static MPI executable for Linux machines. The source code is written in C++. When I compile serial versions of the code, with gcc -static, and run ldd on the executable it shows no libraries.
However, when I try this with MPI, there are certain libraries that are not being statically linked. I have built my openmpi on my machine with see below shell commands. I also see that the exact same libraries that are not statically linking with the executable are not statically linking to the openmpi build. At the end of the openmpi build, I get
/home/bevan/Downloads/openmpi-4.0.1/opal/.libs/libopen-pal.a(dl_dlopen_module.o): In function `dlopen_open':
dl_dlopen_module.c:(.text+0x413): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/bevan/Downloads/openmpi-4.0.1/orte/.libs/libopen-rte.a(plm_rsh_module.o): In function `setup_launch':
plm_rsh_module.c:(.text+0xca2): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/bevan/Downloads/openmpi-4.0.1/opal/.libs/libopen-pal.a(if.o): In function `opal_ifaddrtoname':
if.c:(.text+0x1e4): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/bevan/Downloads/openmpi-4.0.1/orte/.libs/libopen-rte.a(ras_slurm_module.o): In function `init':
ras_slurm_module.c:(.text+0x6e4): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/bevan/Downloads/openmpi-4.0.1/opal/.libs/libopen-pal.a(evutil.o): In function `evutil_unparse_protoname':
/home/bevan/Downloads/openmpi-4.0.1/opal/mca/event/libevent2022/libevent/evutil.c:758: warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
The sort of command chain I feel I should be using is,
tar -xvf openmpi-4.0.1
cd ./openmpi-4.0.1
./configure --prefix="$HOME/.openmpi" --without-memory-manager CXX=g++ CC=gcc LDFLAGS=--static --disable-shared --enable-static
make
sudo make install
export PATH="$PATH:$HOME/.openmpi/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/.openmpi/lib/"
cd application/directory/build
cmake ..
make application
I would like to be able to run ldd application after compiling with MPI and have no dynamically linked libraries associated with the application.
Related
In my C++ project, I'm compiling and linking against a library that makes use of OpenSSL.
I need to compile this project for my BeagleBone which has openssl installed by default. I have downloaded libssl-dev on my development machine.
Thus, I can compile the project fine if I'm compiling for my development machine on x86_64, but I am not able to successfully cross compile:
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lssl
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lcrypto
This indicates I need to have armhf binaries for OpenSSL, which seems a bit of a waste really since I have them on my BeagleBone if it can just be patient and wait until I deploy it.
Is the only way around this cross-compiling OpenSSL myself? Where would I then need to install the .so files (I guess make install would be a bad idea?)
This indicates I need to have armhf binaries for OpenSSL
Correct.
which seems a bit of a waste really since I have them on my BeagleBone if it can just be patient and wait until I deploy it.
You appear to think that shared libraries are only needed at runtime, but that is not the case.
ELF stands for executable and linking format. The .so is very much needed at static link time to construct various tables in the main executable, which will then be used by the loader at runtime to resolve references from the main executable to the .so.
If you are familiar with Win32, you can think of .so as a combination of Win32 .LIB and .DLL packed into a single file.
For reasons beyond the scope of this question I have to statically link the libstdc++ to my executables. The practical downside is that now GDB can't step in to the stdlib symbols. When I was using the shared linked variants of my executable GDB had no issues to show me the accompanying source files (after I had installed the source package through apt-get under ubuntu which installed it under /build)
How can I step into libstdc++ functions when it is statically linked to an executable in GDB, under Ubuntu (14.04)?
Ubuntu package libstdc++6-4.8-dbg provides the static library with debug symbols at /usr/lib/x86_64-linux-gnu/debug/libstdc++.a; try building your executable against it and then run gdb.
I am working on a vision project using a beaglebone white. I am using an i686 machine running Ubuntu 12.04 LTS and the eclipse IDE with CDT plugin as my development machine. My beaglebone is running the latest Angstrom distro provided from beaglebone.org. My question has to do with general cross-compiling methodologies.
My program uses OpenCV and Curl c++ libraries.
So far on my host machine I have downloaded the latest OpenCV and Curl libraries and have crossed compiled them for the arm-linux architecture.
My test program compiles without errors on my development pc and generates an executable.
I use SCP to transfer the executable to the beaglebone over ethernet, and when I run my program I get the following error on the beaglebone:
"error while loading shared libraries: libopencv_core.so.3.0: cannot open shared object file: No such file or directory"
On the host computer OpenCV and Curl source and libraries are in two separate locations.
For OpenCV I used:
sudo cmake -DSOFTFP=ON -DCMAKE_TOOLCHAIN_FILE=../arm-gnueabi.toolchain.cmake ../../..
sudo make
sudo make install
which creates arm-compiled version of OpenCV in the /home/OpenCVArm/opencv/platforms/linux/build_hardfp/install/ on my host.
For Curl I used:
sudo ./configure --host=arm-linux-gnueabi --build=i686-linux CFLAGS='-Os' --with-ssl=/usr/bin/openssl --enable-smtp
sudo make
sudo make install
which creates the Arm compiled curl library is in /usr/local/ on the host.
to link all the libraries in my program I use the following script in Eclipse:
arm-linux-gnueabi-g++ -L/usr/local/lib -L/home/OpenCVArm/opencv/platforms/linux/build_hardfp/install/lib -L/usr/arm-linux-gnueabi/lib -o "HelloWorlTest" ./src/HelloWorlTest.o -lopencv_highgui -lopencv_core -lopencv_imgproc -lcurl
My questions are:
It appears I can get rid of my shared library error on the bone, by copying the appropriate libraries from my arm-compiled versions on the host to the target. So the target needs a copy of all libraries as well in order for the program to run. Since these are shared libraries and they are not included in the final executable, why do I need to compile the source for the target platform on the host in order to make the host linker happy? It appears the arm-compiled versions of the shared libraries are never used on the host. I initially thought it was so they would be packaged with the executable, but that is obviously incorrect.
If I copy the needed shared libraries from the host to the directory where the executable is stored on the target, the program still fails to find the shared libraries. The program will only run if I place a copy of the needed .so files in the /usr/lib/ folder on the target. What folders are searched for shared libraries when running an executable? Why won't it find shared libraries within its own local folder?
As I add more libraries to my project, what is the best way to manage them, and get them on the target. I really do not want to download the source on my host, cross-compile for arm, and then sift through all the libraries generated to only transfer the .so files I need on the bone. What is the proper way to provided the target with only the needed libraries for the executable? Is there a tool/plugin to manage or make this process automated?
How can I determine what are the required libraries irrespective of all the libraries I added to the eclipse linker?
If I wanted to tell eclipse to not use shared libraries how do I change the compile scripts for OpenCV, Curl, and modify eclipse so that static libraries are used instead?
When doing embedded programming, and cross-compiling is it more typical to use shared libraries or static libraries?
Thanks for the help.
You are just making the linker happy having the shared library on the host. It looks in the shared libraries to make sure the symbols your program uses are resolved. They are not linked in or used for anything else.
/lib and /usr/lib are the usual place to find shared libraries. You can add directories to the dynamic loader's search path by defining the LD_LIBRARY_PATH environment variable:
setenv LD_LIBRARY_PATH /home/me/lib:/home/me/lib2
I have no clue if there is some kind of tool/plugin for this. I use scp. ;-)
The ldd command will tell you what shared libraries an executable uses.
Good question. I've never built them. Often packages will build both shared and static libraries.
I don't know if is more typical to use shared libraries or not. I generally use static libraries. In my ELLCC cross compiler project.
I have used ELLCC to build itself. The resulting statically linked executables were actually smaller than the gcc compiled executable that uses shared libraries. Of course that is with an entirely different set of C++ and C standard libraries.
I've compiled a command-line tool against some C++ dynamic libraries using GCC 4.7 on Mac OS X 10.8. On the development system, the compiler was installed by MacPorts into /opt/local and the libraries reside in /usr/local/lib. The dynamic libraries are compiled from source alongside the program. (But they're built by cmake and I don't want to mess with that system.)
When I try to run it on another machine by putting the necessary dylibs into the executable's directory and DYLD_LIBRARY_PATH, it complains about an undefined symbol in the C++ standard library. It appears to be trying to load the older, builtin GNU standard library from /usr/lib/libstdc++.6.dylib.
How can I force the system to load the desired libstdc++?
By default, libboostpython.a is compiled without -fPIC. But I have to make a python extension and it is a dynamic library with -fPIC that links to static libraries.
How can I compile a static library (libboostpython.a) with -fPIC from boost.python?
There are a couple options you could use:
Compile boost from source and pass extra compiler options to bjam. E.g. bjam ... cxxflags='-fPIC'. That would compile every boost source file as position independent code.
Use boost in the form of shared libraries. In this case you probably want to ship boost shared libraries along with your application to make sure the appropriate version of boost is used. You can link your executable with '-Wl,-rpath,$ORIGIN' flag, so that when the dynamic linker searches for shared libraries required by your executable it looks for them in the directory where the executable is. See man ld.so for more details on $ORIGIN.
Note that if you already run bjam once you need to clear the targets first
it is helpful also to print the commands by applying -d+2:
./bjam clean &&
./bjam -d+2 link=static cxxflags="-fPIC" install