CMake/Linker cross compile for OpenWRT - c++

unfortunately I have paused the porting of a software of mine under OpenWRT due to a linking problem, but in order to go deeper into the above issue I need to briefly give you some details about my software.
It is made of three modules: LIB-1, LIB-2 and APP.
I'm developing it under LUbuntu with Netbeans/C++ and CMake. I'm cross compiling by CMake (I successfully followed this guide http://www.vtk.org/Wiki/CMake_Cross_Com … chain_file) and a I have built a toolchain and a target Generic-x86 OpenWRT image by the latest official BUILDROOT tool (it works under VirtualBox like a charm).
I set the CMAKE_LIBRARY_OUTPUT_DIRECTORY and CMAKE_RUNTIME_OUTPUT_DIRECTORY to a common folder (${CMAKE_BINARY_DIR}/bin) in order to have all the binaries into the same folder.
Now, I build the project for LUbuntu and inside the ouput folder I find LIB-1.so, LIB-2.so and APP, as expected. Then I execute 'ldd' on APP and I see it referencing LIB-1.so and LIB-2.so as expected. By the verbose CMake output I see also that CMake sets (as expected) the RPATH which points to '${CMAKE_BINARY_DIR}/bin'.
Now go into the issue details...
I build the project for OpenWRT and inside the output folder I find all the three files, as expected. Then I execute 'ldd' and 'objdump -x' and I see that APP links to '../bin/LIB-1.so' and '../bin/LIB-2.so' (file name plus a relative path prefix!!!). The other difference is that CMake (I don't know why) does not add the RPATH on the linking command line!
In order to get APP linked to the two libraries without the relative '../bin' path, I have to put the two SO files inside the APP linking folder, and manually run the linker stripping the '../bin/' prefix, and adding '-Wl,-rpath,${CMAKE_BINARY_DIR}/bin' too.
Could anybody explain me:
1) Why CMake links the two platforms using different command line parameters?
2) How can I link with CMake avoind embedding the relative .SO path?
3) If none of the readers uses CMake, could somebody tell me if exists an additional parameter for the G++ linker to strip relative paths from the referenced LIBs?
Thank you very much!

for the moment I found a workaround.
I create a bash script like this (openwrt-fix-libs):
#!/bin/bash
TARGET_NAME=$1
OUTPUT_DIR=$2
# Prologue
ECHO_PREFIX=[----]
echo "$ECHO_PREFIX Fixing OpenWRT library imports for \"$TARGET_NAME\"..."
# Dependencies copy
echo "$ECHO_PREFIX Copying libraries locally from \"$OUTPUT_DIR\"...";
cp $OUTPUT_DIR/*.so .
if [ $? != 0 ]; then
exit 1
fi
# Link command hacking
LINK_COMMAND=./CMakeFiles/$TARGET_NAME.dir/link.txt
echo "$ECHO_PREFIX Hacking link command at \"$LINK_COMMAND\"..."
LINK_COMMAND=$(cat $LINK_COMMAND | sed -e 's/\.\.\/bin\///g')
#echo "$ECHO_PREFIX <$LINK_COMMAND>"
if [ $? != 0 ]; then
exit 1
fi
# Re-linking
echo "$ECHO_PREFIX Re-linking...";
$LINK_COMMAND
if [ $? != 0 ]; then
exit 1
fi
# Update rebuilt binary
echo "$ECHO_PREFIX Updating \"$TARGET_NAME\" binary..."
cp *$TARGET_NAME* ${OUTPUT_DIR}
if [ $? != 0 ]; then
exit 1
fi
# Epilogue
echo "$ECHO_PREFIX Libraries imports successfully fixed."
Then I added into the CMakeLists.txt file of any module I need it this custom command:
if (_PLATFORM STREQUAL "OPENWRT")
add_custom_command(
TARGET ${_TARGET_NAME}
POST_BUILD COMMAND ${_GLOBAL_SCRIPTS_DIR}/openwrt-fix-libs ${_TARGET_NAME} ${_GLOBAL_OUTPUT_DIR}
)
endif (_PLATFORM STREQUAL "OPENWRT")
Where _PLATFORM is a symbol set by me on the command line when I compile for OpenWRT, _GLOBAL_SCRIPTS_DIR is my project scripts folder, and _GLOBAL_OUTPUT_DIR then bin inside the CMake building root.
I hope it maybe useful for you too.

Related

CMake include_directories isn't using one directory, but using all others around it

I'm trying to use conda to manage several dependencies for a c++ library I'm building (I was previously using git submodules in my project's external/ dir which worked fine, but for a number of reasons I wanted to migrate to conda).
The libraries I need to use are tbb, zstd, and cspice. I've verified they are all installed correctly by conda, and are in my environment (named vira_env). The headers are very clearly in ~/mamba_envs/vira_env/include/.
However when I add include_directories(${CMAKE_INSTALL_PREFIX}/include) to my root CMakeLists.txt and run cmake using -DCMAKE_INSTALL_PREFIX=${CONDA_PREFIX}, it fails to find any of the headers.
Running make VERBOSE=1 reveals that g++ has no -I argument to include the ~/mamba_envs/vira_env/include/ directory. Which makes sense, as it is failing to find the headers that are there.
However, if I use include_directories(${CMAKE_INSTALL_PREFIX}) and rerun make VERBOSE=1, or it clearly shows that g++ has a -I /home/cgnam/mamba_envs/vira_env/.
Similarly, if I use include_directories(${CMAKE_INSTALL_PREFIX}/include/oneapi) it shows g++ has a -I /home/cgnam/mamba_envs/vira_env/include/oneapi. (oneapi is the directory where the tbb headers are placed).
So include_directories() will happily use either the conda environment root, or even one of the directories inside the conda environment's include/ directory... but it will just completely ignore the include/ directory itself.
EDIT
As discovered in one of my comments below, printing CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES shows /home/cgnam/mamba_envs/vira_env/include. Which explains why my include_directories() is not having any effect on the g++ call by make, since it believes that the compiler should be looking in that directory already.
make VERBOSE=1is showing that the compiler being used is: /home/cgnam/mamba_envs/vira_env/bin/x86_64-conda-linux-gnu-c++. However using which g++ is showing ~/mamba_envs/vira_env/bin/g++. Further, which ld returns: ~/mamba_envs/vira_env/bin/ld.
If I use ld --verbose | grep SEARCH_DIR, it does not show /home/cgnam/mamba_envs/vira_env/include. Neither does ~/mamba_envs/vira_env/bin/x86_64-conda-linux-gnu-ld --verbose | grep SEARCH_DIR.

Cross compiling with CMake: linker errors

I am attempting to cross compile a project for my raspberry pi (32 bit armv8) on my ubuntu machine. I set up a toolchain using crosstool-NG and compiling works, but linking fails. My CMake toolchain file is as follows:
SET(CMAKE_SYSTEM_NAME Linux)
SET(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSROOT /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/sysroot)
SET(CMAKE_C_COMPILER /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/bin/armv8-rpi3-linux-gnueabihf-gcc)
SET(CMAKE_CXX_COMPILER /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/bin/armv8-rpi3-linux-gnueabihf-g++)
SET(CMAKE_FIND_ROOT_PATH /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf)
SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# for libraries and headers in the target directories
SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
I have a few libraries that are not being found by the linker. The problem is with my sysroot. I copied over /usr and /lib from the pi to my sysroot folder that was created by crosstool-NG. The libraries that are being linked reside in sysroot/usr/lib/arm-linux-gnueabihf.
However, the linker is not checking that directory, it checks sysroot/usr/lib and a few others, but for some reason it does not check the arm-linux-gnueabihf directory.
On the pi, I can run ld -lasound --verbose and get the following output:
attempt to open //usr/local/lib/arm-linux-gnueabihf/libasound.so failed
attempt to open //usr/local/lib/arm-linux-gnueabihf/libasound.a failed
attempt to open //lib/arm-linux-gnueabihf/libasound.so failed
attempt to open //lib/arm-linux-gnueabihf/libasound.a failed
attempt to open //usr/lib/arm-linux-gnueabihf/libasound.so succeeded
so its finding it correctly on the pi.
When I do the same using the ld generated by crosstool-NG:
attempt to open /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/bin/../sysroot/home/stone/x-tools/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/lib/libasound.so failed
attempt to open /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/bin/../sysroot/home/stone/x-tools/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/lib/libasound.a failed
attempt to open /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/bin/../sysroot/usr/local/lib/libasound.so failed
attempt to open /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/bin/../sysroot/usr/local/lib/libasound.a failed
attempt to open /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/bin/../sysroot/lib/libasound.so failed
attempt to open /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/bin/../sysroot/lib/libasound.a failed
attempt to open /home/stone/development/raspberry_pi/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/bin/../sysroot/usr/lib/libasound.so failed
It fails to find it because it doesnt look in the arm-linux-gnueabihf directory. I copied the library .so file from sysroot/usr/lib/arm-linux-gnueabihf to sysroot/usr/lib and it compiles and links successfully, but I would like to make it so that I do not have to do that. How can I make the linker check that arm-linux-gnueabihf directory?
EDIT: I also copied /etc/ld.so.conf and /etc/ld.so.conf.d from the pi into my sysroot, but it does not seem to affect the linker.
EDIT: After further research it looks like this might be due to something with gcc-multiarch. I am not sure what that is but hopefully I can figure this out
EDIT: I verified that indeed the linker is not searching the arm-linux-gnueabihf path:
./armv8-rpi3-linux-gnueabihf-ld --verbose | grep -i "search"
outputs:
SEARCH_DIR("=/home/stone/x-tools/armv8-rpi3-linux-gnueabihf/armv8-rpi3-linux-gnueabihf/lib"); SEARCH_DIR("=/usr/local/lib"); SEARCH_DIR("=/lib"); SEARCH_DIR("=/usr/lib");
Alright I got it all figured out. The repo where I learned the tricks to getting this to work can be found here. I am grateful for this repo, without it I would not have gotten this to work.
The issue lies with the linker. The linker that comes with plain old binutils that gets pulled in by crosstool-ng does not search the arm-linux-gnueabihf subdirectories. These directories exist because of Debian multiarch, and in order to get a linker that looks in such a directory, binutils needs to be patched. The instructions that follow assume you have already generated a ct-ng config based on the rpi3 sample.
On the raspberry pi, I installed the binutils-source package:
$ sudo apt install binutils-source
This will give access to a patch file located in /usr/src/binutils/patches.
On the host machine, you need to add a patches directory in the directory where you are building your toolchain (where the ct-ng config file is). It must have a specific structure that reflects the structure of packages in the crosstool-ng repo (ie. inside patches/ you must have a directory with the package name, inside that you must have the version number, and inside that is where you place the patch.):
$ cd your_toolchain_directory
$ mkdir -p patches/binutils/2.31.1/
Now we can copy the patch file from the pi to the host. The specific patch we need is 129_multiarch_libpath.patch:
$ cd patches/binutils/2.31.1/
$ scp pi#raspberrypi:/usr/src/binutils/129_multiarch_libpath.patch
Now that we have the patch, we need to update the config file to tell ct-ng to include local patches as well as enable the multiarch flag for gcc. You can do this using ct-ng menuconfig so change back to the directory with the config file and run:
$ ct-ng menuconfig
In the paths and misc options you need to change the patches origin to bundled then local, and then add ${CT_TOP_DIR}/patches as the local patch directory. This should produce the following lines under the extracting section of the .config file:
CT_PATCH_BUNDLED_LOCAL=y
CT_PATCH_ORDER="bundled,local"
CT_PATCH_USE_LOCAL=y
CT_LOCAL_PATCH_DIR="${CT_TOP_DIR}/patches"
Next you need to add the --enable-multiarch flag in the gcc options. Again using menuconfig, go to the C compiler settings. Add --enable-multiarch to the gcc extra config setting. The config file should have the following in the gcc settings:
CT_CC_GCC_EXTRA_CONFIG_ARRAY="--enable-multiarch"
Save the config. The last thing we need to do is export a variable.
$ export DEB_TARGET_MULTIARCH=arm-linux-gnueabihf
Now we can build the toolchain:
$ ct-ng build
Once its done, you can change into the bin directory of the generated toolchain and run:
$ ./ld --verbose | grep -i "search"
and you should see that the arm-linux-gnueabihf directory is now in the search paths. What a process.
You need to use static library instead. then the executable will include the binary of the libraries.
But keep in mind the executable file will be bigger ( depends obviously which lib...).
When you using cross compile, you should check that the libraries you use exist in the other environment you execute the program.
The solution will be to add STATIC word according to CmakeList.txt file

Building libcsv problems

I'm having some trouble building the csv library libcsv.
I'm using Windows XP, MinGW compiler and MSYS command shell.
It configures ok (./configure) but when I go to build it I get the below error. I realise it has to do with autoconf (it requires 2.65 or above) So what do I do here. Do I download autoconf and have it in a separate directory marked c:\autoconf and have an environmental variable pointing to it? Or,, does it go in the MSYS folder.
I have downloaded the binaries for autoconf and I just get a bunch of files. The binary files are in a SHARE folder and a 'bin' folder. Do I put their contents into the MsYS share and bin folders? Looking around the web I'm also seeing references to putting the autoconf binaries into my MinGW/bin folder. I'm really not sure what to do here?
Can anyone help? Thank you.
xxxx#XXXX-2FEF09FD39 ~
$ cd C:\libcsv-3.0.3
xxxx#XXXX-2FEF09FD39 /c/libcsv-3.0.3
$ make
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /c/libcsv-3.0.3/missing --run autoconf
/bin/m4: unrecognized option `--gnu'
Try `/bin/m4 --help' for more information.
autom4te: /bin/m4 failed with exit status: 1
make: *** [configure] Error 1
xxxx#XXXX-2FEF09FD39 /c/libcsv-3.0.3
$

libSDL2-2.0.so.0: cannot open shared object file

I'm trying to build the SDL library from the source code. I've downloaded the compressed file (i.e. SDL2-2.0.3.tar.gz) and extracted it. I don't want to install the files in /usr/local. According to this link, I need to change the configure
The last command says "sudo" so we can write it to /usr/local (by
default). You can change this to a different location with the
--prefix option to the configure script. In fact, there are a LOT of good options you can use with configure! Be sure to check out its
--help option for details.
This is what I've done.
mkdir build
cd build
../configure
make
sudo make install
In install folder that I've created are the following files
share
lib
include
bin
Now I would like to run the test files. I've picked this testatomic.c and this is the command line
gcc testatomic.c -o test -I/home/xxxx/Desktop/SDL2-2.0.3/install/include/SDL2 -L/home/xxxx/Desktop/SDL2-2.0.3/install/lib -lSDL2 -lSDL2main
I get this error
error while loading shared libraries: libSDL2-2.0.so.0: cannot open shared object file: No such file or directory
In lib, these are the files
Where is the shared object file?
You're getting error when running resulting program because system's dynamic linker cannot find required library. Program requires libSDL2-2.0.so.0, linker looks for it in system-defined directories (/lib, /usr/lib, ..., - defined in /etc/ld.so.conf), but finds none - hence an error.
To inform linker where you want it to look for libraries, you can define LD_LIBRARY_PATH environment variable, e.g. in your case:
export LD_LIBRARY_PATH="$HOME/Desktop/SDL2-2.0.3/install/lib"
./test
Other ways is installing libraries in standard location, defining LD_LIBRARY_PATH in your .bashrc (or whatever shell you use), or using rpath, e.g. adding -Wl,-rpath=$HOME/Desktop/SDL2-2.0.3/install/lib at the end of your compilation line.
I was able to fix this problem with:
sudo apt install libsdl2-dev
I too had:
./01_hello_SDL: error while loading shared libraries: libSDL2-2.0.so.0: cannot open shared object file: No such file or directory
as a result of compiling the first C++ program (using the SDL headers) as part of the Lazy Foo tutorial. I found out that libSDL2-2.0.so.0 was just using the find command in the GUI. It turned out to be in /usr/local/lib
Then in terminal I typed:
export LD_LIBRARY_PATH="/usr/local/lib"
I checked the value of LD_LIBRARY_PATH using:
echo $LD_LIBRARY_PATH
I recompiled (don't know if that was necessary) and voila, it worked.

OpenCV Installation in Linux/Ubuntu

I am doing this Tutorial
http://docs.opencv.org/doc/tutorials/introduction/linux_install/linux_install.html#linux-installation
But I got confused. I stopped at building the OpenCV from source.
I already created an File called Workspace where I made the cmake_binary_dir (named release). I downloaded the sourcefile (which is in my home directory and named: opencv-2.3.1), and now I want to run this
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
where I use:
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/home/markus/opencv-2.3.1
But the Terminal keeps telling me, that this source directory does not exist!?
So what I am doing wrong?
CMAKE_INSTALL_PREFIX defines where to distribute the binary to after its compiled and linked, it defaults to good place (/usr/local/) so avoid defining it
You are leaving out the trailing .. in your cmake command which tells it where to get the source code hence the error message
Here are the typical steps when installing from source code any project which uses cmake
if you see a file :
CMakeLists.txt
in the src directory this indicates it wants you to use cmake
0 cd into dir where your expanded source code lives
1 mkdir build # make a build dir (initially empty)
2 cd build
3 cmake .. # NOTE those little .. which declares relative path to src dir
which says populate current dir (build) with compiled code
and get the source code and configs from parent directory (..)
4 examine the output, if it looks successful go to step 5
if it has errors you may need to install upstream dependent
libraries then try cmake again
5 make -j4 # compile source, -j speeds up by using multicore
6 sudo make install <-- only if above step 4 and 5 are OK
You can do everything cmake related from command line, yet its GUI can be quite handy especially with an unfamiliar project. In above instead of typing :
cmake ..
its GUI version is :
cmake-gui ..
in the GUI its easy to toggle on/off settings like to build examples or not ... the values column on the right is editable ... if you changed settings in the gui at bottom hit button Configure then when its done hit Generate to perform same as the normal cmake .. now return to step 4 above to do the compile