How do I use GCC options -iprefix and -iwithprefix? - c++

I have a c++ application that is separated into modules. The directory structure for each module looks like this:
module_a/
export/ <-- module_a public interface
src/ <-- module_a private source
test/
src/ <-- unittests for module_a
I'm using CMake to setup the module dependencies and to build the application. If module_a depends on module_b then module_b's export directory is added as include path when building module_a.
So, if there is a file module_b/export/foo.h that module_a needs then in a source file you will use #include "foo.h".
I'm looking for a way to to make so that the module name is part of the include directive. So for the above example I want to (have to) write #include "module_b/foo.h".
Is this something that can be done with GCC option -iprefix and -iwithprefix? I've searched for usage examples but all I can find is copies and references back to the GCC manual, which I think doesn't explain it very well.
I have tried to use it this way:
$ find -type f
./src
./src/main.cc
./export
./export/bar.h
$ g++ -iprefix foo/ -iwithprefix export/ src/main.cc
src/main.cc:1:10: fatal error: foo/bar.h: No such file or directory
1 | #include "foo/bar.h"
| ^~~~~~~~~~~
compilation terminated.
$ gcc --version
gcc (Debian 9.2.1-17) 9.2.1 20191102
But as you can see, it doesn't work. How should I use -iprefix and -iwithprefix?
Also, does anyone have another solution to my problem? I'm a bit worried that IDEs and other compilers might not understand -iprefix and -iwithprefix, so any other solutions are welcome as well.
Edit: after posting this question I immediately realized that perhaps -iprefix foo/ -iwithprefix bar is just a fancy way of writing -I foo/bar. However, I tested that and I still didn't get it to work. So it would still be good if someone could explain how these options works, even if they are not going to help me for my problem.

How should I use -iprefix and -iwithprefix?
It's just a shortcut to avoid having to write out a common prefix every time. So instead of:
cpp -Idirafter /sys/root/a -Idirafter /sys/root/b
You instead do:
cpp -iprefix /sys/root/ -iwithprefix a -withprefix b
(substitute -Iwithprefix for -Iwithprefixbefore if you want it to be equivalent to -I instead.)
This can come on handy with very long include directory paths like you may run into when cross-compiling (and may be even required, cf. getconf ARG_MAX)
Also, does anyone have another solution to my problem?
I stumbled upon -iprefix, because I've the same problem you have as well and hoped there would be an out-of-the-box option for that. There isn't, but Linux device tree builds have the same issue:
The device trees for ARCH=arm and ARCH=arm64 are in arch/arm/boot/dts and arch/arm64/boot/dts, respectively. Sometimes, arm64 device trees need some common file from the arm directory, but having the dts directories of all architectures in path isn't that great a solution.
What Linux does instead, is having a dedicated directory in the source tree with suitably-named symlinks into each architecture and this directory's path as well as the directory of the currently active ARCH go into the search path. That way, ARCH=arm can write #include "something.dtsi", but ARCH=arm64 has to write #include <arm/something.dtsi> instead.

Related

CMake include_directories isn't using one directory, but using all others around it

I'm trying to use conda to manage several dependencies for a c++ library I'm building (I was previously using git submodules in my project's external/ dir which worked fine, but for a number of reasons I wanted to migrate to conda).
The libraries I need to use are tbb, zstd, and cspice. I've verified they are all installed correctly by conda, and are in my environment (named vira_env). The headers are very clearly in ~/mamba_envs/vira_env/include/.
However when I add include_directories(${CMAKE_INSTALL_PREFIX}/include) to my root CMakeLists.txt and run cmake using -DCMAKE_INSTALL_PREFIX=${CONDA_PREFIX}, it fails to find any of the headers.
Running make VERBOSE=1 reveals that g++ has no -I argument to include the ~/mamba_envs/vira_env/include/ directory. Which makes sense, as it is failing to find the headers that are there.
However, if I use include_directories(${CMAKE_INSTALL_PREFIX}) and rerun make VERBOSE=1, or it clearly shows that g++ has a -I /home/cgnam/mamba_envs/vira_env/.
Similarly, if I use include_directories(${CMAKE_INSTALL_PREFIX}/include/oneapi) it shows g++ has a -I /home/cgnam/mamba_envs/vira_env/include/oneapi. (oneapi is the directory where the tbb headers are placed).
So include_directories() will happily use either the conda environment root, or even one of the directories inside the conda environment's include/ directory... but it will just completely ignore the include/ directory itself.
EDIT
As discovered in one of my comments below, printing CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES shows /home/cgnam/mamba_envs/vira_env/include. Which explains why my include_directories() is not having any effect on the g++ call by make, since it believes that the compiler should be looking in that directory already.
make VERBOSE=1is showing that the compiler being used is: /home/cgnam/mamba_envs/vira_env/bin/x86_64-conda-linux-gnu-c++. However using which g++ is showing ~/mamba_envs/vira_env/bin/g++. Further, which ld returns: ~/mamba_envs/vira_env/bin/ld.
If I use ld --verbose | grep SEARCH_DIR, it does not show /home/cgnam/mamba_envs/vira_env/include. Neither does ~/mamba_envs/vira_env/bin/x86_64-conda-linux-gnu-ld --verbose | grep SEARCH_DIR.

Clang toolchain fails for system libc files

I wrote a custom (cc_)toolchain for Clang. After some back and forth I now have it mostly working. It is however failing with:
undeclared inclusion(s) in rule '//foo:foo'
this rule is missing dependency declarations for the following files included by 'foo/foobar.cpp':
'/usr/include/features.h'
'/usr/include/stdc-predef.h'
'/usr/include/x86_64-linux-gnu/sys/cdefs.h'
'/usr/include/x86_64-linux-gnu/bits/wordsize.h'
'/usr/include/x86_64-linux-gnu/gnu/stubs.h'
'/usr/include/x86_64-linux-gnu/gnu/stubs-64.h'
'/usr/include/stdint.h'
'/usr/include/x86_64-linux-gnu/bits/wchar.h'
'/usr/include/stdio.h'
'/usr/include/x86_64-linux-gnu/bits/types.h'
'/usr/include/x86_64-linux-gnu/bits/typesizes.h'
'/usr/include/libio.h'
'/usr/include/_G_config.h'
'/usr/include/wchar.h'
'/usr/include/x86_64-linux-gnu/bits/stdio_lim.h'
'/usr/include/x86_64-linux-gnu/bits/sys_errlist.h'
'/usr/include/math.h'
'/usr/include/x86_64-linux-gnu/bits/math-vector.h'
'/usr/include/x86_64-linux-gnu/bits/libm-simd-decl-stubs.h'
'/usr/include/x86_64-linux-gnu/bits/huge_val.h'
'/usr/include/x86_64-linux-gnu/bits/huge_valf.h'
'/usr/include/x86_64-linux-gnu/bits/huge_vall.h'
'/usr/include/x86_64-linux-gnu/bits/inf.h'
'/usr/include/x86_64-linux-gnu/bits/nan.h'
'/usr/include/x86_64-linux-gnu/bits/mathdef.h'
'/usr/include/x86_64-linux-gnu/bits/mathcalls.h'
'/usr/include/assert.h'
'/usr/include/string.h'
'/usr/include/xlocale.h'
So how is one supposed to handle these files? Simply hardcode these for the toolchain? Or is there a switch to ignore dependencies from the system (/usr)? Or rather use a hermetic libc like musl, newlib or bionic.
EDIT:
For more details about the actual toolchain see: https://github.com/abergmeier/bazel_toolchains
You need to put the builtin include paths of the compiler into the cxx_builtin_include_directory field of the crosstool proto, so Bazel knows to ignore them during include dependency validation. You can see the builtin include paths of a gcc-compatible compiler by running something like
$ $CC -x c++ -v - -c -o /dev/null < /dev/null
and looking for the output like
#include "..." search starts here:
#include <...> search starts here:
/usr/include/c++/7
/usr/include/x86_64-linux-gnu/c++/7
/usr/include/c++/7/backward
/usr/lib/gcc/x86_64-linux-gnu/7/include
/usr/local/include
/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed
/usr/include/x86_64-linux-gnu
/usr/include

Issues in c++ compilation for embedded system

I have a few related questions about my issues with compilation for embedded system. My questions are not only about HOW to do something, but more about WHY, because I have solutions for my problems (but maybe there are better ones?), but have no idea why some things works in some conditions, and does not work in others. I already spent some time with this, but until yesterday I was doing things a little blindly, with trials and errors, and without knowing what I was doing. Time to stop that! Please, help.
Scenario
I want to develop an application for Xilinx’s Zynq ARM processor, on Zedboard. The app will involve multithreading, some audio manipulation, and httpserver. So I will need pthread, alsa, sndfile and microhttpd libraries. I created rootfs with yocto. In original conf.local file I added/modified these lines:
BB_NUMBER_THREADS ?= "${#oe.utils.cpu_count()}"
PARALLEL_MAKE ?= "-j ${#oe.utils.cpu_count()}"
MACHINE ?= "zedboard-zynq7"
PACKAGE_CLASSES ?= "package_deb"
EXTRA_IMAGE_FEATURES = "debug-tweaks eclipse-debug"
IMAGE_INSTALL_append = "libgcc alsa-utils mpg123 libstdc++ sthttpd libmicrohttpd libsndfile1"
LICENSE_FLAGS_WHITELIST = "commercial_mpg123"
I also had to add some additional layers to bblayers.conf (and of course downloaded them):
meta-xilinx
meta-multimedia (from meta-openembedded)
meta-oe (from meta-openembedded)
meta-webserver (from meta-openembedded)
Lastly, I generated core-image-minimal with bitbake.
This, together with Linux kernel, and other stuff compiled separately, boots and works fine.
Problems
1. Simple app with this rootfs
It is app for Zynq, so I use XSDK, which is SDK from Xilinx, based on Eclipse. I created new Application project. In dialog window I chose Linux as platform, C++ as language, and I provided path to my unpacked rootfs (excactly the one that system boots with, via NFS). My rootfs path is /home/stas/ZedboardPetalinuxFS (it is not Petalinux, I just used to use it, and this folder name is still the same). This sets proper paths for library and headers search in rootfs.
I started with something very simple:
#include <pthread.h>
int main()
{
int i;
i = 1;
return 0;
}
I also added pthread library for linker (in Eclipse settings). Linking command at this point:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" -o "test.elf" ./src/main.o -lpthread
At this point it compiles. But it stops, when I add sndfile library
#include <sndfile.h>
This is reasonable, because this rootfs does not have all headers. I need to add another path for searching for headers. So I added path in yocto tmp folder, that has all the headers, that was needed for building rootfs. After I add it, it compiles again successfully. But problems started, when I added sndfile library for linking. Here is linking command and error:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" -o "test.elf" ./src/main.o -lpthread -lsndfile
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lsndfile
I looked to usr/lib to check if libsndfile.so is there, and I found only libsndfile.so.1 and ibsndfile.so.1.27. But it is also the case for pthread, and linker does not complain for that. I decided to create libsndfile.so by hand (I linked it to libsndfile.so.1). Linker stopped complaining about it, but started complaining about it’s dependencies. So I also creaded .so files for all the dependencies, and their dependencies, and added them for linking. Then it succeeded. At the end, linking command looked like this:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" -o "test.elf" ./src/main.o -lpthread -lvorbisenc -lvorbis -logg -lFLAC -lsndfile
So here goes the first question – why I did not needed .so file for pthread, but needed it for all other libraries? Or more general – when do I need .so file, and when .so.X file is enough?
2. Simple app - another approach
After the first try, I thought I should make another image, this time more suitable for development. Luckily, in Yocto it is quite easy – I just had to modify one line:
EXTRA_IMAGE_FEATURES = "debug-tweaks eclipse-debug dev-pkgs"
dev-pkgs option adds -dev packages for all installed packages.
So now I have rootfs with all needed headers, and .so files pointing where they should.
Before compilation, I removed unnecessary Include path, leaving only the one from rootfs, and removed all the libraries, except pthread, and sndfile. But then I get new errors:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" -o "test.elf" ./src/main.o -lsndfile -lpthread
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find /lib/libpthread.so.0
makefile:48: polecenia dla obiektu 'test.elf' nie powiodły się (commands for ‘test.elf’ did not succeed)
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find /usr/lib/libpthread_nonshared.a
I spotted, that it looks for libraries in my root folder. Quick search in Google (and SO:)) told me that I should set –-sysroot variable. So I added it to Eclipse option (in Miscelenious card in Linker options) like that:
--sysroot=/home/stas/ZedboardPetalinuxFS
So now linker command looked like this:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" --sysroot=/home/stas/ZedboardPetalinuxFS -o "test.elf" ./src/main.o -lsndfile -lpthread
And all succeed! I also wrote simple example that uses pthreads, and sndfile, and it also worked. But WHY? This leads me to second question:
Why do I need --sysroot option in this case? When do I need to use this option in general? And why this time I didn't have to add all the dependencies to linking command?
3. Another idea
At this point, I had an idea, to check what will happen, if I add --sysroot option having rootfs populated with old, non development image. But this gave me new errors:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" --sysroot=/home/stas/ZedboardPetalinuxFS -o "test.elf" ./src/main.o -lpthread -lvorbisenc -lvorbis -logg -lFLAC -lsndfile
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find crt1.o: No such file or directory
makefile:48: polecenia dla obiektu 'test.elf' nie powiodły się
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find crti.o: No such file or directory
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lpthread
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lm
So third question – what does this errors mean?
Thanks very much in advance!
"why I did not needed .so file for pthread, but needed it for all
other libraries?"
Actually you do need pthread.so file. You included pthread.h but didn't link with -lpthread. So it's normal you don't see any linker errors.
"when do I need .so file, and when .so.X file is enough"
When you give "-lNAME" parameter to g++, the compiler tells the linker to find libNAME.so within library search paths. Since there may exist multiple versions of the same library(libNAME.so.1, libNAME.so.1.20), *.so files link to desired actual library file. (Versioning of shared objects, ld man pages)
"Why do I need --sysroot option in this case? When do I need to use this option in general? And why this time I didn't have to add all the dependencies to linking command?"
The "dev-pkgs" in EXTRA_IMAGE_FEATURES changes your sysroot implicitly to let you link against the dev packages(yoctoproject image-features). That's why you need -sysroot option. You generally need this option when cross compiling to provide a root for standard search paths for headers and libraries. You didn't need it because you didn't have dev-pkgs image feature that changes your sysroot
"So third question – what does this errors mean?"
Even your the most basic hello world code gets linked with standard c library(if you didn't specify otherwise). libm.so, libpthread.so and crt1.o files are parts of libc library and come with libc dev package. So the linker can't see the standard library directories when it looks from your old sysroot
why I did not needed .so file for pthread, but needed it for all other libraries?
A cross compiler will normally come with a C Runtime (including pthread), typically in a directory that is part of the cross compiler installation.
The linker has built in search paths for libraries. These are in respect to the sysroot, which would by default be set to search the cross compiler's own included target C Runtime. If you added any -L options it would search those first and then move on to these pre-defined directories.
When you linked against pthread it would have found at least libpthread.a in the cross compiler's library directory.
Or more general – when do I need .so file, and when .so.X file is enough?
Shared libraries in Linux typically have a major and a minor version number. Libraries are ABI compatible between different minor versions with the same major version, but not between major versions. Sometimes there are three levels of versions but the principal is similar.
When installing libraries it is common to install the actual file with the full name, eg. libmy.so.1.2, then provide symlinks to libmy.so.1 and libmy.so.
If you are linking an application can work with any library version then you would just specify the name, eg. -lmy. In that case you would need symlinks from libmy.so to libmy.so.1.
If you required a specific version you would put -l:libmy.so.1. The ':' indicates a literal file name.
Linker scripts may affect things and may result in specific versions being selected even when you do specify the short name.
Why do I need --sysroot option in this case? When do I need to use
this option in general?
What --sysroot does is prepend the given path onto all the search directories which would normally be used to search for includes and libraries. It is most useful when cross compiling (as you are doing now) to get the compiler and the linker to search inside the target root instead of the build host's own root.
If you have specified a sysroot you probably do not need to specify include paths via -I or linker paths via -L, assuming that the files are within their normal spots inside your target root.
And why this time I didn't have to add all the dependencies to linking command?
One possible scenario is that the first time, sndfile for statically rather than dynamically linked. This would happen if your first root image had only sndfile.a in the lib dir, or elsewhere on the search path. To then satisfy the requirements of sndfile.a you would also need to link the other libs.
When linking against sndfile.so the dependencies will automatically get loaded via the dynamic linking process.
That's just a working theory at present.
So third question – what does this errors mean?
They mean it cannot find even the C runtime library to link.
As described for the first question, it was previously finding the C runtime in the pre-defined search path (relative to the predefined sysroot) which located the C runtime supplied by the cross compiler.
You disturbed this by supplying your own sysroot. It was now only searching the target root. Since this target root filesystem did not have development libs installed, there was no C runtime there to find.
You are doing several things wrong:
looks like you are not using environment variables, but calling cross-compiler directly. So, instead of compiling with arm-linux-gnueabihf-g++ ..., you should do $CXX .... The CXX is the environment variable set by the yocto script to set environment for cross compilation. Using CXX, you do not need to manually pass --sysroot
You should not link directly to pthread library with -lpthread. You should use -pthread

Modifying the "priorities" between include directories within a Makefile

To make it simple, I want to use the OpenCV libs within my embedded ElinOS system, while programming on my Windows machine.
To do so, I downloaded OpenCV sources for Linux on the website and I'm trying to compile it using cygwin and cmake to generate the Makefile. However, I have an error occuring during the "make" step, which is /usr/include/wchar.h:41:25: fatal error : bits/wchar.h : No such file or directory
I understand the problem, which is basically that cygwin should use /opt/elinos-6.0/cdk/x86/x86_64/eglibc-2.17/x86_64-unkown-linux-gnu/include instead of /usr/include but I have basically no idea how to change that.
That far, I added those two lines to the CMakeLists.txt
set(CMAKE_INCLUDE_DIRECTORIES_BEFORE ON)
include_directories(BEFORE C:/sysgo/opt/elinos-6.0/cdk/x86/x86_64/eglibc-2.17/x86_64-unkown-linux-gnu/include/)
and this line to the builded Makefile:
INC = -I/cygdrive/c/sysgo/opt/elinos-6.0/cdk/x86/x86_64/eglibc-2.17/x86_64-unknown-linux-gnu/include/
But it didn't work, I still got the same error. Any help is welcome, and I would be glad if an explanation comes with the magic command line(s) that will help.
Thanks in advance!
one way for you might be to clear the standard include path completly:
first delete standard path with -nostdinc and then put your own directories in it.
you might not want loose all of them, here is a way to see which are used in the standard include path: echo "//" | gcc -xc++ -E -v - (works also with clang)
Here is a fine article for this for gcc :
http://commandlinefanatic.com/cgi-bin/showarticle.cgi?article=art026
here you can read it works with clang too:
http://clang.llvm.org/docs/CommandGuide/clang.html

makefile no such file or directory: libxml/xmlstring.h

I'm not used to making makefiles. My old project had all the makefiles in place and I'm making this from scratch for a small test with a lot of canned lib dependencies that I'm not used to either. It's been a few years since I looked at makefiles.
I have a directory called libopcTest with the following in it:
LibOPCTest.cc
LibOPCTest.h
makefile
makefile has this in it:
INC=-I/usr/include/libxml2/libxml
LIB=-L/usr/include/libxml2/libxml
all: LibOPCTest.exe
LibOPCTest.exe: LibOPCTest.o
>tab> gcc -o LibOPCTest.exe LibOPCTest.o
LibOPCTest.o: LibOPCTest.cpp
>tab> gcc -c $(INC) $(LIB) LibOPCTest.cpp
clean:
>tab> rm LibOPCTest.o LibOPCTest.exe
I looked in /usr/include/libxml2/libxml and it does have xmlstring.h in it. I don't see the libxml reference in opc.h, but apparently that's where it comes in, presumably in an include file, like config.h.
Plus, we have LibOPCTest.cpp which #includes <opc/opc.h> and it's own .h file.
main is in LibOPCTest.cpp.
When I type make at the Linux command prompt, I get the following error:
In file included from /usr/include/opc/opc.h:36:0, from LibOPCTest.cpp:1:
/usr/include/opc/config.h:37:30: fatal error: libxml/xmlstring.h: no such file or directory.
Shouldn't I have the libxml with the LIB and INC definition in the makefile pointing to libxml? I don't think I'm supposed to add anything to opc.h, including build it, since it's a canned library.
I was looking at this makefile example, and I think I have everything I need (probably not since it's not building).
I know it's a basic question, but hopefully someone has a good suggestion. Thanks for being nice in advance!!
We decided to put the Linux version of libOPC on hold because there isn't a good 64 bit version and it needs a lot of work.