I wrote a custom (cc_)toolchain for Clang. After some back and forth I now have it mostly working. It is however failing with:
undeclared inclusion(s) in rule '//foo:foo'
this rule is missing dependency declarations for the following files included by 'foo/foobar.cpp':
'/usr/include/features.h'
'/usr/include/stdc-predef.h'
'/usr/include/x86_64-linux-gnu/sys/cdefs.h'
'/usr/include/x86_64-linux-gnu/bits/wordsize.h'
'/usr/include/x86_64-linux-gnu/gnu/stubs.h'
'/usr/include/x86_64-linux-gnu/gnu/stubs-64.h'
'/usr/include/stdint.h'
'/usr/include/x86_64-linux-gnu/bits/wchar.h'
'/usr/include/stdio.h'
'/usr/include/x86_64-linux-gnu/bits/types.h'
'/usr/include/x86_64-linux-gnu/bits/typesizes.h'
'/usr/include/libio.h'
'/usr/include/_G_config.h'
'/usr/include/wchar.h'
'/usr/include/x86_64-linux-gnu/bits/stdio_lim.h'
'/usr/include/x86_64-linux-gnu/bits/sys_errlist.h'
'/usr/include/math.h'
'/usr/include/x86_64-linux-gnu/bits/math-vector.h'
'/usr/include/x86_64-linux-gnu/bits/libm-simd-decl-stubs.h'
'/usr/include/x86_64-linux-gnu/bits/huge_val.h'
'/usr/include/x86_64-linux-gnu/bits/huge_valf.h'
'/usr/include/x86_64-linux-gnu/bits/huge_vall.h'
'/usr/include/x86_64-linux-gnu/bits/inf.h'
'/usr/include/x86_64-linux-gnu/bits/nan.h'
'/usr/include/x86_64-linux-gnu/bits/mathdef.h'
'/usr/include/x86_64-linux-gnu/bits/mathcalls.h'
'/usr/include/assert.h'
'/usr/include/string.h'
'/usr/include/xlocale.h'
So how is one supposed to handle these files? Simply hardcode these for the toolchain? Or is there a switch to ignore dependencies from the system (/usr)? Or rather use a hermetic libc like musl, newlib or bionic.
EDIT:
For more details about the actual toolchain see: https://github.com/abergmeier/bazel_toolchains
You need to put the builtin include paths of the compiler into the cxx_builtin_include_directory field of the crosstool proto, so Bazel knows to ignore them during include dependency validation. You can see the builtin include paths of a gcc-compatible compiler by running something like
$ $CC -x c++ -v - -c -o /dev/null < /dev/null
and looking for the output like
#include "..." search starts here:
#include <...> search starts here:
/usr/include/c++/7
/usr/include/x86_64-linux-gnu/c++/7
/usr/include/c++/7/backward
/usr/lib/gcc/x86_64-linux-gnu/7/include
/usr/local/include
/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed
/usr/include/x86_64-linux-gnu
/usr/include
Related
I have just installed hidapi in my Ubuntu 20.04 following the instructions, i.e. by doing
sudo apt install libhidapi-dev
I wrote my program in the file mwe.cpp which contains only this line:
#include <hidapi.h>
and now I want to compile it with
g++ -o mwe.o mwe.cpp
but I get
mwe.cpp:1:10: fatal error: hidapi.h: No such file or directory
How am I supposed to use this module? Sorry for such a basic question but cannot find out.
On Ubuntu based systems, the system package libhidapi-dev installs the include files to /usr/include/hidapi, so either include this (-I/usr/include/hidapi) in your command line or #include <hidapi/hidapi.h>
If that header is not within the standard search path for headers, then you can include it manually with the -I flag e.g.
g++ -I/usr/include/hidapi -o mwe.o mwe.cpp
To locate a file on ubuntu, you can run:
sudo updatedb
locate hidapi.h
> /usr/include/hidapi/hidapi.h
You can view the standard include search path with:
gcc -print-search-dirs
Alternatively, because /usr/include is on the standard search path, you can write your include as <hidapi/hidapi.h>.
How to #include <whatever.h> installed with apt in Ubuntu
If the package installs the header within a directory included in the default search path of your compiler - which is typical - then you can include the header using a relative path from the root of the search path where the header is installed. For example, if the file is in the path /usr/include/x/y.h, then you can include <x/y.h>.
If the package doesn't install the header within the default search path, then you must specify the search path for the compiler when you invoke it, and then include the header relative to the specified include directory. For example, if the file is in the path /opt/custom/x/y.h, then you can include <x/y.h> and specify /opt/custom as a search path for the compiler.
If you use GCC or compatible compiler, and the package supports it, then you can use a program called pkg-config to get the compiler options needed to use the library. Besides the header search path, this also takes care of linking with the library as well as any mandatory compiler options. Example:
pkg-config --libs --cflags libhidapi
I don't know, how do I find the location of hidapi.h?
There are several ways to find out the location of a file. A general tool is the program find. Example:
find / -name=hidapi.h
A more specific tool for learning the paths of files installed by an apt package is apt-file. Alternatively, you can look up the list of files in the https://packages.ubuntu.com/ website.
I'm trying to run this test code from the SDL website but I dont understand where to put the downloaded files and how to reference them using the g++ compiler. I've been trying to using the -I command but I dont quite understand that either. I keep getting the "fatal error: SDL.h: No such file or directory #include<SDL.h> "
Using windows and sublime text editor and g++ compiler
Directly from the g++ manual:
"-I dir
Add the directory dir to the list of directories to be searched for header files. Directories named by -I are searched before the standard system include directories. If the directory dir is a standard system include directory, the option is ignored to ensure that the default search order for system directories and the special treatment of system headers are not defeated . If dir begins with "=", then the "=" will be replaced by the sysroot prefix; see --sysroot and -isysroot."
As for your error, I would try #include <SDL2/SDL.h>. This could vary based on the version you are using. If that doesn't work I would just make sure you have the correct path when including in the compiler as a flag.
I have a c++ application that is separated into modules. The directory structure for each module looks like this:
module_a/
export/ <-- module_a public interface
src/ <-- module_a private source
test/
src/ <-- unittests for module_a
I'm using CMake to setup the module dependencies and to build the application. If module_a depends on module_b then module_b's export directory is added as include path when building module_a.
So, if there is a file module_b/export/foo.h that module_a needs then in a source file you will use #include "foo.h".
I'm looking for a way to to make so that the module name is part of the include directive. So for the above example I want to (have to) write #include "module_b/foo.h".
Is this something that can be done with GCC option -iprefix and -iwithprefix? I've searched for usage examples but all I can find is copies and references back to the GCC manual, which I think doesn't explain it very well.
I have tried to use it this way:
$ find -type f
./src
./src/main.cc
./export
./export/bar.h
$ g++ -iprefix foo/ -iwithprefix export/ src/main.cc
src/main.cc:1:10: fatal error: foo/bar.h: No such file or directory
1 | #include "foo/bar.h"
| ^~~~~~~~~~~
compilation terminated.
$ gcc --version
gcc (Debian 9.2.1-17) 9.2.1 20191102
But as you can see, it doesn't work. How should I use -iprefix and -iwithprefix?
Also, does anyone have another solution to my problem? I'm a bit worried that IDEs and other compilers might not understand -iprefix and -iwithprefix, so any other solutions are welcome as well.
Edit: after posting this question I immediately realized that perhaps -iprefix foo/ -iwithprefix bar is just a fancy way of writing -I foo/bar. However, I tested that and I still didn't get it to work. So it would still be good if someone could explain how these options works, even if they are not going to help me for my problem.
How should I use -iprefix and -iwithprefix?
It's just a shortcut to avoid having to write out a common prefix every time. So instead of:
cpp -Idirafter /sys/root/a -Idirafter /sys/root/b
You instead do:
cpp -iprefix /sys/root/ -iwithprefix a -withprefix b
(substitute -Iwithprefix for -Iwithprefixbefore if you want it to be equivalent to -I instead.)
This can come on handy with very long include directory paths like you may run into when cross-compiling (and may be even required, cf. getconf ARG_MAX)
Also, does anyone have another solution to my problem?
I stumbled upon -iprefix, because I've the same problem you have as well and hoped there would be an out-of-the-box option for that. There isn't, but Linux device tree builds have the same issue:
The device trees for ARCH=arm and ARCH=arm64 are in arch/arm/boot/dts and arch/arm64/boot/dts, respectively. Sometimes, arm64 device trees need some common file from the arm directory, but having the dts directories of all architectures in path isn't that great a solution.
What Linux does instead, is having a dedicated directory in the source tree with suitably-named symlinks into each architecture and this directory's path as well as the directory of the currently active ARCH go into the search path. That way, ARCH=arm can write #include "something.dtsi", but ARCH=arm64 has to write #include <arm/something.dtsi> instead.
As we all know,gcc will search some default dir when compiling targets,
I use gcc -print-search-dirs commnd and get this stuff:
install: /usr/lib/gcc/x86_64-redhat-linux/4.8.5/
programs: =/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/:/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/bin/x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/bin/
libraries: =/usr/lib/gcc/x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/lib/x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/:/lib/x86_64-redhat-linux/4.8.5/:/lib/../lib64/:/usr/lib/x86_64-redhat-linux/4.8.5/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/lib/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../:/lib/:/usr/lib/
I have two questions about this:
What does /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/bin/ mean, and why write like that?
Why /usr/include is not there?Doesn't gcc need to know where headfile is?
So as to question 1, I found this in the gcc mailing list:
Note that a/b/c/../../../x will only exist if a/b/c exists.
So your gcc will only search in /usr/x86_64-redhat-linux/bin/ if /usr/lib/gcc/x86_64-redhat-linux/4.8.5/ exists.
Note that I've got similar search-dirs on arch, and the /usr/x86_64-redhat-linux/-equivalent ones don't exist, so I'm not sure WHY they are even included, but Linux has a notoriously unstandardized directory structure, so perhaps it's important for other distros.
Regarding 2, from the man page:
-print-search-dirs
Print the name of the configured installation directory and a list of program and library directories gcc searches---and don't do anything else.
This is useful when gcc prints the error message installation problem, cannot exec cpp0: No such file or directory. To resolve this you either need to put cpp0 and the other
compiler components where gcc expects to find them, or you can set the environment variable GCC_EXEC_PREFIX to the directory where you installed them. Don't forget the
trailing /.
This seems to print not the library paths, but rather the paths where gcc expects to find the components it calls to do it's work.
To find the paths the preprocessor searches, type
`gcc -print-prog-name=cc1plus` -v
or
`gcc -print-prog-name=cc1` -v
for the C preprocessor.
My LIBRARY_PATH environment variable has a custom directory in it: /cs/public/lib/pkg/opencv/lib.
But, when I use g++ --print-search-dirs, I get this instead:
libraries: =
/cs/public/lib/pkg/opencv/lib/x86_64-suse-linux/4.6/:
/cs/public/lib/pkg/opencv/lib/../lib64/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/x86_64-suse-linux/4.6/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/../lib64/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../x86_64-suse-linux/4.6/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/:
/lib/x86_64-suse-linux/4.6/:
/lib/../lib64/:
/usr/lib/x86_64-suse-linux/4.6/:
/usr/lib/../lib64/:
/cs/public/lib/pkg/opencv/lib/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../:
/lib/:
/usr/lib/
Why does g++ look in these alternatives and a whole bunch of other system locations before what I explicitly specify in the LIBRARY_PATH variable, and where is this documented?
I would understand if system defaults were searched before LIBRARY_PATH and LIBRARY_PATH/../lib64, etc, but g++ puts LIBRARY_PATH/../lib64, then system paths, then LIBRARY_PATH. Where is this ordering documented?
My g++ version is g++ (SUSE Linux) 4.6.2
My OS is openSUSE 12.1 (x86_64)
A similar question was asked here:
g++ searches /lib/../lib/, then /lib/
These scary-looking search paths are determined at least in part when the compiler itself it built, for example during the configure phase. It's clear that it goes beyond environment variables because it's possible to have multiple copies of GCC installed and have each of them give different results for gcc --print-search-dirs. Also noting that g++ --print-search-dirs and gcc --print-search-dirs give different results points out that the g++ wrapper is also affecting the search path. Besides configure/build time differences, GCC is definitely aware of the path where its own executable is, and will search subdirectories of that path. A lot of this alchemy can be found in the GCC documentation:
http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/Directory-Options.html#Directory-Options
http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/Environment-Variables.html#Environment-Variables
As far as I know, the most forceful thing that you can without compiling your own copy of GCC is to specify your custom libraries using the -L option. The reason I say this is because -L is searched before e.g. LIBRARY_PATH (see the above link on environment variables). In order to make it more tolerable you could add an alias for g++ including the -L option in your .bashrc file.
If you want a definitive answer then downloading a copy of the GCC source code is one way. For example, in gcc.c the following highly suggestive comment appears:
/* Build a list of search directories from PATHS.
PREFIX is a string to prepend to the list.
If CHECK_DIR_P is true we ensure the directory exists.
If DO_MULTI is true, multilib paths are output first, then
non-multilib paths.
This is used mostly by putenv_from_prefixes so we use `collect_obstack'.
It is also used by the --print-search-dirs flag. */
However the function that follows the comment is not very obvious.
This is multilib at work - a mechanism which allows for having libraries (but also the whole compilation and build toolchain) for multiple architectures on a single machine. This Wiki states that "The multilib suffix is appended to all directories searched for libraries by GCC and passed via -L options to the linker. The linker itself does not have any particular knowledge of multilibs, and will continue to consult its default search directories if a library is not found in the -L paths. If multiple orthogonal ABI-changing options are used in a single compilation, multiple multilib suffixes can be used in series.".
So, according to the above description, the architecture marker string or different variants thereof are appended to each library search path the compiler receives since it doesn't differentiate between default and custom paths. Your custom path is first in the row, but it undergoes the same "expansion" process as other paths.
Due to the need to handle i386 compatibility, multilib mechanisms seem to now be used by default on most x64 distros, which in practice means most of the installations out there.
I have the exact same problem on:
Fedora 17, gcc 4.7 and gcc 4.3
CentOS 6.3, gcc 4.4
Unubuntu 12, gcc 4.6
So it looks like this is a problem with most gcc versions. Probably this strange behavior first appeared in gcc 4.2 at least according to this.
I tried duping the specs and playing with them. It looks like the *multilib spec is used to append specific strings depending on the platform. For example my original spacs looked like:
*multilib:
. !m64 !m32;64:../lib64 m64 !m32;32:../lib !m64 m32;
When I changed 64:../lib64 to 64:../lib then instead of ../lib64 gcc appended ../lib. But I couldn't fully decipher the meaning of *multilib or any of the other specs.
This answer attempts to summarize the search path behaviors for both GCC and Clang.
GCC
Include path: for a command-line of the following format:
CPLUS_INCLUDE_PATH=EDIR g++ -IIDIR -isystemSDIR
the following list of directories are used as search paths for #include <...>:
IDIR # '-I' directories.
SDIR # '-isystem' directories.
EDIR # *_INCLUDE_PATH directories.
GCCDIR/include/c++/GCCVER # libstdc++ directory (C++).
GCCDIR/include/c++/GCCVER/GCCARCH # libstdc++ directory (C++).
GCCDIR/include/c++/GCCVER/backward # libstdc++ directory (C++).
GCCDIR/lib/gcc/GCCARCH/GCCVER/include # GCC arch-specific directory.
/usr/local/include/GCCARCH # Local arch-specific include directory.
/usr/local/include # Local include directory.
GCCDIR/include # GCC include directory.
GCCDIR/lib/gcc/GCCARCH/GCCVER/include-fixed # GCC include-fixed directory.
/usr/include/GCCARCH # System include arch-specific directory.
/usr/include # System include directory.
Library path: for a command-line of the following format:
LIBRARY_PATH=EDIR gcc -BBDIR -LLDIR
the following arguments are passed to the linker:
-LLDIR # '-L' directories.
-LBDIR # '-B' directories.
-LEDIR/../libXX # Multilib directories from LIBRARY_PATH.
-LGCCDIR/lib/gcc/GCCARCH/GCCVER # GCC arch-specific library directory.
-LGCCDIR/libXX # GCC multilib library directory.
-L/libXX # System multilib library directory.
-L/usr/libXX # System multilib library directory.
-LEDIR # LIBRARY_PATH directories.
-LGCCDIR/lib # Other GCC libraries.
Clang
Include path: for a command-line of the following format:
CPLUS_INCLUDE_PATH=EDIR clang++ --gcc-toolchain=GCCDIR -BBDIR -IIDIR -isystemSDIR
the following list of directories are used as search paths for #include <...>:
IDIR # '-I' directories.
SDIR # '-isystem' directories.
EDIR # *_INCLUDE_PATH directories.
# If -stdlib=libstdc++ is used:
GCCDIR/include/c++/GCCVER # libstdc++ directory from the selected GCC toolchain (C++).
GCCDIR/include/c++/GCCVER/GCCARCH # libstdc++ directory from the selected GCC toolchain (C++).
GCCDIR/include/c++/GCCVER/backward # libstdc++ directory from the selected GCC toolchain (C++).
# If -stdlib=libc++ is used:
CLANGDIR/include/c++/v1 # libc++ directory (C++).
/usr/local/include # Local include directory.
CLANGDIR/lib/clang/CLANGVER/include # Clang include directory.
/include # System include directory.
/usr/include # System include directory.
Library path: for a command-line of the following format:
LIBRARY_PATH=EDIR clang --gcc-toolchain=GCCDIR -BBDIR -LLDIR
the following arguments are passed to the linker:
-LLDIR # '-L' directories.
-LGCCDIR/lib/gcc/GCCARCH/GCCVER # GCC arch-specific library directory.
-LGCCDIR/libXX # GCC multilib library directory.
-L/libXX # System multilib library directory.
-L/usr/libXX # System multilib library directory.
-LGCCDIR/lib # Other GCC libraries.
-LCLANGDIR/lib # Clang libraries.
-L/lib # System library directory.
-L/usr/lib # System library directory.
-LEDIR # LIBRARY_PATH directories.
Summary
The search path for includes are pretty much the same in both GCC and Clang. C++-specific paths are omitted if the C frontend is used in both cases. Library search paths differ substantially between GCC and Clang, notably the presence of -B directories and the odd manipulation of LIBRARY_PATH in the GCC frontend.
The library search paths are the same for both C and C++ frontends. Other library search paths are introduced by the linker itself. The following excerpt comes from the vanilla linker script for GNU Binutils:
# Multilib library directories.
SEARCH_DIR("BINUTILSDIR/BINUTILSARCH/libXX");
SEARCH_DIR("BINUTILSDIR/libXX");
SEARCH_DIR("/usr/local/libXX");
SEARCH_DIR("/libXX");
SEARCH_DIR("/usr/libXX");
# Traditional library directories.
SEARCH_DIR("BINUTILSDIR/BINUTILSARCH/lib");
SEARCH_DIR("BINUTILSDIR/lib");
SEARCH_DIR("/usr/local/lib");
SEARCH_DIR("/lib");
SEARCH_DIR("/usr/lib");
It is also imperative to note that library dependencies are not searched for inside the directories listed above. These exclusively rely on -rpath and -rpath-link options passed to the linker, or else they get resolved from the default system library paths. Thus it might be useful to produce both -L and -rpath-link arguments in order to guarantee that the correct libraries are linked.
Finally, special files (such as the CRT objects) are searched for only in -B directories. In Clang, special files are also searched inside the selected GCC toolchain. Other factors (spec files, distribution-specific configuration) might change some or all of the above.
Looks like it's needed for cross-compiling. From the ChangeLog:
Wed Mar 29 14:53:23 1995 Jim Wilson <wilson#cygnus.com>
* gcc.c (process_command): Delete code modifying gcc_exec_prefix.
(main): Put it here after last use of gcc_exec_prefix. For cross
compiler, set startfile_prefixes if gcc_exec_prefix is set and
standard_startfile_prefix is a relative path.
startfile_prefixes is what is being printed out with the search-dirs flags. From gcc/gcc.c:
if (print_search_dirs)
{
printf (_("install: %s%s\n"), standard_exec_prefix, machine_suffix);
printf (_("programs: %s\n"), build_search_list (&exec_prefixes, "", 0));
printf (_("libraries: %s\n"), build_search_list (&startfile_prefixes, "", 0));
return (0);
}
The compiler will first look at the default paths then for the others.
How does it sorted when you print I don't now, but it is documented here, 3.19 Environment Variables Affecting GCC.
The paths are defined by the built-in specs. Specs define how the pipeline processes the source code to obtain a result. GCC just drives the compilation.
You can give GCC your own spec file via -spec=, and you can get the built-in specs with -dumpspecs IIRC.
This is probably explained somewhere in the GCC manual.