How to force clang to use some library by default? - c++

I build clang by clang against libc++, libc++abi, compiler-rt in the following steps:
To download (and update) llvm and sub-projects I use the following script:
svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
cd llvm/tools
svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
svn co http://llvm.org/svn/llvm-project/clang-tools-extra/trunk clang/tools/extra
svn co http://llvm.org/svn/llvm-project/lldb/trunk lldb
svn co http://llvm.org/svn/llvm-project/lld/trunk lld
svn co http://llvm.org/svn/llvm-project/polly/trunk polly
cd ../projects/
svn co http://llvm.org/svn/llvm-project/compiler-rt/trunk compiler-rt
svn co http://llvm.org/svn/llvm-project/libunwind/trunk libunwind
svn co http://llvm.org/svn/llvm-project/openmp/trunk openmp
svn co http://llvm.org/svn/llvm-project/libcxxabi/trunk libcxxabi
svn co http://llvm.org/svn/llvm-project/libcxx/trunk libcxx
First of all I build llvm, clang, libunwind against libgcc and libstdc++ using gcc and install them in /usr/local. In all the following steps except the last one I use this fresh clang/clang++.
Then I separately build libunwind, both the 32-bit and 64-bit versions (they are essential, as will be further, because asan needs the 32-bit versions of some libraries during the final compilation of the whole project tree) and install it in /usr/local/lib and /usr/local/lib32 correspondingly (also I update LD_LIBRARY_PATH respectively).
Build libcxxrt ABI library both 32-bit and 64-bit versions and install them properly.
Build libcxx against libcxxrt both 32-bit and 64-bit versions and install them properly.
Then build libc++abi against libc++ both 32-bit and 64-bit versions and install them properly.
Then build libc++ against libc++abi both 32-bit and 64-bit versions and install them properly over the previous version linked to libcxxrt.
After all I build the whole project tree against libc++, libc++abi, libunwind, compiler-rt and install it over old clang in /usr/local.
(I am almost sure that no step is redundant here.)
At the final step I have a problem: I have to add extra parameters to the linker (I add them to CMAKE_EXE_LINKER_FLAGS and CMAKE_SHARED_LINKER_FLAGS) -lunwind and -lc++abi. Moreover, every time I use the resulting clang++ with -stdlib=libc++ and compiler-rt (or, equally, CLANG_DEFAULT_CXX_STDLIB=libc++ and CLANG_DEFAULT_RTLIB=compiler-rt) in my projects I have to do it over and over. It annoying. Say Qt Creator generated project's CMakeLists.txt should be corrected by hand or by cmake-gui.
How to make clang driver to automatically specify these options to ld at runtime? Is there something similar to RPATH mechanism? Or is there some specific CMake variable (specified before llvm building process) to achieve desired behaviour?
Can I use RPATH for my purposes?
Surely I don't want to make some bash-script like wrappers (similar to clang++-libc++) to specify additional parameters. I want the libraries to be hardcoded somewhere in the clang binary itself.

There are a couple of workarounds have been suggested. I ended up with the following workaround:
mkdir build
cd build
# backup:
cp -vaf /usr/local/lib/libc++.{a,so.1.0} /usr/local/lib/libc++abi.{a,so.1.0} /usr/local/lib/libunwind.{a,so.1.0} .
clang -shared -fPIC -pthread -o fuse.so -Wl,--whole-archive libc++.a libc++abi.a libunwind.a -Wl,--no-whole-archive -ldl -lm
ar x libc++.a
ar x libc++abi.a
ar x libunwind.a
ar rc fuse.a *.o
sudo chown root:root fuse.*
sudo cp -vaf fuse.so /usr/local/lib/
sudo ln -svf /usr/local/lib/libc++.so.1 /usr/local/lib/fuse.so
sudo cp -vaf fuse.a /usr/local/lib/
sudo mv -vf /usr/local/lib/libc++.a /usr/local/lib/libc++.a.bak
sudo ln -svf /usr/local/lib/libc++.a /usr/local/lib/fuse.a
It merges all the libraries used (libc++, libc++abi and libunwind) into the one single *.a or *.so file. Then libc++.a and libc++.so are replaced with (links to) resulting assemblied files, saving a previous versions for possible backup.
It works perfectly for me.
But this is not the answer. Maybe someday clang will not have such a problem right from the box.

Related

GCC & binutils build - C compiler cannot create executables

I'm trying to build gcc-5.3 and binutils-2.26. I've done it like this:
mkdir gcc; cd gcc
wget http://path/to/gcc-5.3.0.tar.bz2
wget http://path/to/binutils-2.26.tar.bz2
tar xf gcc-5.3.0.tar.bz2
tar xf binutils-2.26.tar.bz2
cd gcc-5.3.0
contrib/download_prerequisites
for file in ../binutils-2.26/*; do ln -s "${file}"; done
cd ..
mkdir build
mkdir dist
cd build
../gcc-5.3.0/configure --prefix=/home/teamcity/gcc/dist --disable-multilib --with-system-zlib --enable-languages=c,c++,fortran --program-suffix=-mine
make
This appears to build the first stage executables okay; prev-gas, prev-gcc, prev-ld are all present with plausible-looking executables in them. But the next stage fails:
Configuring stage 2 in ./intl
configure: loading cache ./config.cache
checking whether make sets $(MAKE)... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether NLS is requested... yes
checking for msgfmt... /usr/bin/msgfmt
checking for gmsgfmt... /usr/bin/msgfmt
checking for xgettext... /usr/bin/xgettext
checking for msgmerge... /usr/bin/msgmerge
checking for x86_64-unknown-linux-gnu-gcc... /home/teamcity/gcc/build/./prev-gcc/xgcc -B/home/teamcity/gcc/build/./prev-gcc/ -B/home/teamcity/gcc/dist/x86_64-unknown-linux-gnu/bin/ -B/home/teamcity/gcc/dist/x86_64-unknown-linux-gnu/bin/ -B/home/teamcity/gcc/dist/x86_64-unknown-linux-gnu/lib/ -isystem /home/teamcity/gcc/dist/x86_64-unknown-linux-gnu/include -isystem /home/teamcity/gcc/dist/x86_64-unknown-linux-gnu/sys-include -L/home/teamcity/gcc/build/./ld
checking for C compiler default output file name...
configure: error: in `/home/teamcity/gcc/build/intl':
configure: error: C compiler cannot create executables
See `config.log' for more details.
The relevant bit of config.log appears to be this:
configure:2978: checking for C compiler default output file name
configure:3000: /home/teamcity/gcc/build/./prev-gcc/xgcc -B/home/teamcity/gcc/build/./prev-gcc/ -B/home/teamcity/gcc/dist/x86_64-unknown-linux-gnu/bin/ -B/home/teamcity/gcc/dist/x86_64-unkn
own-linux-gnu/bin/ -B/home/teamcity/gcc/dist/x86_64-unknown-linux-gnu/lib/ -isystem /home/teamcity/gcc/dist/x86_64-unknown-linux-gnu/include -isystem /home/teamcity/gcc/dist/x86_64-unknown-l
inux-gnu/sys-include -L/home/teamcity/gcc/build/./ld -g -O2 -gtoggle -static-libstdc++ -static-libgcc conftest.c >&5
/home/teamcity/gcc/build/./prev-gcc/as: 106: exec: /home/teamcity/gcc/build/./gas/as-new: not found
This looks like prev-gcc's as is expecting to find gas/as-new, when actually it's prev-gas/as-new.
Is there some workaround for this? Is it safe to just ln -s prev-gas gas? I'd sort of expect that to cause problems later on. Is it not possible to build these two versions of gcc and binutils together?
This is my edited answer after trying it on my own on an Ubuntu 14.04 Azure VM.
The following approach worked for me:
Build and install binutils 2.26; for the purposes of this discussion,
let's say it's installed in /opt/gcc-5.3.0.
Configure gcc-5.3.0 in a build directory using /root/objdir/../gcc-5.3.0/configure --prefix=/opt/gcc-5.3.0
--enable-languages=c,c++ --disable-multilib --with-ld=/opt/gcc-5.3.0/bin/ld --with-as=/opt/gcc-5.3.0/bin/as assuming gcc-5.3.0 and the build directory, objdir, are at the same
level.
Do make followed by make install in the objdir build directory.
To verify that the ld used by the newly-built gcc is the one from the new binutils:
/opt/gcc-5.3.0/bin/gcc -print-prog-name=ld
The output should be, in this example:
/opt/gcc-5.3.0/bin/ld
Another test: rename the system ld, in my case /usr/bin/ld; the newly-built gcc should still work.
Of course, this applies to both ld and as.
Setting AS and LD environment variables to point to the newly-built binaries from the binutils package did not work: the -print-prog-name=... still showed default tools, and removing/renaming the default tools caused gcc to fail.
Thus the best way of accomplishing this is to build binutils first and then use the --with-ld and --with-as options to configure. If you also want to ensure that the newly-built binutils are used to build GCC, you may want to put them in the PATH before the system-provided tools, or you can even rename the system-provided tools to keep them out of the picture.
Thank you for checking with another mailing list to verify that building GCC and binutils together doesn't work unless they are pulled from the source control, I guess that option is not applicable when using downloaded tarballs. Overall this was an interesting exercise.

how to force compilation of Boost to use -fPIC

The team on which I work produces a shared library for use in Python. This library is entirely C++ and we use Boost to expose to python. Because we cannot guarantee that our clients have the Boost libraries installed, we pull in the functionality needed from Boost to the shared object file statically. The final stage in compilation will look familiar to many
g++ -o <output> <objects> -Wl,-Bstatic -lboost_python -lboost_regex ... -Wl,-Bdynamic -shared <other_opts>
We've traditionally used our own build of Boost: 1.47. This version is now quite old and so we wish to update. However, oddly, when I install the necessary objects using yum on my CentOS 7 system, I get the following error from gcc:
relocation R_X86_64_32 against '.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
Well, I thought I'd simply download the latest boost (CentOS 7 installs Boost 1.53) and do my own build. This, after all, has always worked for us. I follow the instructions here but I got the same error. How do I force the use of -fPIC for even the static libraries that it builds?
I believe boost automatically uses -fPIC when compiling a shared library (.so file), but the below command uses -fPIC when compiling a static library (.a file) too.
This worked for me on boost 1.46.1:
sudo ./bjam cxxflags=-fPIC cflags=-fPIC -a ... install
The ... is where you add additional flags like threading=multi or --layout=tagged, and optionally the list of projects to build (for example: --with-regex).
Note: using both cflags and cxxflags is unnecessary, only one is needed. See comments below.
Reference links:
https://cmake.org/Wiki/TubeTK/Build_Instructions#Boost_.28optional.29
http://lists.boost.org/boost-users/2010/07/60682.php
Just for convenience, I combined previous answer and comments to it:
sudo ./bjam cxxflags=-fPIC -a --with-system install
--with-system is not necessary, but it's a place where you can add other boost compile options
It works for me at CentOS 7 with boost 1.67
Another solution:
./bootstrap.sh
./b2 cxxflags=-fPIC cflags=-fPIC

Can I build gcc for ARM with an X64 one?

I need a definitive answer because this is still unclear to me and I can't find an explicit answer anywhere in the docs.
Assuming that I have a working gcc toolchain where
host x86_64-linux-gnu
target x86_64-linux-gnu
, the question is can I possibly configure and build gcc from sources with ?
host x86_64-linux-gnu
build x86_64-linux-gnu
target arm-gnu-eabi
The reason why I would like an answer on this is about whether or not I should spend time trying different configurations for my libraries and whether or whether or not the scripts used to build gcc are capable of some implicit stage 1 build that can potentially bootstrap an ARM compiler for me temporarily on this x64, so I can generate the toolchain that I need for the target that I want .
"Can I build gcc for ARM with an X64 one?"
Yes, you can. I have described this process for a suse linux host development system in a blog post of mine.
==================================================================================
I'm going to replicate the steps here:
1. Ensure to have the necessary headers & libraries installed
I have used YAST's 'Install Software' feature, to install the following packages that will be necessary to complete all the build steps (just search for the package names, select and accept):
gmp-devel
mpfr-devel
mpc-devel
texinfo
ncurses-devel
termcap
2. Create a directory skeleton
cd ~
mkdir arm-none-eabi arm-none-eabi-src
cd arm-none-eabi
mkdir src build
cd ~/arm-none-eabi-src
mkdir src build
3. Download the the source packages and extract them
I'm using gcc-4.7.1 here, but the same process will of course apply for newer versions of GCC.
cd ~/arm-none-eabi-src/src
wget ftp://ftp.gnu.org/gnu/gcc/gcc-4.7.1/gcc-4.7.1.tar.bz2
wget ftp://ftp.gnu.org/gnu/binutils/binutils-2.22.tar.bz2
wget ftp://ftp.gnu.org/gnu/gdb/gdb-7.4.tar.bz2
wget ftp://sources.redhat.com/pub/newlib/newlib-1.20.0.tar.gz
tar -xf gcc-4.7.1.tar.bz2
tar -xf binutils-2.22.tar.bz2
tar -xf gdb-7.4.tar.bz2
tar -xf newlib-1.20.0.tar.gz
4. Build the binutils
cd ~/arm-none-eabi-src/build
mkdir binutils-2.22
cd binutils-2.22
../../src/binutils-2.22/configure \
--target=arm-none-eabi \
--prefix=$HOME/arm-none-eabi \
--with-cpu=cortex-m3 \
--with-no-thumb-interwork \
--with-mode=thumb
make all install
export PATH="$PATH:$HOME/arm-none-eabi/bin"
5. Build GCC (Part1)
cd ~/arm-none-eabi-src/build
mkdir gcc-4.7.1
cd gcc-4.7.1
../../src/gcc-4.7.1/configure --target=arm-none-eabi \
--prefix=$HOME/arm-none-eabi --with-cpu=cortex-m3 \
--with-mode=thumb --disable-multilib \
--with-no-thumb-interwork \
--enable-languages="c,c++" --with-newlib \
--with-headers=../../src/newlib-1.20.0/newlib/libc/include
make all-gcc install-gcc
The --enable-cxx-flags configure option might be additionally used to control the build flags of the libstdc++ (included in this step):
--enable-cxx-flags='-fno-exceptions \
-ffunction-sections -fno-omit-frame-pointer'
In general the same C++ compile flags should be used as they'll appear when building the intended target code.
6. Build GCC newlib with the cross compiler (Part2)
cd ~/arm-none-eabi-src/build
mkdir newlib-1.20.0
cd newlib-1.20.0
../../src/newlib-1.20.0/configure --target=arm-none-eabi \
--prefix=$HOME/arm-none-eabi --disable-multilib \
--disable-newlib-supplied-syscalls
make all install
A note about the --disable-newlib-supplied-syscalls option:
Disabling the default newlib syscall stub implementation is generally a good idea when you intend to compile for targets without using a linux like operating system, or no OS at all. It will leave you with linker errors on unimplemented stub functions you'll need to provide for newlib.
Removing the option will still enable you to override the newlib provided stubs with your own implementations.
Though, when you plan to use the cross-toolchain in conjunction with CMake, you should omit this option. CMake does some basic tests using the specified compiler definitions (e.g. from a toolchain.cmake file), that'll fail without the default stub implementations supplied.
7. Complete installing GCC
cd ~/arm-none-eabi-src/build/gcc-4.7.1
make all install
8. Build GDB
cd ~/arm-none-eabi-src/build
mkdir gdb-7.4
cd gdb-7.4
../../src/gdb-7.4/configure --target=arm-none-eabi \
--prefix=$HOME/arm-none-eabi
make all install
UPDATE
The same works pretty well for GCC 4.8.2 also.

NTL header file not found

I have downloaded and installed the NTL library on my Ubuntu. I'm currently using gedit to write my program and having included this ZZ.h header in my program. This is how i compile my program in the terminal: - g++ keygen.cpp -o keygen -I ../include -L ../lib -lntl -lm.
I'm pretty sure this line is correct but for some unknown reason, i get the following error:
KeyGen.cpp:9:20: error: NTL/ZZ.h: No such file or directory
KeyGen.cpp:15: error: expected constructor, destructor, or type conversion before ‘int’
The solution seems pretty straightforward to me: which is to add the NTL library directly to my program folder. I did just that, but still i get the same error.
If you don't need the latest (6.0.0) version of NTL you may do as follows in your Ubuntu:
user#host:~$ sudo apt-get install libntl-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libntl-5.4.2
The following NEW packages will be installed:
libntl-5.4.2 libntl-dev
0 upgraded, 2 newly installed, 0 to remove and 112 not upgraded.
Need to get 2,035 kB of archives.
After this operation, 7,016 kB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://ftp.de.debian.org/debian/ squeeze/main libntl-5.4.2 amd64 5.4.2-4 [767 kB]
Get:2 http://ftp.de.debian.org/debian/ squeeze/main libntl-dev amd64 5.4.2-4 [1,268 kB]
Fetched 2,035 kB in 2s (1,017 kB/s)
Selecting previously deselected package libntl-5.4.2.
(Reading database ... 59184 files and directories currently installed.)
Unpacking libntl-5.4.2 (from .../libntl-5.4.2_5.4.2-4_amd64.deb) ...
Selecting previously deselected package libntl-dev.
Unpacking libntl-dev (from .../libntl-dev_5.4.2-4_amd64.deb) ...
Can not write log, openpty() failed (/dev/pts not mounted?)
Setting up libntl-5.4.2 (5.4.2-4) ...
Setting up libntl-dev (5.4.2-4) ..
user#host:~$
after that the complete compiled NTL library with all development headers is installed in your system and you may compile your program with it without any additional -I<path>.
If you need a newer version that your distro has (check http://packages.ubuntu.com/en/source/trusty/ntl) you may try to build the library package yourself.
The problem with your attempt to compile and output an executable file appears to be compiler's inability to link the necessary library after obtaining an object .o file.
Many people often test the point of fault by separating the two stages by first compiling g++ -c then by linking the libraries for an executable g++ -o. Although -Wall switch does not always work, trying it to provide you with as much information as possible during compilation can also be helpful.
Check this webpage. As for using different switches to link libraries try this webpage.
I'm not certain if it was a typo; but I wonder if the space between the switch and directory:-I ../include and -L ../libwas the problem.
You said in comments:
Icreated a folder called 'include' within the .cpp folder and included the NTL library in that folder already
But your compilation command says:
g++ keygen.cpp -o keygen -I ../include -L ../lib -lntl -lm.
It seems to me, you meant:
g++ keygen.cpp -o keygen -I ./include -L ../lib -lntl -lm.
# ^^^^^^^^^
since .. goes up one directory.

Linking g++ 4.8 to libstdc++

I downloaded and built gcc 4.8.1 on my desktop, running 64-bit Ubuntu 12.04. I built it out of source, like the docs recommend, and with the commands
../../gcc-4.8.1/configure --prefix=$HOME --program-suffix=-4.8
make
make -k check
make install
It seemed to pass all the tests, and I installed everything into my home directory w/ the suffix -4.8 to distinguish from the system gcc, which is version 4.6.3.
Unfortunately when I compile c++ programs using g++-4.8 it links to the system libc and libstdc++ rather than the newer ones compiled from gcc-4.8.1. I downloaded and built gcc 4.8 because I wanted to play around with the new C++11 features in the standard library, so this behaviour is definitely not what I wanted. What can I do to get gcc-4.8 to automatically link to the standard libraries that came with it rather than the system standard libraries?
When you link with your own gcc you need to add an extra run-time linker search path(s) with -Wl,-rpath,$(PREFIX)/lib64 so that at run-time it finds the shared libraries corresponding to your gcc.
I normally create a wrapper named gcc and g++ in the same directory as gcc-4.8 and g++-4.8 which I invoke instead of gcc-4.8 and g++-4.8, as prescribed in Dynamic linker is unable to find GCC libraries:
#!/bin/bash
exec ${0}SUFFIX -Wl,-rpath,PREFIX/lib64 "$#"
When installing SUFFIX and PREFIX should be replaced with what was passed to configure:
cd ${PREFIX}/bin && rm -f gcc g++ c++ gfortran
sed -e 's#PREFIX#${PREFIX}#g' -e 's#SUFFIX#${SUFFIX}#g' gcc-wrapper.sh > ${PREFIX}/bin/gcc
chmod +x ${PREFIX}/bin/gcc
cd ${PREFIX}/bin && ln gcc g++ && ln gcc c++ && ln gcc gfortran
(gcc-wrapper.sh is that bash snippet).
The above solution does not work with some versions of libtool because g++ -Wl,... -v assumes linking mode and fails with an error.
A better solution is to use specs file. Once gcc/g++ is built, invoke the following command to make gcc/g++ add -rpath to the linker command line (replace ${PREFIX}/lib64 as necessary):
g++ -dumpspecs | awk '/^\*link:/ { print; getline; print "-rpath=${PREFIX}/lib64", $0; next } { print }' > $(dirname $(g++ -print-libgcc-file-name))/specs
I just had the same problem when building gcc-4.8.2. I don't have root access on that machine and therefore need to install to my home directory. It took several attempts before I figured out the magic required to get this to work so I will reproduce it here so other people will have an easier time. These are the commands that I used to configure gcc:
prefix=/user/grc/packages
export LDFLAGS=-Wl,-rpath,$prefix/lib
export LD_RUN_PATH=$prefix/lib
export LD_LIBRARY_PATH=$prefix/lib
../../src/gmp-4.3.2/configure --prefix=$prefix
../../src/mpfr-2.4.2/configure --prefix=$prefix
../../src/mpc-0.8.1/configure --prefix=$prefix --with-mpfr=$prefix --with-gmp=$prefix
../../src/gcc-4.8.2/configure --prefix=$prefix --with-mpfr=$prefix --with-gmp=$prefix --with-mpc=$prefix --enable-languages=c,c++
That got me a working binary but any program I built with that version of g++ wouldn't run correctly unless I built it with the -Wl,-rpath,$prefix/lib64 option. It is possible to get g++ to automatically add that option by providing a specs file. If you run
strace g++ 2>&1 | grep specs
you can see which directories it checks for a specs file. In my case it was $prefix/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/specs so I ran g++ -dumpspecs to create a new specs file:
cd $prefix/lib/gcc/x86_64-unknown-linux-gnu/4.8.2
$prefix/bin/g++ -dumpspecs > xx
mv xx specs
and then edited that file to provide the -rpath option. Search for the lines like this:
*link_libgcc:
%D
and edit to add the rpath option:
*link_libgcc:
%D -rpath /user/grc/packages/lib/%M
The %M expands to either ../lib or ../lib64 depending on whether you are building a 32-bit or a 64-bit executable.
Note that when I tried this same trick on an older gcc-4.7 build it didn't work because it didn't expand the %M. For older versions you can remove the %M and just hardcode lib or lib64 but that is only a viable solution if you only ever build 32-bit executables (with lib) or only ever build 64-bit executables (with lib64).
gcc -print-search-dirs will tell you where your compiler is looking for runtime libraries, etc. You can override this with the -B<prefix> option.