I'm generating tvm and istalling and using it in another docker container
TVM wheel generation code:
# install cmake
RUN yum install -y wget
RUN wget https://github.com/Kitware/CMake/releases/download/v3.20.2/cmake-3.20.2.tar.gz -O cmake.tar.gz && \
mkdir cmake && tar -zxvf cmake.tar.gz -C cmake --strip-components 1 && \
cd cmake && ./bootstrap -- -DCMAKE_USE_OPENSSL=OFF && make -j8 && make install && cd .. && \
rm cmake.tar.gz && rm -rf cmake
RUN echo "CUDA $(ls /lib64/ | grep cu)" > /tmp/build_2.output
# install tvm
RUN git clone --recursive https://github.com/apache/incubator-tvm tvm && \
cd tvm && \
git reset --hard 338940dc5044885412f9a6045cb8dcdf9fb639a4 && \
git submodule init && \
git submodule update && \
mkdir ./build && \
cp cmake/config.cmake build && \
cd build && \
cmake --debug-output -DUSE_CUDA=ON -DUSE_CUDNN=ON -DUSE_CUBLAS=ON -DUSE_THRUST=ON -DUSE_LLVM=ON .. &>> /tmp/build_2.output && \
make -j$(nproc) &>> /tmp/build_2.output && \
cd ../python && python3.8 setup.py bdist_wheel
Verified that cuda .so files were created in /tmp/build_2.output
While installing and running the tvm in another place getting below error:
tvm.runtime.load_module
*TVMError: Binary was created using cuda but a loader of that name is not registered. Available loaders are GraphRuntimeFactory, metadata, GraphExecutorFactory, VMExecutable. Perhaps you need to recompile with this runtime enabled.*
How to register cuda loader?
Related
Building WebRTC directly is somewhat painful (& rather time consuming so not ideal in paid for CI environments) so I've opted to use some excellent precompiled static libraries released on GitHub to speed things up. This has worked perfectly for all platforms I'm targetting (Windows, MacOS, iOS, Apple TVOS, linux...) except for Android.
When linking the binary on Android I encounter several duplicate symbol errors related to the std libary:
ld: error: duplicate symbol: std::logic_error::logic_error(char const*)
>>> defined at stdexcept_default.ipp:24 (../../../../../_source/android/webrtc/src/buildtools/third_party/libc++/trunk/src/support/runtime/stdexcept_default.ipp:24)
>>> stdexcept.o:(std::logic_error::logic_error(char const*)) in archive /root/webrtc/lib/armeabi-v7a/libwebrtc.a
>>> defined at stdexcept_default.ipp:24 (/buildbot/src/android/ndk-release-r23/toolchain/llvm-project/libcxx/src/support/runtime/stdexcept_default.ipp:24)
>>> stdexcept.o:(.text._ZNSt11logic_errorC2EPKc+0x1) in archive /app/android-sdk-linux/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libc++_static.a
...
ld: error: duplicate symbol: std::runtime_error::runtime_error(char const*)
>>> defined at stdexcept_default.ipp:35 (../../../../../_source/android/webrtc/src/buildtools/third_party/libc++/trunk/src/support/runtime/stdexcept_default.ipp:35)
>>> stdexcept.o:(std::runtime_error::runtime_error(char const*)) in archive /root/webrtc/lib/armeabi-v7a/libwebrtc.a
>>> defined at stdexcept_default.ipp:35 (/buildbot/src/android/ndk-release-r23/toolchain/llvm-project/libcxx/src/support/runtime/stdexcept_default.ipp:35)
>>> stdexcept.o:(.text._ZNSt13runtime_errorC2EPKc+0x1) in archive /app/android-sdk-linux/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libc++_static.a
It appears that libwebrtc.a for Android includes some libc++ object files and I'm not entirely sure how to successfully link with this library here.
This might be coming down to the libwebrtc.a being compiled with Google's own patched libc++ (Based on what I've read online I believe Google contributes patches to upstream libc++ but they don't wait for the patches to go in - rather they use their fully patched version of libc++ for building WebRTC, Chrome, etc).
When compiling for Linux I actually use the exact versions of clang & libc++ that were downloaded & used in building the static library & I suspect I might need to be doing something similar here - I'm just not entirely sure how given the need to use the Android CMake toolchain.
I've put together a minimal example project on GitHub with Dockerfile that can be used to easily reproduce the above issue.
Does anyone have an idea how I can successfully link the library on Android?
For completeness the source files of interest are:
CMakeLists.txt
cmake_minimum_required(VERSION 3.23)
project(AndroidWebRTC)
set(CMAKE_CXX_STANDARD 17)
add_executable(AndroidWebRTC src/main.cpp)
target_link_libraries(AndroidWebRTC PRIVATE ${WEBRTC_LIBRARY})
add_compile_definitions(WEBRTC_POSIX WEBRTC_ANDROID WEBRTC_LINUX)
include_directories(
${WEBRTC_INCLUDE_DIR}
${WEBRTC_INCLUDE_DIR}/third_party/abseil-cpp
${WEBRTC_INCLUDE_DIR}/third_party/boringssl/src/include
${WEBRTC_INCLUDE_DIR}/third_party/libyuv/include
${WEBRTC_INCLUDE_DIR}/third_party/zlib
)
Dockerfile
# I explicitly specify an amd64 platform as I often build on Apple Silicon
# machines where the default of arm64 breaks things. I believe the alternative
# (and possibly better option) is to use NDK 24 and above, see:
# https://stackoverflow.com/a/69541958
FROM --platform=linux/amd64 ubuntu:20.04 AS build
ENV ANDROID_SDK_ROOT /app/android-sdk-linux
WORKDIR /app
# Install dependencies to build the project
# include apt-get --no-install-recommends
RUN apt-get update && \
apt-get -y upgrade && \
apt-get --no-install-recommends -y install tzdata && \
echo 'Europe/London' > /etc/timezone && \
dpkg-reconfigure -f noninteractive tzdata
RUN apt-get --no-install-recommends -y install git lsb-release python rsync \
emacs wget build-essential sudo pkg-config clang unzip openjdk-8-jdk ant \
android-sdk-platform-tools-common libncurses5 curl
RUN mkdir ${ANDROID_SDK_ROOT}
# ------------------------------------------------------
# --- Android SDK
RUN wget https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip && \
unzip commandlinetools-linux-7583922_latest.zip && \
mkdir ${ANDROID_SDK_ROOT}/cmdline-tools/ &&\
mv cmdline-tools ${ANDROID_SDK_ROOT}/cmdline-tools/latest && \
rm commandlinetools-linux-7583922_latest.zip
ENV PATH ${PATH}:${ANDROID_SDK_ROOT}/cmdline-tools/latest/bin
RUN yes | sdkmanager --licenses
RUN touch /root/.android/repositories.cfg
# Platform tools
RUN yes | sdkmanager "platform-tools"
RUN yes | sdkmanager --update --channel=0
# Keep all sections in descending order!
RUN yes | sdkmanager \
"platforms;android-30" \
"build-tools;31.0.0" \
"ndk;23.1.7779620" \
"extras;android;m2repository" \
"extras;google;m2repository" \
"extras;google;google_play_services" \
"add-ons;addon-google_apis-google-24"
ENV ANDROID_HOME ${ANDROID_SDK_ROOT}
ENV ANDROID_NDK_HOME=${ANDROID_SDK_ROOT}/ndk/23.1.7779620
ENV PATH ${PATH}:${ANDROID_NDK_HOME}:${ANDROID_HOME}/build-tools/31.0.0/
# Get CMake
COPY scripts/get_cmake.sh scripts/get_cmake.sh
RUN ./scripts/get_cmake.sh "3.25.1" linux /root
ENV PATH "/root/cmake/bin:$PATH"
# Get WebRTC
COPY scripts/get_webrtc.sh scripts/get_webrtc.sh
RUN ./scripts/get_webrtc.sh 108.5359.5.0 android /root /root
## Copy resources
COPY src src
COPY CMakeLists.txt ./
RUN cmake -B build \
-DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK_HOME}/build/cmake/android.toolchain.cmake" \
-DANDROID_ABI=armeabi-v7a \
-DANDROID_NATIVE_API_LEVEL=16 \
-DBUILD_SHARED_LIBS=OFF \
-DWEBRTC_INCLUDE_DIR=/root/webrtc/include \
-DWEBRTC_LIBRARY=/root/webrtc/lib/armeabi-v7a/libwebrtc.a \
-DCMAKE_BUILD_TYPE=Release
RUN cmake --build build --config Release --parallel $(nproc) --target AndroidWebRTC
src/main.cpp
#include <iostream>
#include <api/create_peerconnection_factory.h>
int main() {
auto googleSessionDescription = webrtc::CreateSessionDescription(
webrtc::SdpType::kOffer, "sdp");
std::cout << "Hello, World!" << std::endl;
return 0;
}
scripts/get_webrtc.sh
#!/bin/bash
if [ $# -lt 4 ]; then
echo "$0 <webrtc_build_version> <package_name> <output_dir> <source_dir>"
exit 1
fi
WEBRTC_BUILD_VERSION=$1
PACKAGE_NAME=$2
OUTPUT_DIR=$3
SOURCE_DIR=$4
set -ex
if [ ! -e $SOURCE_DIR/webrtc.${PACKAGE_NAME}.${WEBRTC_BUILD_VERSION}.tar.gz ]; then
curl -Lo $SOURCE_DIR/webrtc.${PACKAGE_NAME}.${WEBRTC_BUILD_VERSION}.tar.gz https://github.com/shiguredo-webrtc-build/webrtc-build/releases/download/m${WEBRTC_BUILD_VERSION}/webrtc.${PACKAGE_NAME}.tar.gz
fi
pushd $OUTPUT_DIR
tar xf $SOURCE_DIR/webrtc.${PACKAGE_NAME}.${WEBRTC_BUILD_VERSION}.tar.gz
popd
scripts/get_cmake.sh
#!/bin/bash
# <platform> には Linux か Darwin を指定する
# <output_dir>/cmake に CMake が配置される
if [ $# -lt 3 ]; then
echo "$0 <cmake_version> <platform> <output_dir>"
exit 1
fi
CMAKE_VERSION=$1
PLATFORM=$2
OUTPUT_DIR=$3
set -ex
pushd $OUTPUT_DIR
curl -LO https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64.tar.gz
tar xf cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64.tar.gz
rm cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64.tar.gz
rm -rf cmake
mv cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64 cmake
popd
So I am atempting to build the ARA-RISCV build and it requires a RISC-V LLVM toolchain and they give you a "make toolchain-llvm" command and it starts off ok but midway through the build around 2500/3500 it stops and the terminal closes without pointing out any error while it builds.
The command that are used are the ones below:
mkdir -p /.../rvv/install/riscv-llvm
cd /.../rvv/toolchain/riscv-llvm && rm -rf build && mkdir -p build && cd build && \
cmake -G Ninja \
-DCMAKE_INSTALL_PREFIX=/home/alex/Documents/Project/rvv/install/riscv-llvm \
-DLLVM_ENABLE_PROJECTS="clang;lld" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=gcc \
-DCMAKE_CXX_COMPILER=g++ \
-DLLVM_DEFAULT_TARGET_TRIPLE=riscv64-unknown-elf \
-DLLVM_TARGETS_TO_BUILD="RISCV" \
../llvm
cd /.../rvv/toolchain/riscv-llvm && \
cmake --build build --target install
After everything the build is incomplete and a lot of files have not been downloaded.I dont know what is causing this can anybody help?
Please and thank you.
EDIT:It worked by adding -j8 to the last instruction aka
cmake --build build --target install -j8
I need to run some code that contains both Python 3.8 and R 4.1.0 in a Docker container. Below is my Dockerfile.
FROM python:3.8-slim-buster AS final-image
# R version to install
ARG R_BASE_VERSION=4.1.0
ARG PREBUILD_DEPS="software-properties-common gnupg2"
ARG BUILD_DEPS="build-essential binutils cmake gfortran libblas-dev liblapack-dev libjpeg-dev libpng-dev libnlopt-dev pkg-config"
ARG RUNTIME_DEPS="r-base=${R_BASE_VERSION}-* libcurl4-openssl-dev libssl-dev libxml2-dev"
# venv path
ENV PATH="/opt/venv/bin:$PATH"
RUN apt-get update \
# Adding this to install latest versions of g++
&& echo 'deb http://deb.debian.org/debian testing main' > /etc/apt/sources.list.d/testing.list \
# Install the below packages to add repo which is then used to install R version 4
&& apt-get install -y --no-install-recommends $PREBUILD_DEPS \
&& add-apt-repository 'deb http://cloud.r-project.org/bin/linux/debian buster-cran40/'\
# This key is required to install r-base version 4
&& apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key FCAE2A0E115C3D8A \
# Update again to use the newly added sources
&& apt-get update \
&& apt-get install -y --no-install-recommends $RUNTIME_DEPS $BUILD_DEPS \
&& python -m venv /opt/venv \
&& /opt/venv/bin/python -m pip install --upgrade pip \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt packages.R /
RUN pip install wheel setuptools \
&& pip install --no-cache-dir -r requirements.txt \
&& pip install --no-cache-dir --no-binary xgboost xgboost \
&& Rscript packages.R \
&& strip --strip-unneeded usr/local/lib/R/site-library/Boom/lib/libboom.a \
&& strip --strip-debug /usr/local/lib/R/site-library/*/libs/*so \
# Uninstall unnecessary dependencies
&& rm -rf /tmp/* \
&& apt-get purge -y --auto-remove $BUILD_DEPS $PREBUILD_DEPS \
&& apt-get autoremove -y \
&& apt-get autoclean -y \
&& rm -rf /var/lib/apt/lists/*
ENTRYPOINT XXX
This is packages.R file:
#Setting environment
rm(list = ls())
cat("\014")
print(Sys.time())
# CRAN mirror to use. cran.rstudio.com is a CDN and the recommended mirror.
# Specifying multiple backup CRAN mirrors as Jenkins builds fails
# intermittently due to unavailability of packages in main mirror.
cran_repos = c(MAIN_CRAN_MIRROR = 'https://cran.rstudio.com',
ALT_CRAN_MIRROR = 'http://cran.r-project.org/')
#Loading Libraries
package_ls <- c(
"config",
"crayon",
"aws.s3",
"aws.ec2metadata",
"dplyr",
"data.table",
"imputeTS",
"Metrics",
"StatMeasures",
"tseries",
"purrr",
"log4r",
"lubridate",
"forecast",
"caret",
"MASS",
"stringr",
"tidyr",
"uroot",
"readr",
"Boruta",
"bsts"
)
for (pkg_name in package_ls) {
message("Installing ", pkg_name)
install.packages(pkg_name, repos = cran_repos)
if (!(pkg_name %in% installed.packages()[, 'Package'])) {
stop(pkg_name,
" is a required package and it could not be installed, stopping!")
}
}
The Problem
Building the docker container takes a lot more time than I would like. It is because, some packages (e.g. bsts) needs their dependencies (e.g. the C++ library Boom) to be built from source and this is taking a lot of time.
Is there a way to either:
Speed up the building of R libraries? OR
Build R libraries in local and copy only the binaries to the Docker container. OR
Reduce the build time in any other way for R packages.
Thanks in advance.
Update
Some ideas from the comments:
By #botje
Install R packages in parallel using the Ncpus argument of install.packages R function. (I have 4 CPUs to work with and setting Ncpus = 4 resulted in a 10% speed up.)
install.packages(package_ls, repos = cran_repos, Ncpus = 4)
Create a custom CRAN mirror containing locally-compiled packages for faster installation.
I rewrote the last bit of your packages.R as follows:
install.packages(package_ls, Ncpus=16)
This gave me a 3x speed improvement over a run with Ncpus=1 (189s vs 719s).
I'm trying to build a docker container using the following Dockerfile:
FROM fedora:29
RUN dnf -y update && dnf install -y file gcc gcc-c++ git make wget which libtool python3-pip redhat-rpm-config python3-devel zlib-devel libstdc++ openmpi-devel
RUN cd /tmp && \
wget http://www.mpich.org/static/downloads/3.3/mpich-3.3.tar.gz && \
gzip -dc mpich-3.3.tar.gz | tar xf - && \
cd mpich-3.3 && \
./configure --disable-fortran --prefix=/usr/mpich-3.3 && \
make && \
make install
ENV PATH /usr/mpich-3.3/bin:${PATH}
ENV LD_LIBRARY_PATH /usr/mpich-3.3/lib:${LD_LIBRARY_PATH}
RUN cd /usr && git clone https://github.com/Dowell-Lab/FStitch
RUN cd /usr/FStitch/src && make clean && make
RUN pip3 install FStitch-Bidir --user
ENV PATH /usr/FStitch/src:${PATH}
ENV PATH /root/.local/bin:${PATH}
RUN cd /usr && git clone https://github.com/Dowell-Lab/Tfit
RUN cd /usr/Tfit/src && make clean && make
ENV PATH /usr/Tfit/src:${PATH}
CMD /bin/bash
Where the projects I'm trying to clone are written in c++11 and the second of the two (Tfit) requires openmpi/mpich. The first program compiles successfully, but with the second, I'm getting the following error in the last step of the compiler:
/usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
make: *** [Makefile:20: NU_FIT] Error 1
I searched and found these two links:
cpp: usr/bin/ld: cannot find -l<nameOfTheLibrary>
usr/bin/ld: cannot find -l<nameOfTheLibrary>
But neither of these quite address the problem as I'm guessing I'm just missing a dependency/symlink to a library, but I'm unsure how to achieve this in the build. I can compile this successfully locally, but I have to module load mpi/openmpi-x86_64 to do so. My guess is that this is an openmpi issue when setting the LD_LIBRARY_PATH, but am unsure of how to resolve this in the Docker build.
The first few lines of the Makefile are as follows:
CXX = mpic++
CXXFLAGS = -static-libstdc++ -static-libgcc -Wno-unused-variable -Wno-non-virtual-dtor -std=c++11 -fopenmp -Wno-write-strings -Wno-literal-suffix -D_GLIBCXX_USE_CXX11_ABI=0 -g
EXEC = ${PWD}/Tfit
ARCH = getconf LONG_BIT
CPP_FLAGS_32 = -D32_BIT
CPP_FLAGS_64 = -D64_BIT
GCCVERSION = $(shell ${CXX} -dumpversion)
NU_FIT: main.o load.o split.o model.o across_segments.o template_matching.o \
read_in_parameters.o model_selection.o error_stdo_logging.o \
MPI_comm.o density_profiler.o bootstrap.o prelim_main.o model_main.o select_main.o FDR.o BIC.o ParamWrapper.o old_template_matching.o
#printf "linking : "
#${CXX} ${CXXFLAGS} ${PWD}/main.o ${PWD}/load.o ${PWD}/model_selection.o \
${PWD}/split.o ${PWD}/model.o ${PWD}/across_segments.o \
${PWD}/template_matching.o ${PWD}/read_in_parameters.o \
${PWD}/MPI_comm.o \
${PWD}/bootstrap.o ${PWD}/density_profiler.o \
${PWD}/prelim_main.o ${PWD}/model_main.o ${PWD}/BIC.o ${PWD}/FDR.o \
${PWD}/select_main.o ${PWD}/error_stdo_logging.o ${PWD}/ParamWrapper.o ${PWD}/old_template_matching.o -o ${EXEC} -lmpi
#cp ${PWD}/Tfit ${PWD}/EMGU
Any help is appreciated!
Try add libstdc++-static in your list of dnf install. (https://github.com/numenta/nupic/issues/1901)
I'm trying to build Clang with libc++ from sources. And I'm trying to drop libc++ in-tree while building it out-of-tree with the other components. The recipe I use is below.
If I simply place libcxx and libcxxabi in-tree, then configure does not pick them up, and they are not built automatically. I placed them in llvm/projects per LLVM's libc++ Standard Library.
Additionally, adding make cxx to the recipe does not work as advertised on the LLVM's libc++ Standard Library page. It results in:
llvm[0]: Constructing LLVMBuild project information.
make: *** No rule to make target `cxx'. Stop.
When I configure LLVM/Clang with --with-libcxx and --with-libcxxabi:
# Issued from a scratch 'build' directory, which is next to the 'llvm' directory.
../llvm/configure --enable-optimized --enable-cxx11 --with-libcxx --with-libcxxabi \
$OTHER_OPTIONS --prefix=/usr/local
then I receive the following:
configure: WARNING: Unknown project (libcxx) won't be configured automatically
configure: WARNING: Unknown project (libcxxabi) won't be configured automatically
libcxx and libcxxabi are literally what LLVM calls them, so I'm not sure what names to use if they are not correct.
I tried to examine configure for what the package names should be, but its not very helpful. See below for the logic.
How do I configure and build Clang with libc++ (when libc++ and libc++ ABI are in-tree)?
Configure logic for --with-XXX is shown below.
this is all I can find (its not very helpful):
-with-* | --with-*)
ac_package=`expr "x$ac_option" : 'x-*with-\([^=]*\)'`
# Reject names that are not valid shell variable names.
expr "x$ac_package" : ".*[^-_$as_cr_alnum]" >/dev/null &&
{ echo "$as_me: error: invalid package name: $ac_package" >&2
{ (exit 1); exit 1; }; }
ac_package=`echo $ac_package| sed 's/-/_/g'`
eval with_$ac_package=\$ac_optarg ;;
Related links:
LLVM/Clang download page
LLVM's libc++ Standard Library page
And this Stack Overflow question is related: When is libc++ sources needed when building Clang from sources?
And this discussion of CFE-Dev mailing list: Questions about libc++ for linux and its git repository (if any). The thread says unpacking libcxx into llcm/projects ensures the headers are copied where Clang expects them during make install. But it does not address the --with-XXX question, it does not discuss why libc++ was not built, and it does not discuss how to get make install to actually install the libraries.
Recipe to fetch and build Clang. It works fine when not including libcxx and libcxxabi.
#! /bin/sh
# Clang 3.5 recipe.
# The script should be run from a scratch directory.
# Fetch
if [ ! -e llvm-3.5.0.src.tar.xz ]; then
wget http://llvm.org/releases/3.5.0/llvm-3.5.0.src.tar.xz
fi
if [ ! -e cfe-3.5.0.src.tar.xz ]; then
wget http://llvm.org/releases/3.5.0/cfe-3.5.0.src.tar.xz
fi
if [ ! -e compiler-rt-3.5.0.src.tar.xz ]; then
wget http://llvm.org/releases/3.5.0/compiler-rt-3.5.0.src.tar.xz
fi
if [ ! -e libcxx-3.5.0.src.tar.xz ]; then
wget http://llvm.org/releases/3.5.0/libcxx-3.5.0.src.tar.xz
fi
if [ ! -e libcxxabi-3.5.0.src.tar.xz ]; then
wget http://llvm.org/releases/3.5.0/libcxxabi-3.5.0.src.tar.xz
fi
if [ ! -e clang-tools-extra-3.5.0.src.tar.xz ]; then
wget http://llvm.org/releases/3.5.0/clang-tools-extra-3.5.0.src.tar.xz
fi
# Cleanup
echo "Cleaning up artifacts"
rm -rf llvm build llvm-3.5.0.src
# LLVM
echo "Unpacking LLVM"
tar xf llvm-3.5.0.src.tar.xz
mv llvm-3.5.0.src/ llvm
# Clang Front End
echo "Unpacking Clang Front End"
cd llvm/tools
tar xf ../../cfe-3.5.0.src.tar.xz
mv cfe-3.5.0.src clang
cd ../../
# Compiler RT
echo "Unpacking Compiler RT"
cd llvm/projects
tar xf ../../compiler-rt-3.5.0.src.tar.xz
mv compiler-rt-3.5.0.src/ compiler-rt
cd ../../
# Extra Tools
echo "Unpacking Extra Tools"
cd llvm/tools/clang/tools/
tar xf ../../../../clang-tools-extra-3.5.0.src.tar.xz
mv clang-tools-extra-3.5.0.src extra
cd ../../../../
# libc++
echo "Unpacking libc++"
cd llvm/projects
tar xf ../../libcxx-3.5.0.src.tar.xz
mv libcxx-3.5.0.src/ libcxx
cd ../../
# libc++ ABI
echo "Unpacking libc++ ABI"
cd llvm/projects
tar xf ../../libcxxabi-3.5.0.src.tar.xz
mv libcxxabi-3.5.0.src/ libcxxabi
cd ../../
# Determine if Apple
IS_DARWIN=`uname -s | egrep -i -c "Darwin"`
if [ $IS_DARWIN -ne 0 ]; then
OTHER_OPTIONS=" --enable-libcpp"
fi
# Configure
echo "Configuring build"
mkdir -p build
cd build
../llvm/configure --enable-optimized --enable-cxx11 --with-libcxx --with-libcxxabi $OTHER_OPTIONS --prefix=/usr/local
# Build
# 'make cxx' for libc++ is from http://libcxx.llvm.org/
echo "Running make"
make cxx
make -j2
RET=$?
if [ $RET -eq 0 ];then
echo "****************************************"
read -p "Press [ENTER] to install, or [CTRL]+C to quit"
sudo make install
fi
# ****************************************
# ****************************************
# Install does not install scan-build and scan-view
# Perform the copy, and/or put them on-path
#sudo cp llvm/projects/compiler-rt/lib/asan/scripts/asan_symbolize.py /usr/local/bin
#sudo 2to3 -w /usr/local/bin/asan_symbolize.py
#sudo mkdir /usr/local/bin/scan-build
#sudo cp -r llvm/tools/clang/tools/scan-build /usr/local/bin
#sudo mkdir /usr/local/bin/scan-view
#sudo cp -r llvm/tools/clang/tools/scan-view /usr/local/bin
This is script which I used to build libcxxabi and libcxx. It use previously built Clang (with GCC (4.8.3 in my case) and GCC STL):
if ( $#argv != 2 ) then
echo "Usage: [32|64] <directory>"
exit
endif
set echo on
set CMake=<CMake executable>
set GCCDir=<recent GCC directory>
set LLVMSourceDir=${PWD}/llvm-${LLVM_VERSION}.src
set LLVMOutOfTreeSourceDir=${PWD}
set LLVMPass1Dir=${PWD}/pass1
set PythonDir=<Python directory>
set InstallDir=${PWD}/$argv[2]
if ( $argv[1] == 32 ) then
set GCC_EHDir=${GCCDir}/lib/gcc/x86_64-redhat-linux/4.8.3/32
set BuildMode="-m32"
set LibDirSuffix=""
else
set GCC_EHDir=${GCCDir}/lib/gcc/x86_64-redhat-linux/4.8.3
set BuildMode="-m64"
set LibDirSuffix="64"
endif
set BuildDir=libcxxabi.build
if ( -d ${BuildDir} ) then
rm -rf ${BuildDir}
endif
mkdir ${BuildDir}
cd ${BuildDir}
${CMake} \
-DCMAKE_BUILD_TYPE="Release" \
-DCMAKE_INSTALL_PREFIX:PATH=${InstallDir} \
-DCMAKE_C_COMPILER=${LLVMPass1Dir}/bin/clang \
-DCMAKE_C_FLAGS=${BuildMode} \
-DCMAKE_CXX_COMPILER=${LLVMPass1Dir}/bin/clang++ \
-DCMAKE_CXX_FLAGS=${BuildMode} \
-DCMAKE_SHARED_LINKER_FLAGS="-L ${GCC_EHDir}" \
-DCMAKE_STATIC_LINKER_FLAGS="${GCC_EHDir}/libgcc_eh.a" \
-DLLVM_FORCE_USE_OLD_TOOLCHAIN=YES \
-DLLVM_PATH=${LLVMSourceDir} \
-DLIBCXXABI_LIBCXX_INCLUDES=${LLVMOutOfTreeSourceDir}/libcxx-${LLVM_VERSION}.src/include \
-DLIBCXXABI_LIBCXX_PATH=${LLVMOutOfTreeSourceDir}/libcxx-${LLVM_VERSION}.src \
-DLIBCXXABI_LIBDIR_SUFFIX=${LibDirSuffix} \
${LLVMOutOfTreeSourceDir}/libcxxabi-${LLVM_VERSION}.src
make
make install
cd ..
set BuildDir=libcxx.build
if ( -d ${BuildDir} ) then
rm -rf ${BuildDir}
endif
mkdir ${BuildDir}
cd ${BuildDir}
${CMake} \
-DCMAKE_BUILD_TYPE="Release" \
-DCMAKE_INSTALL_PREFIX:PATH=${InstallDir} \
-DCMAKE_C_COMPILER=${LLVMPass1Dir}/bin/clang \
-DCMAKE_C_FLAGS=${BuildMode} \
-DCMAKE_CXX_COMPILER=${LLVMPass1Dir}/bin/clang++ \
-DCMAKE_CXX_FLAGS=${BuildMode} \
-DCMAKE_SHARED_LINKER_FLAGS="-L ${GCCDir}/lib${LibDirSuffix}" \
-DLLVM_PATH=${LLVMSourceDir} \
-DLIBCXX_CXX_ABI=libcxxabi \
-DLIBCXX_CXX_ABI_INCLUDE_PATHS=${LLVMOutOfTreeSourceDir}/libcxxabi-${LLVM_VERSION}.src/include \
-DLIBCXX_CXX_ABI_LIBRARY_PATH=${InstallDir}/lib \
-DLIBCXX_LIBDIR_SUFFIX=${LibDirSuffix} \
-DLIT_EXECUTABLE=${LLVMSourceDir}/utils/lit/lit.py \
${LLVMOutOfTreeSourceDir}/libcxx-${LLVM_VERSION}.src
make
make install
cd ..
libc++ and libc++abi aren't maintained to work with configure. I think libc++ might work if you invoke it correctly, but there isn't even a configure script for libc++abi.
See our docs for using cmake with these projects:
# In-tree build:
# Check out libcxx and libcxxabi into llvm/projects
cd llvm
mkdir build && cd build
cmake .. # Linux may require -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
make cxx