I'm trying to build libtorch (i.e. the headers and shared object files for pytorch) from source, but get an unexpected result. I expected something that looks like the libtorch downloads from pytorch.org:
bin/
include/
lib/
share/
but instead got this:
abi-check Caffe2Config.cmake cmake_uninstall.cmake empty.cpp modules/ TorchConfig.cmake
aten/ Caffe2ConfigVersion.cmake compile_commands.json FXdiv/ nccl/ TorchConfigVersion.cmake
bin/ CMakeCache.txt confu-deps/ include/ nccl_external-prefix/
build.ninja CMakeFiles/ CTestTestfile.cmake install_manifest.txt sleef/
caffe2/ cmake_install.cmake detect_cuda_version.cc lib/ third_party/
It also contains lots of .o files and other junk, which I didn't expect. And it's missing many of the headers, such as the contents of torch/csrc/autograd/.
I followed the instructions for setting up my environment with Conda. Here's the approximate command I used to build:
export BUILD_CAFFE2=1 # probably don't need this
export BUILD_TEST=0
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
export DEBUG=0
export MAX_JOBS=16
export USE_CUDA=1
export USE_CUDNN=1
export CUDA_HOME=...
export NVCC_EXECUTABLE=...
export CUDNN_LIB_DIR=...
export CUDNN_INCLUDE_DIR=...
export CUDNN_LIBRARY=...
python ../tools/build_libtorch.py
I'm doing this on Linux with Python 3.8.
How can I get it to build the usual libtorch package structure?
I found a way to create the intended libtorch structure by directly using cmake, following this document:
cmake \
-DBUILD_SHARED_LIBS:BOOL=ON \
-DCMAKE_BUILD_TYPE:STRING=Release \
-DPYTHON_EXECUTABLE:PATH=`which python3` \
-DCMAKE_INSTALL_PREFIX:PATH=../pytorch-install \
-DBUILD_CAFFE2=1 \
-DCMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"} \
-DUSE_CUDA=1 \
-DUSE_CUDNN=1 \
-DCUDNN_INCLUDE_DIR=... \
-DCUDNN_LIBRARY=... \
../pytorch
cmake --build . --target install -- -j 16
This produces libtorch in ../pytorch-install.
Related
Building WebRTC directly is somewhat painful (& rather time consuming so not ideal in paid for CI environments) so I've opted to use some excellent precompiled static libraries released on GitHub to speed things up. This has worked perfectly for all platforms I'm targetting (Windows, MacOS, iOS, Apple TVOS, linux...) except for Android.
When linking the binary on Android I encounter several duplicate symbol errors related to the std libary:
ld: error: duplicate symbol: std::logic_error::logic_error(char const*)
>>> defined at stdexcept_default.ipp:24 (../../../../../_source/android/webrtc/src/buildtools/third_party/libc++/trunk/src/support/runtime/stdexcept_default.ipp:24)
>>> stdexcept.o:(std::logic_error::logic_error(char const*)) in archive /root/webrtc/lib/armeabi-v7a/libwebrtc.a
>>> defined at stdexcept_default.ipp:24 (/buildbot/src/android/ndk-release-r23/toolchain/llvm-project/libcxx/src/support/runtime/stdexcept_default.ipp:24)
>>> stdexcept.o:(.text._ZNSt11logic_errorC2EPKc+0x1) in archive /app/android-sdk-linux/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libc++_static.a
...
ld: error: duplicate symbol: std::runtime_error::runtime_error(char const*)
>>> defined at stdexcept_default.ipp:35 (../../../../../_source/android/webrtc/src/buildtools/third_party/libc++/trunk/src/support/runtime/stdexcept_default.ipp:35)
>>> stdexcept.o:(std::runtime_error::runtime_error(char const*)) in archive /root/webrtc/lib/armeabi-v7a/libwebrtc.a
>>> defined at stdexcept_default.ipp:35 (/buildbot/src/android/ndk-release-r23/toolchain/llvm-project/libcxx/src/support/runtime/stdexcept_default.ipp:35)
>>> stdexcept.o:(.text._ZNSt13runtime_errorC2EPKc+0x1) in archive /app/android-sdk-linux/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libc++_static.a
It appears that libwebrtc.a for Android includes some libc++ object files and I'm not entirely sure how to successfully link with this library here.
This might be coming down to the libwebrtc.a being compiled with Google's own patched libc++ (Based on what I've read online I believe Google contributes patches to upstream libc++ but they don't wait for the patches to go in - rather they use their fully patched version of libc++ for building WebRTC, Chrome, etc).
When compiling for Linux I actually use the exact versions of clang & libc++ that were downloaded & used in building the static library & I suspect I might need to be doing something similar here - I'm just not entirely sure how given the need to use the Android CMake toolchain.
I've put together a minimal example project on GitHub with Dockerfile that can be used to easily reproduce the above issue.
Does anyone have an idea how I can successfully link the library on Android?
For completeness the source files of interest are:
CMakeLists.txt
cmake_minimum_required(VERSION 3.23)
project(AndroidWebRTC)
set(CMAKE_CXX_STANDARD 17)
add_executable(AndroidWebRTC src/main.cpp)
target_link_libraries(AndroidWebRTC PRIVATE ${WEBRTC_LIBRARY})
add_compile_definitions(WEBRTC_POSIX WEBRTC_ANDROID WEBRTC_LINUX)
include_directories(
${WEBRTC_INCLUDE_DIR}
${WEBRTC_INCLUDE_DIR}/third_party/abseil-cpp
${WEBRTC_INCLUDE_DIR}/third_party/boringssl/src/include
${WEBRTC_INCLUDE_DIR}/third_party/libyuv/include
${WEBRTC_INCLUDE_DIR}/third_party/zlib
)
Dockerfile
# I explicitly specify an amd64 platform as I often build on Apple Silicon
# machines where the default of arm64 breaks things. I believe the alternative
# (and possibly better option) is to use NDK 24 and above, see:
# https://stackoverflow.com/a/69541958
FROM --platform=linux/amd64 ubuntu:20.04 AS build
ENV ANDROID_SDK_ROOT /app/android-sdk-linux
WORKDIR /app
# Install dependencies to build the project
# include apt-get --no-install-recommends
RUN apt-get update && \
apt-get -y upgrade && \
apt-get --no-install-recommends -y install tzdata && \
echo 'Europe/London' > /etc/timezone && \
dpkg-reconfigure -f noninteractive tzdata
RUN apt-get --no-install-recommends -y install git lsb-release python rsync \
emacs wget build-essential sudo pkg-config clang unzip openjdk-8-jdk ant \
android-sdk-platform-tools-common libncurses5 curl
RUN mkdir ${ANDROID_SDK_ROOT}
# ------------------------------------------------------
# --- Android SDK
RUN wget https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip && \
unzip commandlinetools-linux-7583922_latest.zip && \
mkdir ${ANDROID_SDK_ROOT}/cmdline-tools/ &&\
mv cmdline-tools ${ANDROID_SDK_ROOT}/cmdline-tools/latest && \
rm commandlinetools-linux-7583922_latest.zip
ENV PATH ${PATH}:${ANDROID_SDK_ROOT}/cmdline-tools/latest/bin
RUN yes | sdkmanager --licenses
RUN touch /root/.android/repositories.cfg
# Platform tools
RUN yes | sdkmanager "platform-tools"
RUN yes | sdkmanager --update --channel=0
# Keep all sections in descending order!
RUN yes | sdkmanager \
"platforms;android-30" \
"build-tools;31.0.0" \
"ndk;23.1.7779620" \
"extras;android;m2repository" \
"extras;google;m2repository" \
"extras;google;google_play_services" \
"add-ons;addon-google_apis-google-24"
ENV ANDROID_HOME ${ANDROID_SDK_ROOT}
ENV ANDROID_NDK_HOME=${ANDROID_SDK_ROOT}/ndk/23.1.7779620
ENV PATH ${PATH}:${ANDROID_NDK_HOME}:${ANDROID_HOME}/build-tools/31.0.0/
# Get CMake
COPY scripts/get_cmake.sh scripts/get_cmake.sh
RUN ./scripts/get_cmake.sh "3.25.1" linux /root
ENV PATH "/root/cmake/bin:$PATH"
# Get WebRTC
COPY scripts/get_webrtc.sh scripts/get_webrtc.sh
RUN ./scripts/get_webrtc.sh 108.5359.5.0 android /root /root
## Copy resources
COPY src src
COPY CMakeLists.txt ./
RUN cmake -B build \
-DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK_HOME}/build/cmake/android.toolchain.cmake" \
-DANDROID_ABI=armeabi-v7a \
-DANDROID_NATIVE_API_LEVEL=16 \
-DBUILD_SHARED_LIBS=OFF \
-DWEBRTC_INCLUDE_DIR=/root/webrtc/include \
-DWEBRTC_LIBRARY=/root/webrtc/lib/armeabi-v7a/libwebrtc.a \
-DCMAKE_BUILD_TYPE=Release
RUN cmake --build build --config Release --parallel $(nproc) --target AndroidWebRTC
src/main.cpp
#include <iostream>
#include <api/create_peerconnection_factory.h>
int main() {
auto googleSessionDescription = webrtc::CreateSessionDescription(
webrtc::SdpType::kOffer, "sdp");
std::cout << "Hello, World!" << std::endl;
return 0;
}
scripts/get_webrtc.sh
#!/bin/bash
if [ $# -lt 4 ]; then
echo "$0 <webrtc_build_version> <package_name> <output_dir> <source_dir>"
exit 1
fi
WEBRTC_BUILD_VERSION=$1
PACKAGE_NAME=$2
OUTPUT_DIR=$3
SOURCE_DIR=$4
set -ex
if [ ! -e $SOURCE_DIR/webrtc.${PACKAGE_NAME}.${WEBRTC_BUILD_VERSION}.tar.gz ]; then
curl -Lo $SOURCE_DIR/webrtc.${PACKAGE_NAME}.${WEBRTC_BUILD_VERSION}.tar.gz https://github.com/shiguredo-webrtc-build/webrtc-build/releases/download/m${WEBRTC_BUILD_VERSION}/webrtc.${PACKAGE_NAME}.tar.gz
fi
pushd $OUTPUT_DIR
tar xf $SOURCE_DIR/webrtc.${PACKAGE_NAME}.${WEBRTC_BUILD_VERSION}.tar.gz
popd
scripts/get_cmake.sh
#!/bin/bash
# <platform> には Linux か Darwin を指定する
# <output_dir>/cmake に CMake が配置される
if [ $# -lt 3 ]; then
echo "$0 <cmake_version> <platform> <output_dir>"
exit 1
fi
CMAKE_VERSION=$1
PLATFORM=$2
OUTPUT_DIR=$3
set -ex
pushd $OUTPUT_DIR
curl -LO https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64.tar.gz
tar xf cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64.tar.gz
rm cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64.tar.gz
rm -rf cmake
mv cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64 cmake
popd
I'm writing a library in C++ that implements a few different coroutine primitives, and the library is targeted at the newly released C++20. As a result, it also makes use of things like concepts that were added to the language in C++20.
I want to use github actions to build the library, but builds are failing because ubuntu-latest uses GCC 9 and CLang 9, but my library requires at least GCC 10 or Clang 10 to build.
I attempted to configure the build action by setting -DCMAKE_CXX_COMPILER=g++-10, but the action fails in the Configure CMake phase because g++-10 can't be found on the system.
Is there any way specify github actions should use GCC 10 or Clang 10?
This is the most recent workflow file I tried running:
name: CMake
on: [push]
env:
# Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
BUILD_TYPE: Release
jobs:
build:
# The CMake configure and build commands are platform agnostic and should work equally
# well on Windows or Mac. You can convert this to a matrix build if you need
# cross-platform coverage.
# See: https://docs.github.com/en/actions/configuring-and-managing-workflows/configuring-a-workflow#configuring-a-build-matrix
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Create Build Environment
# Some projects don't allow in-source building, so create a separate build directory
# We'll use this as our working directory for all subsequent commands
run: cmake -E make_directory ${{runner.workspace}}/build
- name: Configure CMake
# Use a bash shell so we can use the same syntax for environment variable
# access regardless of the host operating system
shell: bash
working-directory: ${{runner.workspace}}/build
# Note the current convention is to use the -S and -B options here to specify source
# and build directories, but this is only available with CMake 3.13 and higher.
# The CMake binaries on the Github Actions machines are (as of this writing) 3.12
run: cmake $GITHUB_WORKSPACE -DCMAKE_BUILD_TYPE=$BUILD_TYPE -DCMAKE_CXX_COMPILER=g++-10
- name: Build
working-directory: ${{runner.workspace}}/build
shell: bash
# Execute the build. You can specify a specific target with "--target <NAME>"
run: cmake --build . --config $BUILD_TYPE
- name: Test
working-directory: ${{runner.workspace}}/build
shell: bash
# Execute tests defined by the CMake configuration.
# See https://cmake.org/cmake/help/latest/manual/ctest.1.html for more detail
run: ctest -C $BUILD_TYPE
And this is the point where it fails:
Run cmake $GITHUB_WORKSPACE -DCMAKE_BUILD_TYPE=$BUILD_TYPE -DCMAKE_CXX_COMPILER=g++-10
-- The CXX compiler identification is unknown
CMake Error at CMakeLists.txt:2 (project):
The CMAKE_CXX_COMPILER:
g++-10
is not a full path and was not found in the PATH.
Tell CMake where to find the compiler by setting either the environment
variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
to the compiler, or to the compiler name if it is in the PATH.
-- Configuring incomplete, errors occurred!
See also "/home/runner/work/conduit/build/CMakeFiles/CMakeOutput.log".
See also "/home/runner/work/conduit/build/CMakeFiles/CMakeError.log".
##[error]Process completed with exit code 1.
As Some programmer dude mentioned, installing g++ from apt is the way to go (unless it's installed by default); adds a minute or two to the build. Then you can tell cmake which compiler it should use by passing CC and CXX variables during configure step:
- run: |
sudo apt update
sudo apt install gcc-10 g++-10
shell: bash
# ... #
- run: cmake $GITHUB_WORKSPACE -DCMAKE_BUILD_TYPE=$BUILD_TYPE
shell: bash
env:
CC: gcc-10
CXX: g++-10
Same solution applies when you want to use clang.
I am using both gcc-9 and clang-10 for C (only) project.
- name: Setup dependencies
if: startsWith(matrix.os, 'ubuntu')
run: |
sudo apt-get install -y gcc-9 llvm-10 clang-10
sudo update-alternatives \
--install /usr/bin/gcc gcc /usr/bin/gcc-9 100 \
--slave /usr/bin/gcc-ar gcc-ar /usr/bin/gcc-ar-9 \
--slave /usr/bin/gcc-ranlib gcc-ranlib /usr/bin/gcc-ranlib-9 \
--slave /usr/bin/gcov gcov /usr/bin/gcov-9
sudo update-alternatives \
--install /usr/bin/llvm-ar llvm-ar /usr/bin/llvm-ar-10 100 \
--slave /usr/bin/llvm-ranlib llvm-ranlib /usr/bin/llvm-ranlib-10 \
--slave /usr/bin/llvm-cov llvm-cov /usr/bin/llvm-cov-10
sudo update-alternatives \
--install /usr/bin/clang clang /usr/bin/clang-10 100
PS you need to update more alternatives for C++ project, just example.
You can see what's installed by visiting https://github.com/actions/virtual-environments.
If you're trying this in 2022, now ubuntu-latest has "GNU C++ 9.3.0, 10.3.0". g++ is linked to version 9, but g++-10 is available on the PATH without any extra installation steps.
When I am doing a cmake in the build directory of the project I am getting this error. Initially I got a
protobuf-config.cmake not found
error. So I gave a path of the protobuf-config.cmake file to Protobuf_DIR. Later it started to show this new error:
CMake Error at
/opt/cmake/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137
(message): Could NOT find Protobuf (missing:
Protobuf_PROTOC_EXECUTABLE)
(found suitable version "3.6.1", minimum required is "3.0.0")
I am also attaching the error log file:
https://drive.google.com/open?id=1y7BZ6lDBtxvla7r-o188xM_FjwLqwhCx
I am doing this on Ubuntu-18 with cmake version: 3.13 and protobuf version: 3.6.1
You probably don't have the Protobuf compiler and development files installed. To fix that, run this command:
sudo apt-get install protobuf-compiler libprotobuf-dev
Alternatively, if you're building Protobuf by hand, you can't build it with the build type as RelWithDebInfo because that causes issues with the library and CMake.
Installed from apt on Ubuntu 20.04, dont have permissions to /usr/include/google
To fix: sudo chmod +Xr -R /usr/include/google
Default repositories usually contain outdated protobuf version. It is best to install it manually, from sources:
git clone --progress -b v3.10.0 https://github.com/protocolbuffers/protobuf && \
( \
cd protobuf; \
mkdir build; \
cd build; \
cmake ../cmake \
-DCMAKE_BUILD_TYPE=Release \
-Dprotobuf_BUILD_SHARED_LIBS=ON \
-Dprotobuf_BUILD_TESTS=OFF; \
make -j4 install; \
) && \
rm -rf protobuf
Quickly adding here that after installing Protobuf following this answer, I had to delete the build folder in my workspace to get cmake to run without this error :)
Hy,
list your protobuf libraries with sudo apt list | grep protobuf it should tell you what it will install by default. Run protoc --version so that you see what is recognized by default now. And after that get a version from github if needed build it and install it (this should not take to long). Then run protoc --version again.
From the parquet-cpp home page:
By default, Parquet links to Arrow's shared libraries. If you wish to statically-link the Arrow symbols instead, pass -DPARQUET_ARROW_LINKAGE=static.
I do want to statically link Arrow, because I want to use my program on other servers that won't have Arrow installed. I tried -DPARQUET_ARROW_LINKAGE=static, but I get an error about "missing transitive dependencies":
# cmake -DPARQUET_BUILD_TESTS=Off -DCMAKE_BUILD_TYPE=Release -DPARQUET_MINIMAL_DEPENDENCY=ON -DPARQUET_ARROW_LINKAGE=static .
-- The C compiler identification is GNU 4.8.5
...
-- [ /usr/local/share/cmake-3.9/Modules/FindBoost.cmake:1717 ] Boost_FOUND = 1
-- Boost version: 1.55.0
...
-- THRIFT_HOME:
-- Thrift compiler/libraries NOT found: (THRIFT_INCLUDE_DIR-NOTFOUND, THRIFT_STATIC_LIB-NOTFOUND). Looked in system search paths.
-- Thrift include dir: /root/tmp/parquet-cpp-master/thrift_ep/src/thrift_ep-install/include
-- Thrift static library: /root/tmp/parquet-cpp-master/thrift_ep/src/thrift_ep-install/lib/libthrift.a
-- Thrift compiler: /root/tmp/parquet-cpp-master/thrift_ep/src/thrift_ep-install/bin/thrift
-- Checking for module 'arrow'
-- No package 'arrow' found
-- Could not find the Arrow library. Looked for headers in , and for libs in
-- Building Apache Arrow from commit: 501d60e918bd4d10c429ab34e0b8e8a87dffb732
-- CMAKE_CXX_FLAGS: -O3 -DNDEBUG -Wall -std=c++11
-- Found cpplint executable at /root/tmp/parquet-cpp-master/build-support/cpplint.py
CMake Error at CMakeLists.txt:515 (message):
Missing transitive dependencies for Arrow static linking
So I found the code that generates the error:
if (NOT DEFINED ENV{BROTLI_STATIC_LIB_ENC} OR
NOT DEFINED ENV{BROTLI_STATIC_LIB_DEC} OR
NOT DEFINED ENV{BROTLI_STATIC_LIB_COMMON} OR
NOT DEFINED ENV{SNAPPY_STATIC_LIB} OR
NOT DEFINED ENV{ZLIB_STATIC_LIB} OR
NOT DEFINED ENV{LZ4_STATIC_LIB} OR
NOT DEFINED ENV{ZSTD_STATIC_LIB})
message(FATAL_ERROR "Missing transitive dependencies for Arrow static linking")
But that doesn't really help me, since I don't know what to do to get those environment variable defined.
Do I need to compile Arrow and install myself first? (I would rather have parquet-cpp do it for me.)
I arranged a script to download dependencies sources, set the environment variables and run your cmake line at the end. Just change the DEPDIR variable value, setting it to a directory of choice.
#!/bin/bash
CMKDIR=$PWD
DEPDIR=/tmp
cd $DEPDIR
#snappy
git clone https://github.com/google/snappy.git
cd snappy
mkdir build
cd build
cmake ..
make
export SNAPPY_STATIC_LIB=$DEPDIR/snappy/build/libsnappy.a
cd $DEPDIR
#brotli
git clone https://github.com/google/brotli.git
cd brotli
mkdir out
cd out
../configure-cmake
make
export BROTLI_STATIC_LIB_ENC=$DEPDIR/brotli/out/libbrotlienc-static.a
export BROTLI_STATIC_LIB_DEC=$DEPDIR/brotli/out/libbrotlidec-static.a
export BROTLI_STATIC_LIB_COMMON=$DEPDIR/brotli/out/libbrotlicommon-static.a
cd $DEPDIR
#zlib
git clone https://github.com/madler/zlib.git
cd zlib
./configure
make
export ZLIB_STATIC_LIB=$DEPDIR/zlib/libz.a
cd $DEPDIR
#lz4
git clone https://github.com/lz4/lz4.git
cd lz4
make
export LZ4_STATIC_LIB=$DEPDIR/lz4/lib/liblz4.a
cd $DEPDIR
#zstd
git clone https://github.com/facebook/zstd.git
cd zstd
make
export ZSTD_STATIC_LIB=$DEPDIR/zstd/lib/libzstd.a
cd $CMKDIR
cmake -DPARQUET_BUILD_TESTS=Off -DCMAKE_BUILD_TYPE=Release -DPARQUET_MINIMAL_DEPENDENCY=ON -DPARQUET_ARROW_LINKAGE=static
This script is very simple but should be effective. Just copy it in a new file (in the same CMakeLists.txt directory), give the file the execute permissions (i.e. sudo chmod +x filename) and execute it like this:
./filename.sh
About the fPIC option issue, you have to edit some files:
snappy: add this line in CMakeLists.txt, at the beginning, after the first two lines:
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
lz4 and zstd: edit the Makefile in lib sub-directory, after this line
CFLAGS += $(DEBUGFLAGS) $(MOREFLAGS)
add this line:
CFLAGS += -fPIC
zlib: edit the Makefile, after this line
CFLAGS=-O3 -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN
add this line:
CFLAGS += -fPIC
brotli: as far as I can see from make output, the option is already set.
Before running make again, execute this script:
#!/bin/bash
DEPDIR=/tmp
cd $DEPDIR/snappy/build
cmake ..
make clean
make
cd $DEPDIR/lz4
make clean
make
cd $DEPDIR/zstd
make clean
make
I've tried to build clang-tidy from sources but it complains about an undefined CMake command:
CMake Error at clang-apply-replacements/CMakeLists.txt:5 (add_clang_library):
Unknown CMake command "add_clang_library".
CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as
cmake_minimum_required(VERSION 3.9)
should be added at the top of the file. The version specified may be lower
if you wish to support older CMake versions for this project. For more
information run "cmake --help-policy CMP0000".
This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring incomplete, errors occurred!
How can I build clang-tidy or, alternatively, how can I install the latest version on macOS?
Up-to-date steps:
git clone https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" ../llvm
make install clang-tidy
Reference, ninja, and other details: my own blog post.
EDIT: this answer is out of date — the LLVM project has moved to a single git repository at https://github.com/llvm/llvm-project. See answers below for updated instructions.
clang-tidy is intended to be built inside a checkout of llvm/clang, and depends on CMake macros from the llvm project. You should check out the llvm repo, then the clang repo inside llvm/tools/clang, then the clang-tools-extra repo inside llvm/tools/clang/tools/extra. Then you can run CMake on the top-level directory, and make clang-tidy should work.
If you're not interested in building it yourself, it looks like the Homebrew formula for LLVM also includes the extra tools: https://github.com/Homebrew/homebrew-core/blob/382d3defb5bc48ce2dccd17261be70c4ada9a124/Formula/llvm.rb#L181
I had same problem as Per Mildner. Got is solved with slightly modified code YvesgereY posted (I don't have enough reputation to post a comment to that answer, hence a new answer instead).
In short, I added -G "Unix Makefiles" to cmake. Without this option, no makefile will be generated. Also, I used -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;". It didn't work when just clang-tools-extra was specified.
Here is the whole snippet:
git clone https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build
cd build
cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;" ../llvm
make -j8 install-clang-tidy
#jtbandes: Thank you for the information.
I'd like to share these explicit steps for us noobs:
1. Download the released sources from LLVM Download Page
LLVM source code -> Links to the file llvm-6.0.0.src.tar.xz
Clang source code -> Links to the file cfe-6.0.0.src.tar.xz
clang-tools-extra -> Links to the file clang-tools-extra-6.0.0.src.tar.xz
2. Detar each of these into the proper directory:
$ tar -zxvf <download_dir_path>/llvm-6.0.1.src.tar.xz
$ cd llvm-6.0.1.src/tools
$ tar -zxcf <download_dir_path>/cfe-6.0.1.src.tar.xz
$ cd llvm-6.0.1.src/tools/cfe-6.0.1.src/tools
$ tar -zxvf <download_dir_path>/clang-tools-extra-6.0.1.src.tar.xz
Results in a directory llvm-6.0.1.src/tools/cfe-6.0.1.src/tools/clang-tools-extra-6.0.1.src/clang-tidy; Which is incorrect. The lang-tools-extra-6.0.1.src needs to be renamed to extra (as mentioned by #jtbandes).
3. So rename it or provide a symbolic link:
$ cd llvm-6.0.1.src/tools/cfe-6.0.1.src/tools
$ mv clang-tools-extra-6.0.1.src extra
or
$ ln -s clang-tools-extra-6.0.1.src extra
The path llvm-6.0.1.src/tools/cfe-6.0.1.src/tools/extra/clang-tidy should now be valid
4. Build it:
$ cd llvm-6.0.1.src
$ mkdir build
$ cd build
$ cmake ..
$ make
Everything should make without errors or warnings.
5. Build Output:
The build output can be found in llvm-6.0.1.src/build/bin.
For everyone who are looking for latest (LLVM 11) Windows build instructions (ensure CMake, Visual Studio 2019 and git are installed and set in PATH):
git clone --config core.autocrlf=false https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build
cd build
cmake -G "Visual Studio 16 2019" -Thost=x64 -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" ../llvm
cmake --build . --target clang-tidy --config RelWithDebInfo --parallel
cmake --build . --target clang-query --config RelWithDebInfo --parallel
This worked for me:
mkdir build
files="
llvm-12.0.1.src.tar.xz
clang-12.0.1.src.tar.xz
clang-tools-extra-12.0.1.src.tar.xz
"
for f in $files; do
echo "Untar $f"
tar xf $f
done
mv llvm-12.0.1.src llvm
mv clang-12.0.1.src llvm/tools/clang
mv clang-tools-extra-12.0.1.src llvm/tools/clang/tools/extra
cd build
cmake -G "Unix Makefiles" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr/local/llvm \
-DCLANG_BUILD_TOOLS=ON \
../llvm
make -j16 install
As of LLVM 14.0.0, sparse checkouts do no longer work (at least temporarily) and the top-level directory contains no CMakeLists.txt. I believe the tree layout has changed after LLVM 13.0.1. In consequence, none of the approaches here worked without quite some modification.
Here is how you can build version 15.0.0git (the most recent at the time of this writing). See related issue: https://github.com/llvm/llvm-project/issues/53281.
First, get the compressed code or clone with git (slower)
$ wget "https://github.com/llvm/llvm-project/archive/refs/heads/main.zip" -O llvm.zip
$ unzip llvm.zip
As usual, create a build directory and run cmake in the llvm directory.
$ mkdir /build
$ cd /build
$ cmake -G "Unix Makefiles" \
-DCMAKE_INSTALL_PREFIX=/usr/local/llvm \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" \
/llvm-project-main/llvm
Navigate downwards in the generated files and only build clang-tidy.
$ cd /build/tools/clang/tools/extra/clang-tidy
$ make install
Cmake will install to /usr/local/llvm. Also, if you want to check out a specific version, use tags in the first step like this:
$ wget "https://github.com/llvm/llvm-project/archive/refs/tags/llvmorg-14.0.0.zip"
Note that you need to supply the matching builtin headers for running clang-tidy, which are located in GitHub under llvm-project/clang/lib/Headers and can by pointed to with -extra-arg=-I/path/to/builtin/headers.