I'm trying to build a docker container using the following Dockerfile:
FROM fedora:29
RUN dnf -y update && dnf install -y file gcc gcc-c++ git make wget which libtool python3-pip redhat-rpm-config python3-devel zlib-devel libstdc++ openmpi-devel
RUN cd /tmp && \
wget http://www.mpich.org/static/downloads/3.3/mpich-3.3.tar.gz && \
gzip -dc mpich-3.3.tar.gz | tar xf - && \
cd mpich-3.3 && \
./configure --disable-fortran --prefix=/usr/mpich-3.3 && \
make && \
make install
ENV PATH /usr/mpich-3.3/bin:${PATH}
ENV LD_LIBRARY_PATH /usr/mpich-3.3/lib:${LD_LIBRARY_PATH}
RUN cd /usr && git clone https://github.com/Dowell-Lab/FStitch
RUN cd /usr/FStitch/src && make clean && make
RUN pip3 install FStitch-Bidir --user
ENV PATH /usr/FStitch/src:${PATH}
ENV PATH /root/.local/bin:${PATH}
RUN cd /usr && git clone https://github.com/Dowell-Lab/Tfit
RUN cd /usr/Tfit/src && make clean && make
ENV PATH /usr/Tfit/src:${PATH}
CMD /bin/bash
Where the projects I'm trying to clone are written in c++11 and the second of the two (Tfit) requires openmpi/mpich. The first program compiles successfully, but with the second, I'm getting the following error in the last step of the compiler:
/usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
make: *** [Makefile:20: NU_FIT] Error 1
I searched and found these two links:
cpp: usr/bin/ld: cannot find -l<nameOfTheLibrary>
usr/bin/ld: cannot find -l<nameOfTheLibrary>
But neither of these quite address the problem as I'm guessing I'm just missing a dependency/symlink to a library, but I'm unsure how to achieve this in the build. I can compile this successfully locally, but I have to module load mpi/openmpi-x86_64 to do so. My guess is that this is an openmpi issue when setting the LD_LIBRARY_PATH, but am unsure of how to resolve this in the Docker build.
The first few lines of the Makefile are as follows:
CXX = mpic++
CXXFLAGS = -static-libstdc++ -static-libgcc -Wno-unused-variable -Wno-non-virtual-dtor -std=c++11 -fopenmp -Wno-write-strings -Wno-literal-suffix -D_GLIBCXX_USE_CXX11_ABI=0 -g
EXEC = ${PWD}/Tfit
ARCH = getconf LONG_BIT
CPP_FLAGS_32 = -D32_BIT
CPP_FLAGS_64 = -D64_BIT
GCCVERSION = $(shell ${CXX} -dumpversion)
NU_FIT: main.o load.o split.o model.o across_segments.o template_matching.o \
read_in_parameters.o model_selection.o error_stdo_logging.o \
MPI_comm.o density_profiler.o bootstrap.o prelim_main.o model_main.o select_main.o FDR.o BIC.o ParamWrapper.o old_template_matching.o
#printf "linking : "
#${CXX} ${CXXFLAGS} ${PWD}/main.o ${PWD}/load.o ${PWD}/model_selection.o \
${PWD}/split.o ${PWD}/model.o ${PWD}/across_segments.o \
${PWD}/template_matching.o ${PWD}/read_in_parameters.o \
${PWD}/MPI_comm.o \
${PWD}/bootstrap.o ${PWD}/density_profiler.o \
${PWD}/prelim_main.o ${PWD}/model_main.o ${PWD}/BIC.o ${PWD}/FDR.o \
${PWD}/select_main.o ${PWD}/error_stdo_logging.o ${PWD}/ParamWrapper.o ${PWD}/old_template_matching.o -o ${EXEC} -lmpi
#cp ${PWD}/Tfit ${PWD}/EMGU
Any help is appreciated!
Try add libstdc++-static in your list of dnf install. (https://github.com/numenta/nupic/issues/1901)
Related
Building WebRTC directly is somewhat painful (& rather time consuming so not ideal in paid for CI environments) so I've opted to use some excellent precompiled static libraries released on GitHub to speed things up. This has worked perfectly for all platforms I'm targetting (Windows, MacOS, iOS, Apple TVOS, linux...) except for Android.
When linking the binary on Android I encounter several duplicate symbol errors related to the std libary:
ld: error: duplicate symbol: std::logic_error::logic_error(char const*)
>>> defined at stdexcept_default.ipp:24 (../../../../../_source/android/webrtc/src/buildtools/third_party/libc++/trunk/src/support/runtime/stdexcept_default.ipp:24)
>>> stdexcept.o:(std::logic_error::logic_error(char const*)) in archive /root/webrtc/lib/armeabi-v7a/libwebrtc.a
>>> defined at stdexcept_default.ipp:24 (/buildbot/src/android/ndk-release-r23/toolchain/llvm-project/libcxx/src/support/runtime/stdexcept_default.ipp:24)
>>> stdexcept.o:(.text._ZNSt11logic_errorC2EPKc+0x1) in archive /app/android-sdk-linux/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libc++_static.a
...
ld: error: duplicate symbol: std::runtime_error::runtime_error(char const*)
>>> defined at stdexcept_default.ipp:35 (../../../../../_source/android/webrtc/src/buildtools/third_party/libc++/trunk/src/support/runtime/stdexcept_default.ipp:35)
>>> stdexcept.o:(std::runtime_error::runtime_error(char const*)) in archive /root/webrtc/lib/armeabi-v7a/libwebrtc.a
>>> defined at stdexcept_default.ipp:35 (/buildbot/src/android/ndk-release-r23/toolchain/llvm-project/libcxx/src/support/runtime/stdexcept_default.ipp:35)
>>> stdexcept.o:(.text._ZNSt13runtime_errorC2EPKc+0x1) in archive /app/android-sdk-linux/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libc++_static.a
It appears that libwebrtc.a for Android includes some libc++ object files and I'm not entirely sure how to successfully link with this library here.
This might be coming down to the libwebrtc.a being compiled with Google's own patched libc++ (Based on what I've read online I believe Google contributes patches to upstream libc++ but they don't wait for the patches to go in - rather they use their fully patched version of libc++ for building WebRTC, Chrome, etc).
When compiling for Linux I actually use the exact versions of clang & libc++ that were downloaded & used in building the static library & I suspect I might need to be doing something similar here - I'm just not entirely sure how given the need to use the Android CMake toolchain.
I've put together a minimal example project on GitHub with Dockerfile that can be used to easily reproduce the above issue.
Does anyone have an idea how I can successfully link the library on Android?
For completeness the source files of interest are:
CMakeLists.txt
cmake_minimum_required(VERSION 3.23)
project(AndroidWebRTC)
set(CMAKE_CXX_STANDARD 17)
add_executable(AndroidWebRTC src/main.cpp)
target_link_libraries(AndroidWebRTC PRIVATE ${WEBRTC_LIBRARY})
add_compile_definitions(WEBRTC_POSIX WEBRTC_ANDROID WEBRTC_LINUX)
include_directories(
${WEBRTC_INCLUDE_DIR}
${WEBRTC_INCLUDE_DIR}/third_party/abseil-cpp
${WEBRTC_INCLUDE_DIR}/third_party/boringssl/src/include
${WEBRTC_INCLUDE_DIR}/third_party/libyuv/include
${WEBRTC_INCLUDE_DIR}/third_party/zlib
)
Dockerfile
# I explicitly specify an amd64 platform as I often build on Apple Silicon
# machines where the default of arm64 breaks things. I believe the alternative
# (and possibly better option) is to use NDK 24 and above, see:
# https://stackoverflow.com/a/69541958
FROM --platform=linux/amd64 ubuntu:20.04 AS build
ENV ANDROID_SDK_ROOT /app/android-sdk-linux
WORKDIR /app
# Install dependencies to build the project
# include apt-get --no-install-recommends
RUN apt-get update && \
apt-get -y upgrade && \
apt-get --no-install-recommends -y install tzdata && \
echo 'Europe/London' > /etc/timezone && \
dpkg-reconfigure -f noninteractive tzdata
RUN apt-get --no-install-recommends -y install git lsb-release python rsync \
emacs wget build-essential sudo pkg-config clang unzip openjdk-8-jdk ant \
android-sdk-platform-tools-common libncurses5 curl
RUN mkdir ${ANDROID_SDK_ROOT}
# ------------------------------------------------------
# --- Android SDK
RUN wget https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip && \
unzip commandlinetools-linux-7583922_latest.zip && \
mkdir ${ANDROID_SDK_ROOT}/cmdline-tools/ &&\
mv cmdline-tools ${ANDROID_SDK_ROOT}/cmdline-tools/latest && \
rm commandlinetools-linux-7583922_latest.zip
ENV PATH ${PATH}:${ANDROID_SDK_ROOT}/cmdline-tools/latest/bin
RUN yes | sdkmanager --licenses
RUN touch /root/.android/repositories.cfg
# Platform tools
RUN yes | sdkmanager "platform-tools"
RUN yes | sdkmanager --update --channel=0
# Keep all sections in descending order!
RUN yes | sdkmanager \
"platforms;android-30" \
"build-tools;31.0.0" \
"ndk;23.1.7779620" \
"extras;android;m2repository" \
"extras;google;m2repository" \
"extras;google;google_play_services" \
"add-ons;addon-google_apis-google-24"
ENV ANDROID_HOME ${ANDROID_SDK_ROOT}
ENV ANDROID_NDK_HOME=${ANDROID_SDK_ROOT}/ndk/23.1.7779620
ENV PATH ${PATH}:${ANDROID_NDK_HOME}:${ANDROID_HOME}/build-tools/31.0.0/
# Get CMake
COPY scripts/get_cmake.sh scripts/get_cmake.sh
RUN ./scripts/get_cmake.sh "3.25.1" linux /root
ENV PATH "/root/cmake/bin:$PATH"
# Get WebRTC
COPY scripts/get_webrtc.sh scripts/get_webrtc.sh
RUN ./scripts/get_webrtc.sh 108.5359.5.0 android /root /root
## Copy resources
COPY src src
COPY CMakeLists.txt ./
RUN cmake -B build \
-DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK_HOME}/build/cmake/android.toolchain.cmake" \
-DANDROID_ABI=armeabi-v7a \
-DANDROID_NATIVE_API_LEVEL=16 \
-DBUILD_SHARED_LIBS=OFF \
-DWEBRTC_INCLUDE_DIR=/root/webrtc/include \
-DWEBRTC_LIBRARY=/root/webrtc/lib/armeabi-v7a/libwebrtc.a \
-DCMAKE_BUILD_TYPE=Release
RUN cmake --build build --config Release --parallel $(nproc) --target AndroidWebRTC
src/main.cpp
#include <iostream>
#include <api/create_peerconnection_factory.h>
int main() {
auto googleSessionDescription = webrtc::CreateSessionDescription(
webrtc::SdpType::kOffer, "sdp");
std::cout << "Hello, World!" << std::endl;
return 0;
}
scripts/get_webrtc.sh
#!/bin/bash
if [ $# -lt 4 ]; then
echo "$0 <webrtc_build_version> <package_name> <output_dir> <source_dir>"
exit 1
fi
WEBRTC_BUILD_VERSION=$1
PACKAGE_NAME=$2
OUTPUT_DIR=$3
SOURCE_DIR=$4
set -ex
if [ ! -e $SOURCE_DIR/webrtc.${PACKAGE_NAME}.${WEBRTC_BUILD_VERSION}.tar.gz ]; then
curl -Lo $SOURCE_DIR/webrtc.${PACKAGE_NAME}.${WEBRTC_BUILD_VERSION}.tar.gz https://github.com/shiguredo-webrtc-build/webrtc-build/releases/download/m${WEBRTC_BUILD_VERSION}/webrtc.${PACKAGE_NAME}.tar.gz
fi
pushd $OUTPUT_DIR
tar xf $SOURCE_DIR/webrtc.${PACKAGE_NAME}.${WEBRTC_BUILD_VERSION}.tar.gz
popd
scripts/get_cmake.sh
#!/bin/bash
# <platform> には Linux か Darwin を指定する
# <output_dir>/cmake に CMake が配置される
if [ $# -lt 3 ]; then
echo "$0 <cmake_version> <platform> <output_dir>"
exit 1
fi
CMAKE_VERSION=$1
PLATFORM=$2
OUTPUT_DIR=$3
set -ex
pushd $OUTPUT_DIR
curl -LO https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64.tar.gz
tar xf cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64.tar.gz
rm cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64.tar.gz
rm -rf cmake
mv cmake-${CMAKE_VERSION}-${PLATFORM}-x86_64 cmake
popd
So I am atempting to build the ARA-RISCV build and it requires a RISC-V LLVM toolchain and they give you a "make toolchain-llvm" command and it starts off ok but midway through the build around 2500/3500 it stops and the terminal closes without pointing out any error while it builds.
The command that are used are the ones below:
mkdir -p /.../rvv/install/riscv-llvm
cd /.../rvv/toolchain/riscv-llvm && rm -rf build && mkdir -p build && cd build && \
cmake -G Ninja \
-DCMAKE_INSTALL_PREFIX=/home/alex/Documents/Project/rvv/install/riscv-llvm \
-DLLVM_ENABLE_PROJECTS="clang;lld" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=gcc \
-DCMAKE_CXX_COMPILER=g++ \
-DLLVM_DEFAULT_TARGET_TRIPLE=riscv64-unknown-elf \
-DLLVM_TARGETS_TO_BUILD="RISCV" \
../llvm
cd /.../rvv/toolchain/riscv-llvm && \
cmake --build build --target install
After everything the build is incomplete and a lot of files have not been downloaded.I dont know what is causing this can anybody help?
Please and thank you.
EDIT:It worked by adding -j8 to the last instruction aka
cmake --build build --target install -j8
I want to build and run a c++ opencv code using docker. Here is my dockerfile:
FROM nvidia/cuda:11.5.0-cudnn8-runtime-ubuntu20.04
FROM ubuntu
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y \
g++ git wget cmake sudo
RUN apt-get install -y \
build-essential cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev \
python3-dev python3-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev \
libcanberra-gtk-module libcanberra-gtk3-module
RUN git clone https://github.com/opencv/opencv.git && \
cd /opencv && mkdir build && cd build && \
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local .. && \
make -j"$(nproc)" && \
make install && ldconfig
The above commands makes my opencv libs, but I don't how to use it to run the actual code. I added this two lines at the end of the dockerfile (wav.cpp is the name of my cpp file that I want to run):
COPY . .
RUN g++ -o wav wav.cpp
But at the end I get this error, which obviously says it can't find the opencv headers.
wav.cpp:2:10: fatal error: opencv2/imgproc.hpp: No such file or
directory
2 | #include "opencv2/imgproc.hpp"
| ^~~~~~~~~~~~~~~~~~~~~ compilation terminated.
Now how should I resolve this header (and lib) dependency problem?
Thank you.
You didn't specify the location for the headers and which libraries should be linked by gcc. Please take a look at the manual of gcc/g++ for the flags -I and -L. Should be something like this:
RUN g++ -o wav wav.cpp -I <opencv header location> -L <opencv libs location> -lopencv_core ....
Using #emrhzc's answer I could build my code inside the dockerfile. Now my final working command is:
RUN g++ -o wav wav.cpp `pkg-config --cflags --libs opencv4`
I'm attempting to compile a C++ library called VRPN with musl instead of glibc, and running into linker errors.
Dockerfile
# This build container compiles fully static binaries for alpine
FROM ekidd/rust-musl-builder:1.49.0 AS build
ENV CC=musl-gcc
ENV CXX=musl-g++
ENV CFLAGS=-static
ENV CXXFLAGS=-static
USER root
# Install build dependencies
RUN apt-get update && apt-get install -y cmake
# Install VRPN (VR peripheral device network)
WORKDIR /usr/local/src
RUN git clone --depth 1 --branch v07.34 https://github.com/vrpn/vrpn.git
RUN cd vrpn && git submodule update --init --recursive
RUN mkdir vrpn_Build
WORKDIR /usr/local/src/vrpn_Build
RUN cmake -DCMAKE_BUILD_TYPE=RELEASE \
-DVRPN_USE_GPM_MOUSE=OFF \
-DVRPN_BUILD_CLIENT_LIBRARY=OFF \
-DVRPN_BUILD_JAVA=OFF \
-DVRPN_BUILD_PYTHON_HANDCODED_3X=OFF \
-DVRPN_BUILD_SERVERS=OFF \
-DVRPN_USE_LIBUSB_1_0=OFF \
-DVRPN_USE_DEV_INPUT=OFF \
-DVRPN_USE_HID=OFF \
-DVRPN_USE_I2CDEV=OFF \
-DVRPN_USE_JOYLIN=OFF \
-DVRPN_USE_LIBUSB_1_0=OFF \
../vrpn
RUN make VERBOSE=1
Save the above to a file and run docker build . to get the following error (after lots of successful building):
Output
[ 72%] Linking CXX executable time_test
/usr/bin/cmake -E cmake_link_script CMakeFiles/time_test.dir/link.txt --verbose=1
/usr/bin/musl-g++ -static -fPIC -O3 -DNDEBUG -rdynamic CMakeFiles/time_test.dir/time_test.cpp.o -o time_test -L/usr/lib/x86_64-linux-musl libvrpnserver.a quat/libquat.a -lm atmellib/libvrpn_atmel.a gpsnmealib/libgpsnmea.a -Wl,-Bstatic -lgcc -lgcc_eh
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x12): undefined reference to `__libc_csu_fini'
(.text+0x19): undefined reference to `__libc_csu_init'
CMakeFiles/time_test.dir/time_test.cpp.o: In function `main':
time_test.cpp:(.text.startup+0x45): undefined reference to `__printf_chk'
time_test.cpp:(.text.startup+0xee): undefined reference to `__printf_chk'
time_test.cpp:(.text.startup+0x120): undefined reference to `__printf_chk'
collect2: error: ld returned 1 exit status
CMakeFiles/time_test.dir/build.make:100: recipe for target 'time_test' failed
make[2]: Leaving directory '/usr/local/src/vrpn_Build'
make[2]: *** [time_test] Error 1
CMakeFiles/Makefile2:973: recipe for target 'CMakeFiles/time_test.dir/all' failed
make[1]: Leaving directory '/usr/local/src/vrpn_Build'
make[1]: *** [CMakeFiles/time_test.dir/all] Error 2
Makefile:162: recipe for target 'all' failed
make: *** [all] Error 2
The command '/bin/sh -c make VERBOSE=1' returned a non-zero code: 2
It seems that for some reason, musl's glibc hasn't been linked correctly, but I've been unable to figure out why.
I've read that linking to pre-compiled objects is generally the cause of this type of issue, but in this case, I'm pretty confident that everything is being built from source.
I don't have much experience with compiler internals, and I'm having a hard time finding reading material about musl simple enough that I can understand. Any insight or pointers would be much appreciated.
Thanks!
I managed to figure this one out. I think I was using host libraries rather than a proper musl toolchain. I got it to work by building in the messense/rust-musl-cross docker image (created with musl-cross-make), which contains the gcc toolchain compiled for the x86_64-unknown-linux-musl target.
Here's my working docker build script for future reference.
# This build container compiles fully static binaries for alpine
FROM messense/rust-musl-cross:x86_64-musl AS build
USER root
# Install build dependencies
RUN apt-get update && apt-get install -y cmake
# Install VRPN (VR peripheral device network)
WORKDIR /usr/local/src
RUN git clone --depth 1 --branch v07.34 https://github.com/vrpn/vrpn.git
RUN cd vrpn && git submodule update --init --recursive
RUN mkdir vrpn_Build
WORKDIR /usr/local/src/vrpn_Build
RUN set -x
ENV TARGET=x86_64-unknown-linux-musl
ENV CC=$TARGET-gcc
ENV CXX=$TARGET-g++
RUN cmake -DCMAKE_BUILD_TYPE=Release \
-DVRPN_USE_GPM_MOUSE=OFF \
-DVRPN_BUILD_JAVA=OFF \
-DVRPN_BUILD_PYTHON_HANDCODED_3X=OFF \
-DVRPN_BUILD_SERVERS=OFF \
-DVRPN_BUILD_CLIENTS=OFF \
-DVRPN_USE_LIBUSB_1_0=OFF \
-DVRPN_USE_DEV_INPUT=OFF \
-DVRPN_USE_HID=OFF \
-DVRPN_USE_I2CDEV=OFF \
-DVRPN_USE_JOYLIN=OFF \
-DVRPN_USE_LIBUSB_1_0=OFF \
../vrpn
RUN make VERBOSE=1
Cheers!
I am trying to install caffe by building it from source
After issuing the following command from the caffe root directory
$ make all -j4
I am getting an error
...
CXX src/caffe/layer_factory.cpp
CXX src/caffe/blob.cpp
AR -o .build_release/lib/libcaffe.a
LD -o .build_release/lib/libcaffe.so.1.0.0
/usr/bin/x86_64-linux-gnu-ld: cannot find -lpython3.6
collect2: error: ld returned 1 exit status
Makefile:582: recipe for target '.build_release/lib/libcaffe.so.1.0.0' failed
make: *** [.build_release/lib/libcaffe.so.1.0.0] Error 1
Dependencies installed
$ sudo apt install python3-opencv
$ sudo apt-get install libatlas-base-dev
$ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
$ sudo apt-get install --no-install-recommends libboost-all-dev
$ sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
$ sudo apt-get install the python3-dev
CUDA: CUDA 9 CuDnn 7.4
Ubuntu: Ubuntu 18.04
Makefile.config
I have looked at all the issues in the source Github repository but couldn't find anything useful.
Therefore that your error states cannot find -lpython3.6 you are missing the libpython3.6.so on your system.
Try to:
sudo apt-get install libpython3.6-dev
The issue is resolved, I had to make the following changes in Makefile.config file
From
PYTHON_LIBRARIES := boost_python3 python3.6
PYTHON_INCLUDE := /usr/include/python3.6 \
/usr/lib/python3.6/dist-packages/numpy/core/include
To
PYTHON_LIBRARIES := boost_python3 python3.6m
PYTHON_INCLUDE := /usr/include/python3.6m \
/usr/lib/python3.6/dist-packages/numpy/core/include