I'm new to Linux, so sorry if this my question is really stupid.
I have shared cross-platform library project, which uses third party static library (Libtorrent).
Windows/Android/macOS - builds and works fine.
But I can't figure out how to build it in Linux (Ubuntu) with GCC.
I'm getting the following linker errors:
/usr/bin/ld: /home/user/Desktop/project/libtorrent-build/prebuilt/linux/lib/libtorrent-rasterbar.a(create_torrent.o): relocation R_X86_64_TPOFF32 against symbol `_ZN5boost4asio6detail15keyword_tss_ptrINS1_10call_stackINS1_15task_io_serviceENS1_27task_io_service_thread_infoEE7contextEE6value_E' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/user/Desktop/project/libtorrent-build/prebuilt/linux/lib/libtorrent-rasterbar.a(disk_io_thread.o): relocation R_X86_64_TPOFF32 against symbol `_ZN5boost4asio6detail15keyword_tss_ptrINS1_10call_stackINS1_15task_io_serviceENS1_27task_io_service_thread_infoEE7contextEE6value_E' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/user/Desktop/project/libtorrent-build/prebuilt/linux/lib/libtorrent-rasterbar.a(peer_connection.o): relocation R_X86_64_TPOFF32 against symbol `_ZN5boost4asio6detail15keyword_tss_ptrINS1_10call_stackINS1_15task_io_serviceENS1_27task_io_service_thread_infoEE7contextEE6value_E' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/user/Desktop/project/libtorrent-build/prebuilt/linux/lib/libtorrent-rasterbar.a(session.o): relocation R_X86_64_TPOFF32 against symbol `_ZN5boost4asio6detail15keyword_tss_ptrINS1_10call_stackINS1_15task_io_serviceENS1_27task_io_service_thread_infoEE7contextEE6value_E' can not be used when making a shared object; recompile with -fPIC
I've tried rebuilding Boost and Libtorrent with -fPIC flag: nothing is changed.
Boost build script:
#!/bin/bash
set -e
BOOST_VERSION=1.63.0
BOOST_VERSION_UNDERSCORES=${BOOST_VERSION//./_}
echo "Boost version $BOOST_VERSION"
echo "Downloading..."
BOOST_ARCH_NAME=boost_${BOOST_VERSION_UNDERSCORES}.tar.gz
curl -O -L https://dl.bintray.com/boostorg/release/$BOOST_VERSION/source/$BOOST_ARCH_NAME
echo "Extracting..."
tar -xvzf $BOOST_ARCH_NAME
BOOST_FOLDER_NAME=boost_${BOOST_VERSION_UNDERSCORES}
cd $BOOST_FOLDER_NAME
chmod +x bootstrap.sh
./bootstrap.sh
chmod +x b2
./b2 headers
./b2 \
--layout=versioned \
--with-thread \
--with-date_time \
--with-filesystem \
--with-chrono \
--with-random \
toolset=gcc \
threading=multi \
link=static \
runtime-link=shared \
variant=release \
threadapi=pthread \
debug-symbols=off \
warnings=off \
warnings-as-errors=off \
architecture=x86 \
address-model=64 \
--stagedir=stage/linux \
cxxflags="-std=gnu++0x -fPIC" \
stage
echo "Copying prebuilt..."
mkdir -p ../prebuilt/linux/lib
cp -rf ./stage/linux/lib ../prebuilt/linux/
cd ../
echo "Cleaning up..."
rm $BOOST_ARCH_NAME
rm -rf $BOOST_FOLDER_NAME
echo "Done."
Libtorrent build script:
#!/bin/sh
set -e
MY_PWD=$PWD
LIBTORRENT_VERSION=1_1_8
LIBTORRENT_NAME=libtorrent-$LIBTORRENT_VERSION
export LIBTORRENT_FOLDER_NAME=libtorrent-$LIBTORRENT_NAME
curl -O -L "https://github.com/arvidn/libtorrent/archive/${LIBTORRENT_NAME}.tar.gz"
echo "Extracting..."
rm -rf ./$LIBTORRENT_FOLDER_NAME
tar xfz "${LIBTORRENT_NAME}.tar.gz"
echo "Building for linux..."
cd $LIBTORRENT_FOLDER_NAME
. ./autotool.sh
mkdir -p lib
if [ ! -d "./include/boost" ]; then
ln -s $PWD/../../boost-build/prebuilt/include/boost $PWD/include/boost
ln -s $PWD/../../boost-build/prebuilt/linux/lib/* $PWD/lib/
fi
if [ ! -d "./include/openssl" ]; then
ln -s $PWD/../../openssl-build/prebuilt/include/openssl $PWD/include/openssl
ln -s $PWD/../../openssl-build/prebuilt/linux/lib/* $PWD/lib/
fi
export CXX=g++
export CC=gcc
export CXXFLAGS="-std=gnu++0x -fPIC -fPIE"
export CPPFLAGS="-I"$PWD"/include"
./configure --enable-debug=no --with-boost-python=no --enable-logging=no \
--enable-tests=no --with-openssl=$PWD \
--with-boost=$PWD \
--enable-shared=no \
--enable-encryption
make -j4
mkdir -p ../prebuilt/linux/lib
cp -rf ./src/.libs/libtorrent-rasterbar.a ../prebuilt/linux/lib/
cd $MY_PWD
echo "cleaning up"
rm -rf $LIBTORRENT_FOLDER_NAME
rm -rf ${LIBTORRENT_NAME}.tar.gz
echo "done"
My shared lib itself is a Qt-based library. I build it using Qt Creator, providing all the required headers and libs in .pro file.
The problem was in Libtorrent build script. I removed -fPIE option and added
export CFLAGS="-fPIC"
Rebuilt and now it links without errors.
Related
Bit of a "hole in my bucket problem", here.
To start from the top, I was compiling musl-libc when I got an error claiming "can not be used when making a PIE object; recompile with -fPIE". It was pointing to some crto library that would have been packaged with gcc.
Okay, so...I assume I would have to recompile gcc at this point? That's fine, I'll give it a shot.
Except, now I'm running into this same error while trying to recompile gcc.
EDIT (The Error, or at least some of it):
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: gcov.o: relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIE
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: json.o: relocation R_X86_64_32 against symbol `_ZTVN4json6objectE' can not be used when making a PIE object; recompile with -fPIE
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: hash-table.o: relocation R_X86_64_32S against symbol `prime_tab' can not be used when making a PIE object; recompile with -fPIE
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: ggc-none.o: relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIE
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: libcommon.a(diagnostic.o): relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIE
For ease, I've created a dockerfile that should reliably reproduce the error. The contents of the dockerfile are:
# docker build --rm --squash --tag gcc:dev -f ./Dockerfile
# docker run -it --rm --name gcc-dev gcc:dev
ARG JOBS=8
ARG CC=clang
ARG CXX=clang++
FROM alpine:edge as prep
ARG JOBS
ARG CC
ARG CXX
WORKDIR /root/work
RUN apk update && apk upgrade && apk add --no-cache make g++ clang
FROM prep AS gcc-build
ARG JOBS
ARG CC=gcc
ARG CXX=g++
ENV \
CFLAGS="-fpic -fPIE -fpie -static -static-libgcc" \
CXXFLAGS="-fpic -fPIE -fpie -static" \
LDFLAGS=""
ADD https://bigsearcher.com/mirrors/gcc/releases/gcc-10.2.0/gcc-10.2.0.tar.gz .
# COPY gcc/gcc-10.2.0.tar.gz .
RUN \
echo Adding gcc dependencies... && \
apk add gmp-dev mpfr-dev mpc1-dev isl-dev linux-headers && \
echo Building gcc... && \
echo Unpacking gcc... && \
tar zxf gcc-10.2.0.tar.gz && \
echo Moving gcc source files... && \
mv gcc-10.2.0/* . && rm -rf gcc-10.2.0 *.tar.gz && \
echo Configuring gcc... && \
./configure \
--build=x86_64-unknown-linux-musl \
--host=x86_64-unknown-linux-musl \
--target=x86_64-unknown-linux-musl \
# --prefix=/opt/gcc && \
--enable-languages=c,c++ \
--enable-default-pie \
--disable-shared \
--disable-nls \
--with-static-standard-libraries \
--with-stage1-ldflags="-static-pie" \
--with-boot-ldflags="-static-pie" \
--disable-multilib && \
time make -j${JOBS} bootstrap && \
apk del g++ gcc && \
make install
I had run into the error when leaving out the stage 1 -static-pie LDFLAGS. To make the error reproduce quickly, I've set the make target to bootstrap and included the stage 1 -static-pie LDFLAG, but I'm not totally convinced this is necessarily a good indicator of the nature of the error.
I will note that no matter what CFLAGS or CXXFLAGS I have, much of the bootstrapped and stage 2 gcc compilation occurs with something like the following command:
g++ -std=gnu++98 -fno-PIE -c -g -DIN_GCC -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-error=format-diag -Wno-format -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common -DHAVE_CONFIG_H -I. -I. -I../.././gcc -I../.././gcc/. -I../.././gcc/../include -I../.././gcc/../libcpp/include -I../.././gcc/../libdecnumber -I../.././gcc/../libdecnumber/bid -I../libdecnumber -I../.././gcc/../libbacktrace -o insn-output.o -MT insn-output.o -MMD -MP -MF ./.deps/insn-output.TPo insn-output.c
i.e. Its always passing the -fno-PIE flag. I would expect I need to get rid of that flag, but I can't find where I do such a thing.
So...how do I create a gcc package that I can use to create static PIE executables?
I got this to work but I had to compile basically the entire LLVM toolchain for x64 musl. Not sure if it ended up being compiler-rt, or libc++/libc++abi, but eventually I could bootstrap the LLVM toolchain using the -fPIE flag appropriately.
I've since given up on that entire endeavor. Its a nightmare trying to keep track of who's bootstraps are bootstrapping who's bootstraps.
I'm trying to build GLEW 2.1.0 on Windows (8.1) using MinGW and have been struggling for a while now.
I tried to follow the instructions for "MSYS/Mingw" on the GLEW GitHub page but when running mingw32-make, I only got a couple errors about missing commands (test and config) as well as the following: Makefile:40: *** "Platform '' not supported". Stop.
Under "Requirements" it says that bash is needed, so I installed git, because that apparently comes with a version of bash. Running mingw32-make (in the directory that all the GLEW files and folders are in) using Git Bash results in a long list of warnings and "undefined reference"s. The latter come from the linker as far as I know, but I have no idea what could cause them. As it's so much text, I'll only include the last few lines here:
c:/mingw/bin/../lib/gcc/mingw32/8.2.0/../../../../mingw32/bin/ld.exe: tmp/mingw/default/shared/glewinfo.o:glewinfo.c:(.text.startup+0x181b): undefined reference to `_imp___iob'
collect2.exe: error: ld returned 1 exit status
Makefile:181: recipe for target 'bin/glewinfo.exe' failed
mingw32-make: *** [bin/glewinfo.exe] Error 1
mingw32-make install has the following output:
$ mingw32-make install
install -d -m 0755 "/usr/include/GL"
install -m 0644 include/GL/wglew.h "/usr/include/GL/"
install -m 0644 include/GL/glew.h "/usr/include/GL/"
install -m 0644 include/GL/glxew.h "/usr/include/GL/"
sed \
-e "s|#prefix#|/usr|g" \
-e "s|#libdir#|/usr/lib|g" \
-e "s|#exec_prefix#|/usr/bin|g" \
-e "s|#includedir#|/usr/include/GL|g" \
-e "s|#version#|2.1.0|g" \
-e "s|#cflags#||g" \
-e "s|#libname#|glew32|g" \
-e "s|#requireslib#|glu|g" \
< glew.pc.in > glew.pc
install -d -m 0755 "/usr/lib"
install -d -m 0755 "/usr/bin"
install -m 0755 lib/glew32.dll "/usr/bin/"
install -m 0644 lib/libglew32.dll.a "/usr/lib/"
install -m 0644 lib/libglew32.a "/usr/lib/"
install -d -m 0755 "/usr/lib/pkgconfig"
install -d -m 0755 "/usr/lib/pkgconfig"
install -m 0644 glew.pc "/usr/lib/pkgconfig/"
And mingw32-make install.all produces another long list of undefined references and the same error that came from running mingw32-make. The first lines however are (about) the same as the result of running mingw32-make install.
As expected with all of the errors, running these commands didn't produce a binary file. I guess it could have something to do with me not having make, which is one of the requirements listed on the GitHub page. However I don't understand how I should install make on Windows and why it is even needed when the instructions say to use mingw32-make.
I've also tried some of the suggestions made in this thread, but that doesn't seem to have worked either.
What could be the problem and what can I do to successfully compile GLEW?
I could never build it with the instructions I found around the internet, so here's how I build glew (using https://downloads.sourceforge.net/project/glew/glew/2.2.0/glew-2.2.0.tgz) under MSYS2 with MinGW-w64:
# change the line below to your desired install path
INSTALLPREFIX=/usr/local
VERSION=2.2.0
gcc -O2 -Wall -W -Iinclude -DGLEW_BUILD -o src/glew.o -c src/glew.c &&
gcc -fno-builtin -fno-stack-protector -shared -s -Wl,-soname,libglew32.dll -Wl,--out-implib,lib/libglew32.dll.a -o lib/glew32.dll src/glew.o -Wl,--as-needed -lglu32 -lopengl32 -lgdi32 -luser32 -lkernel32 -nostdlib &&
ar cr lib/libglew32.a src/glew.o &&
gcc -DGLEW_MX -O2 -Wall -W -Iinclude -DGLEW_BUILD -o src/glew.mx.o -c src/glew.c &&
gcc -fno-builtin -fno-stack-protector -shared -s -Wl,-soname,libglew32mx.dll -Wl,--out-implib,lib/libglew32mx.dll.a -o lib/glew32mx.dll src/glew.mx.o -Wl,--as-needed -lglu32 -lopengl32 -lgdi32 -luser32 -lkernel32 -nostdlib &&
ar cr lib/libglew32mx.a src/glew.mx.o &&
gcc -O2 -c -Iinclude -o src/glewinfo.o src/glewinfo.c &&
gcc -s -o src/glewinfo.exe src/glewinfo.o lib/libglew32.dll.a -Wl,--as-needed -lgdi32 -lopengl32 &&
gcc -O2 -c -Iinclude -o src/visualinfo.o src/visualinfo.c &&
gcc -s -o src/visualinfo.exe src/visualinfo.o lib/libglew32.dll.a -Wl,--as-needed -lgdi32 -lopengl32 -lglu32 &&
sed -e "s?#prefix#?$INSTALLPREFIX?; s?#libdir#?\$\{prefix\}/lib?; s?#version#?$VERSION?; s?#requireslib#?glu?; s?#cflags#??; s?#libname#?glew32?" glew.pc.in > glew.pc &&
sed -e "s?#prefix#?$INSTALLPREFIX?; s?#libdir#?\$\{prefix\}/lib?; s?#version#?$VERSION?; s?#requireslib#?glu?; s?#cflags#?-DGLEW_MX?; s?#libname#?glew32mx?" glew.pc.in > glewmx.pc &&
echo Installing... &&
mkdir -p $INSTALLPREFIX/include $INSTALLPREFIX/lib/pkgconfig $INSTALLPREFIX/bin $INSTALLPREFIX/cmake &&
cp -rf include/GL $INSTALLPREFIX/include/ &&
cp -f lib/*.a $INSTALLPREFIX/lib/ &&
cp -f *.pc $INSTALLPREFIX/lib/pkgconfig/ &&
cp -f lib/*.dll src/*.exe $INSTALLPREFIX/bin/ &&
cp -f build/cmake/*.cmake $INSTALLPREFIX/cmake/ &&
echo Success
Using a Dockerfile, I'm attempting to build Clang version v8 along with its dependencies on Ubuntu 18.04. The reason I'm going through all this trouble is because I can't find a 32-bit version of the LLVM libraries. Even the LLVM packages themselves have only 64-bit variants of everything. Because of this, I'm not able to build my 32-bit applications using the LLVM toolchain at version 8. Using version 8 of LLVM is mandatory for reasons I won't go into here.
So far, here is what I have:
FROM ubuntu:18.04 AS build
ARG NUM_PARALLEL=8
RUN true \
&& dpkg --add-architecture i386 \
&& apt-get -qq update \
&& apt-get -qq install \
software-properties-common \
build-essential \
gcc-multilib \
g++-multilib \
git \
wget \
autoconf \
pkg-config \
m4 \
python-dev:i386 \
libcurl4-gnutls-dev:i386 \
libncurses-dev:i386 \
uuid-dev:i386 \
libx11-dev:i386 \
libxext-dev:i386 \
libtinfo-dev:i386 \
libedit-dev:i386 \
swig \
libedit-dev
python-dev
ENV CMAKE_BUILD_PARALLEL_LEVEL=$NUM_PARALLEL
# Ninja
RUN true \
&& git clone --depth 1 --branch v1.8.2 https://github.com/ninja-build/ninja.git \
&& cd ninja \
&& ./configure.py --bootstrap \
&& cp ninja /usr/local/bin
# CMake
RUN true \
&& git clone --depth 1 --branch v3.13.4 https://gitlab.kitware.com/cmake/cmake.git \
&& cd cmake \
&& ./bootstrap --parallel=$NUM_PARALLEL \
&& make -j$NUM_PARALLEL install
# Clang (See: https://clang.llvm.org/get_started.html)
RUN true \
&& git clone --depth 1 --branch llvmorg-8.0.0 https://github.com/llvm/llvm-project.git \
&& cd llvm-project \
&& cmake -G Ninja -B build -S llvm \
-D CMAKE_BUILD_TYPE=Release \
-D LLVM_BUILD_32_BITS:BOOL=ON \
-D LLVM_ENABLE_PROJECTS=all \
-D LLVM_BUILD_TESTS:BOOL=OFF \
-D LLVM_BUILD_EXAMPLES:BOOL=OFF \
-D LLVM_INCLUDE_EXAMPLES:BOOL=OFF \
-D LLVM_INCLUDE_TESTS:BOOL=OFF \
-D LLVM_INCLUDE_BENCHMARKS:BOOL=OFF \
&& cmake --build build -j $NUM_PARALLEL --target install
I get through compiling a little over 5200 translation units, which takes over an hour, only for it to fail:
[5232/6435] Linking CXX shared library lib/readline.so
FAILED: lib/readline.so
: && /usr/bin/c++ -fPIC -fPIC -fvisibility-inlines-hidden -m32 -Werror=date-time -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-deprecated-register -Wno-vla-extension -Wno-macro-redefined -O3 -DNDEBUG -Wl,-z,defs -Wl,-z,nodelete -m32 -shared -Wl,-soname,readline.so -o lib/readline.so tools/lldb/scripts/Python/modules/readline/CMakeFiles/readline.dir/readline.cpp.o /usr/lib/x86_64-linux-gnu/libpython2.7.so /usr/lib/x86_64-linux-gnu/libedit.so && :
/usr/lib/x86_64-linux-gnu/libpython2.7.so: error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status
I followed the getting started page to understand how to build LLVM so far, but I must be missing something. They don't exactly go into 32-bit compilation in their examples. Can someone help me get this building? Or at least (and probably better), point me to a version of Clang v8 that has 32-bit LLVM libraries bundled with it?
Seems like you are linking against the wrong Python: usr/lib/x86_64-linux-gnu/libpython2.7.so.
You may want to for it and add it to the CMake commands:
-DPYTHON_EXECUTABLE=PATH_TO_PYTHON-DEV:i686
You may also need to set PYTHON_LIBRARY and PYTHON_INCLUDE_DIR to make sure that the i386 versions are used.
I'm building my first OpenCV based application in C++. My goal is to build an intermediate docker image that can compile the application statically so that it can run standalone in the resulting smaller image. I'm open to using any docker image for this step, but just so that you can see exactly what I have, here's the dockerfile to reproduce the entire environment:
FROM ubuntu:18.04 as compiler
ENV OPENCV_VERSION='3.4.2' DEBIAN_FRONTEND=noninteractive
RUN apt-get -y update && \
apt-get -y upgrade && \
apt-get -y dist-upgrade && \
apt-get -y autoremove && \
apt-get install -y build-essential cmake
RUN apt-get install -y qt5-default libvtk6-dev
RUN apt-get install -y zlib1g-dev libjpeg-dev libwebp-dev libpng-dev libtiff5-dev libopenexr-dev libgdal-dev
RUN apt-get install -y libdc1394-22-dev libavcodec-dev libavformat-dev libswscale-dev libtheora-dev libvorbis-dev libxvidcore-dev libx264-dev yasm libopencore-amrnb-dev libopencore-amrwb-dev libv4l-dev libxine2-dev
RUN apt-get install -y unzip wget
RUN wget --progress=dot:giga https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip && \
unzip -q ${OPENCV_VERSION}.zip && \
rm ${OPENCV_VERSION}.zip && \
mv opencv-${OPENCV_VERSION} OpenCV && \
cd OpenCV && \
mkdir build && \
cd build && \
cmake \
-D BUILD_SHARED_LIBS=OFF \
-D WITH_QT=ON \
-D WITH_OPENGL=ON \
-D FORCE_VTK=ON \
-D WITH_TBB=ON \
-D WITH_GDAL=ON \
-D WITH_XINE=ON \
-D BUILD_EXAMPLES=OFF \
-D ENABLE_PRECOMPILED_HEADERS=OFF \
-D BUILD_DOCS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_opencv_apps=OFF \
.. && \
make -j4 && \
make install && \
ldconfig
COPY compile-test.cpp compile-test.cpp
RUN g++ -std=c++11 -static compile-test.cpp -o /app $(pkg-config --cflags --libs opencv)
I can currently compile my c++ apps without issue using the dyanmic libs, but this creates a massive docker image, and I really want to be able to build standalone binaries for distribution, with minimal size.
As you can see I'm compiling OpenCV from source including the flag BUILD_SHARED_LIBS=OFF to make sure I get the .a static libs, rather than the .so dynamic libs. I took a hint from a highly recommended build script, and modified it for use with docker omitting a few python things as I'm using c++.
Because I was having so much trouble with my real application, I've gone ahead and created a much simpler app, which also blows up during compilation. I believe this has something to do with the included cflags and libs. The problem is currently beyond my comprehension. I get mountains of errors that seem to change when I adjust a single include on my compile command. Here's the simplest app I'm trying to compile. It really doesn't do anything, but it does include a lib.
#include "opencv2/imgcodecs.hpp"
using namespace cv;
Mat img;
int main( int argc, char** argv ) {
img = cv::imread( argv[1], IMREAD_COLOR );
}
Then I try to compile this like so:
g++ -std=c++11 -static compile-test.cpp -o /app $(pkg-config --cflags --libs opencv)
And it ends up in a pile of errors much too long to completely paste here.
//usr/local/lib/libopencv_imgcodecs.a(grfmt_jpeg.cpp.o): In function `cv::JpegEncoder::write(cv::Mat const&, std::vector<int, std::allocator<int> > const&)':
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0xf8): undefined reference to `jpeg_CreateCompress'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x105): undefined reference to `jpeg_std_error'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x2b5): undefined reference to `jpeg_set_defaults'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x2d0): undefined reference to `jpeg_set_quality'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x2fe): undefined reference to `jpeg_quality_scaling'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x30d): undefined reference to `jpeg_quality_scaling'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x367): undefined reference to `jpeg_default_qtables'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x379): undefined reference to `jpeg_start_compress'
grfmt_jpeg.cpp:
...
collect2: error: ld returned 1 exit status
Some Things I've Already Tried
Beginning to google each of the seemingly unique compile errors and adding related flags to the end of my compile code.
Reordering some of the include flags, but there are just too many to do this effectively
Using the opencv-dev package instead of compiling it myself, but it seems you can't do this and expect to use static libs.
After a lot of experimentation I finally got something working! There were a few issues that are all fixed in this Dockerfile. In order to reproduce this, create a Dockerfile with the following contents, and create another file called app.cpp with the simple code from my question above, in the same folder.
I will explain what the issues were below:
FROM alpine:3.8 as compiler
RUN echo -e '#edgunity http://nl.alpinelinux.org/alpine/edge/community \
#edge http://nl.alpinelinux.org/alpine/edge/main \
#testing http://nl.alpinelinux.org/alpine/edge/testing \
#community http://dl-cdn.alpinelinux.org/alpine/edge/community' \
>> /etc/apk/repositories
RUN apk add --update --no-cache \
build-base \
openblas-dev \
unzip \
wget \
cmake \
g++ \
libjpeg \
libjpeg-turbo-dev \
libpng-dev \
jasper-dev \
tiff-dev \
libwebp-dev \
clang-dev \
linux-headers
ENV CC /usr/bin/clang
ENV CXX /usr/bin/g++
ENV OPENCV_VERSION='3.4.2' DEBIAN_FRONTEND=noninteractive
RUN mkdir /opt && cd /opt && \
wget https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip && \
unzip ${OPENCV_VERSION}.zip && \
rm -rf ${OPENCV_VERSION}.zip
RUN mkdir -p /opt/opencv-${OPENCV_VERSION}/build && \
cd /opt/opencv-${OPENCV_VERSION}/build && \
cmake \
-D BUILD_DOCS=OFF \
-D BUILD_EXAMPLES=OFF \
-D BUILD_opencv_apps=OFF \
-D BUILD_opencv_python2=OFF \
-D BUILD_opencv_python3=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_SHARED_LIBS=OFF \
-D BUILD_TESTS=OFF \
-D CMAKE_BUILD_TYPE=RELEASE \
-D ENABLE_PRECOMPILED_HEADERS=OFF \
-D FORCE_VTK=OFF \
-D WITH_FFMPEG=OFF \
-D WITH_GDAL=OFF \
-D WITH_IPP=OFF \
-D WITH_OPENEXR=OFF \
-D WITH_OPENGL=OFF \
-D WITH_QT=OFF \
-D WITH_TBB=OFF \
-D WITH_XINE=OFF \
-D BUILD_JPEG=ON \
-D BUILD_TIFF=ON \
-D BUILD_PNG=ON \
.. && \
make -j$(nproc) && \
make install && \
rm -rf /opt/opencv-${OPENCV_VERSION}
RUN wget --progress=dot:giga https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.0.0-linux-x86-64.tar.gz && \
pwd && \
tar -xzf libwebp-1.0.0-linux-x86-64.tar.gz && \
mv /libwebp-1.0.0-linux-x86-64/lib/libwebp.a /usr/lib && \
rm -rf /libwebp*
RUN wget --progress=dot:giga http://www.ece.uvic.ca/~frodo/jasper/software/jasper-2.0.10.tar.gz && \
tar -xzf jasper-2.0.10.tar.gz && \
cd jasper-2.0.10 && \
mkdir BUILD && \
cd BUILD && \
cmake -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_SKIP_INSTALL_RPATH=YES \
-DCMAKE_INSTALL_DOCDIR=/usr/share/doc/jasper-2.0.10 \
-DJAS_ENABLE_SHARED=FALSE \
.. && \
make install && \
rm -rf /jasper-2.0.10*
ENV PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig:/usr/lib/pkgconfig
COPY app.cpp app.cpp
RUN g++ -Wl,-Bstatic -static-libgcc -std=c++11 \
app.cpp \
-o /app \
$(pkg-config --cflags --libs -static opencv) \
-lgfortran -lquadmath
FROM alpine
COPY --from=compiler /app /bin/app
Problems
Linker
There were indeed files that needed linking that weren't present, there were two reasons for this:
The pkg-config command is supposed to emit all of the necessary flags for compilation, but in my earlier attempt I hadn't included the -static flag to pkg-config. When you add the -static flag it makes sure to link the extra required packages. I saw a few people run into this problem with the solution of adding the extra flags like -pthread, but I found that the -static flag did this for me and so was preferable.
ld: cannot find -lgcc_s error. This appeared to be fixed by adding the -static-libgcc flag to g++. Some of this is still a mystery to me.
Missing Static Libraries
There were two libraries that I wanted to be included as static which needed to be acquired from sources other than apk. These were libjasper and libwebp. There are build steps above that acquire and build these as necessary and copy the resources into the required place.
More missing links
For reasons I can't yet explain pkg-config didn't provide the last two necessary flags. Those were -lgfortran and -lquadmath.
Notes about this Solution
I switched to alpine linux, just because I had read that some people had success with that, I'm sure the same could be done with Ubuntu. It did result in a much smaller image, so I do like that. This is about 900mb for the intermediate image, which, while huge, is much smaller than the 1.9GB Ubuntu image.
The actual resulting image is about 44mb including all of the statically linked OpenCV libs. This seems like a good solution for those that need a small docker image to run a single C++ bin.
I have a makefile.qt file, and whenever I make it, there are unknown folders generated.Makefile.qt:
include buildconfig
%.mak : %.pro qtconfig
$(MAYA_LOCATION)/bin/$(QMAKE) -o - QMAKE_CC=$(CC) QMAKE_CXX=$(C++) $< | \
LC_CTYPE=C sed -e '/^TARGET.*=/s?=.*$$?= $$(QMAKE_TARGET).$(EXT)?' \
-e 's?-framework QtCore?$$(MAYA_LOCATION)/MacOS/QtCore?' \
-e 's?-framework QtGui?$$(MAYA_LOCATION)/MacOS/QtGui?' \
-e 's?-framework QtOpenGL?$$(MAYA_LOCATION)/MacOS/QtOpenGL?' \
-e 's?-framework QtTest?$$(MAYA_LOCATION)/MacOS/QtTest?' \
-e 's?-framework QtXml?$$(MAYA_LOCATION)/MacOS/QtXml?' \
-e 's?-framework Maya?$$(MAYA_LOCATION)/MacOS/Maya?' \
-e 's?-dynamiclib??' -e 's?-mtune=generic??' >$#
PLUGINS = helixQtCmd.$(EXT)
ifeq ($(QMAKE),)
all:
#echo "Qt not found. Qt-based plug-ins will not be built."
else
all: $(PLUGINS)
endif
# For each plugin, make sure that its individual Makefile is up-to-date then
# use that Makefile to build the plugin.
.PHONY: force
%.$(EXT): force
$(MAKE) -f Makefile.qt $(#:.$(EXT)=.mak)
$(MAKE) -f $(#:.$(EXT)=.mak) $#
cp $(#:.$(EXT)=.bundle) compiled_plugins/maya$(MAYA_VERSION)/plug-ins/Plurii.bundle
clean:
rm -f $(PLUGINS:.$(EXT)=.o) $(PLUGINS:.$(EXT)=.mak) \
moc_output/* ui_* qrc_* *.o
Clean: clean
rm -f $(PLUGINS)
In order to build my plugin, I use the following command:
make -f Makefile.qt <myplugin>.bundle
However this generates a .mak file in which there are plenty of Include and Lib folders that are incorrect and I can't find where they are coming from:
-I/Users/caiwe/qt/qt-4.8.6/include/QtGui
-L/Users/caiwe/qt/qt-4.8.6/lib
/Users/caiwe/qt/qt-4.8.6/bin/moc
...
I have no clue where these /Users/caiwe come from, as this is not me. Any hint where I can find this ?
Issue was coming from the mkspecs folder which I took from maya2016 download. Taking the mkspecs folder which was provided with maya2015 made it work.