Building Apache Arrow inside existing C++ Executable project CMAKE - c++

Question

I'm working on a C++ CMake project that uses Apache Arrow as a dependency. My goal is to be able to include and use arrow/api.h. However, I couldn't find any documentation or tutorial that explains what I can do to achieve that so my first thought was to add it as a third library using CMake's FetchContent, hence, I added the following code to my CMakeLists.txt:
include(FetchContent REQUIRED)
# INCLUDING APACHE ARROW ================================ #
message(STATUS "Searching for Apache Arrow")
FetchContent_Declare(
arrow
GIT_REPOSITORY https://github.com/apache/arrow.git
GIT_TAG apache-arrow-0.15.1
)
FetchContent_GetProperties(arrow)
if(NOT arrow_POPULATED)
message(STATUS "Populating Arrow")
FetchContent_Populate(arrow)
add_subdirectory(${arrow_SOURCE_DIR}/cpp ${arrow_BINARY_DIR})
endif()
I can find the code inside my project now - still can't use include arrow/api.h though - but whenever I try to build it I receive errors from the CMakeLists.txt of Apache Arrow cpp src directory! Here's the CMake build log that contains the error:
-- Searching for Apache Arrow
-- Populating Arrow
-- Building using CMake version: 3.15.3
-- Arrow version: 0.15.1 (full: '0.15.1')
-- Arrow SO version: 15 (full: 15.1.0)
-- clang-tidy not found
-- clang-format not found
-- infer not found
-- Could NOT find PythonInterp (missing: PYTHON_EXECUTABLE)
-- Found cpplint executable at CPPLINT_BIN-NOTFOUND
-- Compiler command: C:/MinGW/bin/g++.exe -v
-- Compiler version: Using built-in specs.
COLLECT_GCC=C:/MinGW/bin/g++.exe
COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/mingw32/8.2.0/lto-wrapper.exe
Target: mingw32
Configured with: ../src/gcc-8.2.0/configure --build=x86_64-pc-linux-gnu --host=mingw32 --target=mingw32 --prefix=/mingw --disable-win32-registry --with-arch=i586 --with-tune=generic --enable-languages=c,c++,objc,obj-c++,fortran,ada --with-pkgversion='MinGW.org GCC-8.2.0-3' --with-gmp=/mingw --with-mpfr=/mingw --with-mpc=/mingw --enable-static --enable-shared --enable-threads --with-dwarf2 --disable-sjlj-exceptions --enable-version-specific-runtime-libs --with-libiconv-prefix=/mingw --with-libintl-prefix=/mingw --enable-libstdcxx-debug --with-isl=/mingw --enable-libgomp --disable-libvtv --enable-nls --disable-build-format-warnings
Thread model: win32
gcc version 8.2.0 (MinGW.org GCC-8.2.0-3)
-- Compiler id: GNU
Selected compiler gcc 8.2.0
-- Arrow build warning level: CHECKIN
Configured for DEBUG build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})
-- Build Type: DEBUG
-- Using approach to find dependencies
-- AWSSDK_VERSION: 1.7.160
-- BOOST_VERSION: 1.67.0
-- BROTLI_VERSION: v1.0.7
-- BZIP2_VERSION: 1.0.8
-- CARES_VERSION: 1.15.0
-- DOUBLE_CONVERSION_VERSION: v3.1.5
-- FLATBUFFERS_VERSION: v1.11.0
-- GBENCHMARK_VERSION: v1.5.0
-- GFLAGS_VERSION: v2.2.0
-- GLOG_VERSION: v0.3.5
-- GRPC_VERSION: v1.20.0
-- GTEST_VERSION: 1.8.1
-- JEMALLOC_VERSION: 5.2.1
-- LZ4_VERSION: v1.8.3
-- MIMALLOC_VERSION: 270e765454f98e8bab9d42609b153425f749fff6
-- ORC_VERSION: 1.5.5
-- PROTOBUF_VERSION: v3.7.1
-- RAPIDJSON_VERSION: 2bbd33b33217ff4a73434ebf10cdac41e2ef5e34
-- RE2_VERSION: 2019-08-01
-- SNAPPY_VERSION: 1.1.7
-- THRIFT_VERSION: 0.12.0
-- THRIFT_MD5_CHECKSUM: 3deebbb4d1ca77dd9c9e009a1ea02183
-- URIPARSER_VERSION: 0.9.3
-- ZLIB_VERSION: 1.2.11
-- ZSTD_VERSION: v1.4.3
-- Boost include dir:
-- Boost libraries:
-- Performing Test DOUBLE_CONVERSION_HAS_CASE_INSENSIBILITY
-- Performing Test DOUBLE_CONVERSION_HAS_CASE_INSENSIBILITY - Failed
-- Building without OpenSSL support. Minimum OpenSSL version 1.0.2 required.
-- Found hdfs.h at: D:/School/Research Project/TestApacheArrow/cmake-build-debug/_deps/arrow-src/cpp/thirdparty/hadoop/include/hdfs.h
-- CMAKE_C_FLAGS: -g -O0 -Wall -Wno-conversion -Wno-sign-conversion -Wno-unused-variable -Werror
-- CMAKE_CXX_FLAGS: -Wno-noexcept-type -fdiagnostics-color=always -g -O0 -Wall -Wno-conversion -Wno-sign-conversion -Wno-unused-variable -Werror
CMake Error at cmake-build-debug/_deps/arrow-src/cpp/src/arrow/CMakeLists.txt:362 (add_dependencies):
add_dependencies called with incorrect number of arguments
CMake Error at cmake-build-debug/_deps/arrow-src/cpp/CMakeLists.txt:814 (export):
export Export set "arrow-targets" not found.
-- ---------------------------------------------------------------------
-- Arrow version: 0.15.1
--
-- Build configuration summary:
-- Generator: MinGW Makefiles
-- Build type: DEBUG
-- Source directory: D:/School/Research Project/TestApacheArrow/cmake-build-debug/_deps/arrow-src/cpp
-- Install prefix: C:/Program Files (x86)/TestApacheArrow
-- Configuring incomplete, errors occurred!
See also "/cmake-build-debug/CMakeFiles/CMakeOutput.log".
See also "/cmake-build-debug/CMakeFiles/CMakeError.log".
Am I missing something here? is this the correct way to include Apache Arrow in an existing project? Any idea how can I fix this error?
I'm using Windows 10 but would finally want to run the project on Linux!

Answer 1

We don't currently support including Apache Arrow in another CMake project using add_subdirectory. You can use ExternalProject_Add, though, and it should work. It should also be possible to use CMake's built-in find_package facility to find a pre-built package.
We would like to provide well-documented and straightforward solutions for C++ developers including Apache Arrow in their CMake projects. We'd be glad to discuss further on our developer mailing list dev#arrow.apache.org