Error building Python extension using boost.python - c++

I installed boost as well as boost-python, and boost-build using homebrew on my Mac with OS X 10.11.6. I am running Python 3.5.2. boost is set up correctly and works in C++ projects. Both, user-config.jam and the jamfile located in my python extension project directory are ok. I tried to compile a shared library from the following source
#include <iostream>
#include <boost/python.hpp>
using namespace std;
void say_hello() {
std::cout << "HELLO!!!!!";
}
BOOST_PYTHON_MODULE(hello) {
using namespace boost::python;
def ("say_hello", say_hello);
}
using the b2 interpreter. It issues the following command:
"g++" -dynamiclib -Wl,-single_module -install_name "hello.so" -L"/usr/local/lib/python3.5" -o "bin/darwin-4.2.1/release/hello.so" "bin/darwin-4.2.1/release/say_hello.o" -lpython3.5 -headerpad_max_install_names -Wl,-dead_strip -no_dead_strip_inits_and_terms
, which crash with
darwin.link.dll bin/darwin-4.2.1/release/hello.so
Undefined symbols for architecture x86_64:
"typeinfo for boost::python::objects::py_function_impl_base", referenced from:
[...long trace back ...]
"boost::python::detail::init_module(PyModuleDef&, void (*)())", referenced from:
_PyInit_hello in say_hello.o ld: symbol(s) not found for architecture x86_64
I am very aware of all the question concerning similar problems, but unfortunately none of them provides a working answer.
What do I have to do I order to get this simple code working as an Python extension module?

You should link it with boost python library as well( as boost.python is not header only). Here is how boost libraries are include in the build command(paths as I have on my machine):
-L/usr/lib/libpython2.7.dylib /usr/local/lib/libboost_system-mt.dylib /usr/local/lib/libboost_python-mt.dylib /usr/lib/libpython2.7.dylib -Wl,-rpath,/usr/lib/libpython2.7.dylib
I assume you can do without libboost_system library. (Such output I get when run make VERBOSE=1 as I'm not running make explicitly.)
Regarding cmake, here is a simple CMakeLists.txt you can use to build a project with Boost.Python:
cmake_minimum_required(VERSION 2.8)
set(LIBRARY_NAME "ext") # ext for extension
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall -ggdb")
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR}/build)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR}/lib)
set(SOURCES
python_interface.cpp
main.cpp
)
set(CMAKE_MACOSX_RPATH ON)
# Get Boost
find_package(Boost COMPONENTS
system
python REQUIRED)
include_directories(${Boost_INCLUDE_DIRS})
link_directories(${Boost_LIBRARY_DIRS})
# Get Python
find_package(PythonLibs REQUIRED)
include_directories(${PYTHON_INCLUDE_DIRS})
link_directories(${PYTHON_LIBRARIES})
add_library(${LIBRARY_NAME} SHARED ${SOURCES})
target_link_libraries(${LIBRARY_NAME}
${Boost_LIBRARIES}
${PYTHON_LIBRARIES}
)

Related

Rust CXX & CMake FFI bindings for C++ library to be used in rust code

I have the following error during cargo build of a project using cxx with cmake for compiling a C++ library to be callable in rust.
Here is the error from cargo build:
ld: warning: object file (/usr/local/lib/libwhisper.a(ggml.c.o)) was built for newer macOS version (12.6) than being linked (12.0)
ld: warning: object file (/usr/local/lib/libwhisper.a(whisper.cpp.o)) was built for newer macOS version (12.6) than being linked (12.0)
Undefined symbols for architecture x86_64:
"_cxxbridge1$whisper_init", referenced from:
whisper_rust::interop::ffi::whisper_init::hbcdc3c0c1120a6a9 in whisper_rust-2cf5d7323dfcdb42.1p80ceksqm40omuj.rcgu.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
This is my code in interop.rs using cxx::bridge macro:
#[cxx::bridge]
pub mod ffi {
unsafe extern "C++" {
type whisper_token;
type whisper_context;
fn whisper_init(path_model: &CxxString) -> *mut whisper_context; // this line causes the error
}
}
the last function declaration is the faulty one, commenting out the line make me build the project successfully but without interacting with the static C++ library this is useless.
That refers to a function decl. in the header:
struct whisper_context * whisper_init(const char * path_model);
Here's my build.rs:
println!("cargo:rerun-if-changed=src/whisper-cpp/*");
let dst = cmake::build("src/whisper-cpp");
println!("cargo:rustc-link-search=native={}", dst.display());
println!("cargo:rustc-link-lib=static=whisper");
And this is my CMakeLists.txt:
cmake_minimum_required(VERSION 3.0)
set(CMAKE_OSX_DEPLOYMENT_TARGET "10.9" CACHE STRING "Minimum OS X deployment version")
project(whisper VERSION 0.1.0)
# add compiler flags
set(CMAKE_CC_FLAGS "${CMAKE_CC_FLAGS} -I. -O3 -pthread")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -I. -O3 -std=c++11 -pthread")
# platform-dependent compiler flags
...
# add source files
aux_source_directory(. SRC_LIST)
# add library
add_library(${PROJECT_NAME} STATIC ${SRC_LIST})
target_include_directories(${PROJECT_NAME} PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
set_target_properties(${PROJECT_NAME} PROPERTIES VERSION ${PROJECT_VERSION})
set_target_properties(${PROJECT_NAME} PROPERTIES PUBLIC_HEADER "whisper.h")
# install
install(TARGETS ${PROJECT_NAME}
LIBRARY DESTINATION lib
PUBLIC_HEADER DESTINATION include)
I'm not sure where my problem lies, I guess this is related the compilation step of CMakeLists but can be also generated by a bad build.rs setup.
I've already tried without success:
rebuilding / clean cache and then rebuilt
including the header file in interop.rs
looking for answers here, google, reddit, etc. for hours
changing the version of OSX required by cmake (this still doesn't make any difference shown in the error, makes me wonder...)
I spent way too hours doing this, I'm still learning FFI and I'll be very thankful if you can help me get out of this mess. Thank you.
EDIT: I updated the system, reinstalled cmake and set file in place of include in build.rs, this fixed the problem.
cxx will generate cpp code for your interop.rs in target/cxxbridge but it seems that you didn't include it in your CMakeLists.txt.
For example,
set(CXXBRIDGE ${CMAKE_CURRENT_SOURCE_DIR}/target/cxxbridge)
add_library(whisper SHARED your_cpp_file.cc ${CXXBRIDGE}/your_project_name/src/bridge.rs.cc)

How to properly link OpenCV in a C++ cross-platform library for Android and iOS?

I am developing a C++ library, including OpenCV, which will be used in a cross-platform Xamarin solution through a wrapper and the NuGet packaging system (see this guide). I configured a CMakeLists.txt file but I simply cannot get OpenCV to be correctly linked for both static (iOS) and dynamic (Android) libraries.
I tried to change the OpenCV_DIR variable, install and build OpenCV from sources and manually include the content of the OpenCV_INCLUDE_DIRS variable but nothing worked. I also noticed that the linking works when only using cv::Point. But the linking does not work when using cv::Mat, which I do not understand the reason.
The following is the CMakeLists.txt that I am using :
cmake_minimum_required (VERSION 3.2)
project (MyLib C CXX)
enable_testing()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
MESSAGE(STATUS "CMAKE_CXX_FLAGS: " ${CMAKE_CXX_FLAGS})
# Source and headers files
set(SOURCES File1.cpp File2.cpp)
set(HEADERS File1.h File2.h)
# Library
if(BUILD_SHARED_LIBS)
add_library (MyLib SHARED ${SOURCES} ${HEADERS})
target_compile_definitions(MyLib PUBLIC IS_BUILDING_SHARED)
else()
add_library (MyLib STATIC ${SOURCES} ${HEADERS})
endif()
# Dependencies
set(OpenCV_DIR /usr/local/Cellar/opencv/4.5.0_1/lib/cmake/opencv4)
find_package(OpenCV REQUIRED)
message(STATUS "OpenCV_INCLUDE_DIRS = ${OpenCV_INCLUDE_DIRS}")
message(STATUS "OpenCV_LIBS = ${OpenCV_LIBS}")
message(STATUS "OpenCV_DIR = ${OpenCV_DIR}")
include_directories(${OpenCV_INCLUDE_DIRS})
target_link_libraries(MyLib ${OpenCV_LIBS})
The following shows the location of OpenCV's files that are used during the build process. Everything seems alright.
-- OpenCV_INCLUDE_DIRS = /usr/local/Cellar/opencv/4.5.0_1/include/opencv4
-- OpenCV_LIBS = opencv_calib3d;opencv_core;opencv_dnn;opencv_features2d;opencv_flann;opencv_gapi;opencv_highgui;opencv_imgcodecs;opencv_imgproc;opencv_ml;opencv_objdetect;opencv_photo;opencv_stitching;opencv_video;opencv_videoio;opencv_alphamat;opencv_aruco;opencv_bgsegm;opencv_bioinspired;opencv_ccalib;opencv_datasets;opencv_dnn_objdetect;opencv_dnn_superres;opencv_dpm;opencv_face;opencv_freetype;opencv_fuzzy;opencv_hfs;opencv_img_hash;opencv_intensity_transform;opencv_line_descriptor;opencv_mcc;opencv_optflow;opencv_phase_unwrapping;opencv_plot;opencv_quality;opencv_rapid;opencv_reg;opencv_rgbd;opencv_saliency;opencv_sfm;opencv_shape;opencv_stereo;opencv_structured_light;opencv_superres;opencv_surface_matching;opencv_text;opencv_tracking;opencv_videostab;opencv_viz;opencv_xfeatures2d;opencv_ximgproc;opencv_xobjdetect;opencv_xphoto
-- OpenCV_DIR = /usr/local/Cellar/opencv/4.5.0_1/lib/cmake/opencv4
Android
The following is the commands that I am using to build the Android dynamic library (.so). I have installed the NDK and am building for each ABI (x86, x86_64, armeabi-v7a, arm64-v8a).
cmake ../.. -DCMAKE_TOOLCHAIN_FILE=/Users/$USER/Library/Android/sdk/ndk-bundle/build/cmake/android.toolchain.cmake -DANDROID_NATIVE_API_LEVEL=21 -DANDROID_ABI=$abi_name -DBUILD_SHARED_LIBS=ON
cmake --build . --config Release
I directly get an error when building the library which is the following.
ld: error: /usr/local/Cellar/opencv/4.5.0_1/lib/libopencv_gapi.4.5.0.dylib: unknown file type
ld: error: /usr/local/Cellar/opencv/4.5.0_1/lib/libopencv_stitching.4.5.0.dylib: unknown file type
[...]
ld: error: /usr/local/Cellar/opencv/4.5.0_1/lib/libopencv_rapid.4.5.0.dylib: unknown file type
ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
iOS
The following is the commands that I am using to build the iOS static library (.a). I am using leetal's ​cmake toolchain file from this repository.
cmake ../.. -G Xcode -DCMAKE_TOOLCHAIN_FILE=../../ios.toolchain.cmake -DPLATFORM=OS64COMBINED -DBUILD_SHARED_LIBS=OFF
cmake --build . --config Release
The compilation of the static library seems to work because no error message is printed. However, when the library is used in the final Xamarin solution, the linked library cannot be found and the following error is shown.
Native linking failed, undefined symbol: cv::Mat::deallocate(). Please verify that all the necessary frameworks have been referenced and native libraries are properly linked in. (MT5210)
Question
What am I missing in order to properly compile and link OpenCV into my C++ library ?
I am working on macOS Big Sur and uses the following tools versions:
cmake : 3.20.0-rc5
ndk : 23.0.7196353
apple clang : 12.0.0
I hope that the description of my problem is clear enough and I thank you in advance for any help.
We had same problems, including Xamarin's DllNotFoundException with message from last comment, which led me to this topic. What fixed the exception for us in the end was linking statically to OpenCV *.a libs instead of linking to the shared libopencv_java4.so file. So we now have a huge 30MB nativelib.so file for each android ABI in build output, instead of a pair of small nativelib.so and libopencv_java4.so per ABI. CMakeLists looks like this:
set( OpenCV_DIR "~/opencv/build/OpenCV-android-sdk/sdk/native/jni" )
find_package( OpenCV REQUIRED )
target_link_libraries( # Specifies the target library.
nativelib
${OpenCV_LIBS})
Another thing in our project is we use OpenCV optional modules and had to create a custom OpenCV build, which I guess ensures our native library and OpenCV are compiled against same NDK version. I suppose using the prebuilt OpenCV distribution and compiling against a different NDK version could lead to problems too otherwise.

CMake: build fails when compiling a C++ shared lib depending on MPI

Problem overview
I'm using CMake to build a project which contains a library based on MPI. The lib has to be built as both shared and static. When I build the project on my Fedora 20 system, everything goes without problem. When I build it on a Mac OS X El Capitan system, the building process fails with an error of the type:
ld: symbol(s) not found for architecture x86_64
Additional remarks
I cannot activate the option forcing automatic compilation of libs as both shared and static because this specific library is the only one that needs to be built as both.
Forcing the replacement of the C++ compiler command by the MPI wrapper (mpicxx) results in a successful build but completely breaks CLion's code analysis.
Minimal (non)working example
File: MPILib.h
#include <mpi.h>
void MPILib();
File: MPILib.c
#include "MPILib.h"
void MPILib() { MPI_Init(NULL, NULL); }
File: CMakeLists.txt
cmake_minimum_required(VERSION 2.6)
enable_language(CXX)
set(CMAKE_MACOSX_RPATH 0)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall")
find_package(MPI REQUIRED)
include_directories(SYSTEM ${MPI_INCLUDE_PATH})
add_library(mpilib_shared SHARED MPILib.cpp MPILib.h)
add_library(mpilib_static STATIC MPILib.cpp MPILib.h)
Build commands:
cmake .
make mpilib_static (succeeds)
make mpilib_shared (fails)

Error linking against boost python

Here's my simple HelloWorld program
#include <boost/python.hpp>
using namespace boost::python;
void greet() {
// do nothing
}
BOOST_PYTHON_MODULE(HelloWorld)
{
def("greet", greet);
}
and here's my CMakeLists.txt file
cmake_minimum_required(VERSION 2.8.4)
project(HW)
find_package(Boost COMPONENTS python3 REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
include_directories(${Boost_INCLUDE_DIRS} /Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m include)
file(GLOB_RECURSE SRC
HelloWorld.cpp
)
add_library(HelloWorld SHARED ${SRC})
target_link_libraries(HelloWorld ${Boost_LIBRARIES})
However, I have been unable to build this simple program, with this build error
Undefined symbols for architecture x86_64:
"__Py_NoneStruct", referenced from:
boost::python::detail::none() in HelloWorld.cpp.o
boost::python::api::object::object() in HelloWorld.cpp.o
"boost::python::detail::init_module(PyModuleDef&, void (*)())", referenced from:
_PyInit_HelloWorld in HelloWorld.cpp.o
ld: symbol(s) not found for architecture x86_64
What am I missing? Sorry if this looks like a newbie question but I'm actually stuck.
I think you're missing the a link to the Python library (as opposed to the Boost Python library)
Try something like find_package(Python) then target_link_libraries(HelloWorld ${Python_LIBRARY})
Additionally (based on this post https://www.preney.ca/paul/archives/107) the name of the library you're building doesn't match the name given in BOOST_PYTHON_MODULE. Change it to BOOST_PYTHON_MODULE(libHelloWorld) because cmake implicitly adds a lib to the module name.

Getting "ld: symbol(s) not found for architecture x86_64" when compiling cmake code

I'm trying to compile a ROS package which uses Boost. The code compiles just fine on Linux, but on OS X I'm getting the error
ld: symbol(s) not found for architecture x86_64
I installed Boost through brew and it seems that it is installed in 64bit (my system is also 64bit - OS X 10.9), as running
file libboost_atomic-mt.dylib
outputs
libboost_atomic-mt.dylib: Mach-O 64-bit dynamically linked shared library x86_64
In the CMakeFiles.txt, I've tried nearly everything in terms of compile and link flags, having tried -stdlib both as libc++ and as libstdc++, as well as -mmacosx-version-min as everything from 10.5 to 10.9. For instance, right now I have:
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libstdc++ -mmacosx-version-min=10.9")
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -stdlib=libstdc++ -mmacosx-version-min=10.9")
Furthermore, building with -m64 produces the same errors, and building with -m32 produces the same errors except that it says "... architecture i136".
The following paste bins contain my CMakefiles.txt and the compiling errors, respectively:
http://pastebin.com/0MD8T916 - CMakeFiles.txt
http://pastebin.com/v3vk9i2r - Errors
I'm running out of ideas for fixing this issue...
Thank you for your help!
You did not actually add/link the boost libraries inside your project.
set(BOOST_COMPONENTS
unit_test_framework
program_options
thread) # And other components you need
set(Boost_USE_STATIC_LIBS ON) # Easier to deploy elsewhere
set(BOOST_ROOT /usr/local/opt/boost) # Useful for boost from brew
set(BOOST_LIBRARYDIR /usr/local/opt/boost/lib64)
find_package(Boost REQUIRED COMPONENTS ${BOOST_COMPONENTS})
include_directories(${Boost_INCLUDE_DIR})
target_link_libraries(<your target> ${Boost_LIBRARIES})