Compile boost as universal library (Intel and Apple Silicon architectures) - c++

Question

I am trying to build boost library as dylib on MacOS. I need to build it for both the Intel architecture and the upcoming Apple Silicon (arm64) architecture.
I downloaded boost and ran the following commands:
./bootstrap.sh
./b2 -address-model=64 architecture=combined -a
lipo -archs always shows produced dylibs architecture is x86_64.
I have Xcode12 beta and MacOS Catalina 10.15.7,
I can build a sample universal library if I create a project in Xcode and set archs arm64 x86_64 in build settings.
Running command ./b2 cxxflags="-arch arm64 -arch x86_64" fails with following errors:
"clang++" -x c++ -fvisibility-inlines-hidden -m64 -O3 -Wall -fvisibility=hidden -Wno-inline -arch arm64 -arch x86_64 -ftemplate-depth-255 -fvisibility=hidden -fvisibility-inlines-hidden -DBOOST_ALL_NO_LIB=1 -DNDEBUG -I"." -c -o "bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_iarchive.o" "libs/serialization/src/polymorphic_xml_iarchive.cpp"
...failed clang-darwin.compile.c++ bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_iarchive.o...
clang-darwin.compile.c++ bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_oarchive.o
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:32:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
#error Unsupported architecture
^
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:33:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
^
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
typedef __int64_t __darwin_blkcnt_t; /* total blocks */
^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t __darwin_blksize_t; /* preferred block size */
^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t __darwin_dev_t; /* dev_t */
^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t __darwin_gid_t; /* [???] process and group IDs */
^
note: '__uint128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t __darwin_id_t; /* [XSI] pid_t, uid_t, or gid_t*/
^

Answer 1

The following works for me on Big Sur using Xcode 12.2. On Catalina 10.15.7 (19H15) I get the same error messages as OP.
./bootstrap.sh --with-libraries=regex,date_time
./b2 architecture=combined cxxflags="-arch x86_64 -arch arm64"

Answer 2

I'm on Monterey with an M1 (XCode 13.1) and failed to get any of the other answers to work, but I combined stuff from Navan and Petr along with some other bits to get the libboost_coroutine to work on x86_64. I arrived at the following script which builds all the boost libraries (why not, it doesn't take long on the M1):
#!/bin/sh
rm -rf arm64 x86_64 universal stage bin.v2
rm -f b2 project-config*
./bootstrap.sh cxxflags="-arch x86_64 -arch arm64" cflags="-arch x86_64 -arch arm64" linkflags="-arch x86_64 -arch arm64"
./b2 toolset=clang-darwin target-os=darwin architecture=arm abi=aapcs cxxflags="-arch arm64" cflags="-arch arm64" linkflags="-arch arm64" -a
mkdir -p arm64 && cp stage/lib/*.dylib arm64
./b2 toolset=clang-darwin target-os=darwin architecture=x86 cxxflags="-arch x86_64" cflags="-arch x86_64" linkflags="-arch x86_64" abi=sysv binary-format=mach-o -a
mkdir x86_64 && cp stage/lib/*.dylib x86_64
mkdir universal
for dylib in arm64/*; do
lipo -create -arch arm64 $dylib -arch x86_64 x86_64/$(basename $dylib) -output universal/$(basename $dylib);
done
for dylib in universal/*; do
lipo $dylib -info;
done
This script prints out the lipo info for each dylib in the universal directory, so you can see quickly that every one has both x86_64 and arm64 inside. I had the following result, maybe it's useful for you to see if your output matches:
Architectures in the fat file: universal/libboost_atomic.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_chrono.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_container.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_context.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_contract.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_coroutine.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_date_time.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_filesystem.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_graph.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_iostreams.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_locale.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_log.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_log_setup.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_numpy27.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_prg_exec_monitor.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_program_options.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_python27.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_random.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_regex.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_serialization.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_stacktrace_addr2line.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_stacktrace_basic.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_stacktrace_noop.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_system.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_thread.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_timer.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_type_erasure.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_unit_test_framework.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_wave.dylib are: x86_64 arm64
Architectures in the fat file: universal/libboost_wserialization.dylib are: x86_64 arm64

Answer 3

After spending considerable amount of time here:
https://github.com/bfgroup/b2/issues/105
I was able to compile boost on arm64. The command line is:
./b2 architecture=arm address-model=64 asmflags=--target=arm64-apple-darwin21.2.0 cflags=--target=arm64-apple-darwin21.2.0 cxxflags=--target=arm64-apple-darwin21.2.0 linkflags=--target=arm64-apple-darwin21.2.0 -s NO_LZMA=1 -s NO_ZSTD=1 abi=aapcs
the LZMA and ZSTD flags are there since I did not have an universal binary for those libs on my machine.

Answer 4

I was facing the same issue and came across this answer, which was for creating universal binaries for i386 and x86_64.
To summarise the answer, you need to run ./b2 twice ( with different toolsets as clang always builds for x86_64, even if you pass the appropriate CXXFlags
This is the script I used to first individually generate the libraries I required and then combined them using lipo
#!/bin/sh
rm -rf arm64 x86_64 universal
./bootstrap.sh --with-toolset=clang --with-libraries=thread,system,filesystem,program_options,serialization
./b2 cxxflags="-arch arm64" toolset=darwin -a
mkdir -p arm64 && cp stage/lib/*.dylib arm64
./b2 toolset=clang cxxflags="-arch arm64 -arch x86_64" -a
mkdir x86_64 && cp stage/lib/*.dylib x86_64
mkdir universal
for dylib in arm64/*; do
lipo -create -arch arm64 $dylib -arch x86_64 x86_64/$(basename $dylib) -output universal/$(basename $dylib);
done

Answer 5

You don't need to use lipo, you can pass
-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk to cxxflags and it will build a universal library out of the box.

Answer 6

I followed the answer by Navan Chauhan which seems to be the way to go on macOS 10.15.7 with Xcode 12.3.
My only issue was that the build system was adding additional -arch armv4t clang option (or something similar) which fails the build. Using the architecture=combine cannot work as well as it is able to manage Intel and PowerPC only. What does work for me instead is:
bootstrap.sh --with-toolset=clang-darwin and cxxflags, cflags and linkflags set to -arch x86_64 -arch arm64 plus additional options.
x86_64 build: b2 toolset=clang-darwin target-os=darwin architecture=x86 stage and cxxflags, cflags and linkflags set to -arch x86_64 only (plus additional options).
arm64 build: b2 toolset=clang-darwin target-os=darwin architecture=arm abi=aapcs stage and cxxflags, cflags and linkflags set to -arch arm64 only (plus additional options).
Merge libraries with lipo…
I've been forced to set the ABI also as the automatic-guess-something is not able to recognize Apple Silicon goal therefor some assembly stuff is not compiled – hence missing symbols later on during the build. Finally the armv4t issue is solved by the clang-darwin toolset.
The only caveat of this at the moment is that both builds go into the same architectural/configuration directories which can be solved by custom user-config.jam or by immediate call to install just after each stage which is what I did.
Works on:
Host platform: Intel
macOS 10.15.7
Xcode 12.3 (with Command Line Tools installed)
Boost 1.75.0 (dependent compression and ICU libraries already compiled as universal)

Answer 7

For macOS with M1 building boost 1.81.0
./bootstrap.sh
sudo ./b2 -address-model=64 architecture=arm+x86 install
Libraries that support both architectures are installed