Protobuf ld undefined symbol ~InternalMetadata() - c++

I'm hitting an issue with some fairly basic Protobuf files in a library of mine. This is built as a static library successfully but then when compiling tests (Catch2) and linking to the same protobuf library I'm getting the below undefined symbol:-
[build] Starting build
[proc] Executing command: /usr/local/bin/cmake --build /Volumes/TB3-1/git/skunkworks/herald-for-cpp/build --config Debug --target herald-tests --
[build] [1/1 100% :: 0.182] Linking CXX executable herald-tests/herald-tests
[build] FAILED: herald-tests/herald-tests
[build] : && /usr/bin/clang++ -g -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names herald-tests/CMakeFiles/herald-tests.dir/test-templates.cpp.o herald-tests/CMakeFiles/herald-tests.dir/test-util.cpp.o herald-tests/CMakeFiles/herald-tests.dir/mesh-tests.cpp.o herald-tests/CMakeFiles/herald-tests.dir/main.cpp.o -o herald-tests/herald-tests herald/libherald.a /usr/local/lib/libprotobuf.a /usr/local/opt/openssl#3/lib/libcrypto.a && :
[build] Undefined symbols for architecture x86_64:
[build] "google::protobuf::internal::InternalMetadata::~InternalMetadata()", referenced from:
[build] google::protobuf::MessageLite::~MessageLite() in libherald.a(modem.pb.cc.o)
[build] google::protobuf::MessageLite::~MessageLite() in libherald.a(mesh.pb.cc.o)
[build] google::protobuf::MessageLite::~MessageLite() in libherald.a(models.pb.cc.o)
[build] ld: symbol(s) not found for architecture x86_64
[build] clang: error: linker command failed with exit code 1 (use -v to see invocation)
[build] ninja: build stopped: subcommand failed.
[build] Build finished with exit code 1
The super interesting thing is it even complains about the missing symbol from the ~MessageLite call when I force SPEED instead of LITE_RUNTIME. Doesn't seem to matter whether I link the library and programme to the dylib or the static lib on my mac, whether the full library or the lite library. Tried both SPEED and LITE_RUNTIME in my proto files with exactly the same result.
I've been struggling with this a while. Any ideas greatly appreciated.
Note: My library's target platform ranges from a container on a server to an embedded hardware device. These particular tests are compiled local (Mac OS X x86_64 Intel Mac in my case currently).

This issue was tracked here and ought to be fixed in the recently released Protobuf 3.21.3.
The library builds that ship with package managers are usually built with NDEBUG defined (-DNDEBUG to the compiler, #define NDEBUG in the code). Usually this just turns off checking assert() macros, but Protobuf keys on this symbol to turn on and off its own internal debugging. It turns out that in release mode the InternalMetadata destructor was always inlined and (with -fvisibility-inlines-hidden in the compiler options) never produced a symbol to link against, while in debug mode it was not always inlined and needed to be linked against.
So trying to build user code that didn't define NDEBUG for the Protobuf headers would result in code that needed to link against that destructor, which wouldn't be in the release Protobuf packaged libraries.
The workaround is to use -DNDEBUG in your application, or to build a debug build of Protobuf to link against, or to use a Protobuf new enough (3.21.3+) or old enough (I think <=3.19) to not have this particular problem.
The only remaining mystery is why the whole Internet noticed this bug in the past few days, when it's been there for much longer.

Related

Make error library not found for -lstdc++fs

When compiling code examples from a textbook, I run into a compilation error: ld: library not found for -lstdc++fs. What does this error mean and how can I get around it?
% make filesystem
Consolidate compiler generated dependencies of target filesystem
[100%] Linking CXX executable filesystem
ld: library not found for -lstdc++fs
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [chapter_17/filesystem] Error 1
make[2]: *** [chapter_17/CMakeFiles/filesystem.dir/all] Error 2
make[1]: *** [chapter_17/CMakeFiles/filesystem.dir/rule] Error 2
make: *** [filesystem] Error 2
In short, linking stdc++fs is not longer necessary as it has been incorporated into the base library.
When the textbook was first written the C++ libraries didn't officially support filesystem yet, so they would require a secondary library called stdc++fs. This was a libstdc++ specific library that imported the C++17 features that weren't in the official library yet.
Now, both GCC's libstdc++ and Clang's libc++ include it in their base library, and the stdc++fs library got dropped.
You can drop the linking to that library without any problems; it patched a problem in compiler support that's no longer relevant

how to skip incompatible architecture when linking with lld

I have a lib that provides both libxxx_x86_64.a and libxxx_arm64.a. When linking using ld x86_64.a is skipped,
ld: warning: ignoring file _pod_work_dir/Pods/____/libs/______x86_64.a, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
however lld won't do so
1d64.11d: error: _pod_work_dir/Pods/_____/libs/______x86_64.a(___+___.o)has architecture x86_64 which is incompatible with target architecture arm64
Is there an option with lld I can make this behavior a warning instead of error.
the lib is downloaded directly into build dir by cmake or pods script like this. So the project just try to link x86_64.a as well.

cmake on Mac with ARM M1 is running linker with x86_64 architecture instead of arm64

I am trying to compile glfw from source on Mac with M1 arm64 processor, and while running the linker, cmake strangely is trying to link the project for x86_64 architecture, while the binaries were built for arm64.
I clone the project, create build folder named cmake-build-debug, generate build system in it with the Makefile etc. as follows:
git clone https://github.com/glfw/glfw.git
cd glfw
mkdir cmake-build-debug
cd cmake-build-debug
cmake -S .. -B . -DCMAKE_BUILD_TYPE=Debug -DCMAKE_HOST_SYSTEM_PROCESSOR=arm64 -DCMAKE_SYSTEM_PROCESSOR=arm64
This works fine. But now that I build it with make or cmake --config Debug --build ., the .o binaries are generated perfectly fine, but linker script is incorrectly invoked by cmake with x86_64 target architecture for some reason:
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Including Cocoa support
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/burkov/Documents/Projects/open-source/glfw/cmake-build-debug
[ 47%] Built target glfw
Scanning dependencies of target wave
[ 50%] Linking C executable wave.app/Contents/MacOS/wave
ld: warning: ignoring file CMakeFiles/wave.dir/wave.c.o, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
ld: warning: ignoring file ../src/libglfw3.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture x86_64:
"_main", referenced from:
implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [examples/wave.app/Contents/MacOS/wave] Error 1
make[1]: *** [examples/CMakeFiles/wave.dir/all] Error 2
make: *** [all] Error 2
I look at the failing Makefile in glfw/cmake-build-debug/examples/CMakeFiles/wave.dir/build.make and see the line, where cmake is crashing:
cd /Users/me/Documents/Projects/open-source/glfw/cmake-build-debug/examples && $(CMAKE_COMMAND) -E cmake_link_script CMakeFiles/wave.dir/link.txt --verbose=$(VERBOSE)
I manually open the file glfw/cmake-build-debug/examples/CMakeFiles/wave.dir/link.txt file and see the following link script code there:
/Library/Developer/CommandLineTools/usr/bin/cc -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/wave.dir/wave.c.o -o wave.app/Contents/MacOS/wave ../src/libglfw3.a -framework Cocoa -framework IOKit -framework CoreFoundation
If I manually execute this line from shell, it successfully builds my binary for arm64 architecture, as expected.
But when this link.txt script is automatically invoked with cmake via cmake -E cmake_link_script CMakeFiles/wave.dir/link.txt --verbose=$(VERBOSE), it fails, apparently, trying to build the binary for the wrong x86_64 architecture.
Why is this happening and how to fix this?
For anyone running into the same problem, it looks like the first version of cmake with an adequate support for Apple Silicon is 3.19.
I was using 3.17.5 as my slightly out-of-date version of CLion does not support versions of cmake above that.
After an update to cmake 3.22.4 the problem is gone.

Trouble linking against OpenCL library/ies in Cygwin with an NVIDIA GPU

I'm trying to build this small OpenCL project:
https://bitbucket.org/Anteru/opencltutorial/src
on Windows, within Cygwin. It is a toy project with one .cpp file and one kernel to demonstrate the use of OpenCL. So, I download the package and cmake it. I get the error message:
CMake Error at /usr/share/cmake-3.14.5/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
Could NOT find OpenCL (missing: OPENCL_LIBRARY)
Call Stack (most recent call first):
/usr/share/cmake-3.14.5/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
cmake/FindOpenCL.cmake:35 (find_package_handle_standard_args)
CMakeLists.txt:6 (FIND_PACKAGE)
I do not balk at this, and proceed to change my CMakeCache.txt so that instead of:
OPENCL_LIBRARY:FILEPATH=OPENCL_LIBRARY-NOTFOUND
I make it:
OPENCL_LIBRARY:FILEPATH=/cygdrive/c/Program Files/NVIDIA Corporation/OpenCL/OpenCL.dll
(this is the legit path of NVIDIA's OpenCL DLL) Ok, so now CMake configuration concludes successfully. The main.cpp file compiles, but linking fails!:
/usr/bin/c++.exe -Wl,--enable-auto-import CMakeFiles/clTut.dir/main.cpp.o -o clTut.exe -Wl,--out-implib,libclTut.dll.a -Wl,--major-image-version,0,--minor-image-version,0 "/cygdrive/c/Program Files/NVIDIA Corporation/OpenCL/OpenCL.dll"
CMakeFiles/clTut.dir/main.cpp.o:main.cpp:(.text+0x610): undefined reference to `clGetPlatformInfo'
CMakeFiles/clTut.dir/main.cpp.o:main.cpp:(.text+0x610): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `clGetPlatformInfo'
CMakeFiles/clTut.dir/main.cpp.o:main.cpp:(.text+0x659): undefined reference to `clGetPlatformInfo'
CMakeFiles/clTut.dir/main.cpp.o:main.cpp:(.text+0x659): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `clGetPlatformInfo'
and so on, dozens of such errors.
Why am I getting these errors and how can I fix them? Is it a result of ELF/WinPE incompatibility because I'm using Cygwin?
Additional info:
My machine has both integrated Intel Graphics (630 - coffee lake), and a Quadro P1000.
I 've installed CUDA 10.2 and AMD CodeXL installed (never mind why) to their default locations.
Looking for An OpenCL DLL file, I find:
$ locate OpenCL.dll
/cygdrive/c/Program Files (x86)/CodeXL/spies/OpenCL.dll
/cygdrive/c/Program Files (x86)/CodeXL/spies64/OpenCL.dll
/cygdrive/c/Program Files/NVIDIA Corporation/OpenCL/OpenCL.dll
/cygdrive/c/Windows/LastGood/system32/OpenCL.dll
/cygdrive/c/Windows/LastGood/SysWow64/OpenCL.dll
/cygdrive/c/Windows/System32/OpenCL.dll
/cygdrive/c/Windows/SysWOW64/OpenCL.dll
Usually Cygwin programs are Unix like and do not call directly Windows DLLs, that use
a different paragdim.
For example the Cygwin cygwin1.dll provides its own C library different from the
MS C library, this cause the sharing the data problematic specially in the 64bit version where the size of long is different. See
https://cygwin.com/cygwin-ug-net/programming.html#gcc-64
Your Tutorial example can be built in a Cygwin pure enviroment, with the proper OpenCL library
$ cygcheck -cd | grep -i opencl
libOpenCL-devel 2.2.12-1
libOpenCL1 2.2.12-1
$ unzip Anteru-opencltutorial-cb1df4439f83.zip
Archive: Anteru-opencltutorial-cb1df4439f83.zip
inflating: Anteru-opencltutorial-cb1df4439f83/.hg_archival.txt
...
inflating: Anteru-opencltutorial-cb1df4439f83/main.cpp
inflating: Anteru-opencltutorial-cb1df4439f83/test.ppm
$ cd Anteru-opencltutorial-cb1df4439f83
/pub/tmp/Anteru-opencltutorial-cb1df4439f83
$ cmake .
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
...
-- Found OpenCL: /usr/lib/libOpenCL.dll.a
-- Configuring done
-- Generating done
-- Build files have been written to: /pub/tmp/Anteru-opencltutorial-cb1df4439f83
$ make
Scanning dependencies of target clTut
[ 50%] Building CXX object CMakeFiles/clTut.dir/main.cpp.o
/pub/tmp/Anteru-opencltutorial-cb1df4439f83/main.cpp: In function β€˜int main()’:
/pub/tmp/Anteru-opencltutorial-cb1df4439f83/main.cpp:249:9: warning: β€˜_cl_mem* clCreateImage2D(cl_context, cl_mem_flags, const cl_image_format*, size_t, size_t, size_t, void*, cl_int*)’ is deprecated [-Wdeprecated-declarations]
&error);
^
In file included from /pub/tmp/Anteru-opencltutorial-cb1df4439f83/main.cpp:10:0:
~~~~~~~^~~~~~
...
[100%] Linking CXX executable clTut.exe
[100%] Built target clTut
The built program seems to run fine
$ ./clTut.exe
Found 1 platform(s)
(1) : Portable Computing Language
Found 1 device(s)
(1) : pthread-Intel(R) Core(TM) i5-8250U CPU # 1.60GHz
Context created

How to resolve build errors in OSX cctools?

I am attempting to build tools from the CCtools collection for OSX, under OSX. I am doing this for research purposes.
When I run the makefile for the entire project I get a linker error:
=========== /Applications/Xcode.app/Contents/Developer/usr/bin/make all for misc =============
cc -Os -DLTO_SUPPORT -DTRIE_SUPPORT -g -Wall -I. -I./../include -I. -I/usr/local/include -c -o ./lipo.o lipo.c
cc -nostdlib -r \
-o ./lipo.private.o \
./lipo.o -L./../libstuff -lstuff
cc -o ./lipo.NEW \
./lipo.private.o
Undefined symbols for architecture x86_64:
"_is_llvm_bitcode_from_memory", referenced from:
_main in lipo.private.o
_check_archive in lipo.private.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [lipo.NEW] Error 1
make: *** [all] Error 1
Running the make files of the individual tools nets me a similar linker error. This occurs for all versions of the CCtools I have tried (Including 877.5, the latest source, and 862, the version that my installed tools are at.)
What am I missing to make the tools compile?
How can I coax the linker to let me know what library I am missing?
Where can I find instruction to make the tools compile?
Additional info:
I have XCode and developer tools installed, and I build projects all the time.
I also have gcc installed from brew.
I must be missing something stupid.
I cannot give you the exact solution for your version 877.5 but the overall problem is that the compilation requires some of the LLVM code on your machine. It might be as simple as missing include directories for the LLVM headers or can require linking some of the LLVM libraries. In your case, they seem to be related to LLVM Bitcode somehow.
In my case, I have just built a subset of the more recent version cctools-895. It complained about a missing symbol related to LLVM's llvm-c/lto.h header. The error went away when I added the following to the problematic file's compilation rules:
-I/opt/llvm-7.0.0/include/
The version of LLVM 7 was a random guess that worked because I had it on my machine. You might want to tailor your include paths to point to the correct version of LLVM that the cctools are supposed to be built with.
The easiest way to get the LLVM on your machine is to get a binary version from the LLVM Download page.