Nm shows symbol in shared library, but undefined symbol when linking - c++

As a mental exercise, I'm trying to write a program that links directly against the GPU driver of my Macbook Pro rather than using Apple's Metal framework. Some exploration led me to this file (presumably specific to my particular hardware):
/System/Library/Extensions/AMDRadeonX6000MTLDriver.bundle/Contents/MacOS/AMDRadeonX6000MTLDriver
Running file on it confirms this is a Mach-O 64-bit dynamically linked shared library.
Running nm on it tells me it's a superset of AMD's ROCr runtime. One symbol in particular that interests me is this one:
$ nm -gD AMDRadeonX6000MTLDriver | grep "hsa_init"
00000000001cca20 T __ZN3HSA8hsa_initEv
$ nm -gCD AMDRadeonX6000MTLDriver | grep "hsa_init"
00000000001cca20 T HSA::hsa_init()
So I wrote this simple program (rocr_test.cpp):
typedef int hsa_status_t;
namespace HSA {
hsa_status_t hsa_init();
}
int main() {
HSA::hsa_init();
return 0;
}
And compiled it like so:
$ clang++ rocr_test.cpp -c
$ clang++ rocr_test.o /System/Library/Extensions/AMDRadeonX6000MTLDriver.bundle/Contents/MacOS/AMDRadeonX6000MTLDriver
Undefined symbols for architecture x86_64:
"HSA::hsa_init()", referenced from:
_main in rocr_main-95c854.o
ld: symbol(s) not found for architecture x86_64
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
However, nm on the object file shows the linker should look for a symbol with the same name:
$ nm rocr_test.o
U __ZN3HSA8hsa_initEv
0000000000000000 T _main
Why am I seeing this linker error, when nm shows that a symbol with this exact name clearly exists in the shared library?

Apple's compiler is a bit different, and in order to link with libraries it needs to use a ".tbd" file. This is a textual file containing the symbol list, a UUID and the basic details of a mach-O it is linked against. You can find plenty of examples of those in the SDK (go to the SDK root and find . -type f -name "*.tbd"). The TBD would look something like:
--- !tapi-tbd-v3
archs: [ x86_64 ]
uuids: ['x86_64: 8891E6F5-0B7C-3CC7-88C1-9F5303311EC7' ]
platform: ios
install-name: /System/Library/Extensions/AMDRadeonX6000MTLDriver.bundle/Contents/MacOS/AMDRadeonX6000MTLDriver
objc-constraint: none
exports:
- archs: [ x86_64 ]
symbols: [ __Z34amdMtl_GFX10_GetFallbackFamilyNameP15GFX10_HwInfoRec, __Z35amdMtl_GFX10_GetFallbackProductNameP15GFX10_HwInfoRec, __Z25amdMtl_GFX10_AllocLsHsMgrP15GFX10_MtlDeviceP14AMDPPMemMgrRec, ...
You'd have to create a TBD for the Bundle, (the above was created using jtool2 --tbd), and direct the compiler to use it (or place it in the SDK directory) and that should (hopefully) work.

If has_init is not part of a class, then you can still call the function by it's mangled name. However, this will only work if it is a free function. If it is part of a class, then you can not really call it without class definition, as you don't know what it does to the class members and you would have to pass the object as the first argument.
#include <iostream>
#include <dlfcn.h>
using namespace std;
typedef int hsa_status_t;
typedef hsa_status_t (*hsa_init_t)();
hsa_init_t hsa_init;
const char *hsa_init_name = "__ZN3HSA8hsa_initEv";
const char *libPath = "/System/Library/Extensions/AMDRadeonX6000MTLDriver.bundle/Contents/MacOS/AMDRadeonX6000MTLDriver";
int main()
{
void *libraryHandle = dlopen(libPath, RTLD_NOW);
if (!libraryHandle)
{
cout << "Error opening library: " << libPath << " Error: " << dlerror() << endl;
return 0;
}
dlerror(); // clear any existing error
hsa_init = (hsa_init_t)dlsym(libraryHandle, hsa_init_name);
if (!hsa_init)
{
cout << "Error importing symbol: " << hsa_init_name << " Error: " << dlerror() << endl;
return 0;
}
hsa_init();
return 0;
}

Related

g++ failing when trying to use GDAL library [duplicate]

This question already has answers here:
What is an undefined reference/unresolved external symbol error and how do I fix it?
(39 answers)
Closed 3 months ago.
I just want to compile this easy example of the GDAL library in my Ubuntu 22.04 system using the system-packed g++, version 11.3.0:
#include <iostream>
#include "gdal_priv.h"
#include "cpl_conv.h"
#include "gdal.h"
using namespace std;
int main(int argc, char* argv[])
{
GDALDataset *poDataset;
GDALAllRegister();
poDataset = (GDALDataset *) GDALOpen(argv[1], GA_ReadOnly);
if (poDataset == NULL)
{
cout << "No dataset loaded for file " << argv[1] << ". Exiting." << endl;
return 1;
}
cout << "Driver: "
<< poDataset->GetDriver()->GetDescription()
<< "/"
<< poDataset->GetDriver()->GetMetadataItem(GDAL_DMD_LONGNAME)
<< endl;
cout << "Size: "
<< poDataset->GetRasterXSize() << "x"
<< poDataset->GetRasterYSize() << "x"
<< poDataset->GetRasterCount()
<< endl;
if (poDataset->GetProjectionRef() != NULL)
{
cout << "Projection: " << poDataset->GetProjectionRef() << endl;
}
}
Of course I installed the GDAL libraries, as it can be seen here:
~$ dpkg -l | grep gdal
ii gdal-bin 3.4.1+dfsg-1build4 amd64 Geospatial Data Abstraction Library - Utility programs
ii gdal-data 3.4.1+dfsg-1build4 all Geospatial Data Abstraction Library - Data files
ii libgdal-dev 3.4.1+dfsg-1build4 amd64 Geospatial Data Abstraction Library - Development files
ii libgdal30 3.4.1+dfsg-1build4 amd64 Geospatial Data Abstraction Library
ii python3-gdal 3.4.1+dfsg-1build4 amd64 Python 3 bindings to the Geospatial Data Abstraction Library
Everything seems to be settled and ready to go, but then, when I trigger this g++ command to compile my little program
g++ -I/usr/include/gdal -L/usr/lib -lgdal open_file.cpp -o open_file -g
it fails with this output:
/usr/bin/ld: /tmp/ccU6PwuP.o: in function `main':
/home/jose/Code/concepts/gdal/open_file.cpp:13: undefined reference to `GDALAllRegister'
/usr/bin/ld: /home/jose/Code/concepts/gdal/open_file.cpp:14: undefined reference to `GDALOpen'
/usr/bin/ld: /home/jose/Code/concepts/gdal/open_file.cpp:29: undefined reference to `GDALDataset::GetRasterXSize()'
/usr/bin/ld: /home/jose/Code/concepts/gdal/open_file.cpp:30: undefined reference to `GDALDataset::GetRasterYSize()'
/usr/bin/ld: /home/jose/Code/concepts/gdal/open_file.cpp:31: undefined reference to `GDALDataset::GetRasterCount()'
/usr/bin/ld: /home/jose/Code/concepts/gdal/open_file.cpp:34: undefined reference to `GDALDataset::GetProjectionRef() const'
/usr/bin/ld: /home/jose/Code/concepts/gdal/open_file.cpp:36: undefined reference to `GDALDataset::GetProjectionRef() const'
collect2: error: ld returned 1 exit status
Which doesn't make any sense, because I am indeed passing the GDAL libraries in -I/usr/include/gdal and the definition of the "undefined" references do exist in the multiple .h files there.
Moreover, this works using clang++:
clang++ -I/usr/include/gdal -L/usr/lib -lgdal open_file.cpp -o open_file -g
Did anyone have a similar issue, or can give some hint on where the problem might be? Thank you.
Include paths have nothing to do with the symbols.
-I/usr/include/gdal -L/usr/lib both are not necessary as they are set by default. But you should use #include <gdal/gdal.h>, not just <gdal.h> and certainly not "gdal.h".
Move -lgdal after all other cpp/object files.
In general, it should be g++ <OPTIONS> <OBJECTS> <LIBRARIES> where library A which uses symbols from lib B should appear after B i.e. -lB -lA, the order matters for ld. Because it will use the library to resolve just the currently missing symbols and then will promptly forget the library ever existed. So any newly found unresolved symbols will not be resolved, hence shifting the library arguments "right". One can resolve circular dependencies by repeating libraries more than once.

dlopen succeeds (or at least seems to) but then dlsym fails to retrieve a symbol from a shared library

In an attempt to undersand how lazily loaded dynamic libraries work, I've made up the following (unfortunately non-working) example.
dynamic.hpp - Header of the library
#pragma once
void foo();
dynamic.cpp - Implementation of the library
#include "dynamic.hpp"
#include <iostream>
void foo() {
std::cout << "Hello world, dynamic library speaking" << std::endl;
}
main.cpp - main function that wants to use the library (edited from the snippet in this question)
#include <iostream>
#include <dlfcn.h>
#include "dynamic.hpp"
int main() {
void * lib = dlopen("./libdynamic.so", RTLD_LAZY);
if (!lib) {
std::cerr << "Error (when loading the lib): " << dlerror() << std::endl;
}
dlerror();
auto foo = dlsym(lib, "foo");
auto error = dlerror();
if (error) {
std::cerr << "Error (when loading the symbol `foo`): " << error << std::endl;
}
dlerror();
using Foo = void (*)();
(Foo(foo)());
}
Compilation and linking¹
# compile main.cpp
g++ -g -O0 -c main.cpp
# compile dynamic.cpp into shared library
g++ -fPIC -Wall -g -O0 -pedantic -shared -std=c++20 dynamic.cpp -o libdynamic.so
# link
g++ -Wall -g -pedantic -L. -ldynamic main.o -o main
Run
LD_LIBRARY_PATH='.' ./main
Error
Error (when loading the symbol `foo`): ./libdynamic.so: undefined symbol: foo
Segmentation fault (core dumped)
As far as I can tell, the error above clearly shows that the library is correctly loaded, but it's the retrieval of the symbol which fails for some reason.
(¹) A few options are redundant or, at least, not necessary. I don't think this really affects what's happening, but if you think so, I can try again with the options you suggest.
auto foo = dlsym(lib, "foo");
Perform the following simple thought experiment: in C++ you can have overloaded functions:
void foo();
void foo(int bar);
So, if your shared library has these two functions, which one would you expect to get from a simple "dlsym(lib, "foo")" and why that one, exactly?
If you ponder and wrap your brain around this simple question you will reach the inescapable conclusion that you must be missing something fundamental. And you are: name mangling.
The actual symbol names used for functions in C++ code are "mangled". That is, if you use objdump and/or nm tools to dump the actual symbols in the shared libraries you will see a bunch of convoluted symbols, with "foo" hiding somewhere in the middle of them.
The mangling is used to encode the "signature" of a function: its name and the type of its parameters, so that different overloads of "foo" produce distinct and unique symbol names.
You need to feed the mangled name into dlsym in order to resolve the symbol.

Linking Error due to symbol(s) not found in Bjarne Stroustrup "Programming and Practices using c++"

I'm new to c++ (and compiled languages in general) and am doing the drill at the end of chapter 8 in Bjarne Stroustrup "Programming and Practices using c++" but I'm getting the following error when I try to compile the code
➜ Desktop g++ -std=c++11 *.cpp -o use
Undefined symbols for architecture x86_64:
"_foo", referenced from:
print_foo() in my-4f7853.o
_main in use-46cb26.o
(maybe you meant: __Z9print_foov)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I've also tried using g++ -c my.cpp use.cpp followed by g++ -o use.exe my.o use.o but this gave the same error. The other approach I tried was g++ -c use.cpp -o use.exe, however use.exe produced no output when it ran. The source code files are
my.h
extern int foo;
void print_foo();
void print_int(int);
my.cpp
#include "my.h"
#include <iostream>
void print_foo() {
std::cout << foo << '\n';
}
void print_int(int num) {
std::cout << num << '\n';
}
use.cpp
#include "my.h"
#include <iostream>
int main() {
std::cout<<"DSGFSGFSG"<< '\n';
foo = 7;
print_foo();
int i = 99;
print_int(i);
}
I've looked at other questions that are similar (if not seemingly the same is in Link-time errors in VS 2013 while compiling the C++ program - B. Stroustrup's PPP using C++: Ch. 8 - Q1 Drill?) but the solutions haven't worked for me. Is the problem to do with my compilation using g++ or have I made a more fundamental error?
The global variable foo is only declared in your header file.
extern int foo;
You also need to define it in my.cpp
int foo;
The declaration is a promise: "it exists somewhere".
The definition actually reserves some storage for this variable.
So your linker complains because some code relying on this
promise needs to access this missing storage.

"Duplicate symbol" when attempting to compile two .cpp files (from XCode) in terminal (MacOSX)

I have two .cpp files, main.cpp and secondFile.cpp:
#include <iostream>
int main()
{
std::cout << "Hello, World!\n" << std::endl;
std::cout << "I was also able to add this line!" << std::endl;
return 0;
}
And
#include <iostream>
int main()
{
std::cout << "This was from the second file!" << std::endl;
return 0;
}
I have successfully run g++ -o main.cpp main and g++ -o secondFile.cpp secondFile, as well as run each of their corresponding executables. However when I attempt to compile them simultaneously into a single executable g++ -o main.cpp secondFile.cpp bothScripts or clang++ main.cpp secondFile.cpp -o bothScripts I receive the following error:
"duplicate symbol _main in:
/var/folders/49/38grlkzs44zcth3v_dw9m9dm0000gn/T/main-d43536.o
/var/folders/49/38grlkzs44zcth3v_dw9m9dm0000gn/T/secondfile-2bee63.o
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)"
Clearly something is being loaded twice, but I am unsure whether this is a library (iostream), that I've named both sections 'main', or something else entirely. There are certainly questions similar to this already, but many are convoluted and not as fundamental for new C++ members (hence my question here).
Context: My rationale is to practice building executables from multiple .cpp files. Is there a better way to go about this? (New to C++ but not to programming/code as a whole.)
The reason for your error is simple. You have 2 main() functions. As you should know, in a C++ program, the function main() generally defines the entry point of a program. When each of the files are compiled together, and have their own main() function, the compiler gets confused and throws an error. To solve this, simply change the name of the main() function in one file, and call it from the other file, if you are planning to run them together.

#include <lib.h> gives symbol not found, why?

I have this code:
#include <iostream>
#include <mp4.h>
int main (int argc, char * const argv[]) {
// insert code here...
std::cout << "Hello, World!\n";
MP4Read("filename", MP4_DETAILS_ALL );
return 0;
}
And i've added -I/opt/local/include and -L/opt/local/lib to the path (where the mp4 library resides after installing it through macports), but all i get is:
Undefined symbols: "_MP4Read",
referenced from:
_main in main.o ld: symbol(s) not found
Even though XCode finds it and autocompletes properly...
You need to link the library most likely, i.e. add -lmp4 or similar to your linking commands.
You have only specified the paths. You need to link in the mp4 library. Something like the following:
g++ -I /.../ -L /.../ -lmp4 -o out main.cpp
The -L flags tell the compiler where to look, the -l flag tells it what to look for.