I'm trying to list all the symbols inside a dylib in order to know which one needs to be updated inside my app (as my app store all links to function as well as their name , to create a dynamic module mechanism that auto detect/update functions automatically at runtime).
Basically what Im searching for is similar as what the nm command do but in code (and not by launching nm on the console then get the output and parse).
I think rsym_macosx could give you some helps to do something like nm. I don't know what kind of symbols you really want to do within your code, but you could modify it based on that and Mach-O format.
Related
I use a symbol store on a network drive to collect debug symbols for our application. The symbols are used while debugging a crash dump file that has been loaded into Visual Studio. I have Powershell scripts in place that manage deleting ‘old’ versions of symbols. Or, so, I thought.
Recently, while working on another script, I noticed that using symstore to delete symbols doesn’t actually delete anything. When I use symstore to delete symbols I see a “0000000161.deleted” file in 000Admin folder for example. This file contains entries for the associated debug symbols. It contains entries like:
"Aggregat.pdb\B4E7044117F0469CB321E9FA9003E4CA5","C:\temp\pdbs\1823\Aggregat.pdb"
The first entry above is the folder for the associated debug symbols for the ‘Aggregat’ module. I would expect that when I delete symbols using the transaction ID, that the corresponding folders (ie. B4E7044117F0469CB321E9FA9003E4CA5) would also be removed. It appears like that’s not the case.
Is my assumption correct? Am I responsible for the full cleanup of the symbols store structure?
I know ldd can only take a binary as its parameter what I am asking here is how to run ldd with a binary, say, mybin, with parameter of the binary. For instance, mybin --myparam ./configfile.conf.
The linker will differ if I add the conf file for my binary because it loads some plugins in runtime with the plugins' shared object files, plugin1.so something like this. I have some undefined reference issue but I still got no idea which .so file I was missing.
If I run ldd ./mybin, everything is linked and running the plain binary is fine. Once I add the conf file for my binary, to let it loads some plugins shared library, then my binary will report errors when loading those library(coded exception throw, with some undefined reference error messages).
So if there is way to run ldd with mybin --myparam ./a.file something like this would help a lot.
Use the LD_DEBUG environment variable for this. To see the relevant options, run any command with LD_DEBUG=help. For example, running LD_DEBUG=help ls on my machine gives this output:
LD_DEBUG=help ls
Valid options for the LD_DEBUG environment variable are:
libs display library search paths
reloc display relocation processing
files display progress for input file
symbols display symbol table processing
bindings display information about symbol binding
versions display version dependencies
scopes display scope information
all all previous options combined
statistics display relocation statistics
unused determined unused DSOs
help display this help message and exit
To direct the debugging output into a file instead of standard output
a filename can be specified using the LD_DEBUG_OUTPUT environment variable.
One way of debugging your dlopens, or whatever late loading mechanism you are using, would be to run your executables with the relevant args with LD_DEBUG=all. This would give you a lengthy output detailing symbol lookups and search paths. This output would also tell you about resolution failures.
We have a large codebase with > 40 projects (in VS lingo) creating several DLLs/SOs (~15) and an EXE.
There are a few Utility projects which are statically linked to create the EXE and also used by most of the DLLs. Ideally, we'd want these Utility projects to be DLLs too, so that the code isn't duplicated in each of the DLLs that depend on them.
Are there any tools to do a binary analysis on the DLLs to see how much of duplication exists (code + data)? Getting an estimate on this would help.
No tools, just the one between your ears. You want to focus on the projects that link a static library, find the ones where the same static library is used more than once. That's the start point for assuming that a function can be linked in more than once.
Then you can use the linker's /VERBOSE option, it shows you which functions are getting linked in from the static library. There's a lot of output from that option, it is however brief and easy to parse.
As an alternative, consider using the linker's /MAP option to generate a .map file. Which shows in detail which functions got linked into the final executable. Having the same function appear more than once in different .map files is your lead that it might be beneficial to put it in a DLL instead. Writing a little program in your favorite scripting language that processes the /VERBOSE output or .map files and finds matches is feasible.
Well, on a Unix/Linux/OSX system you'd do something like
for eachfile in *.exe *.dll ; do
nm $eachfile | sort | uniq > $eachfile.symbols.txt
done
cat *.symbols.txt | sort | uniq -c > count-duplicate-symbols.txt
sort -r count-duplicate-symbols.txt | less
The first three lines say "Dump the symbols out of each .exe and .dll file in the current directory; store each dump in a separate file. By the way, if the same line appears multiple times in a single file, just store it once."
The line beginning with cat says "Count the number of times each line appears across all the files we just produced. Write a new file named count-duplicate-symbols.txt that contains the duplicated lines with their counts."
The final line says "Sort this file by the number of duplicates (in increasing order), and pipe it to the terminal so I can read it."
If you wanted to see which source files contained the offending duplicate symbols, you could use grep for that.
Notice that this approach probably won't work for static symbols (functions and variables), and it may produce false positives for things like inline functions which are supposed to appear everywhere. You could filter out symbols appearing in linkonce sections, prettyprint the output with c++filt, etc. etc.
Some of these tools are definitely available for Windows. I don't know if they all are.
I am having trouble with very large object files being produced. We are working with VxWorks 5.5.1, but we have a GCC 4.1.2 available.
Our modules are roughly 6.2MB in size, and we are looking for ways to reduce that. The problem seems to be mainly caused by excessive use of templates. When dumping the symbols in the file using nm I get a text-file of 1.8MB. This tells me that almost ⅓ of the file is just the names. Is there any way to reduce the file size?
The following approaches have not worked:
--strip-all seems to have no effect - the output is the same as using --strip-debug
I cannot use --gc-sections, because it is not supported for that platform (the option is simply ignored)
I understand that VxWorks links the code at load time, but all it has to link is the C++ runtime library, and I don't want any symbols to be added to the global symbol table, so there should be a way to strip that information, right?
For reference, here is my linker version:
i386-wrs-vxworks-ld.exe --version
>>> GNU ld (Wind River VxWorks G++ DWARF-EH 4.1-131) 2.17.50.20070509
>>> SPR fixes: cq103489 cq111170 cq116027 cq116652 cq118878 cq125145
and my compiler version:
i386-wrs-vxworks-g++.exe --version
>>> i386-wrs-vxworks-g++.exe (GCC) 4.1.2
I see what you mean by wanting to strip the symbols out of the object. But if you were to strip all the symbols, you wouldn't have any symbol to use as the entry point to start your application. But I believe you still have options. Unfortunately VxWorks 5 is known for having a not very effecient C++ compiler.
If you compiled your application into a *.a (archive - aka static library), you would be able to link this into your operating system at build time, and call this from within usrAppInit.c. This should allow for striping out symbols - or at least moving them to an optional downloaded symbol table. From your application build properties, select the Macros tab and add your archive to the LIBS macro.
To make an archive, goto the build properties and select the Rules tab, then select archive from the dropdown box.
To trim the size of your OS (with your application linked in), modify your vxworks settings. Disable as many components as you can. Also be sure to use a downloaded symbol table (development tool components -> symbol table components -> symbol table initialization componts -> selecte symbol table initialization -> downloaded symbol table).
This will strip all the symbols out of the OS, and make a downloadable symbol table, that can be downloaded after boot time to debug.
Good luck!
PS. Make sure you have turned off debug (-g) in your compiler. Maybe we could help more if you post your compiler switches.
The way we have normally handled this is to compress the image. You will also need to build the bootrom so that it will decompress to RAM before running. I believe that there are standard makefile options that will allow you to do this.
I have been working on a cross platform windowing library aimed to be used for OpenGL specifically, currently focusing on linux. I am making use of glload to manage OpenGL extensions, and this is being compiled, along with other libraries that I will use later, into an .so. This `.so is being dynamically loaded as you would expect, but at run time the program gives the following output (manually wrapped so it is easier to read):
_dist/x64-linux-debug/bin/test: Symbol `glXCreateContextAttribsARB' has \
different size in shared object, consider re-linking
Now, obviously I have tried re-linking, going as far as rebuilding the entire project many times (testing things out, not just blindly hoping it will magically make it all better). The program does seem to be willing to run as it will produce some logging output as I would expect it to. I have used nm to confirm that the 'symbol' is in the .so
nm _dist/x64-linux-debug/lib64/libvendor.so | grep glXCreateContextAttribsARB
00000000009e0e78 B glXCreateContextAttribsARB
If I use readelf to look at the symbols being defined I get the following (again, I have manually wrapped the first three lines for formatting sake):
readelf -Ws _dist/x64-linux-debug/bin/test \
_dist/x64-linux-debug/lib64/libvendor.so | \
grep glXCreateContextAttribsARB
348: 000000000062b318 8 OBJECT GLOBAL DEFAULT 26 glXCreateContextAttribsARB
421: 000000000062b318 8 OBJECT GLOBAL DEFAULT 26 glXCreateContextAttribsARB
1370: 00000000009e0e78 8 OBJECT GLOBAL DEFAULT 25 glXCreateContextAttribsARB
17464: 00000000009e0e78 8 OBJECT GLOBAL DEFAULT 25 glXCreateContextAttribsARB
I am afraid that this is about all I can offer to help, as I really do not know what to try or look into. Like I said, I am sure more will info will be need, so please just say an I will provide what I can. I am running these commands from my project root, encase you are wondering.
wilsonmichaelpatrick's answer is mostly correct, but using gdb is likely not the fastest way to find the problem, and will likely not work at all if you have a non-debug build.
First, you should confirm that there in fact is a problem:
readelf -Ws _dist/x64-linux-debug/bin/test _dist/x64-linux-debug/lib64/libvendor.so |
grep glXCreateContextAttribsARB
This should show the symbol being defined in test and libvendor.so, with different size.
Second, re-link test and libvendor.so with -Wl,-y,glXCreateContextAttribsARB flag. That will tell you which object files (or libraries) provide the (different) definitions.
Finally, preprocess the sources that produce above object files with -E and -dD flags, and see what's different between them.
Update:
I need help digesting what it is saying
Don't be helpless. Read man readelf, or just run it by hand. You'll see something like this:
readelf -Ws /bin/date | head -5
Symbol table '.dynsym' contains 75 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __ctype_toupper_loc#GLIBC_2.3 (2)
This tells you the meaning of the data you've got. In particular, this tells you that the size of the symbol in test and in libvendor.so is the same (8). Therefore, the problem is not in these two ELF files, but somewhere else. Run readelf on your other libraries, and look for definition of glXCreateContextAttribsARB that has a different size. Then follow the rest of the procedure.
The runtime is noticing that glXCreateContextAttribsARB as compiled in the shared object, and glXCreateContextAttribsARB as compiled in the main program (or maybe even some other shared object previously linked) have different sizes. This means that, in the separate builds for the shared object and whatever else references that object, they must be looking at different code (probably in a shared object) where this is defined. Sometimes this occurs because they are looking at different files, sometimes this occurs because of different #defines causing different interpretations of the same file. Whatever the reason, you absolutely need to make sure that the same symbol (e.g. a structure) is defined the same way (i.e. with the same member variables and size) across everything that is linked together at runtime.
It's actually a very good thing that it is refusing to run, as this is a catastrophe when two parts of the code interpret the same bit of memory in different ways at runtime. (Not too much of an exaggeration to say anything could happen if this was allowed to proceed.)
You might want to try just loading up the executable in gdb (without running it) and typing
info types
to see where it is defined, and then load the shared object in gdb (without running it) and doing another info types there to see what each of them thinks it's looking at. If it's the same thing, check the preprocessor directives.
I have faced a tedious issue related to objects of different sizes so I want to share my experience - even though it is clear to me that it is only one reason that might explain different object sizes - and not mandatorily the OP's.
The symptoms were objects of different sizes in debug mode, none in release mode. The linker produced the according warnings. The symbol names were hard to decipher but related to some unnamed static variables in instances of class templates.
The reason was the debug logging feature à la LOG("Do something.");. The LOG macro used the C ANSI macro __FILE__ which expanded to another path depending on whether the header was included by the application or by the shared library. And this string was exactly the aforementioned unnamed static variable.
Even more tedious was the fact that due to our make environment the __FILE__ macro sometimes expanded to, let's say, C:\temp\file.h and sometimes to C:\other\..\temp\file.h so that building the application and the library from the same place didn't solve the problem either.
I hope this piece of experience might spare some time to some of you.
In most cases you're probably just linking against the wrong library (a different version). For example, you have libfoo installed twice and link your executable with -L /path/to/version1 -lfoo but during runtime you link with /path/to/version2 (you can see this one with ldd yourprogram).
One reason could be that the executable was linked with -rpath,/path/to/version1 but (as recent versions do) this set the RUNPATH entry in the dynamic section; while you have LD_LIBRARY_PATH=/path/to/version2. When RUNPATH is set, LD_LIBRARY_PATH gets precedence. In this case delete the library from /path/to/version2 (or remove that path from LD_LIBRARY_PATH).
EXAMPLE
$ minimal
/home/carlo/minimal: Symbol `_ZN6libcwd8libcw_doE' has different size in shared object, consider re-linking
COREDUMP : /home/carlo/projects/libcwd/libcwd/elfxx.cc:2381: void libcwd::elfxx::objfile_ct::load_dwarf(): Assertion `size == sizeof(address)' failed.
(libcwd is smart enough to see it too; aka the problem here is with libcwd):
$ ldd minimal | grep libcwd_r
libcwd_r.so.5 => /usr/local/install/6.0.0-1ubuntu2/lib/libcwd_r.so.5 (0x00007f0b69840000)
$ echo $LD_LIBRARY_PATH
/usr/local/install/6.0.0-1ubuntu2/lib
$ objdump -a -x minimal | grep PATH
RUNPATH /opt/gitache/libcwd_r/888f62c44fd64f1486176bf9e35b36f79612790017c31f95e117fc59743a54ca/lib
Unsetting LD_LIBRARY_PATH or removing libcwd from that path results in
$ unset LD_LIBRARY_PATH
$ ldd minimal | grep libcwd_r
libcwd_r.so.5 => /opt/gitache/libcwd_r/888f62c44fd64f1486176bf9e35b36f79612790017c31f95e117fc59743a54ca/lib/libcwd_r.so.5 (0x00007f11d7298000)
and things work again. Or alternatively I could add to my CMakeLists.txt of the project:
$ set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--disable-new-dtags")
After which we get,
$ objdump -a -x minimal | grep PATH
RPATH /opt/gitache/libcwd_r/888f62c44fd64f1486176bf9e35b36f79612790017c31f95e117fc59743a54ca/lib
which now has precedence over LD_LIBRARY_PATH and therefore also solves the issue. This is not the recommended way however: if you set LD_LIBRARY_PATH you should know what you are doing. If that doesn't work, you should fix LD_LIBRARY_PATH or remove the offending library.