Im comparing two binary files generated by versions of same code that could or could not be equal. Meld showed a small difference between the binaries, but completely disassembling both binaries revealed that every function defined was identical.
Suspecting file metadata i did:
readelf -a my_file1.so > log1.txt
readelf -a my_file2.so > log2.txt
diff log1.txt log2.txt
Result is:
772c772
< Build ID: 067b24a4b8afa6c823ffb5462d344c5021496b66
---
> Build ID: 7925f5c881822348c17221fd1420e7df9fdb5633
I found no good explanation of what is elf-file build id, so my main question is:
My recompilations of the code generate identical build IDs. What could have caused a different build ID of the different compilation, aside from last person who compiled it explicitly using --build-id=something?
If it helps, build architecture is powerpc64le.
I found no good explanation of what is elf-file build id
Copying from my comment on your other question: the build-id is a unique checksum added by the linker and computed over all "important" bits of the ELF file. Its primary use is to make sure that the core file matches the binary which produced it (build-id is located very near the start of the binary, and is included in the core for this purpose).
Build-id is also used to locate correct debug info -- the best practice is to build with -O2 -g, save this "full debug" binary, then run strip -g exe exe.stripped and use the exe.stripped in production. When you get a core dump from production, use the original exe to debug it.
What could have caused a different build ID of the different compilation
Your build is likely not bit-identical. Actually achieving bit-identical rebuilds requires some effort.
One common cause of non-bit-identical rebuilds is using __DATE__ and/or __TIME__ in some message.
If you build your shared library from the same sources twice, and get a non-bit-identical rebuild, you can compare contents of each ELF section (using readelf -x $section ...). Chances are you'll find differences in .rodata or .data.
Related
I have an elf file, while analysing the mapfile and elf using elfparser, I saw a section called .Debug_info, which is taking largest memory.
I am compiling for xtensa DSP, using xt-xc++, I haven't used -g option also given -o2 optimization level.
is it possible to remove this for release builds?
section called .debug_info, which is taking largest memory.
Note that this section does not have SHF_ALLOC flag, and therefore does not take any RAM at runtime (it only takes space in the filesystem). Of course if you use ramdisk, then that section still does cost you RAM in the end.
is it possible to remove this for release builds?
Yes: none of the .debug* sections are necessary at runtime, and all of them can be safely stripped.
-g0 and -s option didn't work.
It's likely that you are getting .debug_* sections from libraries that you are linking in, and not from your own code. The -g was present when the libraries were compiled, so building with -g0 doesn't have any effect.
It's surprising that -s didn't work, but maybe your compiler interprets this flag differently.
In any case, you should use strip --strip-debug to get rid of .debug_* sections (note: this does not remove the symbol table).
The best practice is actually to compile all your code with full debug info (-g), save the full debug binary for post-mortem analysis, use strip --strip-debug to make a release binary, and use that binary for actual distribution.
If/when the release binary crashes and leaves a core dump, having (saved) full-debug exactly matching binary greatly improves post-mortem analysis you can do.
We are looking for an procedure through which we can easily list down all the file which are compiled together to make an executable.
Use Case : Suppose, We have large repository and we want to know what all are the files existing in repository which are compiled to make an executable (i.e a.out)
For example :
dwarfdump a.out | grep "NS uri"
0x0000064a [ 9, 0] NS uri: "/home/main.c"
0x000006dd [ 2, 0] NS uri: "/home/zzzz.c"
0x000006f1 [ 2, 0] NS uri: "/home/yyyy.c"
0x00000705 [ 2, 0] NS uri: "/home/xxxx.c"
0x00000719 [ 2, 0] NS uri: "/home/wwww.c"
but it doesn't listed down the all the header files.
please suggest.
How to Extract Source Code From Executable with Debug Symbol Available ?
You cannot do that. I guess you are on Linux/x86-64 (and your question is operating system and ABI specific, and debugging format specific). Of course, you should pass -g (or even -g3) to all the gcc compilation commands for your executable. Without that -g or -g3 option used to compile every translation unit (including perhaps those of shared libraries!) you might not have enough information.
Even with debug information in DWARF format, the ELF executable don't contain source code, but only references to source code (e.g. source file path, position as line and column numbers). So the debug information contains stuff like file src/foo.c, line 34 column 5 (but don't give anything about the content of src/foo.c near that position). Of course once gdb knows the file path src/foo.c it is able to read that source file (if available and up to date w.r.t. executable) so it can list it.
Extracting that debugging meta-data is a different question. Once you have understood DWARF you could use tools like objdump or readelf or addr2line or dwarfdump or libdwarf; and you could also script gdb (recent versions of GDB may be extendable in Python or in Guile) and use it on your ELF executable.
Perhaps you should consider Ian Taylor's libbacktrace. It uses the DWARF information to provide nice looking backtraces at runtime.
BTW, cgdb is (like ddd) only a front-end to gdb which does all the real work of processing that DWARF information. It is free software, you can study its source code.
i have only a.out then i want to list done file names
You might try dwarfdump -i | grep DW_AT_decl_file and you could use some GNU awk command instead of grep. You need to dive into the details of DWARF specifications and you need to understand more about the elf(5) format.
It doesn't listed down the all the header files
This is expected. Most header files don't contain any code, only declarations (e.g. printf is not implemented in <stdio.h> but in some C source file of your C standard library, e.g. in tree/src/stdio/printf.c if you use musl-libc; it is just declared in /usr/include/stdio.h). DWARF (and other debug information formats) are describing the binary code. And some header files get included only to give access to a few preprocessor macros (which get expanded or skipped at preprocessing time).
Maybe you dream of homoiconic programming languages, then try Common Lisp (e.g. with SBCL).
If your question is how to use gdb, then please read the Debugging with GDB manual.
If your question is about decompilers, be aware that it is an impossible task in general (e.g. because of Rice's theorem). BTW, programs inside most Linux distributions are generally free software, so it is quite easy to get the source code (and you could even avoid using proprietary software on Linux).
BTW, you could also do more things at compilation time by passing more flags to gcc. You might pass -H or -M (etc...) to gcc (in addition of -g). You could even consider writing your own GCC plugin to collect the information you want in some database (but that is probably not worth the effort). You could also consider improving your build automation (e.g. adding more into your Makefile) to collect such information. BTW, many large C programs use some metaprogramming techniques by having some .c files perhaps containing #line directives generated by tools (e.g. bison) or scripts, then what kind of file path do you want to keep ??
We are looking for an procedure through which we can easily list down all the files which are compiled together to make an executable.
If you are writing that executable and compiling it from its source code, I would suggest collecting that information at build time. It could be as trivial as passing some -M and/or -H flag to gcc, perhaps into some generated timestamp.c file (see this for inspiration; but your timestamp.c might contain information provided by gcc -M etc...). Your timestamp file might contain git version control metadata (like generated in this Makefile). Read also about reproducible builds and about package managers.
List items 1- 4 are the steps that I did.
List item 5 describes the problem
List item 6 provides additional information
I have compiled a C source code say c1.c with -g flag.
I have also a
dynamic shared library say liba1.so built with -g for all the source
files that it has.
I built the executable say exe1 by linking c1.o (c1.c object code) with the liba1.so .
I do gdb exe1. and am able to step through the sources of c1.c. When c1 calls the shared library, I am also able to put a breakpoint on a function in the shared library.
However, when I try to step through the function, it says that "Single stepping until exit from function foo1 ,which has no line number information" Also it should ordinarily show the value of the parameters passed into the function foo1 but does not do that. This happens for all functions in the shared library including some very big ones so the values cannot be optimized out
I did an objdump -t on the shared library AND the executable - it shows the symbol table (the fact that I can set a breakpoint on the function also supports this). Also, I can see the values of the variables used in the file c1.c So what should I do in order to ensure that I can see the values of the local variables inside the shared library. Here are the other arguments that are being used to compile the shared library "-O2 -std=gnu99 -Werror -fno-stack-protector -Wstack-protector --param ssp-buffer-size=1 -g -nostdinc". doing info f and trying to look at memory addresses on the frame also does not give any information.
I am looking for some suggestion to at least troubleshoot it. Can I know using objdump (or any other utility) if a shared library has line number information.
I am looking for some suggestion to at least troubleshoot it.
The most likely reason for no line number information, is that there is in fact no line number information, and the most likely reason for that is that you have two copies of liba1.so -- one that has debug info, and one that doesn't, and you are loading at runtime the latter.
First step: (gdb) info shared will tell you exactly which liba1.so is loaded.
If it is in fact the version that you've just built with -g, you should verify that it does have the debug info you are expecting. The exact commands for doing so are platform specific (and you didn't tell which platform you are on). On an ELF platform, objdump -g liba1.so or readelf -w liba1.so should work.
One common reason for -g code to not have debug info is presence of -s (strip) flag on the link line; make sure you don't have "stray" flags on your link line. Some platforms also require -g to be used at link time in addition to compile time.
How do I analyze the core dump (using gdb)
which is not compiled with -g GCC option ?
Generate a map file. The map file will tell you the address that each function starts at (as an offset from the start of the exe so you will need to know the base address its loaded too). So you then look at the instruction pointer and look up where it falls in the map file. This gives you a good idea of the location in a given function.
Manually unwinding a stack is a bit of a black art, however, as you have no idea what optimisations the compiler has performed. When you know, roughly, where you are in the code you can generally work out what ought to be on the stack and scan through memory to find the return pointer. its quite involved however. You effectively spend a lot of time reading memory data and looking for numbers that look like memory addresses and then checking that to see if its logical. Its perfectly doable and I, and I'm sure many others, have done it lots of times :)
With ELF binaries it is possible to separate the debug symbols into a separate file. Quoting from objcopy man pages:
Link the executable as normal (using the -g flag). Assuming that is is called foo then...
Run objcopy --only-keep-debug foo foo.dbg to create a file containing the debugging info.
Run objcopy --strip-debug foo to create a stripped executable.
Run objcopy --add-gnu-debuglink=foo.dbg foo to add a link to the debugging info into the stripped executable.
that should not be a problem , you can compile the source again with -g option and pass gdb the core and the new compiled debug binary, it should work without any problem.
BTW You can generate a map file with the below command in gcc
gcc -Wl,-Map=system.map file.c
The above line should generate the map file system.map, Once the map file is generated you can map the address as mentioned above but i am not sure how are you going to map the share library, this is very difficult
When I tried to debug an executable:
(gdb) break +1
No symbol table is loaded. Use the "file" command.
What does that mean exactly?
Is the symbol table appended to the executable?
There are two sets of symbols that gdb uses.
The -g set are debugging symbols, which make things a lot easier as they allow you to see your code and look at variables while debugging.
Another set of symbols is included by default when you compile. These are the linking symbols and live in the ELF (executable linkable format) symbol table. This contains a lot less info than the debug symbols, but contain the most important stuff, such as the addresses of the things in your executable (or library or object file). Without this information gdb won't even know where main is, so (gdb) break main would fail.
If you don't have the debugging symbols ( -g ) then you will still be able to (gdb) break main but you gdb will not have any concept of the lines of code in your source file. When you try to step through the code you will only advance 1 machine instruction at a time, rather than a line at a time.
The strip command is often used to strip off symbols from an executable (or other object file).
This is often used if you don't want someone to be able to see the symbols or if you want to save space in the file. Symbol tables can get big. Strip removes both the debug symbols and the linker symbols, but it has several command line switches which can limit what it removes.
If you run the file command on your program one of the things it will tell you is weather or not the executable is has been stripped.
$ gcc my_prog.c -o my_prog
$ file my_prog
my_prog: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15, not stripped
$ strip my_prog
my_prog: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15, stripped
$
It's because you didn't compile with debugging turned on. Try gcc -g file.c
The symbol table contains debugging information that tells a debugger what memory locations correspond to which symbols (like function names and variable names) in the original source code file. The symbol table is usually stored inside the executable, yes.
gdb is telling you that it can't find that table. If you compiled with gcc, unless you used the -g flag, it will not include the symbol table in the file. The easiest method is probably to recompile your file with -g. gdb should then automatically find the symbol table information.
Either add the -g flag to the command line arguments of gcc or to the Makefile that you used to compile the program. (A lot of times, there will be a variable called CFLAGS or similar inside the Makefile).
If you are trying to debug an arbitrary third-party program, a lot of times the information will have been "stripped" out of it. This is done to make reverse engineering harder and to make the size of the executable file smaller. Unless you have access to the source code and can compile the program yourself, you will have a very hard time using gdb on it.
Find the entry point of the application.
objdump -f main
main: file format elf32-i386
architecture: i386, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x08048054
Put a breakpoint there using the gnu debugger
gdb
exec-file main
break *0x8048054
set disassemble-next-line on
run
Then step through the code
gdb
stepi
Special Notes
If you are using the latest version of Ubuntu you would not be affected by this, but you may run into this bug if you are running Ubuntu 10.04 or older.
https://bugs.launchpad.net/ubuntu/+source/gdb/+bug/151518G
The solution would be to start debugging at the entry point address plus one.