I am working on a shared library (.so in Linux) which has a XML file for a small database and that xml file is encrypted. Here is an abstract of my code:
void my_fucnt(char *in, char *out)
{
static char key[] = {0x34, 0x6c, 0x54....};
enrcryption(key, in, out);
}
First thing first; The other day I was examining the library with objdump and I found out that the many of the symbols (even those declared static) were found to be in the object file which I thought was revealing most of my code logic so I searched on internet and found out about strip utility so did I.
It would be nice to know that what methodology does strip utility applies and does it place addresses of symbols instead of their names?
Secondly, I still see the key in the .data section of object file which is revealing the database key although I have stripped the symbols. Is there any way I can hide that? or what other techniques can be applied so to encrypt my database file?
Any help would be appreciated.
What strip does is remove debug symbols and information. This can often take up a large part of an executable file, which is the reason the utility exists.
As for the key, it's going to be in there somewhere. You can obfuscate it (encrypting the key itself, store each byte of the key in different places, etc.) but if a cracker wants to find it he or she will find it. They are notoriously good at reverse engineering and figuring out what a piece of assembly code does.
Related
I'm working on a bare-metal microcontroller project that needs to be able to execute a blob of code loaded at runtime. The basic implementation is simple enough - write a linker script that can handle it being loaded in to RAM somewhere, leave a little vector table at the beginning of the image, memcpy it in to place, and then from the main firmware read the vector table and jump in to the module's entry point.
My question comes with how to convince the linker for the module to allow calls back in to the main firmware (avoid duplicating utility functions, share data with main firmware, etc).
Suppose the main firmware has some function:
int FunctionInMainFirmware() {
return 1234;
}
The main firmware complies/links to produce an elf that's later objdump'd to a bin flashed to the microcontroller. As expected, the function ends up linked in the main firmware's flash:
080b73b0 g F .text 00000008 _Z22FunctionInMainFirmwarev
So then we get to the loadable module:
using myFunc = int(void);
static const myFunc* f = reinterpret_cast<myFunc*>(0x080b73b0);
int FunctionInLibrary() {
return f();
// How do I make this work?
// return FunctionInMainFirmware();
}
This implementation works - the compiler correctly generates the call to the function in the main firmware. This is of course nasty and too unwieldy to do in the real world, but proved to me that if the code is generated how I want, it'll do what I want.
How do I convince the linker to treat my main firmware elf as if it was a static library? Essentially I just want it to read the linker map from the main firmware to resolve missing functions from the module.
I tried -l:MyMainFirmware.elf, which doesn't fail, but also doesn't produce a valid result:
It copies the entire contents of the main firmware's elf. This makes sense since it sees sections it should copy (ie, .text), so it puts them in the output and re-links it.
It doesn't link correctly, and does some weird math resulting in mangled addresses.
Is there some way to get the linker to just read the .map or .dmp files generated by the main build? Or to read the elf but ignore the contents, and only look at the symbol address (then don't futz with it)?
AVR g++ has a pointer size of 16 bits. However, my particular chip (the ATMega2560) has 256 KB of RAM. To support this, the compiler automatically generates trampoline sections in the same section of ROM as the current executing code that then contains the extended assembly code to jump into high memory or back. In order for trampolines to be generated, you must take the address of something that sits in high memory.
In my scenario, I have a bootloader that I have written sitting in high memory. The application code needs to be able to call a function in the bootloader. I know the address of this function and need to be able to directly address it by hard-coding the address in my code.
How can I get the compiler/linker to generate the appropriate trampoline for an arbitrary address?
Compiler and linker will only generate trampoline code when the far address is a symbolic address rather than a literal constant number already in code. something like (assuming the address you want to jump to is 0x20000).
extern void (*farfun)() = 0x20000;
farfun ();
Will definitely not work, it doesn't cause the linker to do anything because the address is already resolved.
You should be able to inject the symbol address in the linker command line like so:
extern void farfun ();
farfun ();
compiling "normally" and linking with
-Wl,--defsym,farfun=0x20000
I think it's clear that you need to make sure yourself that something sensible sits at farfun.
You will most probably also need --relax.
EDIT
Never tried this myself, but maybe:
You could probably try to store the function address in a table in high memory and declare it like this:
extern void (*farfunctable [10])();
(farfunctable [0])();
and use the very same linker command to resolve the external symbol (now your table at 0x20000 (in the bootloader) needs to look like this:
extern void func1();
extern void func2();
void ((*farfunctab [10])() = {
func1,
func2,....
};
I would recommend to put func1() ... func10() in a different module from farfunctab in order to make the linker know it has to generate trampolins.
I was planning on putting a dispatch struct (that is, a struct with function pointers to all the various functions). Your solution works well, but requires knowing all of the locations of all of the functions ahead of time. Is there a way to execute a function call to a far address that isn't known at compile time?
[...] My goal was to put the struct with pointers to the functions in a fixed location. That way, it would be a single thing that needed a fixed address rather than every external function.
So you have two applications, let's call them App and Boot, where Boot provides some functionalities that App wants to use. The following problems have to be addressed:
How to get addresses from Boot into App.
How to build a jump table for Boot.
Avoid constructs that will crash when App tries to use code from Boot, like: Using indirect calls or jumps, using static constructors or using static storage in Boot.
App uses Addresses of boot.elf directly
Linking with -Wl,-R,boot.elf
A simple way would be to just link app.elf against boot.elf be means of -Wl,-R,boot.elf. Option -R instructs the linker to use symbol values from the specified file without dragging any code. Problem is that there's no way to specify which symbols to use, for example this might lead to a situation where App uses libgcc functions from Boot.
Defining Symbols by means of -Wl,--defsym,symbol=value
A bit more control over which symbols are being defined can be implemented by following a specific naming convention. Suppose that all symbols from Boot that have "boot" in their name should be "exported", then you could just
> avr-nm -g boot.elf | grep ' T ' | awk '/boot/ { printf("--defsym %s=0x%s\n",$3,$1) }' > syms.opt
This prints global symbol values, and grep filters out symbols in the text section. awk then transforms lines like 00020102 T boot1 to lines like
--defsym boot1=0x00020102 which are written to an option file syms.opt. The option file can then be provided to the linker by means of -Wl,#syms.opt.
The advantage of an option file is that it is easier to provide than plain options in a build environment like make: app.elf would depend (amongst others) on syms.opt, which in turn would depend on boot.elf.
Defining Symbols in a Linker Script Snippet
An alternative would be to define the symbols in a linker script augmentation, which you would provide by means of -T syms.ld during link and which would contain
"boot1"=ABSOLUTE(0x00020102);
"boot2"=...
...
INSERT AFTER .text
Defining Symbols in an Assembly Module
Yet another way to define the symbols would be by means of an assembly module which contains definitions like .global boot1 together with boot1 = 0x00020102.
All these approaches have in common that all symbols must be defined, or otherwise the linker will throw an undefined symbol error. This means boot.elf must be available, and it does not matter whether just one symbol is undefined or whether dozends of symbols are undefined.
Let Boot provide a Dispatch Table
The problem with using boot.elf directly, like lined out in the previous section, is that it introduces a direct dependency. This means that if Boot is improved or refactored, then you'll also have to re-compile App each time, even if the interface did not change.
A solution is to let Boot provide a dispatch table whose position and layout are known ahead of time. Only when the interface itself changes, App will have to be rebuilt. Just refactoring Boot will not require to re-build App.
The Assembly Module with the Jump Table
As explained in the "Crash" section below, addresses in a dispatch table (and hence indirect jumps) won't work because EIND has a wrong value. Therefore, let's assume we have a table of JMPs to the desired Boot functions, like in an assembly module boot-table.sx that reads:
;;; Linker description file boot.ld locates input section .boot.table
;;; right after .vectors, hence the address of .boot_table will be
;;; text-section-start + _VECTORS_SIZE, where the latter is
;;; #define'd in <avr/io.h>.
;;; No "x" section flag so that the linker won't relax JMPs to RJMPs.
.section .boot.table,"a",#progbits
.global .boot_table
.type .boot_table,#object
boot_table:
jmp boot1
jmp boot2
.size boot_table, .-boot_table
In this example, we are going to locate the jump table right after .vectors, so that its location is known ahead of time. The respective symbol definitions in App's syms.opt will then read
--defsym boot1=0x20000+vectors_size+0*4
--defsym boot2=0x20000+vectors_size+1*4
provided Boot is located at 0x20000. Symbol vectors_size can be defined in a C/C++ module, here by abusing avr-gcc attribute "address":
#include <avr/io.h>
__attribute__((__address__(_VECTORS_SIZE)))
char vectors_size;
Locating the Jump Table
In order to locate input section .boot.table, we need an own linker description file, which you might already use for Boot anyways. We start with a linker script from avr-gcc installation at ./avr/lib/ldscripts/avr6.xn, copy it to boot.ld, and add the following 2 lines after vectors:
...
.text :
{
*(.vectors)
KEEP(*(.vectors))
*(.boot.table)
KEEP(*(.boot.table))
/* For data that needs to reside in the lower 64k of progmem. */
*(.progmem.gcc*)
...
Auto-Generating Boot's Jump Table Module and the Symbols for App
It's highly advisable to have an interface description used by both App and Boot, say common.h. Moreover, in order to keep Boot's boot-table.sx and App's syms.opt in sync with the interface, it's agood idea to auto-generate these two files from common.h. To that end, assume that common.h reads:
#ifndef COMMON_H
#define COMMON_H
#define EX __attribute__((__used__,__externally_visible__))
EX int boot1 /* #boot_table:0 */ (int);
EX int boot2 /* #boot_table:1 */ (void);
#endif /* COMMON_H */
For the matter of simplicity, let's assume that this is C code or the interfaces are extern "C" so that the symbols in source code match the assembly names, and there's no need to use mangled names. It' easy enough to generate boot-table.sx and syms.opt from common.h using the magic comments. The magic comment follows directly after the symbol, so a regex would retrieve the token left of the magic comment, something like Python:
# ... symbol /* #boot_table:index */...
pat = re.compile (r".*(\b\w+\b)\s*/\* #boot_table:(\d+) \*/.*")
for line in sys.stdin.readlines():
match = re.match (pat, line)
if match:
index = int (match.group(2))
symbol = match.group(1)
Output template for syms.opt would be something like:
asm_line = "--defsym {symbol}=0x20000+vectors_size+4*{index}\n"
Code that will crash
Using Boot code from App is subject to several restrictions:
Indirect Calls and Jumps
These will crash because the start addresses of App resp. Boot are in different 128KiB segments of flash. When the address of a code symbol is taken, the compiler does this per gs(symbol) which instructs the linker to generate a stub and resolve gs() to that stub in .trampolines if the target address is outside the 128KiB segment where the trampolines are located. An explanation of gs() can be found in this answer, there is however more to it: The startup code will effectively initialize
EIND = __vectors >> 17;
see gcrt1.S, the AVR-LibC bits of start-up code crt<device>.o. The compiler assumes EIND never changes during execution, see EIND and more than 128KiB of Flash in the GCC documentation.
This means code in Boot assumes EIND = 1 but is called with EIND = 0 and hence EICALL resp. EIJMP will target the wrong address. This means common code must avoid indirect calls and jumps, and should be compiled with -fno-jump-tables so that switch/case won't generate such tables.
This also implies that the dispatch table described above won't work if it would just held gs(symbol) entries, because App and Boot will disagree on EIND.
Data in Static Storage
If common Boot code is using data in static storage, the data might collide with App's static storage. One way out is to avoid static storage in respective parts of Boot and pass addresses to, say, some data buffer by means of pointer erguments of respective functions.
One could have completely separate RAM areas; one for Boot and one for App, but that would be a waste of RAM because the applications will never run at the same time.
Static Constructors
Boot's static constructors will be bypassed if App uses code from Boot. This includes:
C++ code in Boot that explicitly or implicitly generates such constructors.
C/C++ code in Boot that relies on __attribute__((__constructor__)) or code in section .initN which is supposed to run prior to main.
Start-up code that initializes static storage, EIND etc., which is also run by locating it in some .initN sections, but will be bypassed if App calls Boot code.
I have a string of information that is roughly 17 kb long. My program will not generate this string or read it into a buffer - the data is already initialized, I want it to be compiled as is from within my code, like you would a static variable. Moreover, I'd much prefer it is within my executable, and not stored within a project file. I've never before encountered such an issue, what is the best way to go around this? Should I include as resource, or literally copy and paste the enormous stream of data into a variable? What would you recommend?
Forgot to mention, am using VisualStudio C++ 2015 if that matters
The GNU linker ld has the ability to directly include custom data as the .data section of an object file:
ld -r -b binary -o example.o example.txt
The resulting example.o file has symbols defined to access start and end of the embedded data (just look at the file with objdump to see how they're named).
Now I don't know whether the linker coming with Visual Studio has a similar ability, but I guess you could use the GNU linker either via mingw or also via cygwin (since the generated object file won't reference the standard lib you won't need the emulation lib that comes with cygwin).The resulting object file apparently can just be added to your sources like a regular source file.
Of course this manual workflow isn't too good if the data changes often...
Alternatively you can write a simple program which puts the contents of the file in a C string, like:
unsigned char const * const data = {
0x12, 0x34, 0x56 };
Of course there's already such a program (xdd) but I don't know whether it's available to you. One potential issue is that you could reach the limit for the length of string literals that way. To get around that you could try a (multidimensional) char array.
(When writing this answer I found this blog post very helpful.)
I'd like to identify where all the objects of static storage duration are , in a large codebase; so that I can review to see if there are any potential problems with static initialization order.
Is there a good way to do this?
Merely searching the keyword static is not good enough, as it will miss any objects declared at namespace scope.
The linker's map file does indicate how big bss and data areas are , however it has all the names stripped out for symbols that are not extern.
Currently I am sifting through a dump of each object file looking for DATA and BSS, however that is painful and there is a lot of junk such as class vtables and compiler-generated static data.
Disclaimer: this is a fairly localized and incomplete answer. I leave it here in the hope that someone may benefit from it still (and maybe build on it).
Using the gcc toolchain, at startup __main calls __do_global_ctors which does a backward traversal of __CTOR_LIST__. Using nm on a .so library, for example, I get:
00000000004e2040 d __CTOR_END__
00000000004e2000 d __CTOR_LIST__
00000000004e2050 d __DTOR_END__
00000000004e2048 d __DTOR_LIST__
From then on, I suppose you could get from those addresses to the effective functions being executed; however, as you noticed, mapping back to the source name might be awkward (especially in anonymous namespaces). You may be able to recover them from the debugging information though (source location), however I have not progressed so far.
Our project (C++, Linux, gcc, PowerPC) consists of several shared libraries. When releasing a new version of the package, only those libs should change whose source code was actually affected. With "change" I mean absolute binary identity (the checksum over the file is compared. Different checksum -> different version according to the policy). (I should mention that the whole project is always built at once, no matter if any code has changed or not per library).
Usually this can by achieved by hiding private parts of the included Header files and not changing the public ones.
However, there was a case where a mere delete was added to the destructor of a class TableManager (in the TableManager.cpp file!) of library libTableManager.so, and yet the binary/checksum of library libB.so (which uses class TableManager ) has changed.
TableManager.h:
class TableManager
{
public:
TableManager();
~TableManager();
private:
int* myPtr;
}
TableManager.cpp:
TableManager::~TableManager()
{
doSomeCleanup();
delete myPtr; // this delete has been added
}
By inspecting libB.so with readelf --all libB.so, looking at the .dynsym section, it turned out that the length of all functions, even the dynamically used ones from other libraries, are stored in libB! It looks like this (length is the 668 in the 3rd column):
527: 00000000 668 FUNC GLOBAL DEFAULT UND _ZN12TableManagerD1Ev
So my questions are:
Why is the length of a function actually stored in the client lib? Wouldn't a start address be sufficient?
Can this be suppressed somehow when compiling/linking of libB.so (kind of "stripping")? We would really like to reduce this degree of dependency...
Bingo. It is actually kind of a "bug" in binutils which they found and fixed in 2008. The size information is actually not useful!
What Simon Baldwin wrote in the binutils mailing list describes exactly the problem ( emphases by me):
Currently, the size of an undefined ELF symbol is copied out of the
object file or DSO that supplies the symbol, on linking. This size is
unreliable, for example in the case of two DSOs, one linking to the
other. The lower- level DSO could make an ABI-preserving change that
alters the symbol size, with no hard requirement to rebuild the
higher-level DSO. And if the higher- level DSO is rebuilt, tools that
monitor file checksums will register a change due to the altered size
of the undefined symbol, even though nothing else about the
higher-level DSO has altered. This can lead to unnecessary and
undesirable rebuild and change cascades in checksum-based systems.
We have the problem with an older system (binutils 2.16). I compared it with version 2.20 on the desktop system and - voilà - the lengths of shared global symbols were 0:
157: 00000000 0 FUNC GLOBAL DEFAULT UND _ZN12TableManagerD1Ev
158: 00000000 0 FUNC GLOBAL DEFAULT UND _ZNSs6assignERKSs#GLIBCXX_3.4 (2)
159: 00000000 0 FUNC GLOBAL DEFAULT UND sleep#GLIBC_2.0 (6)
160: 00000000 0 FUNC GLOBAL DEFAULT UND _ZN4Gpio11setErrorLEDENS_
So I compared both binutils source codes, and - voilà again - there is the fix which Alan suggested in the mailing list:
Maybe we just apply the patch and recompile binutils since we need to stay with the olderish platform. Thanks for your patience.
You'd need to read through the code for the loader to be sure, but I think in this case we can make a fairly reasonable guess about what that length field is intended to accomplish.
The loader needs to take all the functions that are going to be put into the process, and map them to memory addresses. So, it gives the first function an address. Then, the second comes after the end of the first -- but to know "the end of the first", it needs to know how long the first function is.
I can see two ways for it to approach getting that length: it can either have it encoded in the file (as you'd seen it is in ELF) or else it can open the file that contains the function, and get the length from there.
The latter seems (to me) to have two fairly obvious disadvantages. The first is speed -- opening all those extra files, parsing their headers, etc., just to get the lengths of the functions is almost certainly slower than reading an extra four bytes for each function from the current file. The second is convenience: as long as you don't call any of the functions in a file, you don't need that file to be present at all. If you read the lengths directly from the file (e.g., like Windows normally does with DLLs) you'd need that file to be present on the target system, even if it's never actually used.
Edit: Since some people apparently missed the (apparently too-) subtle implication of "intended to accomplish", let me be entirely clear: I'm reasonably certain this field is not (and never has been) actually used.
Anybody who thinks that makes this answer wrong, however, needs to go back to programming 101 and learn the difference between an interface and an implementation.
In this case, the file format defines an interface -- a set of capabilities that a loader can use. In the specific case of Linux, it appears that this field isn't ever used.
That, however, doesn't change the fact that the field still exists, nor that the OP asked about why it exists. Simply saying "it's not used", while true in itself, would/does not answer the question he asked.