I'm writing a very simple process loader for Linux. The executables I'm loading are already compiled, and I know where each one expects to be found in memory. The first approach I tried was using mmap() to manually place each code or data section at the correct location, like
mmap(addr, size, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED, -1, 0)
which segfaults unless I remove the MAP_FIXED flag because, it seems, the address of one block conflicts with something already in memory, possibly even the loader itself; the address 0x401000 seems to be the problematic one.
I'm not really even sure where to begin with this one. A friend suggested virtualizing memory access operations; I'm not sure what kind of performance hits I'd take for that, and I have no clue how it's done, but it might be an option. What I'd really love to do is create an "empty" process, which would have, as far as it was concerned, full run of the memory, so nothing would be loaded into the user space until I wanted it to be. The whole concept of an "empty" process might be meaningless, but it's the best way to describe what I want. I'm pretty desperate for some references or examples that might help me.
With your process running (maybe snoozing in "sleep(1000);"), look at its /proc/pid/maps. That will tell you what 0x401000 is used for.
~$ sleep 1h &
[3] 2033
~$ cat /proc/2033/maps
00110000-002af000 r-xp 00000000 08:01 1313056 /lib/i386-linux-gnu/libc-2.15.so
...
Here on my box, /bin/sleep doesn't use that block, and neither does my little one-liner program.
You're probably linking in some library which wants to land there?
So one way would be to allocate the block you need way early (long before main() runs -- look elsewhere for that info).
Another way is to link your code to some address you "know" isn't taken (presumably, you're generating the x86 opcodes yourself, or otherwise "linking", so that shouldn't be a stretch).
Another, better, option is to make your code relocatable. The fact that you don't want to replace the entire process's address space (precisely what exec does) more or less says that your code should be just that.
So find a usable address, load the bits there, and, as needed, perform the relocations (so your on-disk file format, if it's not ELF, will need to include reloc info). That's the high road, and the obvious thing you'll want next from your loader.
Of course, that pretty much means reimplementing dlopen() yourself. I assume you're just trying to learn how it works -- if not, man dlopen. Stephane's Rule Zero: it's already there ;-)
Don't forget to support linking other libraries from your code (without duplication), dlclose(), initializers, the various RTLD_* modes, honor MYCUSTOMLD_LIBRARY_PATH, GCC's __thread specifier, etc. ;-)
Related
I have an MCU (say an STM32) running, and I would like to 'pass' it a separately compiled binary file over UART/USB and use it like calling a function, where I can pass it data and collect its output? After its complete, a second, different binary would be sent to be executed, and so on.
How can I do this? Does this require an OS be running? I'd like to avoid that overhead.
Thanks!
It is somewhat specific to the mcu what the exact call function is but you are just making a function call. You can try the function pointer thing but that has been known to fail with thumb (on gcc)(stm32 uses the thumb instruction set from arm).
First off you need to decide in your overall system design if you want to use a specific address for this code. for example 0x20001000. or do you want to have several of these resident at the same time and want to load them at any one of multiple possible addresses? This will determine how you link this code. Is this code standalone? with its own variables or does it want to know how to call functions in other code? All of this determines how you build this code. The easiest, at least to first try this out, is a fixed address. Build like you build your normal application but based in a ram address like 0x20001000. Then you load the program sent to you at that address.
In any case the normal way to "call" a function in thumb (say an stm32). Is the bl or blx instruction. But normally in this situation you would use bx but to make it a call need a return address. The way arm/thumb works is that for bx and other related instructions the lsbit determines the mode you switch/stay in when branching. Lsbit set is thumb lsbit clear is arm. This is all documented in the arm documentation which completely covers your question BTW, not sure why you are asking...
Gcc and I assume llvm struggles to get this right and then some users know enough to be dangerous and do the worst thing of ADDing one (rather than ORRing one) or even attempting to put the one there. Sometimes putting the one there helps the compiler (this is if you try to do the function pointer approach and hope the compiler does all the work for you *myfun = 0x10000 kind of thing). But it has been shown on this site that you can make subtle changes to the code or depending on the exact situation the compiler will get it right or wrong and without looking at the code you have to help with the orr one thing. As with most things when you need an exact instruction, just do this in asm (not inline please, use real) yourself, make your life 10000 times easier...and your code significantly more reliable.
So here is my trivial solution, extremely reliable, port the asm to your assembly language.
.thumb
.thumb_func
.globl HOP
HOP:
bx r0
I C it looks like this
void HOP ( unsigned int );
Now if you loaded to address 0x20001000 then after loading there
HOP(0x20001000|1);
Or you can
.thumb
.thumb_func
.globl HOP
HOP:
orr r0,#1
bx r0
Then
HOP(0x20001000);
The compiler generates a bl to hop which means the return path is covered.
If you want to send say a parameter...
.thumb
.thumb_func
.globl HOP
HOP:
orr r1,#1
bx r1
void HOP ( unsigned int, unsigned int );
HOP(myparameter,0x20001000);
Easy and extremely reliable, compiler cannot mess this up.
If you need to have functions and global variables between the main app and the downloaded app, then there are a few solutions and they involve resolving addresses, if the loaded app and the main app are not linked at the same time (doing a copy and jump and single link is generally painful and should be avoided, but...) then like any shared library you need to have a mechanism for resolving addresses. If this downloaded code has several functions and global variables and/or your main app has several functions and global variables that the downloaded library needs, then you have to solve this. Essentially one side has to have a table of addresses in a way that both sides agree on the format, could be as a simple array of addresses and both sides know which address is which simply from position. Or you create a list of addresses with labels and then you have to search through the list matching up names to addresses for all the things you need to resolve. You could for example use the above to have a setup function that you pass an array/structure to (structures across compile domains is of course a very bad thing). That function then sets up all the local function pointers and variable pointers to the main app so that subsequent functions in this downloaded library can call the functions in the main app. And/or vice versa this first function can pass back an array structure of all the things in the library.
Alternatively a known offset in the downloaded library there could be an array/structure for example the first words/bytes of that downloaded library. Providing one or the other or both, that the main app can find all the function addresses and variables and/or the caller can be given the main applications function addresses and variables so that when one calls the other it all works... This of course means function pointers and variable pointers in both directions for all of this to work. Think about how .so or .dlls work in linux or windows, you have to replicate that yourself.
Or you go the path of linking at the same time, then the downloaded code has to have been built along with the code being run, which is probably not desirable, but some folks do this, or they do this to load code from flash to ram for various reasons. but that is a way to resolve all the addresses at build time. then part of the binary in the build you extract separately from the final binary and then pass it around later.
If you do not want a fixed address, then you need to build the downloaded binary as position independent, and you should link that with .text and .bss and .data at the same address.
MEMORY
{
hello : ORIGIN = 0x20001000, LENGTH = 0x1000
}
SECTIONS
{
.text : { *(.text*) } > hello
.rodata : { *(.rodata*) } > hello
.bss : { *(.bss*) } > hello
.data : { *(.data*) } > hello
}
you should obviously do this anyway, but with position independent then you have it all packed in along with the GOT (might need a .got entry but I think it knows to use .data). Note, if you put .data after .bss with gnu at least and insure, even if it is a bogus variable you do not use, make sure you have one .data then .bss is zero padded and allocated for you, no need to set it up in a bootstrap.
If you build for position independence then you can load it almost anywhere, clearly on arm/thumb at least on a word boundary.
In general for other instruction sets the function pointer thing works just fine. In ALL cases you simply look at the documentation for the processor and see the instruction(s) used for calling and returning or branching and simply use that instruction, be it by having the compiler do it or forcing the right instruction so that you do not have it fail down the road in a re-compile (and have a very painful debug). arm and mips have 16 bit modes that require specific instructions or solutions for switching modes. x86 has different modes 32 bit and 64 bit and ways to switch modes, but normally you do not need to mess with this for something like this. msp430, pic, avr, these should be just a function pointer thing in C should work fine. In general do the function pointer thing then see what the compiler generates and compare that to the processor documentation. (compare it to a non-function pointer call).
If you do not know these basic C concepts of function pointer, linking a bare metal app on an mcu/processor, bootstrap, .text, .data, etc. You need to go learn all that.
The times you decide to switch to an operating system are....if you need a filesystem, networking, or a few things like this where you just do not want to do that yourself. Now sure there is lwip for networking and some embedded filesystem libraries. And multithreading then an os as well, but if all you want to do is generate a branch/jump/call instruction you do not need an operating system for that. Just generate the call/branch/whatever.
Loading and execution a fully linked binary and loading and calling a single function (and returning to the caller) are not really the same thing. The latter is somewhat complicated and involves "dynamic linking", where the code effectively and secures in the same execution environment as the caller.
Loading a complete stand-alone executable in the other hand is more straightforward and is the function of a bootloader. A bootloader loads and jumps to the loaded executable which then establishes it's own execution environment. Returning to the bootloader requires a processor reset.
In this case it would make sense to have the bootloader load and execute code in RAM if you are going to be frequently loading different code. However be aware that on Harvard Architecture devices like STM32, RAM execution may slow down execution because data and instruction fetch share the same bus.
The actual implementation of a bootloader will depend on the target architecture, but for Cortex-M devices is fairly straightforward and dealt with elsewhere.
STM32 actually includes an on-chip bootloader (you need to configure the boot source pins to invoke it), which I believe can load and execute code in RAM. It is normally used to load a secondary bootloader to load and program flash, but it can be used for loading any code.
You do need to build and link your code to run from RAM at the address tle loader locates it, or if supported build position-indeoendent code that can run from anywhere.
For my university project I need to WRITE protect the whole address space of the process. I was reading the /proc/self/maps file and parsing the mapping.
So for each entry of the format 08048000-0804c000 r-xp 00000000 08:03 7971106 /bin/cat, I am reading the first two entry(here 08048000 & 0804c000), converting them to decimal. Lets assume the decimal equivalent is A & B respectively. Then I do mprotect((int*)A, B-A, PROT_READ). But this approach is giving me segmentation fault. I can't find out what I did wrong here. May be I've some knowledge gap here which is causing the problem. Someone can give me some suggestions?
Assuming that your implementation is correct, I'd still expect to see segmentation faults.
After all, you're telling the kernel that you don't want to be allowed to write to any part of your memory. Afterwards, you'll just continue to run your process and the next time you try to write anything at all you'll get a segmentation fault because that's no longer allowed.
That'll most likely be when you return from mprotect() after "protecting" the stack.
Thinking a bit more, it's even possible that you're getting segmentation faults while executing memory (i.e. a shared lib, or your executable code) after you've "protected" it.
In fact, all of the bits of memory where it's safe to apply read-only / do-not-execute flags already have those flags set.
I suspect that's the insight this univerity project was meant to give you.
Is it possible without any file.out and source code, but just the binary?
Is it possible, knowing the name of a var, found and read at runtime the value?
Is it possible, knowing the name of a var, found and read at runtime the value
It depends.
If the variable is a global, and the binary is not stripped, then you should be able to examine its value with a simple
x/gx &var
print var
The latter may print the variable as if it were of type int (if the binary has no debug info), which may not be what you are looking for.
If the variable is local (automatic), then you can print it only while inside the routine in which it is declared (obviously).
If the binary has debug info, then simple print var in correct context should work.
If the binary doesn't, you'll have to figure out the in-memory address of the variable (usually at fixed offset from stack pointer of frame pointer register), and examine that address. You can often figure out a lot about the given routine by disassembling it.
Update:
if I strip the binary, is harder to do the reverse engineering?
Sure: the less info you provide to the attacker, the harder you make his job.
But you also make your job harder: when your binary doesn't work, often your end-user will know more about his system than you do. Often he will load your binary into GDB, and tell you exactly where your bug is. With a stripped executable, he likely wouldn't be able to do that, so you'll guess back and forth, and after a week of trying will lose that customer.
And there is nothing you can do to prevent a sufficiently determined and sufficiently skilled hacker with root access to his system and hardware from reverse engineering your program.
In the end, in my experience, anti-circumvention techniques are usually much more trouble than they are worth.
My executable was 364KB in size. It did not use a Vector2D class so I implemented one with overloaded operators.
I changed most of my code from
point.x = point2.x;
point.y = point2.y;
to
point = point2;
This resulted in removing nearly 1/3 of my lines of code and yet my exe is still 364KB. What exactly causes it to grow in size?
The compiler probably optimised your operator overload by inlining it. So it effectively compiles to the same code as your original example would. So you may have cut down a lot of lines of code by overloading the assignment operator, but when the compiler inlines, it takes the contents of your assignment operator and sticks it inline at the calling point.
Inlining is one of the ways an executable can grow in size. It's not the only way, as you can see in other answers.
What makes EXE’s grow in size?
External libraries, especially static libraries and debugging information, total size of your code, runtime library. More code, more libraries == larger exe.
To reduce size of exe, you need to process exe with gnu strip utility, get rid of all static libraries, get rid of C/C++ runtime libraries, disable all runtime checks and turn on compiler size optimizations. Working without CRT is a pain, but it is possible. Also there is a wcrt (alternative C runtime) library created for making small applications (by the way, it hasn't been updated/maintained during last 5 years).
The smallest exe that I was able create with msvc compiler is somewhere around 16 kilobytes. This was a windows application that displayed single window and required msvcrt.dll to run. I've modified it a bit, and turned it into practical joke that wipes out picture on monitor.
For impressive exe size reduction techniques, you may want to look at .kkrieger. It is a 3D first person shooter, 96 kilobytes total. The game has a large and detailed level, supports shaders, real-time shadows, etc. I.e. comparable with Saurbraten (see screenshots). The smallest working windows application (3d demo with music) I ever encountered was 4 kilobytes big, and used compression techniques and (probably) undocumented features (i.e. the fact that *.com executbale could unpack and launch win32 exe on windows xp)..
In most cases, size of *.exe shouldn't really bother you (I haven't seen a diskette for a few years), as long as it is reasonable (below 100 megabytes). For example of "unreasonable" file size see debug build of Qt 4 for mingw.
This resulted in removing nearly 1/3 of my lines of code and yet my exe is still 364KB.
Most likely it is caused by external libraries used by compiler, runtime checks, etc.
Also, this is an assignment operation. If you aren't using custom types for x (with copy constructor), "copy" operation is very likely to result in small number of operations - i.e. removing 1/3 of lines doesn't guarantee that your code will be 1/3 shorter.
If you want to see how much impact your modification made, you could "ask" compiler to produce asm listing for both versions of the program then compare results (manually or with diff). Or you could disasm/compare both versions of executable. BUt I'm certain that using GNU strip or removing extra libraries will have more effect than removing assignment operators.
What type is point? If it's two floats, then the compiler will implicitly do a member-by-member copy, which is the same thing you did before.
EDIT: Apparently some people in today's crowd didn't understand this answer and compensated by downvoting. So let me elaborate:
Lines of code have NO relation to the executable size. The source code tells the compiler what assembly line to create. One line of code can cause hundreds if not thousands of assembly instructions. This is particularly true in C++, where one line can cause implicit object construction, destruction, copying, etc.
In this particular case, I suppose that "point" is a class with two floats, so using the assignment operator will perform a member-by-member copy, i.e. it takes every member individually and copies it. Which is exactly the same thing he did before, except that now it's done implicitly. The resulting assembly (and thus executable size) is the same.
Executables are most often sized in 'pages' rather than discrete bytes.
I think this a good example why one shouldn't worry too much about code being too verbose if you have a good optimizing compiler. Instead always code clearly so that fellow programmers can read your code and leave the optimization to the compiler.
Some links to look into
http://www2.research.att.com/~bs/bs_faq.html#Hello-world
GCC C++ "Hello World" program -> .exe is 500kb big when compiled on Windows. How can I reduce its size?
http://www.catch22.net/tuts/minexe
As for Windows, lots of compiler options in VC++ may be activated like RTTI, exception handling, buffer checking, etc. that may add more behind the scenes to the overall size.
When you compile a c or c++ program into an executable, the compiler translates your code into machine code, and applying optimizations as it sees fit.
But simply, more code = more machine code to generate = more size to the executable.
Also, check if you have lot of static/global objects. This substantially increase your exe size if they are not zero initialized.
For example:
int temp[100] = {0};
int main()
{
}
size of the above program is 9140 bytes on my linux machine.
if I initialize temp array to 5, then the size will shoot up by around 400 bytes. The size of the below program on my linux machine is 9588.
int temp[100] = {5};
int main()
{
}
This is because, zero initialized global objects go into .bss segment, which ill be initialized at once during program startup. Where as non zero initialized objects contents will be embedded in the exe itself.
Im trying to get the engine version of a game from a global pointer, but I am fairly new to this. Here is a very small example I found...
http://ampaste.net/mb42243
And this is the disassembly for what I am trying to get, the pointer (gpszVersionString) is the highlighted line (line 5)
http://ampaste.net/m2a8f8887
So what I need to find out is basically using the example approach I found to get it, would I need to basically sig out the first part of the function and find the offset to that line?
Like...
Memory signature - /x56/x8B/x35/x74/xD5/x29/x10/x68/x00/xA8/x38/x10
Then an offset to reach that line? (not sure how to find the offset)
You can't directly do this. Process address space is completely unique to your process -- 0xDEADBEEF can point to "Dog" in one process, while 0xDEADBEEF can point to "Cat" in another. You would have to make operating system calls that allow you to access another process' address space, and even then you'd have to guess. Many times that location will be different each run of the application -- you can't generally predict what the runtime layout of a process will be in all cases.
Assuming you're on Windows you'll need to (EDIT: You don't need A and B in all cases but you usually need them) A. be an administrator, B. take the SeDebugPrivilege for your process, C, open a handle to the process, and then D. use ReadProcessMemory/WriteProcessMemory to do what you want.
Hope that helps :)
EDIT 2: It looks like you're looking at an address taken from a disassembler. If that's the case, then you can't use that value of the address -- the image can be re-based at runtime and the value there would be completely different. Particularly on recent versions of Windows which support Address Space Layout Randomization.