This question already has an answer here:
Get loaded address of a ELF binary, dlopen is not working as expected
(1 answer)
Closed 1 year ago.
On Windows, the HMODULE returned from LoadLibrary is the base pointer of the loaded DLL.
The shared library I use is a headless version of a game. To save its state, I parse the DLL to locate the .data and .bss sections, add their VAs to the DLL's base address, then copy the right amount of data from each section.
In principle, the same should be doable on Linux. However, I'm stuck on how to get the base address of a dlopen()ed ELF library, since the void* returned from dlopen() is a pointer to the shared library's link_map AFAIK.
How might I accomplish this?
EDIT 1: The "state" of the shared library is the state of all the static variables in it. To save that state, I copy the sections that contain them (.data and .bss) to an alternate buffer (in memory). When I restore that state, I write the alternate buffer's data back to the shared library's .data and .bss.
Read and parse the file /proc/self/maps.
That said, the notion of saving the state of a noncooperating library by dumping its data section is rather questionable. Imagine there is a pointer. It points at some other variable else in the data section. There is no way you can tell it's a pointer - from the standpoint of the controlling program, it's just a word in memory that by pure accident contains a value that corresponds to a valid address. If the library is loaded at some other address and the state is restored, the said pointer won't be valid anymore.
Related
This question already has answers here:
Does the address of a function change per runtime [duplicate]
(3 answers)
Why does the address of a function change with every run?
(4 answers)
Closed 3 years ago.
The address of executable code is decided at link time, isn't it?
#include <stdio.h>
int main ()
{
printf("%p", (void*)&main);
return 0;
}
example output #1:
0x563ac3667139
example output #2:
0x55e3903a9139
On many modern systems, at link time it will determine the address of the function relative to the base address module. When the module (exe, dll, or so) is loaded, Address Space Layout Randomization (ASLR) gives it a different base address.
This is for security, it means the addresses of functions is not predictable. This means certain attacks that might for example overflow a stack variable to overwrite the return address or a function pointer with some other function (for malicious purposes), can't easily predict what address to overwrite it with, it will vary from run to run.
The ability to relocate the base address also solves the practical problem of a conflict, if you load a.dll and b.dll which were independently compiled for the same base address, that won't work, so being able to relocate one resolves the conflict.
At the machine code level, this is fine because most jumps and calls use a relative instruction offset, not an absolute. Although certain constructs are dynamically patched when the module is loaded, or use some form of "table" that is populated with the correct addresses.
See also Relocation (computing)
This is a security technique called address space layout randomization.
It deliberately moves things around on each execution, to make it more difficult for attackers to know where bits of data are in your process and hack them.
I've been looking a bit into Cheat Engine, which allows you to inspect and manipulate the memory of running processes on Windows: You scan for variables based on their value, then you can modify them, e.g. to cheat in a game.
In order to write a bot or something similar, you need to find a static address for the variable you want to change - i.e. one that stays the same if the process is restarted. The method for that goes roughly like this:
Look for the address of the variable you're interested in, searching by value
Look for code using that address, e.g. to find the address of the struct it belongs to (since struct offsets are fixed)
Look for another pointer pointing to that pointer until you find one with a static address (shows as green in Cheat Engine)
It seems to work just fine judging from the tutorials I've looked at, but I have trouble understanding why it works.
Don't all variables, including global static ones, get a pretty random address at runtime time?
Bonus questions:
How can Cheat Engine tell if an address is static (i.e. will stay the same on restart)?
A tutorial referred to the fact that many older and some modern games (e.g. Call of Duty 4) use only static addresses. How is that possible?
I will answer the bonus questions first because they introduce some concepts you may need to know to understand the answer for the main question.
Answering the first bonus question is easy if you know how an executable file works: all the global/static variables are inside the .data section, in which the .exe stores the address offset for the section so Cheat Engine just checks if the variable is in this address range (from this section to the next one).
For the second question, it is possible to use only static addresses, but that is nearly impossible for a game. Even the older ones. What the tutorial creator was probably trying to say is that all variables that he wants, actually had a static pointer pointing to them. But solely by the fact that you create a local variable, or even pass an argument to a function, their values are being stored into the stack. That's why it is nearly impossible to have a "static-only" program. Even if you compile a program that actually doesn't do anything, it will probably have some stuff being stored in the stack.
For the whole question itself, not all dynamic address variables are pointed by a global variable. It depends totally on the programmer. I can create a local variable and never assign its address to a global/static pointer in a C program, for example. The only certain way to find that address in this case is to actually know the code when the variable was first assigned a value in the stack.
Some variables have a dynamic address because they are just local variables, which are stored in the stack the first time they have a value assigned to them.
Some other variables have a static address because they are declared either as a global or a static variable to the compiler. These variables have a fixed address offset that is part of the .data section in the executable file.
The executable file has a fixed offset address for each section inside it, and the .data section is no exception.
But it is worth to note that the offset inside the executable itself is fixed. In the operating system things might be different (all random addresses), but that is the job of an OS, abstracting this kind of stuff for you (creating the executable's virtual address space in this case). So it just looks like static variables are actually static, but only inside the executable's memory space. On the RAM things might be anywhere.
Finally, it is difficult to try to explain this to you because you'll have to understand how executable files work. A good start would be to search for some explanations regarding low-level programming, like stack frame, calling conventions, the Assembly language itself and how compilers use some well-known techniques to manage functions (scopes in general), global/static/local/constant variables, and the memory system (sections, the stack, etc.), and maybe some research into PE (and even ELF) files.
As far as I understand it, variables declared static have a permanent offset within the program data. This means that when the program is loaded into RAM, the offset of the variable will always be the same. Because the beginning address of the program is known globally, finding a static variable based on offset, as you mentioned, should be a trivial task. Therefore, while a pointer to a static variable might be random in the scheme of things, its offset to the beginning of program memory should remain the same no matter when the program starts. So Cheat Engine (though I don't know the software) most likely stores the offset of the static variable, and then when the software starts, applies this logic to find that variable.
As to how it can tell it's a static variable in the first place... well, this is partially a guess, but when you declare a variable static in C, I'm assuming the compiler/linker puts some kind of flag so the OS knows that it's a static variable. It could also be that all static variables are stored in a certain way, or at a certain address offset, for all programs compiled for a certain target system. Again, not too sure about that, but from what I understand about memory management, that seems to make the most sense. With these assumptions, it's quite possible for a program to contain solely static variables. The difference is that memory is assigned statically at program runtime, as a opposed to dynamically (as with a call to malloc() or similar). If the variables were stored dynamically, I'm sure there'd be a way to find them easily, so I don't think it matters to Cheat Engine whether or not a variable is static or not. However, as I'm assuming Cheat Engine wants to modify a game upon startup (just like the old GameSharks used to... ahh, miss those days) it's probably more reliable to modify variables that are static, instead of trying to locate pointers and disassemble the code, etc. etc.
If you're interested in learning more, I'd recommend checking out something like this tutorial over at OSDev!
Is it possible to load a library from memory instead of from the filesystem on mac/gcc?
With windows I'm using MemoryModule but it's obviously not cross-platform compatible.
First things first, to do this I advise you use read the OS X ABI Dynamic Loader Reference.
To do this, you must use the NSCreateObjectFileImageFromMemory API.
Given a pointer to a Mach-O file in memory, this function creates and returns an NSObjectFileImage reference. The current implementation works only with bundles, so you must build the Mach-O executable file using the -bundle linker option.
The memory block that address points to, must be allocated with vm_allocate (/usr/include/mach/vm_map.h).
Make sure to abide by the requirement that vm_allocate is used for the memory block containing the module.
Once you acquire the object file image, you must use NSLinkModule function to link the module into the program.
When you call this function, all libraries referenced by the given module are added to the library search list. Unless you pass the NSLINKMODULE_OPTION_PRIVATE, NSLinkModule adds all global symbols in the module to the global symbol list.
After linking, don't forget to clean up by calling the NSDestroyObjectFileImage function.
When this function is called, the dynamic loader calls vm_deallocate (/usr/include/mach/vm_map.h) on the memory pointed to by the objectFileImage parameter.
Note that while these functions are deprecated, there is no substitute (to the best of my knowledge) using the suggested alternative dlopen et. al.
I have read about GCC's Options for Code Generation Conventions, but could not understand what "Generate position-independent code (PIC)" does. Please give an example to explain me what does it mean.
Position Independent Code means that the generated machine code is not dependent on being located at a specific address in order to work.
E.g. jumps would be generated as relative rather than absolute.
Pseudo-assembly:
PIC: This would work whether the code was at address 100 or 1000
100: COMPARE REG1, REG2
101: JUMP_IF_EQUAL CURRENT+10
...
111: NOP
Non-PIC: This will only work if the code is at address 100
100: COMPARE REG1, REG2
101: JUMP_IF_EQUAL 111
...
111: NOP
EDIT: In response to comment.
If your code is compiled with -fPIC, it's suitable for inclusion in a library - the library must be able to be relocated from its preferred location in memory to another address, there could be another already loaded library at the address your library prefers.
I'll try to explain what has already been said in a simpler way.
Whenever a shared lib is loaded, the loader (the code on the OS which load any program you run) changes some addresses in the code depending on where the object was loaded to.
In the above example, the "111" in the non-PIC code is written by the loader the first time it was loaded.
For not shared objects, you may want it to be like that because the compiler can make some optimizations on that code.
For shared object, if another process will want to "link" to that code it must read it to the same virtual addresses or the "111" will make no sense. But that virtual-space may already be in use in the second process.
Code that is built into shared libraries should normally be position-independent code, so that the shared library can readily be loaded at (more or less) any address in memory. The -fPIC option ensures that GCC produces such code.
The link to a function in a dynamic library is resolved when the library is loaded or at run time. Therefore, both the executable file and dynamic library are loaded into memory when the program is run.
The memory address at which a dynamic library is loaded cannot be determined in
advance, because a fixed address might clash with another dynamic library requiring the same address.
There are two commonly used methods for dealing with this problem:
1.Relocation. All pointers and addresses in the code are modified, if necessary, to fit the actual load address. Relocation is done by the linker and the loader.
2.Position-independent code. All addresses in the code are relative to the current position. Shared objects in Unix-like systems use position-independent code by default. This is less efficient than relocation if program run for a long time, especially in 32-bit mode.
The name "position-independent code" actually implies the following:
The code section contains no absolute addresses that need relocation, but only self relative
addresses. Therefore, the code section can be loaded at an arbitrary memory address and shared between multiple processes.
The data section is not shared between multiple processes because it often contains
writeable data. Therefore, the data section may contain pointers or addresses that
need relocation.
All public functions and public data can be overridden in Linux. If a function
in the main executable has the same name as a function in a shared object, then the
the version in main will take precedence, not only when called from main, but also when
called from the shared object. Likewise, when a global variable in the main has the same
name as a global variable in the shared object, then the instance in main will be
used, even when accessed from the shared object. This so-called symbol interposition is intended to mimic the behavior of static libraries.
A shared object has a table of pointers to its functions, called procedure linkage table (PLT), and a table
of pointers to its variables called global offset table (GOT) in order to implement this "override" feature.
All accesses to functions and public variables go through these tables.
p.s. Where dynamic linking cannot be avoided, there are various ways to avoid the time-consuming features of the position-independent code.
You can read more from this article: http://www.agner.org/optimize/optimizing_cpp.pdf
Adding further...
Every process has same virtual address space (If randomization of virtual address is stopped by using a flag in linux OS)
(For more details Disable and re-enable address space layout randomization only for myself)
So if its one exe with no shared linking (Hypothetical scenario), then we can always give same virtual address to same asm instruction without any harm.
But when we want to link shared object to the exe, then we are not sure of the start address assigned to shared object as it will depend upon the order the shared objects were linked.That being said, asm instruction inside .so will always have different virtual address depending upon the process its linking to.
So one process can give start address to .so as 0x45678910 in its own virtual space and other process at the same time can give start address of 0x12131415 and if they do not use relative addressing, .so will not work at all.
So they always have to use the relative addressing mode and hence fpic option.
A minor addition to the answers already posted: object files not compiled to be position independent are relocatable; they contain relocation table entries.
These entries allow the loader (that bit of code that loads a program into memory) to rewrite the absolute addresses to adjust for the actual load address in the virtual address space.
An operating system will try to share a single copy of a "shared object library" loaded into memory with all the programs that are linked to that same shared object library.
Since the code address space (unlike sections of the data space) need not be contiguous, and because most programs that link to a specific library have a fairly fixed library dependency tree, this succeeds most of the time. In those rare cases where there is a discrepancy, yes, it may be necessary to have two or more copies of a shared object library in memory.
Obviously, any attempt to randomize the load address of a library between programs and/or program instances (so as to reduce the possibility of creating an exploitable pattern) will make such cases common, not rare, so where a system has enabled this capability, one should make every attempt to compile all shared object libraries to be position independent.
Since calls into these libraries from the body of the main program will also be made relocatable, this makes it much less likely that a shared library will have to be copied.
I have a cpp file from which I am generating a shared library (using autofoo and the like). Within the cpp file, I have declared a couple of static fields that I use throughout the library functions.
My question is 2-part:
1) Where are these fields stored in memory? It's not as if the system instantiates the entire library and keeps it in memory... the library, after all, really is just a bunch of hooks.
2) Is there a better way to do this? The reason I did it to begin with is that I want to avoid requiring the user to pass the fields into every library function call as parameters.
Thanks!
The code used to load shared libraries:
Generally (each has minor technical differences):
Loads the shared lib into memory
Walks the symbol table and updates the address of function in the DLL
Initializes any global static members using their constructor.
Note: The shared lib loader need not do all this at the load point.
It may do some of these jobs lazily (implementation detail). But they will be done before use.
Any Global staic POD variables (things with no constructor). Will be stored in special memory segments depending on weather they are initialized or not (again an implementation detail). If they were initialized then they will be loaded with the from disk (or shared lib source) with that value already defined.
So the answer to your questions:
undefined.
The library is code segments
Initialized data segments
Uninitialized data segments
Some utility code that knows how to link it into a running application.
Better than what exactly
Good practice would suggest passing values to a function rather than relying on global state. But to be honest that is an over generalization and really down to the problem.
Logically speaking, it is as if the system instantiates the entire library. In practice, only the code is really "shared" in a shared library, anybody who links against it will get a copy of the data. (Well maybe not read-only data). So, as far your questions go:
1) Your process will get a copy of the variable somehow (dependent on how the shared library system on your OS works).
2) I don't see a problem with this approach.