I want to add a 4KB space to the bss section of an executable elf file. How can this be done?
If not bss section, can I increase the size of the data section or define a new data section
and initialize the region ?
If possible, can this be done with binary file formats other than elf ?
PS: I need this to add some extra instructions for instrumentation and saving the data in dedicated locations which have to be printed later.
I want to add a 4KB space to the bss section of an executable elf file. How can this be done?
Assuming you want to do that to an already linked ELF executable, note that sections are not used at all by anything (other than perhaps debugging tools) after the link is done; you want to modify the corresponding segment.
What you are asking for is impossible in general, but might be possible for your particular executable. In particular, it should be possible if the LOAD segment "covering" the .bss section is the last one, or if there is an in-memory gap between that segment and the next LOAD segment.
If there is space (in memory) to extend the LOAD segment in question, then all you have to do is patch its program header's .p_memsz and increment it by 4096.
You would need to understand the output from readelf -Wl a.out in detail.
Update:
assuming that bss occurs last, is there a tool to change .p_memsz of the last segment in a line or two ?
I don't know of any tool to do this, but it's pretty trivial: program headers are fixed sized table starting at file offset ehdr->e_phoff. The table contains ehdr->e_phnum records. You read each record until you find the last PT_LOAD segment, update the .p_memsz member, seek back and write the updated record on top of what was there.
The libelf or elfio libraries may (or may not) make writing this tool easier.
I guess to make the elf conformant, we also need to change the section size of bss accordingly ?
No, you don't:
there is no requirement that sizes of .bss and the load segment match
in fact, there is no requirement that any sections are present at all,
like I said, nothing cares about any sections after the binary is linked.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 12 months ago.
Improve this question
I'm using esp32c3(a riscv architecture chip). I use a global string array initialized with string, and it occupies ram and rom. When I add "const" keyword before the array, the array is put into .rodata by link script. Then I found that the ram consumption reduces(this is in line with my expectation). However, the compiled .bin firmware file size is reduced.
I think even if the const keyword is added to the array, the size of the compiled .bin firmware file should remain unchanged, because the strings are always need to be stored in the firmware.This puzzles me.
Other info:
If a pointer is used to point to the same string content(instead of using string array, and the pointer is is located at .sdata), the .bin file size will increase, ram consumption not increase.
At the same time, I saw this paragraph in the official data of the chip, but I didn't fully understand the meaning:
By default, constant data is placed by the linker into a region mapped to the MMU flash cache. This is the same as the IROM (code executed from flash) section, but is for read-only data not executable code.
The only constant data not placed into this memory type by default are literal constants which are embedded by the compiler into application code. These are placed as the surrounding function’s executable instructions.
You don't specify it in your question, but I'm assuming your array is initialized with some non-zero data.
Your typical executable will look something like this [source]:
When you add the const keyword, the array simply gets moved from the .data section into the .rodata. The net change in file size is 0.
If you also add static keyword to your array, it will be moved into the .text section, but again, the size will remain the same.
See Wikipedia page on Data segment for the information about each section.
If I get you correct, you expect the following to happen when you add const to your array:
the RAM consumption is reduced
the ROM consumption is increased.
But you can only observe 1), not 2).
Without looking at the exact sections that are used, you are missing one thing:
If you have an array like this:
char myArray[] = "This is the String for my Array!";
Then you need 2 chunks of memory:
RAM area for holding your array
ROM area for holding the string literal you are using to initialized that array.
At startup the initialization string is copied from ROM to RAM area.
If you make the array const, there is no more need to copy the string into RAM and you will see RAM consumption drop.
But wrt ROM consumption, there is no big difference as the data is already there.
Therefore ROM area will not be reduced.
The usual reason is that at the OS level memory gets allocated in fixed sized chunks for reasons owing to the hardware in use, if you add an object to .rodata that doesn't cross a boundary of that size then no more memory will need to be allocated than if that object were not part of the program. I can't tell you anything about the embedded platform you are using but for x86 for example that fixed size will be either 4kb, 2mb or 4mb (that is, the page size).
As I understand in PE file Virtual Size shows how much space it allocates for section during loading and Raw Size shows how big the section is on disk.
I came across to this executable which did the following:
It subtracted the virtual size (offset 0x8) from the raw data size (offset 0x10) and made sure there was some space(for example 100 bytes). At offset 0x14 from the text section header it found the offset in the file for the section itself. It added to this the virtual size, finding right where the section ends in the file. it copied some shellcode(which at the end eventually jumped to the original entry point of an executable to make sure original executable ran) to the end of the text section of the binary.
Now I'm little confused here, if the Virtual Size shows the exact space which will be allocated for executable, would not adding code at the end of .txt section overwrite some other data of executable and make it crash? thanks.
This is a great question and illustrates a very important point (or you might say quirk) about how the Windows loader calculates memory sizes.
The PE/COFF spec does indeed describe VirtualSize as "The total size of the section when loaded into memory". This is technically true if you consider total to be the total amount of REAL data that comprises the section, but it is not the total amount of memory Windows allocates for the section. You'll find VirtualSize is often LESS than the amount Windows allocates for the section in memory because VirtualSize must be rounded UP to the nearest memory alignment value (set in the PE image).
In other words, VirtualSize represents the unrounded size of the section while SizeOfRawData is the size of the data in the image file, but rounded up to the nearest file alignment padding value. This is the reason VirtualSize is the better representation of the true "raw" size of the data, in memory or on disk. The PE/COFF spec does not make this distinction. Why one is rounded in the image file and not the other probably has its ancient roots in backwards compatibility.
This is precisely why your shellcode uses VirtualSize to find the "real" end of data even as it resides in the image file. Not surprisingly, you can calculate SizeOfRawData by rounding VirtualSize up to the file alignment's value, at least in well formed PE files.
The shellcode simply used VirtualSize to find the end of the REAL code. Between there and SizeOfRawData bytes, are just unused padding zeroes, making it a prime spot to add new code without affecting the size of the file or breaking addressing offsets within the PE file.
In summary, the Windows loader essentially takes the VirtualSize value and rounds it up to the memory alignment value to get the real size of the memory allocation (and even this may be rounded up to the nearest 4k-minimum memory page). Then up to SizeOfRawData bytes are copied from the file to the beginning of the memory section. If it is less than the size of the section in memory, the remainder is zero filled.
Uhmmm, this hack code seems crafted to hide malicious code between sections.
Going to your question, you are correct VirtualSize is the space really allocated in memory and RawSize is the the space used on disk to hold the section data.
What you missed is (from MS PECOFF spec):
VirtualSize: The total size of the section when loaded into memory. If this value is greater than SizeOfRawData, the section is zero-padded. This field is valid only for executable images and should be set to zero for object files.
This means that if a the result of SizeOfRawData-VirtualSize is positive we have some space available on disk actually filled with 0's.
If that space is enough to hold the malicious code then adding the start of text section on disk with the VirtualSize you can get the start of 0's padded area that can be used to copy the code.
The rest is story...
I need to identify, whether a given pointer is a pointer to a writable or non-writable memory.
I think, it might be identified by that the pointer points to a PE section, and the section has not the IMAGE_SCN_MEM_WRITE flag set.
Is there any simple way, how to obtain the section characteristic based on just pointer? (I don't want to parse the PE in runtime, I expect, there is some direct way.
VirtualQuery will give you the required information, if the PE file has been loaded into memory. It doesn't make sense to have a pointer to something that's not loaded in memory, so this is probably sufficient for your purpose.
Since VirtualQuery works on pages, round down the pointer to the previous page boundary, and set the length to check to just one page.
I have to deal with a huge amount of data that usually doesn't fit into main memory. The way I access this data has high locality, so caching parts of it in memory looks like a good option. Is it feasible to just malloc() a huge array, and let the operating system figure out which bits to page out and which bits to keep?
Assuming the data comes from a file, you're better off memory mapping that file. Otherwise, what you end up doing is allocating your array, and then copying the data from your file into the array -- and since your array is mapped to the page file, you're basically just copying the original file to the page file, and in the process polluting the "cache" (i.e., physical memory) so other data that's currently active has a much better chance of being evicted. Then, when you're done you (typically) write the data back from the array to the original file, which (in this case) means copying from the page file back to the original file.
Memory mapping the file instead just creates some address space and maps it directly to the original file instead. This avoids copying data from the original file to the page file (and back again when you're done) as well as temporarily moving data into physical memory on the way from the original file to the page file. The biggest win, of course, is when/if there are substantial pieces of the original file that you never really use at all (in which case they may never be read into physical memory at all, assuming the unused chunk is at least a page in size).
If the data are in a large file, look into using mmap to read it. Modern computers have so much RAM, you might not enough swap space available.
I have a question to clarify my confusion about the memory organization in computer in C++.
In C++, different data is put in different location. My understanding is like this.
1) data segment section, where global and static data are located;
2) heap section, the objects created by new
3) stack section, the local variable
4) text section, the code itself.
Is that right? Is there anything I missed or did wrong?
Thanks!
I wrote an article called "C++ Internals :: Memory Layout" which will clarify this for you.
Short excerpt from the article:
.text segment
It's a Read-Only, fixed-size segment.
The text segment, a.k.a. code segment, contains executable instructions provided by the compiler and assembler.
.data segment
It's a Read-Write, fixed-size segment.
The data segment, a.k.a. initialized data segment, contains initialized:
global variables (including global static variables)
static local variables.
The segment's size depends on the size of the values in the source code, the values can be altered at run-time.
.rdata/.rodata segment
It's a Read-Only segment
The segments contains static unnamed data (like string constants)
.bss segment
It's a Read-Write and adjacent to the .data segment.
BSS segment, a.k.a. uninitialized data segment, contains statically-allocated (global and static) variables represented solely by zero-valued bits on program start. BSS stands for Block Started by Symbol, a pseudo-operation that existed in a very old assembler developed for the IBM.
heap & stack
You are quite right about those. Anyway, if you want to check some examples and get a closer look, please, refer to mentioned article.
There are typically at least two data sections. One with initialized globals, another without (BSS). A stack section isn't typically emitted in the binary.
Of course, these kind of very implementation specific questions are kinda useless if you don't specify the implementation.