I'm developing some software for microcontrollers, and I would like to be able to easily see which parts of the software are using how much memory. The software does not use dynamic memory allocation, I am only interested in static memory allocations (the bss and data sections).
All of this static memory is actually part of a single struct, which contains (most of the) memory the program works with. This is a hierarchy of structs, corresponding to the components of the program. E.g.:
struct WholeProgram {
int x;
struct ComponentA a;
struct ComponentB b;
};
struct ComponentA {
int y;
struct ComponentC c;
struct ComponentD d;
};
...
struct WholeProgram whole_program;
Ideally, I would like to see the memory usage represented with a multi-level pie chart.
I could not find anything that can descend into structures like this, only programs which print the size of global variables (nm). This isn't too useful for me because it would only tell me the size of the WholeProgram struct, without any details about its parts.
Note that the solution must not be in the form of a program that parses the code. This would be unacceptable for me because I use a lot of C++ template metaprogramming, and the program would surely not be able to handle that.
If such a tool is not available, I would be interested in ways to retrieve this memory usage information (from the binary or the compiler).
Rather than using nm, you could get the same information (and possibly more) by getting the linker to output a map file directly. However this may not solve your problem - the internal offsets of a structure can be resolved by the compiler and the symbols discarded and therefore need not be visible in the final link map - only the external references are preserved for the purposes of linking.
However, the information necessary to achieve your aim must be available to the debugger (since it is able to expand a structure), so some tool that can parse your compiler's specific debug information - perhaps even the debugger itself - but that is a long shot, I imagine that you would have to write such a tool yourself.
The answers to GDB debug info parser/description may help.
If you declare instances of the component structs at global scope instead of inside the whole_program struct, your map file should give you the sizes of each component struct.
Packing all the components into one single structure naturally results in only whole_program being listed in the map file.
Related
I'm fairly new to c++ and am really interested in learning more. Have been reading quite a bit. Recently discovered the init/fini elf sections.
I started to wonder if & how one would use the init section to prepopulate objects that would be used at runtime. Say for example you wanted
to add performance measurements to your code, recording the time, filename, linenumber, and maybe some ID (monotonic increasing int for ex) or name.
You would place for example:
PROBE(0,"EventProcessing",__FILE__,__LINE__)
...... //process event
PROBE(1,"EventProcessing",__FILE__,__LINE__)
......//different processing on same event
PROBE(2,"EventProcessing",__FILE__,__LINE__)
The PROBE could be some macro that populates a struct containing this data (maybe on an array/list, etc using the id as an indexer).
Would it be possible to have code in the init section that could prepopulate all of this data for each PROBE (except for the time of course), so only the time would need to be retrieved/copied at runtime?
As far as I know the __attribute__((constructor)) can not be applied to member functions?
My initial idea was to create some kind of
linked list with each node pointing to each probe and code in the init secction could iterate it populating the id, file, line, etc, but
that idea assumed I could use a member function that could run in the "init" section, but that does not seem possible. Any tips appreciated!
As far as I understand it, you do not actually need an ELF constructor here. Instead, you could emit descriptors for your probes using extended asm statements (using data, instead of code). This also involves switching to a dedicated ELF section for the probe descriptors, say __probes.
The linker will concatenate all the probes and in an array, and generate special symbols __start___probes and __stop___probes, which you can use from your program to access thes probes. See the last paragraph in Input Section Example.
Systemtap implements something quite similar for its userspace probes:
User Space Probe Implementation
Adding User Space Probing to an Application (heapsort example)
Similar constructs are also used within the Linux kernel for its self-patching mechanism.
There's a pretty simple way to have code run on module load time: Use the constructor of a global variable:
struct RunMeSomeCode
{
RunMeSomeCode()
{
// your code goes here
}
} do_it;
The .init/.fini sections basically exist to implement global constructors/destructors as part of the ABI on some platforms. Other platforms may use different mechanisms such as _start and _init functions or .init_array/.deinit_array and .preinit_array. There are lots of subtle differences between all these methods and which one to use for what is a question that can really only be answered by the documentation of your target platform. Not all platforms use ELF to begin with…
The main point to understand is that things like the .init/.fini sections in an ELF binary happen way below the level of C++ as a language. A C++ compiler may use these things to implement certain behavior on a certain target platform. On a different platform, a C++ compiler will probably have to use different mechanisms to implement that same behavior. Many compilers will give you tools in the form of language extensions like __attributes__ or #pragmas to control such platform-specific details. But those generally only make sense and will only work with that particular compiler on that particular platform.
You don't need a member function (which gets a this pointer passed as an arg); instead you can simply create constructor-like functions that reference a global array, like
#define PROBE(id, stuff, more_stuff) \
__attribute__((constructor)) void \
probeinit##id(){ probes[id] = {id, stuff, 0/*to be written later*/, more_stuff}; }
The trick is having this macro work in the middle of another function. GNU C / C++ allows nested functions, but IDK if you can make them constructors.
You don't want to declare a static int dummy#id = something because then you're adding overhead to the function you profile. (gcc has to emit a thread-safe run-once locking mechanism.)
Really what you'd like is some kind of separate pass over the source that identifies all the PROBE macros and collects up their args to declare
struct probe global_probes[] = {
{0, "EventName", 0 /*placeholder*/, filename, linenum},
{1, "EventName", 0 /*placeholder*/, filename, linenum},
...
};
I'm not confident you can make that happen with CPP macros; I don't think it's possible to #define PROBE such that every time it expands, it redefines another macro to tack on more stuff.
But you could easily do that with an awk/perl/python / your fave scripting language program that scans your program and constructs a .c that declares an array with static storage.
Or better (for a single-threaded program): keep the runtime timestamps in one array, and the names and stuff in a separate array. So the cache footprint of the probes is smaller. For a multi-threaded program, stores to the same cache line from different threads is called false sharing, and creates cache-line ping-pong.
So you'd have #define PROBE(id, evname, blah blah) do { probe_times[id] = now(); }while(0)
and leave the handling of the later args to your separate preprocessing.
Say I have a DLL contains a struct, but I don't know the details of this struct. But I have a void pointer which points to address of the struct.
Can anybody tell me how can I get the details of the struct? Such as output the struct to a text file.
Thank you!
You cannot know the details of the struct without the type definition. Copying a region starting with the void pointer without a type definition will give you the raw binary data, but you wont know where it ends, or which pieces represent which variables. Some of the values could be integer values or they could be pointer addresses. There are all sorts of possibilities.
You should try to obtain the header file.
You might be able to glean some information from the debug / symbol file if you have it (example .pdb files on Windows), or debugging the program with GDB on Linux, this will only work if you have a debug build of the program. Refer to the "whatis" and "ptype" commands in GDB.
You never know this without structure definition. Also there can be "holes" between the user's variables in the real memory placement because of the alignment and padding.
Say if you have,
struct mystr {
char x;
int y;
};
by default such structure most likely will have size 8, and after one byte of char x there will be three bytes of padding (in theory random values), and then 4 bytes of int y, but it depends on compiler and its directives.
I have a C++ object which needs a huge amount of data to instantiate. For example:
class object {
public object() {
double a[] = { array with 1 million double element };
/* rest of code here*/};
private:
/* code here*/
}
Now the data (i.e 1 million double numbers) is in a separate text file. The question: How can I put it after "double a[]" in an efficient way and eventually compile the code? I do not want to read the data at run time from a file. I want it compiled with the object. What can be a solution? Ideally I would like the data to sit in the separate text file as it presently resides and somehow also have an assignment like double a[] =..... above.
Is this possible? Thanks in advance!
Something like:
class object
{
public
object(){ double a[] = {
#include "file.h"
};
/* rest of code here*/};
private:
/* code here*/
}
The file has to be formatted correctly though - i.e. contain something like:
//file.h
23, 24, 40,
5, 1.1,
In general, you can use #include directives to paste content into files. I've seen virtual methods being pasted like that, if they were common for most derived classes. I personally don't really like this technique.
One large problem with this design is that 1 million ints on the stack will probably blow the stack. What you probably want is to put the data on the data segment, or in some kind of resource that is stores in your binary file and can be loaded at run time. If you need more than one copy of the data, duplicate it into a std::vector at run time, so you know the data is on the free store (heap). Mayhap even use a shared_ptr to a std::array to reduce the chance of needless accidental duplication(or unique_ptr to reduce the chance of reference duplication).
4mb of data is not going to play all that well is all I am saying. And locality of reference on a 4mb array to your other variables is not going to be your biggest concern.
Depending in your compiled target platform and framework, there will be ways to stuff this kind of data into a binary resource. I've never done it for a multi-meg file, but here is the visual studio help on resource files: http://msdn.microsoft.com/en-us/library/7zxb70x7%28v=vs.80%29.aspx
Note that "the data being in the code" does not make it fundamentally faster to load (other than traversing the filesystem once to find it maybe). The OS still has to load the binary, and larger binaries take more time to load, and a big array of values will take up as much room in a binary as it does in a distinct file. The real advantage is that it isn't a file that can be "misplaced" relative to your executable, but resource fork/resource file/etc methods can deal with that.
As noted in the comments below, static const data (and global data) tends to be loaded into the data segment, which is distinct from both the heap (aka free store) and stack (aka automatic store). I forget what the standard calls it. I do know that a static local variable in a function will behave differently than a static or global non-local variable with regards to initialization order (global (static or not) data gets initialized fully prior to main starting, while static local is initialized the first time the function is called, if I remember correctly).
The answer of Luchian Grigore is quite correct. But compiler can have some limit on length of source code line. See for example https://stackoverflow.com/questions/10519738/source-line-length-limit
So try on your compiler. But I am afraid, more simple solution of your problem will be reading of huge data from file.
Unfortunately I'm not even sure how this sort of static analysis is called. It's not really control flow analysis because I'm not looking for function calls and I don't really need data flow analysis because I don't care about the actual values.
I just need a tool that lists the locations (file, function) where writing access to a specific variable takes place. I don't even care if that list contained lines that are unreachable. I could imagine that writing a simple parser could suffice for this task but I'm certain that there must be a tool out there that does this simple analysis.
As a poor student I would appreciate free or better yet open source tools and if someone could tell me how this type of static analysis is actually called, I would be equally grateful!
EDIT: I forgot to mention there's no pointer arithmetic in the code base.
Why don't you make the variable const and then note down all the errors where your compiler bans write access?
Note: This won't catch errors where the memory underlying the variable is written to in some erroneous manner such as a buffer overrun.
EDIT: For example:
const int a = 1;
a = 2;
a = 3;
My compiler produces:
1>MyProg.c(46): error C3892: 'a' : you cannot assign to a variable that is const
1>MyProg.c(47): error C3892: 'a' : you cannot assign to a variable that is const
Do you mean something like this?
This works for C programs that you have made the effort to analyze with Frama-C's value analysis. It is Open Source and the dependency information is also available programmatically. As static analyzers go, it is rather on the “precise” side of the spectrum. It will work better if your target is embedded C code.
I am not sure such a tool could be written. Pointers can be used to change arbitary data in memory without having any reference to other variables pointing to that data. Think about functions like memset(), which change whole blocks of memory.
If you are not interested in these kind of mutations, you would still have to take transitive pointers into account. In C, you can have any number of pointers pointing to the same data, and you would have to analyze where copies of these pointers are made. And then these copies can be copied again, ...
So even in the "simple" case it would require quite a big amount of code analysis.
I'd like to change the way some types are displayed using either 'dt' or '??' in a manner similar to how you can do that with autoexp.dat. Is there a way to do this?
For example, I have a structure something like this:
struct Foo
{
union Bar
{
int a;
void *p;
} b;
};
And I've got an array of a few hundred of these, all of which I know point to a structure Bar. Is there any way to tell cdb that, in this expression anyway, that 'p' is a pointer to Bar? This is the kind of thing you could do with autoexp. (The concrete example here is that I've got a stashtable that can have keys of any type, but I know they keys are strings. the implementation stores them as void pointers).
Thanks in advance!
I don't think there's anything as simple as autoexp.dat.
You have a couple potential options - you can write a simple script file with the debugger commands to dump the data structure in the way you want and use the "$<filename" command (or one of its variants). Combined with user aliases you can get this to be pretty easy and natural to use.
The second option is quite a bit more involved, but with it comes much more power - write an extension DLL that dumps your data structure. For something like what you're talking about this is probably overkill. But you have immense power with debugger extensions (in fact, much of the power that comes in the Debugging tools package is implemented this way). The SDK is packaged with the debugger, so it's easy to determine if this is what you might need.
You can say du or da to have it dump memory as unicode or ascii strings.