I have the following question and from a systems perspective want to know how to achieve this easily and efficiently.
Given a task 'abc' that has been built with debug information and a global variable "TRACE" that is normally set to 0, I would like to print out to file 'log' the address of each function that is called between the time that TRACE is set to 1 and back again to 0.
I was considering doing this through a front-loading / boot-strapping task that I'd develop which looks at the instructions for a common pattern of jump/frame pointer push, writing down the address and then mapping addresses to function names from the symbolic debug information in abc. There could be better system level ways to do this without a front-loader though, and I'm not sure what is most feasible.
Any implemented techniques out there?
One possibility is to preprocess the source before compiling it. This preprocessing would add code at the beginning of each function that would check the TRACE global and, if set, write to the log. As Mystagogue said, the compiler has preprocessor macros that expand to the name of the function.
You might also look at some profiling tools. Some of them have functionality close to what you're asking for. For example, some will sample the entire callstack periodically, which can tell you a lot about the code flow without actually logging every call.
Looking for a common prologue/epilogue won't work in the presence of frame-pointer omission and tail call optimization. Also, modern optimizers like to split functions into several chunks and merge common tail chunks of different functions.
There is no standard solution.
For Microsoft compiler, check out _penter and _pexit hooks. For GCC, look at -finstrument-functions option and friends.
Also, on x86 Windows you can use a monitor such as WinApiOverride32. It's primarily intended for monitoring DLL and system API calls, but you can generate a description file from your application's map file and monitor internal functions as well.
(Edited: added link to GCC option.)
Make sure you've looked into the __func__ or __FUNCTION__ predefined identifiers. They provide a string literal of the function/method name you are currently executing.
Related
If you set a breakpoint in the debugger over a function ptr, you will see the name of the assembly included in the inspector panel. This seems to work for all function objects including lambdas.
You may see something like this;
Func=0x00FF00FF00{UE4Editor-Game.dll!<lambda_4b5336d9060965465490645>::<lambda_invoker_cdecl>}
Question; How would one programmatically obtain a string containing the assembly information given here, using the function pointer Func and functions that are available a Windows development environment?
For the given example I would call something like this;
const char* details = GetFunctionAssemblyString(Func);
The most important part I would like to obtain is this; UE4Editor-Game.dll However the full string might also be interesting...
This is for development tools only, and not-intended to be cross platform, so using windows specific functions is acceptable. I have access to the debug database .pdb.
Cheers
Note that 'assembly' is a term that is limited to .net, it indicates an image file (whether exe or dll or otherwise) that has attached metadata. For native contexts the analogous term is 'module', a module may or may not have symbol names available but is likely not going to have more than that in the usual case. Note that there may possibly be debugging information but that can be removed and the module will continue to work, the same is not the case if the metadata were removed from a .net assembly.
All of that being said, you can use the Debug Help Library to get as much information about a native process as is available. Note that comments in SymInitialize make it sound like it is not feasible for a process to load information about itself. Once you have initialized dbghelp for a particular process you could use SymFromAddr to get the name associated with a particular address and then SymGetModuleInfo64 to get information for the module containing that address.
I recently posted a question about stack segmentation and boost coroutines but it seems like the -fsplit-stack approach only works with source files that are compiled with that flag, the runtime breaks down when you branch to another function that has not been compiled with -fsplit-stack. For example
This implies that the runtime uses a function local technique to detect when the current stack has been surpassed. And not a "guard page signal" trick, where the end of the stack always has a guard page which will raise a signal on write or read, telling the runtime to allocate a new stack frame and branch to that.
Then what is the use of this flag? If I link to any other library that has not been built with this, code will break (even libstdc++ and libc), then how is this something people use practically with big projects?
From reading the gcc wiki about split stacks it seems like calling a non split stack function from a split stack function results in an allocation of a 64KB stack frame. Good.
But it seems like calling a non split stack function from a function pointer has not yet been implemented to follow the above scheme.
What use is this flag then? If I proceed to call any virtual function will my program break?
Further from the answer below it seems like clang has not implemented split stacks?
You have to compile boost (at least boost.context and boost.coroutine) with segmeented-stacks support AND your application.
compile boost (boost.context and boost.coroutine) with b2 property segmented-stacks=on (enables special code inside boost.coroutine and boost.context).
your app has to be compiled with -DBOOST_USE_SEGMENTED_STACKS and -fsplit-stack (required by boost.coroutines headers).
see boost.coroutine documentation
boost.coroutine contains an example that demonstrates segmented stacks (in directory coroutine/example/asymmetric/ call b2 toolset=gcc segmented-stacks=on).
regarding your last question GCC Wiki states:
For calls from split-stack code to non-split-stack code, the linker
will change the initial instructions in the split-stack (caller)
function. This means that the linker will have to have special
knowledge of the instructions that the compiler emits. The effect of
the changes will be to increase the required framesize by a number
large enough to reasonably work for a non-split-stack. This will be a
target dependent number; the default will be something like 64K. Note
that this large stack will be released when the split-stack function
returns. Note that I'm disregarding the case of split-stack code in a
shared library calling non-split-stack code in the main executable;
that seems like an unlikely problem.
please note: while llvm supports segmented stacks, clang seams not to provide the __splitstack_<xyz> functions.
First I'd say split stack support is somewhat experimental in nature to begin with. It is not a widely supported thing nor has a single implementation become accepted as the way to go. As such, part of the purpose of it existing in the compiler is to enable research in real use.
That said, one generally wants to use such a feature to enable lots of threads with small stacks, but which can get bigger if they need to. In some applications, the code that runs in these threads can be tightly controlled. E.g. fairly specialized request handlers that do not call general purpose libraries such as Boost. High performance systems work often involves tightening down the constraints on what code is used in a given path and this would be an example thereof. It certainly limits the applicability of the feature, but I wouldn't be surprised if someone is using it in production this way.
Note that similar issues exist with flags such as -fno-exceptions and -fno-rtti . Generally C++ requires compiling everything that goes into an executable with a compatible set of flags. Sometimes one can mix and match, but it is often fragile. This is part of the motivation of building everything from source and hermetic build tools like bazel. Other languages have different approaches to non-source components, especially virtual machine based languages such as Java and the .NET family. In those worlds things like split stacks are decided at a lower-level of compilation, but typically one would not have any control over or awareness of them at the source code level.
Is it possible to 'peek' at the stack enough to deduce, perhaps by mapping an address to the debug .map file or something, what the calling function is programmatically?
I have a function that is called from a ton of different places, and basically if possible I would like to be able to programmatically log out who called the function so that I can trace the progression of parameter values over time, and be able to connect them back to where they may be going wrong. I could add a parameter so that the caller must provide a user string or something, but I'd like to do something less intrusive if it's possible.
GCC has features for this, such as __builtin_return_address (see http://gcc.gnu.org/onlinedocs/gcc/Return-Address.html). They should be used only for debugging or special testing purposes and not as part of production code.
You can generate MiniDump files using windows API and load them later in the debugger and if symbols are available you should be able to debug the crash and investigate values of variables.
I have the location/offset of a particular function present inside an executable. Would it be possible to call such a function (while suppressing the CRT's execution of the executable's entry point, hopefully) ?
In effect, you can simulate the Windows loader, assuming you run under Windows, but the basics should be the same on any platform. See e.g. http://msdn.microsoft.com/en-us/magazine/cc301805.aspx.
Load the file into memory,
Replace all relative addresses of functions that are called by the loaded executable with the actual function addresses.
Change the memory page to "executable" (this is the difficult and platform-dependent part)
Initialize the CRT in order to, e.g., initialize static variables.
Call.
However, as the commenters point out correctly, this might only be practical as an exercise using very simple functions. There are many, many things that can go wrong if you don't manage to emulate the complete OS loader.
PS: You could also ask the Google: http://www.cultdeadcow.com/tools/pewrap.html
PPS: You may also find helpful advice in the "security" community: https://www.blackhat.com/presentations/bh-usa-07/Harbour/Whitepaper/bh-usa-07-harbour-WP.pdf
Yes, you can call it, if you will initialize all global variables which this function uses. Probably including CRT global variables. As alternative way, you can hook and replace all CRT functions that callee uses. See disassembly of that function to get right solution.
1) Take a look at the LoadLibraryEx() API. It has some flags that could be able to do all the dirty work described by Sebastian.
2) Edit the executable. Several modified bytes will do the job. Here is some documentation on the file format: http://docsrv.sco.com:507/en/topics/COFF.html
I was wondering if its possible / anyone knows any tools out there to compare the execution of two related programs (for example, assignments on a class) to see how similar they are. For example, not to compare the names of functions, but how they use syscalls. One silly case of this would be testing if a C string is printed as (see example below) in more than one case one separate program.
printf("%s",str)
Or as
for (i=0;i<len;i++) printf("%c",str[i]);
I haven´t put much thought into this, but i would imagine that strace / ltrace (maybe even oprofile) would be a good starting point. Particularly, this is for UNIX C / C++ programs.
Thanks.
If you have access to the source code of the two programs, you may build a graph of the functions (each function is a node, and there is an edge from A to B if A calls B()), and compute some graph similarity metrics. This will catch a source code copy made by renaming and reorganizing.
An initial idea would be to use ltrace and strace to log the calls and then use diff on the logs. This would obviously only cover the library an system calls. If you need a more fine granular logging, the oprofile might help.
If you have access to the source code you could instrument your code by compiling it with profiling information and then parse the gcov output after the runs. A pure static source code analysis may be sufficient if your code is not taking different routes depending on external data/state.
I think you can do this kind of thing using valgrind.
A finer-grained version (and depending on what is the access to the program source and what you exactly want in terms of comparison) would be to use kprobes.
Kernel Dynamic Probes (Kprobes) provides a lightweight interface for kernel modules to implant probes and register corresponding probe handlers. A probe is an automated breakpoint that is implanted dynamically in executing (kernel-space) modules without the need to modify their underlying source. Probes are intended to be used as an ad hoc service aid where minimal disruption to the system is required. They are particularly advocated in production environments where the use of interactive debuggers is undesirable. Kprobes also has substantial applicability in test and development environments. During test, faults may be injected or simulated by the probing module. In development, debugging code (for example a printk) may be easily inserted without having to recompile to module under test.