I am working on this small project where I'd like to generate the call graph of an application - I am not planning to do anything complex, it is mainly for fun/experience. I am working on x64 platform.
The first goal I set myself is to be able to measure the time spent in each function of my test application. So far my strategy has been to use __penter()_ and __pexit()_ - __penter()_ is a function that will get called at the start of every method or function and conversely __pexit()_ will get called at the end of every method or function.
With these two functions I can record each function call as well as the time spent in each of them. What I'd like to do next is get the address of each function being called.
For example if we consider the following callstack (very simplified):
main()
....myFunction()
........_penter()
I am in __penter_ and I want to get the address of the calling function, myFunction(). I already found a way to do it in the case of non-leaf functions, I simply use RtlLookupFunctionEntry. However this solution doesn't seem to work for leaf functions because they don't provide any unwind data.
One thing I was thinking about is to go up one more level in the callstack, in main(), and decode the CALL procedure manually - that would involve getting a pointer to the instruction calling myFunction().
I was wondering if any of you would know how to get the address of the current function in the case of leaf functions. I have this gut feeling that my current approach is a bit overcomplicated.
Thanks,
Clem
I believe SymGetSymFromAddr64, probably along with StackWalk64 should get you (most of?) what you want.
Hmm, x64 code, no assembly hacks at your disposal unless you use ml64.exe. There's one intrinsic that ought to help here, _ReturnAddress() gives you the code location of the call to your __penter() function. The instruction after it btw. That should be enough to help you identify the caller.
Related
So recently I wanted to add an imgui interface to an example window using DirectX, so I watched on a video I had to hook the EndScene function using DirectX9sdk to be able to add my custom imgui interface.
However I have some questions:
Where can I find any documentation for the DirectX9 functions and types,( if there is any, because I do not understand why we specifically have to hook the EndScene function) or where could I find any article explaining more in depth how directX works?
I've seen two version so far of EndScene hooks one with a patternScanning function which scans a signature in the shaderapi dll and another which creates a DirectXDevice and then accesses the vtable from there; are there any sources online, or is it something we have to do ourselves?
Here is the version I have:
while (!DirectXDevice) // loops until it finds the device
DirectXDevice = **(DWORD**)(FindPattern("shaderapidx9.dll", "A1 ?? ?? ?? ?? 50 8B 08 FF 51 0C") + 0x1);
void** pVTable = *reinterpret_cast<void***>(DirectXDevice); // getting the vtable array
oEndScene = (f_EndScene)DetourFunction((PBYTE)pVTable[42], (PBYTE)Hooked_EndScene)//getting the 42th virtual function and detouring it to our own
I don't really understand what __stdcall does here, I do know it is used to call WINAPI functions but what for here?
HRESULT __stdcall Hooked_EndScene(IDirect3DDevice9* pDevice){//some code}
Note: thats the function I hook to the original endscene.
Thank you really much, I'm sorry if there are alot of questions but I really can't wrap my head around this.
How do you know which functions you need to hook?
To put it bluntly, you have to be an experienced DirectX graphics programmer to find that out. Don't expect being able to hook into a framework that you don't understand. It just so happens that EndScene will always be called after all the other draw calls on the render target.
There are tons of D3D9 programming resources available, online and in paper form. Most of them are not free. I'm afraid this is not the answer you were hoping for.
What is the deal with pattern scanning, or creating a temporary D3D9 device?
Microsoft did not put any explicit effort into making EndScene hookable. It just happens to be hookable because every normal function is hookable. You need a way to find the function in memory, because the function will not always be at the same address.
One approach is to scan for known instructions that appear inside the function. Someone needs to be the first person to find out that pattern that you can scan for. You are far from the first person to hook EndScene, so many have reverse-engineered the function before and shared searchable patterns.
NOTE: The pattern does not necessarily need to be directly inside the target function. It might also lead you to something else first, in your case, the ID3D9Device instance. The important thing is that you can find your way to the EndScene function somehow.
Another approach is to get a pointer to the function. If it was a regular C function, that would be easy. It's hard here because OOP tends to make these things hard - you have to fight your way through various interfaces to get the correct vtable.
Both methods have advantages and disadvantages -- creating a D3D9 device is safer, but also more intrusive, because the target process might not expect someone to just randomly create new devices.
Why does the hook function need __stdcall?
Since you replace the original function with your hooked version, the calling convention of the hooked function must be the same as the calling convention of the original function. The caller of EndScene expects (and was compiled with) a __stdcall convention, so the new function must also behave the same way, otherwise the stack will be corrupted. Your act of replacing the function does not change the way the caller calls it.
I would really appreciate your inputs on moving from a YieldTermStructure pointer to that of adding a spread as below::
boost::shared_ptr<YieldTermStructure> depoFutSwapTermStructure(new PiecewiseYieldCurve<Discount,
LogLinear>(settlementDate, depoFutSwapInstruments_New, termStructureDayCounter, 1.0e-15));
I tried adding a spread of 50 bps as below...
double OC_Spread(0.50 / 100);
Rate OCSQuote = OC_Spread;
boost::shared_ptr<Quote> OCS_Handler(new SimpleQuote(OCSQuote));
I then proceed to create a zerospreaded object as below:
ZeroSpreadedTermStructure Z_Spread(Handle<YieldTermStructure>(*depoFutSwapTermStructure), Handle<Quote>(OCS_Handler));
But now I am stuck as the code repeatedly breaks down if I go on ahead to do anything like
Z_Spread.zeroYieldImpl;
What is the issue with above code. I have tried several flavors of above approach and failed on all the fronts.
Also is there a native way of calling directly the discount function just like as I do now with the TermStructure object prior to adding the spread currently as below???
depoFutSwapTermStructure->discount(*it)
I'm afraid you got your interfaces a bit mixed up. The zeroYieldImpl method you're trying to call on your ZeroSpreadedTermStructure is protected, so you can't use it from your code (at least, that's how I'm guessing your code breaks, since you're not reporting the error you get).
The way you interact with the curve you created is through the public YieldTermStructure interface that it inherits; that includes the discount method that you want to call, as well as methods such as zeroRate or forwardRate.
Again, it's hard to say why your call to discount fails precisely, since you're not quoting the error and you're not saying what *it is in the call. From the initialization you do report, and from the call you wrote, I'm guessing that you might have instantiated a ZeroSpreadedTermStructure object but you're trying to use it with the -> syntax as if it were a pointer. If that's the case, calling Z_Spread.discount(*it) should work instead (assuming *it resolves to a number).
If that's not the problem, I'm afraid you'll have to add a few more details to your question.
Finally, for a more general treatment of term structures in QuantLib, you can read here and here.
I'm implementing a logger for an OpenGL application ( the only reason I'm mentioning it is that it runs in a loop ). I'd like to somehow log every method call or some group of method calls of some classes, every time they are called.
My initial approach was to place the required logger function call in all the methods ( which actually kind of works like comments :) ) but I got really tired of it really fast, so I started looking for a more effective way. I searched google for some time, but since I don't really know what I'm looking for, I ran out of ideas.
The best thing for my case would be some kind of magical method, that would be called every time I invoked any other class method, idealy with name and params string as a parameter for this method. ( kind of PHP - like magic method __call() - but that one works only if method is not defined ). I don't know what I am looking for, if something like that even exists, and if it does, what do we call it?
P.S.:
my logging works on macros, so no worries for performance there :)
#if DEV_LOG
#define log_init() logInit()
#define log_write(a,b) writeToLog(to_str(a), to_str(b))
#else
#define log_init()
#define log_write(a,b)
#endif
( And if there's a nicer way to do this, let me know, please :) )
Thank you!
1st I have to re-cite my co-answerer Filip here
C++ doesn't have this kind of "magical method", so you are stuck with explicitly stating a function call inside every member-function, if you'd like one to be made.
Such stuff is implemented as compiler specific features like the GCC profiling. There will be code generated to track for function calls, their parameters, and where these actually were called from and how often.
The general usage is to compile and link your code with special compiler flags that will generate this code. When your code is run, this information will be stored along specific kind of databases, that can be analyzed with a separate tool after running (as e.g. gprof for the GCC toolchain).
A similar tooling suite is used for retrieving code coverage of certain program runs (e.g. testsuites for your code): gcov A Test Coverage Program
C++ doesn't have this kind of "magical method", so you are stuck with explicitly stating a function call inside every member-function, if you'd like one to be made.
You could instead use a debugger to track the calls made, the program you've written shouldn't have to be responsible for questions such as "what code is called, when and with what?"; that's the exact question a profiler, or a debugger, was made to answer.
Having used gprof and callgrind many times, I have reached the (obvious) conclusion that I cannot use them efficiently when dealing with large (as in a CAD program that loads a whole car) programs. I was thinking that maybe, I could use some C/C++ MACRO magic and somehow build a simple (but nice) logging mechanism. For example, one can call a function using the following macro:
#define CALL_FUN(fun_name, ...) \
fun_name (__VA_ARGS__);
We could add some clocking/timing stuff before and after the function call, so that every function called with CALL_FUN gets timed, e.g
#define CALL_FUN(fun_name, ...) \
time_t(&t0); \
fun_name (__VA_ARGS__); \
time_t(&t1);
The variables t0, t1 could be found in a global logging object. That logging object can also hold the calling graph for each function called through CALL_FUN. Afterwards, that object can be written in a (specifically formatted) file, and be parsed from some other program.
So here comes my (first) question: Do you find this approach tractable ? If yes, how can it be enhanced, and if not, can you propose a better way to measure time and log callgraphs ?
A collegue proposed another approach to deal with this problem, which is annotating with a specific comment each function (that we care to log). Then, during the make process, a special preprocessor must be run, parse each source file, add logging logic for each function we care to log, create a new source file with the newly added (parsing) code, and build that code instead. I guess that reading CALL_FUN... macros (my proposal) all over the place is not the best approach, and his approach would solve this problem. So what is your opinion about this approach?
PS: I am not well versed in the pitfalls of C/C++ MACROs, so if this can be developed using another approach, please say it so.
Thank you.
Well you could do some C++ magic to embed a logging object. something like
class CDebug
{
CDebug() { ... log somehow ... }
~CDebug() { ... log somehow ... }
};
in your functions then you simply write
void foo()
{
CDebug dbg;
...
you could add some debug info
dbg.heythishappened()
...
} // not dtor is called or if function is interrupted called from elsewhere.
I am bit late, but here is what I am doing for this:
On Windows there is a /Gh compiler switch which makes the compiler to insert a hidden _penter function at the start of each function. There is also a switch for getting a _pexit call at the end of each function.
You can utilizes this to get callbacks on each function call. Here is an article with more details and sample source code:
http://www.johnpanzer.com/aci_cuj/index.html
I am using this approach in my custom logging system for storing the last few thousand function calls in a ring buffer. This turned out to be useful for crash debugging (in combination with MiniDumps).
Some notes on this:
The performance impact very much depends on your callback code. You need to keep it as simple as possible.
You just need to store the function address and module base address in the log file. You can then later use the Debug Interface Access SDK to get the function name from the address (via the PDB file).
All this works suprisingly well for me.
Many nice industrial libraries have functions' declarations and definitions wrapped into void macros, just in case. If your code is already like that -- go ahead and debug your performance problems with some simple asynchronous trace logger. If no -- post-insertion of such macros can be an unacceptably time-consuming.
I can understand the pain of running an 1Mx1M matrix solver under valgrind, so I would suggest starting with so called "Monte Carlo profiling method" -- start the process and in parallel run pstack repeatedly, say each second. As a result you will have N stack dumps (N can be quite significant). Then, the mathematical approach would be to count relative frequencies of each stack and make a conclusion about the ones most frequent. In practice you either immediately see the bottleneck or, if no, you switch to bisection, gprof, and finally to valgrind's toolset.
Let me assume the reason you are doing this is you want to locate any performance problems (bottlenecks) so you can fix them to get higher performance.
As opposed to measuring speed or getting coverage info.
It seems you're thinking the way to do this is to log the history of function calls and measure how long each call takes.
There's a different approach.
It's based on the idea that mainly the program walks a big call tree.
If time is being wasted it is because the call tree is more bushy than necessary,
and during the time that's being wasted, the code that's doing the wasting is visible on the stack.
It can be terminal instructions, but more likely function calls, at almost any level of the stack.
Simply pausing the program under a debugger a few times will eventually display it.
Anything you see it doing, on more than one stack sample, if you can improve it, will speed up the program.
It works whether or not the time is being spent in CPU, I/O or anything else that consumes wall clock time.
What it doesn't show you is tons of stuff you don't need to know.
The only way it can not show you bottlenecks is if they are very small,
in which case the code is pretty near optimal.
Here's more of an explanation.
Although I think it will be hard to do anything better than gprof, you can create a special class LOG for instance and instantiate it in the beginning of each function you want to log.
class LOG {
LOG(const char* ...) {
// log time_t of the beginning of the call
}
~LOG(const char* ...) {
// calculate the total time spent,
//by difference between current time and that saved in the constructor
}
};
void somefunction() {
LOG log(__FUNCTION__, __FILE__, ...);
.. do other things
}
Now you can integrate this approach with the preprocessing one you mentioned. Just add something like this in the beginning of each function you want to log:
// ### LOG
and then you replace the string automatically in debug builds (shoudn't be hard).
May be you should use a profiler. AQTime is a relatively good one for Visual Studio. (If you have VS2010 Ultimate, you already have a profiler.)
I came across the following weird chunk of code.Imagine you have the following typedef:
typedef int (*MyFunctionPointer)(int param_1, int param_2);
And then , in a function , we are trying to run a function from a DLL in the following way:
LPCWSTR DllFileName; //Path to the dll stored here
LPCSTR _FunctionName; // (mangled) name of the function I want to test
MyFunctionPointer functionPointer;
HINSTANCE hInstLibrary = LoadLibrary( DllFileName );
FARPROC functionAddress = GetProcAddress( hInstLibrary, _FunctionName );
functionPointer = (MyFunctionPointer) functionAddress;
//The values are arbitrary
int a = 5;
int b = 10;
int result = 0;
result = functionPointer( a, b ); //Possible error?
The problem is, that there isn't any way of knowing if the functon whose address we got with LoadLibrary takes two integer arguments.The dll name is provided by the user at runtime, then the names of the exported functions are listed and the user selects the one to test ( again, at runtime :S:S ).
So, by doing the function call in the last line, aren't we opening the door to possible stack corruption? I know that this compiles, but what sort of run-time error is going to occur in the case that we are passing wrong arguments to the function we are pointing to?
There are three errors I can think of if the expected and used number or type of parameters and calling convention differ:
if the calling convention is different, wrong parameter values will be read
if the function actually expects more parameters than given, random values will be used as parameters (I'll let you imagine the consequences if pointers are involved)
in any case, the return address will be complete garbage, so random code with random data will be run as soon as the function returns.
In two words: Undefined behavior
I'm afraid there is no way to know - the programmer is required to know the prototype beforehand when getting the function pointer and using it.
If you don't know the prototype beforehand then I guess you need to implement some sort of protocol with the DLL where you can enumerate any function names and their parameters by calling known functions in the DLL. Of course, the DLL needs to be written to comply with this protocol.
If it's a __stdcall function and they've left the name mangling intact (both big ifs, but certainly possible nonetheless) the name will have #nn at the end, where nn is a number. That number is the number of bytes the function expects as arguments, and will clear off the stack before it returns.
So, if it's a major concern, you can look at the raw name of the function and check that the amount of data you're putting onto the stack matches the amount of data it's going to clear off the stack.
Note that this is still only a protection against Murphy, not Machiavelli. When you're creating a DLL, you can use an export file to change the names of functions. This is frequently used to strip off the name mangling -- but I'm pretty sure it would also let you rename a function from xxx#12 to xxx#16 (or whatever) to mislead the reader about the parameters it expects.
Edit: (primarily in reply to msalters's comment): it's true that you can't apply __stdcall to something like a member function, but you can certainly use it on things like global functions, whether they're written in C or C++.
For things like member functions, the exported name of the function will be mangled. In that case, you can use UndecorateSymbolName to get its full signature. Using that is somewhat nontrivial, but not outrageously complex either.
I do not think so, it is a good question, the only provision is that you MUST know what the parameters are for the function pointer to work, if you don't and blindly stuff the parameters and call it, it will crash or jump off into the woods never to be seen again... It is up to the programmer to convey the message on what the function expects and the type of parameters, luckily you could disassemble it and find out from looking at the stack pointer and expected address by way of the 'stack pointer' (sp) to find out the type of parameters.
Using PE Explorer for instance, you can find out what functions are used and examine the disassembly dump...
Hope this helps,
Best regards,
Tom.
It will either crash in the DLL code (since it got passed corrupt data), or: I think Visual C++ adds code in debug builds to detect this type of problem. It will say something like: "The value of ESP was not saved across a function call", and will point to code near the call. It helps but isn't totally robust - I don't think it'll stop you passing in the wrong but same-sized argument (eg. int instead of a char* parameter on x86). As other answers say, you just have to know, really.
There is no general answer. The Standard mandates that certain exceptions be thrown in certain circumstances, but aside from that describes how a conforming program will be executed, and sometimes says that certain violations must result in a diagnostic. (There may be something more specific here or there, but I certainly don't remember one.)
What the code is doing there isn't according to the Standard, and since there is a cast the compiler is entitled to go ahead and do whatever stupid thing the programmer wants without complaint. This would therefore be an implementation issue.
You could check your implementation documentation, but it's probably not there either. You could experiment, or study how function calls are done on your implementation.
Unfortunately, the answer is very likely to be that it'll screw something up without being immediately obvious.
Generally if you are calling LoadLibrary and GetProcByAddrees you have documentation that tells you the prototype. Even more commonly like with all of the windows.dll you are provided a header file. While this will cause an error if wrong its usually very easy to observe and not the kind of error that will sneak into production.
Most C/C++ compilers have the caller set up the stack before the call, and readjust the stack pointer afterwards. If the called function does not use pointer or reference arguments, there will be no memory corruption, although the results will be worthless. And as rerun says, pointer/reference mistakes almost always show up with a modicum of testing.