Function pointers and unknown number of arguments in C++ - c++

I came across the following weird chunk of code.Imagine you have the following typedef:
typedef int (*MyFunctionPointer)(int param_1, int param_2);
And then , in a function , we are trying to run a function from a DLL in the following way:
LPCWSTR DllFileName; //Path to the dll stored here
LPCSTR _FunctionName; // (mangled) name of the function I want to test
MyFunctionPointer functionPointer;
HINSTANCE hInstLibrary = LoadLibrary( DllFileName );
FARPROC functionAddress = GetProcAddress( hInstLibrary, _FunctionName );
functionPointer = (MyFunctionPointer) functionAddress;
//The values are arbitrary
int a = 5;
int b = 10;
int result = 0;
result = functionPointer( a, b ); //Possible error?
The problem is, that there isn't any way of knowing if the functon whose address we got with LoadLibrary takes two integer arguments.The dll name is provided by the user at runtime, then the names of the exported functions are listed and the user selects the one to test ( again, at runtime :S:S ).
So, by doing the function call in the last line, aren't we opening the door to possible stack corruption? I know that this compiles, but what sort of run-time error is going to occur in the case that we are passing wrong arguments to the function we are pointing to?

There are three errors I can think of if the expected and used number or type of parameters and calling convention differ:
if the calling convention is different, wrong parameter values will be read
if the function actually expects more parameters than given, random values will be used as parameters (I'll let you imagine the consequences if pointers are involved)
in any case, the return address will be complete garbage, so random code with random data will be run as soon as the function returns.
In two words: Undefined behavior

I'm afraid there is no way to know - the programmer is required to know the prototype beforehand when getting the function pointer and using it.
If you don't know the prototype beforehand then I guess you need to implement some sort of protocol with the DLL where you can enumerate any function names and their parameters by calling known functions in the DLL. Of course, the DLL needs to be written to comply with this protocol.

If it's a __stdcall function and they've left the name mangling intact (both big ifs, but certainly possible nonetheless) the name will have #nn at the end, where nn is a number. That number is the number of bytes the function expects as arguments, and will clear off the stack before it returns.
So, if it's a major concern, you can look at the raw name of the function and check that the amount of data you're putting onto the stack matches the amount of data it's going to clear off the stack.
Note that this is still only a protection against Murphy, not Machiavelli. When you're creating a DLL, you can use an export file to change the names of functions. This is frequently used to strip off the name mangling -- but I'm pretty sure it would also let you rename a function from xxx#12 to xxx#16 (or whatever) to mislead the reader about the parameters it expects.
Edit: (primarily in reply to msalters's comment): it's true that you can't apply __stdcall to something like a member function, but you can certainly use it on things like global functions, whether they're written in C or C++.
For things like member functions, the exported name of the function will be mangled. In that case, you can use UndecorateSymbolName to get its full signature. Using that is somewhat nontrivial, but not outrageously complex either.

I do not think so, it is a good question, the only provision is that you MUST know what the parameters are for the function pointer to work, if you don't and blindly stuff the parameters and call it, it will crash or jump off into the woods never to be seen again... It is up to the programmer to convey the message on what the function expects and the type of parameters, luckily you could disassemble it and find out from looking at the stack pointer and expected address by way of the 'stack pointer' (sp) to find out the type of parameters.
Using PE Explorer for instance, you can find out what functions are used and examine the disassembly dump...
Hope this helps,
Best regards,
Tom.

It will either crash in the DLL code (since it got passed corrupt data), or: I think Visual C++ adds code in debug builds to detect this type of problem. It will say something like: "The value of ESP was not saved across a function call", and will point to code near the call. It helps but isn't totally robust - I don't think it'll stop you passing in the wrong but same-sized argument (eg. int instead of a char* parameter on x86). As other answers say, you just have to know, really.

There is no general answer. The Standard mandates that certain exceptions be thrown in certain circumstances, but aside from that describes how a conforming program will be executed, and sometimes says that certain violations must result in a diagnostic. (There may be something more specific here or there, but I certainly don't remember one.)
What the code is doing there isn't according to the Standard, and since there is a cast the compiler is entitled to go ahead and do whatever stupid thing the programmer wants without complaint. This would therefore be an implementation issue.
You could check your implementation documentation, but it's probably not there either. You could experiment, or study how function calls are done on your implementation.
Unfortunately, the answer is very likely to be that it'll screw something up without being immediately obvious.

Generally if you are calling LoadLibrary and GetProcByAddrees you have documentation that tells you the prototype. Even more commonly like with all of the windows.dll you are provided a header file. While this will cause an error if wrong its usually very easy to observe and not the kind of error that will sneak into production.

Most C/C++ compilers have the caller set up the stack before the call, and readjust the stack pointer afterwards. If the called function does not use pointer or reference arguments, there will be no memory corruption, although the results will be worthless. And as rerun says, pointer/reference mistakes almost always show up with a modicum of testing.

Related

Passing function pointers as an API interface to a compiled library

Dearest stack exchange,
I'm programming an MRI scanner. I won't go into too much background, but I'm fairly constrained in how much code I've got access to, and the way things have been set up is...suboptimal. I have a situation as follows:
There is a big library, written in C++. It ultimately does "transcoding" (in the worst possible way), writing out FPGA assembly that DoesThings. It provides a set of functions to "userland" that are translated into (through a mix of preprocessor macros and black magic) long strings of 16 bit and 32 bit words. The way this is done is prone to buffer overflows, and generally to falling over.*
The FPGA assembly is then strung out over a glorified serial link to the relevant electronics, which executes it (doing the scan), and returning the data back again for processing.
Programmers are expected to use the functions provided by the library to do their thing, in C (not C++) functions that are linked against the standard library. Unfortunately, in my case, I need to extend the library.
There's a fairly complicated chain of preprocessor substitution and tokenization, calling, and (in general) stuff happening between you writing doSomething() in your code, and the relevant library function actually executing it. I think I've got it figured out to some extent, but it basically means that I've got no real idea about the scope of anything...
In short, my problem is:
In the middle of a method, in a deep dark corner of many thousands of lines of code in a big blob I have little control over, with god-knows-what variable scoping going on, I need to:
Extend this method to take a function pointer (to a userland function) as an argument, but
Let this userland function, written after the library has been compiled, have access to variables that are local to both the scope of the method where it appears, as well as variables in the (C) function where it is called.
This seems like an absolute mire of memory management, and I thought I'd ask here for the "best practice" in these situations, as it's likely that there are lots of subtle issues I might run into -- and that others might have lots of relevant wisdom to impart. Debugging the system is a nightmare, and I've not really got any support from the scanner's manufacturer on this.
A brief sketch of how I plan to proceed is as follows:
In the .cpp library:
/* In something::something() /*
/* declare a pointer to a function */
void (*fp)(int*, int, int, ...);
/* by default, the pointer points to a placeholder at compile time*/
fp = &doNothing(...);
...
/* At the appropriate time, point the pointer to the userland function, whose address is supplied as an argument to something(): /*
fp= userFuncPtr;
/* Declare memory for the user function to plonk data into */
i_arr_coefficients = (int) malloc(SOMETHING_SENSIBLE);
/* Create a pointer to that array for the userland function */
i_ptr_array=&i_arr_coefficients[0];
/* define a struct of pointers to local variables for the userland function to use*/
ptrStrct=createPtrStruct();
/* Call the user's function: */
fp(i_ptr_array,ptrStrct, ...);
CarryOnWithSomethingElse();
The point of the placeholder function is to keep things ticking over if the user function isn't linked in. I get that this could be replaced with a #DEFINE, but the compiler's cleverness or stupidity might result in odd (to my ignorant mind, at least) behaviour.
In the userland function, we'd have something like:
void doUsefulThings(i_ptr_array, ptrStrct, localVariableAddresses, ...) {
double a=*ptrStrct.a;
double b=*ptrStrct.b;
double c=*localVariableAddresses.c;
double d=doMaths(a, b, c);
/* I.e. do maths using all of these numbers we've got from the different sources */
storeData(i_ptr_array, d);
/* And put the results of that maths where the C++ method can see it */
}
...
something(&doUsefulThings(i_ptr_array, ptrStrct, localVariableAddresses, ...), ...);
...
If this is as clear as mud please tell me! Thank you very much for your help. And, by the way, I sincerely wish someone would make an open hardware/source MRI system.
*As an aside, this is the primary justification the manufacturer uses to discourage us from modifying the big library in the first place!
You have full access to the C code. You have limited access to the C++ library code. The C code is defining the "doUsefullthings" function. From C code you are calling the "Something" function ( C++ class/function) with function pointer to "doUseFullThings" as the argument. Now the control goes to the C++ library. Here the various arguments are allocated memory and initialized. Then the the "doUseFullThings" is called with those arguments. Here the control transfers back to the C code. In short, the main program(C) calls the library(C++) and the library calls the C function.
One of the requirements is that the "userland function should have access to local variable from the C code where it is called". When you call "something" you are only giving the address of "doUseFullThings". There is no parameter/argument of "something" that captures the address of the local variables. So "doUseFullThings" does not have access to those variables.
malloc statement returns pointer. This has not been handled properly.( probably you were trying to give us overview ). You must be taking care to free this somewhere.
Since this is a mixture of C and C++ code, it is difficult to use RAII (taking care of allocated memory), Perfect forwarding ( avoid copying variables), Lambda functions ( to access local varibales) etc. Under the circumstances, your approach seems to be the way to go.

Changing what a function points to

I have been playing around with pointers and function pointers in c/c++. As you can get the adress of a function, can you change where a function call actually ends?
I tried getting the memory adress of a function, then writing a second functions adress to that location, but it gave me a access violation error.
Regards,
Function pointers are variables, just like ints and doubles. The address of a function is something different. It is the location of the beginning of the function in the .text section of the binary. You can assign the address of a function to a function pointer of the same type however the .text section is read only and therefore you can't modify it. Writing to the address of a function would attempt to overwrite the code at the beginning of the function and is therefore not allowed.
Note:
If you want to change, at runtime, where function calls end up you can create something called a vritual dispatch table, or vtable. This is a structure containing function pointers and is used in languages such as c++ for polymorphism.
e.g.:
struct VTable {
int (*foo)(void);
int (*bar)(int);
} vTbl;
At runtime you can change the values of vTbl.foo and vTbl.bar to point to different functions and any calls made to vTbl.foo() or .bar will be directed to the new functions.
If the function you're trying to call is inlined, then you're pretty much out of luck. However, if it's not inlined, then there may be a way:
On Unix systems there's a common feature of the dynamic linker called LD_PRELOAD which allows you to override functions in shared libraries with your own versions. See the question What is the LD_PRELOAD trick? for some discussion of this. If the function you're trying to hijack is not loaded from a shared library (i.e. if it's part of the executable or if it's coming from a statically linked library), you're probably out of luck.
On Windows, there are other attack vectors. If the function to be hooked is exported by some DLL, you could use Import Address Table Patching to hijack it without tinkering with the code of the function. If it's not exported by the DLL but you can get the address of it (i.e. by taking the address of a function) you could use something like the free (and highly recommended) N-CodeHook project.
In some environments, it is possible to "patch" the beginning instructions of a function to make the call go somewhere else. This is an unusual technique and is not used for normal programming. It is sometimes used if you have an existing compiled program and need to change how it interacts with the operating system.
Microsoft Detours is an example of a library that has the ability to this.
You can change what a function pointer points to, but you can't change a normal function nor can you change what the function contains.
You generally can't find where a function ends. There's no such standard functionality in the language and the compiler can optimize code in such ways that the function's code isn't contiguous and really has not a single point of end and in order to find where the code ends one would need to either use some non-standard tools or disassemble the code and make sense of it, which isn't something you can easily write a program for to do automatically.

c++ va_arg typecast issue

All,
I am writing a small c++ app and have been stumped by this issue. Is there a way to create (and later catch ) the error while accessing element from va_list macro using va_arg if element type is not expected. Eg:-
count=va_arg(argp,int);
if (count <= 0 || count > 30)
{
reportParamError(); return;
}
Now, if I am passing a typedef instead of int, I get garbage value on MS compiler but 95% of time count gets value 0 on gcc (on 64 bit sles10 sys). Is there a way I can enforce some typechecking, so that I get an error that can be caught in a catch block?
Any ideas on this would be very helpful to me. Or is there a better way to do this. The function prototype is:-
void process(App_Context * pActx, ...)
The function is called as
process(pAtctx,3,type1,type2,type3);
It is essential for pActx to be passed as 1st parameter and hence cannot pass count as 1st parameter.
Update-1
Ok, this sounds strange but nargs does not seem to part of va_list on sles10 gcc. I had to put in
#ifdef _WIN32
tempCount=va_arg(argp,int)
#endif
After using this, parameters following nargs do not get garbage values. However, this introduces compiler/platform based #ifdefs....Thanks Chris and Kristopher
If you know a count will always be passed as the second argument, then you could always change the signature to this:
void process(App_Context * pActx, int count, ...)
If that's not an option, then there is really no way to catch it. That's just how the variable-argument-list stuff works: there is no way for the callee to know what arguments are being passed, other than whatever information the caller passes.
If you look into how the va_arg macro and related macros are implemented, you may be able to figure out how to inspect all the stuff on the stack. However, this would not be portable, and it is not recommended except as a debugging aid.
You also might want to look into alternatives to variable-arguments, like function overloading, templates, or passing a vector or list of arguments.
No, there is no way. varargs doesn't provide any way to check the types of parameters passed in. You must only read them with the correct type which means that you need another way of communicating type information.
You are likely to be better off avoiding varargs functionality unless you really need it. It's only really a C++ feature for the sake of legacy functions such as printf and friends.

Can I programatically deduce the calling convention used by a C++ dll?

Imagine you'd like to write a program that tests functions in a c++ dll file.
You should enable the user to select a dll (we assume we are talking about c++ dlls).
He should be able to obtain a list of all functions exported by the dll.
Then, the user should be able to select a function name from the list, manually input a list of arguments ( the arguments are all basic types, like int, double, bool or char arrays (e.g. c-type strings) ) and attempt to run the selected function with the specified arguments.
He'd like to know if the function runs with the specified arguments, or do they cause it to crash ( because they don't match the signature for example ).
The main problem is that C++, being a strongly typed language, requires you to know the number and type of the arguments for a function call at compile time.And in my case, I simply don't know what these arguments are, until the user selects them at runtime.
The only solution I came up with, was to use assembly to manually push the arguments on the call stack.
However, I've come to understand that if I want to mess with assembly, I'd better make damn sure that I know which calling convention are the functions in the dll using.
So (finally:) here's my question: can I deduce the calling convention programmaticaly? Dependency Walker won't help me, and I've no idea how to manually read PE format.
The answer is maybe.
If the functions names are C++ decorated, then you can determine the argument count and types from the name decoration, this is your best case scenario, and fairly likely if MSVC was used to write the code in the first place.
If the exported functions are stdcall calling convention (the default for windows api), you can determine the number of bytes to be pushed, but not the types of the arguments.
The bad news is that for C calling convention, there isn't any way to tell by looking at the symbol names. You would need to have access to the source code or the debug info.
http://en.wikipedia.org/wiki/X86_calling_conventions
The name that a function is given as an export is not required to have any relationship with the name that the linker sees, but most of the time, the exported name and the symbol name that the linker sees are the same.
You didn't specify whether you're talking 32-bit or 64-bit here, and the difficulties outlined by you and the other posters mainly apply to 32-bit code. On 64-bit Windows, there's essentially only one calling convention (it's in also in the wikipedia article linked by John Knoeller), which means that you do know the calling convention (of course with the exception of anybody cooking up their own).
Also, with the Microsoft x64 calling convention, not knowing the number of parameters of the function to be called does not stop you from calling it, providing as many parameters as you wish/the user wishes to. This is because you as a caller set aside stack space and clean it up afterwards. -- Of course, not providing the right [number of] parameters may still have the called function do silly things because you're providing invalid input, but that's another story.
The compiled code does not just say 'Here this function is a fastcall, and this one here is stdcall' unfortunately.
Not even modern disassemblers like IDA try to deduce call types by default (there might be a plugin or an option somewhere idk).
Basically if you are a human you cn look at the first few instructions and tell 90% of the time. If they are pop and push, its stdcall, if its passing params through the registers (especially ecx) then its cdecl. Fastcall also uses the registers but does something special.. dunno off the top of my head. But all this info is useless because your program obviously will not be a human.
If you are doing testing, dont you at least have the header files?? This is an awfully hard way to skin a cat..
If you want to know what calling convention a C++ function uses, your best hope is to study
The header that declares that function, and
The documentation for the compiler that compiled your particular DLL.
But this whole thing sounds like a bit of a mess, honestly. Why does your friend want to be able to do this, and why can't he get the information he needs by parsing a header that declares the relevant functions?
This page describes the way VC++6 encodes parameter and calling convention info into a symbol name: http://www.bottledlight.com/docs/mangle.html
I suspect that later versions of VC++ will be compatible but I haven't confirmed this.
There are also some tools that automate this which accompany the compiler: http://msdn.microsoft.com/en-us/library/5x49w699.aspx
The name mangling only applies for C++ functions; if a function is 'extern "C"' then this won't work.

Does an arbitrary instruction pointer reside in a specific function?

I have a very difficult problem I'm trying to solve: Let's say I have an arbitrary instruction pointer. I need to find out if that instruction pointer resides in a specific function (let's call it "Foo").
One approach to this would be to try to find the start and ending bounds of the function and see if the IP resides in it. The starting bound is easy to find:
void *start = &Foo;
The problem is, I don't know how to get the ending address of the function (or how "long" the function is, in bytes of assembly).
Does anyone have any ideas how you would get the "length" of a function, or a completely different way of doing this?
Let's assume that there is no SEH or C++ exception handling in the function. Also note that I am on a win32 platform, and have full access to the win32 api.
This won't work. You're presuming functions are contigous in memory and that one address will map to one function. The optimizer has a lot of leeway here and can move code from functions around the image.
If you have PDB files, you can use something like the dbghelp or DIA API's to figure this out. For instance, SymFromAddr. There may be some ambiguity here as a single address can map to multiple functions.
I've seen code that tries to do this before with something like:
#pragma optimize("", off)
void Foo()
{
}
void FooEnd()
{
}
#pragma optimize("", on)
And then FooEnd-Foo was used to compute the length of function Foo. This approach is incredibly error prone and still makes a lot of assumptions about exactly how the code is generated.
Look at the *.map file which can optionally be generated by the linker when it links the program, or at the program's debug (*.pdb) file.
OK, I haven't done assembly in about 15 years. Back then, I didn't do very much. Also, it was 680x0 asm. BUT...
Don't you just need to put a label before and after the function, take their addresses, subtract them for the function length, and then just compare the IP? I've seen the former done. The latter seems obvious.
If you're doing this in C, look first for debugging support --- ChrisW is spot on with map files, but also see if your C compiler's standard library provides anything for this low-level stuff -- most compilers provide tools for analysing the stack etc., for instance, even though it's not standard. Otherwise, try just using inline assembly, or wrapping the C function with an assembly file and a empty wrapper function with those labels.
The most simple solution is maintaining a state variable:
volatile int FOO_is_running = 0;
int Foo( int par ){
FOO_is_running = 1;
/* do the work */
FOO_is_running = 0;
return 0;
}
Here's how I do it, but it's using gcc/gdb.
$ gdb ImageWithSymbols
gdb> info line * 0xYourEIPhere
Edit: Formatting is giving me fits. Time for another beer.