I'm writing a test suite for an assembler stub on multiple CPU architectures. The stub is meant to pass arguments to a C callback.
I have written tests to cover multiple scenarios (eg passing struct by value, mixing arguments of different native size, mixing float args with ints etc) and I would now like the test callback to do something that will use up lots of registers/stack slots etc.
The idea is to try and flush out implementations that are only working by fluke (eg value not correctly put on stack, but it happens to still be in a certain register so you get away with it etc).
Can anyone recommend a good piece of C/C++ I can use as the test here? I realise register use varies wildly across architectures and there's no guarantee at all that it will get complete coverage, but it would be nice to have something that gave a reasonable amount of confidence. Thanks
There is nothing in the C/C++ standards to help you here. Ultimately the only reliable way is exactly the way that compiler writers do it. They study the code their compiler generates and think up ways to break it.
Having said that, I can think of some strategies that might flush out some common problems.
Make calls with lots of different combinations of argument types and numbers. For example, a function with a single argument or return value that is a char/short/int/long/float/double/pointer etc will exercise a certain range of compiler code generation (and likely be passed in a register if possible). The same function with lots of arguments will use a different strategy, and (in most cases) not have enough registers.
Insert preamble code into the called function, so that passed in arguments are not used immediately but the registers get filled with other values.
Use variadic arguments. The calling conventions for variadic arguments (especially with separate compile and link) virtually guarantee arguments on the stack and not in registers.
Exercise lots of different call types: not just simple scalars but struct by value, pointer to function, pointer to member, etc.
Cheat. Call with one prototype but cast the function pointer so that the callee has a different prototype. For example, call with double on the stack but the callee function has two longs. Requires some inside knowledge of compiler workings.
The only pre-written code you will find to do this stuff is the compiler compliance suite for your chosen compiler.
Breaking compilers is fun. Hopefully something here will help you do it.
Related
Right now, I am trying to call a function in C++ through a Json object. The Json object would provide me with the name of the callee function and all the parameters. I will be able to extract the parameters using a for loop, but I am not sure how I can pass them in. For loop only allows me to pass arguments one by one, and I did not find a way to call a function besides passing in all the arguments at once.
I've made a temporary solution of:
if (parameter_count == 1)
func(param_1);
if (parameter_count == 2)
func(param_1, param_2);
...
This solution seems would not work for all cases since it can only work for functions with a limited number of arguments (depending on how many ifs I write). Is there a better way for this? Thanks!
EDIT: Sorry if I was being unclear. I do not know anything about func. I will be reading func from DLL based on its string name. Since I can't really change the function itself, I wouldn't be able to pass in a vector or struct directly.
Or perhaps did I have the wrong understanding? Are we allowed to pass in a single vector in place of a lot of parameters?
Sorry for making a mess through so many edits on this question. Brandon's solution with libffi works. Thanks!
So the problem as I understand it is that you have a void * pointer (which would come from your platform's DLL loading code) which "secretly" is a pointer to a function with a signature which is only known at runtime. You'd like to call this function at runtime with specified arguments.
Unfortunately, this is not possible to do cleanly with standard C++ alone. C++ cannot work with types that are not present in the program at compile-time, and since there is an infinite number of potential function signatures involved here there is no way to compile them all in.
What you'll want to do instead is manually set up the stack frame on your call stack and then jump to it, either via inline assembly or via some library or compiler extension that accomplishes this for your platform.
Here is a simple example of doing this via inline assembly. (To do this in general you will need to learn your platform's calling convention in detail, and needless to say this will constrain your program to the platform(s) you've implemented this for.)
I haven't actually tried it, but gcc has a compiler extension __builtin_apply that is apparently just meant to forward the arguments from one method wholesale to another but which could perhaps be used to accomplish something like this if you learned the (apparently opaque) description of the method.
[Update: Apparently I missed this in the comments, but Brandon mentioned libffi, a library which implements a bunch of platforms' calling conventions. This sounds like it might be the best option if you want to take this sort of approach.]
A final option would be to constrain the allowed signatures of your functions to a specified list, e.g. something like
switch(mySignature)
{
case VOID_VOID:
dynamic_cast<std::function<void(void)> *>(myPtr)();
break;
case VOID_INT:
dynamic_cast<std::function<void(int)> *>(myPtr)(my_int_arg_1);
break;
// ...
}
(Syntax of the above may not be 100% correct; I haven't tested it yet.) Whether this approach is sensible for your purposes depends on what you're doing.
I have two questions about templates in C++. Let's imagine I have written a simple List and now I want to use it in my program to store pointers to different object types (A*, B* ... ALot*). My colleague says that for each type there will be generated a dedicated piece of code, even though all pointers in fact have the same size.
If this is true, can somebody explain me why? For example in Java generics have the same purpose as templates for pointers in C++. Generics are only used for pre-compile type checking and are stripped down before compilation. And of course the same byte code is used for everything.
Second question is, will dedicated code be also generated for char and short (considering that they both have the same size and there are no specialization).
If this makes any difference, we are talking about embedded applications.
I have found a similar question, but it did not completely answer my question: Do C++ template classes duplicate code for each pointer type used?
Thanks a lot!
I have two questions about templates in C++. Let's imagine I have written a simple List and now I want to use it in my program to store pointers to different object types (A*, B* ... ALot*). My colleague says that for each type there will be generated a dedicated piece of code, even though all pointers in fact have the same size.
Yes, this is equivalent to having both functions written.
Some linkers will detect the identical functions, and eliminate them. Some libraries are aware that their linker doesn't have this feature, and factor out common code into a single implementation, leaving only a casting wrapper around the common code. Ie, a std::vector<T*> specialization may forward all work to a std::vector<void*> then do casting on the way out.
Now, comdat folding is delicate: it is relatively easy to make functions you think are identical, but end up not being the same, so two functions are generated. As a toy example, you could go off and print the typename via typeid(x).name(). Now each version of the function is distinct, and they cannot be eliminated.
In some cases, you might do something like this thinking that it is a run time property that differs, and hence identical code will be created, and the identical functions eliminated -- but a smart C++ compiler might figure out what you did, use the as-if rule and turn it into a compile-time check, and block not-really-identical functions from being treated as identical.
If this is true, can somebody explain me why? For example in Java generics have the same purpose as templates for pointers in C++. Generics are only used for per-compile type checking and are stripped down before compilation. And of course the same byte code is used for everything.
No, they aren't. Generics are roughly equivalent to the C++ technique of type erasure, such as what std::function<void()> does to store any callable object. In C++, type erasure is often done via templates, but not all uses of templates are type erasure!
The things that C++ does with templates that are not in essence type erasure are generally impossible to do with Java generics.
In C++, you can create a type erased container of pointers using templates, but std::vector doesn't do that -- it creates an actual container of pointers. The advantage to this is that all type checking on the std::vector is done at compile time, so there doesn't have to be any run time checks: a safe type-erased std::vector may require run time type checking and the associated overhead involved.
Second question is, will dedicated code be also generated for char and short (considering that they both have the same size and there are no specialization).
They are distinct types. I can write code that will behave differently with a char or short value. As an example:
std::cout << x << "\n";
with x being a short, this print an integer whose value is x -- with x being a char, this prints the character corresponding to x.
Now, almost all template code exists in header files, and is implicitly inline. While inline doesn't mean what most folk think it means, it does mean that the compiler can hoist the code into the calling context easily.
If this makes any difference, we are talking about embedded applications.
What really makes a difference is what your particular compiler and linker is, and what settings and flags they have active.
The answer is maybe. In general, each instantiation of a
template is a unique type, with a unique implementation, and
will result in a totally independent instance of the code.
Merging the instances is possible, but would be considered
"optimization" (under the "as if" rule), and this optimization
isn't wide spread.
With regards to comparisons with Java, there are several points
to keep in mind:
C++ uses value semantics by default. An std::vector, for
example, will actually insert copies. And whether you're
copying a short or a double does make a difference in the
generated code. In Java, short and double will be boxed,
and the generated code will clone a boxed instance in some way;
cloning doesn't require different code, since it calls a virtual
function of Object, but physically copying does.
C++ is far more powerful than Java. In particular, it allows
comparing things like the address of functions, and it requires
that the functions in different instantiations of templates have
different addresses. Usually, this is not an important point,
and I can easily imagine a compiler with an option which tells
it to ignore this point, and to merge instances which are
identical at the binary level. (I think VC++ has something like
this.)
Another issue is that the implementation of a template in C++
must be present in the header file. In Java, of course,
everything must be present, always, so this issue affects all
classes, not just template. This is, of course, one of the
reasons why Java is not appropriate for large applications. But
it means that you don't want any complicated functionality in a
template; doing so loses one of the major advantages of C++,
compared to Java (and many other languages). In fact, it's not
rare, when implementing complicated functionality in templates,
to have the template inherit from a non-template class which
does most of the implementation in terms of void*. While
implementing large blocks of code in terms of void* is never
fun, it does have the advantage of offering the best of both
worlds to the client: the implementation is hidden in compiled
files, invisible in any way, shape or manner to the client.
I'm experimenting with variable arguments in C++, using va_args. The idea is useful, and is indeed something I've used a lot in C# via the params functionality. One thing that frustrates me is the following excerpt regarding va_args, above:
Notice also that va_arg does not determine either whether the retrieved argument is the last argument passed to the function (or even if it is an element past the end of that list).
I find it hard to believe that there is no way to programmatically determine the number of variable arguments passed to the function from within that function itself. I would like to perform something like the following:
void fcn(int arg1 ...)
{
va_list argList;
va_start(argList, arg1);
int numRemainingParams = //function that returns number of remaining parameters
for (int i=0; i<numRemainingParams; ++i)
{
//do stuff with params
}
va_end(argList);
}
To reiterate, the documentation above suggests that va_arg doesn't determine whether the retrieved arg is the last in the list. But I feel this information must be accessible in some manner.
Is there a standard way of achieving this?
I find it hard to believe that there is no way to programmatically determine the number of variable arguments passed to the function from within that function itself.
Nonetheless, it is true. C/C++ do not put markers on the end of the argument list, so the called function really does not know how many arguments it is receiving. If you need to mark the end of the arguments, you must do so yourself by putting some kind of marker at the end of the list.
The called function also has no idea of the types or sizes of the arguments provided. That's why printf and friends force you to specify the precise datatype of the value to interpolate into the format string, and also why you can crash a program by calling printf with a bad format string.
Note that parameter passing is specified by the ABI for a particular platform, not by the C++/C standards. However, the ABI must allow the C++/C standards to be implementable. For example, an ABI might want to pass parameters in registers for efficiency, but it might not be possible to implement va_args easily in that case. So it's possible that arguments are also shadowed on the stack. In almost no case is the stack marked to show the end of the argument list, though, since the C++/C standards don't require this information to be made available, and it would therefore be unnecessary overhead.
The way variable arguments work in C and C++ is relatively simple: the arguments are just pushed on the stack and it is the callee's responsibility to somewhat figure out what arguments there are. There is nothing in the standard which provides a way to determine the number of arguments. As a result, the number of arguments are determined by some context information, e.g., the number of elements referenced in a format string.
Individual compilers may know how many elements there are but there is no standard interface to obtain this value.
What you could do instead, however, is to use variadic templates: you can determine very detailed information on the arguments being passed to the function. The interface looks different and it may be necessary to channel the arguments into some sort of data structure but on the upside it would also work with types you cannot pass using variable arguments.
No, there isn't. That's why variable arguments are not safe. They're a part of C, which lacks the expressiveness to achieve type safety for "convenient" variadic functions. You have to live with the fact that C contains constructions whose very correctness depends on values and not just on types. That's why it is an "unsafe language".
Don't use variable arguments in C++. It is a much stronger language that allows you to write equally convenient code that is safe.
No, there's no such way. If you have such a need, it's probably best to pack those function parameters in a std::vector or a similar collection which can be iterated.
The variable argument list is a very old concept inherited from the C history of C++. It dates back to the time where C programmers usually had the generated assembler code in mind.
At that time the compiler did not check at all if the data you passed to a function when calling it matched the data types the function expected to receive. It was the programmer's responsibility to do that right. If, for example, the caller called the function with a char and the function expected an int the program crashed, although the compiler didn't complain.
Today's type checking prevents these errors, but with a variable argument list you go back to those old concepts including all risks. So, don't use it if you can avoid it somehow.
The fact that this concept is several decades old is probably the reason that it feels wrong compared to modern concepts of safe code.
Suppose I have a dll with 2 functions.name of dll="dll1"
f1(int a, int b, int c);
f2(int a);
My program would take the function name ,the dll name and a "list" of parameters as input.
how would i call the appropriate function with its appropriate parameters.
i.e,
if input is
dll1
f1
list(5,8,9)
this would require me to call f1 with 3 parameters
if input was
dll1
f2
list(8)
it would require me to call f2 with one parameter
how would i call the function without knowing the number of parameters in advance.
further clarification:
how do I write code that will call any
function with all its arguments by building the argument list dynamically
using some other source of information
Since the generated code differs based on the number of parameters, you have two choices: you can write some code in assembly language to do the job (basically walk through the parameter list and push each on the stack before calling the function), or you can create something like an array of pointers to functions, one for each number of parameters you care about (e.g., 0 through 10). Most people find the latter a lot simpler to deal with (if only because it avoids using assembly language at all).
To solve the problem in general you need to know:
The calling conventions (those stdcall, cdecl, fastcall, thiscall (btw, the latter two can be combined in MSVC++), etc things) that govern how the functions receive their parameters (e.g. in special registers, on the stack, both), how they return values (same) and what they are allowed to trash (e.g. some registers).
Exact function prototypes.
You can find all this only in the symbol/debug information produced by the compiler and (likely to a lesser extent) the header file containing the prototypes for the functions in the DLL. There's one problem with the header file. If it doesn't specify the calling convention and the functions have been compiled with non-default calling conventions (via a compiler option), you have ambiguity to deal with. In either case you'll need to parse something.
If you don't have this information, the only option left is reverse engineering of the DLL and/or its user(s).
In order to correctly invoke an arbitrary function only knowing its prototype and calling convention at run time you need to construct code analogous to that produced by the compiler when calling this function when it's known at compile time. If you're solving the general problem, you'll need some assembly code here, not necessarily hand-written, run-time generated machine code is a good option.
Last but not least, you need some code to generate parameter values. This is most trivial with numeric types (ints, floats and the like) and arrays of them and most difficult with structures, unions and classes. Creating the latter on the fly may be at least as difficult as properly invoking functions. Don't forget that they may refer to other objects using pointers and references.
The general problem is solvable, but not cheaply. It's far easier to solve a few simple specific cases and maybe avoid the entire problem altogether by rewriting the functions to have less-variable parameters and only one calling convention OR by writing wrapper functions to do that.
You might want to check out the Named Parameter Idiom.
It uses method chaining to basically accomplish what you want.
It solves the problem where you know what a default set of arguments look like, but you only need to customize a few of them and not necessarily in the order they are declared.
If your clients know at compile-time, then can wrap it this way:
template<class Args...>
void CallFunctionPointer(void* pf, Args&&... args)
{
typedef void(*FunctionType)(Args...);
FunctionType* pf2 = (FunctionType*) pf;
(*pf2)(forward<Args>(args)...);
}
Note, if you pass the wrong number of paramters or the wrong type(s) of parameters behaviour is undefined.
Background:
In C/C++ you can cast a function pointer to any signature you want, however if you get it wrong behavior is undefined.
In your case there are two signatures you have mentioned:
void (*)(int)
and
void (*)(int, int, int)
When you load the function from the DLL it is your responsibility to make sure you cast it to the correct signature, with the correct number and types of parameters before you call it.
If you have control over the design of these functions, I would modify them to take a variable number of arguments. It the base type is always int, than just change the signature of all the functions to:
void (*)(int* begin, size_t n);
// begin points to an array of int of n elements
so that you can safely bind any of the functions to any number of arguments.
I've seen it used in programming (specifically in the C++ domain) and have no idea what it is. Presumably it is a design pattern, but I could be wrong. Can anyone give a good example of a thunk?
A thunk usually refers to a small piece of code that is called as a function, does some small thing, and then JUMPs to another location (usually a function) instead of returning to its caller. Assuming the JUMP target is a normal function, when it returns, it will return to the thunk's caller.
Thunks can be used to implement lots of useful things efficiently
protocol translation -- when calling from code that uses one calling convention to code that uses a different calling convention, a thunk can be used to translate the arguments appropriately. This only works if the return conventions are compatible, but that is often the case
virtual function handling -- when calling a virtual function of a multiply-inherited base class in C++, there needs to be a fix-up of the this pointer to get it to point to the right place. A thunk can do this.
dynamic closures -- when you build a dynamic closure, the closure function needs to be able to get at the context where it was created. A small thunk can be built (usually on the stack) which sets up the context info in some register(s) and then jumps to a static piece of code that implements the closure's function. The thunk here is effectively supplying one or more hidden extra arguments to the function that are not provided by the call site.
The word thunk has at least three related meanings in computer science. A "thunk" may be:
a piece of code to perform a delayed
computation (similar to a closure)
a feature of some virtual function
table implementations (similar to a
wrapper function)
a mapping of machine data from one
system-specific form to another,
usually for compatibility reasons
I have usually seen it used in the third context.
http://en.wikipedia.org/wiki/Thunk
The term thunk originally referred to the mechanism used by the Royal Radar Establishment implementation of pass-by-name in their Algol60 compiler. In general it refers to any way to induce dynamic behavior when referencing an apparently static object. The term was invented by Brian Wichmann, who when asked to explain pass-by-name said "Well you go out to load the value from memory and then suddenly - thunk - there you are evaluating an expression."
Thunks have been put in hardware (cf. KDF9, Burroughs mainframes). There are several ways to implement them in software, all very machine, language and compiler specific.
The term has come to be generalized beyond pass-by-name, to include any situation in which an apparently or nominally static data reference induces dynamic behavior. Related terms include "trampoline" and "future".
Some compilers for object-oriented languages such as C++ generate functions called "thunks" as an optimization of virtual function calls in the presence of multiple or virtual inheritance.
Taken from: http://en.wikipedia.org/wiki/Thunk#Thunks_in_object-oriented_programming
This question has already been asked on SO, see:
What is a 'thunk', as used in Scheme or in general?
From what I can tell, it's akin to a lambda statement, where you may not want to return the value until you need to evaluate it; or it can also be compared to a property getter which by design executes some code in order to return a value while yet having the interface form that comes across more like a variable, but also has polymorphic behavior that can be swapped out whether by inheritance or by swapping out the function pointer that would evaluate and return a value at runtime based on compile-time or environmental characteristics.
There's considerable variation in use. Almost universally, a thunk is a function that's (at least conceptually) unusually small and simple. It's usually some sort of adapter that gives you the correct interface to something or other (some data, another function, etc.) but is at least seen as doing little else.
It's almost like a form of syntactic sugar, except that (at least as usually used) syntactic sugar is supposed to make things look the way the human reader wants to see them, and a thunk is to make something look the way the compiler wants to see it.
I was distressed to find no general 'computer science' definition of this term matching its de-facto usage as known historically to me. The first real-life encounter I can recall where it was actually called that was in the OS/2 days and the 16-32 bit transition. It appears "thunking" is like irony in its application today.
My rough general understanding is that the thunk is a stub routine that just does nothing or routes across some fundamental boundary in kind between systems as in the mentioned historical cases.
So the sense is like a synesthesia of being dropped from the one environment to the other making (metaphorically/as a simile) a "thunk" sound.
I'm going to look this up, but I thought thunking was the process employed by a 32-bit processor to run legacy 16-bit code.
I used to use it as an analogy for how you have to restrict how fast you talk and what words you use when talking to dumb people.
Yeah, it's in the Wikipedia link (the part about 32-bit, not my nerdalogy).
https://en.wikipedia.org/wiki/Thunk
Much of the literature on interoperability thunks relates to various Wintel platforms, including MS-DOS, OS/2,[8]Windows[9][10] and .NET, and to the transition from 16-bit to 32-bit memory addressing. As customers have migrated from one platform to another, thunks have been essential to support legacy software written for the older platforms.
(emphasis added by me)
The earliest use of "thunk" I know of is from late '50s in reference to Algol60 pass-by-name argument evaluation in function calls. Algol was originally a specification language, not a programming language, and there was some question about how pass-by-name could be implemented on a computer.
The solution was to pass the entry point of what was essentially a lambda. When the callee evaluated the parameter, control fell through - thunk! - into the caller's context where the lambda was evaluated and it's result became the value of the parameter in the callee.
In tagged hardware, such as the Burroughs machines, the evaluation was implicit: an argument could be passed as a data value as in ordinary pass-by-value, or by thunk for pass-by-name, with different tags in the argument metadata. A load operation hardware checked the tag and either returned the simple value or automatically invoked the lambda thunk.
Per Kyle Simpson's definition, a thunk is a way to abstract the component of time out of asynchronous code.
An earlier version of The New Hacker's Dictionary claimed a thunk is a function that takes no arguments, and that it was a simple late-night solution to a particularly tricky problem, with "thunk" being the supposed past tense of "think", because they ought to have thunk of it long time ago.
In OCaml, it’s a function which takes unit “()” as a param (takes no params, often used for side effects)