use of __cdecl and whose responsibility is to clean the stack - c++

I am confused about the stack cleaning after knowing about the __cdecl.
Read that __cdecl will pop the stack arguments (https://msdn.microsoft.com/en-us/library/zkwh89ks.aspx)
So do we need explicitly mention in function declaration to delete the stack arguments or is it compiler responsibility to generate the appropriate code to clean the stack
Thanks in advnace

The compiler does the work, as long as the header other people #include to use your library correctly declares the function. If you fail to provide a header with an appropriate declaration, they need to declare the function themselves with the appropriate calling convention, or it will use whatever the default argument passing convention happens to be for their project.

Unless you are writing assembly, you don't need to write anything to clean up the stack. The compiler will generate the required code.

Read that __cdecl will pop the stack arguments
You are misreading the documentation that you have linked to. What it actually says is:
Calling function pops the arguments from the stack.
In cdecl, only the caller knows what arguments have been pushed on the stack (this is the only way that variadic parameters can be supported) so only the caller knows how to clean the stack correctly. As long as you are not writing assembly code, and that your function is explicitly declared as __cdecl or your project's default calling convention is __cdecl, then you don't have to worry about this. The compiler will generate the correct code. Personally, I don't suggest relying on the default project setting if you need to share this function across multiple projects, or with other people. Better to declare the calling convention explicitly to be safe.

Related

How function declared __declspec(naked) stores local variables?

__declspec(naked) void printfive() {
int i = 5;
printf("%i\n", i);
}
for some reason this code works, but I do not understand where the i is stored? In the frame of the calling function? It becomes global variable? If it is stored in the caller's frame, then how compiler knows the displacement, because you can call printfive() from different functions with different frame size and local variables. If it is global, or something like static maybe, I have tried to go recursive and I can see that variable is not changed, it is not truly local indeed. But that's obvious, there is no entry code (prolog). Ok, I understand, no prolog, no frame, no register change, but this is values, but what happens to the scope? Is the behaviour of this specifier defined in any reference? Is this part of C++ standard anyway? This sort of functions are great if you mostly use asm {} inside them, (or use asm to call them and want to be sure the function is not over optimized), but you can mix with C++. But this is sort of brain twister.
I know this topic is more than several years old, Here are my own answers.
Since no references are made to Microsoft documentation regarding this topic, for those who care to learn more about what is needed or not as stated by Keltar, here is the Microsoft documentation that explains most of what Keltar didn't explain here.
According to Microsoft documentation, and should be avoided.
The following rules and limitations apply to naked functions:
The return statement is not permitted.
Structured Exception Handling and C++ Exception Handling constructs are not permitted because they must unwind across the stack frame.
For the same reason, any form of setjmp is prohibited
Use of the _alloca function is prohibited.
To ensure that no initialization code for local variables appears before the prolog sequence, initialized local variables are not
permitted at function scope. In particular, the declaration of C++
objects are not permitted at function scope. There may, however, be
initialized data in a nested scope.
Frame pointer optimization (the /Oy compiler option) is not recommended, but it is automatically suppressed for a naked function.
You cannot declare C++ class objects at the function lexical scope. You can, however, declare objects in a nested block.
From gcc manual:
Use this attribute ... to indicate that the specified function does
not need prologue/epilogue sequences generated by the compiler. It is
up to the programmer to provide these sequences. The only statements
that can be safely included in naked functions are asm statements that
do not have operands. All other statements, including declarations of
local variables, if statements, and so forth, should be avoided. Naked
functions should be used to implement the body of an assembly
function, while allowing the compiler to construct the requisite
function declaration for the assembler.
And it isn't standard (as well as any __declspec or __attribute__)
When entering or exiting a function, the compiler adds code to help with the passing or parameters. When a function is declared naked, non of the parameter variables assignment code is generated, if you want to get at any of the parameters, you will need to directly access the relevant registers or stack (depending on the ABI defined calling convention).
In your case, you are not passing parameters to the function, therefore your code works even though the function is declared naked. Take a look at the dis-assembler if you want to see the difference.

What is the real signature of a C++ constructor after compilation

What is the real signature of a constructor in C++?
Background:
I try to hook internal functions of a dll. I have the pdb-Debug Files and were able to get the exact location of the function, including it's undecorated name.
The function looks like:
protected: __cdecl ClassName::ClassName(bool * __ptr64) __ptr64
So it is obviously a constructor.
I tried void function(void * pThisPointer, bool * pBoolPointer), but as the program crashes after simply forwarding the call (other functions work fine this way), I assume that my signature is wrong.
Do you know what signature to use for a constructor (which is most likely not virtual)?
Or do you have any other ideas what may go wrong?
EDIT:
I use x64 and the visual studio 2010 compiler, the target compiler should be something like the visual studio compiler, as it is a microsoft dll.
You cannot call a constructor directly. As far as C++ is concerned, constructors do not have a name. In other words, you cannot call constructors. Many compilers create two or even three different functions. Which one should you call?
The answer is none of them. You cannot and must not try to call a constructor directly from your code.
The easiest way is to look at the disassembly of the actual constructor and see what it accesses. The usual convention with MSVC (and other compilers) is to pass this as a hidden first parameter. Usually it's done with __thiscall convention (i.e. in ecx on x86), but on x64 there is only one convention, so __thiscall is the same as __cdecl or __stdcall.
However, the other not very well-known convention of MSVC is that constructors must return the this pointer. I suspect that's what's causing your crash.
It's probably not very relevant here, but have a look at my article on MSVC C++ internals. It describes x86 implementation, but a lot of things will apply
Since you can't legally take a function pointer to a constructor, it doesn't really have a meaningful signature, as far as C++ itself is concerned.
Implementations do their own thing, but typically a class can have at least two constructor function bodies in the emitted code - one for normal use, and another for when it's used as a virtual base. The compiler knows how to use each one -- you need to know exactly what parts of object construction are done in this emitted code in your implementation, what's done by the caller prior to calling it, what's done by the caller after it returns, and what's done by the caller if it throws an exception, or you can't correctly call it even given that you've figured out its address from debug info.
To reverse-engineer object construction, you should look at the code emitted at a place where an object is created using this constructor.

Should I prefer "default" calling convention over __fastcall when I don't really care of the calling convention?

We have a huge C++ codebase with lots of COM objects. Each function exposed to COM must have __stdcall calling convention (usually STDMETHODCALLTYPE macro) and so we have lots of functions marked STDMETHODCALLTYPE.
Now I see a function that is not directly called through COM, but rather called only from within our C++ code and this function also has STDMETHODCALLTYPE macro in its signature. I'm completely sure that macro is useless there - no calls through COM to that function ever happen.
Should I drop the __stdcall so that it becomes a "default" calling convention function? How do I make such decisions?
My approach is to use the default compiler calling convention for internal code and to use a well-defined explicitly stated calling convention for any methods which are exported across a module boundary.
The default calling convention for most compilers makes good use of registers for performance reasons so there are advantages to using it where appropriate. It also makes your code easier on the eye since you don't need to specify the convention to get the default.
For exported functions you clearly need to specify the convention. If you are making a library that you anticipate will be called from languages other than C or C++ it would be conventional to use stdcall. If you only expect C or C++ clients then cdecl is probably the most common convention.
When Windows switched from __cdecl to __stdcall as the default calling convention, the size of the product dropped by about 10%. That savings was entirely related to removing the stack adjustments after calling the stdcall methods (__cdecl is a "caller adjusts the stack to remove parameters" calling convention, __stdcall is a "callee adjusts the stack to remove parameters" calling convention, since there are more callers than callees, switching reduces the size of your binaries).
The downside of using __stdcall is that you don't have variable #s of argments (since the callee adjusts the stack, they can't know how many parameters the caller specified).
Bottom line: switching to __stdcall from the "default" calling convention can result in a reduction in size of your binary. That may or may not be important to you.
However as mkaes mentioned above, if your code is EVER accessed in another compiland (for instance if you deliver a .lib file to someone else), it's absolutely critical that you declare the calling convention used.
The only reason why the COM stuff explicitly sets the calling convention is because it is used across DLL boundaries.
So my advise would be to drop the explicit setting of the calling convention and set it by the compiler settings.
In general:
If the functions are exported as a DLL set a macro that defines the calling convention in the Headers. This prevents users from the DLL to use a wrong calling convention when linking to your DLL. Explicit overrides the compiler setting.
Do not use any calling convection on local functions. Convention can be set by a compiler switch. If you decide to set one explicitly, do it on all Functions. Then you still have a central place to change the calling convention.
Of course if it makes sense or you need some special calling convention e.g fastcall for optimization then you need to set explicitly too.
Do you have whole program optimization and link-time code generation enabled? If so, and you don't export the function from your DLL or pass around pointers to it, then the compiler may generate custom calling conventions for that function or inline it (even if it's not defined in a header file).
You could look through your maps to see if it is referenced, by searching the ODL files associated with the solution. If its not there, it doesn't have an interface, and you can change the calling convention. There is the risk that someone else assumes all functions are set up with this calling convention, and they could add an interface at a later date.

In C++, do variadic functions (those with ... at the end of the parameter list) necessarily follow the __cdecl calling convention?

I know that __stdcall functions can't have ellipses, but I want to be sure there are no platforms that support the stdarg.h functions for calling conventions other than __cdecl or __stdcall.
The calling convention has to be one where the caller clears the arguments from the stack (because the callee doesn't know what will be passed).
That doesn't necessarily correspond to what Microsoft calls "__cdecl" though. Just for example, on a SPARC, it'll normally pass the arguments in registers, because that's how the SPARC is designed to work -- its registers basically act as a call stack that gets spilled to main memory if the calls get deep enough that they won't fit into register anymore.
Though I'm less certain about it, I'd expect roughly the same on IA64 (Itanium) -- it also has a huge register set (a couple hundred if memory serves). If I'm not mistaken, it's a bit more permissive about how you use the registers, but I'd expect it to be used similarly at least a lot of the time.
Why does this matter to you? The point of using stdarg.h and its macros is to hide differences in calling convention from your code, so it can work with variable arguments portably.
Edit, based on comments: Okay, now I understand what you're doing (at least enough to improve the answer). Given that you already (apparently) have code to handle the variations in the default ABI, things are simpler. That only leaves the question of whether variadic functions always use the "default ABI", whatever that happens to be for the platform at hand. With "stdcall" and "default" as the only options, I think the answer to that is yes. Just for example, on Windows, wsprintf and wprintf break the rule of thumb, and uses cdecl calling convention instead of stdcall.
The most definitive way that you can determine this is to analyze the calling conventions. For variadic functions to work, your calling convention needs a couple of attributes:
The callee must be able to access the parameters that aren't part of the variable argument list from a fixed offset from the top of the stack. This requires that the compiler push the parameters onto the stack from right to left. (This includes such things as the first parameter to printf, the format specification. Also, the address of the variable argument list itself must also be derived from a known location.)
The caller must be responsible for removing the parameters off the stack once the function has returned, because only the compiler, while generating the code for the caller, knows how many parameters were pushed onto the stack in the first place. The variadic function itself does not have this information.
stdcall won't work because the callee is responsible for popping parameters off the stack. In the old 16-bit Windows days, pascal wouldn't work because it pushed parameters onto the stack from left to right.
Of course, as the other answers have alluded to, many platforms don't give you any choice in terms of calling convention, making this question irrelevant for those ones.
Consider the following function on an x86 system:
void __stdcall something(char *, ...);
The function declares itself as __stdcall, which is a callee-clean convention. But a variadic function cannot be callee-clean since the callee does not know how many parameters were passed, so it doesn’t know how many it should clean.
The Microsoft Visual Studio C/C++ compiler resolves this conflict by silently converting the calling convention to __cdecl, which is the only supported variadic calling convention for functions that do not take a hidden this parameter.
Why does this conversion take place silently rather than generating a warning or error?
My guess is that it’s to make the compiler options /Gr (set default calling convention to __fastcall) and /Gz (set default calling convention to __stdcall) less annoying.
Automatic conversion of variadic functions to __cdecl means that you can just add the /Gr or /Gz command line switch to your compiler options, and everything will still compile and run (just with the new calling convention).
Another way of looking at this is not by thinking of the compiler as converting variadic __stdcall to __cdecl but rather by simply saying “for variadic functions, __stdcall is caller-clean.”
click here
AFAIK, the diversity of calling conventions is unique to DOS/Windows on x86. Most other platforms had compilers come with the OS and standardize the convention.
Do you mean 'platforms supported by MSVC" or as a general rule? Even if you confine yourself to the platforms supported by MSVC, you still have situations like IA64 and AMD64 where there is only "one" calling convention, and that calling convention is called __stdcall, but it's certainly not the same __stdcall you get on x86.

Switching callstack for C++ functions

Here's my previous question about switching C callstacks. However, C++ uses a different calling convention (thiscall) and may require some different asm code. Can someone explain the differences and point to or supply some code snippets that switch C++ callstacks (preferably in GCC inline asm)?
Thanks,
James
The code given in the previous question should work fine.
The thiscall calling convention differs only in who is responsible for popping the arguments off the stack. Under the thiscall calling convention, the callee pops the arguments (and additionally, the this pointer is passed in ecx); under the C calling convention, the caller pops the arguments. This does not affect context switches.
However, if you're going to do context switches yourself, note that you need to save and restore the registers as well (probably on the stack) in addition to switching stacks.
Note, by the way, that C++ doesn't always use thiscall -- it's only used for methods with a fixed number of arguments (and apart from that, it's a Microsoftism... g++ doesn't use it).
Note the ABI for C++ is not explicitly defined.
The idea was that compiler manufactures are able to use the optimal calling convention for the situation and thus make C++ faster.
The down side of this is that each compiler has its own calling convention thus code from different compilers are not compatable (even code form different versions (or even different optimization flags) of the same compiler can be incompatable).