Writing Intel x86 Assembly within C++ wrapper on macOS [duplicate] - c++

This question already has answers here:
What is the difference between 'asm', '__asm' and '__asm__'?
(4 answers)
Why can't local variable be used in GNU C basic inline asm statements?
(4 answers)
Is there any way to complie a microsoft style inline-assembly code on a linux platform?
(3 answers)
Closed 10 days ago.
I'm trying to write Intel x86 Assembly code on macOS similarly to the following screenshot:
This screenshot is taken from my professor's Visual Studio application, where he writes assembly. He's also stated that this wouldn't work on a Mac. To do this, I need the C++ Console app on Visual Studio, which the Mac version does not offer.
I've tried to circumvent this issue by trying to compile it on Visual Studio Code on my Mac (M2 Chip), and also following this reference.
My current code snippet looks like this:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int num = 10;
asm(
"mov eax, num\n"
"add eax, 12\n"
"mov num, eax"
);
return 0;
}
However, upon running this through g++ test.cpp, I receive the following errors:
"mov eax, num\n"
^
<inline asm>:1:6: note: instantiated into assembly here
mov eax, num
^
test.cpp:10:11: error: invalid operand for instruction
"add eax, 12\n"
^
<inline asm>:2:5: note: instantiated into assembly here
add eax, 12
^
test.cpp:11:11: error: invalid operand for instruction
"mov num, eax"
^
<inline asm>:3:5: note: instantiated into assembly here
mov num, eax
^
Any ideas on how to solve this issue? (Note: not sure if this is relevant to the problem but I've also installed nasm through brew install nasm)

Related

How to copy a value of register CR3

I am confused. Take a look at code snippet below.
#pragma warning(2:4235)
int main()
{
uint32_t cr3;
__asm(
"mov %%cr3, %%eax\n\t"
"mov %%eax, %2\n\t"
:"=m" (cr3)
: /* no input */
: "%eax"
);
}
Compiled I see error
Error C2059 syntax error: : va2pa C:\Users\usr\source\repos\1.cpp 80
How do I copy a value of CR3 into variable cr3?
MSVC has intrinsics for everything so they can write a 64-bit kernel without inline asm. (Because MSVC doesn't support inline asm in 64-bit mode).
Use __readcr3(). docs. The intrinsic apparently works when compiling for 32 or 64-bit mode.
GNU C inline asm syntax isn't supported in MSVC. In Visual Studio, you could use clang instead of MSVC; clang supports GNU extensions.
If you are doing that, just use an "=r" output so you can mov directly to the output operand; if the first or last instruction in your asm template is a normal mov (not to/from a control reg), you're usually doing it wrong. Use better constraints to tell the compiler where to put or find data.
You could of course use MSVC inline asm syntax for this, but don't. It's inefficient and a dead-end for portability (to other compilers, and to x86-64).

GCC: Specifying mmx register for a local variable fails

Im trying to achieve the following on a Windows environment c++, MINGW GCC:
Im trying to define a local variable as such:
register DWORD index __asm ("%mm7");
this definition compiles with MINGW GCC x64 , but fails on x86 compilation with the error
error: invalid register name for 'index'
register DWORD index __asm ("%mm7");
I have already tried to reference the register as __asm ("mm7"); but to no avail. any help on the matter would be very appreciated

How to add RDRAND instruction into 64-bit code compiled with VS 2008?

I'm working on a C++ project in Visual Studio 2008 IDE, where I need to use Intel's new RDRAND instruction. I did a quick search, and MSDN recommends using _rdrand64_step intrinsic defined in immintrin.h, which I do not have in VS 2008.
In a 32-bit compiled code I can get away with using asm keyword as such:
__asm
{
xor eax, eax
;RDRAND instruction = Set random value into EAX.
;Will set overflow [C] flag if success
_emit 0x0F
_emit 0xC7
_emit 0xF0
}
But on x64 asm is not supported.
Can you suggest how can I compile my project for 64-bit with the RDRAND instruction?
You either need to upgrade your compiler to one that does support the _rdrand64_step intrinsic (supported since Visual Studio 2012), or use normal (external) assembly to create your own functions (since Visual C++ does not support inline assembly for x86-64 targets).
For example:
_TEXT SEGMENT
PUBLIC rdrand32_step
PUBLIC rdrand32_retry
PUBLIC rdrand64_step
PUBLIC rdrand64_retry
; int rdrand32_step(unsigned *p)
rdrand32_step PROC
xor eax, eax
rdrand edx
; DB 0fh, 0c7h, 0f2h
setc al
mov [rcx], edx
ret
rdrand32_step ENDP
; unsigned rdrand32_retry()
rdrand32_retry PROC
retry:
rdrand eax
; DB 0fh, 0c7h, 0f0h
jnc retry
ret
rdrand32_retry ENDP
; int rdrand64_step(unsigned long long *p)
rdrand64_step PROC
xor eax, eax
rdrand rdx
; DB 048h, 0fh, 0c7h, 0f2h
setc al
mov [rcx], edx
ret
rdrand64_step ENDP
; unsigned long long rdrand64_retry()
rdrand64_retry PROC
retry:
rdrand rax
; DB 048h, 0fh, 0c7h, 0f0h
jnc retry
ret
rdrand64_retry ENDP
_TEXT ENDS
END
If you're using the version of MASM from Visual Studio 2008, you'll probably have to comment out the RDRAND instructions and uncomment the DB directives that follow them.
Wow, it took me a while to figure it out. Here's the steps for Visual Studio 2008 for x64 compilation only:
(A) Create a blank project: File -> New -> Project. Then click on "Visual C++" and select "Empty Project." Name it something, and click OK to create.
(B) Go to your VS installation folder, in my case it was C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\VCProjectDefaults and copy masm.rules file and name it masm64.rules
(C) Open masm64.rules in Notepad and search for Microsoft Macro Assembler and change it to x64 Microsoft Macro Assembler. There will be two places to do it. Then search for ml.exe and change it to ml64.exe. Then save that file and close Notepad.
(D) Right-click your project in "Solution Explorer" and select "Custom build rules" and check x64 Microsoft Macro Assembler and click OK.
(E) Right-click your project in "Solution Explorer" and select Add -> New Item, select Text File (.txt) and name it something with .asm extension. I'll call it funcs_asm_x64.asm. Then click OK.
(F) Open funcs_asm_x64.asm and type your x64 asm. For me I was interested in calling RDRAND with a 64-bit operand. I did the following. This function will take one parameter as a pointer to a 64-bit integer that it will fill out with random bits. It will return 1 in rax if success, otherwise it will return 0.
One thing to remember here is that x64 code uses only __fastcall calling convention, meaning that first 4 parameters for a function are passed in registers: RCX, RDX, R8, and R9:
.code
RdRand64 PROC
; RCX = pointer to receive random 64-bit value
; RETURN: [RAX] = 1 if success, 0 if failed
xor rax, rax
test rcx, rcx
jz lbl_out
;push rdx
xor rdx, rdx
DB 048h, 0fh, 0c7h, 0f2h ;RDRAND RDX
setc al
mov [rcx], rdx
;pop rdx
lbl_out:
ret
RdRand64 ENDP
END
(G) Then right-click your project in "Solution Explorer" and select Add -> New Item, select C++ File (.cpp) and name it main.cpp and click OK to create. Then add the following to the main.cpp file:
extern "C" __int64 __fastcall RdRand64(unsigned __int64* pRndVal);
void main()
{
}
The main part is the extern "C" definition. main() method is needed to satisfy MASM requirements.
(H) Then go to Build -> Configuration Manager and open the drop-down list where it says "Active solution platform" and select New. Then pick "x64" in "Type or select the new platform" and click OK. Then select "x64" as "Active solution platform" and also select "Release" in "Active solution configuration."
(I) Close configuration manager window and build solution. If it succeeded look for funcs_asm_x64.obj file in \x64\Release folder for your solution. Copy that file into your main solution folder (where you needed to use RDRAND instruction.)
(J) Then in your main solution where you need to use RDRAND instruction, right-click on your project in "Solution Explorer" and go to Properties. Then go to Linker -> Command Line and add your obj file name. Obviously do so only for x64 platform for Debug and Release. In my case it was funcs_asm_x64.obj. Click OK to save.
(K) Then to use this function that I just created, first add the extern "C" definition just like you had in the first project:
extern "C" __int64 __fastcall RdRand64(unsigned __int64* pRndVal);
and then you can call it as such (obviously it cannot be inlined):
unsigned __int64 randomNumber = 0;
__int64 bResult = RdRand64(&randomNumber);
(1) Obviously all of the above is not necessary for the Win32 or x86 build. For that simply use the inline assembly like I showed in my original post.
(2) Also obviously you will need to call __cpuid command to ensure that RDRAND instruction is supported. On many CPUs it is still not. So if it is not, then don't call my RdRand64 method, as it will crash! You can use this code to check and store result somewhere in a global variable:
#include <intrin.h>
bool is_RDRAND_supported()
{
int name[4] = {0};
__cpuid(name, 0);
if(name[1] == 0x756e6547 && //uneG
name[2] == 0x6c65746e && //letn
name[3] == 0x49656e69) //Ieni
{
int data[4] = {0};
__cpuid(data, 1);
//Check bit 30 on the 2nd index (ECX register)
if(data[2] & (0x1 << 30))
{
//Supported!
return true;
}
}
return false;
}
(3) There is a way to include the asm file in the same project in VS 2008. Unfortunately if you do that you won't be able to switch the project back to Win32 and compile if you need to. So if you're compiling it only for x64 then save a step and do all of it in the same solution.
It's fairly easy, although indirect: Create a tiny C wrapper for _rdrand64_step, compile it into an .OBJ file using VS2012 with no fancy options (No /LTCG, no /Gs etc), and link this object file as-is into your VS2008 project. The VS2008 compiler may not know the instruction, but the VS2008 linker doesn't care.

"Error: bad register name `%rax'" while compiling with MinGW on 64-bit Windows 7

In my previous topic: How to read registers: RAX, RBX, RCX, RDX, RSP. RBP, RSI, RDI in C or C++? I asked about reading those registers. Now I wrote a code to read (just for now on) RAX and RBX.
I'm using CodeBlocks pm 64-bit Windows 7 with MinGW as a compiler and I'm working on an x86-64 CPU. When I tried to compile the below code, I got those errors:
Error: bad register name `%rax'
Error: bad register name `%rbx'
And the code:
#include <iostream>
#include <cstdlib>
#include <stdint.h>
void read(void)
{
uint64_t rax = 0, rbx = 0;
__asm__ __volatile__ (
/* read value from rbx into rbx */
"movq %%rbx, %0;\n"
/* read value from rax into rax*/
"movq %%rax, %1;\n"
/* output args */
: "=r" (rbx), "=r" (rax)
: /* no input */
/* clear both rdx and rax */
: "%rbx", "%rax"
);
/* print out registers content */
std::cout << "RAX = " << rax << "\n";
std::cout << "RBX = " << rbx << "\n";
}
int main(int argc, char **argv)
{
read();
return 0;
}
I suspect your error comes from the "clobber" line:
: "%rbx", "%rax"
which should read:
: "rbx", "rax"
(Oh, and don't ask me why!)
Edit: You will also need to compile for 64-bit, using -m64 or similar - assuming of course the MingW compiler you have is 64-bit capable in the first place.
MinGW is a 32-bit compiler. It doesn't support the x86-64 target so you can't use 64-bit registers like RAX. At least that's what happened before at the time of this question. I'm not sure about the situation of mingw right now but for quite a long time it didn't support producing 64-bit binaries. Besides, it's broken in many ways because it uses the ancient MSVCRT runtime library which doesn't support standard C99 printf format specifiers or 80-bit long double. The developers were also reluctant to update, or respond to (pull) requests, therefore mingw-w64 was forked which provided a much more complete solution for Windows. It solved all the printf issues, have more complete support for POSIX and Win32 APIs and many more things that you can read later
If the current version of mingw does support x86-64 then you still need to pass the -m64 option to make it output a 64-bit executable, because mingw is still a 32-bit compiler as I looked over their installer and will compile into 32-bit code by default where RAX is non-existent. See also
MinGW-w32 vs. MinGW
What is the difference between MinGW, MinGW-w64 and MinGW-builds?
https://en.wikipedia.org/wiki/MinGW
But the better way would be moving to better compilers such as mingw-w64, TDM-GCC or Clang which support both 64-bit guest and host natively
In 2005, MinGW-w64 was created by OneVision Software under clean room design principles, since the original MinGW project was not prompt on updating its code base, including the inclusion of several key new APIs and the much needed 64-bit support. In 2008, OneVision then donated the code to Kai Tietz, one of its lead developers, under the condition that it remain open source. It was first submitted to the original MinGW project, but refused under suspicion of using non-public or proprietary information. For many reasons, the lead developer and co-founder of the MinGW-w64 project, Kai Tietz, decided not to attempt further cooperation with MinGW.
MinGW-w64 provides a more complete Win32 API implementation, including:
better C99 support
better POSIX Threads (pthreads) support (including the possibility to enable C++11 thread-related functionality in GCC's libstdc++)
GCC multilib, which allows users to install 32-bit and 64-bit libraries in parallel
Unicode entry point (wmain/wWinMain)
DDK (from ReactOS)
DirectX (from Wine)
Large file support
Win64 support
Some useful tools such as gendef (an improved version of MinGW's pexports utility), and widl (an IDL compiler, a free replacement for MIDL).
https://en.wikipedia.org/wiki/MinGW#MinGW-w64

Visual Studio 2010 compiling inline assembly in c++ as if Visual Studio 6?

I have a C++ project created in VS6 that was opened in VS2010 and compiles fine. It contains a class with a bit of inline assembly including the following code:
__asm{
mov eax,this
mov esi,[eax].m_pImage
...
All fine and dandy, until I try and create a new MFC C++ project in VS2010 and add the class with the assembly code shown above. Suddenly, this will not compile as the newer interpretation of __asm requires the code to be as follows (or something similar; this compiles in any case):
__asm{
mov eax,this
mov esi,[eax]this.m_pImage
...
Now, for the life of me, I can not figure out what the difference is between the two projects which allows the converted VS6 project to compile the (presumable currently invalid) inline assembly code in VS2010, while a newly created project can't.
Is there somewhere a (hidden) setting which allows one to use the old VS6 compiler?
The VS6 inline ASM seems like a bug that was fixed. I say that because I am not sure how the compiler could verify that m_pImage was a member of what was loaded in the eax register and therefore could not find the offset. To answer your question, there is no way, I am aware of, to use the old ASM semantics in the VS6 compiler.
I would make a local variable outside of the inline ASM and assign that to esi instead.
void * pointer = this.m_pImage;
__asm
{
mov ebx, pointer
mov esi, ebx
...