Finding opcode assembly instruction [closed] - c++

Question

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 6 years ago.
Improve this question
I have been giving a metamorphic engine a try. I started by trying to analyze the opcode assembly instruction but it does not seem to give me anything. The instruction I am looking for in the function is MOV. Why does it not return anything even though they are in the function?
#include <iostream>
#include <Windows.h>
using namespace std;
struct OPCODE
{
unsigned short usSize;
PBYTE pbOpCode;
bool bRelative;
bool bMutated;
};
namespace MOVRegisters
{
enum MovRegisters
{
EAX = 0xB8,
ECX,
EDX,
EBX,
ESP,
EBP,
ESI,
EDI
};
}
bool __fastcall bIsMOV(PBYTE pInstruction)
{
if (*pInstruction == MOVRegisters::EAX || *pInstruction == MOVRegisters::ECX || *pInstruction == MOVRegisters::EDX || *pInstruction == MOVRegisters::EBX ||
*pInstruction == MOVRegisters::ESP || *pInstruction == MOVRegisters::EBP || *pInstruction == MOVRegisters::ESI || *pInstruction == MOVRegisters::EDI)
return true;
else
return false;
}
void pCheckByte(PVOID pFunction, PBYTE pFirstFive)
{
if (*pFirstFive == 0x0)
memcpy(pFirstFive, pFunction, 5);
else
memcpy(pFunction, pFirstFive, 5);
PBYTE pCurrentByte = (PBYTE)pFunction;
while (*pCurrentByte != 0xC3 && *pCurrentByte != 0xC2 && *pCurrentByte != 0xCB && *pCurrentByte != 0xCA)
{
OPCODE* pNewOp = new OPCODE();
pNewOp->pbOpCode = pCurrentByte;
if (bIsMOV(pCurrentByte))
{
cout << "mov instr.\n";
}
}
}
void function()
{
int eaxVal;
__asm
{
mov eax, 5
add eax, 6
mov eaxVal, eax
}
printf("Testing %d\n", eaxVal);
}
int main()
{
PBYTE pFirstFive = (PBYTE)malloc(5);
RtlZeroMemory(pFirstFive, 5);
while (true)
{
pCheckByte(function, pFirstFive);
system("pause");
}
return 0;
}

Answer 1

Did you look at the disassembly of function()? The first instruction probably won't be mov eax, 5, since MSVC probably makes a stack frame in functions with inline asm. (push ebp / mov ebp, esp).
Does your code actually loop over the bytes of the function? You have a loop, but it leaks memory every iteration. The only occurrence of pNewOp is, so it's write-only.
OPCODE* pNewOp = new OPCODE();
pNewOp->pbOpCode = pCurrentByte;
Note that looping over all the bytes will give false positives, because 0xb3 or whatever can occur as a non-opcode byte. (e.g. a ModR/M or SIB byte, or immediate data.) Similarly, you could have false positives on your 0xC3, ... scan for ret instructions. Again, look at disassembly with the raw machine code.
Writing your own code for parsing x86 machine code seems like a lot of unnecessary work; there are many tools an libraries that already do this.
Also, single-step through your C++ code in a debugger to see what it does.