I have some x86 code in a .asm file and am trying to use it from C++:
C++
#include <iostream>
extern "C" int addInts(int a, int b);
int main() {
int a = 1;
int b = 2;
int result = addInts(a, b);
std::cout << "Result :\t" << result << std::endl;
return 0;
}
Asm
.386
.MODEL FLAT, C
.CODE
addInts PROC
PUSH EBP
MOV EBP, ESP
MOV EAX, [EBP+8]
MOV ECX, [EBP+12]
ADD EAX, ECX
POP EBP
RET
addInts ENDP
END
Attempting to run this results in:
LNK4042 (warn) object specified more than once, ignoring extras (asm object file)
LNK2019 (error) unresolved external symbol _addInts referenced in function _main (asm object file)
Followed by a final act of defiance taking the form of fatal error LNK1120 due to the unresolved external (solution executable)
I'm using Visual Studio 2019 with MSVC v142 and MASM. My other, self-contained assembly code has had no issues, and another function I've written involving reading int arrays in x86 from C++ worked fine. I really can't see what's going wrong here, if its a problem with my code, some esoteric setting, or something else entirely.
If I change the last line of the Asm code to END addInts then the program just runs and immediately exits with nothing in std::cout.
The solution file has no entry point defined in linker settings, which was what I did for the last piece of code that called asm from C++.
The asm file is included in the build, using Microsoft Macro Assembler.
The cpp file is set to compile as C++, just in case.
This problem was dispelled after noticing that the .cpp file and .asm files had the same names (IntegerAddition.cpp and IntegerAddition.asm).
Renaming the assembly file to something else fixed the issue.
I encountered something weird in the MSVC compiler.
it puts function template definition in assembly while optimization eliminates the need for them.
It seems that Clang and GCC successfully remove function definition at all but MSVC does not.
Can it be fixed?
main.cpp:
#include <iostream>
template <int n> int value() noexcept
{
return n;
}
int main()
{
return value<5>() + value<10>();
}
assembly:
int value<5>(void) PROC ; value<5>, COMDAT
mov eax, 5
ret 0
int value<5>(void) ENDP ; value<5>
int value<10>(void) PROC ; value<10>, COMDAT
mov eax, 10
ret 0
int value<10>(void) ENDP ; value<10>
main PROC ; COMDAT
mov eax, 15
ret 0
main ENDP
Sample code on godbolt
The /FA switch generates the listing file for each translation unit. Since this is before the linking stage, MSVC does not determine if those two functions are required anywhere else within the program, and are thus still included in the generated .asm file (Note: this may be for simplicity on MS's part, since it can treat templates the same as regular functions in the generated .obj file, though realistically there's no actual need to store them in the .obj file, as user17732522 points out in the comments).
During linking, MSVC determines that those functions are in fact not actually used / needed anywhere else, and thus can be eliminated (even if they were used elsewhere, since the result can be determined at compile time, they'd still be eliminated) from the compiled executable.
In order to see what's in the final compiled executable, you can view the executable through a disassembler. Example for using MSVC to do this, is put a breakpoint in the main function, run it, then when the breakpoint is hit, right click and "View Disassembly". In this, you will see that the two functions don't exist anymore.
You can also generate the Mapfile using /MAP option, which also shows it does not exist.
If I am reading the documentation correctly, it seems as those MS chose to include explicit instantiations of templates classes and functions because it "is useful" when creating libraries. Uninstantiated templates are not put into the obj files though.
Just add /Zc:inline to your compile statement and it does the same thing as clang/GCC if you also wrap the template in an anonymous namespace to ensure it does not have external visibility.
#include <iostream>
namespace
{
template <int n> int value() noexcept
{
return n;
}
}
or if you mark the template function inline
template <int n> inline int value() noexcept
{
return n;
}
Both result in:
main PROC
mov eax, 15
ret 0
main ENDP
The /Zc:inline (Remove unreferenced COMDAT) switch was added in VS 2015 Update 2 as part of the C++11 Standard conformance which allows this optimization.
It is off-by-default in command-line builds. In MSBuild, <RemoveUnreferencedCodeData> defaults to true.
See Microsoft Docs
OTHERWISE It will be cleaned up in the linker phase with /OPT:REF.
I compiled your code as given on my vs2022 in release mode. I get
return value<5>() + value<10>();
00007FF65CD21000 mov eax,0Fh
}
00007FF65CD21005 ret
I am practicing reverse engineering software. I am using Microsoft Visual Studio. I created an empty project and then created an empty file which I called main.cpp. I then wrote the following code, compiled
int main()
{
char* str = "hello matthew";
int x = 15;
return 0;
}
When I brought the release version of the executable over to BinText and IdaPro, the string "hello matthew" was no where to be found. I could also never find the value 15 either in base 10 or hexadecimal.
I cannot begin to understand reverse engineering if I cannot find the references to the values I am looking for in the executable.
My theory is that because my program does absolutely nothing that the compiler just omitted it all, but I do not know for sure. Does anyone know why I cannot locate that string or the value 15 in the executable when I disassemble it?
I cannot begin to understand reverse engineering ...
The first step is to actually understand how the program is built out.
Before you can understand how to reverse a program, you need to understand how it's compiled and built; reversing a binary built for Windows is vastly different from reversing a binary for a *nix system.
To that, since you're using Visual Studio, you can see this answer (option 2) explaining how to enable the assembly output of your code. Alternatively if you're compiling via command line, you can pass /FAs and /Fa to generate the assembly inlined with the source.
Your code produces the following assembly:
; Listing generated by Microsoft (R) Optimizing Compiler Version 18.00.40629.0
TITLE C:\Code\test\test.cpp
.686P
.XMM
include listing.inc
.model flat
INCLUDELIB LIBCMT
INCLUDELIB OLDNAMES
CONST SEGMENT
$SG2548 DB 'hello matthew', 00H
CONST ENDS
PUBLIC _main
; Function compile flags: /Odtp
; File c:\code\test\test.cpp
_TEXT SEGMENT
_x$ = -8 ; size = 4
_str$ = -4 ; size = 4
_main PROC
; 2 : {
push ebp
mov ebp, esp
sub esp, 8
; 3 : char* str = "hello matthew";
mov DWORD PTR _str$[ebp], OFFSET $SG2548
; 4 :
; 5 : int x = 15;
mov DWORD PTR _x$[ebp], 15 ; 0000000fH
; 6 :
; 7 : return 0;
xor eax, eax
; 8 : }
mov esp, ebp
pop ebp
ret 0
_main ENDP
_TEXT ENDS
END
While this is helpful to understand how and what your code is doing, one of the best way to start reversing, is to throw a binary in a debugger, like attaching Visual Studio to an executable, and viewing the assembly as the program is running.
It can depend on what your after since a binary could potentially be obfuscated; that is to say that there could be strings within the binary, but they could be encrypted or just scrambled so as to be unreadable until decrypted/unscrambled by some function within the binary.
So just searching for strings won't necessarily give you anything, and trying to search for a specific binary value in the assembled code is like trying to find a needle in a stack of needles. Know why your trying to reverse a program, then attack that vector.
Does anyone know why I cannot locate that string or the value 15 in the executable when I disassemble it?
As has been mentioned, and as you have guessed, the "release" binary you're searching through was optimized, and the compiler just removed the unused variables so the assembly was essentially returning 0.
I hope that can help.
the main reason is that your code does nothing useful with x and str, so they are entirely redundant!!, and no need for them to even exist in your code! so the compiler automatically removes them from the compiled code "optimization"!!.
if you really want to see them in the compiled code under debuggers, you need to use them or simply tell the compiler not to optimize this part of the code!!
This is how to tell the compiler not to optimize these variable's locations by using volatile qualifier
#include <iostream>
int main(int argc, char** argv) {
const char* volatile str = "hello matthew";
volatile int x = 15;
return 0;
}
this shows that your variables are included in the compiled code in IDA Pro
or as I also said just use them!!!
#include <iostream>
int main(int argc, char** argv) {
const char* str = "hello matthew";
int x = 15;
std::cout << str << x;
return 0;
}
I am using Visual C++ 2010, and MASM as my x64-Assembler.
This is my C++ code:
// include directive
#include "stdafx.h"
// functions
extern "C" int Asm();
extern "C" int (convention) sum(int x, int y) { return x + y; }
// main function
int main()
{
// print asm
printf("Asm returned %d.\n", Asm());
// get char, return
_getch();
return EXIT_SUCCESS;
}
And my assembly code:
; external functions
extern sum : proc
; code segment
.code
Asm proc
; create shadow space
sub rsp, 20o
; setup parameters
mov ecx, 10
mov edx, 15
; call
call sum
; clean-up shadow space
add rsp, 20o
; return
ret
Asm endp
end
The reason I am doing this is so I can learn the different calling conventions.
I would make sum's calling convention stdcall, and modify the asm code so it would call sum the "stdcall" way. Once I got that working, I would make it, say, fastcall, and then call it in asm the "fastcall" way.
But look at my assembly code right now. When I use that code, no matter if sum is stdcall, fastcall or cdecl, it will compile, execute fine, and print 25 as my sum.
My question: How, and why can __cdecl, __stdcall and __fastcall all be called the exact same way?
The problem is that you're compiling for x64 targets. From MSDN
Given the expanded register set, x64 just uses the __fastcall calling
convention and a RISC-based exception-handling model. The __fastcall
model uses registers for the first four arguments and the stack frame
to pass the other parameters.
Switch over to compiling for x86 targets, and you should be able to see the various calling conventions in action.
As far as i know x64 only uses the __fastcall convention. __cdecl and stdcall will just be compiled as __fastcall.
C++ is a static, compiled language, templates are resolved during compile time and so on...
But is it possible to create a function during runtime, that is not described in the source code and has not been converted to machine language during compilation, so that a user can throw at it data that has not been anticipated in the source?
I am aware this cannot happen in a straightforward way, but surely it must be possible, there are plenty of programing languages that are not compiled and create that sort of stuff dynamically that are implemented in either C or C++.
Maybe if factories for all primitive types are created, along with suitable data structures to organize them into more complex objects such as user types and functions, this is achievable?
Any info on the subject as well as pointers to online materials are welcome. Thanks!
EDIT: I am aware it is possible, it is more like I am interested in implementation details :)
Yes, of course, without any tools mentioned in the other answers, but simply using the C++ compiler.
just follow these steps from within your C++ program (on linux, but must be similar on other OS)
write a C++ program into a file (e.g. in /tmp/prog.cc), using an ofstream
compile the program via system("c++ /tmp/prog.cc -o /tmp/prog.so -shared -fPIC");
load the program dynamically, e.g. using dlopen()
You can also just give the bytecode directly to a function and just pass it casted as the function type as demonstrated below.
e.g.
byte[3] func = { 0x90, 0x0f, 0x1 }
*reinterpret_cast<void**>(&func)()
Yes, JIT compilers do it all the time. They allocate a piece of memory that has been given special execution rights by the OS, then fill it with code and cast the pointer to a function pointer and execute it. Pretty simple.
EDIT: Here's an example on how to do it in Linux: http://burnttoys.blogspot.de/2011/04/how-to-allocate-executable-memory-on.html
Below an example for C++ runtime compilation based on the method mentioned before (write code to output file, compile via system(), load via dlopen() and dlsym()). See also the example in a related question. The difference here is that it dynamically compiles a class rather than a function. This is achieved by adding a C-style maker() function to the code to be compiled dynamically. References:
https://www.linuxjournal.com/article/3687
http://www.tldp.org/HOWTO/C++-dlopen/thesolution.html
The example only works under Linux (Windows has LoadLibrary and GetProcAddress functions instead), and requires the identical compiler to be available on the target machine.
baseclass.h
#ifndef BASECLASS_H
#define BASECLASS_H
class A
{
protected:
double m_input; // or use a pointer to a larger input object
public:
virtual double f(double x) const = 0;
void init(double input) { m_input=input; }
virtual ~A() {};
};
#endif /* BASECLASS_H */
main.cpp
#include "baseclass.h"
#include <cstdlib> // EXIT_FAILURE, etc
#include <string>
#include <iostream>
#include <fstream>
#include <dlfcn.h> // dynamic library loading, dlopen() etc
#include <memory> // std::shared_ptr
// compile code, instantiate class and return pointer to base class
// https://www.linuxjournal.com/article/3687
// http://www.tldp.org/HOWTO/C++-dlopen/thesolution.html
// https://stackoverflow.com/questions/11016078/
// https://stackoverflow.com/questions/10564670/
std::shared_ptr<A> compile(const std::string& code)
{
// temporary cpp/library output files
std::string outpath="/tmp";
std::string headerfile="baseclass.h";
std::string cppfile=outpath+"/runtimecode.cpp";
std::string libfile=outpath+"/runtimecode.so";
std::string logfile=outpath+"/runtimecode.log";
std::ofstream out(cppfile.c_str(), std::ofstream::out);
// copy required header file to outpath
std::string cp_cmd="cp " + headerfile + " " + outpath;
system(cp_cmd.c_str());
// add necessary header to the code
std::string newcode = "#include \"" + headerfile + "\"\n\n"
+ code + "\n\n"
"extern \"C\" {\n"
"A* maker()\n"
"{\n"
" return (A*) new B(); \n"
"}\n"
"} // extern C\n";
// output code to file
if(out.bad()) {
std::cout << "cannot open " << cppfile << std::endl;
exit(EXIT_FAILURE);
}
out << newcode;
out.flush();
out.close();
// compile the code
std::string cmd = "g++ -Wall -Wextra " + cppfile + " -o " + libfile
+ " -O2 -shared -fPIC &> " + logfile;
int ret = system(cmd.c_str());
if(WEXITSTATUS(ret) != EXIT_SUCCESS) {
std::cout << "compilation failed, see " << logfile << std::endl;
exit(EXIT_FAILURE);
}
// load dynamic library
void* dynlib = dlopen (libfile.c_str(), RTLD_LAZY);
if(!dynlib) {
std::cerr << "error loading library:\n" << dlerror() << std::endl;
exit(EXIT_FAILURE);
}
// loading symbol from library and assign to pointer
// (to be cast to function pointer later)
void* create = dlsym(dynlib, "maker");
const char* dlsym_error=dlerror();
if(dlsym_error != NULL) {
std::cerr << "error loading symbol:\n" << dlsym_error << std::endl;
exit(EXIT_FAILURE);
}
// execute "create" function
// (casting to function pointer first)
// https://stackoverflow.com/questions/8245880/
A* a = reinterpret_cast<A*(*)()> (create)();
// cannot close dynamic lib here, because all functions of the class
// object will still refer to the library code
// dlclose(dynlib);
return std::shared_ptr<A>(a);
}
int main(int argc, char** argv)
{
double input=2.0;
double x=5.1;
// code to be compiled at run-time
// class needs to be called B and derived from A
std::string code = "class B : public A {\n"
" double f(double x) const \n"
" {\n"
" return m_input*x;\n"
" }\n"
"};";
std::cout << "compiling.." << std::endl;
std::shared_ptr<A> a = compile(code);
a->init(input);
std::cout << "f(" << x << ") = " << a->f(x) << std::endl;
return EXIT_SUCCESS;
}
output
$ g++ -Wall -std=c++11 -O2 -c main.cpp -o main.o # c++11 required for std::shared_ptr
$ g++ -ldl main.o -o main
$ ./main
compiling..
f(5.1) = 10.2
Have a look at libtcc; it is simple, fast, reliable and suits your need. I use it whenever I need to compile C functions "on the fly".
In the archive, you will find the file examples/libtcc_test.c, which can give you a good head start.
This little tutorial might also help you: http://blog.mister-muffin.de/2011/10/22/discovering-tcc/
#include <stdlib.h>
#include <stdio.h>
#include "libtcc.h"
int add(int a, int b) { return a + b; }
char my_program[] =
"int fib(int n) {\n"
" if (n <= 2) return 1;\n"
" else return fib(n-1) + fib(n-2);\n"
"}\n"
"int foobar(int n) {\n"
" printf(\"fib(%d) = %d\\n\", n, fib(n));\n"
" printf(\"add(%d, %d) = %d\\n\", n, 2 * n, add(n, 2 * n));\n"
" return 1337;\n"
"}\n";
int main(int argc, char **argv)
{
TCCState *s;
int (*foobar_func)(int);
void *mem;
s = tcc_new();
tcc_set_output_type(s, TCC_OUTPUT_MEMORY);
tcc_compile_string(s, my_program);
tcc_add_symbol(s, "add", add);
mem = malloc(tcc_relocate(s, NULL));
tcc_relocate(s, mem);
foobar_func = tcc_get_symbol(s, "foobar");
tcc_delete(s);
printf("foobar returned: %d\n", foobar_func(32));
free(mem);
return 0;
}
Ask questions in the comments if you meet any problems using the library!
In addition to simply using an embedded scripting language (Lua is great for embedding) or writing your own compiler for C++ to use at runtime, if you really want to use C++ you can just use an existing compiler.
For example Clang is a C++ compiler built as libraries that could be easily embedded in another program. It was designed to be used from programs like IDEs that need to analyze and manipulate C++ source in various ways, but using the LLVM compiler infrasructure as a backend it also has the ability to generate code at runtime and hand you a function pointer that you can call to run the generated code.
Clang
LLVM
Essentially you will need to write a C++ compiler within your program (not a trivial task), and do the same thing JIT compilers do to run the code. You were actually 90% of the way there with this paragraph:
I am aware this cannot happen in a straightforward way, but surely it
must be possible, there are plenty of programing languages that are
not compiled and create that sort of stuff dynamically that are
implemented in either C or C++.
Exactly--those programs carry the interpreter with them. You run a python program by saying python MyProgram.py--python is the compiled C code that has the ability to interpret and run your program on the fly. You would need do something along those lines, but by using a C++ compiler.
If you need dynamic functions that badly, use a different language :)
A typical approach for this is to combine a C++ (or whatever it's written on) project with scripting language.
Lua is one of the top favorites, since it's well documented, small, and has bindings for a lot of languages.
But if you are not looking into that direction, perhaps you could think of making a use of dynamic libraries?
Yes - you can write a compiler for C++, in C++, with some extra features - write your own functions, compile and run automatically (or not)...
Have a look into ExpressionTrees in .NET - I think this is basically what you want to achieve. Create a tree of subexpressions and then evaluate them. In an object-oriented fashion, each node in the might know how to evaluate itself, by recursion into its subnodes. Your visual language would then create this tree and you can write a simple interpreter to execute it.
Also, check out Ptolemy II, as an example in Java on how such a visual programming language can be written.
You could take a look at Runtime Compiled C++ (or see RCC++ blog and videos), or perhaps try one of its alternatives.
Expanding on Jay's answer using opcodes, the below works on Linux.
Learn opcodes from your compiler:
write own myfunc.cpp, e.g.
double f(double x) { return x*x; }
compile with
$ g++ -O2 -c myfunc.cpp
disassemble function f
$ gdb -batch -ex "file ./myfunc.o" -ex "set disassembly-flavor intel" -ex "disassemble/rs f"
Dump of assembler code for function _Z1fd:
0x0000000000000000 <+0>: f2 0f 59 c0 mulsd xmm0,xmm0
0x0000000000000004 <+4>: c3 ret
End of assembler dump.
This means the function x*x in assembly is mulsd xmm0,xmm0, ret and in machine code f2 0f 59 c0 c3.
Write your own function in machine code:
opcode.cpp
#include <cstdlib> // EXIT_FAILURE etc
#include <cstdio> // printf(), fopen() etc
#include <cstring> // memcpy()
#include <sys/mman.h> // mmap()
// allocate memory and fill it with machine code instructions
// returns pointer to memory location and length in bytes
void* gencode(size_t& length)
{
// machine code
unsigned char opcode[] = {
0xf2, 0x0f, 0x59, 0xc0, // mulsd xmm0,xmm0
0xc3 // ret
};
// allocate memory which allows code execution
// https://en.wikipedia.org/wiki/NX_bit
void* buf = mmap(NULL,sizeof(opcode),PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANON,-1,0);
// copy machine code to executable memory location
memcpy(buf, opcode, sizeof(opcode));
// return: pointer to memory location with executable code
length = sizeof(opcode);
return buf;
}
// print the disassemby of buf
void print_asm(const void* buf, size_t length)
{
FILE* fp = fopen("/tmp/opcode.bin", "w");
if(fp!=NULL) {
fwrite(buf, length, 1, fp);
fclose(fp);
}
system("objdump -D -M intel -b binary -mi386 /tmp/opcode.bin");
}
int main(int, char**)
{
// generate machine code and point myfunc() to it
size_t length;
void* code=gencode(length);
double (*myfunc)(double); // function pointer
myfunc = reinterpret_cast<double(*)(double)>(code);
double x=1.5;
printf("f(%f)=%f\n", x,myfunc(x));
print_asm(code,length); // for debugging
return EXIT_SUCCESS;
}
compile and run
$ g++ -O2 opcode.cpp -o opcode
$ ./opcode
f(1.500000)=2.250000
/tmp/opcode.bin: file format binary
Disassembly of section .data:
00000000 <.data>:
0: f2 0f 59 c0 mulsd xmm0,xmm0
4: c3 ret
The simplest solution available, if you're not looking for performance is to embed a scripting language interpreter, e.g. for Lua or Python.
It worked for me like this. You have to use the -fpermissive flag.
I am using CodeBlocks 17.12.
#include <cstddef>
using namespace std;
int main()
{
char func[] = {'\x90', '\x0f', '\x1'};
void (*func2)() = reinterpret_cast<void*>(&func);
func2();
return 0;
}