In a main program, I dlopen and dlclose (LoadLibrary and FreeLibrary respectively) a shared library. The shared library contains a static variable that is instantiated upon dlopen, and destroyed upon dlclose. This behavior is consistent on MSVC 2008 and 2013, GCC 3.4.6, and Sunstudio 12.1.
With GCC 4.9.1 and GCC 5.2.1 however, the destructor was no longer called on dlclose. Instead, it was called before program exit.
The particularity of the static variable's class, is that in the constructor, there is a call to a templated function get (of global scope) that returns a local static variable.
I was able to reproduce this behavior with the following one cpp file linked into a shared library:
#include <iostream>
template <typename T> // In my actual code, i is of type T, however, this has no effect
int get()
{
static int i = 0;
return i;
}
class Dictionary {
public:
Dictionary()
{
std::cout << "Calling Constructor" << std::endl;
get<int>();
}
~Dictionary(){
std::cout << "Calling Destructor" << std::endl;
}
private:
Dictionary(const Dictionary&);
Dictionary& operator=(const Dictionary&);
};
static Dictionary d;
I investigated the tweaks that can be made in order to have the destructor called on dlclose, and concluded the following:
If the function get was not templated
else if the variable i in the function get was not static
else if the function get is made static
The main program's code is the following:
#include <dlfcn.h>
#include <cassert>
#include <string>
#include <iostream>
void* LoadLib(std::string name)
{
void* libInstance;
name = "lib" + name + ".so";
libInstance = dlopen(name.c_str(), RTLD_NOW);
if ( ! libInstance ) std::cout << "Loading of dictionary library failed. Reason: " << dlerror() << std::endl;
return libInstance;
}
bool UnloadLib(void* libInstance)
{
int ret = dlclose(libInstance);
if (ret == -1)
{
std::cout << "Unloading of dictionary library failed. Reason: " << dlerror() << std::endl;
return false;
}
return true;
}
int main()
{
void* instance = LoadLib("dll");
assert(instance != 0);
assert(UnloadLib(instance));
std::cout << "DLL unloaded" << std::endl;
}
I built the binaries with the following commands:
g++ -m64 -g -std=c++11 -shared -fPIC dll.cpp -o libdll.so
g++ -m64 -g -std=c++11 -ldl main.cpp -o main.out
The output I get when the destructor is called before program exit is the following:
Calling Constructor
DLL unloaded
Calling Destructor
The output I get when the destructor is called on dlclose is the following:
Calling Constructor
Calling Destructor
DLL unloaded
Questions:
If the change of behavior between the versions of GCC is not a bug, can you please explain why is the destructor not called on dlclose?
Can you please explain for each tweak: Why is the destructor called on dlclose in this case?
There is no guarantee that unloading (destructors are invoked) happens on dlclose. On musl (as opposed to glibc), constructors only run the first time a library is run, and destructors only run on exit. For portable code, dlclose cannot be assumed to unload the symbols immediately.
The unload behavior depends on glibc's symbol binding when doing dynamic linking, and is independent of GCC.
The static variable get::i has a STB_GNU_UNIQUE binding. For static variables in inline functions, the uniqueness of the object is assured by the ELF linker. However, for dynamic loading, the dynamic linker assures uniqueness by marking the symbol STB_GNU_UNIQUE. Hence, another attempt to dlopen the same shared library by some other code will lookup the symbol and find that it is unique and return the existent one from the unique symbols table. A symbol with unique binding cannot be unloaded.
Unique binding can be disabled with -fno-gnu-unique if not needed.
References
Bug that I raised to GCC
STB_GNU_UNIQUE
Right off the bat, I want to say that I've never worked with dynamic libraries so It's possible that I don't even understand how they work properly.
I want to have a fully loaded code running and after some trigger (probably user interaction) I want to load a specific library and execute a function inside that library. Preferably close it afterwards. Essentially allowing me to change it and re-load it during run time.
This is the simple dynamic library (called dynlib.so located in the same directory as the main code):
int getInt(int arg_0)
{
return (arg_0 + 7);
}
And this is the main program:
#include <iostream>
#include <dlfcn.h>
int main() {
void *lib_handle = dlopen("./dynlib.so", RTLD_LAZY | RTLD_NOW);
if (!lib_handle) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
typedef int (*func_ptr)(int);
func_ptr func = (func_ptr)dlsym(lib_handle, "getInt");
std::cout << func(13);
dlclose(lib_handle);
}
I'm compiling it using: g++ -std=c++11 -ldl loadlibtest.cpp -o main.
The error I'm catching is ./libshared.so: file too short In my if (!lib_handle) {.
It works fine for me. I've compiled dynlib.so with
$ gcc dynlib.c -fPIC -shared -o dynlib.so
(Obviously, you need to either compile it as C or C++ with extern "C" to avoid name mangling).
and I needed to place -ldl after the source file in the g++ invocation.
gcc: 4.8.5; g++: 5.3.0
dlsym may fail too and casting from void* to function pointers is technically UB. You should base it on the usage snippet from the
manpage(modified for your function):
dlerror(); /* Clear any existing error */
/* Writing: func = (int (*)(int)) dlsym(handle, "getInt");
would seem more natural, but the C99 standard leaves
casting from "void *" to a function pointer undefined.
The assignment used below is the POSIX.1-2003 (Technical
Corrigendum 1) workaround; see the Rationale for the
POSIX specification of dlsym(). */
*(void **) (&func) = dlsym(handle, "getInt");
if ((error = dlerror()) != NULL) {
fprintf(stderr, "%s\n", error);
exit(EXIT_FAILURE);
}
After some great replies I discovered what I'm doing wrong.
1) I wasn't using extern "C" for my library functions, so dlsym was unable to find the function.
2) I didn't know that dynamic libraries had to be compiled << pretty stupid of me.
I still want to know if there is a way to use uncompiled code as a library, but my initial problem was solved, thanks to everyone.
There are similar questions here but they don't quite answer my question:
When to use __declspec(dllexport) in C++
Why do I need __declspec(dllexport) to make some functions accessible from ctypes?
When I cross-compile a DLL from Mac OS X using MinGW and wclang, why does my DLL work fine without using __declspec?
The MinGW DLL sample docs, and every reference I see to doing this, say to use __declspec(dllexport) before function declarations. Yet none of the code in my 9,000-line library uses it, and the DLL works great!
For example, here's a contrived example library built in the same way:
#include <stdio.h>
extern "C" {
int hello(const char* name) {
printf("Hello, %s!\n", name);
return 0;
}
}
Compiled with this on Mac OS X 10.10.3:
w32-clang++ test.cpp -shared -o test.dll
Produces a fine-looking DLL:
And my Windows application:
#include "stdafx.h"
#include <Windows.h>
#include <iostream>
typedef int(*hellofn)(const char*);
int _tmain(int argc, _TCHAR* argv[])
{
DWORD err;
HINSTANCE dll = LoadLibrary(L"E:\\test.dll");
if (!dll) {
err = GetLastError();
std::cout << "Can't load library: " << err << std::endl;
return 1;
}
hellofn hello = (hellofn)GetProcAddress(dll, "hello");
if (!hello) {
err = GetLastError();
std::cout << "Could not load the function: " << err << std::endl;
return 2;
}
int ret = hello("nerd");
std::cout << "hello() returned " << ret << std::endl;
return 0;
}
Works great:
Am I shooting myself in the foot somehow, or is there some magic that I'm not seeing? I'm thinking that wclang (MinGW+clang) knows to use __stdcall automatically somehow and doesn't mangle the function names?
No, you do not need __declspec(dllexport), when building a DLL with MinGW; (in fact, I frequently omit it myself). The caveat is that, if just one symbol to be included in the DLL is so decorated, then all others you wish to have exported must be likewise decorated, (unless you pass the --export-all-symbols option to the linker when you build the DLL).
If you include only undecorated symbols, then all global symbols will be exported, just as if --export-all-symbols were specified by default.
However, this has nothing whatsoever to do with __stdcall vs. __cdecl calling conventions, or name mangling; it is solely a determinant of the visibility of symbols in the DLL's export table. If you don't declare your functions to be __stdcall or __cdecl, then they will be __cdecl by default; that's no problem, provided both providing DLL and caller agree on that convention. Similarly, if both agree on any name mangling convention, (which normally means that, in the case of C++ in particular, they both used the same compiler at build time), then there will be no linking problems.
C++ is a static, compiled language, templates are resolved during compile time and so on...
But is it possible to create a function during runtime, that is not described in the source code and has not been converted to machine language during compilation, so that a user can throw at it data that has not been anticipated in the source?
I am aware this cannot happen in a straightforward way, but surely it must be possible, there are plenty of programing languages that are not compiled and create that sort of stuff dynamically that are implemented in either C or C++.
Maybe if factories for all primitive types are created, along with suitable data structures to organize them into more complex objects such as user types and functions, this is achievable?
Any info on the subject as well as pointers to online materials are welcome. Thanks!
EDIT: I am aware it is possible, it is more like I am interested in implementation details :)
Yes, of course, without any tools mentioned in the other answers, but simply using the C++ compiler.
just follow these steps from within your C++ program (on linux, but must be similar on other OS)
write a C++ program into a file (e.g. in /tmp/prog.cc), using an ofstream
compile the program via system("c++ /tmp/prog.cc -o /tmp/prog.so -shared -fPIC");
load the program dynamically, e.g. using dlopen()
You can also just give the bytecode directly to a function and just pass it casted as the function type as demonstrated below.
e.g.
byte[3] func = { 0x90, 0x0f, 0x1 }
*reinterpret_cast<void**>(&func)()
Yes, JIT compilers do it all the time. They allocate a piece of memory that has been given special execution rights by the OS, then fill it with code and cast the pointer to a function pointer and execute it. Pretty simple.
EDIT: Here's an example on how to do it in Linux: http://burnttoys.blogspot.de/2011/04/how-to-allocate-executable-memory-on.html
Below an example for C++ runtime compilation based on the method mentioned before (write code to output file, compile via system(), load via dlopen() and dlsym()). See also the example in a related question. The difference here is that it dynamically compiles a class rather than a function. This is achieved by adding a C-style maker() function to the code to be compiled dynamically. References:
https://www.linuxjournal.com/article/3687
http://www.tldp.org/HOWTO/C++-dlopen/thesolution.html
The example only works under Linux (Windows has LoadLibrary and GetProcAddress functions instead), and requires the identical compiler to be available on the target machine.
baseclass.h
#ifndef BASECLASS_H
#define BASECLASS_H
class A
{
protected:
double m_input; // or use a pointer to a larger input object
public:
virtual double f(double x) const = 0;
void init(double input) { m_input=input; }
virtual ~A() {};
};
#endif /* BASECLASS_H */
main.cpp
#include "baseclass.h"
#include <cstdlib> // EXIT_FAILURE, etc
#include <string>
#include <iostream>
#include <fstream>
#include <dlfcn.h> // dynamic library loading, dlopen() etc
#include <memory> // std::shared_ptr
// compile code, instantiate class and return pointer to base class
// https://www.linuxjournal.com/article/3687
// http://www.tldp.org/HOWTO/C++-dlopen/thesolution.html
// https://stackoverflow.com/questions/11016078/
// https://stackoverflow.com/questions/10564670/
std::shared_ptr<A> compile(const std::string& code)
{
// temporary cpp/library output files
std::string outpath="/tmp";
std::string headerfile="baseclass.h";
std::string cppfile=outpath+"/runtimecode.cpp";
std::string libfile=outpath+"/runtimecode.so";
std::string logfile=outpath+"/runtimecode.log";
std::ofstream out(cppfile.c_str(), std::ofstream::out);
// copy required header file to outpath
std::string cp_cmd="cp " + headerfile + " " + outpath;
system(cp_cmd.c_str());
// add necessary header to the code
std::string newcode = "#include \"" + headerfile + "\"\n\n"
+ code + "\n\n"
"extern \"C\" {\n"
"A* maker()\n"
"{\n"
" return (A*) new B(); \n"
"}\n"
"} // extern C\n";
// output code to file
if(out.bad()) {
std::cout << "cannot open " << cppfile << std::endl;
exit(EXIT_FAILURE);
}
out << newcode;
out.flush();
out.close();
// compile the code
std::string cmd = "g++ -Wall -Wextra " + cppfile + " -o " + libfile
+ " -O2 -shared -fPIC &> " + logfile;
int ret = system(cmd.c_str());
if(WEXITSTATUS(ret) != EXIT_SUCCESS) {
std::cout << "compilation failed, see " << logfile << std::endl;
exit(EXIT_FAILURE);
}
// load dynamic library
void* dynlib = dlopen (libfile.c_str(), RTLD_LAZY);
if(!dynlib) {
std::cerr << "error loading library:\n" << dlerror() << std::endl;
exit(EXIT_FAILURE);
}
// loading symbol from library and assign to pointer
// (to be cast to function pointer later)
void* create = dlsym(dynlib, "maker");
const char* dlsym_error=dlerror();
if(dlsym_error != NULL) {
std::cerr << "error loading symbol:\n" << dlsym_error << std::endl;
exit(EXIT_FAILURE);
}
// execute "create" function
// (casting to function pointer first)
// https://stackoverflow.com/questions/8245880/
A* a = reinterpret_cast<A*(*)()> (create)();
// cannot close dynamic lib here, because all functions of the class
// object will still refer to the library code
// dlclose(dynlib);
return std::shared_ptr<A>(a);
}
int main(int argc, char** argv)
{
double input=2.0;
double x=5.1;
// code to be compiled at run-time
// class needs to be called B and derived from A
std::string code = "class B : public A {\n"
" double f(double x) const \n"
" {\n"
" return m_input*x;\n"
" }\n"
"};";
std::cout << "compiling.." << std::endl;
std::shared_ptr<A> a = compile(code);
a->init(input);
std::cout << "f(" << x << ") = " << a->f(x) << std::endl;
return EXIT_SUCCESS;
}
output
$ g++ -Wall -std=c++11 -O2 -c main.cpp -o main.o # c++11 required for std::shared_ptr
$ g++ -ldl main.o -o main
$ ./main
compiling..
f(5.1) = 10.2
Have a look at libtcc; it is simple, fast, reliable and suits your need. I use it whenever I need to compile C functions "on the fly".
In the archive, you will find the file examples/libtcc_test.c, which can give you a good head start.
This little tutorial might also help you: http://blog.mister-muffin.de/2011/10/22/discovering-tcc/
#include <stdlib.h>
#include <stdio.h>
#include "libtcc.h"
int add(int a, int b) { return a + b; }
char my_program[] =
"int fib(int n) {\n"
" if (n <= 2) return 1;\n"
" else return fib(n-1) + fib(n-2);\n"
"}\n"
"int foobar(int n) {\n"
" printf(\"fib(%d) = %d\\n\", n, fib(n));\n"
" printf(\"add(%d, %d) = %d\\n\", n, 2 * n, add(n, 2 * n));\n"
" return 1337;\n"
"}\n";
int main(int argc, char **argv)
{
TCCState *s;
int (*foobar_func)(int);
void *mem;
s = tcc_new();
tcc_set_output_type(s, TCC_OUTPUT_MEMORY);
tcc_compile_string(s, my_program);
tcc_add_symbol(s, "add", add);
mem = malloc(tcc_relocate(s, NULL));
tcc_relocate(s, mem);
foobar_func = tcc_get_symbol(s, "foobar");
tcc_delete(s);
printf("foobar returned: %d\n", foobar_func(32));
free(mem);
return 0;
}
Ask questions in the comments if you meet any problems using the library!
In addition to simply using an embedded scripting language (Lua is great for embedding) or writing your own compiler for C++ to use at runtime, if you really want to use C++ you can just use an existing compiler.
For example Clang is a C++ compiler built as libraries that could be easily embedded in another program. It was designed to be used from programs like IDEs that need to analyze and manipulate C++ source in various ways, but using the LLVM compiler infrasructure as a backend it also has the ability to generate code at runtime and hand you a function pointer that you can call to run the generated code.
Clang
LLVM
Essentially you will need to write a C++ compiler within your program (not a trivial task), and do the same thing JIT compilers do to run the code. You were actually 90% of the way there with this paragraph:
I am aware this cannot happen in a straightforward way, but surely it
must be possible, there are plenty of programing languages that are
not compiled and create that sort of stuff dynamically that are
implemented in either C or C++.
Exactly--those programs carry the interpreter with them. You run a python program by saying python MyProgram.py--python is the compiled C code that has the ability to interpret and run your program on the fly. You would need do something along those lines, but by using a C++ compiler.
If you need dynamic functions that badly, use a different language :)
A typical approach for this is to combine a C++ (or whatever it's written on) project with scripting language.
Lua is one of the top favorites, since it's well documented, small, and has bindings for a lot of languages.
But if you are not looking into that direction, perhaps you could think of making a use of dynamic libraries?
Yes - you can write a compiler for C++, in C++, with some extra features - write your own functions, compile and run automatically (or not)...
Have a look into ExpressionTrees in .NET - I think this is basically what you want to achieve. Create a tree of subexpressions and then evaluate them. In an object-oriented fashion, each node in the might know how to evaluate itself, by recursion into its subnodes. Your visual language would then create this tree and you can write a simple interpreter to execute it.
Also, check out Ptolemy II, as an example in Java on how such a visual programming language can be written.
You could take a look at Runtime Compiled C++ (or see RCC++ blog and videos), or perhaps try one of its alternatives.
Expanding on Jay's answer using opcodes, the below works on Linux.
Learn opcodes from your compiler:
write own myfunc.cpp, e.g.
double f(double x) { return x*x; }
compile with
$ g++ -O2 -c myfunc.cpp
disassemble function f
$ gdb -batch -ex "file ./myfunc.o" -ex "set disassembly-flavor intel" -ex "disassemble/rs f"
Dump of assembler code for function _Z1fd:
0x0000000000000000 <+0>: f2 0f 59 c0 mulsd xmm0,xmm0
0x0000000000000004 <+4>: c3 ret
End of assembler dump.
This means the function x*x in assembly is mulsd xmm0,xmm0, ret and in machine code f2 0f 59 c0 c3.
Write your own function in machine code:
opcode.cpp
#include <cstdlib> // EXIT_FAILURE etc
#include <cstdio> // printf(), fopen() etc
#include <cstring> // memcpy()
#include <sys/mman.h> // mmap()
// allocate memory and fill it with machine code instructions
// returns pointer to memory location and length in bytes
void* gencode(size_t& length)
{
// machine code
unsigned char opcode[] = {
0xf2, 0x0f, 0x59, 0xc0, // mulsd xmm0,xmm0
0xc3 // ret
};
// allocate memory which allows code execution
// https://en.wikipedia.org/wiki/NX_bit
void* buf = mmap(NULL,sizeof(opcode),PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANON,-1,0);
// copy machine code to executable memory location
memcpy(buf, opcode, sizeof(opcode));
// return: pointer to memory location with executable code
length = sizeof(opcode);
return buf;
}
// print the disassemby of buf
void print_asm(const void* buf, size_t length)
{
FILE* fp = fopen("/tmp/opcode.bin", "w");
if(fp!=NULL) {
fwrite(buf, length, 1, fp);
fclose(fp);
}
system("objdump -D -M intel -b binary -mi386 /tmp/opcode.bin");
}
int main(int, char**)
{
// generate machine code and point myfunc() to it
size_t length;
void* code=gencode(length);
double (*myfunc)(double); // function pointer
myfunc = reinterpret_cast<double(*)(double)>(code);
double x=1.5;
printf("f(%f)=%f\n", x,myfunc(x));
print_asm(code,length); // for debugging
return EXIT_SUCCESS;
}
compile and run
$ g++ -O2 opcode.cpp -o opcode
$ ./opcode
f(1.500000)=2.250000
/tmp/opcode.bin: file format binary
Disassembly of section .data:
00000000 <.data>:
0: f2 0f 59 c0 mulsd xmm0,xmm0
4: c3 ret
The simplest solution available, if you're not looking for performance is to embed a scripting language interpreter, e.g. for Lua or Python.
It worked for me like this. You have to use the -fpermissive flag.
I am using CodeBlocks 17.12.
#include <cstddef>
using namespace std;
int main()
{
char func[] = {'\x90', '\x0f', '\x1'};
void (*func2)() = reinterpret_cast<void*>(&func);
func2();
return 0;
}
I want to implement a function tracer, which would trace how much time a function is taking to execute. I have following class for the same:-
class FuncTracer
{
public:
FuncTracer(LPCTSTR strFuncName_in)
{
m_strFuncName[0] = _T('\0');
if( strFuncName_in ||
_T('\0') != strFuncName_in[0])
{
_tcscpy(m_strFuncName,strFuncName_in);
TCHAR strLog[MAX_PATH];
_stprintf(strLog,_T("Entering Func:- <%s>"),m_strFuncName);
LOG(strLog)
m_dwEnterTime = GetTickCount();
}
}
~FuncTracer()
{
TCHAR strLog[MAX_PATH];
_stprintf(strLog,_T("Leaving Func:- <%s>, Time inside the func <%d> ms"),m_strFuncName, GetTickCount()-m_dwEnterTime);
LOG(strLog)
}
private:
TCHAR m_strFuncName[MAX_PATH];
DWORD m_dwEnterTime;
};
void TestClass::TestFunction()
{
// I want to avoid writing the function name maually..
// Is there any macro (__LINE__)or some other way to
// get the function name inside a function ??
FuncTracer(_T("TestClass::TestFunction"));
/*
* Rest of the function code.
*/
}
I want to know if there is any way to get the name of the function from inside of a function? Basically I want the users of my class to simply create an object the same. They may not pass the function name.
C99 has __func__, but for C++ this will be compiler specific. On the plus side, some of the compiler-specific versions provide additional type information, which is particularly nice when you're tracing inside a templatized function/class.
MSVC: __FUNCTION__, __FUNCDNAME__, __FUNCSIG__
GCC: __func__, __FUNCTION__, __PRETTY_FUNCTION__
Boost library has defined macro BOOST_CURRENT_FUNCTION for most C++ compilers in header boost/current_function.hpp. If the compiler is too old to support this, the result will be "(unknown)".
VC++ has
__FUNCTION__ for undecorated names
and
__FUNCDNAME__ for decorated names
And you can write a macro that will itself allocate an object and pass the name-yelding macro inside the constructor. Smth like
#define ALLOC_LOGGER FuncTracer ____tracer( __FUNCTION__ );
C++20 std::source_location::function_name
main.cpp
#include <iostream>
#include <string_view>
#include <source_location>
void log(std::string_view message,
const std::source_location& location = std::source_location::current()
) {
std::cout << "info:"
<< location.file_name() << ":"
<< location.line() << ":"
<< location.function_name() << " "
<< message << '\n';
}
int f(int i) {
log("Hello world!"); // Line 16
return i + 1;
}
int f(double i) {
log("Hello world!"); // Line 21
return i + 1.0;
}
int main() {
f(1);
f(1.0);
}
Compile and run:
g++ -ggdb3 -O0 -std=c++20 -Wall -Wextra -pedantic -o source_location.out source_location.cpp
./source_location.out
Output:
info:source_location.cpp:16:int f(int) Hello world!
info:source_location.cpp:21:int f(double) Hello world!
so note how the call preserves caller information, so we see the desired main call location instead of log.
I have covered the relevant standards in a bit more detail at: What's the difference between __PRETTY_FUNCTION__, __FUNCTION__, __func__?
Tested on Ubuntu 22.04, GCC 11.3.
I was going to say I didn't know of any such thing but then I saw the other answers...
It might interest you to know that an execution profiler (like gprof) does exactly what you're asking about - it tracks the amount of time spent executing each function. A profiler basically works by recording the instruction pointer (IP), the address of the currently executing instruction, every 10ms or so. After the program is done running, you invoke a postprocessor that examines the list of IPs and the program, and converts those addresses into function names. So I'd suggest just using the instruction pointer, rather than the function name, both because it's easier to code and because it's more efficient to work with a single number than with a string.