Go shared library as C++ plugin - c++

I have a project where I would like to load Go plugins inside a C++ application.
After a lot of research, it is not clear for me whether or not Go supports this.
I encountered a lot of discussions pointing out the bad habit of dynamic linking, proning IPC instead. Moreover it is not clear for me if dynamic linking is intended by the language or not (new Go philosophy ?).
cgo provides the ability to call C from Go or Go from C (inside Go), but not from plain old C. Or does it ?
gc doesn't seem to support shared library (even if https://code.google.com/p/go/issues/detail?id=256 mentions it does)
gccgo support Go shared libraries but I couldn't make it work (probably because the main entry point is not in Go ...)
SWIG doesn't seem to help either :(
Apparently something is going on upstream as well (https://codereview.appspot.com/7304104/)
main.c
extern void Print(void) __asm__ ("example.main.Print");
int main() {
Print();
}
print.go
package main
import "fmt"
func Print() {
fmt.Printf("hello, world\n")
}
Makefile :
all: print.o main.c
gcc main.c -L. -lprint -o main
print.o: print.go
gccgo -fno-split-stack -fgo-prefix=example -fPIC -c print.go -o print.o
gccgo -shared print.o -o libprint.so
Output :
/usr/lib/libgo.so.3: undefined reference to `main.main'
/usr/lib/libgo.so.3: undefined reference to `__go_init_main'
Is there a solution for that ? What is the best approach ? forking + IPC ?
References :
cgo - go wiki
callback in cgo
c callbacks and non go threads
cgo - golang
call go function from c
link c code with go code

I don't think you can embed Go into C. However you can embed C into Go and with a little stub C program you can call into C first thing which is the next best thing! Cgo definitely supports linking with shared libraries so maybe this approach will work for you.
Like this
main.go
// Stub go program to call cmain() in C
package main
// extern int cmain(void);
import "C"
func main() {
C.cmain()
}
main.c
#include <stdio.h>
// Defined in Go
extern void Print(void);
// C Main program
int cmain() {
printf("Hello from C\n");
Print();
}
print.go
package main
import "fmt"
import "C"
//export Print
func Print() {
fmt.Printf("Hello from Go\n")
}
Compile with go build, and produces this output when you run it
Hello from C
Hello from Go

AFAIK, you cannot compile a Go package to a shared library witg 'gc' ATM, it may change in the future. There might be some chance with 'gccgo' as 'libgo' (Go runtime, I suppose) is a shared library already. I guess the missing piece is then only to properly initialize the runtime, which normally a Go command handles automatically.
The gccgo expert is I.L. Taylor, he is reachable at the golang-nuts mailing list almost daily. I suggest to ask him directly.
PS: Other problems possibly include the interaction of the Go garbage collector and the (C++) process memory etc. Maybe I'm too optimistic and it is not at all feasible.

Related

Linkage of standard libraries in C++ code called from ASM

as I am developing my "OsDev" project, where I am learning a new stuff (for somebody who did not code in C/C++ for a long time due to web development it is kinda "new"). I figured out in the other thread, that calling a C++ function from ASM needs to have a extern "C" prefix but now I have problem with the lining of standard libraries as a for example cstdio etc. I stuck with this message.
kc.o: In function `kmain':
kernel.cpp:(.text+0x3e4): undefined reference to `strlen`
C++
#include <string.h>
#include <cstdio>
#include "inc/screen.h"
extern "C" void kmain()
{
clearScreen();
kernel_print((char*)"Hello Github! :-)", 0x04);
}
and if I try to use strlen() it won't link. (BTW. including screen.h is working for some reason).
Compiling script
nasm -f elf32 kernel.asm -o kasm.o
g++ -c kernel.cpp -o kc.o -lgcc -m32 -Wall -Wextra -O2
ld -m elf_i386 -T link.ld -o kernel.bin kasm.o kc.o
link.ld
OUTPUT_FORMAT(elf32-i386)
ENTRY(start)
SECTIONS
{
. = 0x100000;
.text : { *(.text) }
.data : { *(.data) }
.bss : { *(.bss) }
}
Thanks for any suggestions. :)
Your code cannot work as kernel's can't directly use shared libraries.
Why can't I use shared libraries directly in my kernel?
When an application is loaded by the operating system, all the required files are brought into its address space. This includes the executable file and any dynamic libraries (all ABI-conforming ELF applications will always link with a system library - the C Standard Library or just libc).
But while loading the kernel, only the original executable is loaded. Multiboot 2 (with GRUB bootloader) will allow you to load kernel-modules which can be dynamic libraries. But still, your kernel must know how to link itself and the kernel-modules in physical memory. To do so, you must implement a ELF parser and dynamic linker in your kernel.
Before implementing one, make sure your kernel is mature enough to systematically handle dynamic memory allocation, pagination, and other basic features.
How can I use the sweet features of libc?
Usually, you won't use all of the userspace functionality of libc. But things like memcpy, strlen, strcpyn and so on are absolutely necessary. You will have to implement these functions on your own, but the better part here is that, you can change the names of these functions. For example, if you prefere camelCase for function names, then you can also use function names like copyMemory, lengthOfString, etc.
https://github.com/SukantPal/Silcos-Kernel
I have built my own kernel, which has a few implementations of the required functions in KernelHost/Source/Util/CircuitPrimitive.cpp. You can look into that. Also, it has a full-fledged module linker. (KernelHost, ModuleFramework, etc. those parent folders contain separate kernel-module source code).
Make sure not to use the standard C headers in your kernel, as for now. Implement all required functions on your own, including printf

"no main" function for linking or execution in C++ [duplicate]

This question already has answers here:
How to change entry point of C program with gcc?
(4 answers)
Closed 5 years ago.
I am trying to compile a function (not called main) that can be integrated in another code or directly executed (after linking).
I try it one my mac, and work well.
I finally test it on Linux (CentOS and ubuntu). However, the task looks harder as expected on Linux.
The source code is the following one (just to explain the problem)
test.cpp:
#include <cstdio>
#ifdef __cplusplus
extern "C" {
#endif
int test(int argc, char const *argv[]);
#ifdef __cplusplus
}
#endif
int test(int argc, char const *argv[]) {
fprintf(stderr, "%s\n", "test");
return 0;
}
Compilation line on MacOS
g++ -c test.cpp -o test.o && g++ test.o -o test -e _test
and on Linux
g++ -c test.cpp -o test.o && g++ test.o -o test -e test
I try on my MacOS with clang, g++ and Intel compiler, all 3 works fine.
And I try with g++ and the Intel compiler on Linux, always, the same error.
usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status
Any advice, explanation or solution, on what I am doing wrong or missing would be very helpful.
Thanks
Edit:
Currently, I have a "define" to create a main, but if we have lots of function we are obligated to do two compilations each time (one for the function version and one for the execution) and make finally the code heavier.
Like discussed in this topic is there a GCC compiler/linker option to change the name of main?
To don't do a XY I inherited from a bunch of small programs that I want to put to gather, that it is easier to use (for remote execution ...). However, each one need to be able to be executed independently if needed, for debugging,... I hesitate, between using "execv" and just convert each main as a function. I probably take the bad chose.
Edit:
The final goal is to be able to have independent programs. But that we can call from an external software too.
Solution:
The solution looks to be, to a call to the main through a dlopen
You cannot do that (and even if it appears to work on MacOSX it is implementation specific and undefined behavior).
Linux crt0 is doing more complex stuff that what you think.
The C standard (e.g. n1570 for C11) requires a main function for hosted implementations (§5.1.2.2.1) :
The function called at program startup is named main. The implementation declares no prototype for this function.
And the C++ standard also requires a main and strongly requires some processing (e.g. construction of static data) to be done before main is running and after it has returned (and various crt0 tricks are implementing that feature on Linux).
If you want to understand gory details (and they are not easy!), study the ABI and the (free software) source code of the implementation of the crt0.
I am trying to compile a function (not called main) that can be integrated in another code
BTW, to use dynamically some code (e.g. plug-ins) from another program, consider using the dynamic linker. I recommend using the POSIX compliant dlopen(3) with dlsym(3) on position-independent code shared libraries. It works on most Unix flavors (including MacOSX & Linux & Solaris & AIX). For C++ code beware of name mangling so read at least the C++ dlopen mini howto.
Read also the Program Library HowTo.
Problems with libraries, they cannot be executed, no ?
I don't understand what that means. You certainly can load a plugin then run code inside it from the main program dlopen-ing it.
(and on Linux, some libraries like libc.so are even specially built to also work as an executable; I don't recommend this practice for your own code)
You might take several days to read Drepper's How To Write Shared Libraries (but it is advanced stuff).
If you want to add some code at runtime, read also this answer and that one.
The final goal is to be able to have independent program. But that we can call from an external software too
You can't do that (and it would make no sense). However, you could have conventions for communicating with other running programs (i.e. processes), using inter-process communication such as pipe(7)-s and many others. Read Advanced Linux Programming first and before coding. Read also Operating Systems : Three Easy Pieces
The solution looks to be, to a call to the main through a dlopen
Calling the main function via dlopen & dlsym is forbidden by the C++ standard (which disallows using a pointer to main). The main function has a very specific status and role (and is compiled specially; the compiler knows about main).
(perhaps calling main obtained by dlsym would appear to work on some Linux systems, but it certainly is undefined behavior so you should not do that)

Writing a plugin system?

After many hours of research I have turned up nothing, so I turn to you good folks in hopes of a solution. I am going to be writing a bot in c++, and at some point would like to make a plugin system for it. Now I know I could just write a scripting language for it, however, I know its possible to just write an api and have the program link to that dynamically at run time. My question is, how do i get that dynamic linkage (like what hexchat has for its plugins)? Are there any elegant solutions, or at least theories on the typical design?
On Linux and Posix systems, you want to use dlopen(3) & dlsym (or some libraries wrapping these functions, e.g. Glib from GTK, Qt, POCO, etc...). More precisely,
Build a position independent code shared library as your plugin:
gcc -fPIC -Wall -c plugin1.c -o plugin1.pic.o
gcc -fPIC -Wall -c plugin2.c -o plugin2.pic.o
Notice that if the plugin is coded in C++ you'll compile it with g++ and you should declare the plugin functions as extern "C" to avoid name mangling.
Then link your plugin as
gcc -shared -Wall plugin1.pic.o plugin2.pic.o -o plugin.so
You may add dynamic libraries (e.g. a -lreadline at end of command above if your plugin wants GNU readline).
At last, call dlopen with a full path in your main program, e.g.
void* dlh = dlopen("./plugin.so", RTLD_NOW);
if (!dlh) { fprintf(stderr, "dlopen failed: %s\n", dlerror());
exit(EXIT_FAILURE); };
(often dlh is a global data)
Then use dlsym to get the function pointers. So declare their signature in some header included both by program and plugin code like
typedef int readerfun_t (FILE*);
declare some (often) global function pointers
readerfun_t* readplugfun;
and use dlsym on the plugin handle dlh:
readplugfun = (readerfun_t*) dlsym(dlh, "plugin_reader");
if (!readplugfun) { fprintf (stderr, "dlsym failed: %s\n", dlerror());
exit(EXIT_FAILURE); };
Of course in your plugin source code (e.g. in plugin1.cc) you'll define
extern "C" int plugin_reader (FILE*inf) { // etc...
You might define some constructor (or destructor) functions in your plugin (see GCC function attributes); the would be called at dlopen (or dlclose) time. In C++ you should simply use static objects. (their constructor is called at dlopen time, their destructor is called at dlclose time; hence the name of the function attributes).
At the end of your program call
dlclose(dlh), dlh = NULL;
In practice, you can do a lot (perhaps a million) of dlopen calls.
You generally want to link your main program with -rdynamic to let its symbols be visible from plugins.
gcc -rdynamic prog1.o prog2.o -o yourprog -ldl
Read Program Library HowTo & C++ dlopen mini HowTo & Drepper's paper: How to Write a Shared Library
The most important part is to define and document a plugin convention (i.e. "protocol"), that is a set (and API) of functions (to be dlsym-ed) required in your plugin and how to use them, in which order they are called, what is the memory ownership policy, etc. If you allow several similar plugins, you might have some well documented hooks in your main program which calls all the dlsym-ed functions of relevant dlopen-ed plugins. Examples: GCC plugins conventions, GNU make modules, Gedit plugins, ...

Calling Ada from C++ in Eclipse

I am trying to create a program that is completely hosted in Eclipse, starts in C++, and calls Ada. I have GNATBench loaded, and can run Ada programs without a problem. What I cannot do is have a C++ project call an Ada project.
After hunting around, I found and executed the code shown below using a make file.
http://www.pegasoft.ca/resources/boblap/book.html
I also found a post stating that my goal has been done.
http://blogs.windriver.com/parkinson/2009/10/yesterday-adacore-announced-the-release-of-gnatbench-231-its-ada-integrated-development-environment-eclipse-plugin-which.html
What else do I need to include to have C++ in Eclipse call Ada in Eclipse?
USING MAKE FILE:
$ c++ -c test.cc
$ gnatgcc -c test_subr
$ gnatbind -n test_subr
$ gnatgcc -c b~test_subr
$ gnatlink -o main test.o test_subr.ali --link=c++
$ ./main
 
CPP Code:
//main.cc
#include extern "C" void adainit(void);
#include extern "C" void adafinal(void);
#include extern "C" void ada_subroutine(void);
int main(int argc, char **argv)
{
puts("C++ main");
adainit();
ada_subroutine();
adafinal();
puts("C++ done");
return 0;
}
Ada Code:
package Test_Subr is
procedure Ada_Subroutine;
pragma export(CPP, Ada_Subroutine);
end Test_Subr;
with Ada.Text_IO;
use Ada.Text_IO;
package body Test_Subr is
procedure Ada_Subroutine is
begin
put("IN ADA");
end Ada_Subroutine;
end Test_Subr;
Have you tried using the External_Name parameter of the Export pragma? (IIRC, C++ linkages can get quite mangled.)
pragma Export
( Convention => CPP,
Entity => Ada_Subroutine,
External_Name => "Ada_Subroutine "
);
I don’t know Eclipse; but, how would you get a C++ project in Eclipse to call up another C++ project? or one written in C?
You might be able to get Eclipse to build the Ada as a library and invoke that from the C++?
In the general case, you need to use extern C on the C++ side and pragma exprort (C, .. on the Ada side to get both linkages (parameter passing schemes) the same. However, if you are using gcc for both Ada and C++ then you could use pragma export (CPP instead.
There is one more nit you have to be aware of. If your "main" (the program's entry point) is not written in Ada, then you will need to manually invoke Ada's elaboration process (via the routine adainit()) once before calling anything. Likewise you should in most cases call adafinal() before exiting your program.

Import C++ function into Python program

I'm experimenting with python functions right now. I've found a way to import python functions into c/c++ code, but not the other way around.
I have a c++ program written and it has a certain function in it. I'd like to "import" the compiled c++ program into my python script and call the c++ function.
For simplicity, say the c++ function is as simple as:
int square(x)
{
return x*x;
}
and the compiled program is named Cprog.
I'd like my python script to be something like:
import Cprog
print Cprog.square(4)
Is this possible? I've searched the internet to no avail and I'm hoping one of you gurus might have a clever way of going about this...
Here is a little working completion of the simple example above. Although the thread is old, I think it is helpful to have a simple all-embracing guide for beginners, because I also had some problems before.
function.cpp content (extern "C" used so that ctypes module can handle the function):
extern "C" int square(int x)
{
return x*x;
}
wrapper.py content:
import ctypes
print(ctypes.windll.library.square(4)) # windows
print(ctypes.CDLL('./library.so').square(4)) # linux or when mingw used on windows
Then compile the function.cpp file (by using mingw for example):
g++ -shared -c -fPIC function.cpp -o function.o
Then create the shared object library with the following command (note: not everywhere are blanks):
g++ -shared -Wl,-soname,library.so -o library.so function.o
Then run the wrapper.py an the program should work.
If you build your program as a shared library/DLL, you could use ctypes to call it.
import ctypes
print ctypes.windll.cprog.square(4) # windows
print ctypes.CDLL('cprog.so').square(4) # linux
You need to create a python module with that function in it. There are three main ways:
Using Swig - this reads your c code and creates a python module from it.
Hand coded using the python c api.
Using Boost::Python (often the easiest way).
This pdf covers 1 and 2. This page will tell you how to use Boost::Python.
You cannot (easily) use a function that is in a c/c++ program - it must be in a static library (which you can also link your c/c++ program against).
EDIT -
Cython Is also worth a mention.
You want to extend python with a C/C++ module. The following Python documentation is a good place to start reading: http://docs.python.org/extending/extending.html
There are a lot of different ways to wrap C++ code to be used in Python. Most are listed on the Python wiki here.
I've found a decently easy way to automate it is to use py++ to automatically generate the wrappers, then compile the generated files. But this is more appropriate for larger projects where wrapping everything by hand is just not very feasible and would cause a maintenence nightmare.