How to wrap 3rd party DLL for use in R? - c++

I need to use an existing 3rd party API that comes with a *.h and a *.dll file to load data into R. The functions provided by the dll are not callable directly, so I need to wrap them to call them from R. In order to familiarize myself with this, I made little example dll (based on the HOWTO from the MINGW page here, I have put the source code of the files at the end of the post). There is just one function in it that doubles an integer input. I can compile the dll just fine and also use it in a exe file, so it is functional. This is on Windows 10.
I am not sure how to correctly use this in R. I have created a package (named testwithdll2 ), placed the header file and the dll in "src", together with the wrapper function. When I try to compile the package, I get the follwing error messages with the undefined reference:
C:/Rtools/mingw_64/bin/gcc -I"C:/PROGRA~1/R/R-35~1.1/include" -DNDEBUG
-O2 -Wall -std=gnu99 -mtune=generic -c mydouble_c.c -o mydouble_c.o
C:/Rtools/mingw_64/bin/gcc -shared -s -static-libgcc -o testwithdll2.dll
tmp.def mydouble_c.o -LC:/PROGRA~1/R/R-35~1.1/bin/x64 -lR
mydouble_c.o:mydouble_c.c:(.text+0xc): undefined reference to `__imp_timestwo'
collect2.exe: error: ld returned 1 exit status
Any pointers on what might have gone wrong are greatly appreciated.
example_dll.h:
#ifndef EXAMPLE_DLL_H
#define EXAMPLE_DLL_H
#ifdef __cplusplus
extern "C" {
#endif
#ifdef BUILDING_EXAMPLE_DLL
#define EXAMPLE_DLL __declspec(dllexport)
#else
#define EXAMPLE_DLL __declspec(dllimport)
#endif
int EXAMPLE_DLL timestwo(int x);
#ifdef __cplusplus
}
#endif
#endif // EXAMPLE_DLL_H
example_dll.cpp:
#include <stdio.h>
#include "example_dll.h"
int timestwo(int x)
{
return 2 * x;
}
mydouble.c (in the src folder of the r package):
#include "example_dll.h"
void mydouble(int* a){
*a = timestwo(*a);
}
timestwo.R (wrapper function, in the R folder):
#' #useDynLib testwithdll2 mydouble
#' #export
timestwo <- function(n){
.C("mydouble",n )
n
}

I figured out what to do.
It was necessary to use a makevars file with the following lines:
MAKEVARS:
PKG_CPPFLAGS= -I.
PKG_LIBS= -L. -lexample_dll
It was also necessary to add the useDynlib call to the example_dll.dll in the namespace before the call to the testwithdll2.dll. This also meant that the .C call needed the PACKAGE parameter to be specified, so I had to change the r wrapper to:
timestwo.R
#' #useDynLib example_dll
#' #useDynLib testwithdll2
#' #export
timestwo <- function(n){
.C("mydouble",n, PACKAGE = "testwithdll2")[[1]]
}
Now everything works as expected.

Related

link cython module in a c++ program

Is it possible to build a cython module with some cdef functions and link the resulting shared library into a C++ program?
I tried a proof of concept:
cymod.pyx:
# distutils: language=c++
from libcpp.string cimport string
cdef public string simple_echo(string test_string):
return test_string
cpp_test.cpp:
#define PyMODINIT_FUNC void
#include <iostream>
#include "cymod.h"
int main(int argc, char const *argv[])
{
std::cout << simple_echo("test") << std::endl;
return 0;
}
setup.py:
from setuptools import setup, Extension
from Cython.Build import cythonize
setup(
name='cymod',
ext_modules=cythonize(
Extension(
"cymod", ["cymod.pyx"],
),
)
)
The cython module builds fine, but when I try to build the c++ code that will use the cython function I get:
$ g++ -L. -l:cymod.so cpp_test.cpp -o cpp_test
/tmp/cc48Vc2z.o: In function `main':
cpp_test.cpp:(.text+0x51): undefined reference to `simple_echo'
collect2: error: ld returned 1 exit status
Which is odd. The generated header file has it:
cymod.h:
/* Generated by Cython 0.29.1 */
#ifndef __PYX_HAVE__cymod
#define __PYX_HAVE__cymod
#ifndef __PYX_HAVE_API__cymod
#ifndef __PYX_EXTERN_C
#ifdef __cplusplus
#define __PYX_EXTERN_C extern "C"
#else
#define __PYX_EXTERN_C extern
#endif
#endif
#ifndef DL_IMPORT
#define DL_IMPORT(_T) _T
#endif
__PYX_EXTERN_C std::string simple_echo(std::string);
#endif /* !__PYX_HAVE_API__cymod */
/* WARNING: the interface of the module init function changed in CPython 3.5. */
/* It now returns a PyModuleDef instance instead of a PyModule instance. */
#if PY_MAJOR_VERSION < 3
PyMODINIT_FUNC initcymod(void);
#else
PyMODINIT_FUNC PyInit_cymod(void);
#endif
#endif /* !__PYX_HAVE__cymod */
and I see my function in cymod.so:
nm cymod.so| grep simple_echo
0000000000001e50 T simple_echo
NOTE: I realize that to actually get this working I'll need to link against the python libraries and initialize the interpreter etc. I left that out to make this a tad shorter and I get the same error either way.
The short answer is that I was putting the -l argument too early in the compilation command. It is also important to handle the library lookup path. The simplest way is to use rpath. I set the rpath to the directory that the executable is in, i.e., .
Additionally, it is necessary to link against the python libraries and set the include and library paths. These can be determined at compile time by using the output of the python-config utility. Here is the compilation command that ultimately did the trick:
g++ cpp_test.cpp -o cpp_test -L. -l:cymod.so $(python-config --libs) $(python-config --includes) $(python-config --cflags) -Wl,-rpath,"\$ORIGIN"
I also updated the c++ file to #include "Python.h" and added calls to Py_Initialize(), Py_Finalize(), and initcymod():
#include <iostream>
#include "Python.h"
#include "cymod.h"
int main(int argc, char *argv[])
{
Py_Initialize();
initcymod();
std::cout << simple_echo("test") << std::endl;
Py_Finalize();
return 0;
}
NOTE: the call to initcymod() is necessary, but python2 specific. On python3 you should call PyImport_AppendInittab("cymod", PyInit_cymod); prior to Py_Initialize(). The cymod part is the module name, substitute your module name.
Thanks to #ead for the informative link to the docs on this topic https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#using-cython-declarations-from-c and his answer to a related question https://stackoverflow.com/a/45424720/2069572
While reading the linked docs, I came across this:
Note On some operating systems like Linux, it is also possible to first build the Cython extension in the usual way and then link against the resulting .so file like a dynamic library. Beware that this is not portable, so it should be avoided.
So it turns out that you should not do what I was trying to do.
Instead, what I should have done was run:
cython --cplus cymod.pyx
And then compiled cpp_test.cpp with the generated cymod.cpp file.
No need to link the cython shared library, and it turns out that it is not a good idea to do so.

C++ dll using a method in a dll

I'm trying to get a simple c++ program to use a method in a dll. I've been receiving a variety of errors as I've adjusted the code and have been stuck mostly, as in the code posted below, with "undefined reference to" the method. The code below is being compiled as follows.
g++ -c testdll.cpp
g++ -shared -o testdll.dll testdll.o
g++ -o test test.cpp -L./ -ltestdll
error
g++ -o test test.cpp -L./ -ltestdll
C:\Users\ROGERF~1\AppData\Local\Temp\cca9YhFn.o:test.cpp:(.text+0x53): undefined
reference to `__imp__ZN7TestDLL9writeDataESs'
collect2.exe: error: ld returned 1 exit status
I have no idea why directory C:\Users\ROGERF~1\AppData\Local\Temp\ is involved in the process. That showed up after I started using code from the Microsoft website in the header file. Previously, I was just getting undefined reference to 'writeData'
testdll.cpp
#include <stdio.h>
#include <string>
using namespace std;
class TestDLL {
public:
string data1;
public: void writeData (string s) {
printf ("%s \n", s.c_str());
}
};
TestDLL.h
#ifndef TESTDLL_H
#define TESTDLL_H
#ifdef TRADITIONALDLL_EXPORTS
#define TRADITIONALDLL_API __declspec(dllexport)
#else
#define TRADITIONALDLL_API __declspec(dllimport)
#endif
#ifdef __cplusplus
extern "C"
{
#endif
class TestDLL {
public:
std::string data1;
public:
TRADITIONALDLL_API void writeData (std::string);
};
#ifdef __cplusplus
}
#endif
#endif // TESTDLL_H
test.cpp
#include <string>
#include "TestDLL.h"
using namespace std;
class TestDLL;
int
main () {
TestDLL testdll;
testdll.writeData ("success");
}
Extended explanation: I've focused this down to something easy to post and hopefully easy for someone to answer. I was a C programmer back at the dawn of the PC era but haven't done much with C++ ever or C since then. I've been a Java programmer for quite some time (along with web stuff). Right now, I'm dealing with an existing program that can be extended with dlls, and the dlls need to be connected to a system written in Java. I've done the first step in JNI, so I have Java connected to a single dll. But the architecture needs to be:
Existing C application - dll extensions - dll for JNI - Java system
with communication both ways.
What happens if you add the two following lines to testdll.cpp:
#define TRADITIONALDLL_EXPORTS 1
#include "TestDLL.h"
I suspect that what's happening is that you're not doing that, so GCC doesn't know to compile TestDLL::writeData() with DLL export linkage.

Linking C compiled static library to C++ Program

I tried to link a static library (compiled with gcc) to a c++ program and I got 'undefined reference'. I used gcc and g++ version 4.6.3 on a ubuntu 12.04 server machine. For example, here is the simple library file for factorial method:
mylib.h
#ifndef __MYLIB_H_
#define __MYLIB_H_
int factorial(int n);
#endif
mylib.c
#include "mylib.h"
int factorial(int n)
{
return ((n>=1)?(n*factorial(n-1)):1);
}
I created object for this mylib.c using gcc:
gcc -o mylib.o -c mylib.c
Again the static library was created from the object file using AR utility:
ar -cvq libfact.a mylib.o
I tested this library with a C program (test.c) and C++ program (test.cpp)
Both C and C++ program have the same body:
#include "mylib.h"
int main()
{
int fact = factorial(5);
return 0;
}
Assuming static library libfact.a is available in /home/test directory, I compiled my C program without any issues:
gcc test.c -L/home/test -lfact
However while testing C++ program, it threw a link error:
g++ test.cpp -L/home/test -lfact
test.cpp:(.text+0x2f): undefined reference to `factorial(int)'
collect2: ld returned 1 exit status
I even tried adding extern command in test.cpp:
extern int factorial(int n) //added just before the main () function
Still the same error.
Can someone tell me what I am wrong here?
Is there anything I missed while creating the static library?
Do I have to add anything in my test.cpp to make it work?
The problem is that you haven't told your C++ program that factorial is written in C. You need to change your test.h header file. Like this
#ifndef __MYLIB_H_
#define __MYLIB_H_
#ifdef __cplusplus
extern "C" {
#endif
int factorial(int n);
#ifdef __cplusplus
}
#endif
#endif
Now your header file should work for both C and C++ programs. See here for details.
BTW names containing a double underscore are reserved for the compliler (so are names starting with an underscore and a capital letter) so #ifndef __MYLIB_H_ is illegal strictly speaking. I would change to #ifndef MYLIB_H #define MYLIB_H
While the accepted answer is absolutely correct, I thought I'd just add an observation. Some editors have trouble with the open / close brace, and will indent the entire extern "C" scope in the header. If mylib.h is a key header for a library, you might consider:
#if defined (__cplusplus)
#define _MYLIB_INIT_DECL extern "C" {
#define _MYLIB_FINI_DECL }
#else
#define _MYLIB_INIT_DECL
#define _MYLIB_FINI_DECL
#endif
All other headers in mylib library, e.g., mylib_aux.h, can be of the form:
#ifndef _MYLIB_AUX_H
#define _MYLIB_AUX_H
#include <mylib.h>
_MYLIB_INIT_DECL
... header content ...
_MYLIB_FINI_DECL
#endif /* _MYLIB_AUX_H */
Obviously, the names I'm using are arbitrary, but for multiple library headers, this approach has been useful to me.

Trouble linking c++ exe against C library (GNU)

I've created some simple basic code to illustrate my problem.
header.h:
#ifdef __cplusplus
# define API extern "C"
#else
# define API
#endif
void callback();
API void libFunction();
testlib.c:
#include "header.h"
void libFunction()
{
Callback();
}
I compile this as a static library like so:
gcc -c testlib.c
ar rsc libtest.a testlib.o
Then my sample c++ code is
main.cpp:
extern "C"{
#include <lib/header.h>
}
#include <stdio.h>
main()
{
libFunction();
}
void Callback()
{
printf("Callback is called \n");
}
and I try to build my exe as so
g++ -I. -L. main.cpp -ltest
and get the following error
./lib/libtest.a(testlib.o): In function `libFunction':
testlib.c:(.text+0x7): undefined reference to `Callback'
collect2: ld returned 1 exit status
I have spent literally all day trying to figure out why. Can anyone please help?
If you want to call Callback from a C file, it needs to be built extern "C" in your C++ file - otherwise C++ name mangling will cause the symbols to not line up. You need to change the definition of Callback() in main.cpp to be:
extern "C" void Callback()
You have a case mismatch, too. The prototype in your header says callback, but everywhere else you use Callback. On re-reading your code, I think just fixing this case mismatch will solve all your problems. I didn't see the extern "C" wrapper around the #include <lib/header.h> on first reading.

How to link app with static library + why this is not working

I have a problem. I wrote example code and I want to build it without the error:
main.cpp(.text+0x5): undefined reference to `test()'
Library
test1.c
#include <stdlib.h>
void test()
{
puts("DziaƂa");
}
test1.h
#ifndef TEST1_H
#define TEST1_H
extern void test();
#endif
makefile
all:
gcc -c ./src/test1.c -o ./lib/test1.o
ar rcs ./lib/libtest1.a ./lib/test1.o
Program
main.cpp
#include <test1.h>
int main()
{
test();
return 0;
}
makefile
all:
g++ -static -I../test1/include -L../test1/lib ./src/main.cpp -o ./build/MyApp -ltest1
What am I doing wrong?
You are compiling a C code function, but you are expecting to link a C++ function.
Because of 'type safe linkage', the function you provide is not the function that the C++ code calls.
Either in test1.h use:
#ifdef __cplusplus
extern "C" {
#endif
extern void test1(void);
#ifdef __cplusplus
}
#endif
Or:
Compile the function with the C++ compiler.
The C++ compiler will mangle the symbol names to provide type-safe linkage (a term which you should be able to search for via your preferred search engine).
The 'compiler' - actually the linker - is looking for a function with a C++ mangled name representing the C++ function with the signature 'void test1(void);'.
For example (but remember - different compilers deliberately mangle things differently), G++ 4.2.1 on MacOS X 10.6.2 generates a symbol '__Z5test1v' for the function; GCC generates a symbol '_test1'. Clearly, when the linker is looking for '__Z5test1v', the symbol '_test1' is not going to be used - it is not spelled the same. This is a good thing.
You can use 'nm -g' on the object file for the main program to see what it is looking for, and on the object file in the library to see what it is providing. And, given that the spellings are different, that is why the loader does not pick up the library function - it is looking for something with a different name.
You are calling a C function from a C++ function. The naming between the two is different (C++ mangles names to include parameter information).
Change the header file to look like this:
#ifdef __cplusplus
extern "C" {
#endif
extern void test();
#ifdef __cplusplus
}
#endif
This will tell the compiler that the function follows the C naming/calling convention.