SWIG wrapper of C++ library for Python - submodules - c++

I'm trying to wrap a C++ library into a Python3 interface using SWIG, and there is a problem I can't quite fix. This library has several namespaces, and I would like to make them modules of the library when wrapped in Python. Assume the following minimal example:
lib_class.hpp lib_class.cpp
lib_ops.hpp lib_ops.cpp
io_ops.hpp io_ops.cpp
The files lib_class define a very small class:
lib_class.hpp
#pragma once
namespace lib {
class dummy {
private:
int a;
public:
dummy();
dummy(int t_a);
~dummy();
void asdf();
};
}
lib_class.cpp
#include "lib_class.hpp"
namespace lib {
dummy::dummy() {}
dummy::dummy(int t_a) : a(t_a) {}
dummy::~dummy() {}
void dummy::asdf() { a = 3; }
}
The files lib_ops.hpp and lib_ops.cpp define a only one function:
lib_ops.hpp:
#pragma once
namespace lib {
void lib_operation();
}
lib_ops.cpp:
#include "lib_ops.hpp"
#include <iostream>
using namespace std;
namespace lib {
void lib_operation() {
cout << "LIBRARY TOP LEVEL" << endl;
}
}
and, finally, the files io_ops.hpp io_ops.cppdefine another function, this time within the namespacelib::io```:
io_ops.hpp
#pragma once
#include "lib_class.hpp"
namespace lib {
namespace io {
void io_operation(dummy& a);
}
}
io_ops.cpp
#include "io_ops.hpp"
#include <iostream>
using namespace std;
namespace lib {
namespace io {
void io_operation(dummy& a) {
cout << "LIBRARY SUBMODULE" << endl;
a.asdf();
}
}
}
I would like to wrap these files into a Python interface so that I can:
import lib
d = lib.dummy(10)
lib.ioop.io_operation(d)
lib.lib_operation()
In other words, I would like the organization of the Python wrapper to be:
lib.dummy # class
lib.lib_operation # function
lib.ioop # submodule
lib.ioop.io_operation # function within submodule
I've written the following *.i files:
lib.i:
%module lib
%import ioop.i
%{
#include "lib_ops.hpp"
#include "lib_class.hpp"
%}
%include "lib_ops.hpp"
%include "lib_class.hpp"
ioop.i:
%module ioop
%{
#include "io_ops.hpp"
using namespace lib;
%}
%include "io_ops.hpp"
This compiles without errors with:
g++ -c -fPIC io_ops.cpp
g++ -c -fPIC lib_ops.cpp
g++ -c -fPIC lib_class.cpp
swig -c++ -python -py3 lib.i
swig -c++ -python -py3 ioop.i
g++ -fPIC -c lib_wrap.cxx -I /usr/include/python3.6
g++ -fPIC -c ioop_wrap.cxx -I /usr/include/python3.6
g++ -fPIC -shared -o _lib.so lib_wrap.o lib_ops.o lib_class.o
g++ -fPIC -shared -o _ioop.so ioop_wrap.o io_ops.o lib_class.o
however, the python script above gives the following error:
Traceback (most recent call last):
File "test.py3", line 5, in <module>
lib.ioop.io_operation(d)
File "/home/lluis/Desktop/example.i/ioop.py", line 66, in io_operation
return _ioop.io_operation(a)
TypeError: in method 'io_operation', argument 1 of type 'dummy &'
Although I managed to "insert" the namespace lib::io into the "main" module lib as a submodule ioop, it looks as though it didn't know anything about the class lib::dummy.
Is it possible to do this? If so, how can I do it?
Thanks to all of you for your time (and sorry for such a lengthy post).

As #Flexo pointed out in one of the comments, the file ioops.i needs an %import. The correct contents of the file are:
%module ioop
%import lib.i
%{
#include "io_ops.hpp"
using namespace lib;
%}
%include "io_ops.hpp"

Related

Error: module declaration must occur at the start of the translation unit export module occupati

I am a beginner to C ++ and I am practicing with structures. Every time I try to compile the program the compiler gives me this error:
FAILED: CMakeFiles/jacoProject.dir/main.cpp.o
/Library/Developer/CommandLineTools/usr/bin/c++ -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk -std=gnu++2a -MD -MT CMakeFiles/jacoProject.dir/main.cpp.o -MF CMakeFiles/jacoProject.dir/main.cpp.o.d -o CMakeFiles/jacoProject.dir/main.cpp.o -c /Users/jacopodosio/CLionProjects/jacoProject/main.cpp
In file included from /Users/jacopodosio/CLionProjects/jacoProject/main.cpp:3:
/Users/jacopodosio/CLionProjects/jacoProject/occupati.ixx:1:8: error: module declaration must occur at the start of the translation unit
export module occupati;
The code in the files is shown below.
main.cpp
#include <iostream>
#include "occupati.ixx"
#include "test.h"
using std::cout;
using std::endl;
using std::cin;
int main()
{
Employes employeesCompany;
employeesCompany.initialEmployee = 'f';
}
occupati.ixx
export module employes;
export struct Employes
{
char initialEmployee {};
char ultimaLetteraDipendente {};
int occupatiTotali {};
int stipendio {};
};
Modules (in *.ixx files) aren't something you #include in other source files. Rather, you import the components that are exported by those module files.
So, try this for your "main.cpp":
#include <iostream>
import employes; // Imports from "occupati.ixx"
#include "test.h"
using std::cout;
using std::endl;
using std::cin;
int main()
{
Employes employeesCompany;
employeesCompany.initialEmployee = 'f';
}
Note that the "occupati.ixx" file should be part of the project/build.
Note: You have said, in comments, that you use CLion as your IDE/build-system. I don't use this, but this blog may offer some advice and/or remedies for how to use C++20 Modules in that environment.
You do not #include modules. #include means to copy-and-paste a file into this one. That's not how C++ modules work.
You import modules. Assuming your build is set up to handle C++ modules, you should import employes; at the top.

Cgo can't find standard libraries like <iostream>

I'm trying to include C++ code in my Go code, but isn't recognized.
I first thought that it considers it as C code and try (and fail) to compile as such, but removing the include line actually gives me c++ error troubleshooting like this
error: ‘cout’ is not a member of ‘std’
The code compiles correctly with g++.
I have tried to add the -lstdc++ LDLFLAG, and add the path to the lib in CXXFLAG but it doesn't change a thing.
I have made some other tests (and all fail) but this is the smallest one.
This is the c++ files
test.cpp
#include "test.hpp"
int test()
{
std::cout << "Hello, World! ";
return 0;
}
test.hpp
#include <iostream>
int test() ;
And this is my go file
//#cgo CXXFLAGS: -I/usr/lib/
//#cgo LDFLAGS: -L/usr/lib/ -lstdc++
//#include "test.hpp"
import "C"
func main() {
C.test()
}
I compile using go build but I have also tried to use env CGO_ENABLED CGO_CXXFLAGS="-std=c++11" go build (the env part is fish specific) and it returns the same error.
It's supposed to compile correctly, but instead I have iostream: No such file or directory.
EDIT :
I tried to add CFLAGS: -x c++ as suggested in the comments, the compiler searches at the right place, but I get another error invalid conversion from ‘void*’ to ‘_cgo_96e70225d9dd_Cfunc_test(void*)::<unnamed struct>*’ [-fpermissive] and I don't know if it's related to this new flafg
cgo makes it very easy to wrap C with Go, but C++ is a bit different. You have to extern "C" the functions that you want to make a function-name in C++ have 'C' linkage, otherwise the linker won't see the function. So, the actual problem is in the C++ header file. If you can't change the C++ code because it's a library, you may have to write wrappers (example).
This will compile:
.
├── test.cpp
├── test.go
└── test.hpp
test.hpp
#ifdef __cplusplus
extern "C" {
#endif
int test();
#ifdef __cplusplus
}
#endif
test.cpp
#include <iostream>
#include "test.hpp"
int test() {
std::cout << "Hello, World! ";
return 0;
}
test.go
package main
// #cgo CXXFLAGS: -I/usr/lib/
// #cgo LDFLAGS: -L/usr/lib/ -lstdc++
// #include "test.hpp"
import "C"
func main() {
C.test()
}
Put the files in the same folder,
run go build
Hello, World!

When using bazel to build TensorFlow C++ call model, .so file doesn't work

I want to use TensorFlow C++ api to call model and predict answers.
At fisrt, I clone the tensorflow repo
git clone --recursive https://github.com/tensorflow/tensorflow
Then I write the C++ code like below:
One code is a class to call TensorFlow api, the header file is like:
#ifndef _DEEPMODEL_H_
#define _DEEPMODEL_H_
#include <iostream>
#include <string>
#include <vector>
#include "tensorflow/core/public/session.h"
#include "tensorflow/core/protobuf/meta_graph.pb.h"
#include "tensorflow/cc/client/client_session.h"
#include "tensorflow/cc/ops/standard_ops.h"
#include "tensorflow/core/framework/tensor.h"
using namespace std;
using namespace tensorflow;
class DeepModel{
public:
DeepModel(const string graph_path, const string checkpoint_path);
virtual ~DeepModel();
bool onInit();
void unInit();
vector<float> predict(vector<vector<float>>& x, string input_name, string output_name);
private:
string graph_path;
string checkpoint_path;
MetaGraphDef graph_def;
Session* my_sess;
};
#endif
After this, I write a simple encapsulation code. I want to compile a .so, and use the .so without tensorflow source code in the future. And my encapsulation code is like below:
#ifndef _MODEL_HELPER_H_
#define _MODEL_HELPER_H_
#include <vector>
#include <string>
using namespace std;
class ModelHelper{
public:
ModelHelper(const string graph_path, const string checkpoint_path);
virtual ~ModelHelper();
vector<float> predict(vector<vector<float> >& x, string input_name, string output_name);
private:
string graph_path;
string checkpoint_path;
};
#endif
I have write code to test above code, it works well. Then I want to compile .so using bazel.
My BUILD file is like below:
load("//tensorflow:tensorflow.bzl", "tf_cc_binary")
tf_cc_binary(
name = "my_helper.so",
srcs = ["model_helper.cc", "model_helper.h", "deepmodel.cc", "deepmodel.h"],
linkshared = 1,
deps = [
"//tensorflow/cc:cc_ops",
"//tensorflow/cc:client_session",
"//tensorflow/core:tensorflow"
],
)
then I rename model_helper.so to libmodel_helper.so, and write cpp code to test the .so file. And I want to compile the code, command is like this
g++ -std=c++11 test_so.cpp -L./ -lmy_helper -I./ -o my_helper
Then I meet the error:
.//libmy_helper.so: undefined reference to `stream_executor::cuda::ScopedActivateExecutorContext::~ScopedActivateExecutorContext()'
.//libmy_helper.so: undefined reference to `stream_executor::cuda::ScopedActivateExecutorContext::ScopedActivateExecutorContext(stream_executor::StreamExecutor*)'
.//libmy_helper.so: undefined reference to `tensorflow::DeviceName<Eigen::GpuDevice>::value[abi:cxx11]'
collect2: error: ld returned 1 exit status
I really don't know why. I can't use the .so alone?
You should reference libtensorflow_frameowork.so in your makefile. Just like code below:
g++ -std=c++11 test_so.cpp -L./ -lmy_helper -ltensorflow_framework -I./ -o my_helper
I guess bazel misses some source code in tensorflow into .so when compile my code.

Xcode won't detect files included from main in included file

here is the structure of my program:
// File: main.cpp
#include <iostream>
#include <math.h>
using namespace std;
#include "do.cpp"
int main()
{
doit();
}
// File: do.cpp
void doit()
{
cout<<sqrt(2)<<endl;
}
When I do
g++ main.cpp
Everything goes fine. Now, if I open this as an Xcode project (I have chosen "Command line utility" as project type) and try to just build and run, it keeps saying:
Use of undeclared identifier sqrt
Unknown type name 'ostream'
What should I be supposed to do? Did I do something wrong?
Thank you very much!
Matteo
Don't do this:
#include "do.cpp"
but instead put that "do.cpp" file in the same Xcode project, alongside your main.m or main.cpp file.
And when you want to build from the command line, you can do:
g++ main.cpp do.cpp -o mytesttool
which would create the command line tool named "mytesttool".
The explanation is quite simple actually. You probably added both files to the project. Xcode tries to compile each file into an object file and then link them together.
When it tries to compile do.cpp it doesn't find the definition of cout because iostream is not included and neither math.h for sqrt, as part of do.cpp.
That file compiles fine when compiled as part of main.cpp, because it is included in the file and it finds iostream and math.h and also the using declaration.
Anyway if you remove do.cpp from the project (just the reference) everything should compile as expected.
The right way without a header file
// File: main.cpp
void doit(); // declare the function
int main()
{
doit();
}
// File: do.cpp
#include <iostream>
#include <math.h>
using namespace std;
void doit()
{
cout<<sqrt(2)<<endl;
}
The right way with a header file
// File do.h
#ifndef __DO_H_
#define __DO_H_
void doit();
#endif // __DO_H_
// File: main.cpp
#include "do.h"
int main()
{
doit();
}
// File: do.cpp
#include <iostream>
#include <math.h>
#include "do.h"
using namespace std;
void doit()
{
cout<<sqrt(2)<<endl;
}
When making the new file, I forgot to de-check the "target" selection, so that when it tried to build the project it tried to build all the single files and then link them together. By disabling the "target", I got it to work.

Dynamic library using static library in c++ name mangling error

I am trying to create a dynamic(.so) wrapper library along mongoDB c++ driver. There is no problem with the compilation but when I test it in a C++ sample program i get the error
undefined symbol: _ZN5mongo18DBClientConnection15_numConne
which i assume has something to do with name mangling issues.
I compiled the library as
g++ -fPIC -shared mongoquery.cpp -I/pathto/mongodriver -lmongoclient -lboost_thread-mt -lboost_filesystem -lboost_program_options -o libmongoquery.so
Here's the program I am using for testing:
#include <iostream>
#include <dlfcn.h>
#include "mongoquery.hpp"
using namespace std;
int main()
{
void *lib_handle;
int (*fn)(int *,string);
lib_handle=dlopen("./libmongoquery.so",RTLD_NOW);
if(!lib_handle)
{
cerr<<"Error"<<dlerror();
return 1;
}
fn=(int (*)(int *,string))dlsym(lib_handle,"count_query");
string q="{}";
int n;
(*fn)(&n,q);
cout<<n;
dlclose(lib_handle);
return 0;
}
the header mongoquery.hpp contains
#include <iostream>
#include <client/dbclient.h>
#define HOST "localhost"
#define COLLECTION "test.rules"
using namespace mongo;
using namespace std;
class mongoquery
{
private:
string q;
mongo::DBClientConnection c;
public:
mongoquery(string);
int result_count();
};
int count_query(int *,string);
Thanks
The answer can be followed from this question
Dynamic library uses statics libraries, undefined symbols appears
Added for achival purpose