Very new to Rust.
I'm trying to build a python module in Rust using rust-cypthon. Currently, I can access cpython types but can't call py_module_initializer! which I believe is required to make the module run in python.
When compiling the code, changing the extension to .so and putting the file in with my python scripts and importing, I get this error:
ImportError: dynamic module does not define init function (initpyext_rust_onboard)
Cargo.toml
[lib]
crate-type = ["cdylib"]
[dependencies]
csv = "1.1.1"
serde = "1.0.99"
serde_derive="1.0"
serde_json= "1.0"
serde_yaml = "0.7.1"
[dependencies.cpython]
version = '0.3.0'
default-features = false
features = ["python27-sys", "extension-module-2-7"]
Dependencies imported to lib.rs
#[macro_use]
extern crate serde;
extern crate serde_derive;
#[macro_use]extern crate cpython;
use std::fs::File;
use std::collections::HashMap;
use std::borrow::Cow;
use std::error::Error;
use cpython::{Python, PyObject, PyResult, PyString, PyFloat};
use csv::DeserializeRecordsIter;
When compiling the code, changing the extension to .so and putting the file in with my python scripts
What are you renaming it from? The rust-cpython documentation says
On Mac OS, you will need to rename the output from *.dylib to *.so. On Windows, you will need to rename the output from *.dll to *.pyd.
and on Linux you shouldn't be renaming anything.
The problem is probably that you are not actually building a shared object. You have
[lib]
crate-type = ["dylib"]
but the rust-cpython readme says
[lib]
crate-type = ["cdylib"]
Mind the c! cdylib and dylib are different things in rust.
According to the cpython homepage, you need to enable the extension-module module feature, so try with:
[dependencies.cpython]
version = '0.3.0'
default-features = false
features = ["python27-sys", "extension-module-2-7"]
Related
I am trying to build a ns3 module and I got stuck on " undefined reference errors" .I have dealt with those in the past by specifying the location of the definitions through #include directives but now I am puzzled by the fact that the undefined references are coming from the shared libraries as shown on the output I am getting on this link:
http://www.blikoon.com/wp-content/uploads/2014/05/wafError-1024x575.png
Some search results pointed out that my problem might be related to the path to my library not being specified but I do have the
module.uselib='Xerces-C++
statement in the wscript file of my module
Is there a way I can get waf to make my compiler aware of the Xerces library location.Searching on my system ,I have found that xerces is installed in :
/usr/include/xercesc
I am building on the latest verion of ns3.19 and it uses waf1.7.13.
Thank you for your time.
I do not know about Xcerces, but this is how I have used libxml library. It should be fairly similar of other libraries. This is from NS-3.19, you can look at other modules that use external libraries such as the stats using SQL
In the wscript file of your module you should start like this:
import wutils
def configure(conf):
have_libxml2 = conf.check_cfg(package='libxml-2.0', atleast_version='2.6',
args=['--cflags', '--libs'], uselib_store='LIBXML2',
mandatory=False)
conf.env['ENABLE_LIBXML2'] = have_libxml2
conf.report_optional_feature("XmlIo", "XmlIo",
conf.env['ENABLE_LIBXML2'],
"library 'libxml-2.0 >= 2.7' not found")
I'm trying to generate Python bindings for some C++ code using SWIG.
It created some blah_wrap.cxx and blah.py files.
I then created this setup.py
from distutils.core import setup, Extension
ext = Extension('_ev3',
sources=[
'ev3_serial_wrap.cxx',
'ev3_serial.hpp'
'ev3_motor_wrap.cxx',
'ev3_motor.hpp'
'ev3_i2c_wrap.cxx',
'ev3_i2c.hpp'
'ev3_analog_wrap.cxx',
'ev3_analog.hpp'
],
language='c++',
)
setup (name = 'evpy',
version = '0.1',
author = "Pepijn de Vos",
description = """
An EV3 API.
""",
ext_modules = [ext],
packages=['evpy'],
)
But then I get
$ python3 setup.py build
running build
running build_py
running build_ext
building '_ev3' extension
error: unknown file type '.hpp' (from 'ev3_analog.hpp')
.hpp is a pretty standard C++ extensions right? Why not .cpp? I don't know, the author of the original code put the implementation in his header files.
You can use the parameter "include_dirs". See the documentation for Extension here:
http://docs.python.org/2/extending/building.html
http://docs.python.org/2/distutils/apiref.html#distutils.core.Extension
Are you sure header files are supposed to go in the sources argument, and not in another like headers?
Basically, .h and .hpp do the same jobs, try to change the extension to .h, your python script might not know .hpp files (which is not a shame)...
I am venturing into the land of creating C/C++ bindings for Python using pybindgen. I've followed the steps outlined under "Building it ( GCC instructions )" to create bindings for the sample files:
http://packages.python.org/PyBindGen/tutorial.html#a-simple-example
Running make produces a .so file. If I understand how .so files work, I should be able to import the classes in the shared object into Python. However, I'm not sure where to place the file and how to let Python know where it is. Additionally, do the original c/c++ source files need to accompany the .so file?
So far I've tried placing the file in /usr/local/lib and adding that path to DYLD_LIBRARY_PATH to the .bash_profile. When I try to import the module from within the Python interpeter an error is thrown stating that the module can not be found.
So, my question is: What needs to be done with the generated .so file in order for it to be used by a Python program?
Python looks for .so modules in the same directories where it searches python ones. So you have to install it as you would normal python module either somewhere that is on python's sys.path by default (/usr/share/python/site-lib or something like that—it'd distribution-dependent) or add the directory to PYTHONPATH environment variable.
It's python that's loading the module using dlopen, not the dynamic linker, so LD_LIBRARY_PATH (note, there is no DY) won't help you.
Same as all other Python modules. It must be within one of the locations given in sys.path.
I'm using distutils to build a Python extension module written in C++. The problem I have is that in order to compile the extension module, I need to link with a certain shared library. This requires setting an additional compiler flag. So, I searched through the Python docs and found out about the extra_compile_args property of the Extension object. So I tried the following:
from distutils.core import setup, Extension
module = Extension('test', sources = ['test.cpp'])
module.extra_compile_args = ['--std=c++0x', '-l mylib'];
setup(name = 'test', version = '1.0', ext_modules = [module])
This seems to compile, except when I import my module in Python it throws an ImportError exception due to an undefined symbol. So, apparently the library didn't link properly. So I tried writing a throw away C++ program which linked with the shared library, and it ran fine. Then I realized something really odd is going on with distutils, because if I add a compile argument that links to a bogus library name, distutils just compiles everything with no problem:
module.extra_compile_args = ['--std=c++0x', '-l some_fake_library'];
When I run setup.py build, the build runs with no errors!
So, what's going on here? How can I compile an extension module that requires linkage to a shared library?
There's actually a special option for that.
For example:
libraries=["rt"]
You leave off the option and lib parts.
One of the purposes of distutils is to make your code not dependent on a single compiler. Your "-l somelib" looks like it's meant to work with GCC (even though it should be "-lsomelib", no space). This is why you use the libraries option to the Extension class. Distutils will then pass the appropriate link command to whatever compiler it's using.
You can also look at the actual build commands distutils is using and execute them yourself to see what is going wrong.
I have a need to wrap an existing C++ library for use in Python. After reading through this answer on choosing an appropriate method to wrap C++ for use in Python, I decided to go with Py++.
I walked through the tutorial for Py++, using the tutorial files, and I got the expected output in generated.cpp, but I haven't figured out what to do in order to actually use the generated code as an extension I can import in Python. I'm sure I have to compile the code, now, but with what? Am I supposed to use bjam?
Py++ generates you syntax you use along with boost::python to generate python entry points in your app.
Assuming everything went well with Py++ you need to download the Boost framework, and add the boost include directory and the boost::python lib to your project then compile with the Py++ generated cpp.
You can use whatever build system you want for your project, but boost is built with bjam. You need to choose whether you want a static lib or a dynamic boost python lib then follow the instructions for building boost here.
If on windows, you need to change the extension on your built library from .dll to.pyd. And yes it needs to be a library project, this does not work with executables.
Then, place the pyd where the python on your machine can find it and go into python and execute import [Your-library-name] and hopefully everything will work.
One final note, the name given in generated.cpp in this macro:
BOOST_PYTHON_MODULE( -name- )
needs to be the exact name of your project, otherwise python will complain.
I just went through all this less than a month ago so I know about the confusion.
One thing I did to make my python extension very easy to use while building the library and testing, was to build boost::python and python myself in my build environment. That way the pyd ends up exactly where I want it and users do not need to install python in order to run with my extension. That may be overkill for what you are doing though.
Edit:
If you want your extension to be easily installed and compiled on a machine, check out python's setuptools. With just a few simple lines you can have python compile and install your package for you. One downside though is its not IDE compatible for those of us who like developing in visual studio.
The following answer was provided to me by Roman Yakovenko on the Python C++-sig mailing list; I'm posting it here, with minor edits, for the benefit of the Stack Overflow community.
I don't fully comprehend the answer yet, but I felt it points me in the right direction.
After you have generated the code, you have to compile it. For this purpose, you can use your favorite build system. I use bjam only to compile boost. After this, I prefer to use scons (on Windows and on Linux).
The following is an example of sconstruct file, which is used to compile one of the Py++ unittests (this is generated code too :-) ):
import sys
env = Environment()
if 'linux' not in sys.platform:
env['MSVS'] = {'VERSION': ''}
env['MSVS_VERSION'] = ''
Tool('msvc')(env)
t = env.SharedLibrary(
target=r'abstract_classes',
source=[r'/home/roman/language-binding/sources/pyplusplus_dev/unittests/temp/abstract_classes.cpp'],
LIBS=[r"boost_python"],
LIBPATH=[r"", r"/home/roman/include/libs"],
CPPPATH=[
r"/home/roman/boost_svn",
r"/usr/include/python2.6",
r"/home/roman/language-binding/sources/pyplusplus_dev/unittests/temp",
r"/home/roman/language-binding/sources/pyplusplus_dev/unittests/data",
r"/home/roman/boost_svn"
],
CCFLAGS=[ ],
SHLIBPREFIX='',
SHLIBSUFFIX='.so'
)
Since your code generator written in Python, you can continue where Py++ stops and generate your favorite "make" file. You can go even father. Py++ tests generate the code, compile, load the new module and test the functionality. All this is done in a single, stand alone process.
I wrote a small makefile with the following:
GNUmakefile:
PYTHON_INC=$(shell python-config --includes)
PYTHON_LIBS=$(shell python-config --libs)
BOOST_LIBS=-lboost_python
all:
g++ -W -Wall $(PYTHON_INC) $(PYTHON_LIBS) $(BOOST_LIBS) -fPIC -shared generated.cpp -o hw.so
and then loaded the created .so into ipython to play around with it:
In [1]: import hw
In [2]: a = hw.animal('zebra')
In [3]: a.name()
Out[3]: 'zebra'