I have a big c++ program in a single .cpp file which defines a lot of classes(interdependent of each other) and finally runs a main function. Now I am interested only in using one of this classes in python, specifically one method of this class which accepts 5 floats as inputs and outputs one float. I am trying to find the simplest method to achieve this. After not having success with boost:python(mainly because of installation issues) I have come to Cython which in the current version supports C++. I could successfully run the Rectangle example given in the Cython tutorial but I can't get how to proceed and adapt this to my case where I don't need a so complicated .pyx file, and where I don't have a .h file. Can somebody explain me in simple words what should I write in setup.py and in the .pyx file if my .cpp file has for example the structure:
...
class Nuclei {
public:
...
double potential(float,float,float,float,float);
...
private:
...
};
...
If all you are looking to do is call a single function, Extending Python With C/C++ is probably the simplest approach. This page provides a good example.
The relevant setup.py code in that example is
from distutils.core import setup, Extension
module1 = Extension('demo',
define_macros = [('MAJOR_VERSION', '1'),
('MINOR_VERSION', '0')],
include_dirs = ['/usr/local/include'],
libraries = ['tcl83'],
library_dirs = ['/usr/local/lib'],
sources = ['demo.c'])
setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
author = 'Martin v. Loewis',
author_email = 'martin#v.loewis.de',
url = 'http://docs.python.org/extending/building',
long_description = '''
This is really just a demo package.
''',
ext_modules = [module1])
If the C++ code you want to call is in demo.c, it could be used with import demo in Python.
Note that it is not nearly that simple- you'll be creating a function that takes a PyObject * with arguments and returns a PyObject *, and a lot can be said about how those are constructed. Take a look at the pages linked to above- they are filled with examples.
Related
I'm using Python 2.6 and PyGTK 2.22.6 from the all-in-one installer on Windows XP, trying to build a single-file executable (via py2exe) for my app.
My problem is that when I run my app as a script (ie. not built into an .exe file, just as a loose collection of .py files), it uses the native-looking Windows theme, but when I run the built exe I see the default GTK theme.
I know that this problem can be fixed by copying a bunch of files into the dist directory created by py2exe, but everything I've read involves manually copying the data, whereas I want this to be an automatic part of the build process. Furthermore, everything on the topic (including the FAQ) is out of date - PyGTK now keeps its files in C:\Python2x\Lib\site-packages\gtk-2.0\runtime\..., and just copying the lib and etc directories doesn't fix the problem.
My questions are:
I'd like to be able to programmatically find the GTK runtime data in setup.py rather than hard coding paths. How do I do this?
What are the minimal resources I need to include?
Update: I may have almost answered #2 by trial-and-error. For the "wimp" (ie. MS Windows) theme to work, I need the files from:
runtime\lib\gtk-2.0\2.10.0\engines\libwimp.dll
runtime\etc\gtk-2.0\gtkrc
runtime\share\icons\*
runtime\share\themes\MS-Windows
...without the runtime prefix, but otherwise with the same directory structure, sitting directly in the dist directory produced by py2exe. But where does the 2.10.0 come from, given that gtk.gtk_version is (2,22,0)?
Answering my own question here, but if anyone knows better feel free to answer too. Some of it seems quite fragile (eg. version numbers in paths), so comment or edit if you know a better way.
1. Finding the files
Firstly, I use this code to actually find the root of the GTK runtime. This is very specific to how you install the runtime, though, and could probably be improved with a number of checks for common locations:
#gtk file inclusion
import gtk
# The runtime dir is in the same directory as the module:
GTK_RUNTIME_DIR = os.path.join(
os.path.split(os.path.dirname(gtk.__file__))[0], "runtime")
assert os.path.exists(GTK_RUNTIME_DIR), "Cannot find GTK runtime data"
2. What files to include
This depends on (a) how much of a concern size is, and (b) the context of your application's deployment. By that I mean, are you deploying it to the whole wide world where anyone can have an arbitrary locale setting, or is it just for internal corporate use where you don't need translated stock strings?
If you want Windows theming, you'll need to include:
GTK_THEME_DEFAULT = os.path.join("share", "themes", "Default")
GTK_THEME_WINDOWS = os.path.join("share", "themes", "MS-Windows")
GTK_GTKRC_DIR = os.path.join("etc", "gtk-2.0")
GTK_GTKRC = "gtkrc"
GTK_WIMP_DIR = os.path.join("lib", "gtk-2.0", "2.10.0", "engines")
GTK_WIMP_DLL = "libwimp.dll"
If you want the Tango icons:
GTK_ICONS = os.path.join("share", "icons")
There is also localisation data (which I omit, but you might not want to):
GTK_LOCALE_DATA = os.path.join("share", "locale")
3. Piecing it together
Firstly, here's a function that walks the filesystem tree at a given point and produces output suitable for the data_files option.
def generate_data_files(prefix, tree, file_filter=None):
"""
Walk the filesystem starting at "prefix" + "tree", producing a list of files
suitable for the data_files option to setup(). The prefix will be omitted
from the path given to setup(). For example, if you have
C:\Python26\Lib\site-packages\gtk-2.0\runtime\etc\...
...and you want your "dist\" dir to contain "etc\..." as a subdirectory,
invoke the function as
generate_data_files(
r"C:\Python26\Lib\site-packages\gtk-2.0\runtime",
r"etc")
If, instead, you want it to contain "runtime\etc\..." use:
generate_data_files(
r"C:\Python26\Lib\site-packages\gtk-2.0",
r"runtime\etc")
Empty directories are omitted.
file_filter(root, fl) is an optional function called with a containing
directory and filename of each file. If it returns False, the file is
omitted from the results.
"""
data_files = []
for root, dirs, files in os.walk(os.path.join(prefix, tree)):
to_dir = os.path.relpath(root, prefix)
if file_filter is not None:
file_iter = (fl for fl in files if file_filter(root, fl))
else:
file_iter = files
data_files.append((to_dir, [os.path.join(root, fl) for fl in file_iter]))
non_empties = [(to, fro) for (to, fro) in data_files if fro]
return non_empties
So now you can call setup() like so:
setup(
# Other setup args here...
data_files = (
# Use the function above...
generate_data_files(GTK_RUNTIME_DIR, GTK_THEME_DEFAULT) +
generate_data_files(GTK_RUNTIME_DIR, GTK_THEME_WINDOWS) +
generate_data_files(GTK_RUNTIME_DIR, GTK_ICONS) +
# ...or include single files manually
[
(GTK_GTKRC_DIR, [
os.path.join(GTK_RUNTIME_DIR,
GTK_GTKRC_DIR,
GTK_GTKRC)
]),
(GTK_WIMP_DIR, [
os.path.join(
GTK_RUNTIME_DIR,
GTK_WIMP_DIR,
GTK_WIMP_DLL)
])
]
)
)
I am trying to write Python 2.7 code that will
Dynamically load a list / array of modules from a config file at startup
Call functions from those modules. Those functions are also designated in a config file (maybe the same config file, maybe a different one).
The idea is my code will have no idea until startup which modules to load. And the portion that calls the functions will have no idea which functions to call and which modules those functions belong to until runtime.
I'm not succeeding. A simple example of my situation is this:
The following is abc.py, a module that should be dynamically loaded (in my actual application I would have several such modules designated in a list / array in a config file):
def abc_fcn():
print("Hello World!")
def another_fcn():
print("BlahBlah")
The following is the .py code which should load abc.py (my actual code would need to import the entire list / array of modules from the config file). Both this .py file and abc.py are in the same folder / directory. Please note comments next to each statement.
module_to_import = "abc" #<- Will normally come from config file
fcn_to_call = "abc.abc_fcn" #<- Will normally come from config file
__import__(module_to_import) #<- No error
print(help(module_to_import)) #<- Works as expected
eval(fcn_to_call)() #<- NameError: name 'abc' is not defined
When I change the second line to the following...
fcn_to_call = "abc_fcn"
...the NameError changes to "name 'abc_fcn' is not defined".
What am I doing wrong? Thanks in advance for the help!
__import__ only returns the module specified, it does not add it to the global namespace. So to accomplish what you want, save the result as a variable, and then dynamically retrieve the function that you want. That could look like
fcn_to_call = 'abc_fcn'
mod = __import__(module_to_import)
func = getattr(mod, fcn_to_call)
func()
On a side note, abc is the name of name of the Abstract Base Classes builtin Python module, although I know you were probably just using this an example.
You should assign the returning value of __import__ to a variable abc so that you can actually use it as a module.
abc = __import__(module_to_import)
I'm trying to write a Python (2.7) library which loads certain classes at runtime. These classes contain a predefined set of methods.
My approach is to define a few Metaclasses which I work with in my library. I would for example define a "Navigation" Metaclass and work with it in the library. Then someone could write a class "Mainmenu" which contains some type of type definition that it is a "Navigation" plugin. And then the Library could use this class.
I am able to load modules and I'm able to write Metaclasses. My problem lies in combining these two things.
First there is the problem that I want the "plugin-classes" to be in a different (configurable) folder. So I can not do:
__metaclass__ = Navigation
because the Navigation class is part of my library and won't be there in the plugin-folder...
How could I solve the Problem of telling the type that the plugin is for? (Navigation, content.... e.g)
EDIT2: I solved the following problem. I found out that I can just ask the module to give me a dict.
My first problem still exists though
EDIT:
I managed registering and loading "normal" classes with a registry up until the following point:
from os import listdir
from os.path import isfile, join
import imp
class State:
registry = {}
def register_class(self,target_class):
self.registry[target_class.__name__] = target_class
print target_class.__name__+" registered!"
def create(self,classname):
tcls = self.registry[classname]
print self.registry[classname]
return tcls()
s = State()
mypath = """C:\metatest\plugins"""
files = [f for f in listdir(mypath) if isfile(join(mypath, f))]
for f in files:
spl = f.split(".")
if spl[1] == "py":
a = imp.load_source(spl[0], mypath + """\\""" + f)
s.register_class(a)
The problem I have at the end now is, that "a" is the loaded module, so it is a module-object. In my case there is only one class in it.
How can I get a Class object from the loaded module, so I can register the class properly??
So - let's check your problem steping back on your current proposal.
You need a way to have plug-ins for a larger system - the larger system won't know about the plug-ins at coding time - but the converse is not true: your plugins should be able to load modules, import base classes and call functions on your larger system.
Unless you really have something so plugable that plug-ins can work with more than one larger system. I doubt so, but if that is the case you need a framework that can register interfaces and retrieve classes and adapter-implementations between different classes. That framework is Zope Interface - you should read the documentation on it here: https://zopeinterface.readthedocs.io/en/latest/
Much more down to earth will be a plug-in system that sacans some preset directories for Python files and import those. As I said above, there is no problem if these files do import base classes (or metaclasses, for the record) on your main system: these are already imported by Python in the running process anyway, their import in the plug-in will just make then show up as available on the plug-in code.
You can use the exact code above, just add a short metaclass to register derived classes from State - you can maketeh convention that each base class for a different plug-in category have the registry attribute:
class RegistryMeta(type):
def __init__(cls, name, bases, namespace):
for base in cls.__mro__:
if 'registry' in base.__dict__:
if cls.__name__ in base.registry:
raise ValueError("Attempting to registrate plug-in with the name {} which is already taken".format(cls.__name__))
base.registry[cls.__name__] = cls
break
super(RegistryMeta, cls).__init__(name, base, namespace)
class State(object):
__metaclass__ = RegistryMeta
registry = {}
...
(keep the code for scanning the directory and loading modules - just switch all directory separation bars to "/" - you still are not doing it right and is subject to surprises by using "\")
and on the plug-in code include:
from mysystem import State
class YourClassCode(State):
...
And finally, as I said in the comment : you should really check the possibility of using Python 3.6 for that. Among other niceties, you could use the __init_subclass__ special method instead of needing a custom metaclass for keeping your registries.
I have c++ code that has grown exponential. I have a number of variables (mostly Boolean) that need to be changed for each time I run my code (different running conditions). I have done this using the argument command line inputs for the main( int argc, char* argv[]) function in the past.
Since this method has become cumbersome (I have 18 different running conditions, hence 18 different argument :-( ), I would like to move to interfacing with python (if need be Bash ). Ideally I would like to code a python script, where I set the values of data members and then run the code.
Does anyone have a any pointer/information that could help me out? Better still a simple coded example or URL I could look up.
Edit From Original Question:
Sorry I don't think I was clear with my question. I don't want to use the main( int argc, char* argv[]) feature in c++. Instead of setting the variables on the command line. Can I use python to declare and initialize the data members in my c++ code?
Thanks again mike
Use subprocess to execute your program from python.
import subprocess as sp
import shlex
def run(cmdline):
process = sp.Popen(shlex.split(cmdline), stdout=sp.PIPE, stderr=sp.PIPE)
output, err = process.communicate()
retcode = process.poll()
return retcode, output, err
run('./a.out '+arg1+' '+arg2+' '+...)
Interfacing between C/C++ and Python is heavily documented and there are several different approaches. However, if you're just setting values then it may be overkill to use Python, which is more geared toward customising large operations within your process by farming it off to the interpreter.
I would personally recommend researching an "ini" file method, either traditionally or by using XML, or even a lighter scripting language like Lua.
You can use subprocess module to launch an executable with defined command-line arguments:
import subprocess
option1 = True
option2 = Frue
# ...
optionN = True
lstopt = ['path_to_cpp_executable',
option1,
option2,
...
optionN
]
lstopt = [str(item) for item in lstopt] # because we need to pass strings
proc = subprocess.Popen(lstrun, close_fds = True)
stdoutdata, stderrdata = proc.communicate()
If you're using Python 2.7 or Python 3.2, then OrderedDict will make the code more readable:
from collections import OrderedDict
opts = OrderedDict([('option1', True),
('option2', False),
]
lstopt = (['path_to_cpp_executable'] +
list(str(item) for item in opts.values())
)
proc = subprocess.Popen(lstrun, close_fds = True)
stdoutdata, stderrdata = proc.communicate()
With the ctypes module, you can call arbitrary C libraries.
There are several ways for interfacing C and C++ code with Python:
SWIG
Boost.Python
Cython
I can only advise to have a look at swig : using director feature, it allows to fully integrate C++ and python, including cross derivation from onle language to the other
I have implemented a class in C++. I want to use it with Python.
Please suggest step by step method and elaborate each step.
Somthing like this...
class Test{
private:
int n;
public:
Test(int k){
n=k;
}
void setInt(int k){
n = k;
}
int getInt(){
return n;
}
};
Now, in Python
>>> T1 = Test(12)
>>> T1.getInt()
12
>>> T1.setInt(32)
>>> T1.getInt()
32
Please suggest.How can I do this ?
NOTE: I would like to know manual way to do that. I don't want any third party library dependency.
Look into Boost.Python. It's a library to write python modules with C++.
Also look into SWIG which can also handle modules for other scripting languages. I've used it in the past to write modules for my class and use them within python. Works great.
You can do it manually by using the Python/C API, writing the interface yourself. It's pretty lowlevel, but you will gain a lot of additional knowledge of how Python works behind the scene (And you will need it when you use SWIG anyway).
ctypes is good. It is really easy to use, and it comes standard with Python. Unfortunately it can only talk to shared libraries (Unix) or DLLs (Windows) that have a C-style interface, which means you can't directly interface to a C++ object. But you could use a handle system where a handle refers to a particular object.
>>> getInt(h)
12
I think that is simple, easy to understand, and doesn't require extra libraries.
I would suggest you try SWIG or sip (KDE/PyQt).
SWIG link : http://www.swig.org/
SIP link: http://freshmeat.net/projects/python-sip/
These can be used to wrap C++ classes and provide a Pythonic interface to them.
This is a very old post, and probably the OP already found a solution to his problem, but since no one has yet provided a valid example, I will give it a try and maybe help someone like me with the same problem in hand.
Since the OP did not specify, I will restrict my answer to Windows, although a Linux solution would be straightforward. I will try to provide a minimal working example that reproduces the code in question.
Step 1) Start with the C++ code (a single file)
class Test{
private:
int n;
public:
Test(int k){
n=k;
}
void setInt(int k){
n = k;
}
int getInt(){
return n;
}
};
extern "C"
{
// include below each method you want to make visible outside
__declspec(dllexport) Test* init(int k) {return new Test(k);}
__declspec(dllexport) void setInt(Test *self, int k) {self->setInt(k);}
__declspec(dllexport) int getInt(Test *self) {return self->getInt();}
// Note: the '__declspec(dllexport)' is only necessary in Windows
}
Step 2) Compile the DLL shared library. E.g. from your working directory:
g++ -shared mytest.cpp -o libtest.dll
Step 3) Create the Python file with the wrapper:
# mytest.py
import ctypes
import platform
# From Python 3.8 onwards, there is a reported bug in CDLL.__init__()
mode = dict(winmode=0) if platform.python_version() >= '3.8' else dict()
lib = ctypes.CDLL('./libtest.dll', **mode)
class Test(object):
def __init__(self, val: int):
# Declare input and output types for each method you intend to use
lib.init.argtypes = [ctypes.c_int]
lib.init.restype = ctypes.c_void_p
lib.setInt.argtypes = [ctypes.c_void_p, ctypes.c_int]
lib.setInt.restype = ctypes.c_void_p
lib.getInt.argtypes = [ctypes.c_void_p]
lib.getInt.restype = ctypes.c_int
self.obj = lib.init(val)
def setInt(self, n):
lib.setInt(self.obj, n)
def getInt(self):
return lib.getInt(self.obj)
T1 = Test(12)
print(T1.getInt())
T1.setInt(32)
print(T1.getInt())
Step 4) Run the code
Below are my results when I run the script with 4 different Python versions:
PS D:\Desktop\mytest> py -3.7 .\mytest.py
12
32
PS D:\Desktop\mytest> py -3.8 .\mytest.py
12
32
PS D:\Desktop\mytest> py -3.9 .\mytest.py
12
32
PS D:\Desktop\mytest> py -3.10 .\mytest.py
Traceback (most recent call last):
File "D:\Desktop\mytest\mytest.py", line 7, in <module>
lib = ctypes.CDLL('./libtest.dll', **mode)
File "D:\Python\3102\lib\ctypes\__init__.py", line 374, in __init__
self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'D:\Desktop\mytest\libtest.dll' (or one of its dependencies). Try using the full path with constructor syntax.
In the last case, I have tried with the full path and still same error.
I'm running on Windows 10 64bit and compiling with g++ 8.1.0 from MinGW64
Comments:
My answer is based in the solutions provided here, here, and here
A common problem arises when the Python and compiler (g++) architectures does not match (i.e. 32 or 64 bit)