This might be a trivial one, but I kind of stuck here.
I have the following setup:
entity.cpp/.hpp: containing my Entity class definition and implementation.
entity_wrap.cpp: my boost python wrapper file which I compile to entity.so
entity_test.cpp: a test file
What I'd like to do in entity_test.cpp is the following:
Py_SetProgramName(argv[0]);
Py_Initialize();
...
Entity* entity = new Entity;
globals["entity"] = entity;
I now get the following exception:
TypeError: No to_python (by-value) converter found for C++ type: Entity
Which is obvious since I do not load the conversion definition of my types. I now tried to load the entity.so with globals["entity_module"] = import("entity"); but I ran into this exception:
ImportError: No module named entity
I can load the module from a python shell as expected.
My question now is: how do I load the converters defined in entity_wrap.cpp?
Solution
As eudoxos stated, I have to ensure that the module I want to load is in the sys.path:
globals["sys"] = import("sys");
exec("sys.path.append('/path/to/my/module')\n"
"import entity", globals);
It now works like a charm. Apparently just using Py_SetProgramName(argv[0]); was not enough.
With boost::python::import; watch for sys.path though so that your module is found, you might want to add a call to
PyRun_SimpleString("import sys; sys.path.append('.');")
first. Well, you can do the import via PyRun_SimpleString as well then :-)
Another option: write the entity_test itself in python.
Related
I am trying to write Python 2.7 code that will
Dynamically load a list / array of modules from a config file at startup
Call functions from those modules. Those functions are also designated in a config file (maybe the same config file, maybe a different one).
The idea is my code will have no idea until startup which modules to load. And the portion that calls the functions will have no idea which functions to call and which modules those functions belong to until runtime.
I'm not succeeding. A simple example of my situation is this:
The following is abc.py, a module that should be dynamically loaded (in my actual application I would have several such modules designated in a list / array in a config file):
def abc_fcn():
print("Hello World!")
def another_fcn():
print("BlahBlah")
The following is the .py code which should load abc.py (my actual code would need to import the entire list / array of modules from the config file). Both this .py file and abc.py are in the same folder / directory. Please note comments next to each statement.
module_to_import = "abc" #<- Will normally come from config file
fcn_to_call = "abc.abc_fcn" #<- Will normally come from config file
__import__(module_to_import) #<- No error
print(help(module_to_import)) #<- Works as expected
eval(fcn_to_call)() #<- NameError: name 'abc' is not defined
When I change the second line to the following...
fcn_to_call = "abc_fcn"
...the NameError changes to "name 'abc_fcn' is not defined".
What am I doing wrong? Thanks in advance for the help!
__import__ only returns the module specified, it does not add it to the global namespace. So to accomplish what you want, save the result as a variable, and then dynamically retrieve the function that you want. That could look like
fcn_to_call = 'abc_fcn'
mod = __import__(module_to_import)
func = getattr(mod, fcn_to_call)
func()
On a side note, abc is the name of name of the Abstract Base Classes builtin Python module, although I know you were probably just using this an example.
You should assign the returning value of __import__ to a variable abc so that you can actually use it as a module.
abc = __import__(module_to_import)
I'm trying to write a Python (2.7) library which loads certain classes at runtime. These classes contain a predefined set of methods.
My approach is to define a few Metaclasses which I work with in my library. I would for example define a "Navigation" Metaclass and work with it in the library. Then someone could write a class "Mainmenu" which contains some type of type definition that it is a "Navigation" plugin. And then the Library could use this class.
I am able to load modules and I'm able to write Metaclasses. My problem lies in combining these two things.
First there is the problem that I want the "plugin-classes" to be in a different (configurable) folder. So I can not do:
__metaclass__ = Navigation
because the Navigation class is part of my library and won't be there in the plugin-folder...
How could I solve the Problem of telling the type that the plugin is for? (Navigation, content.... e.g)
EDIT2: I solved the following problem. I found out that I can just ask the module to give me a dict.
My first problem still exists though
EDIT:
I managed registering and loading "normal" classes with a registry up until the following point:
from os import listdir
from os.path import isfile, join
import imp
class State:
registry = {}
def register_class(self,target_class):
self.registry[target_class.__name__] = target_class
print target_class.__name__+" registered!"
def create(self,classname):
tcls = self.registry[classname]
print self.registry[classname]
return tcls()
s = State()
mypath = """C:\metatest\plugins"""
files = [f for f in listdir(mypath) if isfile(join(mypath, f))]
for f in files:
spl = f.split(".")
if spl[1] == "py":
a = imp.load_source(spl[0], mypath + """\\""" + f)
s.register_class(a)
The problem I have at the end now is, that "a" is the loaded module, so it is a module-object. In my case there is only one class in it.
How can I get a Class object from the loaded module, so I can register the class properly??
So - let's check your problem steping back on your current proposal.
You need a way to have plug-ins for a larger system - the larger system won't know about the plug-ins at coding time - but the converse is not true: your plugins should be able to load modules, import base classes and call functions on your larger system.
Unless you really have something so plugable that plug-ins can work with more than one larger system. I doubt so, but if that is the case you need a framework that can register interfaces and retrieve classes and adapter-implementations between different classes. That framework is Zope Interface - you should read the documentation on it here: https://zopeinterface.readthedocs.io/en/latest/
Much more down to earth will be a plug-in system that sacans some preset directories for Python files and import those. As I said above, there is no problem if these files do import base classes (or metaclasses, for the record) on your main system: these are already imported by Python in the running process anyway, their import in the plug-in will just make then show up as available on the plug-in code.
You can use the exact code above, just add a short metaclass to register derived classes from State - you can maketeh convention that each base class for a different plug-in category have the registry attribute:
class RegistryMeta(type):
def __init__(cls, name, bases, namespace):
for base in cls.__mro__:
if 'registry' in base.__dict__:
if cls.__name__ in base.registry:
raise ValueError("Attempting to registrate plug-in with the name {} which is already taken".format(cls.__name__))
base.registry[cls.__name__] = cls
break
super(RegistryMeta, cls).__init__(name, base, namespace)
class State(object):
__metaclass__ = RegistryMeta
registry = {}
...
(keep the code for scanning the directory and loading modules - just switch all directory separation bars to "/" - you still are not doing it right and is subject to surprises by using "\")
and on the plug-in code include:
from mysystem import State
class YourClassCode(State):
...
And finally, as I said in the comment : you should really check the possibility of using Python 3.6 for that. Among other niceties, you could use the __init_subclass__ special method instead of needing a custom metaclass for keeping your registries.
I am currently using python 2.7 with enthought from pythonxy package.
In my software, I need to use my own user_manager and other permissions tool. So I need to add external sources into apptools.permissions.
In apptools documentation, it said I need to develop another egg with namespace, apptools.permissions.external.
Therefore, I have developed a folder with three level:
apptools,
apptools.permissions,
apptools.permissions.external.
and setup.py.
In setup.py, I wrote:
# 3
from setuptools import setup, find_packages
setup(
name = "apptools.permissions.external",
author = "Airbus",
version = '0.1' ,
include_package_data = True, package_data={'': ['*.*']},
packages = find_packages(),#exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),
test_suite = 'nose.collector',
entry_points = """
[envisage.plugins]
apptools.permissions.external = apptools.permissions.external.permissions_plugin:ExternalPermissionsPlugin
""",
#install_requires = ['Aerocity==1.01'],
zip_safe=True,
namespace_packages = ['apptools',
'apptools.permissions',
'apptools.permissions.external',
],
)
However, after I did python setup.py develop. I went to python and try to import apptools.permissions.external.
Python told me:
>>> import apptools.permissions.external
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named external
So it somehow cannot find this external egg. However, if I quickly changed the name of apptools to like apptools_test and related in folders and setup.py. I actually can import apptools_test.permissions.external.
So I think there is some problems when I merge namespace apptools.permissions.external to apptools. Python somehow gets confused.
Could someone help me with this case?
apptools.permissions was architected a long time ago when it was enthought.permissions and enthought was a namespace package. We have long since stopped doing that and refactored most of ETS into separate packages (sadly, apptools is still something of a grab-bag). When we did that, it seems that no one noticed that it was (ab)using the namespace package like that. Sorry about that. We, uh, don't use it much ourselves. Take that for whatever cold comfort it brings you. :-)
The only places it does this kind of indirection is in _*_default() methods, so you should be able to just assign your own instances for these traits. I'm not really sure why the namespace extension mechanism was attempted at all.
I'm using boost::python to do a hybrid C++/python application: the C++ app calls a collection of python scripts, which in turn use the C++ program's functions, classes, etc., exposed as python objects. (Python 2.x.)
BOOST_PYTHON_MODULE(MyModule) exposes the C++ to python as expected.
My initialization code:
Py_Initialize();
initMyModule(); // import MyModule
namespace bpl = boost::python;
Now I want my C++ code to get at MyModule, too. In python, you just write globals()['MyModule']. But this (and things like it) don't work in C++:
bpl::object globals = bpl::eval("globals()");
This fails at run-time with
File "<string>", line 1, in <module>; NameError: name 'globals' is not defined
As an aside, I see many examples of setting up __main__ like this:
bpl::object m = bpl::import("__main__");
bpl::dict g = m.attr("__dict__"); // like locals(), but not globals()
This doesn't fail, and gives locals, but according to the Py_Initialize docs, __main__ is already set up. And it doesn't let you see globals, where you'd find your imported module.
You don't need the explicit bpl::import("__main__");.
Here are the globals:
bpl::dict globals()
{
bpl::handle<> mainH(bpl::borrowed(PyImport_GetModuleDict()));
return bpl::extract<bpl::dict>(bpl::object(mainH));
}
Since everything is managed by smart pointers, returning and manipulating bpl::dict directly works fine.
bpl::object myMod = globals()["MyModule"];
globals()["myNewGlobal"] = 88;
I have a big c++ program in a single .cpp file which defines a lot of classes(interdependent of each other) and finally runs a main function. Now I am interested only in using one of this classes in python, specifically one method of this class which accepts 5 floats as inputs and outputs one float. I am trying to find the simplest method to achieve this. After not having success with boost:python(mainly because of installation issues) I have come to Cython which in the current version supports C++. I could successfully run the Rectangle example given in the Cython tutorial but I can't get how to proceed and adapt this to my case where I don't need a so complicated .pyx file, and where I don't have a .h file. Can somebody explain me in simple words what should I write in setup.py and in the .pyx file if my .cpp file has for example the structure:
...
class Nuclei {
public:
...
double potential(float,float,float,float,float);
...
private:
...
};
...
If all you are looking to do is call a single function, Extending Python With C/C++ is probably the simplest approach. This page provides a good example.
The relevant setup.py code in that example is
from distutils.core import setup, Extension
module1 = Extension('demo',
define_macros = [('MAJOR_VERSION', '1'),
('MINOR_VERSION', '0')],
include_dirs = ['/usr/local/include'],
libraries = ['tcl83'],
library_dirs = ['/usr/local/lib'],
sources = ['demo.c'])
setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
author = 'Martin v. Loewis',
author_email = 'martin#v.loewis.de',
url = 'http://docs.python.org/extending/building',
long_description = '''
This is really just a demo package.
''',
ext_modules = [module1])
If the C++ code you want to call is in demo.c, it could be used with import demo in Python.
Note that it is not nearly that simple- you'll be creating a function that takes a PyObject * with arguments and returns a PyObject *, and a lot can be said about how those are constructed. Take a look at the pages linked to above- they are filled with examples.