Background
I'm trying to implement a highly non-linear lens to do lens distortion in Panda3D for a complex projection setup. I want to use this implementation following this approach.
The question
Can I do it in Python (and if so, how, what am I doing wrong) or do I have to do it in C++ (and if so, where do I start)?
Attempts so far
I've tried subclassing Lens, but if I let my subclass call the super constructor (or don't override the constructor at all), I get an error:
>>> from panda3d.core import Lens
>>> class MyLens(Lens):
... def __init__(self):
... super(MyLens,self).__init__()
...
>>> l = MyLens()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __init__
TypeError: Error Can Not Init Constant Class (Lens)
If I don't call the super constructor, the class isinstance(Lens), but not recognized as such by Panda3D code:
fcamNode = Camera('fcam')
flens = MyLens.MyLens()
assert isinstance(flens, Lens)
fcamNode.setLens(flens)
results in TypeError: LensNode.set_lens() argument 1 must be Lens, not MyLens.
If I subclass PerspectiveLens instead, I can call the super constructor and pass instances of my class to setLens(), but none of its overridden methods are ever called and the rendered scene looks like it was rendered with the stock PerspectiveLens.
That is all coded in C++, i.e. the Lens class is internally a C++ class and all the other classes are also C++ classes which overload the C++ Lens class.
If you overload such Python-wrapped class and pass that object down to some C++ code again, the C++ code wont recognize the Python overwriting.
You might be able to write a C++ Lens superclass which is able to do that, i.e. which implements all possible virtual functions and in all cases, always looks up wether there is a related Python object attribute which can be called. Note that this is likely to be slow.
In C++, you find the definition of Lens here and here.
Take a look at the FisheyeLens here to see what functions you have to overwrite from Lens.
Related
Objective
Wrap every function of every class of gspread module.
I know there are countless of posts on the subject and most unanimously instruct to use decorators.
I'm not too familiar with decorators and felt like that approach is not as seamless as I hoped for. perhaps I didn't understand correctly.
But, I found this answer which "felt" like what I'm looking for.
(poor) Attempt
import gspread
from oauth2client.service_account import ServiceAccountCredentials
import os
import inspect
class GoogleSheetAPI:
def __init__(self):
f = os.path.join(os.path.dirname(__file__), 'credentials.json')
os.environ.setdefault('GOOGLE_APPLICATION_CREDENTIALS', f)
scope = ['https://spreadsheets.google.com/feeds',
'https://www.googleapis.com/auth/drive']
credentials = ServiceAccountCredentials.from_json_keyfile_name(f, scope)
self.client = gspread.authorize(credentials)
self.client.login()
def SafeCall(f):
try:
print 'before call'
f()
print 'after call'
except:
print 'exception caught'
return None
for class_name, c in inspect.getmembers(gspread, inspect.isclass):
for method_name, f in inspect.getmembers(c, inspect.ismethod):
setattr(c, f, SafeCall(f)) # TypeError: attribute name must be string, not 'instancemethod'
g = GoogleSheetAPI()
spreadsheet = g.client.open_by_key('<ID>') # calls a function in gspread.Client
worksheet = spreadsheet.get_worksheet(0) # calls a function in gspread.Spreadsheet
worksheet.add_rows(['key','value']) # calls a function in gspread.Worksheet
Notes
When I use the word "seamless" I mean that considering my code has many calls to many gspread functions, I want to change as little as possible. Using inspect/setattr seems like the perfect/seamless trick.
There are three obvious issues with your code actually.
The first one is the TypeError - which is easy to solve FWIW: as the error message (raised by setattr() states, "attribute name must be string, not 'instancemethod'". And you're indeed trying to use f (the method itself) instead of method_name. What you want here is of course:
setattr(c, method_name, SafeCall(f))
The second issue is that your SafeCall "decorator" is NOT a decorator. A decorator (well, the kind of decorator you want here at least) returns a function that wraps the original one, your current implementation just calls the original function. Actually, it is almost what SafeCall should actually return. An example of a proper decorator would be:
def decorator(func):
def wrapper(*args, **kw):
print("before calling {}".format(func))
result = func(*args, **kw)
print("after calling {}".format(func))
return result
return wrapper
And finally, the third obvious issue is here:
except:
print 'exception caught'
return None
You certainly don't want this. This
1/ will catch absolutely everything (incuding SysExit, which is what Python raises on sys.exit() calls and StopIteration which is how iterators signals they are exhausted),
2/ discard all the very useful debugging infos - making it impossible to diagnose what actuall went wrong
3/ return something that can be plain unusable so you'll have to test the return value of each method call, and since you won't know what went wrong, you won't be able to handle the issue otherwise than printing "oops, something went wrong but don't ask me what nor where nor why" and exiting the program, which is definitly not better than letting the exception propagate - the program will crash in both cases, but at least if you leave the exception alone you'll have some hints on what caused the issue.
4/ or, much worse, return a valid return value for the method (yes, quite a few method are designed to change state and return None) so you won't even know something went wrong and happily continue execution - which is a sure way to have incorrect result and corrupted data.
5/ not to mention that the methods you're decorating that way are very probably calling each others and using (expected) exceptions internally (with proper exception handling), so you are actually introducing bugs in an otherwise working (or mostly working) library.
IOW, this is probably the worse antipattern you can ever think of...
I'm trying to write a Python (2.7) library which loads certain classes at runtime. These classes contain a predefined set of methods.
My approach is to define a few Metaclasses which I work with in my library. I would for example define a "Navigation" Metaclass and work with it in the library. Then someone could write a class "Mainmenu" which contains some type of type definition that it is a "Navigation" plugin. And then the Library could use this class.
I am able to load modules and I'm able to write Metaclasses. My problem lies in combining these two things.
First there is the problem that I want the "plugin-classes" to be in a different (configurable) folder. So I can not do:
__metaclass__ = Navigation
because the Navigation class is part of my library and won't be there in the plugin-folder...
How could I solve the Problem of telling the type that the plugin is for? (Navigation, content.... e.g)
EDIT2: I solved the following problem. I found out that I can just ask the module to give me a dict.
My first problem still exists though
EDIT:
I managed registering and loading "normal" classes with a registry up until the following point:
from os import listdir
from os.path import isfile, join
import imp
class State:
registry = {}
def register_class(self,target_class):
self.registry[target_class.__name__] = target_class
print target_class.__name__+" registered!"
def create(self,classname):
tcls = self.registry[classname]
print self.registry[classname]
return tcls()
s = State()
mypath = """C:\metatest\plugins"""
files = [f for f in listdir(mypath) if isfile(join(mypath, f))]
for f in files:
spl = f.split(".")
if spl[1] == "py":
a = imp.load_source(spl[0], mypath + """\\""" + f)
s.register_class(a)
The problem I have at the end now is, that "a" is the loaded module, so it is a module-object. In my case there is only one class in it.
How can I get a Class object from the loaded module, so I can register the class properly??
So - let's check your problem steping back on your current proposal.
You need a way to have plug-ins for a larger system - the larger system won't know about the plug-ins at coding time - but the converse is not true: your plugins should be able to load modules, import base classes and call functions on your larger system.
Unless you really have something so plugable that plug-ins can work with more than one larger system. I doubt so, but if that is the case you need a framework that can register interfaces and retrieve classes and adapter-implementations between different classes. That framework is Zope Interface - you should read the documentation on it here: https://zopeinterface.readthedocs.io/en/latest/
Much more down to earth will be a plug-in system that sacans some preset directories for Python files and import those. As I said above, there is no problem if these files do import base classes (or metaclasses, for the record) on your main system: these are already imported by Python in the running process anyway, their import in the plug-in will just make then show up as available on the plug-in code.
You can use the exact code above, just add a short metaclass to register derived classes from State - you can maketeh convention that each base class for a different plug-in category have the registry attribute:
class RegistryMeta(type):
def __init__(cls, name, bases, namespace):
for base in cls.__mro__:
if 'registry' in base.__dict__:
if cls.__name__ in base.registry:
raise ValueError("Attempting to registrate plug-in with the name {} which is already taken".format(cls.__name__))
base.registry[cls.__name__] = cls
break
super(RegistryMeta, cls).__init__(name, base, namespace)
class State(object):
__metaclass__ = RegistryMeta
registry = {}
...
(keep the code for scanning the directory and loading modules - just switch all directory separation bars to "/" - you still are not doing it right and is subject to surprises by using "\")
and on the plug-in code include:
from mysystem import State
class YourClassCode(State):
...
And finally, as I said in the comment : you should really check the possibility of using Python 3.6 for that. Among other niceties, you could use the __init_subclass__ special method instead of needing a custom metaclass for keeping your registries.
In my python code, I have these lines:
def __del__(self):
for shader in self.shaders:
GL.glDetachShader(self.program, shader)
GL.glDeleteShader(shader)
GL.glDeleteProgram(self.program)
But when the code executes these lines, I end up seeing this problem:
Exception ignored in: <bound method Shader.__del__ of <__main__.Shader object at 0x0000016E4AFFE550>>
Traceback (most recent call last):
File "C:\Germano\Dev\Gartic\tests\modernOpenGL2.py", line 214, in __del__
GL.glDetachShader(self.program, shader)
File "C:\Users\Germano\AppData\Local\Programs\Python\Python36\lib\site-packages\OpenGL\platform\baseplatform.py", line 407, in __call__
self.__name__, self.__name__,
OpenGL.error.NullFunctionError: Attempt to call an undefined function glDetachShader, check for bool(glDetachShader) before calling
What is going on? How can I fix this problem??
__del__ is likely only being called after the OpenGL context is being destroyed, leading to the NullFunctionError. If the Program/Shader class is surviving until the end of the lifetime of the application after the window is destroyed, when the Python interpreter is calling all of your custom delete functions, all gl* function calls would basically not be available. You can rely on context destruction to delete all OpenGL-associated objects in this case.
However, it is still a good idea to free up memory when you are no longer using a shader/program object during the lifetime of your application, when the context still exists (see this answer for more info)! I personally also ran into a similar issue myself when deleting textures or buffers. I would recommend wrapping your code in a try/except block, so it would look like this:
def __del__(self):
try:
#if the context is alive, you want to try and delete shader/program stuff manually
#this could be triggered with e.g. `del my_awesome_program_class_object`
for shader in self.shaders:
GL.glDetachShader(self.program, shader)
GL.glDeleteShader(shader)
GL.glDeleteProgram(self.program)
except OpenGL.error.NullFunctionError as error:
pass#do nothing; context already deleted my shader/program stuff!
Hope this explanation makes things clearer, as those NullFunctionErrors are definitely tricky to understand without knowing why they are occurring in the first place!
I keep getting this error each time I try to create a world. How do I fix this?
Traceback (most recent call last):
File "C:/Python Codes/test_3.py", line 3, in
world = b2World(gravity=(0,-10), doSleep=True)
NameError: name 'b2World' is not defined
this is the code I wanted to run.
import Box2D as box2d
world = b2World(gravity=(0,-10), doSleep=True)
If b2World is a function defined in the box2d module, you need to specify that:
import Box2D as box2d
world = box2d.b2World(gravity=(0,-10), doSleep=True)
So I'm working on a little project in which I'm using Python as an embedded scripting engine. So far I've not had much trouble with it using boost.python, but there's something I'd like to do with it if it's possible.
Basically, Python can be used to extend my C++ classes by adding functions and even data values to the class. I'd like to be able to have these persist in the C++ side, so one python function can add data members to a class, and then later the same instance passed to a different function will still have them. The goal here being to write a generic core engine in C++, and let users extend it in Python in any way they need without ever having to touch the C++.
So what I thought would work was that I would store a boost::python::object in the C++ class as a value self, and when calling the python from the C++, I'd send that python object through boost::python::ptr(), so that modifications on the python side would persist back to the C++ class. Unfortunately when I try this, I get the following error:
TypeError: No to_python (by-value) converter found for C++ type: boost::python::api::object
Is there any way of passing an object directly to a python function like that, or any other way I can go about this to achieve my desired result?
Thanks in advance for any help. :)
Got this fantastic solution from the c++sig mailing list.
Implement a std::map<std::string, boost::python::object> in the C++ class, then overload __getattr__() and __setattr__() to read from and write to that std::map. Then just send it to the python with boost::python::ptr() as usual, no need to keep an object around on the C++ side or send one to the python. It works perfectly.
Edit: I also found I had to override the __setattr__() function in a special way as it was breaking things I added with add_property(). Those things worked fine when getting them, since python checks a class's attributes before calling __getattr__(), but there's no such check with __setattr__(). It just calls it directly. So I had to make some changes to turn this into a full solution. Here's the full implementation of the solution:
First create a global variable:
boost::python::object PyMyModule_global;
Create a class as follows (with whatever other information you want to add to it):
class MyClass
{
public:
//Python checks the class attributes before it calls __getattr__ so we don't have to do anything special here.
boost::python::object Py_GetAttr(std::string str)
{
if(dict.find(str) == dict.end())
{
PyErr_SetString(PyExc_AttributeError, JFormat::format("MyClass instance has no attribute '{0}'", str).c_str());
throw boost::python::error_already_set();
}
return dict[str];
}
//However, with __setattr__, python doesn't do anything with the class attributes first, it just calls __setattr__.
//Which means anything that's been defined as a class attribute won't be modified here - including things set with
//add_property(), def_readwrite(), etc.
void Py_SetAttr(std::string str, boost::python::object val)
{
try
{
//First we check to see if the class has an attribute by this name.
boost::python::object obj = PyMyModule_global["MyClass"].attr(str.c_str());
//If so, we call the old cached __setattr__ function.
PyMyModule_global["MyClass"].attr("__setattr_old__")(ptr(this), str, val);
}
catch(boost::python::error_already_set &e)
{
//If it threw an exception, that means that there is no such attribute.
//Put it on the persistent dict.
PyErr_Clear();
dict[str] = val;
}
}
private:
std::map<std::string, boost::python::object> dict;
};
Then define the python module as follows, adding whatever other defs and properties you want:
BOOST_PYTHON_MODULE(MyModule)
{
boost::python::class_<MyClass>("MyClass", boost::python::no_init)
.def("__getattr__", &MyClass::Py_GetAttr)
.def("__setattr_new__", &MyClass::Py_SetAttr);
}
Then initialize python:
void PyInit()
{
//Initialize module
PyImport_AppendInittab( "MyModule", &initMyModule );
//Initialize Python
Py_Initialize();
//Grab __main__ and its globals
boost::python::object main = boost::python::import("__main__");
boost::python::object global = main.attr("__dict__");
//Import the module and grab its globals
boost::python::object PyMyModule = boost::python::import("MyModule");
global["MyModule"] = PyMyModule;
PyMyModule_global = PyMyModule.attr("__dict__");
//Overload MyClass's setattr, so that it will work with already defined attributes while persisting new ones
PyMyModule_global["MyClass"].attr("__setattr_old__") = PyMyModule_global["MyClass"].attr("__setattr__");
PyMyModule_global["MyClass"].attr("__setattr__") = PyMyModule_global["MyClass"].attr("__setattr_new__");
}
Once you've done all of this, you'll be able to persist changes to the instance made in python over to the C++. Anything that's defined in C++ as an attribute will be handled properly, and anything that's not will be appended to dict instead of the class's __dict__.