Wrap all class methods using a meta class - python-2.7

I'm trying to wrap all of the methods inside a class I wrote with a specific wrapper method.
My class inherits from the python dict class, and I want to wrap all of the methods of this parent class, such as __setitem__, __getitem__, etc.
In my attempts to achieve that I have written a meta class that wraps all the methods inside it's child class, using the __init__ method in the meta class, so I can access the child class's object (and not it's class definition which does not include the parent methods).
However, after running the code, I see the wrapper method is never called. Meaning the wrapping didn't succeed.
Can you help with figuring out what went wrong?
My code:
def wrapper(func):
def wrapped(self, *args, **kwargs):
print 'wrapper.__call__()'
res = func(self, *args, **kwargs)
return res
return wrapped
class MyMeta(type):
def __init__(cls, classname, bases, class_dict):
print 'meta.__init__()'
new_class_dict = {}
for attr_name in dir(cls):
attr = getattr(cls, attr_name)
if hasattr(attr, '__call__'):
attr = wrapper(attr)
new_class_dict[attr_name] = attr
return super(MyMeta, cls).__init__(classname, bases, new_class_dict)
class MyDict(dict):
__metaclass__ = MyMeta
def __init__(self, *args):
print 'child.__init__()'
super(MyDict, self).__init__(*args)
d = MyDict({'key': 'value'})
d['new_key'] = 'new_value'
The printout I get is:
meta.__init__()
child.__init__()
without any reference to the wrapper.__call__() print I placed inside the wrapped method...

When the metaclass __init__ gets called, the class object has already been built so modifying the attributes dict (class_dict in your code) at this stage is totally useless indeed. You want to use setattr instead:
class MyMeta(type):
def __init__(cls, classname, bases, class_dict):
for attr_name in dir(cls):
if attr_name == "__class__":
# the metaclass is a callable attribute too,
# but we want to leave this one alone
continue
attr = getattr(cls, attr_name)
if hasattr(attr, '__call__'):
attr = wrapper(attr)
setattr(cls, attr_name, attr)
# not need for the `return` here
super(MyMeta, cls).__init__(classname, bases, class_dict)

Related

Django DRF unit tests added with dynamic mixins via metaclass not being executed

I am trying to test DRF endpoints, and trying to add mixings dynamically to a test in order to execute tests again each method allowed in the endpoint (get, post, put, patch, delete)
So, my idea is to make a base test class that will automatically add some mixings to test endpoints if they are allowed. And I can create the actual test that will inherit from this base class.
The code:
from rest_framework.test import APITestCase
class GetTestMixin:
def test_get_all(self):
response = self.client.get(self.uri)
self.assertEqual(response.status_code,status.HTTP_200_OK)
class AutoMixinMeta(type):
def __call__(cls, *args, **kwargs):
allowed_methods = ['get', 'post']
# cls would be the Test class, for example TestExample
# cls.__bases__ is a tuple with the classes inherited in the Test class, for example:
# (<class 'unit_tests.endpoints.base_test.RESTTestCase'>, <class 'rest_framework.test.APITestCase'>)
bases = cls.__bases__
for method in allowed_methods:
bases += (cls.method_mixins[method.lower()],)
# Create a new instance with all the mixins
cls = type(cls.__name__, bases, dict(cls.__dict__))
return type.__call__(cls, *args, **kwargs)
class RESTTestCase(metaclass=AutoMixinMeta):
uri = None
method_mixins = {
'post': PostTestMixin,
'get': GetTestMixin,
}
class TestExample(RESTTestCase, APITestCase):
uri = reverse('somemodel-list')
I was expecting test_get_all to be executed, but it is not.
Mixings are in place. I made a dummy method inside TestExample and put a debugger in place, and checked it, like this:
(Pdb) self.__class__
<class 'TestExample'>
(Pdb) self.__class__.__bases__
(<class 'RESTTestCase'>, <class 'rest_framework.test.APITestCase'>, <class 'GetTestMixin'>)
The problem there is that the code that collects the classes to be tested will never "see" the class as an instance of the Test classes, or as a subclass of: the class inheriting from the test cases only exist when an instance is created.
The only way for this to work is to create the derived classes at import time, and to bind the desired dynamic classes as top-level names on the module.
To do that, you can do away with the metaclass, and just place the statements in the module body, assigning the new class or classes to names using globals(). Or, if you want just the subclasses, rather than at module top level, the code can be placed in the __init_subclass__ method. This methd is called when the class is created, not when it is instantiated, and it should work.
from rest_framework.test import APITestCase
class GetTestMixin:
def test_get_all(self):
response = self.client.get(self.uri)
self.assertEqual(response.status_code,status.HTTP_200_OK)
class RESTTestCase():
uri = None
method_mixins = {
'post': PostTestMixin,
'get': GetTestMixin,
}
def __init_subclass__(cls, *args, **kw):
super.__init_subclass__(*args, **kw)
if "Dynamic" in cls.__name__:
return
allowed_methods = ['get', 'post']
bases = list(cls.__bases__)
for method in allowed_methods:
bases.append(cls.method_mixins[method.lower()])
# Create a new instance with all the mixins
new_cls = type(cls.__name__ + "Dynamic", bases, dict(cls.__dict__))
globals()[new_cls.__name__] = new_cls
class TestExample(RESTTestCase, APITestCase):
uri = reverse('somemodel-list')
# class TestExampleDynamic is created automatically when the `class` statement above resolves

where does "create()" method comes from in django

I just want to know where create method comes from in django.
I checked Model class, BaseManager class and so on.
I cannot find where create method is defined.
Anyone knows about it ?
https://github.com/django/django/blob/2d6179c819010f6a9d00835d5893c4593c0b85a0/django/
Short answer: In the QuerySet class.
This is defined in the QuerySet. Indeed, in the Manager, it is wrapped to the QuerySet source [GitHub]:
#classmethod
def _get_queryset_methods(cls, queryset_class):
def create_method(name, method):
def manager_method(self, *args, **kwargs):
return getattr(self.get_queryset(), name)(*args, **kwargs)
manager_method.__name__ = method.__name__
manager_method.__doc__ = method.__doc__
return manager_method
new_methods = {}
for name, method in inspect.getmembers(queryset_class, predicate=inspect.isfunction):
# Only copy missing methods.
if hasattr(cls, name):
continue
# Only copy public methods or methods with the attribute `queryset_only=False`.
queryset_only = getattr(method, 'queryset_only', None)
if queryset_only or (queryset_only is None and name.startswith('_')):
continue
# Copy the method onto the manager.
new_methods[name] = create_method(name, method)
return new_methods
The _get_queryset_methods is called when constructing the Manager class [GitHub]:
class Manager(BaseManager.from_queryset(QuerySet)):
pass
and the from_queryset will attach the QuerySet methods to the Manager class [GitHub]:
#classmethod
def from_queryset(cls, queryset_class, class_name=None):
if class_name is None:
class_name = '%sFrom%s' % (cls.__name__, queryset_class.__name__)
return type(class_name, (cls,), {
'_queryset_class': queryset_class,
**cls._get_queryset_methods(queryset_class),
})
This is done to implement for each method in the QuerySet a method in the Manager to prevent implementing the same logic multiple times. If you thus call .create on the .objects manager (or any other manager), it will call .get_queryset(), and then call that method (here create) on the given QuerySet that get_queryset returns.
The QuerySet thus has a method create that is implemented as [GitHub]:
def create(self, **kwargs):
"""
Create a new object with the given kwargs, saving it to the database
and returning the created object.
"""
obj = self.model(**kwargs)
self._for_write = True
obj.save(force_insert=True, using=self.db)
return obj

abstractproperty + classmethod decorators in python

I want to enforce childrens to use a classmethod in python2.7.
I tried this:
import abc
class Base(object):
__metaclass__ = abc.ABCMeta
#abc.abstractproperty
def value(self):
pass
#abc.abstractproperty
#classmethod
def text(cls):
pass
class Imp(Base):
TEXT = "hi im text"
#classmethod
def haba(cls):
print 'HI'
#property
def value(self):
return 'asdasd'
#classmethod
#property
def text(cls):
return 'ho ho p'
print Imp.text
print Imp.TEXT
But I'm getting this output:
<bound method ABCMeta.? of <class 'main.Imp'>>
hi im text
How I can properly enforce childrens to implement classmethod properties?
You can see that Imp.TEXT is working but there is no way to enforce creating this member from base class this way
After re-reading your question a few times I concluded that you want the cl method to behave as if it is a property for the class.
First, Python's implementation of abstract method/property checking is meant to be performed at instantiation time only, not at class declaration. I hope you are aware of that.
Second, Python's descriptor protocol allows for the creation of the equivalent of "class properties", although there is no higher level support for that on the language itself - you can create a class which __get__ method returns your calculated property when the instance argument is None (usually descriptors will return 'self' so that they can be retrieved from the class).
Finally - it is possible that by declaring a custom metaclass being abstract itself, and then declaring it as your class metaclass, abstractproperties will trigger in runtime - let's try that - :
In [1]: import abc
In [2]: class AbsPropertyMeta(abc.ABC, type):
...: #abc.abstractproperty
...: def cl(cls):
...: return "Ho ho ho"
...:
In [3]: class ConcreteExample(metaclass=AbsPropertyMeta):
...: pass
...:
(Note that I will develop the answer using Python 3, which should be what you should be using in any new project or for learning purposes as well)
So, as for the former example, the property in the metaclass does work as a "class property", but Python does not enforce its redefinition in the class body.
So, if you really need this design, you should create a complete custom metaclass for that, and let go of the abc.ABCMeta mechanisms at all:
from functools import partial
def abstractclassproperty(func):
func._abstract_property = True
return func
class clsproperty(object):
def __init__(self, func):
self.func = func
def __get__(self, instance, owner):
return self.func(owner)
class ABCAbstractClsProperty(type):
def __new__(mcls, name, bases, namespace, **kw):
new_cls = super(ABCAbstractClsProperty, mcls).__new__(mcls, name, bases, namespace, **kw)
for attr_name in dir(new_cls): # Dir retrieves attributes from all superclasses
attr = getattr(new_cls, attr_name)
if getattr(attr, "im_func", None): # Python 2 specific normalization.
attr = attr.im_func
if getattr(attr, '_abstract_property', False) and new_cls.__dict__.get(attr_name) is not attr:
raise TypeError("Can't create class {!r}: abstract property {!r} not implemented".format(name, attr_name))
return new_cls
""" # Python 3:
class A(metaclass=ABCAbstractClsProperty):
#abstractclassproperty
def cl(cls):
pass
"""
class A(object):
__metaclass__ = ABCAbstractClsProperty
#abstractclassproperty
def cl(cls):
pass
try:
class B(A):
pass
except TypeError:
print("Check ok")
class C(A):
#clsproperty
def cl(cls):
return "ho ho ho " + cls.__name__
print(C.cl)

`self.response_class` in `TemplateResponseMixin` don't call super()

Referencece to response_class in Django's code:
django/base.py
class TemplateResponseMixin:
"""A mixin that can be used to render a template."""
response_class = TemplateResponse
def render_to_response(self, context, **response_kwargs):
response_kwargs.setdefault('content_type', self.content_type)
#here
return self.response_class(
#here
request=self.request,
template=self.get_template_names(),
context=context,
using=self.template_engine,
**response_kwargs
)
The class attribute setting response_class = TemplateResponse,
while call it through instance's attribute self.response_class,
I guess it might be super().response_class
How to understand it?
You need to use super() when calling superclass's method. But in case of response_class it's just attribute defined inside TemplateResponseMixin so you can simple accessed it through self.response_class. Since response_class is Class to instancinate you need to add () like this: self.response_class(*args, **kwargs).
You can check this question to get more details about super().
Example:
class A:
def method_a(self):
pass
class B(A):
some_class = SomeClass
def method_a(self):
super().method_a() # This code will find A's method_a and call it
self.some_class() # This will only instancinate some_class attribute of current B's instance

Why declare field widget before super init?

I am a noob at Python and Django and attempting to learn it as I go along, however, there is one thing that confuses me.
In a custom field of mine I have:
def __init__(self, *args, **kwargs):
self.request = None
self.widget = ReCaptcha
super(ReCaptchaField, self).__init__(*args, **kwargs)
self.required = True
self.error_messages = {'required': self.default_error_messages['captcha_invalid']}
and I cannot seem to understand why declaring the field widget works where it is but not after:
super(ReCaptchaField, self).__init__(*args, **kwargs)
Instead it actually produces an error whereby:
def value_from_datadict(self, data, files, name):
In my widget actually is not passed self for some reason.
However:
self.error_messages = {'required': self.default_error_messages['captcha_invalid']}
only works after calling the super.
Why is this?
I currently have no way to confirm this, but I would provide a possible way to explain this.
It's clearer to check the source code for answer here. self.widget is expecting a class and widget will later be initialized as a class instance in __init__ in parent class:
widget = widget or self.widget
if isinstance(widget, type):
widget = widget() # the instance initialization
If you do the assignment after super, your widget remains a class and will never be initialized, thus it is not going to work.
On the other hand, error_messages is actually an optional parameter for __init__ method. If you provide that in __init__ function, it will take it to the self.error_messages. Otherwise, it's empty dict:
messages = {}
for c in reversed(self.__class__.__mro__):
messages.update(getattr(c, 'default_error_messages', {}))
# see here. Did you provide any error_messages? If no then {}
messages.update(error_messages or {})
# self.error_messages might be {} because the above code
self.error_messages = messages
So if you do self.error_messsages before the super, it will be overridden with {}.