I have a python class that inherits from storm.py from the Apache Storm MultiLang project.
My class looks like the following:
import storm
class MyClassName(Storm.Bolt):
def initialize(self,conf,context):
self._conf = conf;
self._context = context
def process(self, in_tuple):
storm.ack(in_tuple)
if __name__ == '__main__':
MyClassName().run()
I copied my python file (myfilename.py) out to /usr/lib64/python2.7/site-package. I then logged into the python shell and did an import myfilename. That completed without error. When I run the following inspect.getmro(myfilename.MyClassName()) I get the following error:
AttributeError: 'MyClassName' object has no attribute '__bases__'
I was under the impression that when I declared my class and passed it Storm.Bolt that I was extending Storm.Bolt. My questions are:
Do I need to define __bases__ in my class?
What else am I missing?
Using Python 2.7.13 on CentOs7. Storm version is 1.1.0
The inspect.getmro function expects its argument to be a class, but you're passing it an instance. Get rid of the parentheses that call the class and your code should work:
inspect.getmro(myfilename.MyClassName) # not MyClassName()!
If the call you gave in the question was a simplified example and you don't have the class directly available where you're calling getmro on the instance, you can use type to get the class:
obj = SomeClass() # this happens somewhere earlier on, and we don't know SomeClass below
inspect.getmro(type(obj)) # but we can easily get it using type()
Related
I am using mongoengine in python as an ORM. Now I have a situation where I have a class, actually the a model, of the form:
from mongoengine import *
class SomeDetails(Document):
alias = StringField(required=True)
version = StringField(required=True)
author = StringField(required=True)
someName = StringField(required=True)
Now I need to create the object of this class (and hence a document at runtime)
I can create an object of a normal class at runtime by using inspection, something like this:
from inspect import isclass
# moduleImport is a method made to facilitate importing of modules at runtime.
new_module = moduleImport('myModule')
classes = [x for x in dir(new_module) if isclass(getattr(new_module, x))]
class_object = getattr(new_module, classes[0])
instance = class_object()
And the above works perfectly. I am able to create an object at runtime and then use it as I wish.
However, in the former case above, when I need to create an object (or actually a mongo document in this case), as per the mongoengine documentation here and here, I need to create the object something like this:
docObject = SomeDetails(alias=value, version=value, author=value, someName=value)
docObject.save() # to save the document in the mongodb
wherein I need to specify the values for each of the keyword arguments while creating the object of the class, so that the document gets saved successfully.
So in this case, if I try creating the object of the SomeDetails class at runtime (as shown in the inspect example above) how can I provide these keyword arguments at the runtime while creating the object?
An important catch
So the keyword arguments are also not known before hand. They too are fetched at runtime. At runtime, before object creation, yes, I do know the list of keyword args and it's values, which is available to me as a dictionary as :
{alias:"a", version:"b", author:"c", someName:"d"}
but even then, this dic itself is available only at runtime.
You can try this:
data = {alias:"a", version:"b", author:"c", someName:"d"}
instance = class_object(**data)
More on **kwargs here
Presumably, you would just replace the line
instance = class_object()
With
instance = class_object(alias="a", version="b", author="c", someName="d")
If this doesn't work, please elaborate on what error you get.
I would like to provide default behaviour for a class as illustrated below.
import numpy as np
class Test:
def __init__(self, my_method=None):
self.my_method = my_method or np.min
Test().my_method([1, 2, 3]) # >>> 1
The code works as expected. To keep all the default values together for easier code maintenance I wanted to change the code to
import numpy as np
class Test:
default_method = np.min
def __init__(self, my_method=None):
self.my_method = my_method or Test.default_method
Test().my_method([1, 2, 3]) # >>> TypeError
but the call to my_method fails with the error message unbound method amin() must be called with Test instance as first argument (got list instance instead). Oddly, the code works as expected if I use the builtin min rather than np.min, i.e. the following works as expected.
import numpy as np
class Test:
default_method = min # no np.
def __init__(self, my_method=None):
self.my_method = my_method or Test.default_method
Test().my_method([1, 2, 3]) # >>> 1
What am I missing?
Any function stored as an attribute on a class object is treated as a method by Python. On Python 2, that means it requires the first argument to be an instance of the class (which will be passed automatically if the attribute is requested via an instance). On Python 3, unbound methods no longer check their arguments in that way (so your code would work as written).
To work around the issue on Python 2, try wrapping the default_method value with staticmethod:
class Test(object):
default_method = staticmethod(np.min)
#...
This might not be a bad idea even on Python 3, since you'll also be able to use self.default_method rather than explicitly naming the class.
As for why the code worked with min but not np.min, that's because they are implemented differently. You can see that from their types:
>>> type(min)
<class 'builtin_function_or_method'>
>>> type(np.min)
<class 'function'>
Regular functions (like np.min) act as descriptors when they're attributes of a class (thus getting the "binding" behavior that was causing your issue). Builtin functions like min don't support the descriptor protocol, so the issue doesn't come up.
In python 2.7, I have this function
from slacker import Slacker
def post_message(token, channel, message):
channel = '#{}'.format(channel)
slack = Slacker(token)
slack.chat.post_message(channel, message)
with mock and patch, I can check that the token is used in Slacker class
import unittest
from mock import patch
from slacker_cli import post_message
class TestMessage(unittest.TestCase):
#patch('slacker_cli.Slacker')
def test_post_message_use_token(self, mock_slacker):
token = 'aaa'
channel = 'channel_name'
message = 'message string'
post_message(token, channel, message)
mock_slacker.assert_called_with(token)
how I can check the string use in post_message ?
I try with
mock_slacker.chat.post_message.assert_called_with('#channel_name')
but I get
AssertionError: Expected call: post_message('#channel_name')
Not called
You need to be specific about where the call is taking place. The code is:
slack.chat.post_message
So, as slack is an instance of the mocked class Slacker, you'll need to use return_value to ensure you're talking about that instance:
mock_slacker.return_value.chat.post_message.assert_called_with
You've patched 'slacker_cli.Slacker' so mock_slacker is a patched class. The call itself is taking place on an instance of that patched class. Calling a class returns an instance, hence the use of return_value.
I am trying to learn Django. I have a python program which has class in it. All the examples that use python's class inherits Model.model. My class doesn't require any database. Can anyone help me how to set urls.py in django for this particular case?
What I have done so far is that I have imported the class, and the method inside the class from where I have to display my value. But it is always showing me some error.
This is what I have done in urls.py:
1. from mysite.to_twitter_streaming import StdOutListener
2. url(r'^temp/$', StdOutListener().text_extract)
This StdOutListener is the class and text_extract is the function which has a value that I want to get printed.
I used this for printing:
return render_to_response('TwitterApi.html', {'link': key, 'count':self.counter})
errors that I am getting are:
1. TypeError at /temp/
2. expected string or buffer
ok in your urls.py
Change the format
url(r'^temp/$', text_extract),
I am assuming you have text_extract in your views
In your views.py
def text_extract(request):
# do your stuff
return render_to_response('TwitterApi.html', {'link': key, 'count':self.counter})
The -builtin option of SWIG has the advantage of being faster, and of being exempt of a bug with multiple inheritance.
The setback is I can't set any attribute on the generated classes or any subclass :
-I can extend a python builtin type like list, without hassle, by subclassing it :
class Thing(list):
pass
Thing.myattr = 'anything' # No problem
-However using the same approach on a SWIG builtin type, the following happens :
class Thing(SWIGBuiltinClass):
pass
Thing.myattr = 'anything'
AttributeError: type object 'Thing' has no attribute 'myattr'
How could I work around this problem ?
I found a solution quite by accident. I was experimenting with metaclasses, thinking I could manage to override the setattr and getattr functions of the builtin type in the subclass.
Doing this I discovered the builtins already have a metaclass (SwigPyObjectType), so my metaclass had to inherit it.
And that's it. This alone solved the problem. I would be glad if someone could explain why :
SwigPyObjectType = type(SWIGBuiltinClass)
class Meta(SwigPyObjectType):
pass
class Thing(SWIGBuiltinClass):
__metaclass__ = Meta
Thing.myattr = 'anything' # Works fine this time
The problem comes from how swig implemented the classes in "-builtin" to be just like builtin classes (hence the name).
builtin classes are not extensible - try to add or modify a member of "str" and python won't let you modify the attribute dictionary.
I do have a solution I've been using for several years.
I'm not sure I can recommend it because:
It's arguably evil - the moral equivalent of casting away const-ness in C/C++
It's unsupported and could break in future python releases
I haven't tried it with python3
I would be a bit uncomfortable using "black-magic" like this in production code - it could break and is certainly obscure - but at least one giant corporation IS using this in production code
But.. I love how well it works to solve some obscure features we wanted for debugging.
The original idea is not mine, I got it from:
https://gist.github.com/mahmoudimus/295200 by Mahmoud Abdelkader
The basic idea is to access the const dictionary in the swig-created type object as a non-const dictionary and add/override any desired methods.
FYI, the technique of runtime modification of classes is called monkeypatching, see https://en.wikipedia.org/wiki/Monkey_patch
First - here's "monkeypatch.py":
''' monkeypatch.py:
I got this from https://gist.github.com/mahmoudimus/295200 by Mahmoud Abdelkader,
his comment: "found this from Armin R. on Twitter, what a beautiful gem ;)"
I made a few changes for coding style preferences
- Rudy Albachten April 30 2015
'''
import ctypes
from types import DictProxyType, MethodType
# figure out the size of _Py_ssize_t
_Py_ssize_t = ctypes.c_int64 if hasattr(ctypes.pythonapi, 'Py_InitModule4_64') else ctypes.c_int
# python without tracing
class _PyObject(ctypes.Structure):
pass
_PyObject._fields_ = [
('ob_refcnt', _Py_ssize_t),
('ob_type', ctypes.POINTER(_PyObject))
]
# fixup for python with tracing
if object.__basicsize__ != ctypes.sizeof(_PyObject):
class _PyObject(ctypes.Structure):
pass
_PyObject._fields_ = [
('_ob_next', ctypes.POINTER(_PyObject)),
('_ob_prev', ctypes.POINTER(_PyObject)),
('ob_refcnt', _Py_ssize_t),
('ob_type', ctypes.POINTER(_PyObject))
]
class _DictProxy(_PyObject):
_fields_ = [('dict', ctypes.POINTER(_PyObject))]
def reveal_dict(proxy):
if not isinstance(proxy, DictProxyType):
raise TypeError('dictproxy expected')
dp = _DictProxy.from_address(id(proxy))
ns = {}
ctypes.pythonapi.PyDict_SetItem(ctypes.py_object(ns), ctypes.py_object(None), dp.dict)
return ns[None]
def get_class_dict(cls):
d = getattr(cls, '__dict__', None)
if d is None:
raise TypeError('given class does not have a dictionary')
if isinstance(d, DictProxyType):
return reveal_dict(d)
return d
def test():
import random
d = get_class_dict(str)
d['foo'] = lambda x: ''.join(random.choice((c.upper, c.lower))() for c in x)
print "and this is monkey patching str".foo()
if __name__ == '__main__':
test()
Here's a contrived example using monkeypatch:
I have a class "myclass" in module "mystuff" wrapped with swig -python -builtin
I want to add an extra runtime method "namelen" that returns the length of the name returned by myclass.getName()
import mystuff
import monkeypatch
# add a "namelen" method to all "myclass" objects
def namelen(self):
return len(self.getName())
d = monkeypatch.get_class_dict(mystuff.myclass)
d['namelen'] = namelen
x = mystuff.myclass("xxxxxxxx")
print "namelen:", x.namelen()
Note that this can also be used to extend or override methods on builtin python classes, as is demonstrated in the test in monkeypatch.py: it adds a method "foo" to the builtin str class that returns a copy of the original string with random upper/lower case letters
I would probably replace:
# add a "namelen" method to all "myclass" objects
def namelen(self):
return len(self.getName())
d = monkeypatch.get_class_dict(mystuff.myclass)
d['namelen'] = namelen
with
# add a "namelen" method to all "myclass" objects
monkeypatch.get_class_dict(mystuff.myclass)['namelen'] = lambda self: return len(self.getName())
to avoid extra global variables