What is the best way to extract arguments when calling function together. How can one function be run so that the other arguments get executed correctly?
How can each iterable be extracted when it is run by the executor?
As far as I know, there's no direct way to pass a dictionary and unpacking it using Executor.map.
You need to unpack it manually in single (also need to adjust the function to accept single paramter), or define wrapper function:
import concurrent.futures
def single(a, b):
return (a, b)
def single_wrapper(d): # <--- wrapper function to call single(**dict)
return single(**d)
if __name__ == '__main__':
iterable = [dict(a=1, b=2), dict(a=3, b=4)]
with concurrent.futures.ProcessPoolExecutor() as executor:
results = executor.map(single_wrapper, iterable) # instead of `single`
print(list(results))
Alternative: passing two iterables to map:
import concurrent.futures
def single(a, b):
return (a, b)
if __name__ == '__main__':
iterable = [dict(a=1,b=2), dict(a=3,b=4)]
with concurrent.futures.ProcessPoolExecutor() as executor:
iterable1 = [d['a'] for d in iterable]
iterable2 = [d['b'] for d in iterable]
results = executor.map(single, iterable1, iterable2)
# iterable1[0], iterable2[0] will be passed to single as arguments.
# iterable1[1], iterable2[1] will be passed to single as arguments.
# ....
print(list(results))
Related
How do you set/get the values of attributes of t given by x?
class Test:
def __init__(self):
self.attr1 = 1
self.attr2 = 2
t = Test()
x = "attr1"
There are built-in functions called getattr and setattr
getattr(object, attrname)
setattr(object, attrname, value)
In this case
x = getattr(t, 'attr1')
setattr(t, 'attr1', 21)
If you want to keep the logic hidden inside the class, you may prefer to use a generalized getter method like so:
class Test:
def __init__(self):
self.attr1 = 1
self.attr2 = 2
def get(self,varname):
return getattr(self,varname)
t = Test()
x = "attr1"
print ("Attribute value of {0} is {1}".format(x, t.get(x)))
Outputs:
Attribute value of attr1 is 1
Another apporach that could hide it even better would be using the magic method __getattribute__, but I kept getting an endless loop which I was unable to resolve when trying to get retrieve the attribute value inside that method.
Also note that you can alternatively use vars(). In the above example, you could exchange getattr(self,varname) by return vars(self)[varname], but getattrmight be preferable according to the answer to What is the difference between vars and setattr?.
Note: This answer is very outdated. It applies to Python 2 using the new module that was deprecated in 2008.
There is python built in functions setattr and getattr. Which can used to set and get the attribute of an class.
A brief example:
>>> from new import classobj
>>> obj = classobj('Test', (object,), {'attr1': int, 'attr2': int}) # Just created a class
>>> setattr(obj, 'attr1', 10)
>>> setattr(obj, 'attr2', 20)
>>> getattr(obj, 'attr1')
10
>>> getattr(obj, 'attr2')
20
I am a beginner in Python and I am trying to understand trees better. Why can't I add nodes inside the function f below? I would expect that the final print returns the value 2 instead of an error that f(test) in None.
class Testcls(object):
def __init__(self,data):
self.data= data
self.children = []
def add_child(self,obj):
self.children.append(obj)
def f(test):
data1 = test.data+1
test.add_child(Testcls(data1))
test = Testcls(1)
print f(test).data
Final print in your case should actually return an error. Function f doesn't return anything (hence the None if you type print f(test). To add children using f you can add a return statement:
def f(test):
data1 = test.data+1
test.add_child(Testcls(data1))
return test
Which now returns the object test which you can further access and process,
f(test)
print test.children[0].data
Since it actually returns the test object you can also call it and use it directly:
print (f(test).children)[0].data
You don't have to have a return statement either - f will still change the object you're sending to it. In case you don't want to return the object, you just need to call the function separately and only then use test like shown in the second code snippet.
I'm really at a loss here. I need to pass arguments to the wrapper; these arguments change with runtime. Any idea how this could be solved using wrappers for classes?
def wrapper(x=None):
def decorate(cls):
def fct(self):
print('wrapper argument is: %s' % x)
cls.fct = fct
return cls
return decorate
a = 'first'
#wrapper(x=a)
class Test():
pass
test = Test()
test.fct() # Prints: first
a = 'second'
test.fct() # Prints: first (instead of second)
I can't put code in a comment, so here's how you'd do it.
def wrapper(x=None):
def decorate(cls):
def fct(self):
print('wrapper argument is: %s' % x[0])
cls.fct = fct
return cls
return decorate
a = ['first']
#wrapper(x=a)
class Test():
pass
test = Test()
test.fct() # Prints: first
a[0] = 'second'
test.fct() # Prints: second
The key is that strings are immutable, whereas lists are mutable. When you pass in a string, it's basically copied. When you pass in a list, you're sort of passing a reference to the original list, which is itself mutable.
Here is a minimal example:
This works in both python 3.5 and 2.7:
class A(object):
def __init__(self, foo):
self._foo = foo
class B(A):
def __init__(self, foo):
A.__init__(self, foo=foo)
b = B(1)
Change line:
A.__init__(self, foo=foo)
to
A.__init__(self=self, foo=foo)
In python 3.5 works without problems, but in python 2.7 you will receive the following error:
Traceback (most recent call last):
File "self_key.py", line 9, in <module>
b = B(1)
File "self_key.py", line 7, in __init__
A.__init__(self=self, foo=foo)
TypeError: unbound method __init__() must be called with A instance as first argument (got nothing instead)
Is self as keyword argument forbidden in python 2.7 or is this a bug?
Update
I'm aware that python will use the first parameter of a bound function to pass the reference to the object from which has been called. I also know that the __init__ function expects that this parameter is an instance of the class. In this case A.__init__ is unbound so we must provide that parameter manually.
When I asked about self as keyword argument forbidden I'm speaking about self as "the first parameter of __init__", which is supposed to receive a reference of the object to initialize. The name of the variable itself doesn't matter. We can perfectly change the name to this:, for example:
class A(object):
def __init__(this, foo):
this._foo = foo
class B(A):
def __init__(self, foo):
A.__init__(this=self, foo=foo)
b = B(1)
And it will be the same.
My question is why when calling the function we can perfectly specify that parameter as a positional argument (A.__init__(self, foo=foo)) but when we try to pass it as a keyword argument (A.__init__(this=self, foo=foo)) python 2.7 throws an error.
Actually, you don't have to use self keyword unless you're consistent. Check out this example, where I changed self to test :
class funcs():
def init(test, a, b):
test.a=a
test.b=b
def plus(test):
c = test.a + test.b
print"%d + %d = %d" %(test.a, test.b, c)
def minus(test):
c = test.a - test.b
print"%d - %d = %d" %(test.a, test.b, c)
obj = funcs()
obj.init(10, 6)
obj.plus()
obj.minus()
You can try to mix those instance name, so it won't be named self.
class A(object):
def __init__(a_obj, foo):
a_obj._foo = foo
class B(A):
def __init__(self, test, foo):
A.__init__(self, a_obj=test, foo=foo) # here you pass object of class B and actually another object of class B
a = A(2)
b = B(a, 1)
Giving output:
A.__init__(self, a_obj=test, foo=foo)
TypeError: __init__() got multiple values for keyword argument 'a_obj'
I'm not sure what actually you want to accomplish by passing those object like this. In your code, self in class A and self in class B are not the same objects.
I suppose that here: A.__init__(self=self, foo=foo) you are doing something like A_instance=B_instance (self=self)
Always use self for the first argument to instance methods.
Source: Python Documentation
I have a repeating set of lengthy try/except1/except2/etc blocks in a series of class methods that only differ by the outside class method being called on an outside class instance. Below is a simplified version (there are actually 4 exceptions that I am handling and eight methods that only differ by the instance method being called):
class MyClass(object):
def __init__(self):
self.arg = 'foo'
def method1(self, arg1):
err = -1
y = None
try:
x = AnOutsideClass(self.arg) # Creates a class instance of an imported class
y = x.outsideclassmethod1(arg1) # Calls an instance method that returns another different class instance
except MyException1:
x.dosomething() # Needed to handle error
except MyException2:
err = 0
finally:
del x
return y, err
def method2(self, arg1, arg2, arg3):
err = -1
y = None
try:
x = AnOutsideClass(self.arg)
y = x.outsideclassmethod2(arg1, arg2, arg3) # This is the only thing changed
# A different method with different argument requirements
except MyException1:
x.dosomething()
except MyException2:
err = 0
finally:
del x
return y, err
def method3 ...
I have been trying various ways of condensing this code by trying to wrap the two statements in the try: portion of the code by using nested functions, decorators, etc, but seem to fail due to the fact that I am have trouble translating other examples of this due to: 1) am creating a class instance that needs to be used later in one of the except blocks and 2) am calling an instance method and 3) I need to return the result of the instance method.
Is there anyway of accomplishing this with partial from functools or descriptors or any other means? I have a clunky implementation currently with an extended if/elif block that picks the instance method based on an integer code that I use in a wrapper function, but am thinking there must be a more elegant way. I am relatively new to Python and am at a loss...
You could use a function factory (i.e., a function that returns a function).
def make_method(methname):
def method(self, *args):
err = -1
y = None
try:
x = AnOutsideClass(self.arg) # Creates a class instance of an imported class
y = getattr(x, methname)(*args) # Calls an instance method that returns another different class instance
except MyException1:
x.dosomething() # Needed to handle error
except MyException2:
err = 0
finally:
del x
return y, err
return method
class MyClass(object):
def __init__(self):
self.arg = 'foo'
method1 = make_method('outsideclassmethod1')
method2 = make_method('outsideclassmethod2')
The make_method is passed the outside method name as a string.
getattr is used (inside method) to get the actual method from x given the string methname.
getattr(x, 'foo') is equivalent to x.foo.
The * in def method(self, *args) tells Python that method can accept an arbitrary number of positional arguments.
Inside method, args is a tuple. The * in y = getattr(x, methname)(*args) tells Python to pass the elements in args as individual arguments to the method returned by getattr(x, methname). The * unpacking operator is explained in the docs, here, and also in this blog.