Is it possible to simulate extended tuple unpacking in Python 2?
Specifically, I have a for loop:
for a, b, c in mylist:
which works fine when mylist is a list of tuples of size three. I want the same for loop to work if I pass in a list of size four.
I think I will end up using named tuples, but I was wondering if there is an easy way to write:
for a, b, c, *d in mylist:
so that d eats up any extra members.
You can't do that directly, but it isn't terribly difficult to write a utility function to do this:
>>> def unpack_list(a, b, c, *d):
... return a, b, c, d
...
>>> unpack_list(*range(100))
(0, 1, 2, (3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99))
You could apply it to your for loop like this:
for sub_list in mylist:
a, b, c, d = unpack_list(*sub_list)
You could define a wrapper function that converts your list to a four tuple. For example:
def wrapper(thelist):
for item in thelist:
yield(item[0], item[1], item[2], item[3:])
mylist = [(1,2,3,4), (5,6,7,8)]
for a, b, c, d in wrapper(mylist):
print a, b, c, d
The code prints:
1 2 3 (4,)
5 6 7 (8,)
For the heck of it, generalized to unpack any number of elements:
lst = [(1, 2, 3, 4, 5), (6, 7, 8), (9, 10, 11, 12)]
def unpack(seq, n=2):
for row in seq:
yield [e for e in row[:n]] + [row[n:]]
for a, rest in unpack(lst, 1):
pass
for a, b, rest in unpack(lst, 2):
pass
for a, b, c, rest in unpack(lst, 3):
pass
You can write a very basic function that has exactly the same functionality as the python3 extended unpack. Slightly verbose for legibility. Note that 'rest' is the position of where the asterisk would be (starting with first position 1, not 0)
def extended_unpack(seq, n=3, rest=3):
res = []; cur = 0
lrest = len(seq) - (n - 1) # length of 'rest' of sequence
while (cur < len(seq)):
if (cur != rest): # if I am not where I should leave the rest
res.append(seq[cur]) # append current element to result
else: # if I need to leave the rest
res.append(seq[cur : lrest + cur]) # leave the rest
cur = cur + lrest - 1 # current index movded to include rest
cur = cur + 1 # update current position
return(res)
Python 3 solution for those that landed here via an web search:
You can use itertools.zip_longest, like this:
from itertools import zip_longest
max_params = 4
lst = [1, 2, 3, 4]
a, b, c, d = next(zip(*zip_longest(lst, range(max_params))))
print(f'{a}, {b}, {c}, {d}') # 1, 2, 3, 4
lst = [1, 2, 3]
a, b, c, d = next(zip(*zip_longest(lst, range(max_params))))
print(f'{a}, {b}, {c}, {d}') # 1, 2, 3, None
For Python 2.x you can follow this answer.
Related
I have a list of 51 elements.
keypoints = ['0.49395501613616943', '0.3686272203922272', '0.9999948740005493'...]. The original index of my list goes in order from 0,1,2...50. But i have to set new indices to the elements of the list and order them accordingly in a new list. How can i do that?
My new indices:
NEW_INDEX = [39, 40, 41, 21, 22, 23,
27, 28, 29, 33, 34, 35, 24, 25, 26,
30, 31, 32, 36, 37, 38, 42, 43, 44, 45, 46, 47,
48, 49, 50, 3, 4, 5, 9, 10, 11, 15, 16, 17,
6, 7, 8, 12, 13, 14, 18, 19, 20, 0, 1, 2]
Assuming this question is about python syntax.
In pure python, you can use a list comprehension and indices:
keypoints = ['0.49395501613616943', '0.3686272203922272', '0.9999948740005493', '0.9473770229920709', '0.7699773520016487', '0.540962426318277']
new_index = [2, 1, 5, 4, 0, 3]
ordered_keypoints = [keypoints[i] for i in new_index]
print(ordered_keypoints)
# ['0.9999948740005493', '0.3686272203922272', '0.540962426318277', '0.7699773520016487', '0.49395501613616943', '0.9473770229920709']
Using numpy, there is an even more convenient notation:
import numpy as np
keypoints = np.array(['0.49395501613616943', '0.3686272203922272', '0.9999948740005493', '0.9473770229920709', '0.7699773520016487', '0.540962426318277'])
new_index = np.array([2, 1, 5, 4, 0, 3])
ordered_keypoints = keypoints[new_index]
print(ordered_keypoints)
# ['0.9999948740005493' '0.3686272203922272' '0.540962426318277'
# '0.7699773520016487' '0.49395501613616943' '0.9473770229920709']
I have a list of integers placed in order. What I want is to get groups of consecutive integers from the list.
For example:
Input: [19, 20, 21, 22, 23, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 59, 60, 61, 62, 63, 64, 65, 66, 73, 74, 75, 76, 77]
Desired Output: [[19, 20, 21, 22, 23], [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], [59, 60, 61, 62, 63, 64, 65, 66], [73, 74, 75, 76, 77]]
Here is my code:
private fun getConsecutiveNumbers(srcList: ArrayList<Int>): List<List<Int>> {
val listConsecNums = ArrayList<List<Int>>()
var subList = ArrayList<Int>()
for (i in 0 until srcList.size) {
subList.add(srcList[i])
if ((i + 1 >= srcList.size) || (srcList[i] + 1 != srcList[i + 1])) {
listConsecNums.add(subList)
subList = ArrayList()
}
}
return listConsecNums
}
Here, the code assumes that the numbers are consecutive and collects them in a list until a non-consecutive number is found.
This code works in the general case for what I want, but I was just wondering if there is a more efficient Kolin-y way to do the same.
Any suggestions would be greatly appreciated.
fold with mutable lists. If the element is consecutive, then add it to the last array, otherwise create a new array with this element
fun getConsecutiveNumbers(srcList: List<Int>): List<List<Int>> {
return srcList.fold(mutableListOf<MutableList<Int>>()) { acc, i ->
if (acc.isEmpty() || acc.last().last() != i - 1) {
acc.add(mutableListOf(i))
} else acc.last().add(i)
acc
}
}
Here is a different approach that is more compact, but it's less efficient because it creates two intermediate throw-away Lists.
First get a list of the indices matching the beginning of each consecutive run. Then zip the range starts and ends to pull out the relevant ranges of the original list. But you have to add an extra value at the end before zipping so the last range can be captured.
fun List<Int>.splitByConsecutive(): List<List<Int>> {
val rangeStarts = mapIndexedNotNull { index, i ->
if (index == 0 || i - 1 != this[index - 1]) index else null
}
return (rangeStarts + listOf(size)).zipWithNext(::subList)
}
With sequences it doesn't need intermediate lists, but it's still more steps:
fun List<Int>.splitByConsecutive(): List<List<Int>> {
val rangeStarts = asSequence().mapIndexedNotNull { index, i ->
if (index == 0 || i - 1 != this[index - 1]) index else null
}
return (rangeStarts + sequenceOf(size)).zipWithNext(::subList).toList()
}
I have data frame in pandas and I have written a function to use the information in each row to generate a new column. I want the result to be in a list format:
A B C
3 4 1
4 2 5
def Computation(row):
if row['B'] >= 3:
return [s for s in range(row['C'],50)]
else:
return [s for s in range(row['C']+2,50)]
df['D'] = df.apply(Computation, axis = 1)
However, I am getting the following error:
"could not broadcast input array from shape (308) into shape (9)"
Could you please tell me how to solve this problem?
Say you start with
In [25]: df = pd.DataFrame({'A': [3, 4], 'B': [4, 2], 'C': [1, 5]})
Then there are at least two ways to do it.
You can apply twice on the C column, but switch on the B column:
In [26]: np.where(df.B >= 3, df.C.apply(lambda c: [s for s in range(c, 50)]), df.C.apply(lambda c: [s for s in range(c + 2, 50)]))
Out[26]:
array([ [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
[7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]], dtype=object)
Or you can apply on the entire row and switch on the B value per row:
In [27]: df.apply(lambda r: [s for s in range(r.C, 50)] if r.B >= 3 else [s for s in range(r.C + 2, 50)], axis=1)
Out[27]:
0 [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14...
1 [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, ...
Note that the return types are different, but, in each case, you can still write
df['foo'] = <each one of the above options>
This is my training function:
def train(input_layer_data, output_layer_data, dnn, stn):
ds = SupervisedDataSet(len(input_layer_data), len(output_layer_data))
ds.addSample(input_layer_data, output_layer_data)
if 'network' in dnn[stn]:
net_dumped = dnn[stn]['network']
net = pickle.loads(net_dumped)
else:
net = buildNetwork(len(input_layer_data), 50, len(output_layer_data), hiddenclass=SigmoidLayer, outclass = SigmoidLayer)
trainer = BackpropTrainer(net, ds)
trainer.trainEpochs(1)
trnresult = percentError( trainer.testOnClassData(), input_layer_data )
print "epoch: %4d" % trainer.totalepochs, \
" train error: %5.2f%%" % trnresult
return net
I call this function with a single input and output data repeatedly.
And this is the output it generates,
inp=[48, 48, 8, 69, 69, 8, 57, 57, 8, 67, 67, 8, 71, 71, 8, 75, 75, 8, 71, 71, 8]
out=[27, 27, 8, 71, 71, 8, 75, 75, 8, 71, 71, 8, 67, 67, 8, 57, 57, 8, 69, 69, 8]
epoch: 0 train error: 2100.00%
FeedForwardNetwork-152
Modules:
[<BiasUnit 'bias'>, <LinearLayer 'in'>, <SigmoidLayer 'hidden0'>, <SigmoidLayer 'out'>]
Connections:
[<FullConnection 'FullConnection-148': 'bias' -> 'out'>, <FullConnection 'FullConnection-149': 'bias' -> 'hidden0'>, <FullConnection 'FullConnection-150': 'in' -> 'hidden0'>, <FullConnection 'FullConnection-151': 'hidden0' -> 'out'>]
I don't understand such huge error.
The error continues through the whole program(this is for just one call).
How do I reduce the error?
The relationship between Foo and Bar is through Baz as follows:
class Foo(Model):
# stuff
class Bar(Model)
# stuff
class Baz(Model):
foos = ManyToManyField("Foo")
bar = ForeignKey("Bar")
I basically need to generate the following dict representing the Bars that are related to each Foo through Baz (in dict comprehension pseudo-code):
{ foo.id: [list of unique bars related to the foo through any baz] for foo in all foos}
I can currently generate my data structure with O(N) queries (1 query per Foo), but with lots of data this is a bottleneck, and I need it optimized to O(1) (not a single query per se, but a fixed number of queries irrespective of data size of any of the models), while also minimizing iterations of the data in python.
If you can drop to SQL, you could use the single query (the appname should prefix all the tables names):
select distinct foo.id, bar.id
from baz_foos
join baz on baz_foos.baz_id = baz.id
join foo on baz_foos.foo_id = foo.id
join bar on baz.bar_id = bar.id
baz_foos is the many-to-many table Django creates.
#Alasdair's solution is possibly/probably more readable (although if you're doing this for performance reasons that might not be most important). His solution uses exactly two queries (which is hardly a difference). The only problem I see is if you have a large number of Baz objects since the generated sql looks like this:
SELECT "foobar_baz"."id", "foobar_baz"."bar_id", "foobar_bar"."id"
FROM "foobar_baz"
INNER JOIN "foobar_bar" ON ("foobar_baz"."bar_id" = "foobar_bar"."id")
SELECT
("foobar_baz_foos"."baz_id") AS "_prefetch_related_val",
"foobar_foo"."id"
FROM "foobar_foo"
INNER JOIN "foobar_baz_foos" ON ("foobar_foo"."id" = "foobar_baz_foos"."foo_id")
WHERE "foobar_baz_foos"."baz_id" IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54,
55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74,
75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94,
95, 96, 97, 98, 99, 100, 101)
If you have only a few Bar's and a few hundred Foo's, I would do:
from django.db import connection
from collections import defaultdict
# foos = {f.id: f for f in Foo.objects.all()}
bars = {b.id: b for b in Bar.objects.all()}
c = connection.cursor()
c.execute(sql) # from above
d = defaultdict(set)
for f_id, b_id in c.fetchall():
d[f_id].add(bars[b_id])
Using select_related and prefetch_related, I think you can build the required data structure with 2 queries:
out = {}
bazes = Baz.objects.select_related('bar').prefetch_related('foos')
for baz in bazes:
for foo in baz.foos.all():
out.setdefault(foo.id, set()).add(baz.bar)
The values of the output dictionary are sets, not lists as in your question, to ensure uniqueness.