Groovy/Grails groupBy dynamic list of fieldnames - list

i want to group a list of objects by a dynamic list of fields (given by name). I'll try to explain it with an example.
Let's say I have a class looking similar to this:
def SomeClass{
String one
String two
String three
String four
//etc..
}
Now I have a changeable list of 1-4 field names, therefore hardcoding is not an option, like this:
def fields = ["one", "two"]
//or
def fields2 = ["two", "three", "four"]
I want to sort lists of SomeClass, and as result I need a map with a list of values as key and the objects as value; Looking like this:
//group by fields:
def result = [[someClass.one, someClass.two]:[List<SomeClass>],...]
//group by fields2:
def result2 = [[someClass.two, someClass.three, someClass.four]:[List<SomeClass>],...]
I tried splitting the fields and create closures for .groupBy(), but I retrieve a nested Map.
With .collectEntries() i'm not sure how to pass a changeable list of fields to the closure.
The lists I want to sort contain around 500-10000 elements, with 1-4 fields I want to group by.

You can use groupBy with a Closure and collect your fields inside it:
import groovy.transform.*
#Canonical
class SomeClass{
String one
String two
String three
}
List<SomeClass> list = [
new SomeClass('a', '1', 'i'),
new SomeClass('a', '1', 'ii'),
new SomeClass('a', '2', 'i'),
new SomeClass('a', '2', 'ii'),
new SomeClass('a', '3', 'i'),
]
def fields = ['one', 'two']
Map<List<String>,List<SomeClass>> grouped = list.groupBy { o -> fields.collect { o."$it" } }
assert grouped == [
['a', '1']:[new SomeClass('a', '1', 'i'), new SomeClass('a', '1', 'ii')],
['a', '2']:[new SomeClass('a', '2', 'i'), new SomeClass('a', '2', 'ii')],
['a', '3']:[new SomeClass('a', '3', 'i')]
]

Related

GBQ: SchemaField REPEATED mode writes empty array

I am trying to write a simple dataframe to gbq. One of the field is array so I tried using REPEATED mode. But it seems nothing is written to the array
df = pd.DataFrame(
{
'my_string': ['a', 'b', 'c'],
'my_int64': [1, 2, 3],
'my_float64': [4.0, 5.0, 6.0],
'my_timestamp': [
pd.Timestamp("1998-09-04T16:03:14"),
pd.Timestamp("2010-09-13T12:03:45"),
pd.Timestamp("2015-10-02T16:00:00")
],
'my_array': [
[1,2,3],
[4,5,6],
[7,8,9]
]
}
)
# client = bigquery.Client()
table_id = 'junk.pshah2_new_table'
# Since string columns use the "object" dtype, pass in a (partial) schema
# to ensure the correct BigQuery data type.
job_config = bigquery.LoadJobConfig(schema=[
bigquery.SchemaField("my_string", "STRING"),
bigquery.SchemaField("my_array", "INTEGER","REPEATED"),
])
job = client.load_table_from_dataframe(
df, table_id, job_config=job_config
)
# Wait for the load job to complete.
job.result()
In GBQ I do not find my_array written at all
Am I doing something wrong here?

Lists within dictionaries

I'm using Python and trying to access some items in a list within a dictionary. I do not need to loop, just need the item value. Please see this:
<pre>
BROKER_INFO = {'AMERITRADE': {
'file_info': {
'broker_name' : 'TD Ameritrade',
'filename' : 'ameritrade',
'row_start' : '1',
'active' : 'Y',
'actions' : 'Bought,Sold',
'buy' : 'Bought',
'sell' : 'Sold',
'columns' : [
{'date': '0', 'action': '2', 'ticker': '4'}]
}
</pre>
I'm looking to get the value to "date", "action", "ticker' within "columns", which is within "file_info" and "AMERITRADE".
I tried this: BROKER_INFO[p_broker]['file_info']['columns']['action'], but no luck.
Many many thanks!
Try
BROKER_INFO[p_broker]['file_info']['columns'][0]['action']

How to compare elements of nested lists between each other?

I try to compare elements of nested list between each other. Let's say I have the following list:
list1 = [['v1', '1', '2'],['v1', '2', '2'], ['v2', '1'], ['v3'], ['v4', '1'], ['v4', '2']]
and I would like to reach to:
result = [['v1', '2'],['v1', '2'],['v2', '1'], ['v3'], ['v4'], ['v4']]
I've done a small code, but it does not look to work very well.
for i in range(1, len(list1) - 1):
previousone = list1[i-1]
currentone = list1[i]
nextone = list1[i+1]
lenprevious = len(previousone)
lencurrent = len(currentone)
lennext = len(nextone)
minlen = min(lenprevious,lencurrent,lennext) -1
common = ''
for j in range(minlen):
if j == 0:
if previousone[j] == currentone[j]:
common += str(previousone[j])
if previousone[j] != currentone[j]:
if currentone[j] == nextone[j]:
common += str(currentone[j])
else:
common += currentone
break
else:
if common != '':
if previousone[j] == currentone[j]:
common.join('_',str(nextone[j]))
else:
if currentone[j] == nextone[j]:
common.join('_',str(nextone[j]))
else:
break
else:
break
print common
result.append(common)
The idea, is to compare the 1st element of the sub-list vs the 1st element of the previous sub-list. If no match, then we compare with the next sub-list. If no match, we get in common the 1st element of the current sub-list.
Then, if it's matching, we do the same for the next element of the sub-list and so one until the last one. In the end, I want to have in common, a list of the common elements if any, if not I want the current sub-list.
Does anyone has any idea how to make it work? Thanks in advance!
EDIT ::
The logic would be:
Iteration 1 -> Previous : ['v1', '1', '2'] and Current : ['v1', '2', '2'] and Next : ['v2', '1']
We compare each element from each list.
First, we compare the Previous and the Current.
First element of those lists are 'v1', so we append 'v1' in result and we go to the next element, here '1' and '2'.
They are not the same so we pass until the next element, which are '2' and '2': identical.
We append in result to get Result :
[['v1', '2'], ['v1', '2'], [], [], [], []]
Iteration 2 -> Previous : ['v1', '2', '2'] and Current : ['v2', '1'] and Next : ['v3']
First we compare Previous and Current. 'v1' different from 'v2'.
So we compare Current and Next. 'v2' different from 'v3'.
So we append in result the current and we get:
[['v1', '2'], ['v1', '2'], ['v2', '1'], [], [], []]
Iteration 3 -> Previous : ['v2', '1'] and Current : ['v3'] and Next : ['v4', '1']
Same as above, 'v2' different from 'v3' and 'v3' different from 'v4' so we append the current and get:
[['v1', '2'], ['v1', '2'], ['v2', '1'], ['v3'], [], []]
Iteration 4 -> Previous : ['v3'] and Current : ['v4', '1'] and Next: ['v4', '2']
'v3' different from 'v4' so we compare Current and Next: 'v4' is common so we append 'v4':
[['v1', '2'], ['v1', '2'], ['v2', '1'], ['v3'], ['v4'], []]
Iteration 5 -> Previous : ['v4', '1'] and Current : ['v4', '2'] and Next : ??
'v4' is common so we append 'v4' and get the final result:
Result: [['v1', '2'], ['v1', '2'], ['v2', '1'], ['v3'], ['v4'], ['v4']]
But I do not know how to reach there..
Below is the implementation according to your desired result,
Core Logic
def intersection(lst1, lst2):
return list(set(lst1) & set(lst2))
list1 = [['v1', '1', '2'],['v1', '2', '2'], ['v2', '1'], ['v3'], ['v4', '1'], ['v4', '2']]
result = []
list_len = len(list1)
if list_len == 0:
pass
elif list_len == 1:
result.append(list1)
else:
for i in range(list_len):
if i == 0:
current = list1[i]
next = list1[i+1]
print("Iteration {} -> Previous : No previous available Current : {} Next: {}".format(i, current, next))
if current[0] == next[0]:
result.append(intersection(current, next))
else:
result.append(current)
elif i == list_len - 1:
previous = list1[i-1]
current = list1[i]
print("Iteration {} -> Previous : {} Current : {} Next: No next available".format(i, previous, current))
if current[0] != previous[0]:
result.append(current)
else:
result.append(intersection(current, previous))
else:
previous = list1[i-1]
current = list1[i]
next = list1[i+1]
print("Iteration {} -> Previous : {} Current : {} Next: {}".format(i, previous, current, next))
if current[0] == previous[0]:
result.append(intersection(current, previous))
else:
if current[0] == next[0]:
result.append(intersection(current, next))
else:
result.append(current)
print("Result : {}".format(result))
Output
Iteration 0 -> Previous : No previous available Current : ['v1', '1', '2'] Next: ['v1', '2', '2']
Iteration 1 -> Previous : ['v1', '1', '2'] Current : ['v1', '2', '2'] Next: ['v2', '1']
Iteration 2 -> Previous : ['v1', '2', '2'] Current : ['v2', '1'] Next: ['v3']
Iteration 3 -> Previous : ['v2', '1'] Current : ['v3'] Next: ['v4', '1']
Iteration 4 -> Previous : ['v3'] Current : ['v4', '1'] Next: ['v4', '2']
Iteration 5 -> Previous : ['v4', '1'] Current : ['v4', '2'] Next: No next available
Result : [['v1', '2'], ['v1', '2'], ['v2', '1'], ['v3'], ['v4'], ['v4']]

groovy: create a list of values with all strings

I am trying to iterate through a map and create a new map value. The below is the input
def map = [[name: 'hello', email: ['on', 'off'] ], [ name: 'bye', email: ['abc', 'xyz']]]
I want the resulting data to be like:
[hello: ['on', 'off'], bye: ['abc', 'xyz']]
The code I have right now -
result = [:]
map.each { key ->
result[random] = key.email.each {random ->
"$random"
}
}
return result
The above code returns
[hello: [on, off], bye: [abc, xyz]]
As you can see from above, the quotes from on, off and abc, xyz have disappeared, which is causing problems for me when i am trying to do checks on the list value [on, off]
It should not matter. If you see the result in Groovy console, they are still String.
Below should be sufficient:
map.collectEntries {
[ it.name, it.email ]
}
If you still need the single quotes to create a GString instead of a String, then below tweak would be required:
map.collectEntries {
[ it.name, it.email.collect { "'$it'" } ]
}
I personally do not see any reasoning behind doing the later way. BTW, map is not a Map, it is a List, you can rename it to avoid unnecessary confusions.
You could convert it to a json object and then everything will have quotes
This does it. There should/may be a groovier way though.
def listOfMaps = [[name: 'hello', email: ['on', 'off'] ], [ name: 'bye', email: ['abc', 'xyz']]]
def result = [:]
listOfMaps.each { map ->
def list = map.collect { k, v ->
v
}
result[list[0]] = ["'${list[1][0]}'", "'${list[1][1]}'"]
}
println result

How can I put the elements in a list into a dictionary in Python

I have a list:
txtlst = [
['000001', 'DOE', 'JOHN', 'COMSCI', '', 'MATH', '', 'ENGLISH\n'],
['000002', 'DOE', 'JANE', 'FRENCH', '', 'MUSIC', '', 'COMSCI\n']
]
And I want to put the elements in a dictionary so it looks likes this
mydict = {
'000001': ['000001', 'DOE', 'JOHN', 'COMSCI', '', 'MATH', '', 'ENGLISH\n'],
'000002': ['000002', 'DOE', 'JANE', 'FRENCH', '', 'MUSIC', '', 'COMSCI\n']
}
My problem here is, after I ran the code
for i in txtlst:
key = i[0]
value = i
mydict = {key:value}
The two sublists of txtlst are added to different dictionaries. How can I fix my code so they will be in the same dictionary as I mentioned above?
You can easily create a new dictionary with the first element of each list as key:
mydict = { i[0]: i for i in txtlst }
If you wish to do that in a loop like in your approach, you need to initialize a dictionary beforehand and update it in each iteration:
mydict = {}
for i in txtlst:
key = i[0]
value = i
mydict[key] = value