I have problem with this methode which should return both the training and the validation dataset and examine it to return the index that corresponds to the first occurrence of each class in CIFAR10.
this is code:
def get_cifar10_dataset(): """ Should create the cifar 10 network and identify the dataset index of the first time each new class
appears
:return: tuple of training and validation dataset as well as label indices
:rtype: (gluon.data.Dataset, 'dict_values' object is not subscriptable, gluon.data.Dataset,
dict[int:int])
"""
train_data = None
val_data = None
# YOUR CODE HERE
train_data = datasets.CIFAR10(train=True, root=M5_IMAGES)
val_data = datasets.CIFAR10(train=False, root=M5_IMAGES)
You are asked to return a dictionary with labels and the corresponding indexes. Using the following function can solve your problem.
def get_idx_dict(data):
lis = []
idx = []
indices = {}
for i in range(len(data)):
if data[i][1] not in lis:
lis.append(data[i][1])
idx.append(i)
indices = {lis[i]: idx[i] for i in range(len(lis))}
return indices
The function returns a dictionary with desired output. Use this function on data from train and validation set.
train_indices = get_idx_dict(train_data)
val_indices = get_idx_dict(val_data)
You can do it this
def get_cifar10_dataset():
"""
Should create the cifar 10 network and identify the dataset index of the first time each new class appears
:return: tuple of training and validation dataset as well as label indices
:rtype: (gluon.data.Dataset, dict[int:int], gluon.data.Dataset, dict[int:int])
"""
train_data = None
val_data = None
train_indices = {}
val_indices = {}
# Use `root=M5_IMAGES` for your dataset
train_data = gluon.data.vision.datasets.CIFAR10(train=True, root=M5_IMAGES)
val_data = gluon.data.vision.datasets.CIFAR10(train=False, root=M5_IMAGES)
#for train
for i in range(len(train_data)):
if train_data[i][1] not in train_indices:
train_indices[train_data[i][1]] = i
#for valid
for i in range(len(val_data)):
if val_data[i][1] not in val_indices:
val_indices[val_data[i][1]] = i
#raise NotImplementedError()
return train_data, train_indices, val_data, val_indices
I'm trying to programmatically set a value in a dictionary, potentially nested, given a list of indices and a value.
So for example, let's say my list of indices is:
['person', 'address', 'city']
and the value is
'New York'
I want as a result a dictionary object like:
{ 'Person': { 'address': { 'city': 'New York' } }
Basically, the list represents a 'path' into a nested dictionary.
I think I can construct the dictionary itself, but where I'm stumbling is how to set the value. Obviously if I was just writing code for this manually it would be:
dict['Person']['address']['city'] = 'New York'
But how do I index into the dictionary and set the value like that programmatically if I just have a list of the indices and the value?
Python
Something like this could help:
def nested_set(dic, keys, value):
for key in keys[:-1]:
dic = dic.setdefault(key, {})
dic[keys[-1]] = value
And you can use it like this:
>>> d = {}
>>> nested_set(d, ['person', 'address', 'city'], 'New York')
>>> d
{'person': {'address': {'city': 'New York'}}}
I took the freedom to extend the code from the answer of Bakuriu. Therefore upvotes on this are optional, as his code is in and of itself a witty solution, which I wouldn't have thought of.
def nested_set(dic, keys, value, create_missing=True):
d = dic
for key in keys[:-1]:
if key in d:
d = d[key]
elif create_missing:
d = d.setdefault(key, {})
else:
return dic
if keys[-1] in d or create_missing:
d[keys[-1]] = value
return dic
When setting create_missing to True, you're making sure to only set already existing values:
# Trying to set a value of a nonexistent key DOES NOT create a new value
print(nested_set({"A": {"B": 1}}, ["A", "8"], 2, False))
>>> {'A': {'B': 1}}
# Trying to set a value of an existent key DOES create a new value
print(nested_set({"A": {"B": 1}}, ["A", "8"], 2, True))
>>> {'A': {'B': 1, '8': 2}}
# Set the value of an existing key
print(nested_set({"A": {"B": 1}}, ["A", "B"], 2))
>>> {'A': {'B': 2}}
Here's another option:
from collections import defaultdict
recursivedict = lambda: defaultdict(recursivedict)
mydict = recursivedict()
I originally got this from here: Set nested dict value and create intermediate keys.
It is quite clever and elegant if you ask me.
First off, you probably want to look at setdefault.
As a function I'd write it as
def get_leaf_dict(dct, key_list):
res=dct
for key in key_list:
res=res.setdefault(key, {})
return res
This would be used as:
get_leaf_dict( dict, ['Person', 'address', 'city']) = 'New York'
This could be cleaned up with error handling and such. Also using *args rather than a single key-list argument might be nice; but the idea is that
you can iterate over the keys, pulling up the appropriate dictionary at each level.
Here is my simple solution: just write
terms = ['person', 'address', 'city']
result = nested_dict(3, str)
result[terms] = 'New York' # as easy as it can be
You can even do:
terms = ['John', 'Tinkoff', '1094535332'] # account in Tinkoff Bank
result = nested_dict(3, float)
result[terms] += 2375.30
Now the backstage:
from collections import defaultdict
class nesteddict(defaultdict):
def __getitem__(self, key):
if isinstance(key, list):
d = self
for i in key:
d = defaultdict.__getitem__(d, i)
return d
else:
return defaultdict.__getitem__(self, key)
def __setitem__(self, key, value):
if isinstance(key, list):
d = self[key[:-1]]
defaultdict.__setitem__(d, key[-1], value)
else:
defaultdict.__setitem__(self, key, value)
def nested_dict(n, type):
if n == 1:
return nesteddict(type)
else:
return nesteddict(lambda: nested_dict(n-1, type))
The dotty_dict library for Python 3 can do this. See documentation, Dotty Dict for more clarity.
from dotty_dict import dotty
dot = dotty()
string = '.'.join(['person', 'address', 'city'])
dot[string] = 'New York'
print(dot)
Output:
{'person': {'address': {'city': 'New York'}}}
Use these pair of methods
def gattr(d, *attrs):
"""
This method receives a dict and list of attributes to return the innermost value of the give dict
"""
try:
for at in attrs:
d = d[at]
return d
except:
return None
def sattr(d, *attrs):
"""
Adds "val" to dict in the hierarchy mentioned via *attrs
For ex:
sattr(animals, "cat", "leg","fingers", 4) is equivalent to animals["cat"]["leg"]["fingers"]=4
This method creates necessary objects until it reaches the final depth
This behaviour is also known as autovivification and plenty of implementation are around
This implementation addresses the corner case of replacing existing primitives
https://gist.github.com/hrldcpr/2012250#gistcomment-1779319
"""
for attr in attrs[:-2]:
# If such key is not found or the value is primitive supply an empty dict
if d.get(attr) is None or isinstance(d.get(attr), dict):
d[attr] = {}
d = d[attr]
d[attrs[-2]] = attrs[-1]
Here's a variant of Bakuriu's answer that doesn't rely on a separate function:
keys = ['Person', 'address', 'city']
value = 'New York'
nested_dict = {}
# Build nested dictionary up until 2nd to last key
# (Effectively nested_dict['Person']['address'] = {})
sub_dict = nested_dict
for key_ind, key in enumerate(keys[:-1]):
if not key_ind:
# Point to newly added piece of dictionary
sub_dict = nested_dict.setdefault(key, {})
else:
# Point to newly added piece of sub-dictionary
# that is also added to original dictionary
sub_dict = sub_dict.setdefault(key, {})
# Add value to last key of nested structure of keys
# (Effectively nested_dict['Person']['address']['city'] = value)
sub_dict[keys[-1]] = value
print(nested_dict)
>>> {'Person': {'address': {'city': 'New York'}}}
This is a pretty good use case for a recursive function. So you can do something like this:
def parse(l: list, v: str) -> dict:
copy = dict()
k, *s = l
if len(s) > 0:
copy[k] = parse(s, v)
else:
copy[k] = v
return copy
This effectively pops off the first value of the passed list l as a key for the dict copy that we initialize, then runs the remaining list through the same function, creating a new key under that key until there's nothing left in the list, whereupon it assigns the last value to the v param.
This is much easier in Perl:
my %hash;
$hash{"aaa"}{"bbb"}{"ccc"}=1; # auto creates each of the intermediate levels
# of the hash (aka: dict or associated array)
I did the code for insert records from so_parts table to so_bo table using Query...How can I use ORM method to do this kind of job. Is there any other way(best)to do that? Here is my code`
`
#api.multi
def save_rapair_parts(self, vals):
#get todays date and convert it to string
created_date = datetime.datetime.today().strftime("%m/%d/%Y")
str_date = str(created_date)
so_p_id = self.so_p_id.id
bo_status = self.bo_status
so_part_t = self.so_part_t
so_part_sno = self.so_part_sno
product = self.so_part_product
product_str = 'Repair '+str(product)
part_id = self.id
bench_order_table.search(['id','bo_sno','created_date','bo_number','rep_description','bo_status'])
#insert details intoso bench orders
`
if so_part_t=='r_b':
try:
sequence = self.env['ir.sequence'].next_by_code('so.benchorder') or '/'
str_sequence = str(sequence)
query = """SELECT so_work_authorization FROM my_depots_so WHERE id=%d """ % (so_p_id)
self.env.cr.execute(query)
result = self.env.cr.fetchall()
result_number = json.dumps(result, ensure_ascii=False)
strip_number = result_number.strip('\' \" [] ')
work_auth_no = str(strip_number)
work_auth_no += "-"
work_auth_no += str_sequence
insert ="""INSERT INTO my_depots_so_bo(id,so_bo_id,bo_sno,created_date,bo_number,rep_description,bo_status) values %s """
parameters = (part_id,so_p_id,so_part_sno,str_date,work_auth_no,product_str,bo_status)
self.env.cr.execute(insert,(parameters,))
my_depots_bo(id,bo_sno,created_date,bo_number,rep_description,bo_status) values %s """
# self.env.cr.execute(insert_query, (parameters,))
except Exception:
print "Error in inserting values"`
yes there is a better way because when you use ORM
method you also checks access right for user to:
for your select query:
rec = self.env['my.depots.so'].search_read(['id', '=', so_p_id], ['so_work_authorization'])
if rec:
rec = rec[0] # search_read return a list of dictionary
so_work_authorization = rec['so_work_authorization']
# and do what ever you want with the result
# to create
# call create method witch accept a dictionary
# field_name : value
new_rec = self.env['my.depots.so.bo'].create({
'so_bo_id': so_p_id, # many2one must be an integer value
'bo_sno': bo_nso_value,
'bo_number': value_of_number,
# ....
# ....
# add al field
}) # create return the new created record as model object
for inserting use: self.env['model.name'].create(vals)
for updating use : self.env['model.name'].write(vals)
using ORM method makes sure that user don't pass the security access rigths
Hope you get the idea
I have this class:
class Tumor(object):
"""
Wrapper for the tumor data points.
Attributes:
idNum = ID number for the tumor (is unique) (int)
malignant = label for this tumor (either 'M' for malignant
or 'B' for benign) (string)
featureNames = names of all features used in this Tumor
instance (list of strings)
featureVals = values of all features used in this Tumor
instance, same order as featureNames (list of floats)
"""
def __init__(self, idNum, malignant, featureNames, featureVals):
self.idNum = idNum
self.label = malignant
self.featureNames = featureNames
self.featureVals = featureVals
def distance(self, other):
dist = 0.0
for i in range(len(self.featureVals)):
dist += abs(self.featureVals[i] - other.featureVals[i])**2
return dist**0.5
def getLabel(self):
return self.label
def getFeatures(self):
return self.featureVals
def getFeatureNames(self):
return self.featureNames
def __str__(self):
return str(self.idNum) + ', ' + str(self.label) + ', ' \
+ str(self.featureVals)
and I am trying to use an instance of it in another function later in my code:
def train_model(train_set):
"""
Trains a logistic regression model with the given dataset
train_set (list): list of data points of type Tumor
Returns a model of type sklearn.linear_model.LogisticRegression
fit to the training data
"""
tumor = Tumor()
features = tumor.getFeatures()
labels = tumor.getLabel()
log_reg = sklearn.linear_model.LogisticRegression(train_set)
model = log_reg.fit(features, labels)
return model
However, I keep getting this error when I test my code:
TypeError: __init__() takes exactly 5 arguments (1 given)
I understand that I am not using the five arguments when I create the instance of Tumor in train_model , but how can I do so?
Arguments to the __init__ (or __new__, if you're using that) just go, predictably, where you create the instance in train_model:
tumor = Tumor(idNum, malignant, featureNames, featureVals)
Of course, you actually need values for all of these, as they are all required arguments.
You don't need to include self, however, as that first argument is taken care of automatically.
I am writting a custom widget which I want to return a list as the value. From what I can find to set the value that is returned you create a custom value_from_datadict function. I have done this
def value_from_datadict(self, data, files, name):
value = data.get(name, None)
if value:
# split the sting up so that we have a list of nodes
tag_list = value.strip(',').split(',')
retVal = []
# loop through nodes
for node in tag_list:
# node string should be in the form: node_id-uuid
strVal = str(node).split("-")
uuid = strVal[-1]
node_id = strVal[0]
# create a tuple of node_id and uuid for node
retVal.append({'id': node_id, 'uuid': uuid})
if retVal:
# if retVal is not empty. i.e. we have a list of nodes
# return this. if it is empty then just return whatever is in data
return retVal
return value
I expect this to return a list but when I print out the value it is returned as a string rather than a list. The string itself contains the right text but as i said it is a string and not a list. An example of what is returned could be
[{'id': '1625', 'uuid': None}]
but if I did str[0] it would print out [ instead of {'id': '1625', 'uuid': None}
How can I stop it from converting my list into a string?
Thanks
Well, it's simple: if you have a CharField, then you will get a string as a result, because CharField uses method to_python, that coerces the result to string. You need to create your own Field for this one and return a list.
OLD
Could you post the result of:
x = value_from_datadict(..)
print type(x)
so we can see, what exactly is returned?
And could you post the whole test case you are using to deliver the example?