Dictionary error in Python 2.7 - python-2.7

I have a file in the format:
0000 | a1_1,a3_2 | b2_1, b3_2
0001 | a1_3 | b4_1
and I'm trying to create a dictionary which has
{ 'a1' : set(['b2', 'b3', 'b4']), 'a3': set(['b2', 'b3']) }
and this is how my code looks like:
def get_ids(row, col):
ids = set()
x = row.strip().split('|')
for a in x[col].split(','):
ids.add(a.split('_')[0])
return ids
def add_to_dictionary(funky_dictionary,key, values):
if key in funky_dictionary:
funky_dictionary[key].update(values)
else:
funky_dictionary[key] = values
def get_dict(input_file):
funky_dictionary = {}
with open(input_file,'r') as ip:
for row in ip:
a_ids = get_ids(row,1)
b_ids = get_ids(row,2)
for key in a_ids:
add_to_dictionary(funky_dictionary,key,b_ids)
return funky_dictionary
So my problem is this when I lookup values for certain key in the dictionary, it returns me with way more values than expected. E.g.
For the above example the expected value of a3 would be set(['b2', ' b3'])
However with the code, I'm getting set(['b2', ' b3', 'b4'])
I cant figure out whats wrong with the code. Any help?

The issue you have is that many of your dictionary's values are in fact references to the same set instances. In your example data, when the first line is processed, 'a1' and 'a3' both get mapped to the same set object (containing 'b2' and 'b3'). When you process the second line and call update on that set via the key 'a1', you'll see the added value through 'a3' too, since both values are references to the same set.
You need to change the code so that each value is a separate set object. I'd suggest getting rid of add_to_dictionary and just using the dictionary's own setdefault method, like this:
for key in a_ids:
funky_dictionary.setdefault(key, set()).update(b_ids)
This code always starts with a new empty set for a new key, and always updates it with new values (rather than adding a reference to the b_ids set to the dictionary directly).

Related

Fetch value of selection field instead of key

I defined a selection field in one model.
type = fields.Selection([('a','A'),('b','B'),('c','C')])
In one of function i tried to get string value instead of key.
#api.multi
def testFunc(self):
for res in self:
print'Value',res.type //It prints 'a'.
I need to print 'A'.
How can i do this?
Choose One of the solutions :
The most importing thing that you can get the selection list like this:
self._fields['type'].selection
So try this:
# convert the list to dictionary
dict(self._fields['type'].selection).get(self.type)
IF you want the label to be translated in user language:
# here the label return is translated.
value = dict(self.fields['state']._description_selection(self.evn)).get(self.type)
You can use this method, it returns the string value, translated if it's the case:
#api.multi
def testFunc(self):
for res in self:
print'Value', dict(res.fields_get(["type"],['selection'])['type']["selection"]).get(res.type)
A possible and simple solution would be:
VALUES_TYPE = [('a','A'),('b','B'),('c','C')]
type = fields.Selection(VALUES_TYPE )
dict(VALUES_TYPE )[self.type]

How to ensure that user can input any datatype (str, float, int, boolean...)?

So this is my first question on the forum and I hope I am doing it correct.
General question: How can I ensure that python does not return any errors when writing a script that allows the user to input values of different datatypes depending on the context or parameter they want to change?
More specific: I am new to python and want to write a script that allows users of The Foundry's Nuke to change values on multiple nodes of the same class at once. Depending on whether the desired parameter to change is a checkbox('bool'), and RGBA input ('4 floats')... the input has to be of a different type. Searching the forum I found that the type can be checked by type() function and compared in an if statement with the isinstance() function. I guess I could work with that, but the type of e.g. a Gradenode's multiply knob returns type 'AColor_Knob'. I expected something like float. And comparing it in an isinstance() does not give me a match regardless of the datatype I am comparing to.
Mainscript so far:
nukescripts.clear_selection_recursive()
userInput = nuke.getInput('Which type of nodes would you like to select? (!!!first char has to be capitalized!!!)',
'Shuffle')
matchingNodes = []
for each in nuke.allNodes():
if each.Class() == userInput:
matchingNodes.append(each)
else:
pass
for i in matchingNodes:
i.setSelected(True)
nuke.message(str(len(
matchingNodes)) + ' matching Nodes have been found and are now selected! (if 0 there either is no node of this type or misspelling caused an error!)')
userInput_2 = nuke.getInput('Which parameter of these nodes would you like to change? \n' +
'(!!!correct spelling can be found out by hovering over parameter in Properties Pane!!!)',
'postage_stamp')
userInput_3 = nuke.getInput('To what do you want to change the specified parameter? \n' +
'(allowed input depends on parameter type (e.g. string, int, boolean(True/False)))', 'True')
for item in matchingNodes:
item.knob(userInput_2).setValue(userInput_3)
How I checked the datatypes so far:
selected = nuke.selectedNode()
knobsel = selected.knob('multiply')
print(type(knobsel))
#if type(knobsel) == bool:
if isinstance(knobsel, (str,bool,int,float,list)):
print('match')
else:
print('no match')
You can call a TCL command with nuke.tcl(). In TCL, everything is a string, so type is irrelevant (in some commands).
p = nuke.Panel('Property Changer')
p.addSingleLineInput('class', '')
p.addSingleLineInput('knob', '')
p.addSingleLineInput('value', '')
p.show()
node_class = p.value('class')
knob_name = p.value('knob')
knob_value = p.value('value')
for node in nuke.allNodes(node_class):
tcl_exp = 'knob root.{0}.{1} "{2}"'.format(node.fullName(),knob_name,knob_value)
print tcl_exp
nuke.tcl(tcl_exp)
That should answer your question. There are many ways to approach what you're trying to do - if you want to keep it all in python, you can do type checking on the value of the knob. For example:
b = nuke.nodes.Blur()
print type(b.knob('size').value()).__name__
This creates a Blur node and prints the string value of the type. Although I don't recommend this route, you can use that to convert the value:
example = '1.5'
print type(example)
exec('new = {}(example)'.format('float'))
print type(new)
An alternative route to go down might be building yourself a custom lookup table for knob types and expected values.
Edit:
TCL Nuke Commands:
http://www.nukepedia.com/reference/Tcl/group__tcl__builtin.html#gaa15297a217f60952810c34b494bdf83d
If you press X in the nuke Node Graph or go to File > Comp Script Command, you can select TCL and run:
knob root.node_name.knob_name knob_value
Example:
knob root.Grade1.white "0 3.5 2.1 1"
This will set values for the named knob.

TypeError during executemany() INSERT statement using a list of strings

I am trying to just do a basic INSERT operation to a PostgreSQL database through Python via the Psycopg2 module. I have read a great many of the questions already posted regarding this subject as well as the documentation but I seem to have done something uniquely wrong and none of the fixes seem to work for my code.
#API CALL + JSON decoding here
x = 0
for item in ulist:
idValue = list['members'][x]['name']
activeUsers.append(str(idValue))
x += 1
dbShell.executemany("""INSERT INTO slickusers (username) VALUES (%s)""", activeUsers
)
The loop creates a list of strings that looks like this when printed:
['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo']
I am just trying to have the code INSERT these strings as 1 row each into the table.
The error specified when running is:
TypeError: not all arguments converted during string formatting
I tried changing the INSERT to:
dbShell.executemany("INSERT INTO slackusers (username) VALUES (%s)", (activeUsers,) )
But that seems like it's merely treating the entire list as a single string as it yields:
psycopg2.DataError: value too long for type character varying(30)
What am I missing?
First in the code you pasted:
x = 0
for item in ulist:
idValue = list['members'][x]['name']
activeUsers.append(str(idValue))
x += 1
Is not the right way to accomplish what you are trying to do.
first list is a reserved word in python and you shouldn't use it as a variable name. I am assuming you meant ulist.
if you really need access to the index of an item in python you can use enumerate:
for x, item in enumerate(ulist):
but, the best way to do what you are trying to do is something like
for item in ulist: # or list['members'] Your example is kinda broken here
activeUsers.append(str(item['name']))
Your first try was:
['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo']
Your second attempt was:
(['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo'], )
What I think you want is:
[['b2ong'], ['dune'], ['drble'], ['drars'], ['feman'], ['got'], ['urbo']]
You could get this many ways:
dbShell.executemany("INSERT INTO slackusers (username) VALUES (%s)", [ [a] for a in activeUsers] )
or event better:
for item in ulist: # or list['members'] Your example is kinda broken here
activeUsers.append([str(item['name'])])
dbShell.executemany("""INSERT INTO slickusers (username) VALUES (%s)""", activeUsers)

Python, is there a easier way to add values to a default key?

The program I am working does the following:
Grabs stdout from a .perl program
Builds a nested dict from the output
I'm using the AutoVivification approach found here to build a default nested dictionary. I'm using this method of defaultdict because it's easier for me to follow as a new programmer.
I'd like to add one key value to a declared key per pass of the for line in the below code. Is there a easier way to add values to a key beyond making a [list] of values then adding said values as a group?
import pprint
class Vividict(dict):
def __missing__(self, key):
value = self[key] = type(self)()
return value
reg = 'NtUser'
od = Vividict()
od[reg]
def run_rip():
os.chdir('/Users/ME/PycharmProjects/RegRipper2.8') # Path to regripper dir
for k in ntDict:
run_command = "".join(["./rip.pl", " -r
/Users/ME/Desktop/Reg/NTUSER.DAT -p ", str(k)])
process = subprocess.Popen(run_command,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
out, err = process.communicate() # wait for the process to terminate
parse(out)
# errcode = process.returncode // used in future for errorcode checking
ntDict.popitem(last=False)
def parse(data):
pattern = re.compile('lastwrite|(\d{2}:\d{2}:\d{2})|alert|trust|Value')
grouping = re.compile('(?P<first>.+?)(\n)(?P<second>.+?)
([\n]{2})(?P<rest>.+[\n])', re.MULTILINE | re.DOTALL)
if pattern.findall(data):
match = re.search(grouping, data)
global first
first = re.sub("\s\s+", " ", match.group('first'))
od[reg][first]
second = re.sub("\s\s+", " ", match.group('second'))
parse_sec(second)
def parse_sec(data):
pattern = re.compile(r'^(\(.*?\)) (.*)$')
date = re.compile(r'(.*?\s)(.*\d{2}:\d{2}:\d{2}.*)$')
try:
if pattern.match(data):
result = pattern.match(data)
hive = result.group(1)
od[reg][first]['Hive'] = hive
desc = result.group(2)
od[reg][first]['Description'] = desc
elif date.match(data):
result = date.match(data)
hive = result.group(1)
od[reg][first]['Hive'] = hive
time = result.group(2)
od[reg][first]['Timestamp'] = time
else:
od[reg][first]['Finding'] = data
except IndexError:
print('error w/pattern match')
run_rip()
pprint.pprint(od)
Sample Input:
bitbucket_user v.20091020
(NTUSER.DAT) TEST - Get user BitBucket values
Software\Microsoft\Windows\CurrentVersion\Explorer\BitBucket
LastWrite Time Sat Nov 28 03:06:35 2015 (UTC)
Software\Microsoft\Windows\CurrentVersion\Explorer\BitBucket\Volume
LastWrite Time = Sat Nov 28 16:00:16 2015 (UTC)
If I understand your question correctly, you want to change the lines where you're actually adding values to your dictionary (e.g. the od[reg][first]['Hive'] = hive line and the similar one for desc and time) to create a list for each reg and first value and then extend that list with each item being added. Your dictionary subclass takes care of creating the nested dictionaries for you, but it won't build a list at the end.
I think the best way to do this is to use the setdefault method on the inner dictionary:
od[reg][first].setdefault("Hive", []).append(hive)
The setdefault will add the second value (the "default", here an empty list) to the dictionary if the first argument doesn't exist as a key. It preempts the dictionary's __missing__ method creating the item, which is good, since we want a the value to be list rather than another layer of dictionary. The method returns the value for the key in all cases (whether it added a new value or if there was one already), so we can chain it with append to add our new hive value to the list.

How to remember objects to be treated after QuerySet.update()

I want to update the bar attribute of a bunch of Foo objects using the QuerySet's update() method. Then I want to do something with the modified objects, but I can no more identify them using the bar attribute.
Somehow it doesn't work by remembering the ids.
old_bar = Bar.objects.get(id=1)
new_bar = Bar.objects.get(id=2)
foo_query = Foo.objects.filter(bar=old_bar)
foo_ids = foo_query.values_list('id', flat=True)
print len(foo_ids), foo_query.count(), Foo.objects.filter(id__in=foo_ids).count()
# outputs "42 42 42"
print foo_query.update(bar=new_bar)
# outputs "42"
print len(foo_ids), foo_query.count(), Foo.objects.filter(id__in=foo_ids).count()
# outputs "42 0 0"
Does the update() method modify the ids or what am I doing wrong?
Querysets are lazy, so they are evaluated everytime you use them! Therefore in your example foo_query will be evaluated more than once and return something different depending on the actual objects in your database. foo_ids is therefore not a list of ids, but a ValuesQueryset that evaluates to a list.
Forcing it evaluate to a list should make your example work, as the ids do not change to reflect the actual state of the database: foo_ids = list(foo_query.values_list('id', flat=True)).