TypeError during executemany() INSERT statement using a list of strings - python-2.7

I am trying to just do a basic INSERT operation to a PostgreSQL database through Python via the Psycopg2 module. I have read a great many of the questions already posted regarding this subject as well as the documentation but I seem to have done something uniquely wrong and none of the fixes seem to work for my code.
#API CALL + JSON decoding here
x = 0
for item in ulist:
idValue = list['members'][x]['name']
activeUsers.append(str(idValue))
x += 1
dbShell.executemany("""INSERT INTO slickusers (username) VALUES (%s)""", activeUsers
)
The loop creates a list of strings that looks like this when printed:
['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo']
I am just trying to have the code INSERT these strings as 1 row each into the table.
The error specified when running is:
TypeError: not all arguments converted during string formatting
I tried changing the INSERT to:
dbShell.executemany("INSERT INTO slackusers (username) VALUES (%s)", (activeUsers,) )
But that seems like it's merely treating the entire list as a single string as it yields:
psycopg2.DataError: value too long for type character varying(30)
What am I missing?

First in the code you pasted:
x = 0
for item in ulist:
idValue = list['members'][x]['name']
activeUsers.append(str(idValue))
x += 1
Is not the right way to accomplish what you are trying to do.
first list is a reserved word in python and you shouldn't use it as a variable name. I am assuming you meant ulist.
if you really need access to the index of an item in python you can use enumerate:
for x, item in enumerate(ulist):
but, the best way to do what you are trying to do is something like
for item in ulist: # or list['members'] Your example is kinda broken here
activeUsers.append(str(item['name']))
Your first try was:
['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo']
Your second attempt was:
(['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo'], )
What I think you want is:
[['b2ong'], ['dune'], ['drble'], ['drars'], ['feman'], ['got'], ['urbo']]
You could get this many ways:
dbShell.executemany("INSERT INTO slackusers (username) VALUES (%s)", [ [a] for a in activeUsers] )
or event better:
for item in ulist: # or list['members'] Your example is kinda broken here
activeUsers.append([str(item['name'])])
dbShell.executemany("""INSERT INTO slickusers (username) VALUES (%s)""", activeUsers)

Related

How to iterate and store variables in a dictionary?

I am new to python and my coding experience so far is with MATLAB.
I am trying to understand more about lists and dictionaries as i am using a library about DOEs that takes an dictionary as a passing argument.
But my trouble so far is that this dictionary assumes the form of ex.
DOE={'Elastic Modulus':[10,20,30], 'Density':[1,2,3], 'Thickness':[2,3,5]}
But i need this dictionary to be user defined, for example:
Have an input to define how many variables are needed (in this example are 3: Elastic Modulus','Density'and 'Thickness)
as the variables are defined, it should be able to store values in the dictionary over a for loop.
Is this possible using dictionaries?
Or is it better to use a list and convert in a dicionary later?
Thank you in advance
One can add keys and the corresponding values to a dict one at a time like so:
my_dict = {}
num_entries = int(input("How many entries "))
for _ in range(num_entries):
key = input("Enter the key: ")
value = input("Enter the value: ")
my_dict[key] = value
Presumably you would have a loop to do the entry of key and value for the number of values you wish to enter. Also if you are in python 2 it needs to be raw_input rather than input function. [Edit: Showing how to do the loop, since I noticed that was part of your question]

compare two dictionary, one with list of float value per key, the other one a value per key (python)

I have a query sequence that I blasted online using NCBIWWW.qblast. In my xml blast file result I obtained for a query sequence a list of hit (i.e: gi|). Each hit or gi| have multiple hsp. I made a dictionary my_dict1 where I placed gi| as key and I appended the bit score as value. So multiple values for each key.
my_dict1 = {
gi|1002819492|: [437.702, 384.47, 380.86, 380.86, 362.83],
gi|675820360| : [2617.97, 2614.37, 122.112],
gi|953764029| : [414.258, 318.66, 122.112, 86.158],
gi|675820410| : [450.653, 388.08, 386.27] }
Then I looked for max value in each key using:
for key, value in my_dict1.items():
max_value = max(value)
And made a second dictionary my_dict2:
my_dict2 = {
gi|1002819492|: 437.702,
gi|675820360| : 2617.97,
gi|953764029| : 414.258,
gi|675820410| : 450.653 }
I want to compare both dictionary. So I can extract the hsp with the highest score bits. I am also including other parameters like query coverage and identity percentage (Not shown here). The finality is to get the best gi| with the highest bit scores, coverage and identity percentage.
I tried many things to compare both dictionary like this :
First code :
matches[]
if my_dict1.keys() not in my_dict2.keys():
matches[hit_id] = bit_score
else:
matches = matches[hit_id], bit_score
Second code:
if hit_id not in matches.keys():
matches[hit_id]= bit_score
else:
matches = matches[hit_id], bit_score
Third code:
intersection = set(set(my_dict1.items()) & set(my_dict2.items()))
Howerver I always end up with 2 types of errors:
1 ) TypeError: list indices must be integers, not unicode
2 ) ... float not iterable...
Please I need some help and guidance. Thank you very much in advance for your time. Best regards.
It's not clear what you're trying to do. What is hit_id? What is bit_score? It looks like your second dict is always going to have the same keys as your first if you're creating it by pulling the max value for each key of the first dict.
You say you're trying to compare them, but don't really state what you're actually trying to do. Find those with values under a certain max? Find those with the highest max?
Your first code doesn't work because I'm assuming you're trying to use a dict key value as an index to matches, which you define as a list. That's probably where your first error is coming from, though you haven't given the lines where the error is actually occurring.
See in-code comments below:
# First off, this needs to be a dict.
matches{}
# This will never happen if you've created these dicts as you stated.
if my_dict1.keys() not in my_dict2.keys():
matches[hit_id] = bit_score # Not clear what bit_score is?
else:
# Also not sure what you're trying to do here. This will assign a tuple
# to matches with whatever the value of matches[hit_id] is and bit_score.
matches = matches[hit_id], bit_score
Regardless, we really need more information and the full code to figure out your actual goal and what's going wrong.

How to ensure that user can input any datatype (str, float, int, boolean...)?

So this is my first question on the forum and I hope I am doing it correct.
General question: How can I ensure that python does not return any errors when writing a script that allows the user to input values of different datatypes depending on the context or parameter they want to change?
More specific: I am new to python and want to write a script that allows users of The Foundry's Nuke to change values on multiple nodes of the same class at once. Depending on whether the desired parameter to change is a checkbox('bool'), and RGBA input ('4 floats')... the input has to be of a different type. Searching the forum I found that the type can be checked by type() function and compared in an if statement with the isinstance() function. I guess I could work with that, but the type of e.g. a Gradenode's multiply knob returns type 'AColor_Knob'. I expected something like float. And comparing it in an isinstance() does not give me a match regardless of the datatype I am comparing to.
Mainscript so far:
nukescripts.clear_selection_recursive()
userInput = nuke.getInput('Which type of nodes would you like to select? (!!!first char has to be capitalized!!!)',
'Shuffle')
matchingNodes = []
for each in nuke.allNodes():
if each.Class() == userInput:
matchingNodes.append(each)
else:
pass
for i in matchingNodes:
i.setSelected(True)
nuke.message(str(len(
matchingNodes)) + ' matching Nodes have been found and are now selected! (if 0 there either is no node of this type or misspelling caused an error!)')
userInput_2 = nuke.getInput('Which parameter of these nodes would you like to change? \n' +
'(!!!correct spelling can be found out by hovering over parameter in Properties Pane!!!)',
'postage_stamp')
userInput_3 = nuke.getInput('To what do you want to change the specified parameter? \n' +
'(allowed input depends on parameter type (e.g. string, int, boolean(True/False)))', 'True')
for item in matchingNodes:
item.knob(userInput_2).setValue(userInput_3)
How I checked the datatypes so far:
selected = nuke.selectedNode()
knobsel = selected.knob('multiply')
print(type(knobsel))
#if type(knobsel) == bool:
if isinstance(knobsel, (str,bool,int,float,list)):
print('match')
else:
print('no match')
You can call a TCL command with nuke.tcl(). In TCL, everything is a string, so type is irrelevant (in some commands).
p = nuke.Panel('Property Changer')
p.addSingleLineInput('class', '')
p.addSingleLineInput('knob', '')
p.addSingleLineInput('value', '')
p.show()
node_class = p.value('class')
knob_name = p.value('knob')
knob_value = p.value('value')
for node in nuke.allNodes(node_class):
tcl_exp = 'knob root.{0}.{1} "{2}"'.format(node.fullName(),knob_name,knob_value)
print tcl_exp
nuke.tcl(tcl_exp)
That should answer your question. There are many ways to approach what you're trying to do - if you want to keep it all in python, you can do type checking on the value of the knob. For example:
b = nuke.nodes.Blur()
print type(b.knob('size').value()).__name__
This creates a Blur node and prints the string value of the type. Although I don't recommend this route, you can use that to convert the value:
example = '1.5'
print type(example)
exec('new = {}(example)'.format('float'))
print type(new)
An alternative route to go down might be building yourself a custom lookup table for knob types and expected values.
Edit:
TCL Nuke Commands:
http://www.nukepedia.com/reference/Tcl/group__tcl__builtin.html#gaa15297a217f60952810c34b494bdf83d
If you press X in the nuke Node Graph or go to File > Comp Script Command, you can select TCL and run:
knob root.node_name.knob_name knob_value
Example:
knob root.Grade1.white "0 3.5 2.1 1"
This will set values for the named knob.

Print value from a lua table if pattern matches

Okay, so I just recently got into lua and find myself stuck with the following:
I have the function peripheral.getNames() (which is a custom function)
it will return a table with the structure key,value, whereas key is always a number and starts from 1 and value will be what the function finds (it searches for devices connected to it)
In my example it creates a table which looks like this
1 herp
2 derp
3 monitor_1
4 morederp
I can print the values with the following
local pgn = peripherals.getNames()
for key,value in pairs(pgn) do
setCursorPos(1,key)
write(value)
end
end
this will output the corresponding value of the table at key on my display like this
herp
derp
monitor_1
morederp
now, I try to filter my results so it only prints something if value contains 'monitor'
I tried to achive this with
for key,value in pairs(pgn) do
if string.match(value, monitor) then
#dostuff
end
end
but it always returns 'bad argument: string expected, got nil'
so obviously string.match either does not accept 'value' or, value is not a string
so i tried converting value first
for key,value in pairs(pgn) do
value = tostring(value)
if ....
#dostuff
end
end
but it still throws the same error
Do any of you have an idea how i might either get string.match to accept 'value' or if there is another method to check the contents of 'value' for a pattern while in this for loop?
The error message is talking about the variable monitor, which is not defined and so has a nil value.
Try string.match(value, "monitor").

How not to order a list of pk's in a query?

I have a list of pk's and I would like to get the result in the same order that my list is defined... But the order of the elements is begging changed. How any one help me?
print list_ids
[31189, 31191, 31327, 31406, 31352, 31395, 31309, 30071, 31434, 31435]
obj_opor=Opor.objects.in_bulk(list_ids).values()
for o in obj_oportunidades:
print o
31395 31435 31434 30071 31309 31406 31189 31191 31352 31327
This object should be used in template to show some results to the user... But how you can see, the order is different from the original list_ids
Would have been nice to have this feature in SQL - sorting by a known list of values.
Instead, what you could do is:
obj_oportunidades=Opor.objects.in_bulk(list_ids).values()
all_opor = []
for o in obj_oportunidades:
print o
all_opor.append(o)
for i in list_ids:
if i in all_opor:
print all_opor.index(i)
Downside is that you have to get all the result rows first and store them before getting them in the order you want. (all_opor could be a dictionary above, with the table records stored in the values and the PKeys as dict keys.)
Other way, create a temp table with (Sort_Order, Pkey) and add that to the query:
Sort_Order PKey
1 31189
2 31191
...
So when you sort on Sort_Order and Opor.objects, you'll get Pkeys it in the order you specify.
I found a solution in: http://davedash.com/2010/02/11/retrieving-elements-in-a-specific-order-in-django-and-mysql/ it's suited me perfectly.
ids = [a_list, of, ordered, ids]
addons = Addon.objects.filter(id__in=ids).extra(
select={'manual': 'FIELD(id,%s)' % ','.join(map(str,ids))},
order_by=['manual'])
This code do something similiar to MySQL "ORDER BY FIELD".
This guy: http://blog.mathieu-leplatre.info/django-create-a-queryset-from-a-list-preserving-order.html
Solved the problem for both MySQL and PostgreSQL!
If you are using PostgreSQL go to that page.