python sqlite3 .executemany() with named placeholders? - python-2.7

This works:
ss = 'insert into images (file_path) values(?);'
dddd = (('dd1',), ('dd2',))
conn.executemany(ss, dddd)
However this does not:
s = 'insert into images (file_path) values (:v)'
ddddd = ({':v': 'dd11'}, {':v': 'dd22'})
conn.executemany(s, ddddd)
Traceback (most recent call last):
File "/Users/Wes/.virtualenvs/ppyy/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 3035, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-31-a999de59f73b>", line 1, in <module>
conn.executemany(s, ddddd)
ProgrammingError: You did not supply a value for binding 1.
I am wondering if it is possible to use named parameters with executemany and, if so, how.
The documentation at section 11.13.3 talks generally about parameters but doesn't discuss the two styles of parameters that are described for other flavors of .executexxx().
I have checked out Python sqlite3 execute with both named and qmark parameters which does not pertain to executemany.

The source shows that execute() simply constructs a one-element list and calls executemany(), so the problem is not with executemany() itself; the same call fails with execute():
>>> conn.execute('SELECT :v', {':v': 42})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.ProgrammingError: You did not supply a value for binding 1.
As shown in the Python documentation, named parameters do not include the colon:
# And this is the named style:
cur.execute("select * from people where name_last=:who and age=:age", {"who": who, "age": age})
So you have to use ddddd = ({'v': 'dd11'}, {'v': 'dd22'}).

The : isn't part of the parameter name.
>>> s = 'insert into images (file_path) values (:v)'
>>> ddddd = ({'v': 'dd11'}, {'v': 'dd22'})
>>> conn.executemany(s, ddddd)
<sqlite3.Cursor object at 0x0000000002C0E500>
>>> conn.execute('select * from images').fetchall()
[(u'dd11',), (u'dd22',)]

Related

Python RPy2 function with multiple input arguments

I am looking to call an rPy2 function with multiple input parameters. Here is the R function write.csv that I am trying to use. It has multiple input parameters and I need to specify more than one such parameter.
If I use it without the optional parameter row.names and column.names, it works like this:
r("write.csv")(d,file='myfilename.csv')
For my requirements, I must issue this command with the optional parameters row.names and column.names. So, I tried:
r('write.csv')(d, file='myfilename.csv', row.names=FALSE, column.names=FALSE)
but I got this error message:
File "/home/UserName/test.py", line 12
r("write.csv")(d,file='myfilename.csv',row.names=FALSE, column.names=FALSE)
SyntaxError: keyword can't be an expression
[Finished in 0.0s with exit code 1]
[shell_cmd: python -u "/home/UserName/test.py"]
[dir: /home/UserName]
[path: /home/UserName/bin:/home/UserName/.local/bin:/usr/local/sbin:/usr/local/bin:
.../usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin]
How can I achieve write.csv with row.names=FALSE and column.names=FALSE, in rPy2?
You can use Python's **.
See the note here: http://rpy2.readthedocs.io/en/version_2.8.x/robjects_functions.html#callable
Ony of my mistakes was that I should have replaced . by _, as shown in the docs here:
from rpy2.robjects.packages import importr
base = importr('base')
base.rank(0, na_last = True)
so I would analogously need row_names = TRUE. However, the . in write.csv() still remained, so this only solved part of the question. Ok, so I tried a few things to get an answer:
Generating sample data:
from rpy2.robjects import r, globalenv
from rpy2.robjects import IntVector, DataFrame
d = {'a': IntVector((1,2,3)), 'b': IntVector((4,5,6))}
dataf = DataFrame(d)
Attempts follow - 1. did not work, 2. and 3. did work:
1:
r('write_csv')(x=dataf,file='testing.csv',row_names=False)
Traceback (most recent call last):
File "C:\Users\UserName\FileD\test.py", line 18, in <module>
r('write_csv')(x=dataf,file='testing.csv',row_names=False)
File "C:\Python27\lib\site-packages\rpy2\robjects\__init__.py", line 321, in __call__
res = self.eval(p)
File "C:\Python27\lib\site-packages\rpy2\robjects\functions.py", line 178, in __call__
return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
File "C:\Python27\lib\site-packages\rpy2\robjects\functions.py", line 106, in __call__
res = super(Function, self).__call__(*new_args, **new_kwargs)
rpy2.rinterface.RRuntimeError: Error in eval(expr, envir, enclos) : object 'write_csv'
..not found
Error in eval(expr, envir, enclos) : object 'write_csv' not found
2.
r('''
write_csv <- function(x,verbose=FALSE)
write.csv(x,file='testing.csv',row.names=FALSE)
''')
r['write_csv'](dataf)
3.
globalenv['dataf'] = dataf
r("write.csv(dataf,file='testing2.csv',row.names=FALSE)")
I was really hoping attempt 1. would have worked. It seemed I had reproduced the example in the docs base.rank(0, na_last = True), but I think something might have still been missing.

PYTHON: How to convert Date/Time/Tz to Epoch in Python 2.7 with Standard Library

I am scripting on a pfSense router box that is running python 2.7 and the standard library, as a result I don't have the modules pytz or dateutil to work with, and the strptime module doesn't support %z.
I need to convert date/times coming from another server (not local time) to a unix time stamp.
Is there a way I can insert the desired tz offset into a naive structured time and then convert it to GMT?
For Example consider the time 2016/08/14 02:15:10 [-0600/1] MDT coming from a remote server. My TZ is EDT -0400.
If the value was from my time zone, I could easily do something like this:
>>> t=datetime.datetime(2016,8,14,2,15,10).timetuple()
>>> print(t)
time.struct_time(tm_year=2016, tm_mon=8, tm_mday=14, tm_hour=2, tm_min=15, tm_sec=10, tm_wday=6, tm_yday=227, tm_isdst=-1)
>>> ts=time.mktime(t)
>>> print(ts)
1471155310.0
I need some way to do the conversion that allows me to pass a time zone offest (I need something like: time.mktime(t,offset=-21600) but of course it doesn't exist)
I have hard coded everything because I can easily get values, it's the time manipulation logic I need help with.
I also tried the following approach:
In strptime %z does not work.
>>> time.strptime("20160814 021510 -0600","%Y%m%d %H%M%S %z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/_strptime.py", line 478, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/local/lib/python2.7/_strptime.py", line 324, in _strptime
(bad_directive, format))
ValueError: 'z' is a bad directive in format '%Y%m%d %H%M%S %z'
Unless it is my time zone even %Z does not work.
>>> time.strptime("20160814 021510 MDT","%Y%m%d %H%M%S %Z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/_strptime.py", line 478, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/local/lib/python2.7/_strptime.py", line 332, in _strptime
(data_string, format))
ValueError: time data '20160814 021510 MDT' does not match format '%Y%m%d %H%M%S %Z'
but if on my time zone strptime DOES work:
>>> time.strptime("20160814 021510 EDT","%Y%m%d %H%M%S %Z")
time.struct_time(tm_year=2016, tm_mon=8, tm_mday=14, tm_hour=2, tm_min=15, tm_sec=10, tm_wday=6, tm_yday=227, tm_isdst=1)
Using %z for numeric timezone is only supported from python 3.x here is a fix for python 2.7
Instead of using:
datetime.strptime(t,'%Y-%m-%dT%H:%M %z')
use the timedelta to account for the timezone, like this:
from datetime import datetime,timedelta
def dt_parse(t):
ret = datetime.strptime(t[0:16],'%Y-%m-%dT%H:%M')
if t[18]=='+':
ret+=timedelta(hours=int(t[19:22]),minutes=int(t[23:]))
elif t[18]=='-':
ret-=timedelta(hours=int(t[19:22]),minutes=int(t[23:]))
return ret

name from Orange is not defined

I've setup Orange and tried to execute this code in PythonWin
And got error on 2nd line
Was my setup of Orange incomplete or it's something else?
>>> from Orange.data import *
>>> color = DiscreteVariable("color", values=["orange", "green", "yellow"])
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
NameError: name 'DiscreteVariable' is not defined
I'm not sure what the guy in the blog post is doing, or maybe there are some other steps that he explained in previous blog posts, but this code 'as is' is not going to work.
I searched the source code for Orange, and DiscreteVariable isn't mentioned anywhere, not as class, not as regular word, nothing.
What I did find however is
Discrete = core.EnumVariable
in Orange/feature/__init__.py. As you can see this points to core.EnumVariable, which appears, looking at it's usage:
orange.EnumVariable('color', values = ["green", "red"])\
to be the same as DiscreteVariable in your link.
So I suggest you use from Orange.feature import Discrete instead and use that.

What is wrong with following piece of code?

I have the following piece of code copied from book programming collective intelligence page 118, chapter "Document Filtering". This function breaks up the text into words by dividing the text on any character that isn't a letter. This leaves only actual words,all converted to lower-case.
import re
import math
def getwords(doc):
splitter=re.compile('\\W*')
words=[s.lower() for s in splitter.split(doc)
if len(s)>2 and len(s)<20]
return dict([(w,1) for w in words])
I implemented the function and got the following error:
>>> import docclas
>>> t=docclass.getwords(s)
Traceback (most recent call last):
File "<pyshell#15>", line 1, in <module>
t=docclass.getwords(s)
File "docclass.py", line 6, in getwords
words=[s.lower() for s in splitter.split(doc)
NameError: global name 'splitter' is not defined
It works here
>>> import re
>>>
>>> def getwords(doc):
... splitter=re.compile('\\W*')
... words=[s.lower() for s in splitter.split(doc)
... if len(s)>2 and len(s)<20]
... return dict([(w,1) for w in words])
...
>>> getwords ("He's fallen in the water!");
{'water': 1, 'the': 1, 'fallen': 1}
I'm gueesing you made a typo in your code, but got it right when you pasted it here.

How to store the triples in 4store

File "<console>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/django_gstudio-0.3.dev-py2.7.egg/gstudio/testing1.py", line 129, in rdf_description
store.add(self,(subject, predicate, object),context)
File "/usr/local/lib/python2.7/dist-packages/rdflib-3.2.0-py2.7.egg/rdflib/plugins/memory.py", line 298, in add
Store.add(self, triple, context, quoted)
File "/usr/local/lib/python2.7/dist-packages/rdflib-3.2.0-py2.7.egg/rdflib/store.py", line 177, in add
def add(self, (subject, predicate, object), context, quoted=False):
in
store.add(self, (subject, predicate, object), context, quoted=False)
AFAIK - rdflib does not support 4store. But you can easily assert the triples using curl and python and the 4store SPARQL Server. Here there is an example:
import subprocess
command = ["curl","-s",
"-T","/some/file/with/triples",
"-H","Content-Type: application/x-turtle",
"http://localhost:port/data/http://graph.to/save/triples"]
p = subprocess.Popen(command,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
output, err = p.communicate()
ret = p.poll()
if ret <> 0:
raise Exception, "Error asserting triples"
In this example the content type is turtle but you can use any of the other RDF serializations (ntriples, rdfxml).
If you do not want to deal with subprocesses you can also translate this call into a urllib/urllib2 function.
There are more examples in the 4store SparqlServer documentation. And, optionally, you can use any of the Python 4store client libraries.