Ruby NArray.to_na() and Python numpy.array() - python-2.7

Suppose I have the following string.
irb(main):074:0> line = "#!/usr/bin/ruby\n#\n# Gen"
irb(main):078:0> NArray.to_na(line,Float)
=> NArray.float(3):
[ 9.05457e+164, 3.30198e-258, 6.1531e+223 ]
How do I mimic this behavior with Python using numpy.array?
I tried the following, but it did not work.
>>> line = '#!/usr/bin/ruby\n#\n# Gen'
>>> np.array(line,float)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: could not convert string to float: #!/usr/bin/ruby
#
# Gen
Then I tried to convert it to bytes, but that did not work either.
>>> bytes = bytearray(line, 'utf-8')
>>> np.array(bytes,float)
array([ 35., 33., 47., 117., 115., 114., 47., 98., 105.,
110., 47., 114., 117., 98., 121., 10., 35., 10.,
35., 32., 32., 71., 101., 110.])
How do I resolve this?

You can easily achieve this by using the fromstring method of numpy:
import numpy as np
line = "#!/usr/bin/ruby\n#\n# Gen"
array = np.fromstring(line, dtype=float)
print array
Executing the above code results in
[ 9.05457127e+164 3.30197767e-258 6.15310337e+223]

Related

python sqlite3 .executemany() with named placeholders?

This works:
ss = 'insert into images (file_path) values(?);'
dddd = (('dd1',), ('dd2',))
conn.executemany(ss, dddd)
However this does not:
s = 'insert into images (file_path) values (:v)'
ddddd = ({':v': 'dd11'}, {':v': 'dd22'})
conn.executemany(s, ddddd)
Traceback (most recent call last):
File "/Users/Wes/.virtualenvs/ppyy/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 3035, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-31-a999de59f73b>", line 1, in <module>
conn.executemany(s, ddddd)
ProgrammingError: You did not supply a value for binding 1.
I am wondering if it is possible to use named parameters with executemany and, if so, how.
The documentation at section 11.13.3 talks generally about parameters but doesn't discuss the two styles of parameters that are described for other flavors of .executexxx().
I have checked out Python sqlite3 execute with both named and qmark parameters which does not pertain to executemany.
The source shows that execute() simply constructs a one-element list and calls executemany(), so the problem is not with executemany() itself; the same call fails with execute():
>>> conn.execute('SELECT :v', {':v': 42})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.ProgrammingError: You did not supply a value for binding 1.
As shown in the Python documentation, named parameters do not include the colon:
# And this is the named style:
cur.execute("select * from people where name_last=:who and age=:age", {"who": who, "age": age})
So you have to use ddddd = ({'v': 'dd11'}, {'v': 'dd22'}).
The : isn't part of the parameter name.
>>> s = 'insert into images (file_path) values (:v)'
>>> ddddd = ({'v': 'dd11'}, {'v': 'dd22'})
>>> conn.executemany(s, ddddd)
<sqlite3.Cursor object at 0x0000000002C0E500>
>>> conn.execute('select * from images').fetchall()
[(u'dd11',), (u'dd22',)]

Understanding struct.pack in python 2.7 and 3.5+

I am attempting to understand; and resolve, why the following happens:
$ python
>>> import struct
>>> list(struct.pack('hh', *(50,50)))
['2', '\x00', '2', '\x00']
>>> exit()
$ python3
>>> import struct
>>> list(struct.pack('hh', *(50, 50)))
[50, 0, 50, 0]
I understand that hh stands for 2 shorts. I understand that struct.pack is converting the two integers (shorts) to a c style struct. But why does the output in 2.7 differ so much from 3.5?
Unfortunately I am stuck with python 2.7 for right now on this project and I need the output to be similar to one from python 3.5
In response to comment from Some Programmer Dude
$ python
>>> import struct
>>> a = list(struct.pack('hh', *(50, 50)))
>>> [int(_) for _ in a]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: ''
in python 2, struct.pack('hh', *(50,50)) returns a str object.
This has changed in python 3, where it returns a bytes object (difference between binary and string is a very important difference between both versions, even if bytes exists in python 2, it is the same as str).
To emulate this behaviour in python 2, you could get ASCII code of the characters by appling ord to each char of the result:
map(ord,struct.pack('hh', *(50,50)))

PYTHON: How to convert Date/Time/Tz to Epoch in Python 2.7 with Standard Library

I am scripting on a pfSense router box that is running python 2.7 and the standard library, as a result I don't have the modules pytz or dateutil to work with, and the strptime module doesn't support %z.
I need to convert date/times coming from another server (not local time) to a unix time stamp.
Is there a way I can insert the desired tz offset into a naive structured time and then convert it to GMT?
For Example consider the time 2016/08/14 02:15:10 [-0600/1] MDT coming from a remote server. My TZ is EDT -0400.
If the value was from my time zone, I could easily do something like this:
>>> t=datetime.datetime(2016,8,14,2,15,10).timetuple()
>>> print(t)
time.struct_time(tm_year=2016, tm_mon=8, tm_mday=14, tm_hour=2, tm_min=15, tm_sec=10, tm_wday=6, tm_yday=227, tm_isdst=-1)
>>> ts=time.mktime(t)
>>> print(ts)
1471155310.0
I need some way to do the conversion that allows me to pass a time zone offest (I need something like: time.mktime(t,offset=-21600) but of course it doesn't exist)
I have hard coded everything because I can easily get values, it's the time manipulation logic I need help with.
I also tried the following approach:
In strptime %z does not work.
>>> time.strptime("20160814 021510 -0600","%Y%m%d %H%M%S %z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/_strptime.py", line 478, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/local/lib/python2.7/_strptime.py", line 324, in _strptime
(bad_directive, format))
ValueError: 'z' is a bad directive in format '%Y%m%d %H%M%S %z'
Unless it is my time zone even %Z does not work.
>>> time.strptime("20160814 021510 MDT","%Y%m%d %H%M%S %Z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/_strptime.py", line 478, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/local/lib/python2.7/_strptime.py", line 332, in _strptime
(data_string, format))
ValueError: time data '20160814 021510 MDT' does not match format '%Y%m%d %H%M%S %Z'
but if on my time zone strptime DOES work:
>>> time.strptime("20160814 021510 EDT","%Y%m%d %H%M%S %Z")
time.struct_time(tm_year=2016, tm_mon=8, tm_mday=14, tm_hour=2, tm_min=15, tm_sec=10, tm_wday=6, tm_yday=227, tm_isdst=1)
Using %z for numeric timezone is only supported from python 3.x here is a fix for python 2.7
Instead of using:
datetime.strptime(t,'%Y-%m-%dT%H:%M %z')
use the timedelta to account for the timezone, like this:
from datetime import datetime,timedelta
def dt_parse(t):
ret = datetime.strptime(t[0:16],'%Y-%m-%dT%H:%M')
if t[18]=='+':
ret+=timedelta(hours=int(t[19:22]),minutes=int(t[23:]))
elif t[18]=='-':
ret-=timedelta(hours=int(t[19:22]),minutes=int(t[23:]))
return ret

Failure to scale in pyclipper - TypeError: 'Zero' object is not iterable

I've just come up against a TypeError I've not seen before and can't figure out why it's occurring. Googling for the error TypeError: 'Zero' object is not iterable returns no results. I've tested in python 2.7 and 3.5 and the error is the same in both cases.
Here's the MCVE:
from sympy.geometry.polygon import Polygon
import pyclipper as pc
start_list = [(0, 2), (2, 2), (2, 0), (0, 0)]
scaled = pc.scale_to_clipper(start_list) # this works fine
as_poly = Polygon(*start_list)
new_list = [(pt.x, pt.y) for pt in as_poly.vertices]
assert new_list == start_list # check that the lists are the same (this passes)
fail_to_scale = pc.scale_to_clipper(new_list) # this fails
And the traceback:
Traceback (most recent call last):
File "C:\Users\Jamie\<blah>\mcve.py", line 10, in <module>
fails = pc.scale_to_clipper(new_list)
File "pyclipper/pyclipper.pyx", line 544, in pyclipper.scale_to_clipper (pyclipper/pyclipper.cpp:3535)
File "pyclipper/pyclipper.pyx", line 542, in pyclipper.scale_to_clipper.scale_value (pyclipper/pyclipper.cpp:3454)
File "pyclipper/pyclipper.pyx", line 542, in pyclipper.scale_to_clipper.scale_value (pyclipper/pyclipper.cpp:3454)
File "pyclipper/pyclipper.pyx", line 542, in pyclipper.scale_to_clipper.scale_value (pyclipper/pyclipper.cpp:3416)
TypeError: 'Zero' object is not iterable
Does anyone know what the source of and solution to this error could be?
This has been fixed in PyClipper version 1.0.2, which is preferable to using the workaround described below
Ok, I've tracked the problem back to the values stored in Polygon. The issue is that the value 0 is stored by SymPy as sympy.Zero.
These sympy.Zero values are not accepted by polyclipper.scale_to_clipper() and so are raising the TypeError when checked.
To avoid the issue I can generate new_list casting pt.x and pt.y to float:
new_list = [(float(pt.x), float(pt.y)) for pt in as_poly.vertices]

What is wrong with following piece of code?

I have the following piece of code copied from book programming collective intelligence page 118, chapter "Document Filtering". This function breaks up the text into words by dividing the text on any character that isn't a letter. This leaves only actual words,all converted to lower-case.
import re
import math
def getwords(doc):
splitter=re.compile('\\W*')
words=[s.lower() for s in splitter.split(doc)
if len(s)>2 and len(s)<20]
return dict([(w,1) for w in words])
I implemented the function and got the following error:
>>> import docclas
>>> t=docclass.getwords(s)
Traceback (most recent call last):
File "<pyshell#15>", line 1, in <module>
t=docclass.getwords(s)
File "docclass.py", line 6, in getwords
words=[s.lower() for s in splitter.split(doc)
NameError: global name 'splitter' is not defined
It works here
>>> import re
>>>
>>> def getwords(doc):
... splitter=re.compile('\\W*')
... words=[s.lower() for s in splitter.split(doc)
... if len(s)>2 and len(s)<20]
... return dict([(w,1) for w in words])
...
>>> getwords ("He's fallen in the water!");
{'water': 1, 'the': 1, 'fallen': 1}
I'm gueesing you made a typo in your code, but got it right when you pasted it here.