using pyshark on python 2.7 encounter lxml.etree.XMLSyntaxError - python-2.7

This is really a simple script written in python, which I can run it normally on Linux. But when I moved it to Windows, there is a strange error. I wish some helps.
Before running the code, I have made some preparation for the environment:
1. Install Microsoft Visual C++ Compiler for python 2.7
2. Install python 2.7.11
3. pip install pyinstaller
4. easy_install pyshark
Below is part of my code.
# -*- coding: utf-8 -*-
from __future__ import print_function
import pyshark
import lxml
import os
def analysis_method(file_name):
cap = pyshark.FileCapture(input_file=file_name)
for packet in cap:
if hasattr(packet, "http"):
http_layer = packet["http"]
Below is the error information:
Traceback (most recent call last):
File "packet_offline_analysis.py", line 36, in analysis_method
for packet in cap:
File "C:\Python27\lib\site-packages\pyshark-0.3.6.1-py2.7.egg\pyshark\capture\capture.py", line 173, in _packets_from_tshark_sync
self._get_packet_from_stream(tshark_process.stdout, data, psml_structure=psml_structure))
File "C:\Python27\lib\site-packages\trollius-1.0.4-py2.7-win32.egg\trollius\base_events.py", line 300, in run_until_complete
return future.result()
File "C:\Python27\lib\site-packages\trollius-1.0.4-py2.7-win32.egg\trollius\futures.py", line 287, in result
raise self._exception
lxml.etree.XMLSyntaxError: Input is not proper UTF-8, indicate encoding !
Bytes: 0xD6 0xD0 0xB9 0xFA, line 6, column 58

Uninstall your Chinese version of Wireshark, and install a english version of Wireshark instead.
Then the problem is solved.

Related

Python 2.7 Import libsoundtouch

I want install libsoundtouch ("pip install libsoundtouch"). after install process, everything seems correct, but when I run my script python(2.7) can import this module.
After check I have files in "/usr/local/lib/python2.7/dist-packages/libsoundtouch".
#!/usr/bin/python
# -*- coding: utf-8 -*-
import libsoundtouch
./home/pi/myprog/scripts/python/test.py
Traceback (most recent call last):
File "./home/pi/myprog/scripts/python/test.py", line 5, in
import libsoundtouch
File "/usr/local/lib/python2.7/dist-packages/libsoundtouch/init.py", line 9, in
from zeroconf import Zeroconf, ServiceBrowser
File "/usr/local/lib/python2.7/dist-packages/zeroconf.py", line 189
def current_time_millis() -> float:
^
SyntaxError: invalid syntax

Python 2.7 import unicode_literals from __future__ gives UnicodeDecodeError while reading the file with umauts

I have a Python script which read and write a file with german umlauts (äöü) in an input file "myfile.in". I used Python version 2.7. Here a reduced version of my script:
#!/usr/bin/env python
# -*- encoding: utf-8 -*-
if __name__=='__main__':
with open("myfile.in", "r") as f:
lines = f.readlines()
txt = ""
for line in lines:
txt = txt + line
with open("myfile.out", "w") as f:
f.write(txt)
This works fine.
Now I got the requirement from my customer to used the Future statement definitions and I added the following line to my Python script:
from __future__ import unicode_literals
Now I get the following error message:
Traceback (most recent call last):
File "myscript.py", line 9, in <module>
txt = txt + line
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 23: ordinal not in range(128)
How can I resolve this problem.
Thanks for your hints Thomas

Jython 2.7.1 + ftfy 4.4

What can be wrong with this import?
I downloaded version 4.4 for Jython 2.7
import ftfy
import sys
print (ftfy.fix_encoding("н368вв777"))
Traceback (most recent call last):
File "D:/rs_al/IdeaProjects/XLStoSQL/src/main/java/BrokenUTF8.py", line 4,
in <module>
import ftfy
File "C:\jython\Lib\site-packages\ftfy\__init__.py", line 12, in <module>
from ftfy import fixes
UnicodeDecodeError: 'unicodeescape' codec can't decode bytes in position 2-8:
illegal Unicode character
File "C:\jython\Lib\site-packages\ftfy\__init__.py", line 12, in <module>
from ftfy import fixes
UnicodeDecodeError: 'unicodeescape' codec can't decode bytes in position 2-8: illegal Unicode character
With Python3 + ftfy 5 everything works, but I thought about using java + jython to convert wrong UTF8 characters with ftfy package and return data back to java.
Also, I set default decoding of source to UTF-8, because when I use jython 2.7 default decoding of sources is ascii.
At full power ftfy works only with Python 3. Moved project to Python. Solved

Theano ImportError: cannot import name inplace_increment

I've installed Theano as such:
(python2) bash-3.2$ sudo pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git
I'm using conda with Python 2.7. The python2 in parentheses is my conda environment. I'm trying to run the code from iGAN: https://github.com/junyanz/iGAN. I'm on OS X 10.11.6.
The error I get is ImportError: cannot import name inplace_increment. I've tried manually removing the Theano cache (as described in https://github.com/Theano/Theano/issues/5564) and the same error still arises. (I can't use theano-cache purge because it requires installing a version of scipy that clashes with the version required by OpenCV.) I've also tried uninstalling and reinstalling Theano via pip and still get the same error. Any suggestions?
Here's the context:
(python2) bash-3.2$ python iGAN_main.py
//anaconda/envs/python2/lib/python2.7/site-packages/numpy/lib/utils.py:254: FutureWarning: Numpy has detected that you (may be) writing to an array returned
by numpy.diagonal or by selecting multiple fields in a record
array. This code will likely break in the next numpy release --
see numpy.diagonal or arrays.indexing reference docs for details.
The quick fix is to make an explicit copy (e.g., do
arr.diagonal().copy() or arr[['f0','f1']].copy()).
ai = a.__array_interface__
Traceback (most recent call last):
File "iGAN_main.py", line 9, in <module>
import constrained_opt
File "/Users/.../Desktop/Code/iGAN/constrained_opt.py", line 3, in <module>
from lib.rng import np_rng
File "/Users/.../Desktop/Code/iGAN/lib/rng.py", line 2, in <module>
from theano.sandbox.rng_mrg import MRG_RandomStreams as RandomStreams
File "//anaconda/envs/python2/lib/python2.7/site-packages/theano/__init__.py", line 102, in <module>
from theano.scan_module import (scan, map, reduce, foldl, foldr, clone,
File "//anaconda/envs/python2/lib/python2.7/site-packages/theano/scan_module/__init__.py", line 41, in <module>
from theano.scan_module import scan_opt
File "//anaconda/envs/python2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 60, in <module>
from theano import tensor, scalar
File "//anaconda/envs/python2/lib/python2.7/site-packages/theano/tensor/__init__.py", line 9, in <module>
from theano.tensor.subtensor import *
File "//anaconda/envs/python2/lib/python2.7/site-packages/theano/tensor/subtensor.py", line 27, in <module>
from cutils_ext.cutils_ext import inplace_increment
ImportError: cannot import name inplace_increment
I had the same problem.
Deleting the cache manually worked.
rm -rf ~/.theano
If that doesn't help try installing from source.
python setup.py install

Failing to import Shapely / GEOS in Python on OSX

I am running a brewed Python 2.7.11 on OS X El Capitan 10.11.3. I fail to import Shapely / GEOS:
Failed `CDLL(/usr/local/lib/libgeos_c.dylib)`
Failed `CDLL(/Library/Frameworks/GEOS.framework/Versions/Current/GEOS)`
Failed `CDLL(/opt/local/lib/libgeos_c.dylib)`
Traceback (most recent call last):
File "example.py", line 2, in <module>
from shapely.geometry import Point
File "/Library/Frameworks/GEOS.framework/Versions/3/Python/2.7/site-packages/shapely/geometry/__init__.py", line 4, in <module>
from .base import CAP_STYLE, JOIN_STYLE
File "/Library/Frameworks/GEOS.framework/Versions/3/Python/2.7/site-packages/shapely/geometry/base.py", line 9, in <module>
from shapely.coords import CoordinateSequence
File "/Library/Frameworks/GEOS.framework/Versions/3/Python/2.7/site-packages/shapely/coords.py", line 8, in <module>
from shapely.geos import lgeos
File "/Library/Frameworks/GEOS.framework/Versions/3/Python/2.7/site-packages/shapely/geos.py", line 92, in <module>
mode=(DEFAULT_MODE | 16))
File "/Library/Frameworks/GEOS.framework/Versions/3/Python/2.7/site-packages/shapely/geos.py", line 61, in load_dll
libname, fallbacks or []))
OSError: Could not find lib geos_c or load any of its variants ['/Library/Frameworks/GEOS.framework/Versions/Current/GEOS', '/opt/local/lib/libgeos_c.dylib'].
This error is strikingly similar to those reported here:
Python can't import shapely and
OSError geos_c could not be found when Installing Shapely. After implementing all suggestions that "did the trick" for others, I continue to get this error.
I installed geos with brew install geos today. It installed in /usr/local/Cellar/geos/3.5.0.
If I run find / -name "libgeos_c.dylib", I find three versions:
/Library/Frameworks/GEOS.framework/Versions/3/unix/lib/libgeos_c.dylib
/usr/local/Cellar/geos/3.5.0/lib/libgeos_c.dylib
/usr/local/lib/libgeos_c.dylib
Note that the last line is exactly the same path as in the first line of the error. So Python fails to import an existing file? I know it is an alias (to the file in Cellar), but would be surprised if that's the reason.
Also, $DYLD_LIBRARY_PATH and $GEOS_LIBRARY_PATH are empty at startup, but setting them to any of the following does not resolve the problem:
export DYLD_LIBRARY_PATH=/usr/local/lib
export GEOS_LIBRARY_PATH=/usr/local/Cellar/geos/3.5.0
export GEOS_LIBRARY_PATH=/usr/local/Cellar/geos/3.5.0/lib
Any insight will be tremendously welcome! This has taken me more than half of the day already...