AWS Lambda using PIL/Pillow with Python 2.7 - python-2.7

I'm using Python 2.7 (for various reasons) on AWS Lambda, and I need to use the PIL/Pillow library for image processing. I'm using a ZIP file to get code into my Lambda function.
One of the items in this zip is a folder with the PIL library in it. I've tried several different methods of getting the library in there, including using pip install pillow -t ., and using pre-compiled PIL libraries from here and here.
Using the first pre-compiled source, I get the error:
no module named PIL
If I rename the subdirectory from PIL to pillow, I get the same error but with pillow.
Using the second pre-compiled source and the pip method, I get the error:
Could not import the Python Imaging Library (PIL) required to load image files
Here's my import statement and the line that the error is occurring on:
from pillow import *
...
I = imread(filename, flatten=True)
Here's a screenshot of my file structure (using the 1st pre-compiled source):
Archive.zip is what I've been uploading to Lambda. Each time I make a code or library change, I recreate this by selecting each of the files I would like to zip, right-clicking and selecting "Compress". I'm using macOS High Sierra.
Btw, I've seen this question and answer, but I'm not getting the same error message and I couldn't get the suggested commands to work anyway.
I'm wondering if it could be an issue with PIL vs Pillow or with Python 3.6 vs Python 2.7.
Any help is much appreciated!

I solved this by using a Docker solution, as outlined in this post.

Related

Python : Can I separately use maintain and install ANN libraries and Image processing libraries in same project on windows10

I want to use Opencv library for image processing which supports for python 2.7 and tensorflow library for ANN which supports for python 3.5 in the same project.But when I try to include opencv library after including tensor flow library in to pycharm,it shows an error.Is there any way to handle this problem?I have installed both python2.7 and python3.5 in my computer using anaconda.
Yes, you can use both. Any errors you get are due to a bad installation of either Tensorflow of OpenCV. Best would be to find out what the exact problem is and post a new question to describe that.
Also: it would be a good idea to use Docker to get an image with OpenCV and tensorflow already installed. This is one I like: https://github.com/pkmital/CADL/tree/master/#docker-installation. It really removes the dependency and installation problems!

How to install GDAL/scipy using cmd in window?

I downloaded scipy-0.17.0-cp27-none-win_amd64.whl and GDAL-1.11.4-cp27-none-win_amd64.whl from gohlke in C:\Python27\Scripts
To install I used
pip install scipy-0.17.0-cp27-none-win_amd64.whl
pip install GDAL-1.11.4-cp27-none-win_amd64.whl
It says the installation is complete but when I import the libraries as
import scipy
import gdal
it shows error as
No module named gdal
No module named gdal
However, I installed the matplotlib, numpy in the very same way and they are working absolutely fine.
I solved this problem eventually to found the mistake in my approach.
This problem can occur with anyone using ArcGIS in one's system.
ArcGIS comes with it's default Python package and if one installs python separately, each time the new libraries gets installed in the newer Python installation not in ArcGIS.
Therefore, the pyhton IDLE which one uses need to be from another Python installation.
Here in my case, ArcGIS has Python 2.6 and I have made an separate installation using Python 2.7.11.
All the libraries were getting installed in right place but I was opening the wrong IDLE to write scripts hence getting error.

pyinstaller single exe of program which uses google api client lib

I have a python program which I've successfully packaged up as a single exe using pyinstaller in the past. Recently I added new features which make use of the google api python client ( https://developers.google.com/api-client-library/python/ ). I've attempted to make a new single exe package of the new version and it fails to run.
I enable debugging and the console and initially the issue was that it hadn't picked up the oauth lib. I fixed that by adding the following to my spec file:
hiddenimports=['googleapiclient', 'apiclient']
When I build I can see this:
53092 INFO: Hidden import 'googleapiclient' has been found otherwise
53093 INFO: Hidden import 'apiclient' has been found otherwise
However, now when I run the rebuilt exe I get the following error before it exits:
pkg_resources.DistributionNotFound: google-api-python-client
I can't see any reference to that and I'm not sure how to force it to be packaged up with the exe.
I figure I can't be the only person to have ever wanted to package up a python program that makes use of the google api, but I failed to find any help during a lot of time with my friend google...
Any tips?
Many People Have same question but I can't find any answer
Try This way
find site package of your project
enter the Pyinstaller/hooks and find hokk-google.api_core.py
add line ( edit line ) datas += copy_metadata('google-api-python-client')
example
from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata('google-api-core')
datas += copy_metadata('google-api-python-client')
excute pyinstaller hidden-import google-api-python-client
pyinstaller --hidden-import google-api-python-client
you can avoid error
pkg_resources.DistributionNotFound: google-api-python-client
I was able to dig deeper into this issue after I switched from using pip to easy_install to install the google-api-python-client package. Manually installing the egg file gave me a much neater way of testing.
I then started building using pyinstaller with out the --onefile option, this way I was able to start messing with the egg file that was packaged up with my program. Eventually I found out that my problem was stemming from the fact I'm making use of Pandas and the Google API.
Here's the relevant Pandas bug on github.
To solve the problem I had to update my version of Pandas.
I did find that if I moved to the most recent version of Pandas (1.6.0) I ran into a new issue because it imports dateutil and it causes problems looking for the zoneinfo file. I've landed on Pandas 1.5.1 and it is all working.
You can also add site-packages using:
--add-data "path_to_your_package:package_name(e.g googleapiclient)"

Aptana Python stdlib issue with virtualenv

I recently started working on a project using just vim as my text editor with a virtualenv setup. I installed a few API's on this virtualenv from GitHub. Eventually, the project got a little bigger than vim could handle so I had to move the project to an IDE.
I chose Aptana Studio 3. When I started up Aptana, I pointed the project directory to the virtualenv folder that I had created to house my project. I then pointed the interpreter at the Python executable in App/bin (created from virtualenv)/python2.7. When I started reworking the code to make sure I had everything mapped correctly, I was able to import the API's that I had installed just fine. CherryPy came through with no problems, but I've been having an issue with importing a module that I believe is part of the stdlib--urlparse. At first, I thought it was that my python interpreter was 2.7.1 rather than 2.7.5 (I found the documentation in the 2.7.5 section with no option to review 2.7.1), but my terminal is using 2.7.1 and is able to import the module without any errors (I'm using OSX, Mountain Lion). I am also able to import the module when I activate the virtualenv and run my python interpreter. But when I plug "from urlparse import parse_qsl" into Aptana, I'm getting an error: "Unresolved_import: parse_qsl".
Should I have pointed this at a different interpreter and, if so, will I need to reinstall the API modules I had been working with in the new interpreter?
Update: I finally ended up restarting the project. It turns out that not all of the standard Python tools are selected when you select the virtualenv interpreter. After I selected all of the python tools from the list (just after choosing the interpreter), I was able to get access to the entire standard library.
Do NOT just import the modules into your project. Many of the stdlib modules are interdependent and the import function will only import a module into your main project directory, not a libary!

ImportError when importing certain modules from SciPY

I have used Scipy for some time. This is the first time I am using it for Signal processing!
But when I import modules like
from scipy import signal
from scipy import special
I get the error:
ImportError: DLL load failed: The specified module could not be found.
I am using Python 2.7.3 with Scipy 0.12.0 on 32-Bit Windows.
What should I do ?
This problem can be solved if instead of installing the usual numpy distribution, the numpy-MKL package is installed.
This package is available here.
Do remove the previous installation before going with the new one!
I already had numpy+mkl installed, but still I faced similar error.
Reinstalling has solved the issue:
pip uninstall numpy-1.13.1+mkl-cp35-cp35m-win_amd64.whl
pip install numpy-1.13.1+mkl-cp35-cp35m-win_amd64.whl
I installed numpy-MKL from here for Python 3.5.1, but it didn't solve the problem until I added the folder C:\Program Files\Python35\Lib\site-packages\numpy\core to system path.
Similar to the OP, I already had the Intel MKL libraries installed on my system. I was unable to load scipy.linalg with the same error message. I uninstalled the old version of numpy and scipy (which I installed before installing the Intel compilers and math libraries). Then ran pip install scipy, and magically I could now import scipy.linalg without the error.
I'm not entirely sure what caused it, and why it was unable to find the library it needed. But it somehow fixed the problem for me on Python 3.7.1 with Anaconda.
I had an issue importing sklearn because of my Scipy installation.
I fixed this by going to here and downloading the right version of numpy for my computer. Then I did the same for Scipy by going here and downloading the MKL version for my computer.
Once I did that, everything worked!
To check the supported tags for wheel version for your system you can run the following command in the command prompt: pip debug --verbose.
You can install the .whl files for numpy and scipy by doing: pip install {filename}.whl
I had this issue on 3.6 and reinstalling didn't work,downloading the wheel didn't work. I found a solution that did work:
go to "site-packages/scipy" folder and open __init__.py file for editting.
At the very bottom add this line of code:
from . import signal
from . import special
from . import linalg
from . import <insert missing submodule here>
this is the only solution that has worked for me and it should work for any one