Need a Suggestion on the below. I was trying to do a DMS to aurora and i have installed python3.8.8. I have installed all required modules. Openssl is also installed.
When i trigger the script i encounter the below error.
Traceback (most recent call last): File "task_runner.py", line 2, in
import boto3 File "/usr/local/lib/python3.8/site-packages/boto3/init.py", line 16,
in
from boto3.session import Session File "/usr/local/lib/python3.8/site-packages/boto3/session.py", line 17, in
import botocore.session File "/usr/local/lib/python3.8/site-packages/botocore/session.py", line 30,
in
import botocore.credentials File "/usr/local/lib/python3.8/site-packages/botocore/credentials.py", line
34, in
from botocore.config import Config File "/usr/local/lib/python3.8/site-packages/botocore/config.py", line 16,
in
from botocore.endpoint import DEFAULT_TIMEOUT, MAX_POOL_CONNECTIONS File
"/usr/local/lib/python3.8/site-packages/botocore/endpoint.py", line
22, in
from botocore.awsrequest import create_request_object File "/usr/local/lib/python3.8/site-packages/botocore/awsrequest.py", line
26, in
import botocore.utils File "/usr/local/lib/python3.8/site-packages/botocore/utils.py", line 33,
in
import botocore.httpsession File "/usr/local/lib/python3.8/site-packages/botocore/httpsession.py", line
8, in
from urllib3.util.ssl_ import ( ImportError: cannot import name 'ssl' from 'urllib3.util.ssl_'
(/usr/local/lib/python3.8/site-packages/urllib3/util/ssl_.py)
I have tried changing the version of python as suggested in another post and also reinstalled awscli but nothing works. no matter what version of python used, i always end up with the same error.
Lastly, the server where am doing this doesnt have an internet connection.
Kindly suggest.
Cleaning up the entire Python setup and editing the Setup.dist to use SSL during build resolved the issue.
Related
I am currently starting the process of preparing an App Engine app for Python 3 migration.
During the first step:
Migrate the App Engine bundled services in your Python 2 app to Google Cloud services ...
Following all the instructions to switch the datastore module from google.appengine.ext.ndb to google.cloud.ndb, I immediately get the following Import Error:
File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 311, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/home/---.py", line 8, in <module>
from google.cloud import ndb
File "/home/test_env/local/lib/python2.7/site-packages/google/cloud/ndb/__init__.py", line 28, in <module>
from google.cloud.ndb.client import Client
File "/home/test_env/local/lib/python2.7/site-packages/google/cloud/ndb/client.py", line 23, in <module>
from google.cloud import _helpers
File "/home/test_env/local/lib/python2.7/site-packages/google/cloud/_helpers.py", line 29, in <module>
from six.moves import http_client
ImportError: No module named moves
This happens whether or not I am testing in a virtual environment. Importing six.moves works in a python console.
Apparently this is an issue with the bundled test server dev_appserver.py. Found it solved here by Andrewsg:
I think we've identified an issue with devappserver related to the six library specifically. Could you please try a workaround? Add the line: import six; reload(six) to the top of your app, before NDB is loaded
I am trying to implement firebase functions cron job from this link : https://github.com/firebase/functions-cron
Everything worked properly
But when I try to run google cloud cron job it gives me below error :
(/base/alloc/tmpfs/dynamic_runtimes/python27/c5586dbb532f7e5f_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py:263)
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27/c5586dbb532f7e5f_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/alloc/tmpfs/dynamic_runtimes/python27/c5586dbb532f7e5f_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/c5586dbb532f7e5f_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/s~debitcredit-7ecc0/20180506t121449.409523654918066893/main.py", line 18, in <module>
import pubsub_utils
File "/base/data/home/apps/s~debitcredit-7ecc0/20180506t121449.409523654918066893/pubsub_utils.py", line 24, in <module>
import oauth2client.contrib.appengine as gae_oauth2client
File "./lib/oauth2client/contrib/appengine.py", line 36, in <module>
from oauth2client import client
File "./lib/oauth2client/client.py", line 39, in <module>
from oauth2client import transport
File "./lib/oauth2client/transport.py", line 255, in <module>
redirections=httplib2.DEFAULT_MAX_REDIRECTS,
AttributeError: 'module' object has no attribute 'DEFAULT_MAX_REDIRECTS'
I tried this solution : Getting AttributeError: 'module' object has no attribute 'DEFAULT_MAX_REDIRECTS' when running Google Sheets API quickstart
But still no luck.
Can anyone please help me with this.
Issue is with your httplib Module.
When you installed this module for your project you must have installed it with pip for Python 3
If you want to check whether this module is for python 3 or python 2,
Go to httplib2 module and go inside its init.py
later see this line Requires Python 3 or later
if it is written like that means you have installed this library with pip for python 3. Now delete all the httplib2 from your lib folder.
Create a seperate enviroment for python 2.7 and again install all your modules with pip install -t lib -r requirements.txt
I had followed the following instructions for setting up google-cloud-ml on LOCAL: MAC/LINUX
google-cloud-ml setup
But I am getting the following errors while verifying the setup with this command
curl https://raw.githubusercontent.com/GoogleCloudPlatform/cloudml-samples/master/tools/check_environment.py | python
Traceback (most recent call last):
File "", line 70, in
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/init.py", line 16, in
from google.cloud.ml.dataflow._analyzer import AnalyzeModel
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/dataflow/init.py", line 22, in
from _ml_transforms import DeployVersion
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/dataflow/_ml_transforms.py", line 24, in
import _ml_functions as ml_func
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/dataflow/_ml_functions.py", line 25, in
from google.cloud.ml.io.coders import TrainingJobResult
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/io/init.py", line 21, in
from transforms import LoadFeatures
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/io/transforms.py", line 23, in
from google.cloud.ml.dataflow.io import tfrecordio
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/dataflow/io/init.py", line 15, in
import tfrecordio
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/dataflow/io/tfrecordio.py", line 16, in
_crc32c_fn = snappy._crc32c # pylint: disable=protected-access
AttributeError: 'module' object has no attribute '_crc32c'
Looks like there is some issue with dataflow library of google cloud.
I tried upgrading the dataflow library with the following command
pip install --upgrade google-api-python-client
but now, another error. Following is the stacktrace:
Traceback (most recent call last):
File "", line 70, in
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/init.py", line 16, in
from google.cloud.ml.dataflow._analyzer import AnalyzeModel
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/dataflow/init.py", line 17, in
from _analyzer import AnalyzeModel
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/google/cloud/ml/dataflow/_analyzer.py", line 19, in
import apache_beam as beam
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apache_beam/init.py", line 78, in
from apache_beam import io
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apache_beam/io/init.py", line 21, in
from apache_beam.io.avroio import *
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apache_beam/io/avroio.py", line 29, in
from apache_beam.io import filebasedsource
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apache_beam/io/filebasedsource.py", line 32, in
from apache_beam.io import concat_source
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apache_beam/io/concat_source.py", line 24, in
from apache_beam.io import iobase
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apache_beam/io/iobase.py", line 853, in
from apache_beam.runners.dataflow.native_io.iobase import *
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apache_beam/runners/init.py", line 23, in
from apache_beam.runners.dataflow_runner import DataflowRunner
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apache_beam/runners/dataflow_runner.py", line 32, in
from apache_beam.internal import json_value
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apache_beam/internal/json_value.py", line 20, in
from apitools.base.py import extra_types
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apitools/base/py/init.py", line 23, in
from apitools.base.py.credentials_lib import *
File "/Users/pratyusha/miniconda2/envs/cloudml/lib/python2.7/site-packages/apitools/base/py/credentials_lib.py", line 50, in
from oauth2client import locked_file
This looks like a different version of 'snappy' is expected. Can you check which version of python-snappy you have installed?
You can do this by running:
pip freeze | grep python-snappy
It worked for me with version 0.5, which is the latest version. If you have an earlier version, try running:
pip install --upgrade python-snappy
(using 'sudo' or '--user' as necessary.)
Actually I am uploading python app (which using pusher module) on GAE. Its working fine on my local system. But in GAE(Google App Engine) Its giving following error when I host this app to GAE.
Traceback (most recent call last): (/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/cgi.py:122)
File "/base/data/home/apps/s~pushapi-1117/1.388696378960329626/main.py", line
26, in <module>
secret=secret
File "lib/pusher/pusher.py", line 42, in __init__ from pusher.requests
import RequestsBackend
File "lib/pusher/requests.py", line 12, in <module>
import urllib3.contrib.pyopenssl
File "lib/urllib3/contrib/pyopenssl.py", line 53, in <module>
import OpenSSL.SSL
File "lib/OpenSSL/__init__.py", line 8, in <module>
from OpenSSL import rand, crypto, SSL
File "lib/OpenSSL/rand.py", line 11, in <module>
from OpenSSL._util import (
File "lib/OpenSSL/_util.py", line 6, in <module>
from cryptography.hazmat.bindings.openssl.binding import Binding
File "lib/cryptography/hazmat/bindings/openssl/binding.py", line 13, in
<module>
from cryptography.hazmat.bindings._openssl import ffi, lib
**ImportError: No module named _openssl**
I have installed all the dependent modules cryptography, OpenSSL also then also I am getting this problem. This error is coming only when I am hosting my app on GAE.
GAE doesn't support C extensions and pyca/cryptography (which pyOpenSSL depends on) imports C extensions (via cffi). Google has more information about the requirements for Python libraries on GAE.
I use the Python distribution. Python 2.7 x64 with Windows 7 SP1 x64 Ultimate.
After a conda update --all, whenever I try to import matplotlib.pyplot I get ImportError: DLL load failed: %1 is not a valid Win32 application.. Why?
Full error stack:
Traceback (most recent call last):
File "C:\svn\hw4\code\test_con.py", line 1, in <module>
import matplotlib.pyplot as plt
File "C:\Anaconda\lib\site-packages\matplotlib\pyplot.py", line 109, in <module>
_backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
File "C:\Anaconda\lib\site-packages\matplotlib\backends\__init__.py", line 32, in pylab_setup
globals(),locals(),[backend_name],0)
File "C:\Anaconda\lib\site-packages\matplotlib\backends\backend_qt4agg.py", line 17, in <module>
from .backend_qt5agg import NavigationToolbar2QTAgg
File "C:\Anaconda\lib\site-packages\matplotlib\backends\backend_qt5agg.py", line 18, in <module>
from .backend_qt5 import QtCore
File "C:\Anaconda\lib\site-packages\matplotlib\backends\backend_qt5.py", line 31, in <module>
from .qt_compat import QtCore, QtGui, QtWidgets, _getSaveFileName, __version__
File "C:\Anaconda\lib\site-packages\matplotlib\backends\qt_compat.py", line 91, in <module>
from PyQt4 import QtCore, QtGui
ImportError: DLL load failed: %1 is not a valid Win32 application.
I had the same issue after running conda update anaconda. The solution that worked for me was to simply to download the latest windows installer, then uninstall and reinstall.
I suspect this is related to the pathname of your conda environment, as if you do a clean install anaconda now wants to live in C:\Users\yourname\AppData\Local\Continuum\Anaconda2, where previously it was just Anaconda (no 2). I suspect after running the upgrade scripts some things are pointing to the nonexistant (for you) "new" path.
I found that if I explicitly activate a conda environment with activate myenvname before running anything it works fine. Likewise if you you run from the "anaconda prompt" (which activates your default environment for you) it works fine. But to get my default environment to stay active from anywhere like its supposed to I had to reinstall.
Here is what worked for me:
Run conda update -f matplotlib (from the Anaconda command prompt)
This gave me a new error:
RuntimeError: module compiled against API version a but this version of numpy is 9
Traceback (most recent call last):
File "C:\svn\hw4\code\test_con.py", line 1, in <module>
import matplotlib.pyplot
File "C:\Anaconda\lib\site-packages\matplotlib\pyplot.py", line 27, in <module>
import matplotlib.colorbar
File "C:\Anaconda\lib\site-packages\matplotlib\colorbar.py", line 32, in <module>
import matplotlib.artist as martist
File "C:\Anaconda\lib\site-packages\matplotlib\artist.py", line 12, in <module>
from .transforms import Bbox, IdentityTransform, TransformedBbox, \
File "C:\Anaconda\lib\site-packages\matplotlib\transforms.py", line 39, in <module>
from matplotlib._path import (affine_transform, count_bboxes_overlapping_bbox,
ImportError: numpy.core.multiarray failed to import
Run conda update -f numpy
This gave me a new error:
Traceback (most recent call last):
File "C:\svn\hw4\code\test_con.py", line 1, in <module>
import matplotlib.pyplot
File "C:\Anaconda\lib\site-packages\matplotlib\pyplot.py", line 27, in <module>
import matplotlib.colorbar
File "C:\Anaconda\lib\site-packages\matplotlib\colorbar.py", line 34, in <module>
import matplotlib.collections as collections
File "C:\Anaconda\lib\site-packages\matplotlib\collections.py", line 27, in <module>
import matplotlib.backend_bases as backend_bases
File "C:\Anaconda\lib\site-packages\matplotlib\backend_bases.py", line 56, in <module>
import matplotlib.textpath as textpath
File "C:\Anaconda\lib\site-packages\matplotlib\textpath.py", line 22, in <module>
from matplotlib.mathtext import MathTextParser
File "C:\Anaconda\lib\site-packages\matplotlib\mathtext.py", line 63, in <module>
import matplotlib._png as _png
ImportError: DLL load failed: The specified module could not be found.
Run conda install anaconda. This rolled your system back to a stable anaconda distribtution, and solved the issue. (I guess you could skip steps 1 and 2, but I don't feel like going through it again to confirm…)