I am new to Python and coding in general. I am trying to build a scraper through Scrapy. I am trying to do this on Mac OS 10.12.6. I've followed the instructions as closely as possible (https://doc.scrapy.org/en/latest/intro/tutorial.html) and yet when I call
scrapy startproject tutorial I get the following output below. What's the best way to solve this and to understand future error messages as such?
Thanks!
File "/usr/local/bin/scrapy", line 9, in <module>
load_entry_point('Scrapy==1.4.0', 'console_scripts', 'scrapy')()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 565, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 2697, in load_entry_point
return ep.load()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 2370, in load
return self.resolve()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 2376, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/Library/Python/2.7/site-packages/Scrapy-1.4.0-py2.7.egg/scrapy/cmdline.py", line 9, in <module>
from scrapy.crawler import CrawlerProcess
File "/Library/Python/2.7/site-packages/Scrapy-1.4.0-py2.7.egg/scrapy/crawler.py", line 7, in <module>
from twisted.internet import reactor, defer
File "/Library/Python/2.7/site-packages/Twisted-17.9.0-py2.7-macosx-10.12-intel.egg/twisted/internet/reactor.py", line 38, in <module>
from twisted.internet import default
File "/Library/Python/2.7/site-packages/Twisted-17.9.0-py2.7-macosx-10.12-intel.egg/twisted/internet/default.py", line 56, in <module>
install = _getInstallFunction(platform)
File "/Library/Python/2.7/site-packages/Twisted-17.9.0-py2.7-macosx-10.12-intel.egg/twisted/internet/default.py", line 50, in _getInstallFunction
from twisted.internet.selectreactor import install
File "/Library/Python/2.7/site-packages/Twisted-17.9.0-py2.7-macosx-10.12-intel.egg/twisted/internet/selectreactor.py", line 18, in <module>
from twisted.internet import posixbase
File "/Library/Python/2.7/site-packages/Twisted-17.9.0-py2.7-macosx-10.12-intel.egg/twisted/internet/posixbase.py", line 18, in <module>
from twisted.internet import error, udp, tcp
File "/Library/Python/2.7/site-packages/Twisted-17.9.0-py2.7-macosx-10.12-intel.egg/twisted/internet/tcp.py", line 28, in <module>
from twisted.internet._newtls import (
File "/Library/Python/2.7/site-packages/Twisted-17.9.0-py2.7-macosx-10.12-intel.egg/twisted/internet/_newtls.py", line 21, in <module>
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
File "/Library/Python/2.7/site-packages/Twisted-17.9.0-py2.7-macosx-10.12-intel.egg/twisted/protocols/tls.py", line 63, in <module>
from twisted.internet._sslverify import _setAcceptableProtocols
File "/Library/Python/2.7/site-packages/Twisted-17.9.0-py2.7-macosx-10.12-intel.egg/twisted/internet/_sslverify.py", line 38, in <module>
TLSVersion.TLSv1_1: SSL.OP_NO_TLSv1_1,
AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'
I fixed this by updating to pyOpenSSL 0.14.
Shortly after though I had more problems with the local and user versions of python but sorted this by downloading Anaconda.
Related
I'm automating my EC2 instances with Patch Manager and I'm getting this error.
Error loading patching payloadfailed to run commands: exit status 156
The error output logs are these:
/usr/bin/python3
/usr/bin/apt-get
Reading package lists...
Building dependency tree...
Reading state information...
python3-apt is already the newest version (2.3.0ubuntu2).
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
Using python binary: 'python3'
Using Python Version: Python 3.10.4
/usr/bin/curl
/usr/bin/wget
05/29/2022 01:11:51 root [INFO]: Downloading payload from https://s3.dualstack.sa-east-1.amazonaws.com/aws-ssm-sa-east-1/patchbaselineoperations/linux/payloads/patch-baseline-operations-1.86.tar.gz
05/29/2022 01:11:52 root [INFO]: Attempting to import entrance file os_selector
05/29/2022 01:11:52 root [ERROR]: Error loading entrance module.
Traceback (most recent call last):
File "/var/log/amazon/ssm/patch-baseline-operations/common_startup_entrance.py", line 154, in execute
entrance_module = __import__(module_name)
File "/var/log/amazon/ssm/patch-baseline-operations/os_selector.py", line 11, in <module>
import common_os_selector_methods
File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 11, in <module>
from patch_common.baseline_override import load_baseline_override
File "/var/log/amazon/ssm/patch-baseline-operations/patch_common/baseline_override.py", line 6, in <module>
from patch_common.downloader import download_file, load_json_file, is_access_denied
File "/var/log/amazon/ssm/patch-baseline-operations/patch_common/downloader.py", line 1, in <module>
import boto3
File "/var/log/amazon/ssm/patch-baseline-operations/boto3/__init__.py", line 16, in <module>
from boto3.session import Session
File "/var/log/amazon/ssm/patch-baseline-operations/boto3/session.py", line 17, in <module>
import botocore.session
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/session.py", line 29, in <module>
import botocore.configloader
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/configloader.py", line 19, in <module>
from botocore.compat import six
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/compat.py", line 25, in <module>
from botocore.exceptions import MD5UnavailableError
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/exceptions.py", line 15, in <module>
from botocore.vendored import requests
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/__init__.py", line 58, in <module>
from . import utils
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/utils.py", line 26, in <module>
from .compat import parse_http_list as _parse_list_header
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/compat.py", line 7, in <module>
from .packages import chardet
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/__init__.py", line 3, in <module>
from . import urllib3
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/urllib3/__init__.py", line 10, in <module>
from .connectionpool import (
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 38, in <module>
from .response import HTTPResponse
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/urllib3/response.py", line 9, in <module>
from ._collections import HTTPHeaderDict
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/urllib3/_collections.py", line 1, in <module>
from collections import Mapping, MutableMapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)
05/29/2022 01:11:52 root [ERROR]: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)
Traceback (most recent call last):
File "/var/log/amazon/ssm/patch-baseline-operations/common_startup_entrance.py", line 154, in execute
entrance_module = __import__(module_name)
File "/var/log/amazon/ssm/patch-baseline-operations/os_selector.py", line 11, in <module>
import common_os_selector_methods
File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 11, in <module>
from patch_common.baseline_override import load_baseline_override
File "/var/log/amazon/ssm/patch-baseline-operations/patch_common/baseline_override.py", line 6, in <module>
from patch_common.downloader import download_file, load_json_file, is_access_denied
File "/var/log/amazon/ssm/patch-baseline-operations/patch_common/downloader.py", line 1, in <module>
import boto3
File "/var/log/amazon/ssm/patch-baseline-operations/boto3/__init__.py", line 16, in <module>
from boto3.session import Session
File "/var/log/amazon/ssm/patch-baseline-operations/boto3/session.py", line 17, in <module>
import botocore.session
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/session.py", line 29, in <module>
import botocore.configloader
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/configloader.py", line 19, in <module>
from botocore.compat import six
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/compat.py", line 25, in <module>
from botocore.exceptions import MD5UnavailableError
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/exceptions.py", line 15, in <module>
from botocore.vendored import requests
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/__init__.py", line 58, in <module>
from . import utils
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/utils.py", line 26, in <module>
from .compat import parse_http_list as _parse_list_header
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/compat.py", line 7, in <module>
from .packages import chardet
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/__init__.py", line 3, in <module>
from . import urllib3
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/urllib3/__init__.py", line 10, in <module>
from .connectionpool import (
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 38, in <module>
from .response import HTTPResponse
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/urllib3/response.py", line 9, in <module>
from ._collections import HTTPHeaderDict
File "/var/log/amazon/ssm/patch-baseline-operations/botocore/vendored/requests/packages/urllib3/_collections.py", line 1, in <module>
from collections import Mapping, MutableMapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)
Does anyone have an idea what it could be?
What's the Ubuntu version being used here?
Looking at the Python version (Python 3.10.4) in the error logs, that target instance appears to be running Ubuntu 22.04, which is not yet supported by Patch Manager.
Below is the link for the supported operating systems in Patch Manager prerequisites documentation:
https://docs.aws.amazon.com/systems-manager/latest/userguide/patch-manager-prerequisites.html
I tried using "from sklearn.decomposition import PCA" on windows python 2.7 to my program, but the result was an error and it said like this:
Traceback (most recent call last):
File "<pyshell#6>", line 1, in <module>
from sklearn.decomposition import PCA
File "C:\Python27\lib\site-packages\sklearn\decomposition\__init__.py", line 10, in <module>
from .kernel_pca import KernelPCA
File "C:\Python27\lib\site-packages\sklearn\decomposition\kernel_pca.py", line 14, in <module>
from ..preprocessing import KernelCenterer
File "C:\Python27\lib\site-packages\sklearn\preprocessing\__init__.py", line 8, in <module>
from .data import Binarizer
File "C:\Python27\lib\site-packages\sklearn\preprocessing\data.py", line 18, in <module>
from scipy import stats
File "C:\Python27\lib\site-packages\scipy\stats\__init__.py", line 343, in <module>
from .stats import *
File "C:\Python27\lib\site-packages\scipy\stats\stats.py", line 171, in <module>
from . import distributions
File "C:\Python27\lib\site-packages\scipy\stats\distributions.py", line 10, in <module>
from ._distn_infrastructure import (entropy, rv_discrete, rv_continuous,
File "C:\Python27\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 16, in <module>
from scipy.misc import doccer
File "C:\Python27\lib\site-packages\scipy\misc\__init__.py", line 97, in <module>
from .pilutil import *
File "C:\Python27\lib\site-packages\scipy\misc\pilutil.py", line 21, in <module>
import Image
File "C:\Python27\lib\site-packages\PIL\Image.py", line 27, in <module>
from . import VERSION, PILLOW_VERSION, _plugins
ValueError: Attempted relative import in non-package
what's wrong with that? yesterday back then, I used that "from sklearn.decomposition import PCA" to my program and there's no problem whit that. help me please
I have installed Anaconda2 on my windows 10 with python 2.7.15 version. Now, I need to install tensorflow and Keras, however, I am consistently getting the error while installing using "conda" or "pip". I googled it a lot about this error, but to no avail. Please help me out in finding the lead to resolving this issue. TIA, Priya Arora
The stack trace is as below:
(base) C:\Users\ARORAP1\anaconda2>conda install jupyter
Traceback (most recent call last):
File "C:\Users\ARORAP1\anaconda2\Scripts\conda-script.py", line 10, in <module
>
sys.exit(main())
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\cli\main.py", line 11
3, in main
return conda_exception_handler(_main, *args)
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\exceptions.py", line
1112, in conda_exception_handler
return_value = exception_handler(func, *args, **kwargs)
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\exceptions.py", line
822, in __call__
return self.handle_exception(exc_val, exc_tb)
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\exceptions.py", line
864, in handle_exception
return self.handle_unexpected_exception(exc_val, exc_tb)
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\exceptions.py", line
876, in handle_unexpected_exception
self.print_unexpected_error_report(error_report)
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\exceptions.py", line
947, in print_unexpected_error_report
from .cli.main_info import get_env_vars_str, get_main_info_str
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\cli\main_info.py", li
ne 25, in <module>
from ..core.subdir_data import SubdirData
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\core\subdir_data.py",
line 28, in <module>
from ..core.package_cache_data import PackageCacheData
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\core\package_cache_da
ta.py", line 13, in <module>
from .path_actions import CacheUrlAction, ExtractPackageAction
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\core\path_actions.py"
, line 25, in <module>
from ..gateways.connection.download import download
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\conda\gateways\connection\_
_init__.py", line 16, in <module>
from pip._vendor.requests import ConnectionError, HTTPError, Session
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\pip\_vendor\requests\__init
__.py", line 83, in <module>
from pip._internal.compat import WINDOWS
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\pip\_internal\__init__.py",
line 42, in <module>
from pip._internal import cmdoptions
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\pip\_internal\cmdoptions.py
", line 16, in <module>
from pip._internal.index import (
File "C:\Users\ARORAP1\anaconda2\lib\site-packages\pip\_internal\index.py", li
ne 15, in <module>
from pip._vendor import html5lib, requests, six
ImportError: cannot import name requests
(base) C:\Users\ARORAP1\anaconda2>
I have installed tensor flow from tensorflow website. While trying to import tensor flow its showing following error
>>>import tensorflow as tf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import *
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/__init__.py", line 75, in <module>
from tensorflow.core.framework.graph_pb2 import *
File "/usr/local/lib/python2.7/dist-packages/tensorflow/core/framework/graph_pb2.py", line 6, in <module>
from google.protobuf import descriptor as _descriptor
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/__init__.py", line 37, in <module>
__import__('pkg_resources').declare_namespace(__name__)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 36, in <module>
import email.parser
File "/usr/lib/python2.7/email/parser.py", line 12, in <module>
from email.feedparser import FeedParser
File "/usr/lib/python2.7/email/feedparser.py", line 27, in <module>
from email import message
File "/usr/lib/python2.7/email/message.py", line 16, in <module>
import email.charset
File "/usr/lib/python2.7/email/charset.py", line 13, in <module>
import email.base64mime
File "/usr/lib/python2.7/email/base64mime.py", line 40, in <module>
from email.utils import fix_eols
File "/usr/lib/python2.7/email/utils.py", line 28, in <module>
import socket
File "/home/tamarind/socket.py", line 5, in <module>
This module provides socket operations and some related functions.
TypeError: 'module' object is not callable
I think its problem with socket please suggest. Please suggest.
It looks like /home/tamarind/socket.py is taking precedence over the python socket module. Just rename/remove that (or import TensorFlow from a different directory) and it should work. You may need to remove socket.pyc too.
I am encountering the below stacktrace when running import paramiko (this is in windows platform)
File "/usr/lib/python2.7/site-packages/paramiko/__init__.py", line 30, in <module>
from paramiko.transport import SecurityOptions, Transport
File "/usr/lib/python2.7/site-packages/paramiko/transport.py", line 50, in <module>
from paramiko.dsskey import DSSKey
File "/usr/lib/python2.7/site-packages/paramiko/dsskey.py", line 26, in <module>
from Crypto.PublicKey import DSA
File "/usr/lib/python2.7/site-packages/Crypto/PublicKey/DSA.py", line 89, in <module>
from Crypto import Random
File "/usr/lib/python2.7/site-packages/Crypto/Random/__init__.py", line 29, in <module>
from Crypto.Random import _UserFriendlyRNG
File "/usr/lib/python2.7/site-packages/Crypto/Random/_UserFriendlyRNG.py", line 38, in <module>
from Crypto.Random.Fortuna import FortunaAccumulator
File "/usr/lib/python2.7/site-packages/Crypto/Random/Fortuna/FortunaAccumulator.py", line 39, in <module>
import FortunaGenerator
File "/usr/lib/python2.7/site-packages/Crypto/Random/Fortuna/FortunaGenerator.py", line 35, in <module>
from Crypto.Util import Counter
File "/usr/lib/python2.7/site-packages/Crypto/Util/Counter.py", line 59, in <module>
from Crypto.Util import _counter
ImportError: cannot import name _counter
I see that _counter.pyd exists in Pycrypto folder. Why is this not getting picked?