Can't install Scrapy for Python 2.7 - python-2.7

I'm trying to install Scrapy for Python 2.7 according to Installation guide - Scrapy 0.22.2 documentation. I've installed pip, but when I try to run the following from the Windows 8 command prompt:
pip install Scrapy
it doesn't appear to install properly, here is the end of the pip.log
C:\Python27\lib\disutils\dist.py:267: UserWarning: Unknown distribution option: 'butrack_url'
warnings.warn(msg)
error: Unable to find vcvarsall.bat
----------------------------------------
Cleaning up...
Removing temporary dir c:\users\mgarrett\appdata\local\temp\pip_build_mgarrett...
Command C:\Python27\python.exe -c "import setuptools, tokenize;__file__='c:\\users\\mgarrett\\appdata\\local\\temp\\pip_build_mgarrett\\lxml\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record c:\users\mgarrett\appdata\local\temp\pip-7wgh3w-record\install-record.txt --single-version-externally-managed --compile failed with error code 1 in c:\users\mgarrett\appdata\local\temp\pip_build_mgarrett\lxml
Exception information:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\pip\basecommand.py", line 122, in main
status = self.run(options, args)
File "C:\Python27\lib\site-packages\pip\commands\install.py", line 283, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File "C:\Python27\lib\site-packages\pip\req.py", line 1435, in install
requirement.install(install_options, global_options, *args, **kwargs)
File "C:\Python27\lib\site-packages\pip\req.py", line 706, in install
cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
File "C:\Python27\lib\site-packages\pip\util.py", line 697, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command C:\Python27\python.exe -c "import setuptools, tokenize;__file__='c:\\users\\mgarrett\\appdata\\local\\temp\\pip_build_mgarrett\\lxml\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record c:\users\mgarrett\appdata\local\temp\pip-7wgh3w-record\install-record.txt --single-version-externally-managed --compile failed with error code 1 in c:\users\mgarrett\appdata\local\temp\pip_build_mgarrett\lxml
I've previously installed setuptools, so I'm not sure what the log file is referring to at the end?

Related

Unable to pip install psycopg2

I have been trying to google this all over the internet and still can't figure out how to get this package installed. Any help is much appreciated. I have tried numerous examples from google to no avail.
Mac OS 10.15.3
pip freeze
asgiref==3.2.5
dj-database-url==0.5.0
Django==3.0.4
entrypoints==0.3
flake8==3.7.9
gunicorn==20.0.4
mccabe==0.6.1
postgres==3.0.0
psycopg2-binary==2.8.4
psycopg2-pool==1.1
pycodestyle==2.5.0
pyflakes==2.1.1
pytz==2019.3
sqlparse==0.3.1
whitenoise==5.0.1
pip install psycopg2
Collecting psycopg2
Using cached psycopg2-2.8.4.tar.gz (377 kB)
ERROR: Command errored out with exit status 1:
command: /Users/keeganleary/Documents/Coding/life_cal/venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"
'"'/private/var/folders/s6/f4q9mnjn7q9b34gw7bz35l680000gn/T/pip-install-qwbzap8s/psycopg2/setup.py'"'"'; __file__='"'"'/private/va
r/folders/s6/f4q9mnjn7q9b34gw7bz35l680000gn/T/pip-install-qwbzap8s/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open
)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
--egg-base /private/var/folders/s6/f4q9mnjn7q9b34gw7bz35l680000gn/T/pip-install-qwbzap8s/psycopg2/pip-egg-info
cwd: /private/var/folders/s6/f4q9mnjn7q9b34gw7bz35l680000gn/T/pip-install-qwbzap8s/psycopg2/
Complete output (12 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/keeganleary/Documents/Coding/life_cal/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 20, in <mo
dule>
from setuptools.dist import Distribution, Feature
File "/Users/keeganleary/Documents/Coding/life_cal/venv/lib/python3.8/site-packages/setuptools/dist.py", line 34, in <module
>
from setuptools.depends import Require
File "/Users/keeganleary/Documents/Coding/life_cal/venv/lib/python3.8/site-packages/setuptools/depends.py", line 7, in <modu
le>
from .py33compat import Bytecode
File "/Users/keeganleary/Documents/Coding/life_cal/venv/lib/python3.8/site-packages/setuptools/py33compat.py", line 2, in <m
odule>
import array
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/array.cpython-38-darwin.so, 2)
: no suitable image found. Did find:
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/array.cpython-38-darwin.so: code signature in
(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/array.cpython-38-darwin.so) not valid for use in proc
ess using Library Validation: Library load disallowed by System Policy
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Try
pip install psycopg2-binary
Hope this helps you

'Import Error: No module named msvc' when running pip install ipdb on Ubuntu 18.04

I keep getting this error when trying to run pip install ipdb on Ubuntu 18.04:
Collecting ipdb
Using cached https://files.pythonhosted.org/packages/df/78/3d0d7253dc85549db182cbe4b43b30c506c84008fcd39898122c9b6306a9/ipdb-0.12.2.tar.gz
ERROR: Command errored out with exit status 1:
command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-PpJ5WV/ipdb/setup.py'"'"'; __file__='"'"'/tmp/pip-install-PpJ5WV/ipdb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
cwd: /tmp/pip-install-PpJ5WV/ipdb/
Complete output (11 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/symphony-client/setuptools/__init__.py", line 160, in <module>
monkey.patch_all()
File "/opt/symphony-client/setuptools/monkey.py", line 104, in patch_all
patch_for_msvc_specialized_compiler()
File "/opt/symphony-client/setuptools/monkey.py", line 156, in patch_for_msvc_specialized_compiler
msvc = import_module('setuptools.msvc')
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named msvc
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
I did a pip search msvc and it found msvc-runtime but when I try and pip install msvc-runtime I get the following:
Collecting msvc-runtime
ERROR: Could not find a version that satisfies the requirement msvc-runtime (from versions: none)
ERROR: No matching distribution found for msvc-runtime
Any help is appreciated.

How to install django-channels in ubuntu?

I have tried to install django channels using this command.
pip install -U channels
but am getting this error
Downloading/unpacking channels Downloading
channels-2.0.2-py2.py3-none-any.whl Cleaning up... Exception:
Traceback (most recent call last): File
"/home/praneet/pral/myvenv/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py",
line 2482, in _dep_map
return self.dep_map File "/home/praneet/pral/myvenv/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py",
line 2344, in __getattr
raise AttributeError(attr) AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
"/home/praneet/pral/myvenv/lib/python3.4/site-packages/pip/basecommand.py",
line 122, in main
status = self.run(options, args) File "/home/praneet/pral/myvenv/lib/python3.4/site-packages/pip/commands/install.py",
line 278, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle) File
"/home/praneet/pral/myvenv/lib/python3.4/site-packages/pip/req.py",
line 1266, in prepare_files
req_to_install.extras): File "/home/praneet/pral/myvenv/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py",
line 2291, in requires
dm = self._dep_map File "/home/praneet/pral/myvenv/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py",
line 2484, in _dep_map
self.__dep_map = self._compute_dependencies() File "/home/praneet/pral/myvenv/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py",
line 2508, in _compute_dependencies
parsed = next(parse_requirements(distvers)) File "/home/praneet/pral/myvenv/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py",
line 2605, in parse_requirements
line, p, specs = scan_list(VERSION,LINE_END,line,p,(1,2),"version spec") File
"/home/praneet/pral/myvenv/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py",
line 2573, in scan_list
raise ValueError("Expected "+item_name+" in",line,"at",line[p:]) ValueError: ('Expected version spec in', 'asgiref ~=2.1', 'at', '
~=2.1')
Storing debug log for failure in /home/praneet/.pip/pip.log
I was also stuck with same issue.
Now,I have fixed it by installing following packages:
python2
sudo apt-get install python-dev python-pip libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev
python3
sudo apt-get install python3-dev python3-pip libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev

pip install produces OSError: [Errno 13] Permission denied:

I'm wanting to install ten packages via pip in virtualenv.
I possibly used sudo improperly in my haste to get it "working" as suggested by https://stackoverflow.com/a/27939356/1063287, ie I installed virtualenv with sudo:
sudo virtualenv --no-site-packages ENV
I did this because without sudo I got this:
me#my-comp:/var/www/html$ virtualenv --no-site-packages ENV
Running virtualenv with interpreter /usr/bin/python2
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/virtualenv.py", line 2364, in <module>
main()
File "/usr/lib/python3/dist-packages/virtualenv.py", line 719, in main
symlink=options.symlink)
File "/usr/lib/python3/dist-packages/virtualenv.py", line 942, in create_environment
site_packages=site_packages, clear=clear, symlink=symlink))
File "/usr/lib/python3/dist-packages/virtualenv.py", line 1144, in install_python
mkdir(lib_dir)
File "/usr/lib/python3/dist-packages/virtualenv.py", line 324, in mkdir
os.makedirs(path)
File "/usr/lib/python2.7/os.py", line 150, in makedirs
makedirs(head, mode)
File "/usr/lib/python2.7/os.py", line 150, in makedirs
makedirs(head, mode)
File "/usr/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/var/www/html/ENV'
In Ubuntu 16.04 I cannot see "Disk Utility" to test the solution offered however.
Trying to pip install lxml results in this final error:
Command "/var/www/html/ENV/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-jcCDbh/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-_oNugl-record/install-record.txt --single-version-externally-managed --compile --install-headers /var/www/html/ENV/include/site/python2.7/lxml" failed with error code 1 in /tmp/pip-build-jcCDbh/lxml/
Whilst two other examples are below:
pip install bottle:
(ENV) me#my-comp:/var/www/html/ENV$ pip install bottle
Collecting bottle
Installing collected packages: bottle
Exception:
Traceback (most recent call last):
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/commands/install.py", line 335, in run
prefix=options.prefix_path,
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/req/req_set.py", line 732, in install
**kwargs
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/req/req_install.py", line 835, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/req/req_install.py", line 1030, in move_wheel_files
isolated=self.isolated,
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/wheel.py", line 344, in move_wheel_files
clobber(source, lib_dir, True)
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/wheel.py", line 322, in clobber
shutil.copyfile(srcfile, destfile)
File "/usr/lib/python2.7/shutil.py", line 83, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: '/var/www/html/ENV/lib/python2.7/site-packages/bottle.pyc'
pip install requests:
(ENV) me#my-comp:/var/www/html/ENV$ pip install requests
Collecting requests
Using cached requests-2.9.1-py2.py3-none-any.whl
Installing collected packages: requests
Exception:
Traceback (most recent call last):
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/commands/install.py", line 335, in run
prefix=options.prefix_path,
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/req/req_set.py", line 732, in install
**kwargs
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/req/req_install.py", line 835, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/req/req_install.py", line 1030, in move_wheel_files
isolated=self.isolated,
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/wheel.py", line 344, in move_wheel_files
clobber(source, lib_dir, True)
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/wheel.py", line 315, in clobber
ensure_dir(destdir)
File "/var/www/html/ENV/local/lib/python2.7/site-packages/pip/utils/__init__.py", line 83, in ensure_dir
os.makedirs(path)
File "/var/www/html/ENV/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/var/www/html/ENV/lib/python2.7/site-packages/requests-2.9.1.dist-info'
If I use sudo pip install bottle, I get:
sudo: pip: command not found
Update:
I ran this suggestion:
$sudo chown -R $(whoami) /var/www/html/ENV
and can now pip install bottle, requests, pymongo, beautifulsoup4, Beaker, pycrypto and tldextract. However, lxml and pillow are failing.
lxml fail:
Failed building wheel for lxml
Command "/var/www/html/ENV/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-yHLQQe/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-hLznuQ-record/install-record.txt --single-version-externally-managed --compile --install-headers /var/www/html/ENV/include/site/python2.7/lxml" failed with error code 1 in /tmp/pip-build-yHLQQe/lxml/
pillow fail:
Failed building wheel for pillow
Command "/var/www/html/ENV/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-IkuM34/pillow/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-60McJh-record/install-record.txt --single-version-externally-managed --compile --install-headers /var/www/html/ENV/include/site/python2.7/pillow" failed with error code 1 in /tmp/pip-build-IkuM34/pillow/
I have tried the suggestion here:
https://stackoverflow.com/a/6504860/1063287
for troubleshooting these remaining errors and libxml2-dev, libxslt1-dev and python2.7-dev are already installed.
Update 2:
Installed zlib1g-dev as per:
https://stackoverflow.com/a/19289133/1063287
and can install lxml now.
Still can't install pillow.
Update 3:
Installed libjpeg8-dev as per:
https://stackoverflow.com/a/33582789/1063287
and can now install pillow.
Have you installed pip?
Try installing pip by
sudo apt-get install python
,download pip from https://pip.pypa.io/en/stable/installing/ then do a 'python get-pip.py'. This will install pip
Then for the issue of permission denied use
$sudo chown -R $(whoami) /var/www/html/ENV
The solution was a combination of this suggestion:
$sudo chown -R $(whoami) /var/www/html/ENV
And making sure the following packages were installed:
python2.7-dev
libxml2-dev
libxslt1-dev ( for ^, as per https://stackoverflow.com/a/6504860/1063287)
apache2-dev
zlib1g-dev (for lxml, as per https://stackoverflow.com/a/19289133/1063287)
libjpeg8-dev (for pillow, as per https://stackoverflow.com/a/33582789/1063287)

Scipy autotest finished with one error

Under Ubuntu 14.04 I installed python 2.76. then scipy as shown here, namely through
sudo apt-get install python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose
The python -c "import numpy; numpy.test()" test finishes with no error, but python -c "import scipy; scipy.test()" results in:
======================================================================
ERROR: test_fitpack.TestSplder.test_kink
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/dist-packages/scipy/interpolate/tests/test_fitpack.py", line 333, in test_kink
splder(spl2, 1) # Should work
File "/usr/lib/python2.7/dist-packages/scipy/interpolate/fitpack.py", line 1186, in splder
"and is not differentiable %d times") % n)
ValueError: The spline has internal repeated knots and is not differentiable 1 times
Ran 8936 tests in 77.338s
FAILED (KNOWNFAIL=115, SKIP=191, errors=1)
What should I do?