I have Tensorflow with GPU installed on my main usr/bin/python, e.g. on an AWS EC2 machine and I'd like to use it within a conda environment. Is this possible without reinstalling tensorflow for each environment ?
EDIT: I found out about the virtual-env --system-site-packages option. Maybe an equivalent for Conda ? And especially for packages like Tensorflow-gpu that need to be built for the specific hardware, and are not available directly on pip.
If tensorflow is installed in the system, then it will probably be present in one of these four directories:
/usr/local/lib/python2.7/dist-packages
/usr/local/lib/python2.7/site-packages
/usr/lib/python2.7/dist-packages
/usr/lib/python2.7/site-packages
To have the system packages also be available in conda, you can do the following:
(root) ~/condaexpts $ cat <<EOF > $CONDA_PREFIX/lib/python2.7/site-packages/systempkgs.pth
/usr/local/lib/python2.7/dist-packages
/usr/local/lib/python2.7/site-packages
/usr/lib/python2.7/dist-packages
/usr/lib/python2.7/site-packages
EOF
Then verify that these paths have been added:
(root) ~/condaexpts $ python
Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul 2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import sys
>>> sys.path
['', '/home/ubuntu/condaexpts/m2/lib/python27.zip', '/home/ubuntu/condaexpts/m2/lib/python2.7', '/home/ubuntu/condaexpts/m2/lib/python2.7/plat-linux2', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-tk', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-old', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-dynload', '/home/ubuntu/condaexpts/m2/lib/python2.7/site-packages', '/home/ubuntu/condaexpts/m2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/site-packages', '/usr/lib/python2.7/dist-packages']
And now you can import any system package.
If you don't want this change to be permanent for your conda environment, you can always add the directory containing system packages to PYTHONPATH environment variable:
(root) ~/condaexpts $ PYTHONPATH=$PYTHONPATH:/usr/local/lib/python2.7/dist-packages/ python
Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul 2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import sys
>>> sys.path
['', '/home/ubuntu/condaexpts', '/usr/local/lib/python2.7/dist-packages', '/home/ubuntu/condaexpts/m2/lib/python27.zip', '/home/ubuntu/condaexpts/m2/lib/python2.7', '/home/ubuntu/condaexpts/m2/lib/python2.7/plat-linux2', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-tk', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-old', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-dynload', '/home/ubuntu/condaexpts/m2/lib/python2.7/site-packages', '/home/ubuntu/condaexpts/m2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg']
Related
(python2.7) (23:28:22) ~$ python
Python 2.7.14 |Anaconda, Inc.| (default, Dec 7 2017, 17:05:42)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import six
>>> six.__version__
'1.5.2'
>>> quit()
(python2.7) (23:30:09) ~$ conda install six
Fetching package metadata ...........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /home/hyejinj/anaconda3/envs/python2.7:
#
six 1.11.0 py27h5f960f1_1
I need the six package with version 1.11.0, but my python in the conda environment keeps using the default six, whose version is 1.5.2. How can I resolve this issue?
I have various versions of python (python2.4 and python2.7) installed on the CentOS machine. I installed MySQL-python, but I can only import the MySQLdb module on python2.4 but not python2.7
root# python2.7
Python 2.7.11 |Continuum Analytics, Inc.| (default, Dec 6 2015, 18:08:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import MySQLdb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named MySQLdb
root# python
Python 2.4.3 (#1, Mar 5 2011, 21:26:05)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
>>>
I've tried alias'ing python to /usr/bin/python2.7 and then doing a 'yum remove MySQL-python' and then doing a 'yum install MySQL-python' but no luck. Any clues/insights? Thanks.
The yum will install the MySQL-python to the system default python's directory.
maybe you can try virtualenv and pip.
install the pip
pip install virtualenv
virtualenv -p /the/localtion/of/python27 python27
active the new python envirement, and pip install MySQL-python
I use Python 2.7.6 on my machine
$ python --version
Python 2.7.6
I have on my machine Spark 1.1.0 depended from Python 2.7.6. If I execute:
user#user:~/bin/spark-1.1.0$ ./bin/pyspark
I get
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
.
.
.
Today I install the new pre-built version of Spark 1.3.1 (I don't know why, but depended from python 2.7.5). If I execute now the same command for the new version:
user#user:~/bin/spark-1.3.1-bin-hadoop2.6$ ./bin/pyspark
I get back the older Python version
Python 2.7.5 (default, Jun 18 2014, 09:37:37)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
.
.
.
The main difference is that in the older Spark version I can execute import numpy, in the new one not.
I created the next path for Python in .bashrc file:
export PYTHONPATH=$PYTHONPATH:usr/lib/python2.7
I don't find the way to distinguish between version 2.7.6 and 2.7.5 in Python, thus I don't know a place, where python 2.7.6 is stored (command find is missing).
You can set the python executable which is used in pyspark and spark-submit by an environmet variable "PYSPARK_PYTHON".
For example,
PYSPARK_PYTHON=/opt/local/python-2.7/bin/python pyspark
I solve this problem with the removing of python2.7.5 from HD. The topic can be closed
I run this and it worked perfectly fine. "export PYSPARK_PYTHON=python3"
i have a problem:
I use virtualbox, vagrant and pycharm to create environment code
Then install vagrant,pycharm and virtualbox on win 8 64bit, i begin create a project and all ok
But i check version python on machine virtual(on virtualbox, i use centos6.4), version python is 2.6.3,i need python 2.7 for my project
Then i install python 2.7 change python2.6, all completed and success
But when i create other project use environment that i install python 2.7 but version python is python 2.6
I want all my project will run on the virtualbox in the future via Vagrant which always runs on Python 2.7. What should i do?
project new
$ vagrant ssh
Last login: Tue Jul 1 14:45:18 2014 from 10.0.2.2
Welcome to your Vagrant-built virtual machine.
[vagrant#localhost ~]$ python
Python 2.6.6 (r266:84292, Feb 21 2013, 23:54:59)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
project old
[vagrant#localhost ~]$ python
Python 2.7.6 (default, Jul 1 2014, 10:42:45)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
Help me
If I understand you correctly, you like to use a centos 6.4 box but with a non-standard python version for all your projects? In this case you may have to provision your box using a custom shell script that will handle the python upgrade process each time a centos 6.4 box is used.
First time ever I'm installing Django (and using python). I run Mac OSX 10.8 Mountain Lion and by default, python is already installed:
$ python
Python 2.7.2 (default, Jun 20 2012, 16:23:33)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
I've followed the steps at https://docs.djangoproject.com/en/1.4/intro/install/ namely:
Download Django at https://www.djangoproject.com/download/ and run the following:
$ tar xzvf Django-1.4.1.tar.gz
$ cd Django-1.4.1
$ sudo python setup.py install
This has created a symlink in /user/local/bin which allows me to invoke Django from anywhere with $ django-admin.py
However, trying the following fails:
$ python
Python 2.7.2 (default, Jun 20 2012, 16:23:33)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>>
I should be getting something after typing >>> import django
>>> print django.get_version()
1.4
Is it me missing something or the documentation not so accurate?
Actually, the fact that import django doesn't print anything means that Django was installed correctly.
import django only imports the Django module; it doesn't start a webserver or anything.