I posted the question below, but none of the answers I was pointed to worked, though they look like they should.
I activated (again) the virtualenv. It still tells me that pip can't be found by apt when doing an 'apt install' command. But here is where I am now, and very confused.
I pointed my directory to "/home/.../q7root/bin/pip" and did an "ls". It shows a sub-directory with pip in it (or, I think, a link to it - I'm not the best at Unix). When I type "which pip" I get the path to this point ('q7root/pip'). bit if I just type "pip" at the CLI I get I get this error:
[![pip error][1]][1]
I have looked at my PATH, and this q7root/bin is the first place to look on the path. And, despite trying mightily with all the references people gave me, pip3 never gets installed.
But even pip is challenged. "which pip" points to this copy in the virtual environment site, but typing "pip" as a command tells me 'No module named pip.'
So pip seems to need more stuff installed (?), or there is some mess. Any advice?
Original Question:
At the suggestion of others working on what was a functional Django project, I upgraded to a more recent version of Ubuntu (18).
However, when I first try to run it it blows up at line 3 of the initial script module when asked to import django as a package.
I tried pip -r requirements.txt, but the system said pip was an unknown package. I dropped down and used apt to load pip onto my machine (sudo apt-get pip), then tried using pip itself (pip update pip) which failed.:
[![Pip load error message][2]][2]
I also tried pip install django, and got this:
[![django not found][3]][3]
I would have thought an OS upgrade would not require re-installing all currently installed packages (seems like a no-brainer to do the work of installing everything that had been installed). But right now I am terribly stuck...obviously, having 'pip' let's you (at least) have a basic CLI tool.
Any advice?
[1]: https://i.stack.imgur.com/OPfgc.png
[2]: https://i.stack.imgur.com/shLOc.png
[3]: https://i.stack.imgur.com/bEhDB.png
It depends on the version of python.
Python 3
sudo apt update
sudo apt install python3-pip
Python 2
sudo apt update
sudo apt install python-pip
How to Install Pip on Ubuntu 18.04
Start with a fresh Ubuntu install. I think you've run too many commands for your current setup to be reproduceable.
Install python3 and python3-venv.
sudo apt update
sudo apt install python3 python3-venv
Use the venv module to create the virtual env.
python3 -m venv myenv
source myenv/bin/activate
You now have access to pip in the venv.
It's OK to upgrade pip in the virtual env, I suggest you don't ever upgrade the system pip otherwise you might hit issues like this.
(myenv) python -m pip --version
(myenv) python -m pip install --upgrade pip
Now you can use pip to install your requirements in the virtual env.
(myenv) python -m pip install -r requirements.txt
In the above commands I've used python -m pip instead of pip. This is the recommended way, as it ensures that you are using the version of pip that matches python.
In the end, this was a state of deep computer confusion. I was already disk-limited so I bought a new computer, and this error did not recur with the same code being used.
I am a newbie to Django and Python installation. Intermediate with Ubuntu 14.04.
These are my installations so far in my Ubuntu 14.04.
apt-get install python3-setuptools --force-yes -y ## for python3
easy_install3 pip ## for python3
apt-get install python-setuptools --force-yes -y ## for python2.7 or above
easy_install pip ## for python2.7 or above
apt-get install python-dev --force-yes -y ## because ubuntu 14.04 does not have dev version of python 2
apt-get install python3-dev --force-yes -y ## because ubuntu 14.04 does not have dev version of python 3.4
apt-get install links --force-yes -y ##a command line web browser
apt-get install python-flup --force-yes -y ## connects python to uwsgi)
apt-get install build-essential --force-yes -y ##
pip2 install django uwsgi virtualenv ## use pip to install django and uwsgi and virtualenv for python2
pip3 install django uwsgi ## use pip to install django and uwsgi for python3
For the full list, please look at https://gist.github.com/simkimsia/41c55e0c08eda42e2cb3#file-install-sh-L88
I am confused about the use of virtualenv.
I want to prepare my ubuntu 14.04 server edition for a production level of Django 1.7 as much as possible.
The reason why I installed multiple Python environment because I may have other Python apps running which require 3.4.
My Django files are from bitbucket repository and I have git cloned them into /var/virtual/WebApps/DjangoProject
Inside /var/virtual/WebApps/DjangoProject, I have manage.py and other files and folders.
Please advise on how do I get the Django project running for this situation.
I am currently testing this setup on my virtualbox and vagrant.
EDIT
There will be at least 2 Django applications. 1 requires 2.7 python. The other requires 3.4 python.
Let me add that this is a single server that will host the application, frontend, and database.
EDIT 1
I have restarted with a fresh install of Ubuntu 14.04 and I started with Python 2.7.6 and Python 3.4.0.
I then did sudo apt-get install python-virtualenv which I checked its version: 1.11.4.
I have created ~/virtualenvs/py2.7 and ~/virtualenvs/py3.4.
Inside ~/virtualenvs, I did virtualenv -p /usr/bin/python2 py2.7
and ~/virtualenvs, I did virtualenv -p /usr/bin/python3 py3.4
So how do I install python2 only libraries for the python 2 app?
E.g. are django-adminfiles, sorl-thumbnail, psycopg2
EDIT 5
Use virtualenv --system-site-packages -p /usr/bin/python2 py2.7 instead
I have restarted with a fresh install of Ubuntu 14.04 and I started
with Python 2.7.6 and Python 3.4.0.
Okay, so now in your system you have two base versions of Python. Base version just means, the versions that are supported by your operating system; which you have installed globally.
In other words you have installed them using the operating system's package installers and did not compile them separately.
In practice, this above only matters in Linux, because in Windows you cannot install "locally" without going through a few hoops; all Python installers will register themselves in the registry thus making them global, base Python versions.
I then did sudo apt-get install python-virtualenv which I checked its
version: 1.11.4.
This package is outdated (current version is 12.0.7).
Now you have virtual environment installed against the base Python 2 version because the package requires Python 2.
In practical terms it means if you need to upgrade Python 2, you'll have to make sure python-virtualenv is also updated for the base versions of both Pythons that are supported by your operating system. This means, when you apt-get update and apt-get upgrade, virtualenv will be upgraded.
Usually this doesn't matter as its a rare case if python2 is upgraded and then python-virtualenv is not upgraded to match its dependency.
However this is not recommended because you want to control the versions of critical software "manually" to avoid any surprises. There are ways to control this on Ubuntu and other debian-like distributions by pinning versions; but even if you do so, you may not be getting the latest version of the library which may force you later on to uninstall the version that came with your operating system, and reinstall it from source.
I have created ~/virtualenvs/py2.7 and ~/virtualenvs/py3.4.
Inside ~/virtualenvs, I did virtualenv -p /usr/bin/python2 py2.7 and
~/virtualenvs, I did virtualenv -p /usr/bin/python3 py3.4
So how do I install python2 only libraries for the python 2 app?
E.g. are django-adminfiles, sorl-thumbnail, psycopg2
In order for solr-thumbnail and psycopg2 to be installed correctly, you need to build their dependencies; so
sudo apt-get install libjpeg62 libjpeg62-dev zlib1g-dev
sudo apt-get install libgraphicsmagick++-dev libboost-python1.55-dev
sudo apt-get install libexpat1-dev libpython-dev libpython3-dev libssl-dev libpq-dev
To install libraries for the Python 2 app:
Activate the virtual environment; by typing source ~/virtualenvs/py2.7/bin/activate
Type pip install _____ (name of the library)
To support multiple Python applications with different major versions; your system should have both Python major versions installed (you already have done this).
You then install virtualenv for each Python major version. You can skip this step if your applications are fully contained (that is, they include the Python runtime required - but this is a rare case) or if you have a single purpose server.
You should avoid installing anything but the base Python libraries in your system's global python. That is avoid (as root, or using sudo) to pip install things; because these will be installed for all users of Python and may cause problems (on some systems; like Fedora/RedHat - critical system packages like yum rely on the base system Python).
Next step is to make sure you have a suitable build environment available. This means for Debian-sourced systems to install build-essential and further the support libraries for common Python drivers and modules. The exact libraries you need to install will depend on the applications you are planning to host, but at a minimum you should make sure PIL (or Pillow) can be installed and database drivers' support libraries are available. To do so, you can apt-get build-deb python-imaging psycopg2 python-mysqldb (for PostgreSQL, MySQL and PIL).
Now you have a system ready to support most Python applications. You can then optionally add other utilities, but I would try to avoid assuming too much about what applications will require.
To host an app:
Create a virtual environment with the base version of Python required. So virtualenv-2.7 or virtualenv-3 as the normal non-root user account.
Install required packages into the virtual environment.
Adjust the bootstrap script for your application to use the correct Python binary. This is usually done from whatever process you are using to manage your application server. For example, on my server I use supervisord.
That's all you have to do. Everything else will depend on the individual application's requirements and setup (so if you need to serve static files, you'll have to configure that mapping, etc.)
After reading your shell script, it seems you are trying to build a server that will support both the application, the front end and the database.
In order to support such a system; you will need to install the following:
Database server(s) that you would like to support. As this is a single purpose server, you will also need to install database command line clients.
Source code tools (git, etc.)
A global process manager (like supervisor or circus).
Base Python versions you intend to support; and their development headers (sudo apt-get install python-dev)
setuptools, pip, and then virtualenv. These tools should be installed from source rather than your package manager; to ensure the latest versions are installed. You should install these globally (ie, as root) so they are available for all users.
A build tool chain (ie, "Development Tools" or build-essential)
Support libraries for any extensions (but not the extensions themselves). The easiest way to do this is to use the package manager to build the dependencies and not the packages sudo apt-get build-dep python-imaging psycopg2 python-mysqldb.
The next thing you need to do is decide how you will run your application servers (the "django code"). You can use uwsgi, gunicorn, etc. as these are the ones most tested with django.
You need to be able to support multiple versions of these runtimes, so instead of installing them globally across the system; just build their dependencies and install the specific version required for each application in its own isolated environment.
The next thing you need to install is a front end proxy for your applications. You can install whatever proxy suits your needs (nginx is the most popular); but please install from source rather than the packages as those are almost always out of date.
Once all this is setup, the process of hosting a django application is the following:
Create a separate user account with a non-login shell.
Create a virtual environment in this user's home directory. I recommend keeping some standard here, like env for the virtual environments.
Download/copy the source code of your application.
Create a standard directory where you will store the static files. For example I use $HOME/www/static.
Create an entry in your process manager.
Create an entry in your proxy for front end routing.
Reload your proxy server.
Reload your process manager.
You can automate/script a lot of the above. For example, you can create a custom skeleton directory to create the base directories for you when adding new users; and you can create custom templates for other areas using tools like cookiecutter.
Understanding execution path
The first principle to understand is that when you install packages on your operating system (virtual machine, whatever) with sudo is that you are installing all those packages in a global location, that is, a file directory which your system knows about via the $PATH (PATH) environment variable.
This $PATH variable is often set-up by default in your .bashrc or your .bash_profile or your .profile whenever a new linux/unix user is created. Whether you have any of these files depends on what is inside the /etc/skel on your system. /etc/skel holds a "template" of these files that gets duplicated whenever a new user with a home directory is created.
When you type echo $PATH in command line, you would see a list of execution search path delimited by :, for example:
/usr/local/sbin:/usr/local/bin:/usr/bin
This is an example I will use for the purpose of this discussion. Different OS will give you slightly different default $PATH but the simple idea here is that the binaries of your globally installed packages gets thrown into one of these directories and the reason you are able to run these binaries is because these binaries (programs) are now available in the execution search path (simply called $PATH).
Ok great, but what does this mean for the python 2 and python 3 interpreters I installed?
So in relation to your question, this means that when you sudo apt-get or sudo aptitude install python2 or python3 binaries (python interpreters), they are both available and to differentiate which interpeter you are using at any moment, you would run python2.7 or python3.4 that correspondingly calls /usr/bin/python2.7 or /usr/bin/python3.4
You can always check this easily by using the which linux/unix command. which python3.4 will return you the exact path where your python3.4 binary is installed.
Similarly, when type pip2.7 in your command line, you are asking your system to execute the pip2.7 program that came installed with your python2.7 package. And naturally which pip2.7 will reveal to you where this pip2.7 binary has been installed.
All this is possible simply because these binaries are have been installed by you using sudo apt-get and are placed in the directories listed in $PATH. If you move one of these binaries forcibly to another directory not listed in $PATH, you will realize that you can no longer run the binary in your command line without typing out the specific path to the binary.
Additional python specific information
The Python Interpreter has another attribute call $PYTHONPATH. This - as you will rightly deduce - is a variable that holds a list of directories ("search path") where the python interpreter will search for python modules to load. If you want to know where your python interpreter is currently looking for modules (your own python modules or 3rd party python modules), run
python -c "import sys; print(sys.path)"
where python is your specific python interpreter. If /usr/bin/python is symlinked to /usr/bin/python2.7, then you are in fact calling python2.7.
When would python path matter? It matters inside your own .py source code when you ask to import other modules. The import line of code in your .py source code is where you are asking the python interpreter to go forth and search for the module that you want to import. As you can imagine, if your sys.path (in python) is empty, you will not be able to import any 3rd party modules.
Beyond the gory details
Now that we have a clear understanding of the underlying principles behind $PATH and $PYTHONPATH, we can now understand what the virtualenv (and additionally, virtualenvwrapper is useful) does for us.
When we create a new virtualenv giving it a directory, what we are saying is that we want to symlink a specific python interpreter (via -p python2.7 flag for the virtualenv command) for example to that virtualenv.
When we source activate that virtualenv we created, we are in fact invoking shell scripts that come with virtualenv to dynamically modify the $PATH. Try running echo $PATH after you have source activated a virtualenv you created. What do you see?
That's right, you will be seeing something like this:-
/Users/calvin/.virtualenvs/myproject/bin:/usr/local/sbin:/usr/local/bin:/usr/bin
And if you type which python, what do you get?
/Users/calvin/.virtualenvs/myproject/bin/python
That's right, the python interpreter we are using is the one that is inside the virtualenv directory and no longer from the /usr/bin directory.
If you run
ls -la /Users/calvin/.virtualenvs/myproject/bin
what do you see?
Ah! A symlink to the specific global python interpreter you specified when you created this virtualenv. If you don't see a symlink, this means that your entire python interpreter was copied over from the globally installed one.
So this is what our virtualenv tool does. It lets us isolate specific projects and choose which python interpreter to use for a specific project.
Once you have source activated a virtualenv, your pip is also the pip that is located in your virtualenv directory. This pip is python path-aware. When you issue pip install commands, python packages now gets installed into your
/Users/calvin/.virtualenvs/myproject/lib/site-packages
python packages directory; and are available ONLY to your project when you source activate that particular project.
What if you install packages with sudo pip?
sudo pip calls the pip tool that comes with your globally installed python interpreter. When you use sudo pip, you are installing your python packages into a global location (which is not jailed within a virtualenv).
I did not give you specific answers to your how-can-I-make-two-projects-use-two-different-python-interpreters question. I explained the principles and now that you know the principles, it is clear what you need to do. Cheers!
:D
you should have to try following steps to make your django work with python-2.7.
As Ubuntu-14 already come with Python-2.7 installed. so no need to install python-2.7.
First of all install python setuptools for python-2.7:
https://pypi.python.org/packages/source/s/setuptools/setuptools-12.0.5.tar.gz
tar -zxvf setuptools-12.0.5.tar.gz
cd setuptools-12.0.5/
sudo python2.7 setup.py install
after install setuptools install pip for python-2.7
wget https://pypi.python.org/packages/source/p/pip/pip-6.0.8.tar.gz
tar -zxvf pip-6.0.8.tar.gz
cd pip-6.0.8/
sudu python2.7 setup.py install
after installing pip now we have to go for install python virtual environment:
sudo pip2.7 install virtualenv
then to folder where you want to create new virtualenv using python 2.7
virtualenv-2.7 "name for new virtualenv for example '/var/virtual/DjangoProject'"
cd /var/virtual/
virtualenv-2.7 DjangoProject
Note just created virtualenv name is "DjangoProject"
Now time to activate new virtualenv
cd DjangoProject
source bib/activate
Now time to clone your project here.
git clone "your git url"
install rest of apps and django here. if you have created simply install by requirement.txt
cd "projectFolder"
pip install -r requirement.txt
after doing all these your new python2.7 django with virtualenv are ready to use. just run django wsgi server for test is every thing working:
python manage.py runserver.
Now you can use this virtualenv and project to setup for web server.
Hope this will help-full to you.
You should use pyenv (and pyenv virtualenvwrapper). With these tools you can have multiple Python versions and switch between them.
Here is how you setup one environment with Python 3.4.2 and one with Python 2.7.8 (and each environment can have its own Python packages)
# install pyenv
git clone https://github.com/yyuu/pyenv.git ~/.pyenv
# setup pyenv and load when shell is loaded
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc
# install pyenv virtualenvwrapper
git clone https://github.com/yyuu/pyenv-virtualenvwrapper.git ~/.pyenv/plugins/pyenv-virtualenvwrapper
# setup pyenv virtualenvwrapper and load when shell is loaded
echo 'export PYENV_VIRTUALENVWRAPPER_PREFER_PYVENV="true"' >> ~/.bashrc
echo "pyenv virtualenvwrapper" >> ~/.bashrc
Now everything is setup. Restart your shell.
# install python 2.7.8
pyenv install 2.7.8
pyenv rehash
# install python 3.4.2
pyenv install 3.4.2
pyenv rehash
# switch the global python version to 2.7.8
pyenv global 2.7.8
# create a virtual environment using python 2.7.8
mkvirtualenv project278
# -- now you can install you python 2 modules
# leave the virtual environment
deactivate
# switch the global python to 3.4.2
pyenv global 3.4.2
# create a virtual environment using python 3.4.2
mkvirtualenv project342
# -- now you can install you python 2 modules
# leave the virtual environment
deactivate
# switch the global python to the system python.
pyenv global system
# swith to environment project278 and check the python version
workon project278
python --version
# swith to environment project342 and check the python version
workon project342
python --version
Voilà! You now have two environments with different Python versions. Just use the workon (and the deactivate) commands to switch between them.
Make sure you read the documentation of the used packages, so you understand what is going on!
sudo apt-get install python-Orange
or
sudo apt-get install python-orange
doesn't work
sudo python setup.py install
sudo python setup.py build
is not working as well.
Can anyone help??
Python has two tools for easy installation of all programs that are listed on the Python Package Index, also known as PyPi: These are easy_install and pip. Both retrieve very recent versions of Orange (and of any other package that is updating its PyPi entry regularly).
I installed Orange on Ubuntu 12.04 (LTS) with
pip install orange.
You will see lots of log lines indicating that Pip is downloading and compiling Orange for you. Simply wait. When pip is ready, fire up python and try to import orange. If that works, quit python and try the GUI with python /usr/local/lib/python2.7/dist-packages/Orange/OrangeCanvas/orngCanvas.pyw (you probably want to create a shell alias or bash script for that one :-)
NOTE: on 12.04 I needed to first upgrade 'distribute' itself with sudo easy_install -U distribute but this was clearly indicated by pip.
https://pypi.python.org/pypi/Orange/2.6/
You need to extract the dowloaded tarball on that page to a folder and then change directory to that folder. Then the sudo python setup.py... instructions will work (but you should 'build' the application before you 'install' it).
go to the given link "https://pypi.python.org/pypi/Orange/2.6/"
download the package and extract the file
install with given command
python setup.py build
python setup.py install
note:- during installation make sure that your net is working because it downloads required packages. Also it may ask for C++ or gcc compilers while installing and could be terminate just read the errors care fully and install requires packages from the synaptic package manage in ubuntu.
I'm trying to install eyeD3 on Webfaction for my Django application and I tried to this command
easy_install-2.7 eyeD3
and it didn't work, also Webfaction prevents me from downloading packages as well right?
I could install "boto" with the very same command tho.
Any ideas?
Thanks in advance
You can always pull down whatever packages you need and place them into your /lib/python2.x folder within your application directory. WebFaction doesn't prevent you from downloading or installing packages using pip or easy_install.
Have a look at their documentation: http://docs.webfaction.com/software/python.html?highlight=install%20python%20pacakge#installing-python-packages
If that doesn't help, just contact technical support, their staff is extremely helpful.
eyeD3 is not packaged with a setup.py script, so its installation is a bit more complicated than simply easy_installing it.
Here are the commands you can run in a SSH session to install it in your home directory on your WebFaction server:
mkdir -p ~/tmp ~/src ~/lib/python2.7
export TMPDIR=~/tmp
cd ~/src
wget http://eyed3.nicfit.net/releases/eyeD3-0.6.17.tar.gz
tar zxf eyeD3-0.6.17.tar.gz
cd eyeD3-0.6.17
./configure --prefix=$HOME
cd ..
easy_install-2.7 eyeD3-0.6.17
Hope that helps!
I'm working on a django application on the bitnami djangostack. I want to use the lxml library, but I haven't been able to install it. Or rather, I haven't been able to install it where django can find it.
I've already used apt-get to install the libxml2, libxslt, and python-dev dependencies. Both of these commands report success*:
sudo pip install lxml
sudo apt-get install python-lxml
easy_install fails with a super-long error message that makes me think it can't find the dependencies. (I've run into this problem before.)
When I open up python or call python manage.py shell and try "import lxml", I get
"ImportError: No module named lxml"
As best I can tell, bitnami has set up a virtual environment for django, and pip and aptitude are installing lxml perfectly -- to the wrong python. Assuming that's all correct, how do I get lxml installed to the right one?
When you use apt-get install you are installing system libraries. BitNami DjangoStack is self-contained and independent. You could upgrade or remove your system libraries with apt-get and it would not be affected. Unfortunately lxml is not included in the stack nor libxslt which is a depency. We will include it in a future version however please find below the steps for manualing installing lxml on top of the python version included in BitNami DjangoStack.
You will need to use the system libraries for libxslt and libxml2. Be sure that you have them installed:
sudo apt-get install libxml2 libxml2-dev libxslt1.1 libxslt1-dev
Download lxml and uncompress it:
wget http://lxml.de/files/lxml-2.3.2.tgz
tar zxvf lxml-2.3.2.tgz
cd lxml-2.3.3
Load the BitNami environment:
. path_to_your_djangostack_installation/scripts/setenv.sh <-- notice the space between the dot and the path to the script.
which python <-- the output should be the python version from BitNami.
Install lxml specifying the path to your system libraries (notice that you should execute this command in the lxml directory):
python setup.py install --with-xslt-config=/usr/bin/xslt-config --with-xml2-config=/usr/bin/xml2-config
Now executing import lxml in the python console should work.
(This was already replied here)
There have been a couple of blog postings on installing this library on shared hosting. http://rhodesmill.org/brandon/2009/installing-lxml-on-webfaction/
How to install lxml for python without administative rights on linux?