We're getting "django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library", GeoDjango's common installation issue.
Dev environment is Win10 and GDAL libraries were resolved following this answer on the same question for windows, which supplies filepaths directly. I now can't deploy because prod is Ubuntu 20.04 and that answer doesn't address how to resolve on Ubuntu.
I'm new to Ubuntu / Linux and I think I've installed GDAL with sudo apt install libpq-dev gdal-bin libgdal-dev following these instructions, but Django doesn't know how to find the libraries nor do I. This question mentions "GDAL_LIBRARY_PATH = '/usr/local/lib/libgdal.so.1.15.1'", but my "/usr/local/lib/" only includes the dirs of python3.8 and 3.9.
Also, unsure if relevant but I'm deploying on Dokku, and am unsure if containerizing things precludes Django from finding libs outside of Dokku's fancy ecosystem I don't well understand. This post is the only other post mentioning both Dokku and GDAL, and although it asks something unrelated its requirements.txt includes "GDAL==2.4.1". Could we somehow resolve this by supplying it in a similar way?
What simple thing am I missing or doing wrong?
Figured it out. Install the dokku-apt plugin and place a file named exactly "apt-packages" in the project's root. In that file list the packages you want installed, which in this case was:
libpq-dev
gdal-bin
libgdal-dev
I have been searching the web hours on end for several days and I am unable to install psycopg2 library on my Linux machine (CentOS - 2.6.32-431.3.1.el6.x86_64 GNU/Linux).
I know that the problem is that I am missing the libpq header files since I am getting this message after attempting pip install psycopg2: libpq-fe.h: No such file or directory
http://initd.org/psycopg/docs/install.html#install-from-source
Almost all the articles I found pointed me to use apt-get on CentOS but apt-get is not a standard tool on CentOS 6.3 so I've been trying yum install instead.
However, every time I try to use sudo yum install to download something the package is not available. For example:
yum install postgresql-devel
Loaded plugins: fastestmirror, refresh-packagekit, security
Setting up Install Process
Loading mirror speeds from cached hostfile
drivesrvr | 2.2 kB 00:00
No package postgresql-devel available.
Error: Nothing to do
I've tried this for:
yum install postgresql-server
yum install python-devel
service postgresql initdb
service postgresql start
yum install python-psycopg2
Any ideas? Without the the libpq header files I can't install the psycopg2 module that is necessary for my Python program. This is for Python 2.7.12. And PostgreSQL 9.3.13.
I had this exact issue on Fedora 2016.09 box on Amazon. I was able to install postgresql-devel via yum, but that didn't do the trick; the version seemed to be out of date.
I solved it using:
sudo yum install /usr/include/libpq-fe.h
This installs an updated version of postgresql-devel which allows psycopg2 to compile correctly when installing through pip.
I want to update my django version (1.8 ->1.9).
My current version was installed using pip (pip install Django) .
Now I need to install a newer version without pip.
I think I should run from the django package folder, using the setup.py file:
python setup.py install
[I've installed some other packages using a setup.py file .]
The question - will that create some problems? And, does the setup.py install method take care for uninstalling the older version? Or I have to uninstall manually somehow?
I don't have internet connection in this computer, so no pip.
Download a release (they are available on GitHub), and copy to your computer.
Then use pip install to install the archive. For example
pip install django-1.9.3.tar.gz
For more ways to use pip install, see the docs.
I am a newbie to Django and Python installation. Intermediate with Ubuntu 14.04.
These are my installations so far in my Ubuntu 14.04.
apt-get install python3-setuptools --force-yes -y ## for python3
easy_install3 pip ## for python3
apt-get install python-setuptools --force-yes -y ## for python2.7 or above
easy_install pip ## for python2.7 or above
apt-get install python-dev --force-yes -y ## because ubuntu 14.04 does not have dev version of python 2
apt-get install python3-dev --force-yes -y ## because ubuntu 14.04 does not have dev version of python 3.4
apt-get install links --force-yes -y ##a command line web browser
apt-get install python-flup --force-yes -y ## connects python to uwsgi)
apt-get install build-essential --force-yes -y ##
pip2 install django uwsgi virtualenv ## use pip to install django and uwsgi and virtualenv for python2
pip3 install django uwsgi ## use pip to install django and uwsgi for python3
For the full list, please look at https://gist.github.com/simkimsia/41c55e0c08eda42e2cb3#file-install-sh-L88
I am confused about the use of virtualenv.
I want to prepare my ubuntu 14.04 server edition for a production level of Django 1.7 as much as possible.
The reason why I installed multiple Python environment because I may have other Python apps running which require 3.4.
My Django files are from bitbucket repository and I have git cloned them into /var/virtual/WebApps/DjangoProject
Inside /var/virtual/WebApps/DjangoProject, I have manage.py and other files and folders.
Please advise on how do I get the Django project running for this situation.
I am currently testing this setup on my virtualbox and vagrant.
EDIT
There will be at least 2 Django applications. 1 requires 2.7 python. The other requires 3.4 python.
Let me add that this is a single server that will host the application, frontend, and database.
EDIT 1
I have restarted with a fresh install of Ubuntu 14.04 and I started with Python 2.7.6 and Python 3.4.0.
I then did sudo apt-get install python-virtualenv which I checked its version: 1.11.4.
I have created ~/virtualenvs/py2.7 and ~/virtualenvs/py3.4.
Inside ~/virtualenvs, I did virtualenv -p /usr/bin/python2 py2.7
and ~/virtualenvs, I did virtualenv -p /usr/bin/python3 py3.4
So how do I install python2 only libraries for the python 2 app?
E.g. are django-adminfiles, sorl-thumbnail, psycopg2
EDIT 5
Use virtualenv --system-site-packages -p /usr/bin/python2 py2.7 instead
I have restarted with a fresh install of Ubuntu 14.04 and I started
with Python 2.7.6 and Python 3.4.0.
Okay, so now in your system you have two base versions of Python. Base version just means, the versions that are supported by your operating system; which you have installed globally.
In other words you have installed them using the operating system's package installers and did not compile them separately.
In practice, this above only matters in Linux, because in Windows you cannot install "locally" without going through a few hoops; all Python installers will register themselves in the registry thus making them global, base Python versions.
I then did sudo apt-get install python-virtualenv which I checked its
version: 1.11.4.
This package is outdated (current version is 12.0.7).
Now you have virtual environment installed against the base Python 2 version because the package requires Python 2.
In practical terms it means if you need to upgrade Python 2, you'll have to make sure python-virtualenv is also updated for the base versions of both Pythons that are supported by your operating system. This means, when you apt-get update and apt-get upgrade, virtualenv will be upgraded.
Usually this doesn't matter as its a rare case if python2 is upgraded and then python-virtualenv is not upgraded to match its dependency.
However this is not recommended because you want to control the versions of critical software "manually" to avoid any surprises. There are ways to control this on Ubuntu and other debian-like distributions by pinning versions; but even if you do so, you may not be getting the latest version of the library which may force you later on to uninstall the version that came with your operating system, and reinstall it from source.
I have created ~/virtualenvs/py2.7 and ~/virtualenvs/py3.4.
Inside ~/virtualenvs, I did virtualenv -p /usr/bin/python2 py2.7 and
~/virtualenvs, I did virtualenv -p /usr/bin/python3 py3.4
So how do I install python2 only libraries for the python 2 app?
E.g. are django-adminfiles, sorl-thumbnail, psycopg2
In order for solr-thumbnail and psycopg2 to be installed correctly, you need to build their dependencies; so
sudo apt-get install libjpeg62 libjpeg62-dev zlib1g-dev
sudo apt-get install libgraphicsmagick++-dev libboost-python1.55-dev
sudo apt-get install libexpat1-dev libpython-dev libpython3-dev libssl-dev libpq-dev
To install libraries for the Python 2 app:
Activate the virtual environment; by typing source ~/virtualenvs/py2.7/bin/activate
Type pip install _____ (name of the library)
To support multiple Python applications with different major versions; your system should have both Python major versions installed (you already have done this).
You then install virtualenv for each Python major version. You can skip this step if your applications are fully contained (that is, they include the Python runtime required - but this is a rare case) or if you have a single purpose server.
You should avoid installing anything but the base Python libraries in your system's global python. That is avoid (as root, or using sudo) to pip install things; because these will be installed for all users of Python and may cause problems (on some systems; like Fedora/RedHat - critical system packages like yum rely on the base system Python).
Next step is to make sure you have a suitable build environment available. This means for Debian-sourced systems to install build-essential and further the support libraries for common Python drivers and modules. The exact libraries you need to install will depend on the applications you are planning to host, but at a minimum you should make sure PIL (or Pillow) can be installed and database drivers' support libraries are available. To do so, you can apt-get build-deb python-imaging psycopg2 python-mysqldb (for PostgreSQL, MySQL and PIL).
Now you have a system ready to support most Python applications. You can then optionally add other utilities, but I would try to avoid assuming too much about what applications will require.
To host an app:
Create a virtual environment with the base version of Python required. So virtualenv-2.7 or virtualenv-3 as the normal non-root user account.
Install required packages into the virtual environment.
Adjust the bootstrap script for your application to use the correct Python binary. This is usually done from whatever process you are using to manage your application server. For example, on my server I use supervisord.
That's all you have to do. Everything else will depend on the individual application's requirements and setup (so if you need to serve static files, you'll have to configure that mapping, etc.)
After reading your shell script, it seems you are trying to build a server that will support both the application, the front end and the database.
In order to support such a system; you will need to install the following:
Database server(s) that you would like to support. As this is a single purpose server, you will also need to install database command line clients.
Source code tools (git, etc.)
A global process manager (like supervisor or circus).
Base Python versions you intend to support; and their development headers (sudo apt-get install python-dev)
setuptools, pip, and then virtualenv. These tools should be installed from source rather than your package manager; to ensure the latest versions are installed. You should install these globally (ie, as root) so they are available for all users.
A build tool chain (ie, "Development Tools" or build-essential)
Support libraries for any extensions (but not the extensions themselves). The easiest way to do this is to use the package manager to build the dependencies and not the packages sudo apt-get build-dep python-imaging psycopg2 python-mysqldb.
The next thing you need to do is decide how you will run your application servers (the "django code"). You can use uwsgi, gunicorn, etc. as these are the ones most tested with django.
You need to be able to support multiple versions of these runtimes, so instead of installing them globally across the system; just build their dependencies and install the specific version required for each application in its own isolated environment.
The next thing you need to install is a front end proxy for your applications. You can install whatever proxy suits your needs (nginx is the most popular); but please install from source rather than the packages as those are almost always out of date.
Once all this is setup, the process of hosting a django application is the following:
Create a separate user account with a non-login shell.
Create a virtual environment in this user's home directory. I recommend keeping some standard here, like env for the virtual environments.
Download/copy the source code of your application.
Create a standard directory where you will store the static files. For example I use $HOME/www/static.
Create an entry in your process manager.
Create an entry in your proxy for front end routing.
Reload your proxy server.
Reload your process manager.
You can automate/script a lot of the above. For example, you can create a custom skeleton directory to create the base directories for you when adding new users; and you can create custom templates for other areas using tools like cookiecutter.
Understanding execution path
The first principle to understand is that when you install packages on your operating system (virtual machine, whatever) with sudo is that you are installing all those packages in a global location, that is, a file directory which your system knows about via the $PATH (PATH) environment variable.
This $PATH variable is often set-up by default in your .bashrc or your .bash_profile or your .profile whenever a new linux/unix user is created. Whether you have any of these files depends on what is inside the /etc/skel on your system. /etc/skel holds a "template" of these files that gets duplicated whenever a new user with a home directory is created.
When you type echo $PATH in command line, you would see a list of execution search path delimited by :, for example:
/usr/local/sbin:/usr/local/bin:/usr/bin
This is an example I will use for the purpose of this discussion. Different OS will give you slightly different default $PATH but the simple idea here is that the binaries of your globally installed packages gets thrown into one of these directories and the reason you are able to run these binaries is because these binaries (programs) are now available in the execution search path (simply called $PATH).
Ok great, but what does this mean for the python 2 and python 3 interpreters I installed?
So in relation to your question, this means that when you sudo apt-get or sudo aptitude install python2 or python3 binaries (python interpreters), they are both available and to differentiate which interpeter you are using at any moment, you would run python2.7 or python3.4 that correspondingly calls /usr/bin/python2.7 or /usr/bin/python3.4
You can always check this easily by using the which linux/unix command. which python3.4 will return you the exact path where your python3.4 binary is installed.
Similarly, when type pip2.7 in your command line, you are asking your system to execute the pip2.7 program that came installed with your python2.7 package. And naturally which pip2.7 will reveal to you where this pip2.7 binary has been installed.
All this is possible simply because these binaries are have been installed by you using sudo apt-get and are placed in the directories listed in $PATH. If you move one of these binaries forcibly to another directory not listed in $PATH, you will realize that you can no longer run the binary in your command line without typing out the specific path to the binary.
Additional python specific information
The Python Interpreter has another attribute call $PYTHONPATH. This - as you will rightly deduce - is a variable that holds a list of directories ("search path") where the python interpreter will search for python modules to load. If you want to know where your python interpreter is currently looking for modules (your own python modules or 3rd party python modules), run
python -c "import sys; print(sys.path)"
where python is your specific python interpreter. If /usr/bin/python is symlinked to /usr/bin/python2.7, then you are in fact calling python2.7.
When would python path matter? It matters inside your own .py source code when you ask to import other modules. The import line of code in your .py source code is where you are asking the python interpreter to go forth and search for the module that you want to import. As you can imagine, if your sys.path (in python) is empty, you will not be able to import any 3rd party modules.
Beyond the gory details
Now that we have a clear understanding of the underlying principles behind $PATH and $PYTHONPATH, we can now understand what the virtualenv (and additionally, virtualenvwrapper is useful) does for us.
When we create a new virtualenv giving it a directory, what we are saying is that we want to symlink a specific python interpreter (via -p python2.7 flag for the virtualenv command) for example to that virtualenv.
When we source activate that virtualenv we created, we are in fact invoking shell scripts that come with virtualenv to dynamically modify the $PATH. Try running echo $PATH after you have source activated a virtualenv you created. What do you see?
That's right, you will be seeing something like this:-
/Users/calvin/.virtualenvs/myproject/bin:/usr/local/sbin:/usr/local/bin:/usr/bin
And if you type which python, what do you get?
/Users/calvin/.virtualenvs/myproject/bin/python
That's right, the python interpreter we are using is the one that is inside the virtualenv directory and no longer from the /usr/bin directory.
If you run
ls -la /Users/calvin/.virtualenvs/myproject/bin
what do you see?
Ah! A symlink to the specific global python interpreter you specified when you created this virtualenv. If you don't see a symlink, this means that your entire python interpreter was copied over from the globally installed one.
So this is what our virtualenv tool does. It lets us isolate specific projects and choose which python interpreter to use for a specific project.
Once you have source activated a virtualenv, your pip is also the pip that is located in your virtualenv directory. This pip is python path-aware. When you issue pip install commands, python packages now gets installed into your
/Users/calvin/.virtualenvs/myproject/lib/site-packages
python packages directory; and are available ONLY to your project when you source activate that particular project.
What if you install packages with sudo pip?
sudo pip calls the pip tool that comes with your globally installed python interpreter. When you use sudo pip, you are installing your python packages into a global location (which is not jailed within a virtualenv).
I did not give you specific answers to your how-can-I-make-two-projects-use-two-different-python-interpreters question. I explained the principles and now that you know the principles, it is clear what you need to do. Cheers!
:D
you should have to try following steps to make your django work with python-2.7.
As Ubuntu-14 already come with Python-2.7 installed. so no need to install python-2.7.
First of all install python setuptools for python-2.7:
https://pypi.python.org/packages/source/s/setuptools/setuptools-12.0.5.tar.gz
tar -zxvf setuptools-12.0.5.tar.gz
cd setuptools-12.0.5/
sudo python2.7 setup.py install
after install setuptools install pip for python-2.7
wget https://pypi.python.org/packages/source/p/pip/pip-6.0.8.tar.gz
tar -zxvf pip-6.0.8.tar.gz
cd pip-6.0.8/
sudu python2.7 setup.py install
after installing pip now we have to go for install python virtual environment:
sudo pip2.7 install virtualenv
then to folder where you want to create new virtualenv using python 2.7
virtualenv-2.7 "name for new virtualenv for example '/var/virtual/DjangoProject'"
cd /var/virtual/
virtualenv-2.7 DjangoProject
Note just created virtualenv name is "DjangoProject"
Now time to activate new virtualenv
cd DjangoProject
source bib/activate
Now time to clone your project here.
git clone "your git url"
install rest of apps and django here. if you have created simply install by requirement.txt
cd "projectFolder"
pip install -r requirement.txt
after doing all these your new python2.7 django with virtualenv are ready to use. just run django wsgi server for test is every thing working:
python manage.py runserver.
Now you can use this virtualenv and project to setup for web server.
Hope this will help-full to you.
You should use pyenv (and pyenv virtualenvwrapper). With these tools you can have multiple Python versions and switch between them.
Here is how you setup one environment with Python 3.4.2 and one with Python 2.7.8 (and each environment can have its own Python packages)
# install pyenv
git clone https://github.com/yyuu/pyenv.git ~/.pyenv
# setup pyenv and load when shell is loaded
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc
# install pyenv virtualenvwrapper
git clone https://github.com/yyuu/pyenv-virtualenvwrapper.git ~/.pyenv/plugins/pyenv-virtualenvwrapper
# setup pyenv virtualenvwrapper and load when shell is loaded
echo 'export PYENV_VIRTUALENVWRAPPER_PREFER_PYVENV="true"' >> ~/.bashrc
echo "pyenv virtualenvwrapper" >> ~/.bashrc
Now everything is setup. Restart your shell.
# install python 2.7.8
pyenv install 2.7.8
pyenv rehash
# install python 3.4.2
pyenv install 3.4.2
pyenv rehash
# switch the global python version to 2.7.8
pyenv global 2.7.8
# create a virtual environment using python 2.7.8
mkvirtualenv project278
# -- now you can install you python 2 modules
# leave the virtual environment
deactivate
# switch the global python to 3.4.2
pyenv global 3.4.2
# create a virtual environment using python 3.4.2
mkvirtualenv project342
# -- now you can install you python 2 modules
# leave the virtual environment
deactivate
# switch the global python to the system python.
pyenv global system
# swith to environment project278 and check the python version
workon project278
python --version
# swith to environment project342 and check the python version
workon project342
python --version
Voilà! You now have two environments with different Python versions. Just use the workon (and the deactivate) commands to switch between them.
Make sure you read the documentation of the used packages, so you understand what is going on!
I want to upgrade Django from 1.2.5 to 1.3. I uninstalled 1.2.5 version, by the Ubuntu Software Center, for future upgrading, but it still in dist-packages and it still imported in python shell. What should I do? Would it be normal if I would brutally deleted the folder and egg from dist-packages?
It runs on Ubuntu 10.04
I recommend using setup-tools. Then run
easy_install --upgrade django
It will remove current django path from PYTHON_PATH and will add it's own path. To get easy_install do:
apt-get install python-setuptools
For both actions sudo is needed.
Go to the /usr/local/lib/python2.6/dist-packages. Find django folder there and delete it. then download django 1.3 and run python setup.py install. This should do the trick.
Remove django version using pip:
pip uninstall django
And install the version you want, for example 1.5:
pip install django==1.5