I have installed cmake3 through yum on my AWS Elastic Beanstalk instance:
packages:
yum:
cmake3: []
However when running a pip install that requires cmake, I'm getting an error:
FileNotFoundError: [Errno 2] No such file or directory: 'cmake': 'cmake'
How can I alias cmake3 for cmake? I have tried the following and it is resulting in the same error message.
container_commands:
01_link_cmake:
command: "ln -s /usr/bin/cmake3 /usr/bin/cmake"
The reason I do not install cmake: [] is because I need version 3.
I have solved it. I realized that the pip package install happened before the ln -s container command. So even if my alias was happening, it was happening too late in the process.
Instead, what I needed to do was
container_commands:
01_link_cmake:
command: ln -s /usr/bin/cmake3 /usr/bin/cmake
02_upgrade_pip:
command: "python -m pip install --upgrade pip"
leader_only: true
03_install_package:
command: "python -m pip install <package that requires cmake>"
leader_only: true
Related
I installed GDAL and necessary libraries by following this link: https://github.com/vancityhuddy/aws-eb-gdal-example
Had to use aws s3 instead of s3cmd tho, and removed PYTHONPATH from environment variables since EB's configuration already comes with PYTHONPATH. Anyways, here is a copy of my .config file.
container_commands:
01_install_geos:
command: |
sudo wget ***RemovedTheURLForSecurity***/geos-3.6.4.tar.gz
sudo mkdir -p /usr/local/geos
sudo tar -xvf geos-3.6.4.tar.gz -C /usr/local/geos
sudo rm -f geos-3.6.4.tar.gz
test: "[ ! -d /usr/local/geos ]"
02_install_proj4:
command: |
sudo wget ***RemovedTheURLForSecurity***/proj4-5.2.0.tar.gz
sudo mkdir -p /usr/local/proj4
sudo tar -xvf proj4-5.2.0.tar.gz -C /usr/local/proj4
sudo rm -f proj4-5.2.0.tar.gz
test: "[ ! -d /usr/local/proj4 ]"
03_install_gdal:
command: |
sudo wget ***RemovedTheURLForSecurity***/gdal-2.4.4.tar.gz
sudo mkdir -p /usr/local/gdal
sudo tar -xvf gdal-2.4.4.tar.gz -C /usr/local/gdal
sudo rm -f gdal-2.4.4.tar.gz
test: "[ ! -d /usr/local/gdal ]"
04_migrate:
command: "source /var/app/venv/*/bin/activate && python3 manage.py makemigrations && python3 manage.py migrate"
leader_only: true
05_createsu:
command: "source /var/app/venv/*/bin/activate && python3 manage.py createsu"
leader_only: true
option_settings:
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: core.settings
PATH: /usr/local/gdal/bin:$PATH
LD_LIBRARY_PATH: /usr/local/proj4/lib:/usr/local/geos/lib:/usr/local/gdal/lib:$LD_LIBRARY_PATH
GDAL_LIBRARY_PATH: /usr/local/gdal/lib/libgdal.so
But still, this error occurs even thought the geos, proj4 and gdal commands completed successfully,
django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal3.1.0", "gdal3.0.0", "gdal2.4.0", "gdal2.3.0", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.
When I add GDAL_LIBRARY_PATH in settings.py according to the message. This is what it shows:
OSError: /usr/local/gdal/lib/libgdal.so: cannot open shared object file: No such file or directory
I'm clueless here. I have been trying to deploy this for two days. Any help would be appreciated.
So I'm trying to build a new docker image with Python2.7 and pip for python 2.7 however I'm getting a "The command '/bin/sh -c pip2 install -r requirements.txt' returned a non-zero code: 1" error when trying to build the image.
FROM colstrom/python:legacy
MAINTAINER **REDACTED**
RUN pip2 install -r requirements.txt
CMD ["python2.7", "parser.py"]
Any ideas?
You have to COPY/ADD or mount your data (at least requirements.txt and parser.py) into the container.
Assuming your Dockerfile resides at the root directory of your project:
FROM colstrom/python:legacy
MAINTAINER **REDACTED**
COPY . .
RUN pip2 install -r requirements.txt
CMD ["python2.7", "parser.py"]
I'm trying to deploy a geodjango application on AWS Elastic Beanstalk. The configuration is 64bit Amazon Linux 2017.09 v2.6.6 running Python 3.6. I am getting this error when trying to deploy:
Requires: libpoppler.so.5()(64bit) Error: Package: gdal-java-1.9.2-8.rhel6.x86_64 (pgdg93) Requires: libpoppler.so.5()(64bit)
How do I install the required package? I read through Setting up Django with GeoDjango Support in AWS Beanstalk or EC2 Instance but I am still getting problems. My ebextensions currently looks like:
commands:
01_yum_update:
command: sudo yum -y update
02_epel_repo:
command: sudo yum-config-manager -y --enable epel
03_install_gdal_packages:
command: sudo yum -y install gdal gdal-devel
packages:
yum:
git: []
postgresql95-devel: []
gettext: []
libjpeg-turbo-devel: []
libffi-devel: []
I'm going to answer my own question for the sake my future projects and anyone else trying to get started with geodjango. Updating this answer as of July 2020
Create an ebextensions file to install GDAL on the EC2 instance at deployment:
01_gdal.config
commands:
01_install_gdal:
test: "[ ! -d /usr/local/gdal ]"
command: "/tmp/gdal_install.sh"
files:
"/tmp/gdal_install.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo yum-config-manager --enable epel
sudo yum -y install make automake gcc gcc-c++ libcurl-devel proj-devel geos-devel
# Geos
cd /
sudo mkdir -p /usr/local/geos
cd usr/local/geos/geos-3.7.2
sudo wget geos-3.7.2.tar.bz2 http://download.osgeo.org/geos/geos-3.7.2.tar.bz2
sudo tar -xvf geos-3.7.2.tar.bz2
cd geos-3.7.2
sudo ./configure
sudo make
sudo make install
sudo ldconfig
# Proj4
cd /
sudo mkdir -p /usr/local/proj
cd usr/local/proj
sudo wget -O proj-5.2.0.tar.gz http://download.osgeo.org/proj/proj-5.2.0.tar.gz
sudo wget -O proj-datumgrid-1.8.tar.gz http://download.osgeo.org/proj/proj-datumgrid-1.8.tar.gz
sudo tar xvf proj-5.2.0.tar.gz
sudo tar xvf proj-datumgrid-1.8.tar.gz
cd proj-5.2.0
sudo ./configure
sudo make
sudo make install
sudo ldconfig
# GDAL
cd /
sudo mkdir -p /usr/local/gdal
cd usr/local/gdal
sudo wget -O gdal-2.4.4.tar.gz http://download.osgeo.org/gdal/2.4.4/gdal-2.4.4.tar.gz
sudo tar xvf gdal-2.4.4.tar.gz
cd gdal-2.4.4
sudo ./configure
sudo make
sudo make install
sudo ldconfig
As shown, the script checks whether gdal already exists using the test function. It then downloads the Geos, Proj, and GDAL libraries and installs them in the usr/local directory. At the time of writing this, geodjango (Django 3.0) supports up to Geos 3.7, Proj 5.2 (which also requires projdatum. Current releases do not require it), and GDAL 2.4 Warning: this installation process can take a long time. Also I am not a Linux professional so some of those commands may be redundant, but it works.
Lastly I add the following two environment variables to my Elastic Beanstalk configuration:
LD_LIBRARY_PATH: /usr/local/lib:$LD_LIBRARY_PATH
PROJ_LIB: usr/local/proj
If you still have troubles I recommend checking the logs and ssh-ing in the EC2 instance to check that installation took place. Original credit to this post
I am currently using Tika to extract text from files uploaded to my Rails app running on AWS Elastic Beanstalk (64bit Amazon Linux 2016.03 v2.1.2 running Ruby 2.2). I'd like to index scanned images as well, so I need to install Tesseract.
I was able to get it to work by installing it from source like so, but it added 10 minutes to my deploys to a fresh instance. Is there a faster way to do this?
.ebextensions/02-tesseract.config
packages:
yum:
autoconf: []
automake: []
libtool: []
libpng-devel: []
libtiff-devel: []
zlib-devel: []
container_commands:
01-command:
command: mkdir -p install
cwd: /home/ec2-user
02-command:
command: cp .ebextensions/scripts/install_tesseract.sh /home/ec2-user/install/
03-command:
command: bash install/install_tesseract.sh
cwd: /home/ec2-user
.ebextensions/scripts/install_tesseract.sh
#!/usr/bin/env bash
cd_to_install () {
cd /home/ec2-user/install
}
cd_to () {
cd /home/ec2-user/install/$1
}
if ! [ -x "$(command -v tesseract)" ]; then
# Add `usr/local/bin` to PATH
echo 'pathmunge /usr/local/bin' > /etc/profile.d/usr_local.sh
chmod +x /etc/profile.d/usr_local.sh
# Install leptonica
cd_to_install
wget http://www.leptonica.org/source/leptonica-1.73.tar.gz
tar -zxvf leptonica-1.73.tar.gz
cd_to leptonica-1.73
./configure
make
make install
rm -rf /home/ec2-user/install/leptonica-1.73.tar.gz
rm -rf /home/ec2-user/install/leptonica-1.73
# Install tesseract ~ the jewel of Odin's treasure room
cd_to_install
wget https://github.com/tesseract-ocr/tesseract/archive/3.04.01.tar.gz
tar -zxvf 3.04.01.tar.gz
cd_to tesseract-3.04.01
./autogen.sh
./configure
make
make install
ldconfig
rm -rf /home/ec2-user/install/3.04.01.tar.gz
rm -rf /home/ec2-user/install/tesseract-3.04.01
# Install tessdata
cd_to_install
wget https://github.com/tesseract-ocr/tessdata/archive/3.04.00.tar.gz
tar -zxvf 3.04.00.tar.gz
cp /home/ec2-user/install/tessdata-3.04.00/eng.* /usr/local/share/tessdata/
rm -rf /home/ec2-user/install/3.04.00.tar.gz
rm -rf /home/ec2-user/install/tessdata-3.04.00
fi
Short answer
.ebextensions/02-tesseract.config
commands:
01-libwebp:
command: "yum --enablerepo=epel --disablerepo=amzn-main -y install libwebp"
02-tesseract:
command: "yum --enablerepo=epel -y install tesseract"
Long answer
I'm not familiar with non-Ubuntu package managers or ebextensions, so after some digging, I found that there are precompiled binaries that can be installed on Amazon Linux in the stable EPEL repo.
The first obstacle was figuring out how to use the EPEL repo. The easiest way is to use the enablerepo option on the yum command.
That gets us here:
yum --enablerepo=epel install tesseract
Next, I had to resolve this dependency error:
[root#ip-10-0-1-193 ec2-user]# yum install --enablerepo=epel tesseract
Loaded plugins: priorities, update-motd, upgrade-helper
951 packages excluded due to repository priority protections
Resolving Dependencies
--> Running transaction check
---> Package tesseract.x86_64 0:3.04.00-3.el6 will be installed
--> Processing Dependency: liblept.so.4()(64bit) for package: tesseract-3.04.00-3.el6.x86_64
--> Running transaction check
---> Package leptonica.x86_64 0:1.72-2.el6 will be installed
--> Processing Dependency: libwebp.so.5()(64bit) for package: leptonica-1.72-2.el6.x86_64
--> Finished Dependency Resolution
Error: Package: leptonica-1.72-2.el6.x86_64 (epel)
Requires: libwebp.so.5()(64bit)
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
I found the solution here
Just adding the epel repo doesn't solve it, as the packages in the
amzn-main repository seem to overrule those in the epel repository. If
the libwebp package in the amzn-main repo are excluded it should work
The Tesseract install has some dependencies found in the amzn-main repo. This is why I first install libwebp with --disablerepo=amzn-main.
yum --enablerepo=epel --disablerepo=amzn-main install libwebp
yum --enablerepo=epel install tesseract
Finally, here's how you can install yum packages on Elastic Beanstalk with options:
.ebextensions/02-tesseract.config
commands:
01-libwebp:
command: "yum --enablerepo=epel --disablerepo=amzn-main -y install libwebp"
02-tesseract:
command: "yum --enablerepo=epel -y install tesseract"
Fortunately, this is also the easiest way to install Tesseract on Elastic Beanstalk!
I'm trying to setup a Django app on Docker with Nginx, uWSGI and Postgres. I found this great guide on setting up Compose for Django and Postgres: https://docs.docker.com/v1.5/compose/django/
However, now I need to add Nginx and uWSGI. I've tried using files of this repo (https://github.com/baxeico/django-uwsgi-nginx) with the Compose setup of the Docker docs but without succes, sadly.
This is what happens when I enter docker-compose run web:
Step 17 : RUN pip install -r /home/docker/code/app/requirements.txt
---> Running in e1ec89e80d9c
Collecting Django==1.9.1 (from -r /home/docker/code/app/requirements.txt (line 1))
/usr/local/lib/python2.7/dist-packages/pip-7.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading Django-1.9.1-py2.py3-none-any.whl (6.6MB)
Collecting psycopg2 (from -r /home/docker/code/app/requirements.txt (line 2))
Downloading psycopg2-2.6.1.tar.gz (371kB)
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-XRgbSA/psycopg2
ERROR: Service 'web' failed to build: The command '/bin/sh -c pip install -r /home/docker/code/app/requirements.txt' returned a non-zero code: 1
This is my Dockerfile:
from ubuntu:precise
run echo "deb http://us.archive.ubuntu.com/ubuntu/ precise-updates main restricted" | tee -a /etc/apt/sources.list.d/precise-updates.list
# update packages
run apt-get update
# install required packages
run apt-get install -y python python-dev python-setuptools python-software-properties
run apt-get install -y sqlite3
run apt-get install -y supervisor
# add nginx stable ppa
run add-apt-repository -y ppa:nginx/stable
# update packages after adding nginx repository
run apt-get update
# install latest stable nginx
run apt-get install -y nginx
# install pip
run easy_install pip
# install uwsgi now because it takes a little while
run pip install uwsgi
# install our code
add . /home/docker/code/
# setup all the configfiles
run echo "daemon off;" >> /etc/nginx/nginx.conf
run rm /etc/nginx/sites-enabled/default
run ln -s /home/docker/code/nginx-app.conf /etc/nginx/sites-enabled/
run ln -s /home/docker/code/supervisor-app.conf /etc/supervisor/conf.d/
# run pip install
run pip install -r /home/docker/code/app/requirements.txt
run cd /home/docker/code/app && ./manage.py syncdb --noinput
expose 80
cmd ["supervisord", "-n"]
And the docker-compose.yml:
db:
image: postgres
web:
build: .
command: python vms/manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db
There are also files named nginx-app.conf, supervisor-app.conf, uwsgi_params and uwsgi.ini. These are all from the aforementioned repo. Requirements.txt contains Django 1.9.1, psycopg2 and requests.
If there is a better alternative to this Frankenstein project, I'd love to hear it.
On Ubuntu, make sure that python-dev and libpq-dev have been installed using apt-get, before trying to install psycopg2 using pip.
See the installation docs for more info.