I installed GDAL and necessary libraries by following this link: https://github.com/vancityhuddy/aws-eb-gdal-example
Had to use aws s3 instead of s3cmd tho, and removed PYTHONPATH from environment variables since EB's configuration already comes with PYTHONPATH. Anyways, here is a copy of my .config file.
container_commands:
01_install_geos:
command: |
sudo wget ***RemovedTheURLForSecurity***/geos-3.6.4.tar.gz
sudo mkdir -p /usr/local/geos
sudo tar -xvf geos-3.6.4.tar.gz -C /usr/local/geos
sudo rm -f geos-3.6.4.tar.gz
test: "[ ! -d /usr/local/geos ]"
02_install_proj4:
command: |
sudo wget ***RemovedTheURLForSecurity***/proj4-5.2.0.tar.gz
sudo mkdir -p /usr/local/proj4
sudo tar -xvf proj4-5.2.0.tar.gz -C /usr/local/proj4
sudo rm -f proj4-5.2.0.tar.gz
test: "[ ! -d /usr/local/proj4 ]"
03_install_gdal:
command: |
sudo wget ***RemovedTheURLForSecurity***/gdal-2.4.4.tar.gz
sudo mkdir -p /usr/local/gdal
sudo tar -xvf gdal-2.4.4.tar.gz -C /usr/local/gdal
sudo rm -f gdal-2.4.4.tar.gz
test: "[ ! -d /usr/local/gdal ]"
04_migrate:
command: "source /var/app/venv/*/bin/activate && python3 manage.py makemigrations && python3 manage.py migrate"
leader_only: true
05_createsu:
command: "source /var/app/venv/*/bin/activate && python3 manage.py createsu"
leader_only: true
option_settings:
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: core.settings
PATH: /usr/local/gdal/bin:$PATH
LD_LIBRARY_PATH: /usr/local/proj4/lib:/usr/local/geos/lib:/usr/local/gdal/lib:$LD_LIBRARY_PATH
GDAL_LIBRARY_PATH: /usr/local/gdal/lib/libgdal.so
But still, this error occurs even thought the geos, proj4 and gdal commands completed successfully,
django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal3.1.0", "gdal3.0.0", "gdal2.4.0", "gdal2.3.0", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.
When I add GDAL_LIBRARY_PATH in settings.py according to the message. This is what it shows:
OSError: /usr/local/gdal/lib/libgdal.so: cannot open shared object file: No such file or directory
I'm clueless here. I have been trying to deploy this for two days. Any help would be appreciated.
Related
what are the correct steps to install geodjango on elastic beanstalk?
got eb instance, installed env and made it two instances now I want to use geodjango on it, I'm already using it on a separate ec2 instance for testing
that's my django.config file and it fails
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: hike.project.wsgi:application
commands:
01_gdal:
command: "wget http://download.osgeo.org/gdal/2.1.3/gdal-2.1.3.tar.gz && tar -xzf gdal-2.1.3.tar.gz && cd gdal-2.1.3 && ./configure && make && make install"
then tried this instead and also failed from 100% cpu and time limit
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: hike.project.wsgi:application
commands:
01_install_gdal:
test: "[ ! -d /usr/local/gdal ]"
command: "/tmp/gdal_install.sh"
files:
"/tmp/gdal_install.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo yum-config-manager --enable epel
sudo yum -y install make automake gcc gcc-c++ libcurl-devel proj-devel geos-devel
# Geos
cd /
sudo mkdir -p /usr/local/geos
cd usr/local/geos/geos-3.7.2
sudo wget geos-3.7.2.tar.bz2 http://download.osgeo.org/geos/geos-3.7.2.tar.bz2
sudo tar -xvf geos-3.7.2.tar.bz2
cd geos-3.7.2
sudo ./configure
sudo make
sudo make install
sudo ldconfig
# Proj4
cd /
sudo mkdir -p /usr/local/proj
cd usr/local/proj
sudo wget -O proj-5.2.0.tar.gz http://download.osgeo.org/proj/proj-5.2.0.tar.gz
sudo wget -O proj-datumgrid-1.8.tar.gz http://download.osgeo.org/proj/proj-datumgrid-1.8.tar.gz
sudo tar xvf proj-5.2.0.tar.gz
sudo tar xvf proj-datumgrid-1.8.tar.gz
cd proj-5.2.0
sudo ./configure
sudo make
sudo make install
sudo ldconfig
# GDAL
cd /
sudo mkdir -p /usr/local/gdal
cd usr/local/gdal
sudo wget -O gdal-2.4.4.tar.gz http://download.osgeo.org/gdal/2.4.4/gdal-2.4.4.tar.gz
sudo tar xvf gdal-2.4.4.tar.gz
cd gdal-2.4.4
sudo ./configure
sudo make
sudo make install
sudo ldconfig
and I've no idea what to do,
Rather than necro-post on a two-year old thread, I decided to create a new question.
I want add brew (homebrew) to a Docker container, but I get a brew: not found error.
The suggested solution in that previous article doesn't seem to work. This new Dockerfile...
FROM rust:1.63.0-buster
WORKDIR app
RUN apt-get update && \
apt-get install -y -q --allow-unauthenticated \
git \
sudo
RUN useradd -m -s /bin/zsh linuxbrew && \
usermod -aG sudo linuxbrew && \
mkdir -p /home/linuxbrew/.linuxbrew && \
chown -R linuxbrew: /home/linuxbrew/.linuxbrew
USER linuxbrew
RUN /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
USER root
RUN chown -R $CONTAINER_USER: /home/linuxbrew/.linuxbrew
RUN brew install hello
gives this error... What am I missing? Thanks.
=> ERROR [6/6] RUN brew install hello 0.2s
------
> [6/6] RUN brew install hello:
#9 0.181 /bin/sh: 1: brew: not found
------
executor failed running [/bin/sh -c brew install hello]: exit code: 127
This Dockerfile installs brew in /home/linuxbrew/.linuxbrew/bin/brew. Including that directory in the path (with the ENV command) does the trick.
...
ENV PATH="/home/linuxbrew/.linuxbrew/bin:${PATH}"
RUN brew install hello
I have installed cmake3 through yum on my AWS Elastic Beanstalk instance:
packages:
yum:
cmake3: []
However when running a pip install that requires cmake, I'm getting an error:
FileNotFoundError: [Errno 2] No such file or directory: 'cmake': 'cmake'
How can I alias cmake3 for cmake? I have tried the following and it is resulting in the same error message.
container_commands:
01_link_cmake:
command: "ln -s /usr/bin/cmake3 /usr/bin/cmake"
The reason I do not install cmake: [] is because I need version 3.
I have solved it. I realized that the pip package install happened before the ln -s container command. So even if my alias was happening, it was happening too late in the process.
Instead, what I needed to do was
container_commands:
01_link_cmake:
command: ln -s /usr/bin/cmake3 /usr/bin/cmake
02_upgrade_pip:
command: "python -m pip install --upgrade pip"
leader_only: true
03_install_package:
command: "python -m pip install <package that requires cmake>"
leader_only: true
I'm trying to use the pyarrow Filesystem interface with HDFS. I receive a libhdfs.so not found error when calling the fs.HadoopFileSystem constructor even though libhdfs.so is apparently at the indicated location.
from pyarrow import fs
hfs = fs.HadoopFileSystem(host="10.10.0.167", port=9870)
OSError: Unable to load libhdfs: /hadoop-3.3.1/lib/native/libhdfs.so: cannot open shared object file: No such file or directory
I've tried different python and pyarrow versions and setting ARROW_LIBHDFS_DIR. For testing, I'm using the following dockerfile on linuxmint.
FROM openjdk:11
RUN apt-get update &&\
apt-get install wget -y
RUN wget -nv https://dlcdn.apache.org/hadoop/common/hadoop-3.3.1/hadoop-3.3.1-aarch64.tar.gz &&\
tar -xf hadoop-3.3.1-aarch64.tar.gz
ENV PATH=/miniconda/bin:${PATH}
RUN wget -nv https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh &&\
bash miniconda.sh -b -p /miniconda &&\
conda init
RUN conda install -c conda-forge python=3.9.6
RUN conda install -c conda-forge pyarrow=4.0.1
ENV JAVA_HOME=/usr/local/openjdk-11
ENV HADOOP_HOME=/hadoop-3.3.1
RUN printf 'from pyarrow import fs\nhfs = fs.HadoopFileSystem(host="10.10.0.167", port=9870)\n' > test_arrow.py
# 'python test_arrow.py' fails with ...
# OSError: Unable to load libhdfs: /hadoop-3.3.1/lib/native/libhdfs.so: cannot open shared object file: No such file or directory
RUN python test_arrow.py || true
CMD ["/bin/bash"]
I have created a docker file for the pyarrow fs hadoopfilesystem client.
HDFS needs to be installed to use the libhdfs.so file.
RUN mkdir -p /data/hadoop
RUN apt-get -q update
RUN apt-get install software-properties-common -y
RUN add-apt-repository "deb http://deb.debian.org/debian/ sid main"
RUN apt-get -q update
RUN apt-get install openjdk-8-jdk -y
RUN apt-get clean
RUN rm -rf /var/lib/apt/lists/*
RUN wget "https://dlcdn.apache.org/hadoop/common/hadoop-3.3.2/hadoop-3.3.2.tar.gz" -O hadoop-3.3.2.tar.gz
RUN tar xzf hadoop-3.3.2.tar.gz
ENV HADOOP_HOME=/app/hadoop-3.3.2
ENV HADOOP_INSTALL=$HADOOP_HOME
ENV HADOOP_MAPRED_HOME=$HADOOP_HOME
ENV HADOOP_COMMON_HOME=$HADOOP_HOME
ENV HADOOP_HDFS_HOME=$HADOOP_HOME
ENV YARN_HOME=$HADOOP_HOME
ENV HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
ENV PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
ENV HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/nativ"
ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
ENV CLASSPATH="$HADOOP_HOME/bin/hadoop classpath --glob"
ENV ARROW_LIBHDFS_DIR=$HADOOP_HOME/lib/native
ADD pyarrow-app.py /app/
CMD [ "python3" "-u" "/app/pyarrow-app.py.py"]
I'm trying to use shadowsocksR in my ubuntu 16 .
sudo apt-get install git
git clone -b manyuser https://github.com/shadowsocksr-backup/shadowsocksr.git
cd shadowsocksr/shadowsocks
python local.py -s server_ip -p server_port -k password -m chacha20-ietf -o tls1.2_ticket_auth
then he said " libsodium not found" so I install libsodium
download the LATEST.tar.gz from https://download.libsodium.org/libsodium/releases/
tar -zxf LATEST.tar.gz
cd libsodium-stable
./configure
make && make check
sudo make install
```
but this time
python local.py -s server_ip -p server_port -k password -m chacha20-ietf -o tls1.2_ticket_auth
He said OSError: libsodium.so.23: cannot open shared object file: No such file or directory
thankyou for your help
You need to install libsodium-dev
Try running "apt-get install libsodium-dev"
If you've compiled libsodium, make sure libsodium is in the linker cache:
ldconfig -p | grep libsodium
If there is no output, chances are that the linker cache is not updated in time.
You can update the cache by:
sudo ldconfig