I need to install and configure postgresql and django in one container, I can't use docker-compose because the hosting doesn't have support for docker-compose.
i try of following in dockerfile
FROM ubuntu:16.04
RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" > /etc/apt/sources.list.d/pgdg.list
RUN apt-get update && apt-get install -y python-software-properties software-properties-common postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3
USER postgres
RUN /etc/init.d/postgresql start &&\
psql --command "CREATE USER docker WITH SUPERUSER PASSWORD 'postgres';" &&\
createdb -O docker docker
RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/9.3/main/pg_hba.conf
RUN echo "listen_addresses='*'" >> /etc/postgresql/9.3/main/postgresql.conf
# Expose the PostgreSQL port
EXPOSE 5432
# Add VOLUMEs to allow backup of config, logs and databases
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
CMD ["/usr/lib/postgresql/9.3/bin/postgres", "-D", "/var/lib/postgresql/9.3/main", "-c", "config_file=/etc/postgresql/9.3/main/postgresql.conf"]
FROM python
ENV PYTHONUNBUFFERED 1
RUN mkdir /app
WORKDIR /app/roche
COPY requirements.txt /app/roche
RUN pip install -r requirements.txt
COPY . /app/roche
CMD ["python3", "manage.py", "runserver", "0.0.0.0:8000"]
but I get this error
This is my configure postgres in settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'postgres',
'USER': 'docker',
'PASSWORD': 'postgres',
'HOST': 'localhost',
'PORT': 5432,
}
}
this is the link of the answer why I can't use more than one container in plesk
https://talk.plesk.com/threads/networking-between-docker-containers.353312/
thanks to all who read and respond
Related
I'm very new for docker, now I am trying to run django with mariadb in docker through docker-compose, but I always get this error:
I use Docker version 17.09.1-ce, build 19e2cf6, docker-compose version 1.18.0, build 8dd22a9
django.db.utils.OperationalError: (2003, 'Can\'t connect to MySQL
server on \'mariadb55\' (111 "Connection refused")')
I can connect db correctly after run docker-compose up db in local or remote, and I even can run python manage.py runserver 0.0.0.0:6001 correctly in anaconda virtual environment to connect db service in docker by setting parameters of settings.py file like below:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'belter',
# 'HOST': 'mariadb55',
'HOST': '127.0.0.1',
'PORT': '3302',
'PASSWORD': 'belter_2017',
'default-character-set': 'utf8',
'OPTIONS': {
'sql_mode': 'traditional',
}
}
}
This is my docker-compose.yml file
version: '3'
services:
db:
image: mariadb:5.5
restart: always
environment:
- MYSQL_HOST=localhost
- MYSQL_PORT=3306
- MYSQL_ROOT_HOST=%
- MYSQL_DATABASE=test
- MYSQL_USER=belter
- MYSQL_PASSWORD=belter_2017
- MYSQL_ROOT_PASSWORD=123456_abc
volumes:
- /home/belter/mdbdata/mdb55:/var/lib/mysql
ports:
- "3302:3306"
web:
image: onlybelter/django_py35
command: python3 manage.py runserver 0.0.0.0:6001
volumes:
- /mnt/data/www/mysite:/djcode
ports:
- "6001:6001"
depends_on:
- db
links:
- db:mariadb55
I almost tried everything I can find, but still cannot figure it out, any help would be nice!
What I have tried:
Docker compose mysql connection failing
Linking django and mysql containers using docker-compose
Django connection to postgres by docker-compose
Finally, I figured it out!
The key point is, just as #SangminKim said, I need to use 3306 not 3302 in settings.py, and use db as HOST not 127.0.0.1.
So this is my docker-compose.yml file now:
version: '3'
services:
db:
image: mariadb:5.5
restart: always
environment:
- MYSQL_HOST=localhost
- MYSQL_PORT=3306 # cannot change this port to other number
- MYSQL_ROOT_HOST=%
- MYSQL_DATABASE=test
- MYSQL_USER=belter
- MYSQL_PASSWORD=belter_2017
- MYSQL_ROOT_PASSWORD=123456_abc
volumes:
- /home/belter/mdbdata/mdb55:/var/lib/mysql
ports:
- "3302:3306"
web:
image: onlybelter/django_py35
command: python3 manage.py runserver 0.0.0.0:6001
volumes:
- .:/djcode
ports:
- "6001:6001"
depends_on:
- db
So now we can connect this docker-mysql by mysql -h 127.0.0.1 -P 3302 -u root -p in shell directly, but we have to use db and 3306 in django settings.py file:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'belter',
# 'HOST': 'mariadb55',
'HOST': 'db', #<---
'PORT': '3306', #<---
'PASSWORD': 'belter_2017',
'default-character-set': 'utf8',
'OPTIONS': {
'sql_mode': 'traditional',
}
}
}
And we can still check if this port is open, by running extra command in docker-compose.yml file:
...
web:
image: onlybelter/django_py35
command: /bin/sh -c "python check_db.py --service-name mysql --ip db --port 3306"
volumes:
- .:/djcode
...
Here is check_db.py file:
# check_db.py
import socket
import time
import argparse
""" Check if port is open, avoid docker-compose race condition """
parser = argparse.ArgumentParser(description='Check if port is open, avoid\
docker-compose race condition')
parser.add_argument('--service-name', required=True)
parser.add_argument('--ip', required=True)
parser.add_argument('--port', required=True)
args = parser.parse_args()
# Get arguments
service_name = str(args.service_name)
port = int(args.port)
ip = str(args.ip)
# Infinite loop
while True:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = sock.connect_ex((ip, port))
if result == 0:
print("{0} port is open! Bye!".format(service_name))
break
else:
print("{0} port is not open! I'll check it soon!".format(service_name))
time.sleep(3)
By the way, this is my Dockerfile for build django-py35:
FROM python:3.5-alpine
MAINTAINER Xin Xiong "xiongxin20008#126.com"
ENV PYTHONUNBUFFERED 1
RUN set -e; \
apk add --no-cache --virtual .build-deps \
gcc \
libc-dev \
linux-headers \
mariadb-dev \
python3-dev \
postgresql-dev \
freetype-dev \
libpng-dev \
g++ \
;
RUN mkdir /djcode
WORKDIR /djcode
ENV REFRESHED_AT 2017-12-25
ADD requirements.txt /djcode/
RUN pip install --no-cache-dir -r /djcode/requirements.txt
RUN pip install uwsgi
ADD . /djcode/ # copy . to /djcode/
EXPOSE 6001
See more details from here: https://github.com/OnlyBelter/django-compose
You should use the container name instead of localhost (or 127.0.0.1) in your settings.py file. Try providing a container name to the db service in the docker-compose.yml file using container_name attribute and replace the host name in the settings.py by the value of the container_name. (Make sure that they are in the same network that docker compose creates for you.)
Build container with this:
docker run --name mysql-latest \
-p 3306:3306 -p 33060:33060 \
-e MYSQL_ROOT_HOST='%' -e MYSQL_ROOT_PASSWORD='strongpassword' \
-d mysql/mysql-server:latest
Make sure MYSQL_ROOT_HOST='%', that means root can connect from any IP.
I'm trying to create a docker setup such that I can easily build and deploy a geodjango app (with postgis backend). I have the following folder structure:
|-- Dockerfile
|-- Pipfile
|-- Pipfile.lock
|-- README.md
|-- app
| |-- manage.py
| |-- app
| `-- app_web
In my Dockerfile to setup Django I have the following:
# Pull base image
FROM python:3.7
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install dependencies
RUN pip install pipenv
COPY . /code
WORKDIR /code/
RUN pipenv install --system
# Setup GDAL
RUN apt-get update &&\
apt-get install -y binutils libproj-dev gdal-bin python-gdal python3-gdal
# set work directory
WORKDIR /code/app
CMD ["python", "manage.py", "migrate", "--no-input"]
In my docker-compose.yml file:
version: '3.7'
services:
postgis:
image: kartoza/postgis:12.1
volumes:
- postgres_data:/var/lib/postgresql/data/
web:
build: .
command: python /code/app/manage.py runserver 0.0.0.0:8000
ports:
- 8000:8000
volumes:
- .:/code
depends_on:
- postgis
volumes:
postgres_data:
And finally in settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'postgis',
},
}
Now when I: run docker-compose up --build
It all seems to work (both the database as well as the django app spin up their containers). But whenever I try to actually work with the database (the app is blank rightnow, so I still need to migrate) django doesn't seem to recognize the database.
Can anybody tell me what is going wrong?
run docker-compose up --build It all seems to work (both the database as well as the django app spin up their containers)
Make sure to check the logs because containers not necessarily stop after finding an error.
docker-compose logs
Also you're already setting your working dir in Dockerfile
WORKDIR /code/app
so take a look at the command you execute in docker-compose.yml:
command: python /code/app/manage.py runserver 0.0.0.0:8000
Did you install the app?
INSTALLED_APPS = [
# [...]
'django.contrib.gis'
]
In settings.py, PASSWORD needs to be specified:
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': 'postgres',
'HOST': 'postgis',
},
}
I'm running into problems when using Docker. To connect from Django application to the Postgres database I have to use:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'db',
'PORT': 5432,
}
}
However to run tests via pytest in my Pipenv shell I have to change 'Host': from db to localhost. Is there a way that I can always use localhost?
Docker-Compose:
version: '3'
services:
db:
image: postgres
ports:
- "5432:5432"
web:
build: .
env_file: .env
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
container_name: test
Dockerfile:
# Pull base image
FROM python:3
# Set environment varibles
ENV PYTHONUNBUFFERED 1
# Set work directory
RUN mkdir /code
WORKDIR /code
# Install dependencies
RUN pip install --upgrade pip
RUN pip install pipenv
COPY ./Pipfile /code/Pipfile
RUN pipenv install --deploy --system --skip-lock --dev
# Define ENTRYPOINT
COPY ./docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
# Copy project
COPY . /code/
I have the following .gitlab-ci.yml :
image: python:3.6
stages:
- lint
- test
services:
- postgres:10.1-alpine
cache:
paths:
- /root/.local/share/virtualenvs/
before_script:
- python -V
- pip install pipenv
- pipenv install --dev
lint:
stage: lint
script:
- pipenv run pylint --output-format=text --load-plugins pylint_django project/ | tee pylint.txt
- score=$(sed -n 's/^Your code has been rated at \([-0-9.]*\)\/.*/\1/p' pylint.txt)
- echo "Pylint score was $score"
- pipenv run anybadge --value=$score --file=pylint.svg pylint
artifacts:
paths:
- pylint.svg
test:
stage: test
script:
- pipenv run python manage.py test
And I am connecting to the database like this :
# settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': '',
'HOST': 'db', # set in docker-compose.yml
'PORT': 5432 # default postgres port
}
}
For now, I just have this tests in users/tests.py :
from .models import CustomUser
from django.test import TestCase
class LogInTest(TestCase):
def setUp(self):
self.credentials = {
'username': 'testuser',
'password': 'secret'}
CustomUser.objects.create_user(**self.credentials)
def testLogin(self):
# send login data
response = self.client.post('/users/login/', self.credentials, follow=True)
# should be logged in now
self.assertTrue(response.context['user'].is_authenticated)
Which fails with the following error :
psycopg2.OperationalError: could not translate host name "db" to address: Name or service not known
In development, I use the following docker-compose.yml file :
version: '3.3'
services:
db:
image: postgres:10.1-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
web:
build: .
command: python /code/manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- 8000:8000
environment:
- SECRET_KEY=changemeinprod
depends_on:
- db
volumes:
postgres_data:
With this Dockerfile :
FROM python:3.6
# Set environment varibles
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Set work directory
WORKDIR /code
# Install dependencies
RUN pip install --upgrade pip
RUN pip install pipenv
COPY ./Pipfile /code/Pipfile
RUN pipenv install --deploy --system --skip-lock --dev
# Copy project
COPY . /code/
I don't understand why the app cab't connect to the database in the CI but connect just fine in the development ENV with docker
In your docker-compose.yml you set the service name as db, and django settings.py for use db, but in gitlab will use the image name as the name of the service, postgres in case.
You have two options:
1 - Use environments variables in settings.py, some example here
2 - Set a alias in gitlab-ci.yml, like:
services:
- name: postgres:10.1-alpine
alias: db
link: gitlab docs
I am trying to dockerize my existing Django Rest project. I am using MySQL database instead of default SqlLite.
My Dockerfile looks like following:
FROM python:2.7
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY . /code/
RUN pip install -r requirements.txt
and Docker-compose:
version: '3'
services:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: docker
MYSQL_DATABASE: docker
MYSQL_USER: docker
MYSQL_PASSWORD: docker
ports:
- "3306:3306"
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
I did not run docker-compose run web python manage.py migrate
docker-compose build is successful
However docker-compose up fails eventually saying Can't connect to local MySQL server. I am guessing that I need to install MySQl in my container as well, but do not know how. What am I missing in my Dockerfile or docker-compose?
UPDATE: My settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql', #django.db.backends.mysql
'NAME': 'libraries', #local: libraries #server:
'USER': 'root', #root #root
'PASSWORD': 'root', #local: root #server:
'HOST': 'localhost', #local: localhost #server:
'PORT': '3306',
}
}
Change the HOST in the database settings from localhost to db. There's no MySQL in the web container so the Python code couldn't connect to the db.