I've been trying to debug my kibana-elasticsearch service for a couple of days now.
I want to be able to access my Docker Kibana container ui in a web browser on a separate host but the service is not available?
It only for exploring and testing so I don't need any authentication on it for now. I've locked down the security group to trusted ip addresses.
Both Kibana and Elasticsearch containers are running. I can access Kibana via localhost:5601.
After trawling through loads of posts and documentation I know the issue is with binding the container to a an ip address to make it accessible to external hosts.
my kibana config file:
# Kibana is served by a back end server. This setting specifies the port to use.
#server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"
my docker compose file:
version: '3.6'
services:
# creates a fluentd service with mountpoints
fluentd:
container_name: fluentd
user: root
build:
context: .
image: fluentd
ports:
- "9880:9880"
volumes:
# LOCAL_HOST_DIR:CONTAINER_DIR
- /var/lib/docker/containers:/fluentd/log/containers # Example: Reading docker logs on
- ./file:/fluentd/log/files/ #Example: Reading logs from a file
- ./configurations:/fluentd/etc/ #where defualt config file is located
# - ./logs:/output/ # Example:Fluentd will collect logs and store it here for demo
logging:
driver: "local"
# This app sends logs to Fluentd endpoint via HTTP
http-myapp:
container_name: http-myapp
image: alpine
volumes:
- ./http:/app
command: [ /bin/sh , -c , "apk add --no-cache curl && chmod +x /app/http_app.sh && ./app/http_app.sh"]
# write test files to a local volume
file-myapp:
image: alpine
container_name: log-generator
# restart: always
volumes:
- ./file:/app
command: [/bin/sh, -c , "chmod +x /app/app.sh && ./app/app.sh"]
elasticsearch: # port 9200
image: docker.elastic.co/elasticsearch/elasticsearch:7.13.1
container_name: elasticsearch
environment:
- node.name=elasticsearch
# - cluster.initial_master_nodes=elasticsearch
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xmx256m -Xms256m"
- discovery.type=single-node
volumes:
- esdata:/usr/share/elasticsearch/data
ulimits:
memlock:
soft: -1
hard: -1
kibana:
image: docker.elastic.co/kibana/kibana:7.13.1
container_name: kibana
volumes:
- ./kibana.yml:/usr/share/kibana/config/kibana.yml
ports:
- "5601:5601"
environment:
ELASTICSEARCH_URL: http://elasticsearch:9200
ELASTICSEARCH_HOSTS: http://elasticsearch:9200
SERVER_NAME: kibana
# SERVER.HOST: "0.0.0.0"
volumes:
esdata:
Related
Yes i know this question has been asked, and the usual error is making the DB_HOST = docker service name, in my case db.
Here's my docker-compose.yml:
version: "3"
services:
db:
image: postgres:latest
restart: always
ports:
- "54320:5432"
expose:
- "54320"
volumes:
- ./database-data:/var/lib/postgresql/data/ # persist data even if container shuts down
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
# POSTGRES_HOST_AUTH_METHOD: trust
django:
build: ./api
restart: always
command: ["python", "manage.py", "runserver", "0.0.0.0:8000"]
volumes:
- ./api:/app/api
ports:
- "8000:8000"
depends_on:
- db
raddl_admin:
build: ./frontend/raddl_admin
stdin_open: true # docker run -i
tty: true # docker run -t
command: ["npm", "start"]
ports:
- "3000:3000"
volumes:
- ./frontend/raddl_admin:/app/frontend/raddl_admin
- /app/frontend/raddl_admin/node_modules
volumes:
database-data: # named volumes can be managed easier using docker-compose
Logging the django service gives:
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "db" (172.18.0.2) and accepting
TCP/IP connections on port 54320?
So it's clearly trying to connect to db on that port and my environment vars are set for the django instance.
I can even ping my db service from my django service:
docker exec -it 1f764333ace2 bash | ping db
PING db (23.221.222.250): 56 data bytes
64 bytes from 23.221.222.250: icmp_seq=0 ttl=50 time=42.055 ms
64 bytes from 23.221.222.250: icmp_seq=1 ttl=50 time=42.382 ms
6
Why can't I connect to the db service from Django?
Ah, so my issue was with the port number. For some reason, keeping the db exposed port to "5432" resolved this. I thought that it would have conflicted with my host's localhost instance of postgres. I guess not.
I have the following setup:
docker-compose.yml
# Mentioning which format of dockerfile
version: "3.9"
# services or nicknamed the container
services:
# web service for the web
web:
# you should use the --build flag for every node package added
build: .
# Add additional commands for webpack to 'watch for changes and bundle it to production'
command: python manage.py runserver 0.0.0.0:8000
volumes:
- type: bind
source: .
target: /code
ports:
- "8000:8000"
depends_on:
- db
environment:
- "DJANGO_SECRET_KEY=django-insecure-m#x2vcrd_2un!9b4la%^)ou&hcib&nc9fvqn0s23z%i1e5))6&"
- "DJANGO_DEBUG=True"
expose:
- 8000
db:
image: postgres:13
#
volumes:
- postgres_data:/var/lib/postgresql/data/
# unsure of what this environment means.
environment:
- "POSTGRES_HOST_AUTH_METHOD=trust"
# - "POSTGRES_USER=postgres"
# Volumes set up
volumes:
postgres_data:
and a settings file as
ALLOWED_HOSTS = ['0.0.0.0', 'localhost', '127.0.0.1']
#127.0.0.1 is my localhost address.
With my host's IP as 192.168.0.214
Can you please help me deploy the django site on my host's local network?
Do I have to set up something on my router?
Or could you direct me towards resources(understanding networking) which will help me understand the same.
I'm currently trying to make a connection between two of my docker containers (The requesting container running Gunicorn/Django and the api container running kroki).
I've had a look at other answers but seem to be coming up blank with a solution, so was hoping for a little poke in the right direction.
Docker-compose:
version: '3.8'
services:
app:
build:
context: ./my_app
dockerfile: Dockerfile.prod
command: gunicorn my_app.wsgi:application --bind 0.0.0.0:8000 --access-logfile -
volumes:
- static_volume:/home/app/web/staticfiles
expose:
- 8000
environment:
- DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 kroki
env_file:
- ./.env.prod
depends_on:
- db
db:
image: postgres:13.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- ./.env.prod.db
nginx:
build: ./nginx
volumes:
- static_volume:/home/app/web/staticfiles
ports:
- 1337:80
depends_on:
- app
kroki:
image: yuzutech/kroki
ports:
- 7331:8000
volumes:
postgres_data:
static_volume:
settings.py
ALLOWED_HOSTS = os.environ.get("DJANGO_ALLOWED_HOSTS").split(" ")
Requesting code in django
url = 'http://kroki:7331/bytefield/svg/' + base64_var
try:
response = requests.get(url)
return response.text
except ConnectionError as e:
print("Connection to bytefield module, unavailable")
return None
I'm able to access both containers via my browser successfully, however initiating the code for an internal call between the two throws out
requests.exceptions.ConnectionError: HTTPConnectionPool(host='kroki', port=7331): Max retries exceeded with url: /bytefield/svg/<API_URL_HERE> (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f286f5ecaf0>: Failed to establish a new connection: [Errno 111] Connection refused'))
I've had a go accessing the url via localhost:7331 & 127.0.0.1:7331 however neither seem to help at all
When you access to other containers within the same network you don't have to access to them using the exposed port in the host but using instead the actual port where the application within the container is listening.
I made a really simple example so you can understand what's your problem:
version: '3.8'
services:
app:
image: busybox
entrypoint: tail -f /dev/null
kroki:
image: yuzutech/kroki
ports:
- 7331:8000
From Host
❯ curl -s -o /dev/null -w "%{http_code}" localhost:7331
200
From App
/ # wget kroki:7331
Connecting to kroki:7331 (172.18.0.3:7331)
wget: can't connect to remote host (172.18.0.3): Connection refused
/ # wget kroki:8000
Connecting to kroki:8000 (172.18.0.3:8000)
saving to 'index.html'
index.html 100% |************************************************************************| 51087 0:00:00 ETA
'index.html' saved
I want to monitoring my database using prometheus, django rest framework and docker,
all is my local machine, the error is below:
well the error is the url http://127.0.0.1:9000/metrics, the http://127.0.0.1:9000 is the begging the my API, and I don't know what's the problem, my configuration is below
my requirements.txt
django-prometheus
my file docker: docker-compose-monitoring.yml
version: '2'
services:
prometheus:
image: prom/prometheus:v2.14.0
volumes:
- ./prometheus/:/etc/prometheus/
command:
- '--config.file=/etc/prometheus/prometheus.yml'
ports:
- 9090:9090
grafana:
image: grafana/grafana:6.5.2
ports:
- 3060:3060
my folder and file prometheus/prometheus.yml
global:
scrape_interval: 15s
rule_files:
scrape_configs:
- job_name: prometheus
static_configs:
- targets:
- 127.0.0.1:9090
- job_name: monitoring_api
static_configs:
- targets:
- 127.0.0.1:9000
my file settings.py
INSTALLED_APPS=[
...........
'django_prometheus',]
MIDDLEWARE:[
'django_prometheus.middleware.PrometheusBeforeMiddleware',
......
'django_prometheus.middleware.PrometheusAfterMiddleware']
my model.py
from django_promethues.models import ExportMOdelOperationMixin
class MyModel(ExportMOdelOperationMixin('mymodel'), models.Model):
"""all my fields in here"""
my urls.py
url('', include('django_prometheus.urls')),
well the application is running well, when in the 127.0.0.1:9090/metrics, but just monitoring the same url, and I need monitoring different url, I think the problem is not the configuration except in the file prometheus.yml, because I don't know how to call my table or my api, please help me.
bye.
you need to change your config of prometheus and add python image in docker-compose like this:
config of prometheus(prometheus.yaml):
global:
scrape_interval: 15s # when Prometheus is pulling data from exporters etc
evaluation_interval: 30s # time between each evaluation of Prometheus' alerting rules
scrape_configs:
- job_name: django_project # your project name
metrics_path: /metrics
static_configs:
- targets:
- web:8000
docker-compose file for prometheus and django , you can also include grafana image, I have installed grafana locally:
version: '3.7'
services:
web:
build:
context: . # context represent path of your dockerfile(dockerfile present in the root dir)
command: sh -c "python3 manage.py migrate &&
gunicorn webapp.route.wsgi:application --pythonpath webapp --bind 0.0.0.0:8000"
volumes:
- .:/app
ports:
- "8000:8000"
prometheus:
image: prom/prometheus
ports:
- "9090:9090"
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml #prometheus.yaml present in the root dir
Dockerfile:
FROM python:3.8
COPY ./webapp/django /app
COPY ./requirements.txt /app/requirements.txt
WORKDIR /app
RUN pip3 install -r requirements.txt*strong text*
For prometheus settings in django:
https://pypi.org/project/django-prometheus/
Hit django app api's.
Hit localhost:8000/metrics api.
Hit localhost:9090/ and search for the required metrics from the dropdown and click on execute it will generate result in console and create graph
To show graph in the grafana hit localhost:3000 and create new dashboard.
django views.py
import redis
import jwt
from access import utils
import os
redis_url = os.environ['REDIS_URI']
R = redis.StrictRedis(redis_url)
def set(request):
R.set('foo', 'bar')
return JsonResponse({"code":200,"msg":"success"})
docker-compose
version: "3"
services:
rango:
container_name: rango
build: ./
command: python backend/manage.py runserver 0.0.0.0:8000
# command: npm start --prefix frontend/rango-frontend/
working_dir: /usr/src/rango
environment:
REDIS_URI: redis://redis_db:6379
ports:
- "8000:8000"
tty: true
links:
- elasticsearch
- node
- redis
#elastic search
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.0
ports:
- "9200:9200"
#node
node:
image: node:10.13.0
#redis
redis:
image: redis
environment:
- ALLOW_EMPTY_PASSWORD=yes
ports:
- "6379:6379"
here i am connecting redis from django inside docker.
it is giving me exceptions connexctions refused.
Please have a look into my code and shared screenshot below
By default, docker compose makes containers discoverable with a hostname identical to the container name. Your redis container is thus discoverable via the hostname redis. However, your Django container is using the hostname redis_db.
Update your docker-compose.yml and change the REDIS_URI to reference the correct hostname:
REDIS_URI: redis://redis:6379