Django, Nginx, uwsgi, Kubernetes: unable to connect nginx to uwsgi stream - django

I have a django container and an Ngix container. they work fine with docker-compose, and now im trying to use the images with kubernetes. Everything works fine, except the fact that the nginx container cannot connect to the uwsgi upstream. No response is being returned.
Here are my configuration:
# Nginx congifuration
upstream django {
server admin-api-app:8001 max_fails=20 fail_timeout=10s; # for a web port socket (we'll use this first),
}
server {
# the port your site will be served on
listen 80;
server_name server localhost my-website-domain.de;
charset utf-8;
location / {
uwsgi_pass django;
include /etc/nginx/uwsgi_params;
}
}
# Uwsgi file
module = site_module.wsgi
master = true
processes = 5
socket = :8001
enable-threads = true
vacuum=True
# Kubernetes
# Backend Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: container-backend
labels:
app: backend
spec:
replicas: 1
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
containers:
- name: container-backend
image: my-djangoimage:latest
command: ["./docker/entrypoint.sh"]
ports:
- containerPort: 8001
name: uwsgi
- name: nginx
image: my-nginx-image:latest
imagePullPolicy: Always
ports:
- containerPort: 80
name: http
---
# Backend Service
kind: Service
apiVersion: v1
metadata:
name: admin-api-app
spec:
selector:
app: backend
ports:
- port: 80
targetPort: 80
type: LoadBalancer

You probably need to change host in your django upstream because, as far as I understand, you want to connect to your django app located in the same pod where is nginx so try to change:
server admin-api-app:8001 max_fails=20 fail_timeout=10s;
to
server localhost:8001 max_fails=20 fail_timeout=10s;
Edit:
To make it work you need to change socket to http-socket but it can be painful/pointless as described here: Should I have separate containers for Flask, uWSGI, and nginx?

Related

Redis deployed in AWS - Connection time out from localhost SpringBoot app

Small question regarding Redis deployed in AWS (not AWS Elastic Cache) and an issue connecting to it.
Here is the setup of the Redis deployed in AWS: (pasting only the Kubernetes StatefulSet and Service)
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis
spec:
serviceName: redis
replicas: 3
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
initContainers:
- name: config
image: redis:7.0.5-alpine
command: [ "sh", "-c" ]
args:
- |
cp /tmp/redis/redis.conf /etc/redis/redis.conf
echo "finding master..."
MASTER_FDQN=`hostname -f | sed -e 's/redis-[0-9]\./redis-0./'`
if [ "$(redis-cli -h sentinel -p 5000 ping)" != "PONG" ]; then
echo "master not found, defaulting to redis-0"
if [ "$(hostname)" = "redis-0" ]; then
echo "this is redis-0, not updating config..."
else
echo "updating redis.conf..."
echo "slaveof $MASTER_FDQN 6379" >> /etc/redis/redis.conf
fi
else
echo "sentinel found, finding master"
MASTER="$(redis-cli -h sentinel -p 5000 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-\d{1,})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})')"
echo "master found : $MASTER, updating redis.conf"
echo "slaveof $MASTER 6379" >> /etc/redis/redis.conf
fi
volumeMounts:
- name: redis-config
mountPath: /etc/redis/
- name: config
mountPath: /tmp/redis/
containers:
- name: redis
image: redis:7.0.5-alpine
command: ["redis-server"]
args: ["/etc/redis/redis.conf"]
ports:
- containerPort: 6379
name: redis
volumeMounts:
- name: data
mountPath: /data
- name: redis-config
mountPath: /etc/redis/
volumes:
- name: redis-config
emptyDir: {}
- name: config
configMap:
name: redis-config
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: nfs-1
resources:
requests:
storage: 50Mi
---
apiVersion: v1
kind: Service
metadata:
name: redis
spec:
ports:
- port: 6379
targetPort: 6379
name: redis
selector:
app: redis
type: LoadBalancer
The pods are healthy, I can exec into it and perform operations fine. Here is the get all:
NAME READY STATUS RESTARTS AGE
pod/redis-0 1/1 Running 0 22h
pod/redis-1 1/1 Running 0 22h
pod/redis-2 1/1 Running 0 22h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/redis LoadBalancer 192.168.45.55 10.51.5.2 6379:30315/TCP 26h
NAME READY AGE
statefulset.apps/redis 3/3 22h
Here is the describe of the service:
Name: redis
Namespace: Namespace
Labels: <none>
Annotations: <none>
Selector: app=redis
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 192.168.22.33
IPs: 192.168.22.33
LoadBalancer Ingress: 10.51.5.2
Port: redis 6379/TCP
TargetPort: 6379/TCP
NodePort: redis 30315/TCP
Endpoints: 192.xxx:6379,192.xxx:6379,192.xxx:6379
Session Affinity: None
External Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal IPAllocated 68s metallb-controller Assigned IP ["10.51.5.2"]
Normal nodeAssigned 58s (x5 over 66s) metallb-speaker announcing from node "someaddress.com" with protocol "bgp"
Normal nodeAssigned 58s (x5 over 66s) metallb-speaker announcing from node "someaddress.com" with protocol "bgp"
I then try to connect to it, i.e. inserting some data with a very straightforward Spring Boot application. The application has no business logic, just trying to insert data.
Here are the relevant parts:
#Configuration
public class RedisConfiguration {
#Bean
public ReactiveRedisConnectionFactory reactiveRedisConnectionFactory() {
return new LettuceConnectionFactory("10.51.5.2", 30315);
}
#Repository
public class RedisRepository {
private final ReactiveRedisOperations<String, String> reactiveRedisOperations;
public RedisRepository(ReactiveRedisOperations<String, String> reactiveRedisOperations) {
this.reactiveRedisOperations = reactiveRedisOperations;
}
public Mono<RedisPojo> save(RedisPojo redisPojo) {
return reactiveRedisOperations.opsForValue().set(redisPojo.getInput(), redisPojo.getOutput()).map(__ -> redisPojo);
}
Each time I am trying to write the data, I am getting this exception:
2022-12-02T20:20:08.015+08:00 ERROR 1184 --- [ctor-http-nio-3] a.w.r.e.AbstractErrorWebExceptionHandler : [8f16a752-1] 500 Server Error for HTTP POST "/save"
org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.translateException(LettuceConnectionFactory.java:1602) ~[spring-data-redis-3.0.0.jar:3.0.0]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
*__checkpoint ⇢ Handler com.redis.controller.RedisController#test(RedisRequest) [DispatcherHandler]
*__checkpoint ⇢ HTTP POST "/save" [ExceptionHandlingWebHandler]
Original Stack Trace:
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.translateException(LettuceConnectionFactory.java:1602) ~[spring-data-redis-3.0.0.jar:3.0.0]
Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to 10.51.5.2/<unresolved>:30315
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:78) ~[lettuce-core-6.2.1.RELEASE.jar:6.2.1.RELEASE]
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:56) ~[lettuce-core-6.2.1.RELEASE.jar:6.2.1.RELEASE]
at io.lettuce.core.AbstractRedisClient.getConnection(AbstractRedisClient.java:350) ~[lettuce-core-6.2.1.RELEASE.jar:6.2.1.RELEASE]
at io.lettuce.core.RedisClient.connect(RedisClient.java:216) ~[lettuce-core-6.2.1.RELEASE.jar:6.2.1.RELEASE]
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /10.51.5.2:30315
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:261) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.85.Final.jar:4.1.85.Final]
This is particularly puzzling, because I am quite sure the code of the Spring Boot app is working. When I change the IP of return new LettuceConnectionFactory("10.51.5.2", 30315);: to
a regular Redis on my laptop ("localhost", 6379),
a dockerized Redis on my laptop,
a dockerized Redis on prem, all are working fine.
Therefore, I am quite puzzled what did I do wrong with the setup of this Redis in AWS.
What should I do in order to connect to it properly.
May I get some help please?
Thank you
By default, Redis binds itself to the IP addresses 127.0.0.1 and ::1 and does not accept connections against non-local interfaces. Chances are high that this is your main issue and you may want to review your redis.conf file to bind Redis to the interface you need or to the generic * -::*, as explained in the comments of the config file itself (which I have linked above).
With that being said, Redis also does not accept connections on non-local interfaces if the default user has no password - a security layer named Protected mode. Thus you should either give your default user a password or disable protected mode in your redis.conf file.
Not sure if this applies to your case but, as a side note, I would suggest to always avoid exposing Redis to the Internet.
You are mixing 2 things.
To enable this service for pods in different namespaces you do not need external load balancer, you can just try to use redis.namespace-name:6379 dns name and it will just work. Such dns is there for every service you create (but works only inside kubernetes)
Kubernetes will make sure that your traffic will be routed to proper pods (assuming there is more than one).
If you want to expose redis from outside of kubernetes then you need to make sure there is connectivity from the outside and then you need network load balancer that will forward traffic to your kubernetes service (in your case node port, so you need NLB with eks worker nodes: 30315 as a targets)
If your worker nodes have public IP and their SecurityGroups allow connecting to them directly, you could try to connect to worker node's IP directly just to test things out (without LB).
And regardless off yout setup you can always create proxy via kubectl
kubectl port-forward -n redisNS svc/redis 6379:6379
and connect from spring boot app to localhost:6379
How do you want to connect from app to redis in a final setup?

Nginx upstream server values when serving an API using docker-compose and kubernetes

I'm trying to use docker-compose and kubernetes as two different solutions to setup a Django API served by Gunicorn (as the web server) and Nginx (as the reverse proxy). Here are the key files:
default.tmpl (nginx) - this is converted to default.conf when the environment variable is filled in:
upstream api {
server ${UPSTREAM_SERVER};
}
server {
listen 80;
location / {
proxy_pass http://api;
}
location /staticfiles {
alias /app/static/;
}
}
docker-compose.yaml:
version: '3'
services:
api-gunicorn:
build: ./api
command: gunicorn --bind=0.0.0.0:8000 api.wsgi:application
volumes:
- ./api:/app
api-proxy:
build: ./api-proxy
command: /bin/bash -c "envsubst < /etc/nginx/conf.d/default.tmpl > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
environment:
- UPSTREAM_SERVER=api-gunicorn:8000
ports:
- 80:80
volumes:
- ./api/static:/app/static
depends_on:
- api-gunicorn
api-deployment.yaml (kubernetes):
apiVersion: apps/v1
kind: Deployment
metadata:
name: release-name-myapp-api-proxy
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: myapp-api-proxy
template:
metadata:
labels:
app.kubernetes.io/name: myapp-api-proxy
spec:
containers:
- name: myapp-api-gunicorn
image: "helm-django_api-gunicorn:latest"
imagePullPolicy: Never
command:
- "/bin/bash"
args:
- "-c"
- "gunicorn --bind=0.0.0.0:8000 api.wsgi:application"
- name: myapp-api-proxy
image: "helm-django_api-proxy:latest"
imagePullPolicy: Never
command:
- "/bin/bash"
args:
- "-c"
- "envsubst < /etc/nginx/conf.d/default.tmpl > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
env:
- name: UPSTREAM_SERVER
value: 127.0.0.1:8000
volumeMounts:
- mountPath: /app/static
name: api-static-assets-on-host-mount
volumes:
- name: api-static-assets-on-host-mount
hostPath:
path: /Users/jonathan.metz/repos/personal/code-demos/kubernetes-demo/helm-django/api/static
My question involves the UPSTREAM_SERVER environment variable.
For docker-compose.yaml, the following values have worked for me:
Setting it to the name of the gunicorn service and the port it's running on (in this case api-gunicorn:8000). This is the best way to do it (and how I've done it in the docker-compose file above) because I don't need to expose the 8000 port to the host machine.
Setting it to MY_IP_ADDRESS:8000 as described in this SO post. This method requires me to expose the 8000 port, which is not ideal.
For api-deployment.yaml, only the following value has worked for me:
Setting it to localhost:8000. Inside of a pod, all containers can communicate using localhost.
Are there any other values for UPSTREAM_SERVER that work here, especially in the kubernetes file? I feel like I should be able to point to the container's name and that should work.
You could create a service to target container myapp-api-gunicorn but this will expose it outside of the pod:
apiVersion: v1
kind: Service
metadata:
name: api-gunicorn-service
spec:
selector:
app.kubernetes.io/name: myapp-api-proxy
ports:
- protocol: TCP
port: 8000
targetPort: 8000
You might also use hostname and subdomain fields inside a pod to take advantage of FQDN.
Currently when a pod is created, its hostname is the Pod’s metadata.name value.
The Pod spec has an optional hostname field, which can be used to specify the Pod’s hostname. When specified, it takes precedence over the Pod’s name to be the hostname of the pod. For example, given a Pod with hostname set to “my-host”, the Pod will have its hostname set to “my-host”.
The Pod spec also has an optional subdomain field which can be used to specify its subdomain. For example, a Pod with hostname set to “foo”, and subdomain set to “bar”, in namespace “my-namespace”, will have the fully qualified domain name (FQDN) “foo.bar.my-namespace.svc.cluster-domain.example”.
Also here is a nice article from Mirantis which talks about exposing multiple containers in a pod

Hosting django on aks behind nginx-ingress

I am trying to host a django website on Azure kubernetes service behide nginx-ingress, and I would like my django web show under a path.
e.g. when access the default admin site, I would like to access it at http://example.com/django/admin instead of http://example.com/admin
I tried the configure below, when I access http://example.com/django/admin it will forward me to http://example.com/admin and show me 404 error from default ingress backend, as I set django debug to ture I assume this mean ingress did not send my request to my django service
# path example
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: django-ingress
labels:
app: django
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- backend:
serviceName: django-service
servicePort: 80
path: /django(/|$)(.*)
so I try to curl -I -k http://example.com/django/admin, and it show something like below
HTTP/1.1 301 Moved Permanently
Server: openresty/1.15.8.2
Date: Wed, 06 Nov 2019 04:14:14 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Location: /admin/
The same thing happen to any valid page in the site, if I curl -I -k http://example.com/django/any_valid_page it show below:
HTTP/1.1 301 Moved Permanently
Server: openresty/1.15.8.2
Date: Wed, 06 Nov 2019 04:14:14 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Location: /any_valid_page/
I wonder it is caused by I am doing the test with the default django development web server? (i.e. python manage.py runserver)
If I try to host it at root like below, everything is fine...
# root example
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: django-ingress
labels:
app: django
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- backend:
serviceName: django-service
servicePort: 80
path: /
Trying adding this
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: django-ingress
labels:
app: django
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/rewrite-target: /django
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- backend:
serviceName: django-service
servicePort: 80
path: /django
Starting in Version 0.22.0, ingress definitions using the annotation
nginx.ingress.kubernetes.io/rewrite-target are not backwards
compatible with previous versions. In Version 0.22.0 and beyond, any
substrings within the request URI that need to be passed to the
rewritten path must explicitly be defined in a capture group. So make
sure you have right version.
When using SSL offloading outside of cluster it may be useful to enforce a redirect to HTTPS even when there is no TLS certificate available. This can be achieved by using the nginx.ingress.kubernetes.io/force-ssl-redirect: "true" annotation in the particular resource.
I think your Ingress configuration file should look like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: django-ingress
labels:
app: django
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: example.com
http:
paths:
- path: /django(/|$)(.*)
backend:
serviceName: django-service
servicePort: 80
If you get 404 error, there is possible solution:
Please change https to http in the curl command?:
curl --resolve your-host:80:xx.xxx.xx.xxx http://my-host:80
To get the IP from kubectl get ing command, it is necessary to
enable the reporting Ingress status feature. Take a look on: reporting-ingress-status.
There is the default server in the Ingress controller. It returns
the Not Found page with the 404 status code for all requests for
domains for which there are no Ingress rules defined. Those requests
are not shown in the access log.
Since you're getting a 404, this means that the host header of your
requests doesn't match with the host field in the Ingress resource.
To set the host header in curl, please see previous curl
commands. Optionally, you can also do:
curl http://<ip> -H "host: example.com"
Please take a look on ngnix-ingress, server-side-https-enforcement-nginx.
This is a problem from Django's side. Whenever the admin is not logged in, the /django/admin results in a redirect to /admin/. In this case, if you just replace /django/admin/ with /django/admin/ in the browser URL field it will work and open django admin login.
So basically Django's built-in redirect conflicts with the Ingress's rewrite module.

Ingress Controller on Minikube not routing correctly flask POST request

I have a simple Flask app. It worked fine when I connected to it via port-forwarding to send the HTTP Post request directly to the Service.
from flask import Flask, request
import redis
from rq import Queue
from worker import job_worker
UPLOAD_FOLDER = './uploads/'
app = Flask(__name__)
r = redis.Redis()
q = Queue(connection = r)
#app.route('/', methods=['POST'])
def upload():
scale = int(request.form['scale'])
q.enqueue(job_worker, scale)
return ""
if __name__ == "__main__":
app.run()
I also have a simple index.html file in an nginx container which is served at port 80. It does an ajax POST request to "/upload". Which if you look at the ingress controller, should convert that to a port 5000 request and strip away the "upload"
The flask app gets served at port 5000
Here is the ingress controller:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: emoji-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /upload
backend:
serviceName: emoji-backend
servicePort: 5000
- path: /
backend:
serviceName: emoji-frontend
servicePort: 80
And for completeness, the emoji-backend service:
apiVersion: v1
kind: Service
metadata:
name: emoji-backend
labels:
app: emoji-backend
tier: backend
spec:
type: LoadBalancer
ports:
- port: 5000
selector:
app: emoji-backend
tier: backend
I get a 502 bad gateway without really any indication except the ingress log does say this:
2019/09/29 21:41:04 [error] 2021#2021: *78651 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.64.1, server: _,
request: "POST /upload HTTP/2.0", upstream: "http://172.17.0.4:5000/", host: "192.168.64.5", referrer: "https://192.168.64.5/"
"http://172.17.0.4:5000/" is the correct endpoint and port for the emoji-backend service.
Adding the following line fixed it:
app.run(debug=True,host='0.0.0.0',port=5000)
However, it took me a while to figure that out because at first when I tried it my docker image was not updating when I re-deployed.

how to run celery with django on openshift 3

What is the easiest way to launch a celery beat and worker process in my django pod?
I'm migrating my Openshift v2 Django app to Openshift v3. I'm using Pro subscription. I'm really a noob on Openshift v3 and docker and containers and kubernetes. I have used this tutorial https://blog.openshift.com/migrating-django-applications-openshift-3/ to migrate my app (which works pretty well).
I'm now struggling on how to start celery. On Openshift 2 I just used an action hook post_start:
source $OPENSHIFT_HOMEDIR/python/virtenv/bin/activate
python $OPENSHIFT_REPO_DIR/wsgi/podpub/manage.py celery worker\
--pidfile="$OPENSHIFT_DATA_DIR/celery/run/%n.pid"\
--logfile="$OPENSHIFT_DATA_DIR/celery/log/%n.log"\
python $OPENSHIFT_REPO_DIR/wsgi/podpub/manage.py celery beat\
--pidfile="$OPENSHIFT_DATA_DIR/celery/run/celeryd.pid"\
--logfile="$OPENSHIFT_DATA_DIR/celery/log/celeryd.log" &
-c 1\
--autoreload &
It is a quite simple setup. It just uses the django database as a message broker. No rabbitMQ or something.
Would a openshift "job" be appropriated for that? Or better use powershift image (https://pypi.python.org/pypi/powershift-image) action commands? But I did not understand how to execute them.
here is the current deployment configuration for my only app "
apiVersion: v1
kind: DeploymentConfig
metadata:
annotations:
openshift.io/generated-by: OpenShiftNewApp
creationTimestamp: 2017-12-27T22:58:31Z
generation: 67
labels:
app: django
name: django
namespace: myproject
resourceVersion: "68466321"
selfLink: /oapi/v1/namespaces/myproject/deploymentconfigs/django
uid: 64600436-ab49-11e7-ab43-0601fd434256
spec:
replicas: 1
selector:
app: django
deploymentconfig: django
strategy:
activeDeadlineSeconds: 21600
recreateParams:
timeoutSeconds: 600
resources: {}
rollingParams:
intervalSeconds: 1
maxSurge: 25%
maxUnavailable: 25%
timeoutSeconds: 600
updatePeriodSeconds: 1
type: Recreate
template:
metadata:
annotations:
openshift.io/generated-by: OpenShiftNewApp
creationTimestamp: null
labels:
app: django
deploymentconfig: django
spec:
containers:
- image: docker-registry.default.svc:5000/myproject/django#sha256:6a0caac773acc65daad2e6ac87695f9f01ae3c99faba14536e0ec2b65088c808
imagePullPolicy: Always
name: django
ports:
- containerPort: 8080
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /opt/app-root/src/data
name: data
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
volumes:
- name: data
persistentVolumeClaim:
claimName: django-data
test: false
triggers:
- type: ConfigChange
- imageChangeParams:
automatic: true
containerNames:
- django
from:
kind: ImageStreamTag
name: django:latest
namespace: myproject
lastTriggeredImage: docker-registry.default.svc:5000/myproject/django#sha256:6a0caac773acc65daad2e6ac87695f9f01ae3c99faba14536e0ec2b65088c808
type: ImageChange
I'm using mod_wsgi-express and this is my app.sh
ARGS="$ARGS --log-to-terminal"
ARGS="$ARGS --port 8080"
ARGS="$ARGS --url-alias /static wsgi/static"
exec mod_wsgi-express start-server $ARGS wsgi/application
Help is very appreciated. Thank you
I have managed to get it working, though I'm not quite happy with it. I will move to a postgreSQL database very soon. Here is what I did:
wsgi_mod-express has an option called service-script which starts an additional process besides the actual app. So I updated my app.sh:
#!/bin/bash
ARGS=""
ARGS="$ARGS --log-to-terminal"
ARGS="$ARGS --port 8080"
ARGS="$ARGS --url-alias /static wsgi/static"
ARGS="$ARGS --service-script celery_starter scripts/startCelery.py"
exec mod_wsgi-express start-server $ARGS wsgi/application
mind the last ARGS=... line.
I created a python script that starts up my celery worker and beat.
startCelery.py:
import subprocess
OPENSHIFT_REPO_DIR="/opt/app-root/src"
OPENSHIFT_DATA_DIR="/opt/app-root/src/data"
pathToManagePy=OPENSHIFT_REPO_DIR + "/wsgi/podpub"
worker_cmd = [
"python",
pathToManagePy + "/manage.py",
"celery",
"worker",
"--pidfile="+OPENSHIFT_REPO_DIR+"/%n.pid",
"--logfile="+OPENSHIFT_DATA_DIR+"/celery/log/%n.log",
"-c 1",
"--autoreload"
]
print(worker_cmd)
subprocess.Popen(worker_cmd, close_fds=True)
beat_cmd = [
"python",
pathToManagePy + "/manage.py",
"celery",
"beat",
"--pidfile="+OPENSHIFT_REPO_DIR+"/celeryd.pid",
"--logfile="+OPENSHIFT_DATA_DIR+"/celery/log/celeryd.log",
]
print(beat_cmd)
subprocess.Popen(beat_cmd)
this was actually working, but I kept receiving a message when I tried to launch the celery worker saying
"Running a worker with superuser privileges when the worker accepts messages serialized with pickle is a very bad idea!
If you really want to continue then you have to set the C_FORCE_ROOT environment variable (but please think about this before you do)."
Eventhough I added these configurations to my settings.py in order to remove pickle serializer, it kept giving me that same error message.
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ACEEPT_CONTENT = ['json']
I don't know why.
At the end I added C_FORCE_ROOT to my .s2i/enviroment
C_FORCE_ROOT=true
Now it's working, at least I thinks so. My next job will only run in some hours. I'm still open for any further suggestions and tipps.