I have recently learned to integrate Celery + RabbitMQ in local machine, all three of them were running in separate servers.
During production, should I run celery & RabbitMQ in separate server ?
I have searched for Celery hosting options but couldn't find useful links.
I usually use Pythonanywhere to host my Django app
I found this https://stackoverflow.com/a/46105723/10515390 answer useful but did't get a clarification
Related
I created a website that uses vuejs as the frontend and django as the backend with another service running behind everything that im making api calls to.
So django is setup in a way to look at the dist folder of Vuejs and serve it if you run manage.py runserver. but the problem is that my service that I created is
is also in python and it needs to run in a virtualenv in order to work (It uses tensorflow 1.15.2 and this can only run in a contained environment)
I'm sitting here and wondering how I can deploy the django application and keep the virtualenv active and Im coming up with nothing, I've tried doing some research on this but everything I found was not relevant to my problem. I've deployed it and when I close the ssh connection the virtualenv stops.
If there is anyone that can enlighten my ways I would appreciate it.
i think you need to nginx: https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04
if you are search for keep states just in terminal i suggest tmux https://github.com/tmux/tmux/wiki
You can use uWSGI and nginx to deploy Django apps on server. Here's helpful articles:
https://www.digitalocean.com/community/tutorials/how-to-set-up-uwsgi-and-nginx-to-serve-python-apps-on-centos-7
https://www.digitalocean.com/community/tutorials/how-to-set-up-uwsgi-and-nginx-to-serve-python-apps-on-centos-7
Django official docs also has a page about it: https://docs.djangoproject.com/en/3.1/howto/deployment/wsgi/uwsgi/
There are articles from developers so you can refer them in case you get stuck anywhere:
https://www.freecodecamp.org/news/django-uwsgi-nginx-postgresql-setup-on-aws-ec2-ubuntu16-04-with-python-3-6-6c58698ae9d3/
https://medium.com/#biswashirok/deploying-django-python-3-6-to-digital-ocean-with-uwsgi-nginx-ubuntu-18-04-3f8c2731ade1
I've been reading a lot of blog posts (like this one) on how to deploy Django in a containerized Docker environment.
They all use the runserver command in the docker-compose.yml.
Even the Docker documentation does this.
This suprises me, since using the Django web server is not recommended for production!
What is recommended is pointing the webserver to wsgi.py.
However, none of the articles I've found on Django and Docker explain why they use runserver instead of pointing apache or nginx to wsgi.py.
Why would all these articles use the built-in Django development webserver to handle requests, instead of a full blown webserver like apache or nginx?
Isn't the point of using containers in development, to keep that environment as close to production as possible? Then why build a not-production-ready environment?
The aim of most guides including this you given is to give you a ABC guide to containerize your django applicaiton quickly with docker.
When you decided to read these guides, you are certainly ansumed as experienced django developer, but a new docker user. So the emphasis of these article will not tell you how to use production server(like uwsgi, gunicorn) to manage your django application, because it assumes your are familiar with that.
As a new docker user, it will put more effort to tell you how to dockerize them in container with a start django project. Then, a simple hello-world-like project with development django http server will be the most suitable option.
But, you still need to use uwsgi, gunicorn etc to deploy you apps, E.g. https://hub.docker.com/r/dockerfiles/django-uwsgi-nginx
I'm building a Django project on Heroku.
I understand that gunicorn is recommended as a webserver so I need an event loop type of worker and I use gevent for that.
It seems that monkey patching gevent does most of the work for me so I can have concurrency, but how am I supposed to connect to the RabbitMQ without real threads or jamming the whole loop?
I am baffled by this since Heroku themselves recommend gunicorn, celery and RabbitMQ but I don't see how all of these work together.
Do you understand that celery and gunicorn are used for different purposes?
Gunicorn is the webserver responding to requests made by users, serving them web pages or JSON data.
Celery is an asynchronous task manager, i.e. it lets you run arbitrary python code irrespective of web requests to your server.
Do you understand this distinction?
I have recently switched to Django for a web app I'm developing and I followed the instructions at Heroku for getting a Django app running on Heroku. I have a virtual environment in which my app is developed and I use git for version control and to push to Heroku. The link above suggests that I intall gunicorn:
The examples above used the default HTTP server for Django. For
production apps, you may wish to use a more production-ready embedded
webserver, such as Tornado, gevent’s WSGI server, or Gunicorn.
They then walk the user through installing Gunicorn.
My question is: what problems might I run into if I skip this step and just stay with the default? What benefits will Gunicorn give me?
Gunicorn is production ready and really easy to use. I use it for my websites. You usually should run it via a reverse proxy like Nginx. I'm not sure what Heroku is using. You really should try it.
In my experience it's much easier to use and configure than apache & mod_wsgi, and the other similar setups.
edit/update:
As a summary of the comments below, Heroku already uses Nginx as a reverse proxy
Much better performance, and probably better security and stability, too. Django's development web server (which is used by Heroku by default) isn't really designed to serve production applications.
django's server, is a development server . It is light weigh and easy to use but should not be used in production because it is not production ready. it cannot handle many requests. This link offers a comparison between gunicorn, uwsgi and django's development server.
I am quite a noob when it comes to deploying a Django project. I'd like to know what are the various methods to deploy Django project and which one is the most preferred.
The Django documentation lists Apache/mod_wsgi, Apache/mod_python and FastCGI etc.
mod_python is deprecated now, one should use mod_wsgi instead.
Django with mod_wsgi is easy to setup, but:
you can only use one python version at a time [edit: you even can only use the python version mod_wsgi was compiled for]
[edit: seems if I'm wrong on mod_wsgi not supporting virtualenv: it does]
So for multiple sites (targeting different django/python versions) on a server mod_wsgi is not
the best solution.
FastCGI can be used with virtualenv, also with different python versions, as you run it with
./manage.py runfcgi …
and then configure your webserver to use this fcgi interface.
The new, hot stuff about django deployment seems to be gunicorn. It's a webserver that implements wsgi and is typically used as backend with a "big" webserver as proxy.
Deployment with gunicorn feels a lot like fcgi: you run a process doing the django processing stuff with manage.py, and a webserver as frontend to the world.
But gunicorn deployment has some advantages over fcgi:
speed - I didn't find the sources, but benchmarks say fcgi is not as fast as the f suggests
config files, for fcgi you must do all configuration on the commandline when executing the manage.py command. This comes unhandy when running multiple django instances via an init.d (unix-like OS' system service startup). It's always the same cmdline, with just different configuration files
gunicorn can drop privileges: no need to do this in your init.d script, and it's easy to switch to one user per django instance
gunicorn behaves more like a daemon: writing pidfile and logfile, forking to the background etc. makes again using it in an init.d script easier.
Thus, I would suggest to use the gunicorn solution, unless you have a single site on a single server with low traffic, than you could use the wsgi solution. But I think in the long run you're more happy with gunicorn.
If you have a django only webserver, I would suggest to use nginx as frontendproxy, as it's the best performing (again this is based on benchmarks I read in some blogposts - don't have the url anymore).
Personally I use apache as frontendproxy, as I need it for other sites hosted on the server.
A simple setup instruction for django deployment could be found here:
http://ericholscher.com/blog/2010/aug/16/lessons-learned-dash-easy-django-deployment/
My init.d script for gunicorn is located at github:
https://gist.github.com/753053
Unfortunately I did not yet blog about it, but an experienced sysadmin should be able to do the required setup.
Use the Nginx/Apache/mod-wsgi and you can't go wrong.
If you prefer a simple alternative, just use Apache.
There is a very good deployment document: http://lethain.com/entry/2009/feb/13/the-django-and-ubuntu-intrepid-almanac/
I myself have faced a lot of problems in deploying Django Projects and automating the deployment process. Apache and mod_wsgi were like curse for Django Deployment. There are several tools like Nginx, Gunicorn, SupervisorD and Fabric which are trending for Django deployment. At first I used/configured them individually without Deployment automation which took a lot of time(I had to maintain testing as well as production servers for my client and had to update them as soon as a new feature was tested and approved.) but then I stumbled upon django-fagungis, which totally automates my Django Deployment from cloning my project from bitbucket to deploying on my remote server (it uses Nginx, Gunicorn, SupervisorD, Fabtic and virtualenv and also installs all the dependencies on the fly), all with just three commands :) You can find more about it in my blog post here. Now I even don't have to get involved in this process(which used to take a lot of my time) and one of my junior developers runs those three commands of django-fagungis mentioned here on his local machine and we get a crisp new copy of our project deployed in minutes without any hassle:)