According to celery using redis docs
Configuration
Configuration is easy, just configure the location of your Redis database:
BROKER_URL = 'redis://localhost:6379/0'
Where the URL is in the format of:
redis://:password#hostname:port/db_number
all fields after the scheme are optional, and will default to localhost on port 6379, using database 0.
Where can i find this configuration file. Is this in setting.py of my project.
Related
I am getting the below error after I deployed my website on heroku.
Error 111 connecting to 127.0.0.1:6379. Connection refused.
Request Method: POST
Request URL: https://website.herokuapp.com/account/register
Django Version: 3.2.8
Exception Type: OperationalError
Exception Value:
Error 111 connecting to 127.0.0.1:6379. Connection refused.
Exception Location: /app/.heroku/python/lib/python3.8/site-packages/kombu/connection.py, line 451, in _reraise_as_library_errors
Python Executable: /app/.heroku/python/bin/python
Python Version: 3.8.12
Python Path:
['/app',
'/app/.heroku/python/bin',
'/app',
'/app/.heroku/python/lib/python38.zip',
'/app/.heroku/python/lib/python3.8',
'/app/.heroku/python/lib/python3.8/lib-dynload',
'/app/.heroku/python/lib/python3.8/site-packages']
Server time: Sat, 11 Dec 2021 21:17:12 +0530
So basically my website has to send email regarding otp, after registration and also some contract related emails. These email are neccessary to be sent hence can't be avoided. I posted a question earlier here regardig how to minimize the time that sending emails takes so that the user doesn't have to wait the entire time. I was suggested to use asynchronous code for this. So i decided to use celery for this. I followed the youtube video that taught how to use it.
Now after I pushed the code in the website I am getting this error. I am beginner andd have no idea how to rectify it. Please suggest me what shoul I do. Below are the details and configurations.
settings.py
CELERY_BROKER_URL = 'redis://127.0.0.1:6379'
CELERY_RESULT_BACKEND = 'redis://127.0.0.1:6379'
CELERY_ACCEPT_CONTENT =['application/json']
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TASK_SELERLIZER = 'json'
requirements.txt
amqp==5.0.6
asgiref==3.4.1
billiard==3.6.4.0
celery==5.2.1
click==8.0.3
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
colorama==0.4.4
Deprecated==1.2.13
dj-database-url==0.5.0
Django==3.2.8
django-ckeditor==6.1.0
django-filter==21.1
django-js-asset==1.2.2
django-multiselectfield==0.1.12
dnspython==2.1.0
As I mentioned I am beginer, please suggest me a detailed ans as to how I can recctify this error.
Here's the problem:
CELERY_BROKER_URL = 'redis://127.0.0.1:6379'
Redis won't be running on your local dyno. You'll have to run it somewhere else and configure your code to connect to it. A common choice is to run Redis via an addon:
Once you’ve chosen a broker, create your Heroku app and attach the add-on to it. In the examples we’ll use Heroku Redis as the Redis provider but there are plenty of other Redis providers in the Heroku Elements Marketplace.
If you choose to use Heroku Redis, you'll be able to get the connection string to your instance via the REDIS_URL environment variable:
Heroku add-ons provide your application with environment variables which can be passed to your Celery app. For example:
import os
app.conf.update(BROKER_URL=os.environ['REDIS_URL'],
CELERY_RESULT_BACKEND=os.environ['REDIS_URL'])
Your Celery app now knows to use your chosen broker and result store for all of the tasks you define in it.
Other addons will provide similar configuration mechanisms.
All quoted documentation here, and most links, come from Heroku's Using Celery on Heroku article. I suggest you read the entire document for more information.
I deployed a Django App to AWS Elastic Beanstalk and I'm getting "Invalid HTTP_HOST header" error even when I've added it to my allowed hosts settings.
I'm getting this error:
Invalid HTTP_HOST header: 'recordings-env.kf4qfzaijd.us-west-2.elasticbeanstalk.com'. You may need to add 'recordings-env.kf4qfzaijd.us-west-2.elasticbeanstalk.com' to ALLOWED_HOSTS.
This is what I have in my settings.py
settings.py:
ALLOWED_HOSTS = [
'127.0.0.1',
'localhost',
'.amazonaws.com',
'.elasticbeanstalk.com',
'recordings-env.kf4qfzaijd.us-west-2.elasticbeanstalk.com',
]
I think the problem is that my settings aren't being updated, because isn't working either if I put ALLOWED_HOSTS = ['*']. I left DEBUG = True and in request information I'm getting: ALLOWED_HOSTS: ['localhost']
after modifying it I run eb deploy without errors.
I've realized that my changes wasn't being deployed because I needed to commit first and I didn't know that (my first time deploying to eb). So, that was the problem.
It's a little tricky because you need to dynamically declare your ALLOWED_HOSTS with EB. This article provides some good information in Gotcha #3 on how you could achieve this
I would create a separate settings file called something like settings_production.py then you could place the following code in there:
mysite/settings_production.py
from mysite.settings import *
def is_ec2_linux():
"""Detect if we are running on an EC2 Linux Instance
See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/identify_ec2_instances.html
"""
if os.path.isfile("/sys/hypervisor/uuid"):
with open("/sys/hypervisor/uuid") as f:
uuid = f.read()
return uuid.startswith("ec2")
return False
def get_linux_ec2_private_ip():
"""Get the private IP Address of the machine if running on an EC2 linux server"""
from urllib.request import urlopen
if not is_ec2_linux():
return None
try:
response = urlopen('http://169.254.169.254/latest/meta-data/local-ipv4')
return response.read().decode("utf-8")
except:
return None
finally:
if response:
response.close()
# ElasticBeanstalk healthcheck sends requests with host header = internal ip
# So we detect if we are in elastic beanstalk,
# and add the instances private ip address
private_ip = get_linux_ec2_private_ip()
if private_ip:
ALLOWED_HOSTS += [private_ip, 'your-django-env.elasticbeanstalk.com']
# Other production overrides
DEBUG = False
Now you set "DJANGO_SETTINGS_MODULE" env variable to mysite.production_settings for your production (.i.e your EB environment).
UPDATE:
I decided to take this for a test spin and managed to get it up and running. I discovered a few things though. The above code adds the internal IP of each instance to the ALLOWED_HOSTS. This is purely for health checks so that AWS console can ping the instances internally and receive a 200OK response. I'm leaving the above solution as it is still useful for that purpose. It will not solve your particular error though. To serve you simply need to add your EB url, that is all. You can find it in the AWS console (highlighted in red below) or in the cli by typing eb status and checking the CNAME property.
CONFIG:
Here are my basic config files I manually created in my source:
.ebextensions/django.config
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: mysite/wsgi.py
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: mysite.settings_production
.ebextensions/db-migrate.config
container_commands:
01_migrate:
command: "django-admin.py migrate"
leader_only: true
option_settings:
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: mysite.settings_production
I have a web app written in Django with mongoengine in a Digital Ocean droplet. During the developing step I used a sample of the main database that is located in a remote enviroment. But now I want to connect the Django web app to the main MongoDB database. How can I do this? With an ssh tunnel (I already use it between both environments)? If so, how do I do such connection? I know how to establish an ssh tunnel with pymongo:
from sshtunnel import SSHTunnelForwarder
import pymongo
server = SSHTunnelForwarder(
(remote_ip_address, 22),
ssh_private_key="/home/username/.ssh/id_rsa",
ssh_username="username",
remote_bind_address=('127.0.0.1', 27017),
)
server.start()
client = pymongo.MongoClient('127.0.0.1', server.local_bind_port)
But how do I do the equivalent connection with the Django web app? At the present moment I have the following definitions in the settings.py file.
# MongoDB settings
MONGODB_DATABASES = {'default': {'name':'dbname'} }
DATABASES = {'default': {'ENGINE': 'django.db.backends.dummy'} }
After setting DEBUG = False, and SECURE_SSL_REDIRECT = True and deploying a version on my app to the server, I wish now to develop further locally. The problem is, I think at one point I forgot to remove the SECURE_SSL_REDIRECT = True from settings.py, and I ran the local dev server with heroku local. My local browser always tries not to connect to localhost with SSL, so it just hangs.
I tried removing the site-specific cookies for localhost in the browser settings (Chrome) but localhost now still always tries to establish an SSL connection.
I am trying just to get back to using a non-SSL local connection for development. Any Ideas?
Django version 1.10.2
Heroku
Thanks
EDIT
Seems if I clear ALL the cache and cookies and restart the browser then it will not ask for SSL again. So it seems to be a browser problem. Anyway if anyone has an idea of how to accomplish this without having to clear all the data in Chrome, that would be appreciated.
UPDATE
I have learned a better way to handle this situation. I have set up some code to automatically sense if the software is running on the local environment or the cloud production environment, like this:
if os.environ.get('LOCAL'):
DEBUG = True
SECURE_SSL_REDIRECT = False
else:
DEBUG = False
SECURE_SSL_REDIRECT = True
Of course you have to take care of setting up the environ object, which happens automatically in heroku.
I am using custom settings both for local (development) and production environments.
For instance:
myproject/settings/dev.py
DEBUG = True
SECURE_SSL_REDIRECT = False
...
myproject/settings/production.py
DEBUG = False
SECURE_SSL_REDIRECT = True
...
And then I specify the settings I want to use. On localhost like this:
python myproject/manage.py runserver --settings=settings.dev
and for production using the Heroku Procfile:
web: gunicorn myproject.wsgi.production --log-file -
Content of myproject/wsgi/production.py is:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings.production")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
If you use heroku local for your local development, you can create similar Procfile.local with local wsgi file.
I'm deploying a Django app which uses celery task and has redis as the broker backend. I'm using docker for deployment and my production server is an amazon aws instance. The problem I'm facing is that the django settings is different for localhost:
BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
and all my unit tests work. For docker it fails unless I change it to
BROKER_URL = 'redis://redis:6379'
CELERY_RESULT_BACKEND = 'redis://redis:6379'
My question is, how do I identify the redis broker url in my deployment server? Will it be redis://redis:6379?
PS: For heroku server there's an add-on for identifying the redis url call REDISTOGO_URL. Is there something similar for amazon aws server?
BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
The above implies that both redis and celery are running on localhost, the same machine on which your django app is running.
Please check:
1) Redis is installed on the server, and is running. (sudo service redis-server start)
2) Celery is installed on the server, and is running.
BROKER_URL = 'redis://redis:6379'
CELERY_RESULT_BACKEND = 'redis://redis:6379'
If you are using docker, the above implies that there is another docker container which is running redis, and your code container is linked to the redis container with the alias 'redis'