Cookiecutter - Django: Anymail[SES] boto3 region error - django

I am trying to deploy to AWS(EC2) a Cookiecutter Django project.
The AWS user with this credentials has ful S3, SES and SNS policies.
The EC2 server has also a role with full SES/S3 policies.
In production file in envs I have the keys set up like this.
DJANGO_AWS_ACCESS_KEY_ID=xxxxxxxxx
DJANGO_AWS_SECRET_ACCESS_KEY=xxxxxxxxxx
DJANGO_AWS_STORAGE_BUCKET_NAME=xxxxxxxxxx
In settings I have
AWS_S3_REGION_NAME = env("DJANGO_AWS_S3_REGION_NAME", default=None)
AWS_ACCESS_KEY_ID = env("DJANGO_AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = env("DJANGO_AWS_SECRET_ACCESS_KEY")
AWS_STORAGE_BUCKET_NAME = env("DJANGO_AWS_STORAGE_BUCKET_NAME")
EMAIL_BACKEND = "anymail.backends.amazon_ses.EmailBackend"
ANYMAIL = {}
All nice and fine until the project tries to send an email using SES and it crashes with the error bellow .
Until now I have tried:
adding DJANGO_AWS_S3_REGION_NAME to the production file in envs - no result
adding the region in aws config using aws cli - no result
overriding the settings in ANYMAIL ={} with credetials and region - no result
making a blank project, just adding the aws credentials and not changing anything else - no result
creating manually on another project a boto3.session.client with the same credentials and sending a mail - it works
This is the error. The second part with 'NoneType' object has no attribute 'send_raw_email' repeats a lot after this.
django_1 | [2021-08-13 13:58:14 +0000] [12] [ERROR] Error handling request /accounts/signup/
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
django_1 | response = get_response(request)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
django_1 | response = wrapped_callback(request, *callback_args, **callback_kwargs)
django_1 | File "/usr/local/lib/python3.9/contextlib.py", line 79, in inner
django_1 | return func(*args, **kwds)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
django_1 | return self.dispatch(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
django_1 | return bound_method(*args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/views/decorators/debug.py", line 89, in sensitive_post_parameters_wrapper
django_1 | return view(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 230, in dispatch
django_1 | return super(SignupView, self).dispatch(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 74, in dispatch
django_1 | response = super(RedirectAuthenticatedUserMixin, self).dispatch(
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 204, in dispatch
django_1 | return super(CloseableSignupMixin, self).dispatch(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
django_1 | return handler(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 102, in post
django_1 | response = self.form_valid(form)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 248, in form_valid
django_1 | return complete_signup(
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/utils.py", line 209, in complete_signup
django_1 | return perform_login(
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/utils.py", line 175, in perform_login
django_1 | send_email_confirmation(request, user, signup=signup, email=email)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/utils.py", line 346, in send_email_confirmation
django_1 | email_address.send_confirmation(request, signup=signup)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/models.py", line 62, in send_confirmation
django_1 | confirmation.send(request, signup=signup)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/models.py", line 169, in send
django_1 | get_adapter(request).send_confirmation_mail(request, self, signup)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/adapter.py", line 464, in send_confirmation_mail
django_1 | self.send_mail(email_template, emailconfirmation.email_address.email, ctx)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/adapter.py", line 136, in send_mail
django_1 | msg.send()
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/mail/message.py", line 284, in send
django_1 | return self.get_connection(fail_silently).send_messages([self])
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/base.py", line 89, in send_messages
django_1 | created_session = self.open()
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/amazon_ses.py", line 44, in open
django_1 | self.client = boto3.session.Session(**self.session_params).client("ses", **self.client_params)
django_1 | File "/usr/local/lib/python3.9/site-packages/boto3/session.py", line 258, in client
django_1 | return self._session.create_client(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/session.py", line 847, in create_client
django_1 | client = client_creator.create_client(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 86, in create_client
django_1 | client_args = self._get_client_args(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 355, in _get_client_args
django_1 | return args_creator.get_client_args(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/args.py", line 71, in get_client_args
django_1 | final_args = self.compute_client_args(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/args.py", line 148, in compute_client_args
django_1 | endpoint_config = self._compute_endpoint_config(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/args.py", line 220, in _compute_endpoint_config
django_1 | return self._resolve_endpoint(**resolve_endpoint_kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/args.py", line 302, in _resolve_endpoint
django_1 | return endpoint_bridge.resolve(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 430, in resolve
django_1 | resolved = self.endpoint_resolver.construct_endpoint(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/regions.py", line 133, in construct_endpoint
django_1 | result = self._endpoint_for_partition(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/regions.py", line 148, in _endpoint_for_partition
django_1 | raise NoRegionError()
django_1 | botocore.exceptions.NoRegionError: You must specify a region.
django_1 |
django_1 | During handling of the above exception, another exception occurred:
django_1 |
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
django_1 | response = get_response(request)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/deprecation.py", line 114, in __call__
django_1 | response = response or self.get_response(request)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 49, in inner
django_1 | response = response_for_exception(request, exc)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 104, in response_for_exception
django_1 | log_response(
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/log.py", line 224, in log_response
django_1 | getattr(logger, level)(
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 1475, in error
django_1 | self._log(ERROR, msg, args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 1589, in _log
django_1 | self.handle(record)
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 1599, in handle
django_1 | self.callHandlers(record)
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 1661, in callHandlers
django_1 | hdlr.handle(record)
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 952, in handle
django_1 | self.emit(record)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/log.py", line 122, in emit
django_1 | self.send_mail(subject, message, fail_silently=True, html_message=html_message)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/log.py", line 125, in send_mail
django_1 | mail.mail_admins(subject, message, *args, connection=self.connection(), **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/mail/__init__.py", line 104, in mail_admins
django_1 | mail.send(fail_silently=fail_silently)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/mail/message.py", line 284, in send
django_1 | return self.get_connection(fail_silently).send_messages([self])
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/base.py", line 94, in send_messages
django_1 | sent = self._send(message)
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/base.py", line 124, in _send
django_1 | response = self.post_to_esp(payload, message)
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/amazon_ses.py", line 67, in post_to_esp
django_1 | response = payload.call_send_api(self.client)
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/amazon_ses.py", line 127, in call_send_api
django_1 | return ses_client.send_raw_email(**self.params)
django_1 | AttributeError: 'NoneType' object has no attribute 'send_raw_email'
I'll appreciate any input. I'm out of ideas.
Thanks!

As you've probably figured out, the problem is boto3 doesn't know what AWS region you're trying to operate in:
botocore.exceptions.NoRegionError: You must specify a region.
The region name comes from boto3 configuration. From Anymail's SES docs:
you must make sure boto3 is configured with AWS credentials having the necessary IAM permissions. There are several ways to do this; see Credentials in the Boto docs for options. Usually, an IAM role for EC2 instances, standard Boto environment variables, or a shared AWS credentials file will be appropriate. For more complex cases, use Anymail’s AMAZON_SES_CLIENT_PARAMS setting to customize the Boto session.
It seems like you might be try to mix some of the "several ways" to provide boto3 credentials, which could be causing confusion.
Note that your AWS_* Django settings don't come into play. boto3 doesn't know about Django settings. Anymail's SES settings docs describe which Django settings Anymail will pass along to boto3. (And AWS_S3_REGION_NAME wouldn't be relevant anyway, because S3 is not the same service as SES. I'm guessing those AWS_* Django settings may be for some other app, maybe django-storages.)
If you want to provide AWS credentials specifically for Anymail in settings.py, you can do that with Anymail's AMAZON_SES_CLIENT_PARAMS setting. E.g.:
# (Be sure to add DJANGO_AWS_REGION_NAME to your env to use this example)
ANYMAIL = {
"AMAZON_SES_CLIENT_PARAMS": {
"aws_access_key_id": env("DJANGO_AWS_ACCESS_KEY_ID"),
"aws_secret_access_key": env("DJANGO_AWS_SECRET_ACCESS_KEY"),
"region_name": env("DJANGO_AWS_REGION_NAME"),
},
}

Related

I'm unable to send emails from Django App on AWS

I'm unable to receive emails from my app after deploying to Aws, My app sits on Ec2 and I host my env on Elastic Bean, The emailing situation works fine locally but not on AWS.
Note: I use Amazon SES
I'm not sure on what to do next at this point, I don't seem to be able to find the problem.
HERE IS MY SETTINGS FOR EMAIL
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = os.getenv('EMAIL_HOST')
EMAIL_PORT = 587
EMAIL_USE_TLS = True
# EMAIL_USE_SSL=True
EMAIL_HOST_USER = os.getenv('EMAIL_HOST_USER')
EMAIL_HOST_PASSWORD = os.getenv('EMAIL_HOST_PASSWORD')
HERE IS MY LOG FILE
django_1 | Exception in thread Thread-184:
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.9/threading.py", line 973, in _bootstrap_inner
django_1 | self.run()
django_1 | File "/build/authentication/utils.py", line 15, in run
django_1 | self.email.send()
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/mail/message.py", line 284, in send
django_1 | return self.get_connection(fail_silently).send_messages([self])
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/mail/backends/smtp.py", line 102, in send_messages
django_1 | new_conn_created = self.open()
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/mail/backends/smtp.py", line 67, in open
django_1 | self.connection.starttls(keyfile=self.ssl_keyfile, certfile=self.ssl_certfile)
django_1 | File "/usr/local/lib/python3.9/smtplib.py", line 769, in starttls
django_1 | self.ehlo_or_helo_if_needed()
django_1 | File "/usr/local/lib/python3.9/smtplib.py", line 611, in ehlo_or_helo_if_needed
django_1 | if not (200 <= self.ehlo()[0] <= 299):
django_1 | File "/usr/local/lib/python3.9/smtplib.py", line 451, in ehlo
django_1 | self.putcmd(self.ehlo_msg, name or self.local_hostname)
django_1 | File "/usr/local/lib/python3.9/smtplib.py", line 378, in putcmd
django_1 | self.send(f'{s}{CRLF}')
django_1 | File "/usr/local/lib/python3.9/smtplib.py", line 365, in send
django_1 | raise SMTPServerDisconnected('please run connect() first')
django_1 | smtplib.SMTPServerDisconnected: please run connect() first
django_1 | [23/Mar/2022 11:29:11] "POST /api/auth/register HTTP/1.1" 201 1246```

Django + Mongo + Docker getting pymongo.errors.ServerSelectionTimeoutError

I've been struggling to get a simple app running using Django, Djongo, Mongo, and Docker Compose. My setup looks like this:
docker-compose.yml
services:
mongodb:
image: mongo:latest
restart: always
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: mongoadmin
MONGO_INITDB_DATABASE: django_mongodb_docker
ports:
- 27017:27017
web:
build: ./src
restart: always
command: python src/manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- 8000:8000
links:
- mongodb
Dockerfile
FROM python:3
ENV PYTHONUNBUFFERED=1
RUN mkdir /code
COPY . /code/
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
settings.py
DATABASES = {
'default': {
'ENGINE': 'djongo',
'HOST': 'mongodb',
'PORT': 27017,
'USER': 'root',
'PASSWORD': 'mongoadmin',
'AUTH_SOURCE': 'admin',
'AUTH_MECHANISM': 'SCRAM-SHA-1',
}
}
What is annoying is that I am able to use pymongo from my web container to connect to the container running mongo. That works as follows.
from pymongo import MongoClient
c = MongoClient(
'mongodb://mongodb:27017',
username='root',
password='mongoadmin',
authSource='admin',
authMechanism='SCRAM-SHA-1')
print(c.server_info())
The issue is that when I go to run migrations from within my web container I get the following error.
Traceback (most recent call last):
File "/code/src/manage.py", line 22, in <module>
main()
File "/code/src/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 330, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 371, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 85, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/migrate.py", line 92, in handle
executor = MigrationExecutor(connection, self.migration_progress_callback)
File "/usr/local/lib/python3.9/site-packages/django/db/migrations/executor.py", line 18, in __init__
self.loader = MigrationLoader(self.connection)
File "/usr/local/lib/python3.9/site-packages/django/db/migrations/loader.py", line 53, in __init__
self.build_graph()
File "/usr/local/lib/python3.9/site-packages/django/db/migrations/loader.py", line 216, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/usr/local/lib/python3.9/site-packages/django/db/migrations/recorder.py", line 77, in applied_migrations
if self.has_table():
File "/usr/local/lib/python3.9/site-packages/django/db/migrations/recorder.py", line 56, in has_table
tables = self.connection.introspection.table_names(cursor)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/introspection.py", line 48, in table_names
return get_names(cursor)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/introspection.py", line 43, in get_names
return sorted(ti.name for ti in self.get_table_list(cursor)
File "/usr/local/lib/python3.9/site-packages/djongo/introspection.py", line 47, in get_table_list
for c in cursor.db_conn.list_collection_names()
File "/usr/local/lib/python3.9/site-packages/pymongo/database.py", line 863, in list_collection_names
for result in self.list_collections(session=session, **kwargs)]
File "/usr/local/lib/python3.9/site-packages/pymongo/database.py", line 825, in list_collections
return self.__client._retryable_read(
File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1460, in _retryable_read
server = self._select_server(
File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1278, in _select_server
server = topology.select_server(server_selector)
File "/usr/local/lib/python3.9/site-packages/pymongo/topology.py", line 241, in select_server
return random.choice(self.select_servers(selector,
File "/usr/local/lib/python3.9/site-packages/pymongo/topology.py", line 199, in select_servers
server_descriptions = self._select_servers_loop(
File "/usr/local/lib/python3.9/site-packages/pymongo/topology.py", line 215, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 5f9ecaa5bbdc0433baa13966, topology_type: Single, servers: [<ServerDescription ('localhost', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:27017: [Errno 111] Connection refused')>]>
In addition, I've tried to create a Djongo model and save it to see if maybe the problem is specific to migrations (essentially just trying to make any connection to mongo using djongo).
models.py
from djongo import models
class Blog(models.Model):
name = models.CharField(max_length=100)
test.py
b = Blog(name='test')
b.save()
That returns the following error:
The above exception was the direct cause of the following exception:
web_1 |
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.9/site-packages/djongo/cursor.py", line 51, in execute
web_1 | self.result = Query(
web_1 | File "/usr/local/lib/python3.9/site-packages/djongo/sql2mongo/query.py", line 783, in __init__
web_1 | self._query = self.parse()
web_1 | File "/usr/local/lib/python3.9/site-packages/djongo/sql2mongo/query.py", line 884, in parse
web_1 | raise exe from e
web_1 | djongo.exceptions.SQLDecodeError:
web_1 |
web_1 | Keyword: None
web_1 | Sub SQL: None
web_1 | FAILED SQL: INSERT INTO "game_blog" ("name") VALUES (%(0)s)
web_1 | Params: ['test']
web_1 | Version: 1.3.3
web_1 |
web_1 | The above exception was the direct cause of the following exception:
web_1 |
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
web_1 | return self.cursor.execute(sql, params)
web_1 | File "/usr/local/lib/python3.9/site-packages/djongo/cursor.py", line 59, in execute
web_1 | raise db_exe from e
web_1 | djongo.database.DatabaseError
web_1 |
web_1 | The above exception was the direct cause of the following exception:
web_1 |
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
web_1 | response = get_response(request)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 179, in _get_response
web_1 | response = wrapped_callback(request, *callback_args, **callback_kwargs)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
web_1 | return self.dispatch(request, *args, **kwargs)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
web_1 | return handler(request, *args, **kwargs)
web_1 | File "/code/src/game/views.py", line 9, in get
web_1 | b.save(using='mongo')
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 753, in save
web_1 | self.save_base(using=using, force_insert=force_insert,
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 790, in save_base
web_1 | updated = self._save_table(
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 895, in _save_table
web_1 | results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 933, in _do_insert
web_1 | return manager._insert(
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
web_1 | return getattr(self.get_queryset(), name)(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1254, in _insert
web_1 | return query.get_compiler(using=using).execute_sql(returning_fields)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1397, in execute_sql
web_1 | cursor.execute(sql, params)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 98, in execute
web_1 | return super().execute(sql, params)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
web_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
web_1 | return executor(sql, params, many, context)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
web_1 | return self.cursor.execute(sql, params)
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
web_1 | raise dj_exc_value.with_traceback(traceback) from exc_value
web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
web_1 | return self.cursor.execute(sql, params)
web_1 | File "/usr/local/lib/python3.9/site-packages/djongo/cursor.py", line 59, in execute
web_1 | raise db_exe from e
web_1 | django.db.utils.DatabaseError
Any help on this would be greatly appreciated.
I have tried editing my mongodb.conf like this already.
According to this document, the settings.py should have a CLIENT section which contains:
A set of key-value pairs that will be passed directly to MongoClient as kwargs while creating a new client connection.
So try setting your settings.py to:
DATABASE = {
'default': {
'ENGINE': 'djongo',
'NAME': 'your-database-name',
'CLIENT': {
'host': 'mongodb://mongodb:27017',
'username': 'root',
'password': 'mongoadmin',
'authSource': 'admin',
'authMechanism': 'SCRAM-SHA-1',
}
}
}

Docker-compose does not work in docker-machine

I have a cookiecutter-django application with docker that works fine if I run it locally with docker compose -f local.yml up. Now I am trying to deploy it so first I created a docker-machine in my computer (using macOS Catalina) and activated it.
Now, inside the docker-machine, the docker-compose build works fine, but when I run it, the application crashes.
Any idea what can be happening? I have been trying to solve this for almost a week now...
This are my logs when I do docker-compose up in the docker-machine:
Creating network "innovacion_innsai_default" with the default driver
Creating innovacion_innsai_postgres_1 ... done
Creating innovacion_innsai_django_1 ... done
Creating innovacion_innsai_node_1 ... done
Attaching to innovacion_innsai_postgres_1, innovacion_innsai_django_1, innovacion_innsai_node_1
postgres_1 | 2020-03-16 08:41:12.472 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-03-16 08:41:12.472 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-03-16 08:41:12.473 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-03-16 08:41:12.494 UTC [21] LOG: database system was shut down at 2020-03-16 08:31:09 UTC
postgres_1 | 2020-03-16 08:41:12.511 UTC [1] LOG: database system is ready to accept connections
django_1 | PostgreSQL is available
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | psycopg2.errors.UndefinedTable: relation "innovation_sector" does not exist
django_1 | LINE 1: ...n_sector"."id", "innovation_sector"."sector" FROM "innovatio...
django_1 | ^
django_1 |
django_1 |
django_1 | The above exception was the direct cause of the following exception:
django_1 |
django_1 | Traceback (most recent call last):
django_1 | File "manage.py", line 30, in <module>
django_1 | execute_from_command_line(sys.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
django_1 | utility.execute()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
django_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
django_1 | self.execute(*args, **cmd_options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 361, in execute
django_1 | self.check()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 390, in check
django_1 | include_deployment_checks=include_deployment_checks,
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 65, in _run_checks
django_1 | issues.extend(super()._run_checks(**kwargs))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 377, in _run_checks
django_1 | return checks.run_checks(**kwargs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks
django_1 | new_errors = check(app_configs=app_configs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
django_1 | all_namespaces = _load_all_namespaces(resolver)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
django_1 | url_patterns = getattr(resolver, 'url_patterns', [])
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 584, in url_patterns
django_1 | patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 577, in urlconf_module
django_1 | return import_module(self.urlconf_name)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/app/config/urls.py", line 18, in <module>
django_1 | path("", include("innovacion_innsai.innovation.urls", namespace="innovation")),
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/conf.py", line 34, in include
django_1 | urlconf_module = import_module(urlconf_module)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/app/innovacion_innsai/innovation/urls.py", line 2, in <module>
django_1 | from innovacion_innsai.innovation import views
django_1 | File "/app/innovacion_innsai/innovation/views.py", line 9, in <module>
django_1 | from .analytics import alimentacion_cases, agro_cases, turismo_cases, movilidad_cases
django_1 | File "/app/innovacion_innsai/innovation/analytics.py", line 17, in <module>
django_1 | for case in Case.objects.filter(sector__sector=sectors[0]):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 308, in __getitem__
django_1 | qs._fetch_all()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all
django_1 | self._result_cache = list(self._iterable_class(self))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 55, in __iter__
django_1 | results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1133, in execute_sql
django_1 | cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 99, in execute
django_1 | return super().execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
django_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
django_1 | return executor(sql, params, many, context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
django_1 | raise dj_exc_value.with_traceback(traceback) from exc_value
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | django.db.utils.ProgrammingError: relation "innovation_sector" does not exist
django_1 | LINE 1: ...n_sector"."id", "innovation_sector"."sector" FROM "innovatio...
django_1 | ^
django_1 |
innovacion_innsai_django_1 exited with code 1
node_1 |
node_1 | > innovacion_innsai#1.1.0 dev /app
node_1 | > gulp
node_1 |
node_1 | [08:41:22] Using gulpfile /app/gulpfile.js
node_1 | [08:41:22] Starting 'default'...
node_1 | [08:41:22] Starting 'styles'...
node_1 | [08:41:22] Starting 'scripts'...
node_1 | [08:41:22] Starting 'imgCompression'...
node_1 | [08:41:22] gulp-imagemin: Minified 0 images
This is my local.yml:
version: '3'
volumes:
local_postgres_data: {}
local_postgres_data_backups: {}
services:
django:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: innovacion_innsai_local_django
depends_on:
- postgres
volumes:
- .:/app
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.postgres
ports:
- "8000:8000"
command: /start
postgres:
build:
context: .
dockerfile: ./compose/production/postgres/Dockerfile
image: innovacion_innsai_production_postgres
volumes:
- local_postgres_data:/var/lib/postgresql/data
- local_postgres_data_backups:/backups
env_file:
- ./.envs/.local/.postgres
#Estas dos siguientes lineas las he añadido yo luego
ports:
- "5432:5432"
node:
build:
context: .
dockerfile: ./compose/local/node/Dockerfile
image: innovacion_innsai_local_node
depends_on:
- django
volumes:
- .:/app
# http://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html
- /app/node_modules
command: npm run dev
ports:
- "3000:3000"
# Expose browsersync UI: https://www.browsersync.io/docs/options/#option-ui
- "3001:3001"
And this is my .postgres file inside the .envs/local folder:
# PostgreSQL
# ------------------------------------------------------------------------------
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_DB=innovacion_innsai
POSTGRES_USER=debug
POSTGRES_PASSWORD=debug
File "/app/innovacion_innsai/innovation/analytics.py", line 17, in <module>
django_1 | for case in Case.objects.filter(sector__sector=sectors[0]):
It means that when django is initializing (the moment it loads the modules) it's doing DB queries on database that has no migrations applied. You need to transform whatever your code is doing on line 17 to run it lazy.

Problem with docker-compose up for django cookiecutter ->Django Error: relation "users_user" does not exist

I'm using Django Cookiecutter template to my project and when I build docker image with docker-compose every think is fine, but when I execute
docker-compose -f local.yml up
I get this output
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default timezone ... Etc/UTC
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | creating configuration files ... ok
postgres_1 | running bootstrap script ... ok
mailhog_1 | 2020/02/22 16:26:28 Using in-memory storage
mailhog_1 | 2020/02/22 16:26:28 [SMTP] Binding to address: 0.0.0.0:1025
mailhog_1 | 2020/02/22 16:26:28 Serving under http://0.0.0.0:8025/
mailhog_1 | [HTTP] Binding to address: 0.0.0.0:8025
mailhog_1 | Creating API v1 with WebPath:
mailhog_1 | Creating API v2 with WebPath:
django_1 | Waiting for PostgreSQL to become available...
postgres_1 | performing post-bootstrap initialization ... ok
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 |
postgres_1 | WARNING: enabling "trust" authentication for local connections
postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run initdb.
postgres_1 | waiting for server to start....2020-02-22 16:26:29.864 UTC [42] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-02-22 16:26:29.876 UTC [43] LOG: database system was shut down at 2020-02-22 16:26:29 UTC
postgres_1 | 2020-02-22 16:26:29.880 UTC [42] LOG: database system is ready to accept connections
postgres_1 | done
postgres_1 | server started
django_1 | Waiting for PostgreSQL to become available...
postgres_1 | CREATE DATABASE
postgres_1 |
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1 |
postgres_1 | 2020-02-22 16:26:30.416 UTC [42] LOG: received fast shutdown request
postgres_1 | waiting for server to shut down....2020-02-22 16:26:30.417 UTC [42] LOG: aborting any active transactions
postgres_1 | 2020-02-22 16:26:30.421 UTC [42] LOG: background worker "logical replication launcher" (PID 49) exited with exit code 1
postgres_1 | 2020-02-22 16:26:30.423 UTC [44] LOG: shutting down
postgres_1 | 2020-02-22 16:26:30.432 UTC [42] LOG: database system is shut down
postgres_1 | done
postgres_1 | server stopped
postgres_1 |
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2020-02-22 16:26:30.535 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-02-22 16:26:30.535 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-02-22 16:26:30.537 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-02-22 16:26:30.548 UTC [60] LOG: database system was shut down at 2020-02-22 16:26:30 UTC
postgres_1 | 2020-02-22 16:26:30.552 UTC [1] LOG: database system is ready to accept connections
django_1 | PostgreSQL is available
postgres_1 | 2020-02-22 16:26:32.030 UTC [68] ERROR: relation "users_user" does not exist at character 34
postgres_1 | 2020-02-22 16:26:32.030 UTC [68] STATEMENT: SELECT "users_user"."email" FROM "users_user" ORDER BY "users_user"."email" ASC
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | psycopg2.errors.UndefinedTable: relation "users_user" does not exist
django_1 | LINE 1: SELECT "users_user"."email" FROM "users_user" ORDER BY "user...
django_1 | ^
django_1 |
django_1 |
django_1 | The above exception was the direct cause of the following exception:
django_1 |
django_1 | Traceback (most recent call last):
django_1 | File "manage.py", line 30, in <module>
django_1 | execute_from_command_line(sys.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
django_1 | utility.execute()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
django_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
django_1 | self.execute(*args, **cmd_options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 361, in execute
django_1 | self.check()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 390, in check
django_1 | include_deployment_checks=include_deployment_checks,
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 65, in _run_checks
django_1 | issues.extend(super()._run_checks(**kwargs))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 377, in _run_checks
django_1 | return checks.run_checks(**kwargs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks
django_1 | new_errors = check(app_configs=app_configs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
django_1 | all_namespaces = _load_all_namespaces(resolver)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
django_1 | url_patterns = getattr(resolver, 'url_patterns', [])
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 579, in url_patterns
django_1 | patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 572, in urlconf_module
django_1 | return import_module(self.urlconf_name)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/app/config/urls.py", line 8, in <module>
django_1 | from myapp.users.views import login_view, set_password_view
django_1 | File "/app/myapp/users/views.py", line 24, in <module>
django_1 | from myapp.users.forms import (
django_1 | File "/app/myapp/users/forms.py", line 328, in <module>
django_1 | class InviteForm(forms.Form):
django_1 | File "/app/myapp/users/forms.py", line 331, in InviteForm
django_1 | required=False)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/forms/fields.py", line 767, in __init__
django_1 | self.choices = choices
django_1 | File "/usr/local/lib/python3.7/site-packages/django/forms/fields.py", line 784, in _set_choices
django_1 | value = list(value)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 274, in __iter__
django_1 | self._fetch_all()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all
django_1 | self._result_cache = list(self._iterable_class(self))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 144, in __iter__
django_1 | return compiler.results_iter(tuple_expected=True, chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1052, in results_iter
django_1 | results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1100, in execute_sql
django_1 | cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 99, in execute
django_1 | return super().execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
django_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
django_1 | return executor(sql, params, many, context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
django_1 | raise dj_exc_value.with_traceback(traceback) from exc_value
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | django.db.utils.ProgrammingError: relation "users_user" does not exist
django_1 | LINE 1: SELECT "users_user"."email" FROM "users_user" ORDER BY "user...
django_1 | ^
django_1 |
myapp_django_1 exited with code 1
I tried also run
docker-compose -f local.yml run django python manage.py migrate
first, but I get same django error.
If you need code from other files please comment and I will update then this question content.
UPDATE
Here's fragment of code's which can be cause of problem.
user/forms.py
class EmailListField(forms.MultipleChoiceField):
def validate(self, value):
if self.required and not value:
raise forms.ValidationError(self.error_messages['required'], code=['required'])
email_validator = EmailValidator()
for val in value:
try:
email_validator(val)
except forms.ValidationError:
raise forms.ValidationError(
self.error_messages['invalid_choice'],
code='invalid_choice',
params={'value': val}
)
class InviteForm(forms.Form):
emails = EmailListField(choices=User.objects.all().values_list('email'),
widget=forms.SelectMultiple(),
required=False)
groups = forms.ModelMultipleChoiceField(queryset=Group.objects.none(), required=False)
file = forms.FileField(required=False)
def __init__(self, *args, **kwargs):
from_user = kwargs.pop('from_user')
super().__init__(*args, **kwargs)
self.fields['groups'].queryset = Group.objects.filter(company=from_user.profile.company)
self.from_user = from_user
def clean(self):
if not (self.cleaned_data['emails'] or self.cleaned_data['file']):
raise forms.ValidationError('Wprowadź adresy w polu tekstowy lub prześlij plik')
return self.cleaned_data
You can try use ModelMultipleChoiceField (https://docs.djangoproject.com/en/2.2/ref/forms/fields/#django.forms.ModelMultipleChoiceField) instead of MultipleChoiceField and then change choices to queryset in EmailListField, i.e.
class EmailListField(forms.ModelMultipleChoiceField):
def validate(self, value): (...)
class InviteForm(forms.Form):
emails = EmailListField(queryset=User.objects.all().values_list('email'),
widget=forms.SelectMultiple(),
required=False)
(...)

Django-Channels fails while giving name or service unknown error

I've a fairly simple django-channels/daphne/asgi/redis app. The app never has any problem when using chrome but fails miserably when we use firefox (latest version ~ 60). I currently have the following error:
api_1 | 2018-06-12 16:02:24,378 INFO In the connect method
api_1 | xxxxxx:41647 - - [12/Jun/2018:16:02:24] "WSCONNECT /updates/" - -
api_1 | 2018-06-12 16:02:25,049 ERROR Exception inside application: [Errno -2] Name or service not known
api_1 | File "/usr/local/lib/python3.6/site-packages/channels/consumer.py", line 54, in __call__
api_1 | await await_many_dispatch([receive, self.channel_receive], self.dispatch)
api_1 | File "/usr/local/lib/python3.6/site-packages/channels/utils.py", line 48, in await_many_dispatch
api_1 | await dispatch(result)
api_1 | File "/usr/local/lib/python3.6/site-packages/asgiref/sync.py", line 110, in __call__
api_1 | return await asyncio.wait_for(future, timeout=None)
api_1 | File "/usr/local/lib/python3.6/asyncio/tasks.py", line 339, in wait_for
api_1 | return (yield from fut)
api_1 | File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
api_1 | result = self.fn(*self.args, **self.kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/channels/db.py", line 13, in thread_handler
api_1 | return super().thread_handler(loop, *args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/asgiref/sync.py", line 125, in thread_handler
api_1 | return self.func(*args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/channels/consumer.py", line 99, in dispatch
api_1 | handler(message)
api_1 | File "/usr/local/lib/python3.6/site-packages/channels/generic/websocket.py", line 19, in websocket_connect
api_1 | self.connect()
api_1 | File "./flypoll/consumers.py", line 146, in connect
api_1 | async_to_sync(self.channel_layer.group_add)("flypoll_socket_clients", self.channel_name)
api_1 | File "/usr/local/lib/python3.6/site-packages/asgiref/sync.py", line 64, in __call__
api_1 | return call_result.result()
api_1 | File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 432, in result
api_1 | return self.__get_result()
api_1 | File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
api_1 | raise self._exception
api_1 | File "/usr/local/lib/python3.6/site-packages/asgiref/sync.py", line 78, in main_wrap
api_1 | result = await self.awaitable(*args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/channels_redis/core.py", line 283, in group_add
api_1 | async with self.connection(self.consistent_hash(group)) as connection:
api_1 | File "/usr/local/lib/python3.6/site-packages/channels_redis/core.py", line 403, in __aenter__
api_1 | self.conn = await aioredis.create_redis(**self.kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/aioredis/commands/__init__.py", line 174, in create_redis
api_1 | loop=loop)
api_1 | File "/usr/local/lib/python3.6/site-packages/aioredis/connection.py", line 107, in create_connection
api_1 | timeout, loop=loop)
api_1 | File "/usr/local/lib/python3.6/asyncio/tasks.py", line 339, in wait_for
api_1 | return (yield from fut)
api_1 | File "/usr/local/lib/python3.6/site-packages/aioredis/stream.py", line 19, in open_connection
api_1 | lambda: protocol, host, port, **kwds)
api_1 | File "/usr/local/lib/python3.6/asyncio/base_events.py", line 734, in create_connection
api_1 | infos = f1.result()
api_1 | File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
api_1 | result = self.fn(*self.args, **self.kwargs)
api_1 | File "/usr/local/lib/python3.6/socket.py", line 745, in getaddrinfo
api_1 | for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
api_1 | [Errno -2] Name or service not known
It says there is a problem with connect method but the connect method is as follows:
def connect(self):
"""
standard connect handler, accepts the connection, and sets up some basic parameters
:return:
"""
logging.info("In the connect method")
self.accept()
async_to_sync(self.channel_layer.group_add)("flypoll_socket_clients", self.channel_name)
The app is dockerized and is hosted on ec2. The redis, python, django are dockerized too. I've updated the modules (asgiref, txaio, twisted, channels, daphne etc) to latest versions but i'm not able to get rid of this.
What could be wrong?
It looks like your django container can't access redis.
In your django settings you must have something like this :
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": [('[redis-host-here]', 6379)], # REPLACE redis host here
},
},
}
And that host must be accessible from your django container