Related
I am trying to deploy to AWS(EC2) a Cookiecutter Django project.
The AWS user with this credentials has ful S3, SES and SNS policies.
The EC2 server has also a role with full SES/S3 policies.
In production file in envs I have the keys set up like this.
DJANGO_AWS_ACCESS_KEY_ID=xxxxxxxxx
DJANGO_AWS_SECRET_ACCESS_KEY=xxxxxxxxxx
DJANGO_AWS_STORAGE_BUCKET_NAME=xxxxxxxxxx
In settings I have
AWS_S3_REGION_NAME = env("DJANGO_AWS_S3_REGION_NAME", default=None)
AWS_ACCESS_KEY_ID = env("DJANGO_AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = env("DJANGO_AWS_SECRET_ACCESS_KEY")
AWS_STORAGE_BUCKET_NAME = env("DJANGO_AWS_STORAGE_BUCKET_NAME")
EMAIL_BACKEND = "anymail.backends.amazon_ses.EmailBackend"
ANYMAIL = {}
All nice and fine until the project tries to send an email using SES and it crashes with the error bellow .
Until now I have tried:
adding DJANGO_AWS_S3_REGION_NAME to the production file in envs - no result
adding the region in aws config using aws cli - no result
overriding the settings in ANYMAIL ={} with credetials and region - no result
making a blank project, just adding the aws credentials and not changing anything else - no result
creating manually on another project a boto3.session.client with the same credentials and sending a mail - it works
This is the error. The second part with 'NoneType' object has no attribute 'send_raw_email' repeats a lot after this.
django_1 | [2021-08-13 13:58:14 +0000] [12] [ERROR] Error handling request /accounts/signup/
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
django_1 | response = get_response(request)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
django_1 | response = wrapped_callback(request, *callback_args, **callback_kwargs)
django_1 | File "/usr/local/lib/python3.9/contextlib.py", line 79, in inner
django_1 | return func(*args, **kwds)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
django_1 | return self.dispatch(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
django_1 | return bound_method(*args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/views/decorators/debug.py", line 89, in sensitive_post_parameters_wrapper
django_1 | return view(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 230, in dispatch
django_1 | return super(SignupView, self).dispatch(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 74, in dispatch
django_1 | response = super(RedirectAuthenticatedUserMixin, self).dispatch(
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 204, in dispatch
django_1 | return super(CloseableSignupMixin, self).dispatch(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
django_1 | return handler(request, *args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 102, in post
django_1 | response = self.form_valid(form)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 248, in form_valid
django_1 | return complete_signup(
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/utils.py", line 209, in complete_signup
django_1 | return perform_login(
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/utils.py", line 175, in perform_login
django_1 | send_email_confirmation(request, user, signup=signup, email=email)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/utils.py", line 346, in send_email_confirmation
django_1 | email_address.send_confirmation(request, signup=signup)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/models.py", line 62, in send_confirmation
django_1 | confirmation.send(request, signup=signup)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/models.py", line 169, in send
django_1 | get_adapter(request).send_confirmation_mail(request, self, signup)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/adapter.py", line 464, in send_confirmation_mail
django_1 | self.send_mail(email_template, emailconfirmation.email_address.email, ctx)
django_1 | File "/usr/local/lib/python3.9/site-packages/allauth/account/adapter.py", line 136, in send_mail
django_1 | msg.send()
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/mail/message.py", line 284, in send
django_1 | return self.get_connection(fail_silently).send_messages([self])
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/base.py", line 89, in send_messages
django_1 | created_session = self.open()
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/amazon_ses.py", line 44, in open
django_1 | self.client = boto3.session.Session(**self.session_params).client("ses", **self.client_params)
django_1 | File "/usr/local/lib/python3.9/site-packages/boto3/session.py", line 258, in client
django_1 | return self._session.create_client(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/session.py", line 847, in create_client
django_1 | client = client_creator.create_client(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 86, in create_client
django_1 | client_args = self._get_client_args(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 355, in _get_client_args
django_1 | return args_creator.get_client_args(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/args.py", line 71, in get_client_args
django_1 | final_args = self.compute_client_args(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/args.py", line 148, in compute_client_args
django_1 | endpoint_config = self._compute_endpoint_config(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/args.py", line 220, in _compute_endpoint_config
django_1 | return self._resolve_endpoint(**resolve_endpoint_kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/args.py", line 302, in _resolve_endpoint
django_1 | return endpoint_bridge.resolve(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 430, in resolve
django_1 | resolved = self.endpoint_resolver.construct_endpoint(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/regions.py", line 133, in construct_endpoint
django_1 | result = self._endpoint_for_partition(
django_1 | File "/usr/local/lib/python3.9/site-packages/botocore/regions.py", line 148, in _endpoint_for_partition
django_1 | raise NoRegionError()
django_1 | botocore.exceptions.NoRegionError: You must specify a region.
django_1 |
django_1 | During handling of the above exception, another exception occurred:
django_1 |
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
django_1 | response = get_response(request)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/deprecation.py", line 114, in __call__
django_1 | response = response or self.get_response(request)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 49, in inner
django_1 | response = response_for_exception(request, exc)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 104, in response_for_exception
django_1 | log_response(
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/log.py", line 224, in log_response
django_1 | getattr(logger, level)(
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 1475, in error
django_1 | self._log(ERROR, msg, args, **kwargs)
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 1589, in _log
django_1 | self.handle(record)
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 1599, in handle
django_1 | self.callHandlers(record)
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 1661, in callHandlers
django_1 | hdlr.handle(record)
django_1 | File "/usr/local/lib/python3.9/logging/__init__.py", line 952, in handle
django_1 | self.emit(record)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/log.py", line 122, in emit
django_1 | self.send_mail(subject, message, fail_silently=True, html_message=html_message)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/utils/log.py", line 125, in send_mail
django_1 | mail.mail_admins(subject, message, *args, connection=self.connection(), **kwargs)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/mail/__init__.py", line 104, in mail_admins
django_1 | mail.send(fail_silently=fail_silently)
django_1 | File "/usr/local/lib/python3.9/site-packages/django/core/mail/message.py", line 284, in send
django_1 | return self.get_connection(fail_silently).send_messages([self])
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/base.py", line 94, in send_messages
django_1 | sent = self._send(message)
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/base.py", line 124, in _send
django_1 | response = self.post_to_esp(payload, message)
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/amazon_ses.py", line 67, in post_to_esp
django_1 | response = payload.call_send_api(self.client)
django_1 | File "/usr/local/lib/python3.9/site-packages/anymail/backends/amazon_ses.py", line 127, in call_send_api
django_1 | return ses_client.send_raw_email(**self.params)
django_1 | AttributeError: 'NoneType' object has no attribute 'send_raw_email'
I'll appreciate any input. I'm out of ideas.
Thanks!
As you've probably figured out, the problem is boto3 doesn't know what AWS region you're trying to operate in:
botocore.exceptions.NoRegionError: You must specify a region.
The region name comes from boto3 configuration. From Anymail's SES docs:
you must make sure boto3 is configured with AWS credentials having the necessary IAM permissions. There are several ways to do this; see Credentials in the Boto docs for options. Usually, an IAM role for EC2 instances, standard Boto environment variables, or a shared AWS credentials file will be appropriate. For more complex cases, use Anymail’s AMAZON_SES_CLIENT_PARAMS setting to customize the Boto session.
It seems like you might be try to mix some of the "several ways" to provide boto3 credentials, which could be causing confusion.
Note that your AWS_* Django settings don't come into play. boto3 doesn't know about Django settings. Anymail's SES settings docs describe which Django settings Anymail will pass along to boto3. (And AWS_S3_REGION_NAME wouldn't be relevant anyway, because S3 is not the same service as SES. I'm guessing those AWS_* Django settings may be for some other app, maybe django-storages.)
If you want to provide AWS credentials specifically for Anymail in settings.py, you can do that with Anymail's AMAZON_SES_CLIENT_PARAMS setting. E.g.:
# (Be sure to add DJANGO_AWS_REGION_NAME to your env to use this example)
ANYMAIL = {
"AMAZON_SES_CLIENT_PARAMS": {
"aws_access_key_id": env("DJANGO_AWS_ACCESS_KEY_ID"),
"aws_secret_access_key": env("DJANGO_AWS_SECRET_ACCESS_KEY"),
"region_name": env("DJANGO_AWS_REGION_NAME"),
},
}
I have a cookiecutter-django application with docker that works fine if I run it locally with docker compose -f local.yml up. Now I am trying to deploy it so first I created a docker-machine in my computer (using macOS Catalina) and activated it.
Now, inside the docker-machine, the docker-compose build works fine, but when I run it, the application crashes.
Any idea what can be happening? I have been trying to solve this for almost a week now...
This are my logs when I do docker-compose up in the docker-machine:
Creating network "innovacion_innsai_default" with the default driver
Creating innovacion_innsai_postgres_1 ... done
Creating innovacion_innsai_django_1 ... done
Creating innovacion_innsai_node_1 ... done
Attaching to innovacion_innsai_postgres_1, innovacion_innsai_django_1, innovacion_innsai_node_1
postgres_1 | 2020-03-16 08:41:12.472 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-03-16 08:41:12.472 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-03-16 08:41:12.473 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-03-16 08:41:12.494 UTC [21] LOG: database system was shut down at 2020-03-16 08:31:09 UTC
postgres_1 | 2020-03-16 08:41:12.511 UTC [1] LOG: database system is ready to accept connections
django_1 | PostgreSQL is available
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | psycopg2.errors.UndefinedTable: relation "innovation_sector" does not exist
django_1 | LINE 1: ...n_sector"."id", "innovation_sector"."sector" FROM "innovatio...
django_1 | ^
django_1 |
django_1 |
django_1 | The above exception was the direct cause of the following exception:
django_1 |
django_1 | Traceback (most recent call last):
django_1 | File "manage.py", line 30, in <module>
django_1 | execute_from_command_line(sys.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
django_1 | utility.execute()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
django_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
django_1 | self.execute(*args, **cmd_options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 361, in execute
django_1 | self.check()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 390, in check
django_1 | include_deployment_checks=include_deployment_checks,
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 65, in _run_checks
django_1 | issues.extend(super()._run_checks(**kwargs))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 377, in _run_checks
django_1 | return checks.run_checks(**kwargs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks
django_1 | new_errors = check(app_configs=app_configs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
django_1 | all_namespaces = _load_all_namespaces(resolver)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
django_1 | url_patterns = getattr(resolver, 'url_patterns', [])
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 584, in url_patterns
django_1 | patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 577, in urlconf_module
django_1 | return import_module(self.urlconf_name)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/app/config/urls.py", line 18, in <module>
django_1 | path("", include("innovacion_innsai.innovation.urls", namespace="innovation")),
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/conf.py", line 34, in include
django_1 | urlconf_module = import_module(urlconf_module)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/app/innovacion_innsai/innovation/urls.py", line 2, in <module>
django_1 | from innovacion_innsai.innovation import views
django_1 | File "/app/innovacion_innsai/innovation/views.py", line 9, in <module>
django_1 | from .analytics import alimentacion_cases, agro_cases, turismo_cases, movilidad_cases
django_1 | File "/app/innovacion_innsai/innovation/analytics.py", line 17, in <module>
django_1 | for case in Case.objects.filter(sector__sector=sectors[0]):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 308, in __getitem__
django_1 | qs._fetch_all()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all
django_1 | self._result_cache = list(self._iterable_class(self))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 55, in __iter__
django_1 | results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1133, in execute_sql
django_1 | cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 99, in execute
django_1 | return super().execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
django_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
django_1 | return executor(sql, params, many, context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
django_1 | raise dj_exc_value.with_traceback(traceback) from exc_value
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | django.db.utils.ProgrammingError: relation "innovation_sector" does not exist
django_1 | LINE 1: ...n_sector"."id", "innovation_sector"."sector" FROM "innovatio...
django_1 | ^
django_1 |
innovacion_innsai_django_1 exited with code 1
node_1 |
node_1 | > innovacion_innsai#1.1.0 dev /app
node_1 | > gulp
node_1 |
node_1 | [08:41:22] Using gulpfile /app/gulpfile.js
node_1 | [08:41:22] Starting 'default'...
node_1 | [08:41:22] Starting 'styles'...
node_1 | [08:41:22] Starting 'scripts'...
node_1 | [08:41:22] Starting 'imgCompression'...
node_1 | [08:41:22] gulp-imagemin: Minified 0 images
This is my local.yml:
version: '3'
volumes:
local_postgres_data: {}
local_postgres_data_backups: {}
services:
django:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: innovacion_innsai_local_django
depends_on:
- postgres
volumes:
- .:/app
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.postgres
ports:
- "8000:8000"
command: /start
postgres:
build:
context: .
dockerfile: ./compose/production/postgres/Dockerfile
image: innovacion_innsai_production_postgres
volumes:
- local_postgres_data:/var/lib/postgresql/data
- local_postgres_data_backups:/backups
env_file:
- ./.envs/.local/.postgres
#Estas dos siguientes lineas las he añadido yo luego
ports:
- "5432:5432"
node:
build:
context: .
dockerfile: ./compose/local/node/Dockerfile
image: innovacion_innsai_local_node
depends_on:
- django
volumes:
- .:/app
# http://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html
- /app/node_modules
command: npm run dev
ports:
- "3000:3000"
# Expose browsersync UI: https://www.browsersync.io/docs/options/#option-ui
- "3001:3001"
And this is my .postgres file inside the .envs/local folder:
# PostgreSQL
# ------------------------------------------------------------------------------
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_DB=innovacion_innsai
POSTGRES_USER=debug
POSTGRES_PASSWORD=debug
File "/app/innovacion_innsai/innovation/analytics.py", line 17, in <module>
django_1 | for case in Case.objects.filter(sector__sector=sectors[0]):
It means that when django is initializing (the moment it loads the modules) it's doing DB queries on database that has no migrations applied. You need to transform whatever your code is doing on line 17 to run it lazy.
I'm using Django Cookiecutter template to my project and when I build docker image with docker-compose every think is fine, but when I execute
docker-compose -f local.yml up
I get this output
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default timezone ... Etc/UTC
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | creating configuration files ... ok
postgres_1 | running bootstrap script ... ok
mailhog_1 | 2020/02/22 16:26:28 Using in-memory storage
mailhog_1 | 2020/02/22 16:26:28 [SMTP] Binding to address: 0.0.0.0:1025
mailhog_1 | 2020/02/22 16:26:28 Serving under http://0.0.0.0:8025/
mailhog_1 | [HTTP] Binding to address: 0.0.0.0:8025
mailhog_1 | Creating API v1 with WebPath:
mailhog_1 | Creating API v2 with WebPath:
django_1 | Waiting for PostgreSQL to become available...
postgres_1 | performing post-bootstrap initialization ... ok
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 |
postgres_1 | WARNING: enabling "trust" authentication for local connections
postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run initdb.
postgres_1 | waiting for server to start....2020-02-22 16:26:29.864 UTC [42] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-02-22 16:26:29.876 UTC [43] LOG: database system was shut down at 2020-02-22 16:26:29 UTC
postgres_1 | 2020-02-22 16:26:29.880 UTC [42] LOG: database system is ready to accept connections
postgres_1 | done
postgres_1 | server started
django_1 | Waiting for PostgreSQL to become available...
postgres_1 | CREATE DATABASE
postgres_1 |
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1 |
postgres_1 | 2020-02-22 16:26:30.416 UTC [42] LOG: received fast shutdown request
postgres_1 | waiting for server to shut down....2020-02-22 16:26:30.417 UTC [42] LOG: aborting any active transactions
postgres_1 | 2020-02-22 16:26:30.421 UTC [42] LOG: background worker "logical replication launcher" (PID 49) exited with exit code 1
postgres_1 | 2020-02-22 16:26:30.423 UTC [44] LOG: shutting down
postgres_1 | 2020-02-22 16:26:30.432 UTC [42] LOG: database system is shut down
postgres_1 | done
postgres_1 | server stopped
postgres_1 |
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2020-02-22 16:26:30.535 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-02-22 16:26:30.535 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-02-22 16:26:30.537 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-02-22 16:26:30.548 UTC [60] LOG: database system was shut down at 2020-02-22 16:26:30 UTC
postgres_1 | 2020-02-22 16:26:30.552 UTC [1] LOG: database system is ready to accept connections
django_1 | PostgreSQL is available
postgres_1 | 2020-02-22 16:26:32.030 UTC [68] ERROR: relation "users_user" does not exist at character 34
postgres_1 | 2020-02-22 16:26:32.030 UTC [68] STATEMENT: SELECT "users_user"."email" FROM "users_user" ORDER BY "users_user"."email" ASC
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | psycopg2.errors.UndefinedTable: relation "users_user" does not exist
django_1 | LINE 1: SELECT "users_user"."email" FROM "users_user" ORDER BY "user...
django_1 | ^
django_1 |
django_1 |
django_1 | The above exception was the direct cause of the following exception:
django_1 |
django_1 | Traceback (most recent call last):
django_1 | File "manage.py", line 30, in <module>
django_1 | execute_from_command_line(sys.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
django_1 | utility.execute()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
django_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
django_1 | self.execute(*args, **cmd_options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 361, in execute
django_1 | self.check()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 390, in check
django_1 | include_deployment_checks=include_deployment_checks,
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 65, in _run_checks
django_1 | issues.extend(super()._run_checks(**kwargs))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 377, in _run_checks
django_1 | return checks.run_checks(**kwargs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks
django_1 | new_errors = check(app_configs=app_configs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
django_1 | all_namespaces = _load_all_namespaces(resolver)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
django_1 | url_patterns = getattr(resolver, 'url_patterns', [])
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 579, in url_patterns
django_1 | patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 572, in urlconf_module
django_1 | return import_module(self.urlconf_name)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/app/config/urls.py", line 8, in <module>
django_1 | from myapp.users.views import login_view, set_password_view
django_1 | File "/app/myapp/users/views.py", line 24, in <module>
django_1 | from myapp.users.forms import (
django_1 | File "/app/myapp/users/forms.py", line 328, in <module>
django_1 | class InviteForm(forms.Form):
django_1 | File "/app/myapp/users/forms.py", line 331, in InviteForm
django_1 | required=False)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/forms/fields.py", line 767, in __init__
django_1 | self.choices = choices
django_1 | File "/usr/local/lib/python3.7/site-packages/django/forms/fields.py", line 784, in _set_choices
django_1 | value = list(value)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 274, in __iter__
django_1 | self._fetch_all()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all
django_1 | self._result_cache = list(self._iterable_class(self))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 144, in __iter__
django_1 | return compiler.results_iter(tuple_expected=True, chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1052, in results_iter
django_1 | results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1100, in execute_sql
django_1 | cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 99, in execute
django_1 | return super().execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
django_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
django_1 | return executor(sql, params, many, context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
django_1 | raise dj_exc_value.with_traceback(traceback) from exc_value
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | django.db.utils.ProgrammingError: relation "users_user" does not exist
django_1 | LINE 1: SELECT "users_user"."email" FROM "users_user" ORDER BY "user...
django_1 | ^
django_1 |
myapp_django_1 exited with code 1
I tried also run
docker-compose -f local.yml run django python manage.py migrate
first, but I get same django error.
If you need code from other files please comment and I will update then this question content.
UPDATE
Here's fragment of code's which can be cause of problem.
user/forms.py
class EmailListField(forms.MultipleChoiceField):
def validate(self, value):
if self.required and not value:
raise forms.ValidationError(self.error_messages['required'], code=['required'])
email_validator = EmailValidator()
for val in value:
try:
email_validator(val)
except forms.ValidationError:
raise forms.ValidationError(
self.error_messages['invalid_choice'],
code='invalid_choice',
params={'value': val}
)
class InviteForm(forms.Form):
emails = EmailListField(choices=User.objects.all().values_list('email'),
widget=forms.SelectMultiple(),
required=False)
groups = forms.ModelMultipleChoiceField(queryset=Group.objects.none(), required=False)
file = forms.FileField(required=False)
def __init__(self, *args, **kwargs):
from_user = kwargs.pop('from_user')
super().__init__(*args, **kwargs)
self.fields['groups'].queryset = Group.objects.filter(company=from_user.profile.company)
self.from_user = from_user
def clean(self):
if not (self.cleaned_data['emails'] or self.cleaned_data['file']):
raise forms.ValidationError('Wprowadź adresy w polu tekstowy lub prześlij plik')
return self.cleaned_data
You can try use ModelMultipleChoiceField (https://docs.djangoproject.com/en/2.2/ref/forms/fields/#django.forms.ModelMultipleChoiceField) instead of MultipleChoiceField and then change choices to queryset in EmailListField, i.e.
class EmailListField(forms.ModelMultipleChoiceField):
def validate(self, value): (...)
class InviteForm(forms.Form):
emails = EmailListField(queryset=User.objects.all().values_list('email'),
widget=forms.SelectMultiple(),
required=False)
(...)
I just want to connect my local oracle db with my django project but my database credential is not working. Actually, I'm able to connect my oracle database via sql developer with that credential:
I just used that credential in django settings_py like that
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'INTERNAL',
'USER': 'system',
'PASSWORD': 'oracle',
'HOST':'localhost/xe',
'PORT':'1521'
}
}
and error is:
Traceback (most recent call last):
web_1 | File "manage.py", line 22, in <module>
web_1 | execute_from_command_line(sys.argv)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
web_1 | utility.execute()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
web_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
web_1 | self.execute(*args, **cmd_options)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
web_1 | output = self.handle(*args, **options)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/makemigrations.py", line 110, in handle
web_1 | loader.check_consistent_history(connection)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/migrations/loader.py", line 282, in check_consistent_history
web_1 | applied = recorder.applied_migrations()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 65, in applied_migrations
web_1 | self.ensure_schema()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 52, in ensure_schema
web_1 | if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 254, in cursor
web_1 | return self._cursor()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 229, in _cursor
web_1 | self.ensure_connection()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
web_1 | self.connect()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
web_1 | six.reraise(dj_exc_type, dj_exc_value, traceback)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
web_1 | raise value.with_traceback(tb)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
web_1 | self.connect()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 189, in connect
web_1 | self.connection = self.get_new_connection(conn_params)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/oracle/base.py", line 212, in get_new_connection
web_1 | return Database.connect(self._connect_string(), **conn_params)
web_1 | django.db.utils.DatabaseError: ORA-12545: Connect failed because target host or object does not exist
here its my listener status
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC_FOR_XE)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.2.0 - Production
Start Date 28-DEC-2017 15:51:21
Uptime 0 days 2 hr. 8 min. 36 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Default Service XE
Listener Parameter File /u01/app/oracle/product/11.2.0/xe/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/e48c7c272f44/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC_FOR_XE)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=e48c7c272f44)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=e48c7c272f44)(PORT=8080))(Presentation=HTTP)(Session=RAW))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "XE" has 1 instance(s).
Instance "XE", status READY, has 1 handler(s) for this service...
Service "XEXDB" has 1 instance(s).
Instance "XE", status READY, has 1 handler(s) for this service...
here its my listener status
You should change HOST to localhost' or '127.0.0.1 and SID is NAME.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'xe',
'USER': 'system',
'PASSWORD': 'oracle',
'HOST':'127.0.0.1',
'PORT':'1521'
}
}
For future references, if Oracle is configured with Service name instead of SID, then the configuration would be:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': '127.0.0.1:1521/service.name',
'USER': 'system',
'PASSWORD': 'oracle',
}
}
Another thing to consider when working with Oracle in Django is that when you connect to Other Users (schema) database, you have to set db_table Meta option in Django models:
class OracleTable(models.Model):
... fields ...
class Meta:
db_table = '\"OTHERUSER\".\"ORACLETABLE\"'
I just want to connect my local oracle db with my django project but my database credential is not working. Actually, I'm able to connect my oracle database via sql developer with that credential:
I just used that credential in django settings_py like that
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'INTERNAL',
'USER': 'system',
'PASSWORD': 'oracle',
'HOST':'localhost/xe',
'PORT':'1521'
}
}
and error is:
Traceback (most recent call last):
web_1 | File "manage.py", line 22, in <module>
web_1 | execute_from_command_line(sys.argv)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
web_1 | utility.execute()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
web_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
web_1 | self.execute(*args, **cmd_options)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
web_1 | output = self.handle(*args, **options)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/makemigrations.py", line 110, in handle
web_1 | loader.check_consistent_history(connection)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/migrations/loader.py", line 282, in check_consistent_history
web_1 | applied = recorder.applied_migrations()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 65, in applied_migrations
web_1 | self.ensure_schema()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 52, in ensure_schema
web_1 | if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 254, in cursor
web_1 | return self._cursor()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 229, in _cursor
web_1 | self.ensure_connection()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
web_1 | self.connect()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
web_1 | six.reraise(dj_exc_type, dj_exc_value, traceback)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
web_1 | raise value.with_traceback(tb)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
web_1 | self.connect()
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 189, in connect
web_1 | self.connection = self.get_new_connection(conn_params)
web_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/oracle/base.py", line 212, in get_new_connection
web_1 | return Database.connect(self._connect_string(), **conn_params)
web_1 | django.db.utils.DatabaseError: ORA-12545: Connect failed because target host or object does not exist
here its my listener status
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC_FOR_XE)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.2.0 - Production
Start Date 28-DEC-2017 15:51:21
Uptime 0 days 2 hr. 8 min. 36 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Default Service XE
Listener Parameter File /u01/app/oracle/product/11.2.0/xe/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/e48c7c272f44/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC_FOR_XE)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=e48c7c272f44)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=e48c7c272f44)(PORT=8080))(Presentation=HTTP)(Session=RAW))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "XE" has 1 instance(s).
Instance "XE", status READY, has 1 handler(s) for this service...
Service "XEXDB" has 1 instance(s).
Instance "XE", status READY, has 1 handler(s) for this service...
here its my listener status
You should change HOST to localhost' or '127.0.0.1 and SID is NAME.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'xe',
'USER': 'system',
'PASSWORD': 'oracle',
'HOST':'127.0.0.1',
'PORT':'1521'
}
}
For future references, if Oracle is configured with Service name instead of SID, then the configuration would be:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': '127.0.0.1:1521/service.name',
'USER': 'system',
'PASSWORD': 'oracle',
}
}
Another thing to consider when working with Oracle in Django is that when you connect to Other Users (schema) database, you have to set db_table Meta option in Django models:
class OracleTable(models.Model):
... fields ...
class Meta:
db_table = '\"OTHERUSER\".\"ORACLETABLE\"'