Google App Engine (GAE) Improperly Configured Database - django

I have a working Django project that will deploy using Heroku. I am having trouble getting the app to deploy on GAE. When I run it locally, I get an error referring to an Improperly Configured database backend.
Any help would be appreciated.
Error:
...
raise ImproperlyConfigured(error_msg)
ImproperlyConfigured: 'postgresql' isn't an available database backend.
Try using django.db.backends.XXX, where XXX is one of:
'dummy', 'mysql', 'oracle', 'postgresql_psycopg2', 'sqlite3' <br>
Error was: No module named postgresql.base
...
app.yaml
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.app
libraries:
- name: django
version: "latest"
beta_settings:
cloud_sql_instances: <cloudsql-connection-string>
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'xxx',
'USER': '*****',
'PASSWORD': '******',
'HOST': 'xx.xx.xx.xx',
'PORT': '5432',
}
}
If I change the Engine to
'ENGINE': 'django.db.backends.postgresql_psycopg2'
I get the error:
ImportError: No module named psycopg2.extensions
pip freeze returns:
Django==1.11.4 psycopg2==2.7.3.1 pytz==2017.2

The GAE standard env. does not allow for psycopg2, and it seems that my original app.yaml (above) instructed GAE to allocate a standard env. instance. This disconnect was highlighted by Dan's comments (above). The correct procedure here is to 1) change the app to be able to deploy on GAE standard, or 2) change the app.yaml to deploy on a GAE flex (according to my current understanding). The following pages seem to help on the second option (https://cloud.google.com/appengine/docs/flexible/python/upgrading) and (https://cloud.google.com/appengine/docs/flexible/python/testing-and-deploying-your-app).

Related

How to solve this error deploying to render.com: django.db.utils.OperationalError: could not translate host name "***" to address?

What I'm trying to do: Deploy my django app to render.com with a postgres database. I'm following the render guide: Getting Started with Django on Render.
Problem: I am getting a build failed log error saying the following: django.db.utils.OperationalError: could not translate host name "***" to address: Name or service not known (I have omitted the actual host name here).
What research I have done: I have searched the error extensively, however all of the highly rated solutions I have encountered so far are based on using Docker like this which I am not using.
settings.py (snippet):
import dj_database_url
DEBUG = 'RENDER' not in os.environ
if DEBUG:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
}
}
if not DEBUG:
DATABASES = {
'default': dj_database_url.config(
default=os.environ.get('DATABASE_URL'),
)
}
In my render.com environmental variables, DATABASE_URL is saved with the postgres URL given by render which includes the database name, hostname, username and password. It follows this format: postgres://USER:PASSWORD#INTERNAL_HOST:PORT/DATABASE
If you are using render.yaml for deploying then you have to specify the services region. The region should be same as your database region.
source: https://community.render.com/t/django-could-not-translate-host-name-to-address/6187/2
render.yaml
databases:
- name: berry
services:
- type: web
name: berry-service
plan: free
env: python
region: singapore
buildCommand: "./build.sh"
startCommand: "gunicorn core.wsgi:application"
envVars:
- key: DATABASE_URL
fromDatabase:
name: berry
property: connectionString
- key: SECRET_KEY
generateValue: true
- key: WEB_CONCURRENCY
value: 4
Screenshots
I make the following and it work fine for me:
if DJANGO_ENV == 'development':
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
elif len(sys.argv) > 0 and sys.argv[1] != 'collectstatic':
if os.environ.get('DATABASE_URL', None) is None:
raise Exception('DATABASE_URL environment variable not defined')
DATABASES = {
'default': dj_database_url.config(conn_max_age=60, ssl_require=True)
}

GCP, Django problem connection refused when trying to connect to the server

My django project runns normally on localhost and on heroku also, but when I deployed it to google cloud platform I am getting this error:
could not connect to server: Connection refused
Is the server running locally and accepting connections on Unix domain socket "/cloudsql/<connection_name>/.s.PGSQL.5432"?
The connection to the database in settings.py looks like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'database_name',
'USER': 'postgres',
'PASSWORD': "password",
# production
'HOST': '/cloudsql/connection_name',
'PORT': '5432',
}
Additionally, my app.yaml looks like
runtime: python37
handlers:
- url: /static
static_dir: static/
- url: /.*
script: auto
env_variables:
DJANGO_SETTINGS_MODULE: fes_app.settings
requirements.txt looks like this plus
sqlparse==0.4.2
toml==0.10.2
uritemplate==4.1.1
urllib3==1.25.11
whitenoise==5.2.0
twilio==6.9.0
I have tried using the binary version of psycopg, also gave a role client sql to the service account in the cloud.
**NOTE : ** I am using an app engine standard environment
Likely need more information to help you out, but did you follow the tutorial below in the official Google Docs? That's usually how I get started and then I make modifications from there.
I would compare how Google is deploying a Django app to your own settings and see what's missing. For example, your requirements.txt file does not look complete (unless you only pasted part of it) so I would start there.
https://cloud.google.com/python/django/appengine

Django over Docker does not install mysqlclient

I want to integrate MySQL with Django, MySQL is running over Docker and I put the config like this to connect to the db docker with Django:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'restaurant',
'HOST': 'db',
'PORT': '3306',
'USER': 'root',
'PASSWORD': 'root',
'default-character-set': 'utf8',
'OPTIONS': {
'init_command': 'SET default_storage_engine=INNODB',
}
}
}
But when Django is trying to connect to the mysql db, it throws me this error:
I tried to install mysqlclient with pip, but I have this error:
These are the docker-compose.dev.yml and Dockerfile configs.
If someone needs the complete code, here you can find it, and test it with docker-compose -f docker-compose.dev.yml up --build.
Thanks :).
mysqlclient has native dependencies that must be installed before you can pip install it. When running in docker, and especially in alpine, you probably want to switch over to using mysql-connector-python which is a pure python library that does not have any native dependencies l,ike mysqlclient. Update your requirements file and update your settings to use mysql.connector.django if you want to use mysql-connector-python.
Another response would be adding:
RUN apk add --no-cache bash mariadb-dev mariadb-client mariadb-libs python3-dev build-base
On the dockerfile.

Django server in a Docker

I'm starting to develop an application with a Django back-end, and I wish to do it inside a Docker. I almost managed to do it, but I'm still having an issue. Currently I have two containers running :
The first one contains my django app and the complete command is
python3 manage.py runserver 0.0.0.0:8000
and the second one is hosting my database.
My docker-compose.yml file is this one :
version: '3'
services:
db:
image: mysql:5.7
environment:
MYSQL_ROOT_PASSWORD : root
MYSQL_DATABASE : ml_gui
back:
build: ./back/
command: python3 manage.py runserver
ports:
- "8000:8000"
depends_on:
- db
and my django settings concerning the database is :
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'ml_gui',
'USER': 'root',
'PASSWORD': 'root',
'HOST': 'db',
'PORT': '3306',
'OPTIONS': {
'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
},
'TEST': {
'CHARSET': 'utf8',
'COLLATION': 'utf8_general_ci',
},
},
}
The problem is, when I do requests outside of the container (I've tried in my browser, with curl and with Postman) on localhost:8000, I have no answer. But, when I do the same request inside the container with curl, it works.
How could I make those requests work from outside of the containers ?
you should never run ./manage.py runserver in production, it is only for testing... also if you dont specify the IP address that the server listens to it is available only for localhost (see here)
so in you docker-compose.yml the command should be
./manage.py runserver 0:8000

Why do I lose connectivity to my SQL Server when running through Apache?

I am trying to deploy my Django project on my company LAN. I am able to get the site working. However, I lose connectivity to my Microsoft SQL Server when I run the site through Apache. Everything works fine in the development environment. I suspect that I am losing Windows authentication when I work through the Apache server. This is what my DB settings are in my settings.py file:
'mosaiq': {
'ENGINE': 'sql_server.pyodbc',
'NAME': 'MOSAIQ',
'USER': '',
'PASSWORD': '',
'HOST': 'HHCMOSAIQ01T',
'PORT': '',
'OPTIONS': {'driver': 'ODBC Driver 11 for SQL Server',
},
Any idea how to connect to my SQL Server using Windows authentication or does my problem lie elsewhere?
This is the error I get:
('28000', "[28000] [Microsoft][ODBC Driver 11 for SQL Server][SQL Server]Login failed for user 'MEDSTAR\$'. (18456) (SQLDriverConnect)")