Connect Django to Google Cloud SQL - django

I'm trying to connect Django to the Google cloud SQL, working with python 2.7 and django 1.5 under windows. I went through the instructions on this page: https://developers.google.com/appengine/docs/python/cloud-sql/django
My settings.py file has basic database settings of the form:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'INSTANCE': 'my_project:instance1',
'NAME': 'my_database',
}
}
With of course a proper could SQL instance and a database created through the SQL prompt of the google apis console
When I try to run manage.py syncdb for the first time in order to obtain an OAuth2 token, I get this:
OperationalError: (2003, "Can't connect to MySQL server on 'localhost'
(10061)")
Before you ask, I did make sure that both the django and google packages are in my PYTHONPATH, as well as "C:\Program Files (x86)\Google\google_appengine\lib\django-1.5"
Any help would be really welcome!

That database configuration only makes sense when connecting from AppEngine. If you want to access your CloudSQL database from your local machine using django, you should use the google.appengine.ext.django.backends.rdbms engine.
You can see the different configuration options here:
https://developers.google.com/appengine/docs/python/cloud-sql/django#development-settings
EDIT: The google.appengine.ext.django.backends.rdbms engine has been deprecated. If you want to connect to Google Cloud SQL from your local machine you should use IP connectivity. You can use the Cloud SQL instance IP (IPv4 or IPv6) and connect using the standard django.db.backends.mysql engine.

Example connection to Google Cloud SQL in Django:
AppEngine Standard, Python 2.7:
try:
import MySQLdb # noqa: F401
except ImportError:
import pymysql
pymysql.install_as_MySQLdb()
if os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine'):
# Running on production App Engine, so connect to Google Cloud SQL using
# the unix socket at /cloudsql/<your-cloudsql-connection string>
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '/cloudsql/<your-cloudsql-connection-string>',
'NAME': '<your-database-name>',
'USER': '<your-database-user>',
'PASSWORD': '<your-database-password>',
}
}
else:
# Running locally so connect to either a local MySQL instance or connect to
# Cloud SQL via the proxy. To start the proxy via command line:
#
# $ cloud_sql_proxy -instances=[INSTANCE_CONNECTION_NAME]=tcp:3306
#
# See https://cloud.google.com/sql/docs/mysql-connect-proxy
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '127.0.0.1', # DB's IP address
'PORT': '3306',
'NAME': '<your-database-name>',
'USER': '<your-database-user>',
'PASSWORD': '<your-database-password>',
}
}
Source: GCP Python Django Samples AppEngine Standard Python 2.7
AppEngine Standard, Python 3.7:
# Install PyMySQL as mysqlclient/MySQLdb to use Django's mysqlclient adapter
# See https://docs.djangoproject.com/en/2.1/ref/databases/#mysql-db-api-drivers
# for more information
import pymysql # noqa: 402
pymysql.install_as_MySQLdb()
if os.getenv('GAE_APPLICATION', None):
# Running on production App Engine, so connect to Google Cloud SQL using
# the unix socket at /cloudsql/<your-cloudsql-connection string>
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '/cloudsql/[YOUR-CONNECTION-NAME]',
'USER': '[YOUR-USERNAME]',
'PASSWORD': '[YOUR-PASSWORD]',
'NAME': '[YOUR-DATABASE]',
}
}
else:
# Running locally so connect to either a local MySQL instance or connect to
# Cloud SQL via the proxy. To start the proxy via command line:
#
# $ cloud_sql_proxy -instances=[INSTANCE_CONNECTION_NAME]=tcp:3306
#
# See https://cloud.google.com/sql/docs/mysql-connect-proxy
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '127.0.0.1', # DB's IP address
'PORT': '3306',
'NAME': '[YOUR-DATABASE]',
'USER': '[YOUR-USERNAME]',
'PASSWORD': '[YOUR-PASSWORD]',
}
}
Source GCP Python Django Samples AppEngine Standard Python 3.7
AppEngine Flexible:
DATABASES = {
'default': {
# If you are using Cloud SQL for MySQL rather than PostgreSQL, set
# 'ENGINE': 'django.db.backends.mysql' instead of the following.
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'polls',
'USER': '<your-database-user>',
'PASSWORD': '<your-database-password>',
# For MySQL, set 'PORT': '3306' instead of the following. Any Cloud
# SQL Proxy instances running locally must also be set to tcp:3306.
'PORT': '5432',
}
}
# In the flexible environment, you connect to CloudSQL using a unix socket.
# Locally, you can use the CloudSQL proxy to proxy a localhost connection
# to the instance
DATABASES['default']['HOST'] = '/cloudsql/<your-cloudsql-connection-string>'
if os.getenv('GAE_INSTANCE'):
pass
else:
DATABASES['default']['HOST'] = '127.0.0.1' # DB's IP address
Source GCP Python Django Samples AppEngine Flexible

Related

PostgreSQL with Django failing to connect in Standar App Engine

I have a Django 4 app with PostgreSQL. I connected the SQL instance vie google-proxy correctly, however when I upload it into App Engine it fails to connect.
settings.py
if os.getenv('GAE_APPLICATION', None):
# Running on production App Engine, so connect to Google Cloud SQL using
# the unix socket at /cloudsql/<your-cloudsql-connection string>
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'HOST': '/cloudsql/instancename',
'USER': 'user',
'PASSWORD': '********',
'NAME': 'database',
'PORT': '5432',
}
}
else:
DATABASES = {
'default': {
"ENGINE": "django.db.backends.postgresql",
'NAME': 'database',
'USER': 'user',
'PASSWORD': '******',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
The error that on the screen was related with psycopg2:
could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/cloudsql/instancename/.s.PGSQL.5432"?
/layers/google.python.pip/pip/lib/python3.9/site-packages/psycopg2/__init__.py, line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
I updated the requirements.txt because locally with the proxy everything was correct.
asgiref==3.5.0
Django==4.0
django_better_admin_arrayfield==1.4.2
djangorestframework==3.13.1
django-cors-headers==3.7.0
django-filter==2.4.0
django-ratelimit==3.0.1
gunicorn==20.1.0
django-cors-headers==3.7.0
psycopg2==2.8.6
psycopg2-binary==2.9.3
pytz==2020.4
sqlparse==0.4.1
google-api-core==1.26.0
google-api-python-client==1.12.8
google-auth==1.27.0
google-auth-httplib2==0.0.4
google-cloud-core==1.6.0
google-cloud-storage==1.36.1
google-crc32c==1.1.2
google-resumable-media==1.2.0
googleapis-common-protos==1.53.0
I also added all the APIs necessaries according to the google docs
Lastly, I read the logs in App Engine and the first error is about IAM
rpc error: code = PermissionDenied desc = IAM permission denied for service account services#service.iam.gserviceaccount.com. "
I added all the possible permisions to that service account, however it continuous to happend.
I don't know what else to do.
Thanks you all

Connecting to a remote db through a jump server in Django

I'm trying to tunnel my db connection in a django application through a jump server but can't seem to get it working because django manage.py handles & process the connections.
here's that I have in the settings.py
#process ssh_key first
ssh_key= os.getenv('SSH_KEY', '').encode('utf8').decode('unicode_escape')
server ={}
with sshtunnel.open_tunnel(
(os.environ.get('SSH_HOST'),int(os.getenv('SSH_PORT'))),
ssh_pkey=paramiko.RSAKey.from_private_key(io.StringIO(ssh_key)),
ssh_username= os.environ.get('SSH_USERNAME'),
remote_bind_address=(os.environ.get('DB_HOST'), int(os.getenv('DB_PORT'))),
) as ssh_proxy_host:
server={
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'HOST': 'localhost',
'PORT': ssh_proxy_host.local_bind_port,
'NAME': os.environ.get('DB_NAME'),
'USER': os.environ.get('DB_USER'),
'PASSWORD': os.environ.get('DB_PASS'),
}
# here's where I should have the connection function to db, but don't know if django has that option available
The true remote host is specified upon tunnel creation. The HOST used for the db connection should be localhost because you need it to find your end of the tunnel, not the other end of it.

Why am I getting error "Client with IP address x.x.x.x isnt allowed to connect to this MySql server" when trying to connect to MySql from Django?

I am trying to create tables for my models in this full stack project (Django/React). After I run the python manage.py migrate command I'm expecting to see:
Migrations for X:
X\migrations\0001_initial_py
- Create model Departments
- Create model Employees
Instead I get the following error: (9000, "Client with IP address 'X' is not allowed to connect to this MySQL server."
I have tried the command pip install pymysql
then edited the settings.py file adding
import pymysql
pymysql.install_as_MySQLdb()
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mytestdb',
'USER': 'testadmin#mytestmysql',
'PASSWORD': 'XXXXXXXXXX#',
'HOST': 'mytestmysql.mysql.database.XXXXXXXXXXXXX.com',
'PORT': '3306'
MySQL permissions are such that you have to specify 'user'#'host' when defining privileges. My guess is that you didn't set the MySQL database user up correctly.
Read here to understand what I'm referring to.

Connect django web app to postgresql in gcloud

i have been struggling to connect my django web app to a PostgreSQL instance which I set up inside my gcloud account for testing purposes.
I have done the following DB configs in settings.py in Django:
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'instance connection name from gcloud',
'USER' : 'postgres',
'PASSWORD': 'passsss',
'HOST': 'ip-address',
'PORT': '5432',
}
}
the error that I receive in Django after trying to migrate is :
django.db.utils.OperationalError: FATAL: database "..instance connection name from gcloud.." does not exist
I have tried creating a new database "django" and adding it to the NAME with:. This did not work as well. I have also configured in gcloud connections my own computer IP as an authorized network
Normally if I use service like elephantsql it works fine.
Any help would be appreciated. Thanks!
From what I understood from your post, is that you wish to connect from your local pc to a Cloud SQL PostgreSQL instance.
I see that you seem to be following the official documentation already, as you mentioned that you added your IP to authorized networks in order to be able to connect externally.
The cause of your error seems to be due to the format of the NAME parameter.
Are you passing it in this format?
PROJECT:ZONE:INSTANCENAME
Here you have all the available options on how to connect externally to a Cloud PostgreSQL instance.
And here you have the documentation of the requirements to connect an external application to a Cloud SQL PostgreSQL instance.
ok finally after a lot of research and wasting couple of hours I managed to find Django configs in digital ocean docu... pretty weird
This works:
'''
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'my_django_db',
'USER' : 'user1',
'PASSWORD': 'pass',
'HOST': 'ipv4',
'PORT': '',
}
}
'''

django can't drop the test DB via pgbouncer

I'm using pgbouncer with Django. I've added test_foo database to its config to be able to run tests, because apparently Django can't use a different port for the test DB. Now the test run but at the end, when Django tries to drop the test DB, I receive
django.db.utils.DatabaseError: database "test_foo" is being accessed by other users
DETAIL: There are 1 other session(s) using the database.
I suppose that is caused by the open connection stored by pgbouncer. What can I do?
This is not the perfect solution, but it does the trick. You can force Django to use different database settings when running unit tests by adding to your settings.py:
if 'test' in sys.argv or 'test_coverage' in sys.argv:
# Use 5432 as db port (avoid going through pgbouncer, can't delete test DB).
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'xxx',
'USER': 'xxx',
'PASSWORD': 'xxx',
'HOST': '',
'PORT': '5432'
},
}