I'm trying to tunnel my db connection in a django application through a jump server but can't seem to get it working because django manage.py handles & process the connections.
here's that I have in the settings.py
#process ssh_key first
ssh_key= os.getenv('SSH_KEY', '').encode('utf8').decode('unicode_escape')
server ={}
with sshtunnel.open_tunnel(
(os.environ.get('SSH_HOST'),int(os.getenv('SSH_PORT'))),
ssh_pkey=paramiko.RSAKey.from_private_key(io.StringIO(ssh_key)),
ssh_username= os.environ.get('SSH_USERNAME'),
remote_bind_address=(os.environ.get('DB_HOST'), int(os.getenv('DB_PORT'))),
) as ssh_proxy_host:
server={
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'HOST': 'localhost',
'PORT': ssh_proxy_host.local_bind_port,
'NAME': os.environ.get('DB_NAME'),
'USER': os.environ.get('DB_USER'),
'PASSWORD': os.environ.get('DB_PASS'),
}
# here's where I should have the connection function to db, but don't know if django has that option available
The true remote host is specified upon tunnel creation. The HOST used for the db connection should be localhost because you need it to find your end of the tunnel, not the other end of it.
Related
I am super beginner, but want to learn super fast building web application.
I am right now developing an Income-Expense web app on Django Framework (Python, js and Ajax).
I am now stuck with the server and get different errors. Anyone can support me ?
ERROR
"django.db.utils.OperationalError: connection to server on socket "/tmp/.s.PGSQL.5432" failed: fe_sendauth: no password supplied"
I think I shut down everything not properly and when a came back my virtual environment was not working.
Thank You
Don't know what to try more
To resolve the issue, make sure that you have the correct credentials to access your PostgreSQL database. You may need to update the database settings in your Django settings.py file to include the correct database name, username, password, host, and port information.
for example
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'mydatabase',
'USER': 'mydatabaseuser',
'PASSWORD': 'mypassword',
'HOST': 'localhost',
'PORT': '5432',
}
}
I have a Django 4 app with PostgreSQL. I connected the SQL instance vie google-proxy correctly, however when I upload it into App Engine it fails to connect.
settings.py
if os.getenv('GAE_APPLICATION', None):
# Running on production App Engine, so connect to Google Cloud SQL using
# the unix socket at /cloudsql/<your-cloudsql-connection string>
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'HOST': '/cloudsql/instancename',
'USER': 'user',
'PASSWORD': '********',
'NAME': 'database',
'PORT': '5432',
}
}
else:
DATABASES = {
'default': {
"ENGINE": "django.db.backends.postgresql",
'NAME': 'database',
'USER': 'user',
'PASSWORD': '******',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
The error that on the screen was related with psycopg2:
could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/cloudsql/instancename/.s.PGSQL.5432"?
/layers/google.python.pip/pip/lib/python3.9/site-packages/psycopg2/__init__.py, line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
I updated the requirements.txt because locally with the proxy everything was correct.
asgiref==3.5.0
Django==4.0
django_better_admin_arrayfield==1.4.2
djangorestframework==3.13.1
django-cors-headers==3.7.0
django-filter==2.4.0
django-ratelimit==3.0.1
gunicorn==20.1.0
django-cors-headers==3.7.0
psycopg2==2.8.6
psycopg2-binary==2.9.3
pytz==2020.4
sqlparse==0.4.1
google-api-core==1.26.0
google-api-python-client==1.12.8
google-auth==1.27.0
google-auth-httplib2==0.0.4
google-cloud-core==1.6.0
google-cloud-storage==1.36.1
google-crc32c==1.1.2
google-resumable-media==1.2.0
googleapis-common-protos==1.53.0
I also added all the APIs necessaries according to the google docs
Lastly, I read the logs in App Engine and the first error is about IAM
rpc error: code = PermissionDenied desc = IAM permission denied for service account services#service.iam.gserviceaccount.com. "
I added all the possible permisions to that service account, however it continuous to happend.
I don't know what else to do.
Thanks you all
Currently I have a requirement,I need to communicate with external server using django application. The server is already up,Next section is the data transfer. I need some time samples and values from server and I need to send responses back to server. How django port can listen to external server.? How it can response back? I need asynchronous communication and REST responses
According to the comments, the external server is a PostgreSQL database.
Since you're using Django, you can easily set this up as a secondary database in your settings:
DATABASES = {
'default': # that SQLite config...,
'game': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'mydatabase',
'USER': 'mydatabaseuser',
'PASSWORD': 'mypassword',
'HOST': 'somewhere-else',
'PORT': '5432',
},
}
You don't have to route any models to that database, but if you do create any models to correspond to the data in the game server, you'll want to set managed = False so Django doesn't do migrations or anything.
If you don't want to use models, just open a cursor to the secondary database and query away:
from django.db import connections
with connections['game'].cursor() as cursor:
cursor.execute('SELECT something FROM some_table')
# etc...
I'm trying to connect Django to the Google cloud SQL, working with python 2.7 and django 1.5 under windows. I went through the instructions on this page: https://developers.google.com/appengine/docs/python/cloud-sql/django
My settings.py file has basic database settings of the form:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'INSTANCE': 'my_project:instance1',
'NAME': 'my_database',
}
}
With of course a proper could SQL instance and a database created through the SQL prompt of the google apis console
When I try to run manage.py syncdb for the first time in order to obtain an OAuth2 token, I get this:
OperationalError: (2003, "Can't connect to MySQL server on 'localhost'
(10061)")
Before you ask, I did make sure that both the django and google packages are in my PYTHONPATH, as well as "C:\Program Files (x86)\Google\google_appengine\lib\django-1.5"
Any help would be really welcome!
That database configuration only makes sense when connecting from AppEngine. If you want to access your CloudSQL database from your local machine using django, you should use the google.appengine.ext.django.backends.rdbms engine.
You can see the different configuration options here:
https://developers.google.com/appengine/docs/python/cloud-sql/django#development-settings
EDIT: The google.appengine.ext.django.backends.rdbms engine has been deprecated. If you want to connect to Google Cloud SQL from your local machine you should use IP connectivity. You can use the Cloud SQL instance IP (IPv4 or IPv6) and connect using the standard django.db.backends.mysql engine.
Example connection to Google Cloud SQL in Django:
AppEngine Standard, Python 2.7:
try:
import MySQLdb # noqa: F401
except ImportError:
import pymysql
pymysql.install_as_MySQLdb()
if os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine'):
# Running on production App Engine, so connect to Google Cloud SQL using
# the unix socket at /cloudsql/<your-cloudsql-connection string>
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '/cloudsql/<your-cloudsql-connection-string>',
'NAME': '<your-database-name>',
'USER': '<your-database-user>',
'PASSWORD': '<your-database-password>',
}
}
else:
# Running locally so connect to either a local MySQL instance or connect to
# Cloud SQL via the proxy. To start the proxy via command line:
#
# $ cloud_sql_proxy -instances=[INSTANCE_CONNECTION_NAME]=tcp:3306
#
# See https://cloud.google.com/sql/docs/mysql-connect-proxy
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '127.0.0.1', # DB's IP address
'PORT': '3306',
'NAME': '<your-database-name>',
'USER': '<your-database-user>',
'PASSWORD': '<your-database-password>',
}
}
Source: GCP Python Django Samples AppEngine Standard Python 2.7
AppEngine Standard, Python 3.7:
# Install PyMySQL as mysqlclient/MySQLdb to use Django's mysqlclient adapter
# See https://docs.djangoproject.com/en/2.1/ref/databases/#mysql-db-api-drivers
# for more information
import pymysql # noqa: 402
pymysql.install_as_MySQLdb()
if os.getenv('GAE_APPLICATION', None):
# Running on production App Engine, so connect to Google Cloud SQL using
# the unix socket at /cloudsql/<your-cloudsql-connection string>
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '/cloudsql/[YOUR-CONNECTION-NAME]',
'USER': '[YOUR-USERNAME]',
'PASSWORD': '[YOUR-PASSWORD]',
'NAME': '[YOUR-DATABASE]',
}
}
else:
# Running locally so connect to either a local MySQL instance or connect to
# Cloud SQL via the proxy. To start the proxy via command line:
#
# $ cloud_sql_proxy -instances=[INSTANCE_CONNECTION_NAME]=tcp:3306
#
# See https://cloud.google.com/sql/docs/mysql-connect-proxy
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '127.0.0.1', # DB's IP address
'PORT': '3306',
'NAME': '[YOUR-DATABASE]',
'USER': '[YOUR-USERNAME]',
'PASSWORD': '[YOUR-PASSWORD]',
}
}
Source GCP Python Django Samples AppEngine Standard Python 3.7
AppEngine Flexible:
DATABASES = {
'default': {
# If you are using Cloud SQL for MySQL rather than PostgreSQL, set
# 'ENGINE': 'django.db.backends.mysql' instead of the following.
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'polls',
'USER': '<your-database-user>',
'PASSWORD': '<your-database-password>',
# For MySQL, set 'PORT': '3306' instead of the following. Any Cloud
# SQL Proxy instances running locally must also be set to tcp:3306.
'PORT': '5432',
}
}
# In the flexible environment, you connect to CloudSQL using a unix socket.
# Locally, you can use the CloudSQL proxy to proxy a localhost connection
# to the instance
DATABASES['default']['HOST'] = '/cloudsql/<your-cloudsql-connection-string>'
if os.getenv('GAE_INSTANCE'):
pass
else:
DATABASES['default']['HOST'] = '127.0.0.1' # DB's IP address
Source GCP Python Django Samples AppEngine Flexible
I'm using pgbouncer with Django. I've added test_foo database to its config to be able to run tests, because apparently Django can't use a different port for the test DB. Now the test run but at the end, when Django tries to drop the test DB, I receive
django.db.utils.DatabaseError: database "test_foo" is being accessed by other users
DETAIL: There are 1 other session(s) using the database.
I suppose that is caused by the open connection stored by pgbouncer. What can I do?
This is not the perfect solution, but it does the trick. You can force Django to use different database settings when running unit tests by adding to your settings.py:
if 'test' in sys.argv or 'test_coverage' in sys.argv:
# Use 5432 as db port (avoid going through pgbouncer, can't delete test DB).
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'xxx',
'USER': 'xxx',
'PASSWORD': 'xxx',
'HOST': '',
'PORT': '5432'
},
}