How to connect to snowflake database from Django framework - django

I'm new to Django and I'm trying to display the result that comes from a Snowflake database. I know that Django has multiple built-in database backend engines like: django.db.backends.postgresql and django.db.backends.mysql among the other few it supports.
Unfortunately, I couldn't find a proper way of configuring a database backend engine in the
settings.py
When I enter sqlalchemy or snowflake-sqlalchemy as the engine, I get this error:
Try using 'django.db.backends.XXX', where XXX is one of:
'mysql', 'oracle', 'postgresql', 'sqlite3'
My guess was to go with sqlalchemy as that's what I usually use to connect to Snowflake outside of Django but for some reason, it's not working properly.
I'd appreciate any guidance on that.

2022 update: There's now a Snowflake backend for Django funded by
Snowflake customers and implemented by Django's Tim Graham:
https://github.com/cedar-team/django-snowflake
From their docs:
Install and usage
Use the version of django-snowflake that corresponds to your version of Django. For example, to get the latest compatible release for Django 3.2.x:
pip install django-snowflake==3.2.*
The minor release number of Django doesn't correspond to the minor release number of django-snowflake. Use the latest minor release of each.
Configure the Django DATABASES setting similar to this:
DATABASES = {
'default': {
'ENGINE': 'django_snowflake',
'NAME': 'MY_DATABASE',
'SCHEMA': 'MY_SCHEME',
'WAREHOUSE': 'MY_WAREHOUSE',
'USER': 'my_user',
'PASSWORD': 'my_password',
'ACCOUNT': 'my_account',
},
}
Some of the discussion while implementing it:
https://groups.google.com/g/django-developers/c/po9dS-2h4lg/m/UeKBoL8dBgAJ?pli=1

You should install a custom Snowflake engine like the following ones. Note that, as of today, those are incomplete. Though, it should not be difficult to implement missing Django features by completing the operations.pyfile.
-> https://github.com/pricemoov/django-snowflake
or
-> https://pypi.org/project/django-snowflake-backend/

please install snowflake-connector-python .E.g. below
pip3 install snowflake-connector-python==1.8.1
Here is the code to connect from SQL Alchemy.
=====================================================================
#!/usr/bin/env python
from snowflake.sqlalchemy import URL
from sqlalchemy import create_engine
engine = create_engine(URL(
account = 'XXXX',
user = 'XXXX',
password = 'XXXXX',
database = 'XXXXXX',
schema = 'XXXXXX',
warehouse = 'XXXXX',
role='XXXXXXXX',
))
try:
connection = engine.connect()
connection.execute(
"CREATE OR REPLACE TABLE test_async(c1 TIMESTAMP_NTZ,c2 VARIANT)",_no_results=True)
finally:
connection.close()
engine.dispose()
=========================================================================

Related

Django, using oracledb error msg that I need Oracle 19

I am confused since the documentation for oracledb clearly states that everything past 12.1 should work fine. Could someone please explain to me where I went wrong? The error was created when I tried to create migrations.
The document I am referencing is: oracledb docs
Here is the error:
django.db.utils.NotSupportedError: Oracle 19 or later is required (found 12.2.0.1.0).
And here is my databases string in my settings.py:
from pathlib import Path
import sys
import oracledb
oracledb.version = "8.3.0"
sys.modules["cx_Oracle"] = oracledb
#the above line was added because of error (django.core.exceptions.ImproperlyConfigured: Error
#loading cx_Oracle module: No module named 'cx_Oracle')
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': (
'(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=server123)(PORT=1521))'
'(CONNECT_DATA=(SERVICE_NAME=server.domain.com)))'
),
'USER': 'user123',
'PASSWORD': 'password',
'OPTIONS': {
'threaded': True,
},
}
}
It's a Django thing. From docs.djangoproject.com/en/4.1/ref/databases/#oracle-notes:
"Django supports Oracle Database Server versions 19c and higher."
Also see the Django 4.0 release notes
"Dropped support for Oracle 12.2 and 18c".
Try an older version of Django if you can't upgrade the DB

Connecting Django to Microsoft SQL Database

I want to connect my django application to MS-SQL server 2014 database.
I wrote this code for making connections.
DATABASES = {
'default': {
'ENGINE': 'sql_server.pyodbc',
'HOST':'DESKTOP-6UNRAN0',
'PORT':'1433',
'NAME': 'MOVIE',
'COLLATION' : '',
}
}
I have installed sql_server.pyodbc
pip install django-pyodbc-azure
as mentioned in the documentation https://pypi.org/project/django-pyodbc-azure/. I am still getting error
django.db.utils.InterfaceError: ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
I no longer recommend using django-pyodbc-azure, as it is no longer maintained by the author. The active PyPI project for SQL Server in Django is currently django-mssql-backend. However, it only supports Django 2.2 and above. I would highly recommend upgrading to Django 2.2 (a long term support release), if not Django 3.0. 2.1 is no longer supported, and this will save you headaches down the road for a little bit of work now. I'm going to assume you're on Linux.
Step One: Install Microsoft's Driver for Linux (You May Also Use FreeTDS)
If you want to use Microsoft's driver, you can install it like this:
sudo curl https://packages.microsoft.com/config/rhel/7/prod.repo > /etc/yum.repos.d/mssql-release.repo
sudo yum remove unixODBC-utf16 unixODBC-utf16-devel
sudo ACCEPT_EULA=Y yum install msodbcsql17
Step Two: Create a Database and Service User in SQL Server
In SQL Server, set up a service user to your Django database. This script will create a user with the minimum permissions needed to the underlying database.
/*
This Script Creates a SQL Server Database, Login and User
With Appropriate Permissions for a Production Django Project
with migrations. Simply fill out the variables below (#db_name and #db_password)
Username will be set to database name + '_user' by default.
*/
DECLARE #db_name VARCHAR(MAX) = 'project'
DECLARE #db_password VARCHAR(MAX) = 'project_password'
DECLARE #db_user VARCHAR(MAX) = #db_name + '_user'
--
--
USE master
DECLARE #cmd VARCHAR(MAX)
-- Server scope: create SQL Server login and permissions
SET #cmd = 'CREATE LOGIN ' + #db_user + ' WITH PASSWORD = ''' + #db_password + ''''
EXEC(#cmd)
SET #cmd = 'GRANT VIEW SERVER STATE TO ' + #db_user
EXEC(#cmd)
SET #cmd = 'CREATE DATABASE [' + #db_name + ']'
EXEC(#cmd)
-- DB scope: create user for server login and permissions
SET #cmd = 'USE [' + #db_name + '];'
SET #cmd = #cmd + 'CREATE USER ' + #db_user + ' FOR LOGIN ' + #db_user + ';'
SET #cmd = #cmd + 'GRANT SELECT, INSERT, UPDATE, DELETE, ALTER, CREATE TABLE, REFERENCES, EXEC TO ' + #db_user
EXEC(#cmd)
Step Three: Configure Django
Finally, let's set up Django itself to point to SQL Server. In your Django project with your venv activated:
pip install django-mssql-backend
DATABASES = {
'default': {
'ENGINE': 'sql_server.pyodbc',
'HOST': 'dbserver.your-domain.com',
'PORT': '1433',
'NAME': 'project',
'USER': 'project_user',
'PASSWORD': 'project_password',
'OPTIONS': {
'driver': 'ODBC Driver 17 for SQL Server',
'unicode_results': True,
},
},
}
If you're using FreeTDS or another driver, change the OPTIONS line, 'driver': 'ODBC Driver 17 for SQL Server'. That should do it.
Good luck!
It is expected that you know if you want to connect to SQL Server you'll have to use/install ODBC as it is native data access API
Regarding documentation lets look into following lines
a Django Microsoft SQL Server external DB backend that uses ODBC by
employing the pyodbc library
Compatible with Micosoft ODBC Driver for SQL Server, SQL Server
Native Client, and FreeTDS ODBC drivers
OPTIONS
Dictionary. Current available keys are:
driver
String.
Server Native Client 11.0", "FreeTDS" etc). Default is "ODBC Driver 13
for SQL Server".

django 2.1 + PostgreSQL 11 + Python 3.7 - Cannot do makemigrations

I'm trying to create a geospatial database with geodjango and postgis following the recommendations of the book : Python Geospatial development, 3rd Edition of Erik Westra, in order to do it I'm trying to configure my django database and to connect it to my PostgreSQL db.
After having launched my PostgreSQL database, I've created my django project and django apps. From then I'd like to apply makemigrations command to my shared app with :
python manage.py makemigrations shared
But then I've go the following error :
File "C:\Users\[...]\Anaconda3\lib\site-packages\psycopg2\__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError
I've even tried to check migrations with showmigrations but it makes the same error message so I've absolutely no clue what's going on.
here's my settings.py file:
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'shapeeditor',
'USER': 'shapeeditor',
'PASSWORD': '(password)',
}
}
I've put (password) to hide the real one but I've checked it's the good one.
The NAME corresponds to the database name with a USER who has the same name
The shared app is written in INSTALLED_APPS so I've checked I didn't forget it.
I've looked at the many posts in StackOverflow about the error I got but it doesn't correspond to what I've facing here
After writing my own question I've found where it bugged...
my USER shapeeditor didn't have the privileges, so it couldn't work, just switched with postgres and I worked, I'll have to change privileges in order to make it work

convert_datetimefield_value() takes 4 positional arguments but 5 were given using django with mongodb

Django Admin login page, I am using mongodb for django, after that only I got error
mongodb configure:
DATABASES = {
'default': {
'ENGINE': 'djongo',
'NAME': 'django-mongo-db',
}
}
Click Here
You are using djongo as your backend engine, the requirement of djongo is Django==2.1.2. Please install the same as -
pip install django==2.1.2
I have faced this error in the past. Doing this resolved it.

How to properly grant access to a database user in Google Cloud SQL?

I've been stuck with this problem for a couple of days.
I developed an application for appengine using Django and I'd like to use Google Cloud SQL for my database. Everything works fine until I want to apply migrations on the development server when it fails with the following error:
django.db.utils.OperationalError: (1045, "Access denied for user 'MY_DB_USER'#'MY_IP' (using password: YES)")
What I've done is as follows:
I followed the instructions in the Django Support page to
develop my application.
In order to create a 1st generation Cloud SQL instance I followed
the steps outlined here, using the Cloud SDK.
I then created a new user following the instructions here and assigned it a password.
I deployed the application using the following command line:
gcloud preview app deploy MY-APP-DIR/app.yaml --version 0-1-0
I authorized my IP and my AppEngine Application ID. They are both listed in the ''Authorization'' section under ''Access Control'' in my SQL instance.
Finally, I tried to apply migrations using the following command line:
SETTINGS_MODE='prod' MY-APP-DIR/manage.py migrate
settings.py
The relevant portion of my settings.py looks as follows:
if os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine'):
DEBUG = False
# Running on production App Engine, so use a Google Cloud SQL database.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST' : '/cloudsql/[MY-PROYECT-ID]:[MY-CLOUD-SQL-INSTANCE]',
'NAME': '[MY-DB-NAME]',
'USER': 'root',
}
}
elif os.getenv('SETTINGS_MODE') == 'prod':
DEBUG = False
# Running in development, but want to access the Google Cloud SQL instance
# in production.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': '[MY-DB-NAME]', # db name.
'USER': '[MY-DB-USER]',
'PASSWORD' : '[MY-DB-USER-PASSWORD]',
'HOST' : '[IPV4 ASSIGNED IN GOOGLE CONSOLE]',
'PORT': '3306',
}
}
else:
# Running in development, so use a local MySQL database.
DEBUG = True
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': '[MY-LOCAL-DB]',
'USER': 'root',
'PASSWORD': 'root',
}
}
Any idea as to what might be causing the problem?
Thank you!
I finally figured out what was the problem.
The proper way to grant a user database access, in order to apply migrations, is the following:
White list your IP. It should be shown under 'Authorized Networks'
Create a new database user account, but do not choose the 'Allow any host (%)' wildcard, instead select the "Restrict host by name, address, or address range" option and assign your IP ( The one you just whitelisted ).
You should now be able to run migrations with the command: SETTINGS_MODE='prod' PROJECT_DIR/manage.py migrate
As a side note, make sure the root user whose host is localhost doesn't have a password, else your App Engine application won't be able to connect to the database.
Hope this helps someone else!