I am considering using Google BigQuery as a back-end for Django but cannot be certain if this is possible, and if it is, what settings would apply.
Currently, my Django application uses Postgresql, and the code in settings.py is as follows:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'mydatabase',
'USER': 'mydatabaseuser',
'PASSWORD': 'mypassword',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
Ideally, I'd like to setup a database connection to Google BigQuery through settings.py and then use views and models as usual.
It's not possible, or at least not supported. You could use the API directly, but obviously you won't get any advantages of the ORM.
You should have a db motor like postgres, mysql, whatever...
The point is, this db motor is necessary to have it, because the structure works in that way.
but of course, you can invoke google cloud from librarys in django and use it as
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession
from google.cloud import datastore
from google.cloud import bigquery
in my case I used to connect
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'OAuth2Credential.json'
for generate your .json you should go to the documentation in:
https://cloud.google.com/iam/docs/creating-managing-service-account-keys
It is possible by using SQLAlchemy with Django.
SQLAlchemy can connect to bigquery with the pybigquery driver.
See the following about how to
Configuring Django to use SQLAlchemy.
Everything is possible for sure. To create an interface would not be such a big job. But, I would just keep one note:
Bigquery is not intended to be backend database, rather it is more like data warehouse as it is defined within business intelligence discipline. This means, google will make it very hard for you to perform fast multi-user operations. As far as I can recall, update statements for example have have some thresholds.
On another hand, if this is purely for example data input, or visualisation of data, then why not. But then again, I think Azure power apps is kind of a product for it.
Related
I have a running and working database in SQLite but multiple users are writing to it in the same time and i seem to have concurrency problems, because of the database being locked.
What do i need in order to create and access a PostgreSQL database in Python? Is there a way in which i dont install anything else than python libraries? I have a company laptop and would need 1XXXX approvals to install a third party software.
The setup is simple: The users go to the upload page on the site(built in django), uploads a xls file and the info gets extracted into a database(while being compared with info from inside the database). If 2 users try to upload files in the same time, i get database locked error.
Any help would be great, either to solve this SQLite problem or to get me started in Postgre if this should solve it. THX!
SQLite is not suitable for multiple user access since it is just a file acting as a database. To get the real power of DBMS you have to switch either MySQL or PostgreSQL which are officially supported by Django. Remember that PostgreSQL is preferred by Django for production use. Anyway you can use Oracle DB if you company already have it.
You need to setup settings.py in your project folder in order connect any database with Django as follows.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'dbname',
'USER': 'postgres',
'PASSWORD': 'postgres',
'HOST': '',
'PORT': '',
}
}
You may need to install relevant db driver to connect it with Django.
Before committing to migrating to PostgreSQL, you may want to simply increase the busy timeout of your SQLite3 database and see if that produces acceptable results.
I am new to Django; intermediate in Python3.6. Now, I've created a basic Django app which displays index.html. Now, I want to create a very simple form in index.html which will query the database and print the results in the same form. I already have the postgresql database in my local with all the data needed; it is very simple with just one table (right now, my Django app is connected to the default sqlit3 db). Where do I go about creating the query to fetch the data and populate it?
You need to install these libraries first :
sudo apt-get install libpq-dev postgresql postgresql-contrib
Configuration
in settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'db_name',
'USER': 'db_user',
'PASSWORD': 'db_user_password',
'HOST': '',
'PORT': 'db_port_number',
}
}
You can refer to these two links:
For More Detailed information about setup :Tutorial for setting up application using PostgreSql
Documentation link for writing Queries
Glad you're coming on board with Django! It's a great language with plenty of useful features, you won't regret learning it.
To connect to an existing database, check out the docs here. This explains how to use your existing db to create models in your project.
I also recommend checking out the tutorials in the docs starting here. This tutorial will explain how to connect to your postgres db, as well as how to begin writing queries for your views. This is the place to start learning the basics of Django, just follow it along and you will get the hang of everything in no time
I am trying to use ElasticSearch as a NoSQL database in my Django project.
The goal is to plug Kibana on the other side and be able to visualize my data there.
I do not have a choice of adding another database and using ElasticSearch only for indexing as my project is plugging to an existing infrastructure.
So I have installed the following two modules:
django-haystack
elasticsearch
I was expecting to not need SQLite (or any other database) for storage and to use ElasticSearch as a NoSQL storage (is that wrong?)
I added the Haystack connections settings to my project settings and then, looked for getting rid of "DATABASES" or replacing it with values pointing to my ElasticSearch with no success :(
Here is what I'm stuck with:
settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://127.0.0.1:9200/',
'INDEX_NAME': 'haystack'
},
}
On all the answers I could find on StackOverflow and code snippets I would never see these DATABASES settings changed, so I assume people were using a database for storage and ElasticSearch only for their search engine/indexing which is not what I am looking for.
Is there a way I can have ElasticSearch as my storage DB (just like I would with MongoDB), not breaking the Models and interface of Django?
It's not possbile (officially) to use ElasticSearch as your django database backend (I assume that this is what you want) - currently supported backends are:
postgresql
mysql
sqlite3
oracle
The above is official list, unofficial (3rd parties) is as follows:
SAP SQL
IBM DB2
Microsoft SQL
Firebird
ODBC
Anyway, the people around django are very nice and hard working developers :) And there is package for that (you can give it a try - never used it before):
https://github.com/aparo/django-elasticsearch
One more thing - why do you need django? Is kibana not enough? You gonna make some changes in data on the ES index in your Django App? Or should it be just readonly?
Because if you want use the ES like a normal DB storage - it won't work, as you will wait each time you update/insert data about 1-2 seconds till ES index new data. This is just not the purpose of the ES.
Hope this help, happy coding.
Can django use an external database? I mean, if you have one server for the db and other n-servers with the web server, can django use the db on an external machine? Can django do queries via internet to another django db?
Yes, your database and web server can be on separate servers. You just have to specify in your settings file the host. See https://docs.djangoproject.com/en/dev/ref/settings/#databases
As the relevant documentation states, Django is capable of using multiple databases. Whether remote access is supported will depend on which one you choose to use - but, as a general rule, it is supported, with the notable exception of sqlite
Yes you can access the database from anywhere. But they(you) need to provide database privileges for your IP. some code is like,
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'mydatabase',
'USER': 'mydatabaseuser',
'PASSWORD': 'mypassword',
'HOST': Remote Host,
'PORT': '5432',
}
}
You can also use external django package https://github.com/kennethreitz/dj-database-url for deployment.
I reach this post, but in my case, I need to access another database. My intention is create an application that report informationa from another database. I needed to use two databases. I found the way in the django documentation.
https://docs.djangoproject.com/en/3.2/topics/db/multi-db/
I am using mysql database. I have many schemas with many tables. I want to create a Django admin interface for various tables in different schemas. Currently for a single schema I am using a setting like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'schema1',
'USER': 'test',
'PASSWORD': 'test',
'HOST': 'example.abc.com',
'PORT': '',
}
After creating an app, admin interface is created for whatever models I register in the admin.py of that app for this schema. Now I want to create an other app where I register models of another schema in its admin.py. These models will belong to different schema. Now how do I create an admin interface for the second app that points to different schema?
Is it possible to create two Django projects with two different settings.py and two different admin interfaces? (So that each will point to different schema.)
I have googled a lot about this. But couldn't find any info. May be there is a simple way and I am approaching this in a wrong way. Any help is appreciated.
Thanks in advance.
This is documented well on the django doc here http://docs.djangoproject.com/en/dev/topics/db/multi-db/#exposing-multiple-databases-in-django-s-admin-interface
I'm not quite sure if you mean that you want to handle different databases or just have different models registered. If you want to have different models in different admin sites, you can register multiple admin sites with different models. You can then access one site eg with '/admin' the other with '/otheradmin'. Maybe you find django-admin-tools useful to customize the display of your models/apps within the admin!