In Django, since version 1.11 we have a class for PostgreSQL GinIndex (https://docs.djangoproject.com/en/1.11/ref/contrib/postgres/indexes/). I'd like to create a migration that constructs such index on a VectorSearchField I added to one of my tables. So far, I've tried to simply add db_index=True to the VectorSearchField, but that fails, because it tries to create a B-Tree index (I think) and the VectorSearchField values are too long.
I managed to create the index I want by running a migrations.RunSQL() migration with:
CREATE INDEX entryline_sv_index ON entryline USING GIN (sv);
However, I guess, since there is a special GinIndex class in Django, maybe there is a way to create such index without executing raw SQL?
Here's a model class:
import django.contrib.postgres.search as pg_search
class EntryLine(models.Model):
speaker = models.CharField(max_length=512, db_index=True)
text = models.TextField()
sv = pg_search.SearchVectorField(null=True) # I want a GIN index on this field.
Any idea how to properly create an index for sv field in a migration? Or is executing the CREATE INDEX ... query the best way?
Haven't yet had a chance to migrate my old manual CREATE INDEX codes to the new system introduced in 1.11 but my understanding is
from django.contrib.postgres.indexes import GinIndex
import django.contrib.postgres.search as pg_search
class EntryLine(models.Model):
speaker = models.CharField(max_length=512, db_index=True)
text = models.TextField()
sv = pg_search.SearchVectorField(null=True)
class Meta:
indexes = [GinIndex(fields=['sv'])]
Is what's required. Raw SQL CREATE INDEX statements need not be used any more.
Related
I was using django-hitcont to count the views on my Post model. I am trying to get the most viewed post in my ListView using this query objects.order_by('hit_count_generic__hits') and it is working fine on SQLite but on PostgreSQL, it is giving me this error :
django.db.utils.ProgrammingError: operator does not exist: integer = text LINE 1: ...R JOIN "hitcount_hit_count" ON ("posts_post"."id" = "hitcoun....
models.py
class Post(models.Model, HitCountMixin):
author = models.ForeignKey(User, related_name='authors', on_delete=models.CASCADE)
title = models.CharField('Post Title', max_length = 150)
description = models.TextField('Description', max_length=1000, blank = True)
date_posted = models.DateTimeField('Date posted', default = timezone.now)
date_modifed = models.DateTimeField('Date last modified', default = timezone.now)
document = models.FileField('Document of Post', upload_to='documents', \
validators=[FileExtensionValidator(allowed_extensions = ['pdf', 'docx']), validate_document_size] \
)
hit_count_generic = GenericRelation(
HitCount,
object_id_field='object_pk',
related_query_name='hit_count_generic_relation'
)
views.py
queryset = Post.objects.order_by('hit_count_generic__hits')
I found this issue on Github related to the problem, but I am still not able to figure out the mentioned workaround.
When comparing different types (in this example integer and text), equals operator throws this exception. To fix that, convert HitCount model pk field to integer and you are good to go. To do that, you need to create and apply migration operation. Django is a really good framework to handle this kind of operations. You just need to check values are not null and are "convertable" to integer. Just change the field type and run two commands below.
python manage.py makemigrations
python manage.py migrate
Before updating your model, I highly recommend you to take a backup in case of failure. This is not an easy operation but you can follow the these links to understand what is going on during this the process.
migrations dump and restore initial data
If you don't care the data on table, just drop table and create a brand new migration file and recreate table.
I want dont want to use the django id directly for viewsets in django, since that potentially reveals the number of items.
I already figured out how to generate UUIDs, which work, but are unwieldy and hexadecimal.
Is there a way to generate a guaranteed unique sufficently long alphanummeric string in django similar to a uuid?
You can probably do something like this (although untested):
import random, string
def random_id_field():
rnd_id = ''.join(random.choices(string.ascii_letters + string.digits, k=16))
return rnd_id
class MyModel(models.Model):
id = models.Charfield(max_length=16, unique=True, primary_key=True, default=random_id_field)
Update
In Python 3.6, they introduced the concept of secrets. Here is an example:
from secrets import token_urlsafe
random_string = token_urlsafe(16)
print(random_string)
Result: 'x3jFt0X_hZr2B4j6CexixQ'
unique_together doesn't work, it only set the unique constraints on the first field and ignore the second field. Is there any way to enforce unique constraints?
class BaseModel(models.Model):
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
deleted = models.DateTimeField(db_index=True, null=True, blank=True)
last_modified_at = models.DateTimeField(auto_now=True)
class Meta:
abstract = True
class Book(BaseModel):
first_form_number = models.CharField(max_length=8)
class Meta:
unique_together = (("first_form_number", "deleted"),)
Your models work correctly in that extent that the right unique index is created:
$ python manage.py sqlmigrate app 0001_initial
...
CREATE UNIQUE INDEX "app_base_slug_version_a455c5b7_uniq" ON "app_base" ("slug", "version");
...
(expected like the name of your application is "app")
I must roughly agree with user3541631's answer. It depends on the database in general, but all four db engines supported directly by Django are similar. They expect that "nulls are distinct in a UNIQUE column" (see NULL Handling in SQLite Versus Other Database Engines)
I verified your problem with and without null:
class Test(TestCase):
def test_without_null(self):
timestamp = datetime.datetime(2017, 8, 25, tzinfo=pytz.UTC)
book_1 = Book.objects.create(deleted=timestamp, first_form_number='a')
with self.assertRaises(django.db.utils.IntegrityError):
Book.objects.create(deleted=timestamp, first_form_number='a')
def test_with_null(self):
# this test fails !!! (and a duplicate is created)
book_1 = Book.objects.create(first_form_number='a')
with self.assertRaises(django.db.utils.IntegrityError):
Book.objects.create(first_form_number='a')
A solution is possible for PostgreSQL if you are willing to manually write a migration to create two special partial unique indexes:
CREATE UNIQUE INDEX book_2col_uni_idx ON app_book (first_form_number, deleted)
WHERE deleted IS NOT NULL;
CREATE UNIQUE INDEX book_1col_uni_idx ON app_book (first_form_number)
WHERE deleted IS NULL;
See:
Answer for Create unique constraint with null columns
Django docs Writing database migrations
Django docs migrations.RunSQL(sql)
depending on your database, it is possible that NULL isn't equal to any other NULL.
Therefore the rows you create are not the same, if one of the values is NULL, will be unique only by the non null field, in your case 'first_form_number'.
Also take in consideration that is case sensitive so "char" and "Char" are not the same.
I had a similar situation and I did my own check by overriding the save method on the model.
You check if exist in the database, but also exclude the current instance, in case of updating, not to compare with itself..
if not deleted:
exists = model.objects.exclude(pk=instance.pk).filter(first_form_number__iexact=first_form_number).exists()
Make sure you actually extend the inherited Meta class, rather than defining your own Meta class (which is ignored by Django):
class Meta(BaseModel.Meta):
unique_together = (("first_form_number", "deleted"),)
models.py:
class Ingredient(models.Model):
_est_param = None
param = models.ManyToManyField(Establishment, blank=True, null=True, related_name='+', through='IngredientParam')
def est_param(self, establishment):
if not self._est_param:
self._est_param, created = self.ingredientparam_set\
.get_or_create(establishment=establishment)
return self._est_param
class IngredientParam(models.Model):
#ingredient params
active = models.BooleanField(default=False)
ingredient = models.ForeignKey(Ingredient)
establishment = models.ForeignKey(Establishment)
I need to fetch all Ingredient with parametrs for Establishment. First I fetch Ingredients.objects.all() and use all params like Ingredients.objects.all()[0].est_param(establishment).active. How I can use django 1.4 prefetch_related to make less sql queries? May be I can use other way to store individual Establishment properties for Ingredient?
Django 1.7 adds the Prefetch object you can put into prefetch_related. It allows you to specify a queryset which should provide the filtering. I'm having some problems with it at the moment for getting a singular (latest) entry from a list, but it seems to work very well when trying to get all the related entries.
You could also checkout django-prefetch which is part of this question which does not seem a duplicate of this question because of the vastly different wording.
The following code would fetch all the ingredients and their parameters in 2 queries:
ingredients = Ingredients.objects.all().prefetch_related('ingredientparam_set')
You could then access the parameters you're interested in without further database queries.
this is a model of the view table.
class QryDescChar(models.Model):
iid_id = models.IntegerField()
cid_id = models.IntegerField()
cs = models.CharField(max_length=10)
cid = models.IntegerField()
charname = models.CharField(max_length=50)
class Meta:
db_table = u'qry_desc_char'
this is the SQL i use to create the table
CREATE VIEW qry_desc_char as
SELECT
tbl_desc.iid_id,
tbl_desc.cid_id,
tbl_desc.cs,
tbl_char.cid,
tbl_char.charname
FROM tbl_desC,tbl_char
WHERE tbl_desc.cid_id = tbl_char.cid;
i dont know if i need a function in models or views or both. i want to get a list of objects from that database to display it. This might be easy but im new at Django and python so i having some problems
Django 1.1 brought in a new feature that you might find useful. You should be able to do something like:
class QryDescChar(models.Model):
iid_id = models.IntegerField()
cid_id = models.IntegerField()
cs = models.CharField(max_length=10)
cid = models.IntegerField()
charname = models.CharField(max_length=50)
class Meta:
db_table = u'qry_desc_char'
managed = False
The documentation for the managed Meta class option is here. A relevant quote:
If False, no database table creation
or deletion operations will be
performed for this model. This is
useful if the model represents an
existing table or a database view that
has been created by some other means.
This is the only difference when
managed is False. All other aspects of
model handling are exactly the same as
normal.
Once that is done, you should be able to use your model normally. To get a list of objects you'd do something like:
qry_desc_char_list = QryDescChar.objects.all()
To actually get the list into your template you might want to look at generic views, specifically the object_list view.
If your RDBMS lets you create writable views and the view you create has the exact structure than the table Django would create I guess that should work directly.
(This is an old question, but is an area that still trips people up and is still highly relevant to anyone using Django with a pre-existing, normalized schema.)
In your SELECT statement you will need to add a numeric "id" because Django expects one, even on an unmanaged model. You can use the row_number() window function to accomplish this if there isn't a guaranteed unique integer value on the row somewhere (and with views this is often the case).
In this case I'm using an ORDER BY clause with the window function, but you can do anything that's valid, and while you're at it you may as well use a clause that's useful to you in some way. Just make sure you do not try to use Django ORM dot references to relations because they look for the "id" column by default, and yours are fake.
Additionally I would consider renaming my output columns to something more meaningful if you're going to use it within an object. With those changes in place the query would look more like (of course, substitute your own terms for the "AS" clauses):
CREATE VIEW qry_desc_char as
SELECT
row_number() OVER (ORDER BY tbl_char.cid) AS id,
tbl_desc.iid_id AS iid_id,
tbl_desc.cid_id AS cid_id,
tbl_desc.cs AS a_better_name,
tbl_char.cid AS something_descriptive,
tbl_char.charname AS name
FROM tbl_desc,tbl_char
WHERE tbl_desc.cid_id = tbl_char.cid;
Once that is done, in Django your model could look like this:
class QryDescChar(models.Model):
iid_id = models.ForeignKey('WhateverIidIs', related_name='+',
db_column='iid_id', on_delete=models.DO_NOTHING)
cid_id = models.ForeignKey('WhateverCidIs', related_name='+',
db_column='cid_id', on_delete=models.DO_NOTHING)
a_better_name = models.CharField(max_length=10)
something_descriptive = models.IntegerField()
name = models.CharField(max_length=50)
class Meta:
managed = False
db_table = 'qry_desc_char'
You don't need the "_id" part on the end of the id column names, because you can declare the column name on the Django model with something more descriptive using the "db_column" argument as I did above (but here I only it to prevent Django from adding another "_id" to the end of cid_id and iid_id -- which added zero semantic value to your code). Also, note the "on_delete" argument. Django does its own thing when it comes to cascading deletes, and on an interesting data model you don't want this -- and when it comes to views you'll just get an error and an aborted transaction. Prior to Django 1.5 you have to patch it to make DO_NOTHING actually mean "do nothing" -- otherwise it will still try to (needlessly) query and collect all related objects before going through its delete cycle, and the query will fail, halting the entire operation.
Incidentally, I wrote an in-depth explanation of how to do this just the other day.
You are trying to fetch records from a view. This is not correct as a view does not map to a model, a table maps to a model.
You should use Django ORM to fetch QryDescChar objects. Please note that Django ORM will fetch them directly from the table. You can consult Django docs for extra() and select_related() methods which will allow you to fetch related data (data you want to get from the other table) in different ways.