How to create complex multi model join query set with Django ORM - django

I want to create a query set with multiple tables. I cannot find any resource that helps
Beside some small 2 table join examples, I cannot find any resource that is helpful for this, not in stackoverflow, the Django docs or even in the ORM Cookbook.
Below is firstly the SQL I want to recreate followed by the models classes, simplified for the purpose of this question. They have in fact a LOT more fields.
SELECT doc_uid,
docd_filename,
doct_name,
docd_media_url
FROM vugd_detail,
vug_virtual_user_group,
vugdoc_vug_docs,
doc_documents,
docd_detail,
doct_type
WHERE vugd_vug_uid = vug_uid
AND vugdoc_vug_uid = vug_uid
AND vugdoc_doc_uid = doc_uid
AND docd_doc_uid = doc_uid
AND doct_uid = doc_doct_uid
AND vugd_status = 1
AND docd_status = 1
AND NOW() BETWEEN vugd_start_date AND vugd_end_date
AND NOW() BETWEEN docd_start_date AND docd_end_date
AND vug_uid = {{Some ID}};
class VugVirtualUserGroup(models.Model):
objects = None
vug_uid = models.AutoField(primary_key=True)
vug_name = models.CharField(unique=True, max_length=30)
vug_short_code = models.CharField(max_length=6)
vug_created_on = models.DateTimeField()
vug_created_by = models.CharField(max_length=30)
class Meta:
managed = False
db_table = 'vug_virtual_user_group'
app_label = 'main'
class VugdDetail(models.Model):
objects = None
vugd_uid = models.AutoField(primary_key=True)
vugd_vug_uid = models.ForeignKey(VugVirtualUserGroup, models.DO_NOTHING, db_column='vugd_vug_uid')
vugd_long_name = models.CharField(max_length=50)
vugd_status = models.IntegerField()
vugd_start_date = models.DateTimeField()
vugd_end_date = models.DateTimeField()
class Meta:
managed = False
db_table = 'vugd_detail'
app_label = 'main'
class VugdocVugDocs(models.Model):
onjects = None
vugdoc_uid = models.AutoField(primary_key=True)
vugdoc_vug_uid = models.ForeignKey(VugVirtualUserGroup, models.DO_NOTHING, db_column='vugdoc_vug_uid')
vugdoc_doc_uid = models.ForeignKey(DocDocuments, models.DO_NOTHING, db_column='vugdoc_doc_uid')
class Meta:
managed = False
db_table = 'vugdoc_vug_docs'
app_label = 'main'
class DocdDetail(models.Model):
objects = None
docd_uid = models.AutoField(primary_key=True)
docd_doc_uid = models.ForeignKey(DocDocuments, models.DO_NOTHING, db_column='docd_doc_uid')
docd_filename = models.CharField(max_length=200)
docd_media_url = models.CharField(max_length=300)
docd_status = models.IntegerField()
docd_start_date = models.DateTimeField()
docd_end_date = models.DateTimeField()
class Meta:
managed = False
db_table = 'docd_detail'
app_label = 'main'
class DoctType(models.Model):
objects = None
doct_uid = models.AutoField(primary_key=True)
doct_vug_uid = models.ForeignKey('VugVirtualUserGroup', models.DO_NOTHING, db_column='doct_vug_uid')
doct_name = models.CharField(max_length=30)
doct_start_date = models.DateTimeField()
doct_end_date = models.DateTimeField()
class Meta:
managed = False
db_table = 'doct_type'
app_label = 'main'
class DocDocuments(models.Model):
object = None
doc_uid = models.AutoField(primary_key=True)
doc_doct_uid = models.ForeignKey('DoctType', models.DO_NOTHING, db_column='doc_doct_uid')
class Meta:
managed = False
db_table = 'doc_documents'
app_label = 'main'
As one can see the actual query is not so complicated.
The goal here is just to get the list of valid(active, not deleted) files avilable for a specific VUG(Simple user group), if the group has a valid status.
So if there is any Django ORM experts who can help with what is really a common SQL type that should be should be possible to be converted to a Django ORM script, your help will sincerely be appreciated.

The Django way to query multiple related tables is to chain them using dot notation.
A sudo code example might help point you in the right direction.
vug_virtual_user_group = VugVirtualUserGroup.objects.get(vug_uid="Some ID")
print(f'vug_virtual_user_group {vug_virtual_user_group}')
A lookup using the foreign key to get all related objects
vugdoc_vugdocs = vug_virtual_user_group.vugdocvugdocs_set.all()
print(f'vugdoc_vugdocs {vugdoc_vugdocs}')
This example assumes that the default modelmanger objects is not overridden by objects = None.

Related

Django ModelForm foreign key values are not displayed

Am new to Django, and I am trying to create a form using model form. The form has a foreign key value (connection_type below in forms.py) and it's not displaying the values it is referring to.
For the image below, the columns,
Connection name: displayed
Connection type: text box has not appeared
Endpoint: displayed
Image from the browser
forms.py
class ConnectionForm(forms.ModelForm):
connection_type = forms.ModelChoiceField(queryset=ConnectionTypes.objects.all(), to_field_name='connection_type_id')
class Meta:
model = ConnectionDetails
exclude = ['connection_id','last_update_date']
Models.py
class ConnectionDetails(models.Model):
connection_id = models.IntegerField(primary_key=True,default=re_sequence('connection_seq'))
connection_name = models.CharField(max_length=200)
connection_type = models.IntegerField()
endpoint = models.CharField(max_length=100)
port = models.IntegerField()
login_id = models.CharField(max_length=100)
login_password = fields.CharPGPPublicKeyField(max_length=100)
connection_string_1 = fields.CharPGPPublicKeyField(max_length=100)
connection_string_2 = fields.CharPGPPublicKeyField(max_length=100)
connection_string_3 = fields.CharPGPPublicKeyField(max_length=100)
aws_region = models.CharField(max_length=20)
owner_id = models.IntegerField()
last_update_date = models.DateTimeField(default=dbcurr_ts)
working_schema = models.CharField(max_length=100)
service = models.CharField(max_length=100)
def generate_enc(mystrenc):
return 'pass'
class Meta:
managed = False
db_table = 'connection_details'
verbose_name = 'connection_details'
verbose_name_plural = 'connection_details'
class ConnectionTypes(models.Model):
connection_type_id = models.IntegerField(primary_key=True,default=re_sequence('connection_type_seq'))
connection_type_name = models.CharField(max_length=100)
connection_type_desc = models.CharField(max_length=300)
connection_type_category = models.CharField(max_length=100)
last_update_date = models.DateTimeField(default=dbcurr_ts)
class Meta:
managed = False
db_table ='connection_types'
verbose_name = 'connection_types'
verbose_name_plural = 'connection_types'
Can you please let me know what is mistake am making?
One issue is that the model field for connecton_types takes an IntegerField. Yet, you are giving it a ModelChoiceField in the form. I think if you set connection_type as models.ForeignKey(ConnectionType, on_delete=models.CASCADE) or something similar in your models.py, it should work better. This way the ConnectionType is directly linked to whatever ConnectionDetails you have.

How to reduces number of sqls generating Django restful services

I need help to improve API performance. Below is my approach also i have tried to do other combination of fields with select_related or prefetch_related but still i am getting like 400sqls.
model.py
class VisVisits(models.Model):
visit_id = models.IntegerField(primary_key=True)
null=True)
class Meta:
managed = False
db_table = 'vis_visits'
def __str__(self):
return str(self.visit_id)
class VisVisitData(models.Model):
vdata_id = models.IntegerField(primary_key=True)
app_local_id = models.IntegerField(blank=True, null=True)
visit = models.ForeignKey('VisVisits', models.DO_NOTHING, blank=True, null=True, related_name='data')
class Meta:
managed = False
db_table = 'vis_visit_data'
def __str__(self):
return str(self.vdata_id)
Serializer
class VisVisitDataSerializer(serializers.ModelSerializer):
class Meta:
model = VisVisitData
field = '__all__'
class VisVisitsSerializer(serializers.ModelSerializer):
data = VisVisitDataSerializer(many=True)
class Meta:
model = VisVisits
fields = ('visit_id','data')
views.py
visit_data = VisVisits.objects.filter(is_valid=1,user_id=u).prefetch_related('school_program__school')
visit_data_serializer = VisVisitsSerializer(visit_data,context={'request':request},many=True)
To reduce the number of queries you must prefetch the data that you serialize in the VisVisitsSerializer, i.e. data. In this serializer you don't include school_program so it's not necessary prefetch this. You can use the django debug toolbar (https://django-debug-toolbar.readthedocs.io/en/latest/) to inspect the repeated queries and find out what to put in select_releted/prefetch_related

Django ORM query with multiple inner join

I want to be able to do queries involving multiple inner joins using Django ORM, here's my model (showing only relevant fields)
class PuntoMedida(models.Model):
id_punto_medida = models.AutoField(primary_key=True,db_column='id_punto_medida')
nombre = models.CharField(max_length=100, blank=False,db_column='nombre')
class Meta:
db_table = 'punto_medida'
class Instalacion(models.Model):
id_instalacion = models.AutoField(primary_key=True,db_column='id_instalacion')
activo = models.CharField(max_length=1, blank=False,default='1',db_column='activo')
usuarios=models.ManyToManyField(User,through='InstalacionUsuario')
class Meta:
db_table = 'instalacion'
class InstanciaInstalacion(models.Model):
id_instancia_instalacion = models.AutoField(primary_key=True,db_column='id_instancia_instalacion')
id_instalacion = models.ForeignKey(Instalacion, blank=False, null=False,db_column='id_instalacion')
puntos_medida=models.ManyToManyField('PuntoMedida',through='InstInstalacionPuntoMedida')
activo = models.CharField(max_length=1, blank=True,default='1',db_column='activo')
class Meta:
db_table = 'instancia_instalacion'
class InstInstalacionPuntoMedida(models.Model):
id= models.AutoField(primary_key=True,db_column='id')
id_instancia_instalacion = models.ForeignKey(InstanciaInstalacion,db_column='id_instancia_instalacion', blank=False, null=False)
id_punto_medida = models.ForeignKey('PuntoMedida',db_column='id_punto_medida', blank=False, null=False)
class Meta:
unique_together = ('id_instancia_instalacion','id_punto_medida')
db_table = 'instancia_instalacion_punto_medida'
class InstalacionUsuario(models.Model):
id= models.AutoField(primary_key=True,db_column='id')
id_instalacion = models.ForeignKey(Instalacion,db_column='id_instalacion', blank=False, null=False)
id_usuario = models.ForeignKey(User,db_column='id_usuario', blank=False, null=False)
class Meta:
unique_together = ('id_instalacion','id_usuario')
db_table = 'instalacion_usuario'
I want to obtain all the objects PuntoMedida related to all the objects Instalacion that a certain user can see. Until now, I've been able to do this using a raw sql like this one:
SELECT a.id_punto_medida, a.nombre
FROM punto_medida a
INNER JOIN instancia_instalacion_punto_medida b ON
a.id_punto_medida=b.id_punto_medida
INNER JOIN instancia_instalacion c ON
b.id_instancia_instalacion=c.id_instancia_instalacion
INNER JOIN instalacion d ON
c.id_instalacion=d.id_instalacion
INNER JOIN instalacion_usuario f ON
d.id_instalacion=f.id_instalacion
AND c.activo='1'
AND d.activo='1'
and f.id_usuario=7
However, I am looking forward to stop using raw queries and only use Django ORM just in case I need to change my database in the future (currently using PostgreSQL, might migrate to Oracle). I've been trying to create the equivalent ORM query with no success. Any ORM expert out there than can help me with this syntax or give me an example of where to start?Thanks in advance...
Can you try this:
PuntoMedida.objects.filter(
instanciainstalacion__activo='1',
instanciainstalacion__id_instalacion__activo='1',
instanciainstalacion__id_instalacion__instalacionusuario_set__id_usuario=7,
)

Get_object_or_None and prefetch_related

curr = curr = get_object_or_None('Page', menu__slug=slug1, menu__parent__slug=slug2).prefetch_related('menu')
Need to use prefetch_related to get info from ForeignKey like an object and get None if it is not in table. Who knows, can I realize it by using get_object_or_None? It seems to work, but read, that it will not. Why so?
class Menu(models.Model):
name = models.CharField()
parent = models.ForeignKey('self', related_name='children')
slug = models.SlugField()
region = models.ForeignKey()
pos = models.IntegerField()
on_top = models.BooleanField()
nofollow = models.BooleanField()
class Page(models.Model):
title = models.CharField()
name = models.CharField()
menu = models.ForeignKey('Menu')
meta_key = models.TextField()
meta_desc = models.TextField()
body = models.TextField()
has_certificate = models.BooleanField()
get_object_or_None() is meant to be a shortcut to specifying a try except block, and it doesn't return a QuerySet, the class where prefetch_related() is from.
Also, if you're trying to get just one instance of your model you're not getting any performance enhancement by prefetching, in fact you'll make your query heavier on the db.

don't want inner joins on django models

i've migrated an existing database that i would like to work with django. i have the following models:
class Device(models.Model):
class Meta:
db_table = u'DEVICES'
managed=False
id = models.CharField(primary_key=True, max_length=22, db_column='DEVICE_ID')
name = models.CharField(max_length=40, db_column='DEVICE_NAME')
status = models.CharField(max_length=10, db_column='STATUS')
class DevicePort(models.Model):
class Meta:
db_table = u'DEVICE_PORT'
managed=False
id = models.CharField( primary_key=True, max_length=22, db_column='DEVICE_PORT_ID')
device = models.OneToOneField(Device, db_column='DEVICE_ID')
type = models.CharField( max_length=22, db_column='PORT_TYPE_ID')
port_num = models.CharField( max_length=30, db_column='DEVICE_PORT_NUM')
class IP(models.Model):
class Meta:
db_table = u'IP_NODE'
managed=False
ip_address = models.CharField(primary_key=True, max_length=15, db_column='IP_NODE_NO')
hostname = models.CharField(max_length=40, db_column='IP_HOST')
port = models.OneToOneField(DevicePort, db_column='DEVICE_PORT_ID')
status = models.CharField(max_length=50, db_column='IP_NODE_STATUS')
i want a list of Devices and their relational IP.ip_address and IP.hostnames. In SQL, i would do something like:
SELECT UNIQUE
d.device_name device
FROM
IP_NODE c,
DEVICES d,
DEVICE_PORT e
WHERE
c.ip_node_no = b.ip_node_no
AND c.device_port_id = e.device_port_id
AND e.device_id = d.device_id
AND d.device_name LIKE 'something';
How do i do this with my django models?
So instead of an inner join you want...and implicit join? Is there something I'm missing here because an implicit join is slower than an explicit one most all cases.
[http://postgresql.1045698.n5.nabble.com/explicit-JOIN-faster-than-implicit-td1920335.html][1]
I can't see any reason you would want to do this, but as stated above you can certainty get this by writing your own queries with raw, or even connection.cursor. Maybe even could pull it off with extra(tables=[...]), though I'm not positive as I've never used tables like this.