I am building a data processing app which allows user to upload multiple excel sheets, which is processed and returned back to user. I have created 3 models - 1st model is overall, 2nd model is capturing individual excel workbooks, in the 3rd model excel workbooks are opened and data for each excel sheet (sheets within workbooks) is captured. I need advise on 3 things:
Is my model structure efficient given that users will be uploading multiple excel sheets?
Given that users may do upload multiple times in a day, how do I retrieve the latest batch of files for processing?
I need to take user inputs against each sheet uploaded by user (3rd model) in a single view while parallelly showing a preview of the table to the user?
Please help with your opinions.
class UserUpload(models.Model):
number_of_workbooks = models.IntegerField(blank=True,null=True)
file_size = models.FloatField(blank=True,null=True)
user = models.ForeignKey(User,on_delete=models.CASCADE,null=False,blank=False)
multiple_sheets = models.BooleanField(blank=False,default=False)
var_1 = models.BooleanField(blank=False,default=False)
class FileUpload(models.Model):
file_field = models.FileField(blank=False,upload_to=user_directory_path)
userupload = models.ForeignKey(UserUpload,related_name='user_uploads',on_delete=models.CASCADE)
class FileSheetData(models.Model):
fileupload = models.ForeignKey(FileUpload,related_name='file_sheets',on_delete=models.CASCADE)
sheetname = models.CharField(blank=False,max_length=256)
var_2 = models.BooleanField(blank=False,default=False)
var_3 = models.PositiveIntegerField(blank=False,default=0)
After doing a bit of research I was able to get answers to some of the problems above -
In case you want to allow multiple file uploads you would need to create a separate model and link it with the parent model through foreign key. So my model structure for multiple file uploads is fine.
In my example, I have 3 models which are sequentially linked by a Foreign Key. Instances in FileSheetData can be accessed by following the relationship backwards. Here is the link to the documentation - https://docs.djangoproject.com/en/3.2/topics/db/queries/#backwards-related-objects
I am still struggling with this but understand as of now that ModelFormSet could be a potential way to solve it but I am working on it.
Related
I am pretty new to python/django, but previously programmed in C# and VB.Net. I need to create a list on my django form that contains data elements from a PGSQL table. The user should be able to select multiple rows and then I would formulate a query that I would send back to the server. I have researched, but have not found anything to point me in the right direction. I will show my code below.
forms.py
PROD_MULTI_RECORDS=[
('darryl.dillman','darryl.dillman'),
('richard.mcgarry','richard.mcgarry'),
('janet.delage','janet.delage')
]
class rdsprodform(forms.Form):
selectmulti =
forms.CharField(widget=forms.SelectMultiple(choices=PROD_MULTI_RECORDS),
required = False, label = "Please select records to process")
I have an existing table called empname in my postgres database
(Projectid,empid,name,Location) as
(1,101,Raj,India),
(2,201,David,USA)
So in the app console it will have like the following
1)Projectid=Textbox
2)Ops =(view,insert,Edit)-Dropdown
Case1:
So if i write project id as 1 and select View Result:It will display all the records for Projectid =1(Here 1 record)
Case2:
If i write projectid as 3 and select insert it will ask for all the inputs like empid,name,address and based on that it will update the table .
Case3:
If i write projectid as 2 and select edit.Then it will show all the field for that id and user can edit any column and can save which will update the records in backend for the existing table
If there is not data found for the respective project id then it will display no records found
Please help me on this as I am stuck up with models
Once you have your models created, the next task should be the form models. I can identify atleast 3 form classes that you will need to create. One to display the information(case 1), another to collect information(case 2) and the last class to edit the information. Wire up the form to the views and add the urls.
A good reference could be a django a user registration form since it will have all the three cases taken care of.http://www.tangowithdjango.com/book17/chapters/login.html
I have a Category model and Video model
Category:
name=Charfield()
Video:
name=CharField()
category=ManyToManyField()
is_live=BooleanField()
And I want to have the get all categories with a video count but I want to exclude videos who are not live.
This my start state:
Category.objects.annotate(video_count=Count('video'))
# I tried this but I'm not sure if this the right way
Category.objects.exclude(video__is_liive=False)
Any Ideas?
If you want to filter the field you are annotating, you need to use raw SQL as you can't do it through the ORM yet. I wrote a blog post about this:
http://timmyomahony.com/blog/filtering-annotations-django/
Your situation is a little more complicated as you have a M2M relationship which uses an intermediate table. You need something like the following which joins all 3 tables and counts only those that are marked is_live=True (this is totally untested so you will need to play around with it)
categories = Category.objects.all().extra(select = {
"video_count" : """
SELECT COUNT(*)
FROM myapp_videocategory
JOIN myapp_videocategory on myapp_videocategory.category_id = myapp_category.id
JOIN myapp_video on myapp_videocategory.video_id = myapp_video.id
WHERE myapp_video.is_live = True
"""
}).order_by("-live_video_count",)
I've got 2 existing models that I need to join that are non-relational (no foreign keys). These were written by other developers are cannot be modified by me.
Here's a quick description of them:
Model Process
Field filename
Field path
Field somethingelse
Field bar
Model Service
Field filename
Field path
Field servicename
Field foo
I need to join all instances of these two models on the filename and path columns. I've got existing filters I have to apply to each of them before this join occurs.
Example:
A = Process.objects.filter(somethingelse=231)
B = Service.objects.filter(foo='abc')
result = A.filter(filename=B.filename,path=B.path)
This sucks, but your best bet is to iterate all models of one type, and issue queries to get your joined models for the other type.
The other alternative is to run a raw SQL query to perform these joins, and retrieve the IDs for each model object, and then retrieve each joined pair based on that. More efficient at run time, but it will need to be manually maintained if your schema evolves.
I've been drooling over Django all day while coding up an internal website in record time, but now I'm noticing that something is very inefficient with my ForeignKeys in the model.
I have a model which has 6 ForeignKeys, which are basically lookup tables. When I query all objects and display them in a template, it's running about 10 queries per item. Here's some code, which ought to explain it better:
class Website(models.Model):
domain_name = models.CharField(max_length=100)
registrant = models.ForeignKey('Registrant')
account = models.ForeignKey('Account')
registrar = models.ForeignKey('Registrar')
server = models.ForeignKey('Server', related_name='server')
host = models.ForeignKey('Host')
target_server = models.ForeignKey('Server', related_name='target')
class Registrant(models.Model):
name = models.CharField(max_length=100)
...and 5 more simple tables. There are 155 Website records, and in the view I'm using:
Website.objects.all()
It ends up executing 1544 queries. In the template, I'm using all of the foreign fields, as in:
<span class="value">Registrant:</span> {{ website.registrant.name }}<br />
So I know it's going to run a lot of queries...but it seems like this is excessive. Is this normal? Should I not be doing it this way?
I'm pretty new to Django, so hopefully I'm just doing something stupid. It's definitely a pretty amazing framework.
You should use the select_related function, e.g.
Website.objects.select_related()
so that it will automatically do a join and follow all of those foreign keys when the query is performed instead of loading them on demand as they are used. Django loads data from the database lazily, so by default you get the following behavior
# one database query
website = Website.objects.get(id=123)
# first time account is referenced, so another query
print website.account.username
# account has already been loaded, so no new query
print website.account.email_address
# first time registrar is referenced, so another query
print website.registrar.name
and so on. If you use selected related, then a join is performed behind the scenes and all of these foreign keys are automatically followed and loaded on the first query, so only one database query is performed. So in the above example, you'd get
# one database query with a join and all foreign keys followed
website = Website.objects.select_related().get(id=123)
# no additional query is needed because the data is already loaded
print website.account.username
print website.account.email_address
print website.registrar.name