Extending a Django Database dynamically? - django

I am building an app in Django that allows users to add items to their profile, such as previous work experience. The problem that I am running into is that different users have different amounts of experience. My current approach is to have a model that looks something like:
class MyModel(models.Model):
owner = models.ForeignKey(User, related_name='modelowner', on_delete=models.CASCADE)
experience1_company = models.TextField(null=True, blank=True)
experience2_company = models.TextField(null=True, blank=True)
experience3_company = models.TextField(null=True, blank=True)
...
experience10_company = models.TextField(null=True, blank=True)
experience1_title = models.TextField(null=True, blank=True)
experience2_title = models.TextField(null=True, blank=True)
experience3_title = models.TextField(null=True, blank=True)
...
experience10_title = models.TextField(null=True, blank=True)
In my present model the user can enter up to 10 previous roles, which covers about 95%+ of users. However, this obviously has the problem where each previous experience a user enters has lots five other attributes (start date, end date, title, description, location). meaning that the model ends up with something like 60 total attributes... and if I need to add something new, I have to add it 10 times.
Question: Is there a way to dynamically build new model attributes for the users that have more than X number of experiences or is best practice to have each one explicitly defined in the model and just have a practical cutoff (e.g., you can only enter up to 10)?

You would have to change your model to represent one experience like this:
class Experience(models.Model):
owner = models.ForeignKey(User, on_delete=models.CASCADE)
company = models.TextField(null=True, blank=True)
title = models.TextField(null=True, blank=True)
# and as much other fields as you like:
start_date = models.DateField()
end_date = models.DateField()
description = models.TextField(null=True, blank=True)
location = models.TextField(null=True, blank=True)
Now you can create unlimited Experience instances for each user, and fetch them with:
user = User.objects.get(username='bob')
Experience.objects.filter(owner=user)
or
user.experience_set.all()
Documentation on this topic is in Making queries - following relationships "backward"

You need a model to store work experience information such as(job_title,company_name, start_date, end_date, duties) and link this work experience model to the user profile. something like below:
class WorkExperience(models.Model):
person = models.ForeignKey(User, blank=True, null=True)
job_title = models.CharField(max_length=125)
company_name = models.CharField(max_length=125)
start_date = models.DateField()
end_date = models.DateField()
duties_description = models.CharField(max_length=125)
This way, your application can accommodate any number of WorkExperience per user

Related

How to make a query across multiple models in Django

I'm using Django and I want to know how to get objects through 3 models
These are my models
class Participant(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
is_leader = models.BooleanField(default=False)
team = models.ForeignKey(Team, on_delete=models.CASCADE, null=True, related_name="participants")
application_date = models.DateField(auto_now_add=True, null=True)
resolution_date = models.DateField(null=True, blank=True)
accepted = models.BooleanField(default=False)
class Team(models.Model):
name = models.TextField(default="")
is_public = models.BooleanField(default=False)
institution = models.ForeignKey(Institution, on_delete=models.CASCADE, null=True, related_name='teams')
campaign = models.ForeignKey(Campaign, on_delete=models.CASCADE, null=True, related_name='teams')
class Campaign(models.Model):
name = models.TextField(default="")
description = models.TextField(default="")
initial_date = models.DateTimeField(auto_now_add=False, null=True, blank=True)
end_date = models.DateTimeField(auto_now_add=False, null=True, blank=True)
qr_step_enabled = models.BooleanField(default=True)
image_resolution = models.IntegerField(default=800)
sponsor = models.ForeignKey(Sponsor, on_delete=models.CASCADE, null=True, related_name='campaigns')
I have the user through a request, and I want to get all campaigns of that user.
I tried doing it with for loops but I want to do it with queries
this is what I had:
user = request.user
participants = user.participant_set.all()
for participant in participants:
participant.team.campaign.name
is there a way to make a query through these models and for all participants?
A user can have many participants, and each participant has a Team, each team has a campaign
The best way is to merge the two modela Team and Campaign in one model.
Something as simple as this should work:
Campaign.objects.filter(team__participant__user=request.user)
The Django ORM is smart enough to follow foreign key relationships in both directions.
Thanks to Daniel W. Steinbrook to guide me to the answer, I had to do this to get the query:
Campaign.objects.filter(teams__participants__user__exact=request.user)

Generic model that can be used by several other models but only one at a time

I'm trying to make a Description model that can be used by several different models.
But the Description model can only have one at a time.
Lets say that i have these two models.
class Account(models.Model):
user = models.OneToOneField(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
birthdate = models.DateField()
phoneNumber = models.CharField(max_length=20)
class CompanyProfile(models.Model):
companyName = models.CharField(max_length=50)
VAT = models.CharField(max_length=35, unique=True)
I want both to have multiple descriptions. But the description object can only be linked to one model. A (dirty) solution that i found was this.
class Description(models.Model):
title = models.CharField(max_length=100)
text = models.CharField(max_length=2500, null=True, blank=True)
account = models.ForeignKey(Account, on_delete=models.CASCADE, null=True, blank=True)
company = models.ForeignKey(CompanyProfile, on_delete=models.CASCADE, null=True, blank=True)
In normal python you would just be able to have a list of description. But in django, this isn't possible. I've thought about using generic relations but that seems to work differently.
Why I want to do it like this?
Because I want to expand Description by using inheritance and I want to be able to use all those types of 'Descriptions' for multiple classes.
Django has got Generic ForeignKey
But I feel a better way to implement this is to add OneToOne field in each model mapping to Description model
description= models.ForeignKey(Description, on_delete=models.SET_NULL, null=True, blank=True)

Manipulate Excel Data Before Adding to Database with Django - Order of Operations?

I receive 6 weekly excel reports that I've been manually compiling into a very large monthly report. Each report has between 5-30 columns, and 4000 to 130,000 rows.
I'm putting together a simple Django app that allows you to upload each report, and the data ends up in the database.
Here's my models.py:
#UPEXCEL models
from django.db import models
############## LISTS ###############
class TransactionTypeList(models.Model):
name = models.CharField(max_length=100)
def __str__(self):
return self.name
class TransactionAppTypeList(models.Model):
name = models.CharField(max_length=100)
def __str__(self):
return self.name
class CrmCaseOriginList(models.Model):
name = models.CharField(max_length=100)
def __str__(self):
return self.name
############## CLIENTS AND STAFF ###############
class Staff(models.Model):
name = models.CharField(max_length=40)
employee_id = models.CharField(max_length=40)
start_date = models.TimeField(blank=True, null=True)
end_date = models.DateField(blank=True, null=True)
first_name = models.CharField(blank=True, null=True, max_length=40)
last_name = models.CharField(blank=True, null=True, max_length=40)
email = models.EmailField(blank=True, null=True)
phone = models.CharField(blank=True, null=True, max_length=20)
street = models.CharField(blank=True, null=True, max_length=100)
city = models.CharField(blank=True, null=True, max_length=100)
state = models.CharField(blank=True, null=True, max_length=2)
zipcode = models.CharField(blank=True, null=True, max_length=10)
is_team_lead = models.BooleanField(default=False)
boss = models.ForeignKey('Staff', related_name='Boss', null=True, blank=True)
def __str__(self):
return self.name
class Meta:
app_label="upexcel"
class Client(models.Model):
name = models.CharField(max_length=40)
short_name = models.CharField(max_length=20, blank=True, null=True)
start_date = models.DateField(default=timezone.now, blank=True, null=True)
end_date = models.DateField(blank=True, null=True)
team_lead = models.ForeignKey(Staff, related_name='client_team_lead')
def __str__(self):
return self.name
class ClientNameChart(models.Model):
client_name = models.ForeignKey(Client, related_name='client_corrected_name')
name_variation = models.CharField(max_length=100)
date_added = models.DateTimeField(auto_now_add=True)
def __str__(self):
return '%s becomes %s' % (self.name_variation, self.client_name)
class StaffNameChart(models.Model):
staff_name = models.ForeignKey(Staff, related_name='staff_corrected_name')
name_variation = models.CharField(max_length=100)
date_added = models.DateTimeField(auto_now_add=True)
def __str__(self):
return '%s becomes %s' % (self.name_variation, self.staff_name)
############## DATA FROM REPORTS ###############
class CrmNotes(models.Model):
created_by = models.ForeignKey(Staff, related_name='note_creator')
case_origin = models.CharField(max_length=20)
client_regarding = models.ForeignKey(Client, related_name='note_client_regarding')
created_on = models.DateTimeField()
case_number = models.CharField(max_length=40)
class Transactions(models.Model):
client_regarding = models.ForeignKey(Client, related_name='transaction_client')
created_by = models.ForeignKey(Staff, related_name='transaction_creator')
type = models.ForeignKey(TransactionTypeList, related_name='transaction_type')
app_type = models.ForeignKey(TransactionAppTypeList, related_name='transaction_app_type')
class Meta:
app_label="upexcel"
class Timesheets(models.Model):
staff = models.ForeignKey(Staff, related_name='staff_clocked_in')
workdate = models.DateField()
start_time = models.DateTimeField()
end_time = models.DateTimeField()
total_hours = models.DecimalField(decimal_places=2, max_digits=8)
class Provider(models.Model):
name = models.CharField(max_length=40)
street = models.CharField(max_length=100)
city = models.CharField(max_length=40)
state = models.CharField(max_length=11)
zip = models.CharField(max_length=10)
class StudentsApplication(models.Model):
app_number = models.CharField(max_length=40)
program = models.CharField(max_length=40)
benefit_period = models.CharField(max_length=40)
student_name = models.CharField(max_length=40)
student_empl_id = models.CharField(max_length=40)
requested_amount = models.DecimalField(max_digits=8, decimal_places=2)
provider = models.ForeignKey(Provider, related_name='app_provider')
provider_code = models.CharField(max_length=40)
class AuditReport(models.Model):
was_audited = models.BooleanField(default=False)
auditor = models.ForeignKey('upexcel.Staff', related_name='auditor')
payment_defect = models.BooleanField(default=False)
grant_discount_error = models.BooleanField(default=False)
math_error = models.BooleanField(default=False)
fees_book_error = models.BooleanField(default=False)
other_error = models.BooleanField(default=False)
overpayment_amount = models.DecimalField(max_digits=8, decimal_places=2)
underpayment_amount = models.DecimalField(max_digits=8, decimal_places=2)
doc_defect = models.BooleanField(default=False)
status_change = models.BooleanField(default=False)
admin_savings_defect = models.BooleanField(default=False)
network_savings_defect = models.BooleanField(default=False)
admin_adjustments = models.DecimalField(max_digits=8, decimal_places=2)
network_adjustments = models.DecimalField(max_digits=8, decimal_places=2)
error_corrected = models.BooleanField(default=False)
comments = models.TextField(max_length=500)
client = models.ForeignKey(Client, related_name='audited_client')
staff = models.ForeignKey(Staff, related_name='processor_audited')
application = models.ForeignKey(StudentsApplication, related_name='app_audited')
class Meta:
app_label="upexcel"
However the excel reports I'm taking in need some work done to them, and I'm trying figure out exactly how I should go about processing them and routing them.
The first challenge is that each report references the associated Staff and Client with different data. For example, if the Staff.name is "Bob Dole", one report has it as "Dole, Bob". Another has it as "Dole, Robert". Still another has "Robert Dole" then "103948210", which is his employee ID number.
Also, these change and new ones sprout up, which is why I made ClientNameChart and StaffNameChart, to where a user can input the string as it shows up in a report, and attach it to a Client or Staff. Then when processing, we can lookup StaffNameChart.name_variation, and return the associated StaffNameChart.Staff.employee_id, which should work great as a foreign key within the respective report's table (ie. AuditReport.staff)
The second challenge is to take a report, and route some of the columns to one database table, and others to another. For example, the big one is the Audit Report sheet. Many of the columns just transpose directly into the AuditReport(models.Model). However, it also has data for each StudentsApplication and Provider, where I need to take several columns, store them as a new record in their destination table, and replace the columns with one column containing a foreign key for that item within that destination table.
So that is my quest.
Here's the order of operations I have in my head - I will use the most complex Audit_Report_Wk_1.xlsx report to address all challenges in one upload:
Upload File
Using openpyxl, load read-only data:
from openpyxl.worksheet.read_only import ReadOnlyWorksheet
myexcelfile = request.FILES['file']
myworkbook = load_workbook(myexcelfile, read_only=True)
mysheet = myworkbook['Sheet1']
Write a script that matches the names strings of the staff, auditor, and client columns with StaffNameChart.name_variation, and replace it with StaffNameChart.Staff.name.
Part B: If the client or staff columns are blank, or contain strings not found in the name charts, all of those rows get saved in a new excel document. Edit: I suppose I could also create a new model class called IncompleteAuditReport that just have fields that match up with each column and store it there, then if someone adds a new NameChart variation, it could trigger a quick look-up to see if that could allow this process to complete and the record to be properly added?)
Check the columns in mysheet that will be replaced by foreign keys for the Provider and StudentsApplication tabes. If their respective data doesn't yet exist in their respective tables, add the new record. Either way, then replace their columns with the foreign key that points to the resulting record.
Is this the correct order of operations? Any advice on what specific tools to use from openpyxl etc. to manipulate the data in the most efficient ways, so I can use the fewest resources possible to look-up and then change several hundred thousand fields?
Thank you so much if you've read this far. I'm currently a bit intimidated by the more complex data types, so it's not crystal clear to me the best way to store the data in memory and to manipulate it while it's there.

django many to many field - programmatically retrieve relations

I have been working on a model for tags and am trying to avoid using contenttypes. I have couple questions related to ManyToManyField in django.
I have the following model
taggables/models.py
class Tag(models.Model):
tag_statuses = (
(u'P', _('Pending approval')),
(u'A', _('Approved')),
)
slug = models.SlugField()
created_at = models.DateTimeField(null=True, blank=True)
created_by = models.ForeignKey(User, related_name='tagged_item_created_by')
status = models.CharField(max_length=20, choices=tag_statuses)
site = models.ForeignKey(Site, default=settings.SITE_ID, related_name='tagged_item_site')
def __unicode__(self):
return self.slug
class TagI18n(models.Model):
tag = models.CharField(max_length=100)
descriptor = models.TextField(null=True, blank=True)
# i18n properties
item = models.ForeignKey(Tag)
language = models.CharField(max_length=6, choices=settings.LANGUAGES, default=settings.LANGUAGE_CODE)
class Meta:
unique_together = (("language", "item"))
def __unicode__(self):
return self.tag
I also have different apps around my project that uses tag model as many to many field. such as events for example
evetns/models.py
class Item(models.Model):
event_status_list = (
(u'P', _('Pending approval')),
(u'A', _('Approved')),
(u'R', _('Rejected')),
(u'S', _('Spam')),
)
published_at = models.DateTimeField(null=True, blank=True)
published_by = models.ForeignKey(User, null=True, blank=True, related_name='item_published_by')
updated_by = models.ForeignKey(User, null=True, blank=True, related_name='item_updated_by')
updated_at = models.DateTimeField(null=True, blank=True)
site = models.ForeignKey(Site, default=settings.SITE_ID, related_name='events_item_site')
event_slug = models.SlugField(null=True, blank=True)
# event timing
event_start_date = models.DateField()
event_start_time = models.TimeField(null=True, blank=True)
event_end_date = models.DateField()
event_end_time = models.TimeField(null=True, blank=True)
event_recurrent = models.BooleanField(default=False)
event_status = models.CharField(max_length=20, choices=event_status_list, default=u'P')
# relations
media = models.ManyToManyField(ImageFile, null=True, blank=True)
comments = models.ManyToManyField(Comment, null=True, blank=True)
votes = models.ManyToManyField(Vote, null=True, blank=True)
tags = models.ManyToManyField(Tag, null=True, blank=True)
audience = models.ManyToManyField(Audience, null=True, blank=True)
Now what am trying to do here is run a query to programmatically retrieve all the related models to Tag and then count how many a times a tag was used. Am sure I can do that with contenttypes (generic types) but I don't know how it will perform under heavy usage that's why I wanted to do the many to many fields.
If you are interested in the total number of usage ( aka reference count ) of a tag very often, I think you should store it in the database, example put one extra field to the Tag model, like
referencecount = models.IntegerField( default=0 )
Than in the appropriate places, ( example models .save() )you can increment or decrements it's value.
For your use case, the performance of generic wouldn't matter, because you need anyway to do N queries over 2N tables (one for each "taggable" model and one for each m2m join table, at least).
With the m2m approach, you should have the list of 'taggable' models stored somewhere, at least as a list of ('app_name', 'model') pairs. Then use ContentType (it's very performant) to get the actual model class or query directly from there:
counts = {}
for m in taggable_models:
ct = ContentType.get_by_natural_key(*m)
c = ct.model_class().objects.filter(tags=yourtag).distinct().count()
counts[ct.name] = c

Get a dropdown of states in Django Admin

I made some basic models for a listing of a business, like so:
class Business(models.Models):
id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=225, blank=True)
address = models.CharField(max_length=150, blank=True)
city = models.CharField(max_length=150, blank=True)
state_id = models.IntegerField(null=True, blank=True)
zip = models.CharField(max_length=33, blank=True)
country = models.CharField(max_length=150, blank=True)
url = models.CharField(max_length=765, blank=True)
class States(models.Model):
id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=96)
state_abbr = models.CharField(max_length=24, blank=True)
In the admin when I edit each business it shows the state_id field. But how do I connect it with the state model to show a select dropdown listing of the states?
Also, how do I show the state abbreviation in the view of a business?
An alternative that doesn't require a separate state table:
from django.contrib.localflavor.us.us_states import STATE_CHOICES
class Business(models.Models):
...
state = models.CharField(max_length=2, choices=STATE_CHOICES, null=True, blank=True)
...
Edit in 2015 (django 1.8)
you should check the django official localflavor repo: https://github.com/django/django-localflavor.
from localflavor.us.models import USStateField
class Business(models.Models):
…
state = USStateField(null=True, blank=True)
…
Some tests are available on the repo for this specific usage.
Docs available here.
You need to use a ForeignKey field.
Make the following changes.
class Business(models.Models):
id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=225, blank=True)
address = models.CharField(max_length=150, blank=True)
city = models.CharField(max_length=150, blank=True)
#state_id = models.IntegerField(null=True, blank=True)
# Define a new state field that creates a ForeignKey relationship with States
state = models.ForeignKey('States', null=True, blank=True)
zip = models.CharField(max_length=33, blank=True)
country = models.CharField(max_length=150, blank=True)
url = models.CharField(max_length=765, blank=True)
class States(models.Model):
id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=96)
state_abbr = models.CharField(max_length=24, blank=True)
#Define the __unicode__ method, which is used by related models by default.
def __unicode__(self):
return self.state_abbr
By default ForeignKey fields append '_id' to the field name when creating the column name in the database. So, the new "state" field in the Business class will automatically use the column "state_id" that you've previously defined, unless you've changed some of the default behavior of Django.
For more on this:
Check out Django's documentation of
the ForeignKey field
Search "ForeignKey" on stackoverflow.com