Related
New to Django and I wanted to append a key:value pair to each Job object in a Queryset/list before passing it to a template. Researching on here it says you can't just append a field, but rather need to create a new dict or could possibly add attributes or something, but messing around I got it to work great, but logically it seems it shouldn't work, and I worry I may run into problems or data inconsistency somewhere.
My "jobs" view gets all Jobs currently not assigned to a User, filters out Jobs based on the current signed in Users listed skillset(a custom field for my User model), and then the functionatily I'm adding is to check the current Users schedule, and add a value "conflict":True if the schedule conflicts so I can highlight it red when rendering the list of jobs to the screen.
views.py (abbreviated):
def jobs (request):
//get all unassigned jobs, and create a list of only jobs where the user has matching skills
available = Job.objects.filter(assigned_to__isnull=True).order_by('start_time').all()
available = [job for job in available if request.user.is_skilled(job)]
//go through each job, make it a dict, and add "conflict":True to it if scheudle confict
for job in available:
if not request.user.is_available(job):
job = job.__dict__
job["conflict"] = True
//run custom serializer() to format each field and return list of jobs
return JsonResponse({'available': [job.serialize() for job in available]})
The part that doesn't make sense to me is job.__dict__ should be converting the job object to a dict, so I actually expected to get an error when I attempt to run job.serialize(). Infact, there is another view where I attempt the same thing with a single Job object(not a list or Queryset) and it gives error Dict does not contain method serialize(). And if I don't first convert to dict I get an error TypeError: 'Job' object does not support item assignment. But somehow when working with a list, the conversion doesn't occur, but my Job object now has a new field "conflict" that I can check with job.conflict in JS, templates, and in the serialize() model method:
models.py (abbreviated):
class Job (models.Model):
title = models.CharField(max_length=50)
description = models.CharField(max_length=500)
start_time = models.DateTimeField()
end_time = models.DateTimeField()
skillsets = models.ManyToManyField(Skillset, related_name="jobs")
def serialize(self):
return {
'id': self.id,
'title': self.title,
'description': self.description,
'start_time': self.start_time.strftime("%b %d %Y, %I:%M %p"),
'end_time': self.end_time.strftime("%b %d %Y, %I:%M %p"),
'skillsets': [skill.title for skill in self.skillsets.all()],
'conflict': self.conflict if 'conflict' in self.__dict__ else False,
}
So question is, why does available contain a list of only Job objects after I should have converted some of them to Dict? I do this again on the User's object later to highlight users who are available for a particular Job, and same thing, even though I supposedly convert to Dict and add a field, the final output is a list of User objects with an added field available. I was expecting assignable to be a list of dicts, not a list of user objects:
Also in views.py:
def job_details (request, job_id):
//get a single Job, and list of users who are specialist, then filter out users who don't have required skills
job = Job.objects.get(pk=job_id)
users = User.objects.filter(user_type=User.SPECIALIST)
users = list(filter(lambda usr: usr.is_skilled(job), users))
//for each user, check their schedule against this job, and add "available":True/False as determined
for usr in users:
if (usr.is_available(job)):
usr = usr.__dict__
usr["available"] = True
else:
usr = usr.__dict__
usr["available"] = False
//return job and users to be rendered on screen, colored based on their availability
return render(request, "jobs/job_details.html", {
'job': job.serialize(),
'assignable': users,
'conflict': True if request.user.user_type == User.SPECIALIST and not request.user.is_available(job) else False})
Is there a logical explaination? Is it because I'm doing it in a for loop that the conversion doesn't stick, but the field gets added to the User/Job objects, like is it converting back after it exits the for loop?
To address your immediate question - the statement usr = usr.__dict__ done when iterating over the list simply assigns a variable usr. It does not change the element in the original list. To transform the list of model instances to dicts, you can use map function, or list comprehension. For example:
users = list(map(lambda usr: {**usr.__dict__, "available": usr.is_available(job)}, users))
BTW, probably slightly better approach would be to set "available" attribute right on the model instance, and then access it in serialize method via getattr: getattr(self, "available", False).
Or, you can take it a step further and add available as a python property to model class. Then you will be able to document its meaning and usage:
class User(models.Model):
...
#property
def available(self):
"""Document the meaning..."""
return getattr(self, "_available", False)
#available.setter
def available(self, value):
self._available = value
I want to enable filtering on a table. Some of the filters are simple, eg a dropdown to select user. But others are more complex, eg the from date needs to display table rows that have a date that's greater than the filter date. And the search box should search multiple fields in the table.
I started attacking the problem by using a lot of IF statements to built a search query based on the filters the user has applied. I then apply the query to model.objects.filter(a_string_built_depending_on_filters). So far so good but it looks like I'm going to have to start using Q() objects. I'm not sure if I can string together Q() queries in the same way.
#login_required
def entries_show_all(request):
journal_entry_filter_form = JournalEntryFilterForm()
line_item_filter_form = LineItemFilterForm()
order_by = request.GET.get('order_by', '-journal_entry')
filter_query = dict()
url_params = ""
if request.GET.get('user') and int(request.GET.get('user')):
filter_query['journal_entry__user'] = str(request.GET.get('user'))
url_params+='user='+urllib.parse.quote_plus(str(request.GET.get('user')))+'&'
if request.GET.get('type'):
filter_query['journal_entry__type'] = request.GET.get('type')
url_params+='type='+urllib.parse.quote_plus(request.GET.get('type'))+'&'
if filter_query:
#logger.warning('Filter query:')
#logger.warning(filter_query)
#logger.warning('URL Params:'+url_params)
line_items = LineItem.objects.filter(**filter_query).order_by(order_by)
else:
logger.warning('Filters have not been set')
line_items = LineItem.objects.all().order_by(order_by)
paginator = Paginator(line_items,20)
page = request.GET.get('page')
line_items2 = paginator.get_page(page)
return render(request,"journal/entries_show_all.html", {'line_items': line_items2, })
The URL Params variable is used in the template to add to the pagination links, so that the filter holds together while the user moves through pages.
Have found django-filter and started implementing this add-on. Seems to do most of what I want it to.
Update Two
So unfortunately, #Reza Torkaman Ahmadi's idea didn't work out in the end. This is because our program has a filtering function that relies on get_queryset, and overriding the get_queryset method in our views was messing up that function.
So, my partner and I discussed it, and here's what he came up with.
class OrderbyFilter(filters.OrderingFilter):
def get_ordering(self, request, queryset, view):
"""
Ordering is set by a comma delimited ?$orderby=... query parameter.
Extends the OrderingFilter of the django rest framework to redefine
ordering parameters to "asc" and "desc".
"""
params = request.query_params.get(self.ordering_param)
if params:
field_queries = [param.strip() for param in params.split(',')]
fields = []
for field_query in field_queries:
field_query = field_query.split()
if len(field_query) <= 2:
while "asc" in field_query:
field_query.remove("asc")
for i, field in enumerate(field_query):
if field == "desc":
field_query[i-1] = "-" + field_query[i-1]
while "desc" in field_query:
field_query.remove("desc")
fields.append(field_query[0])
else:
fields.append([param.strip() for param in params.split(',')])
ordering = self.remove_invalid_fields(queryset, fields, view, request)
if ordering:
return ordering
return self.get_default_ordering(view)
Basically, this function over-rides Django REST's source code, specifically the get_ordering function in the OrderingFilter. What it does is, if 'asc' is in the query after the field, it removes it and treats it like normal (for normal ascension ordering)
Otherwise if 'desc' is there, it removed the 'desc', and applies a hyphen.
Update Answered. Applied #Reza Torkaman Ahmadi idea and it works great after modifying it to fit my needs. Thanks mate!
Currently, in Django API rest framework, if a user wants to see a list of something in ascending or descending order, they have to do the following:
'Datastream&order_by=-name' shows names in descending order
'Datastream&order_by=name' shows names in ascending order.
I want to make a custom query where typing in 'Datastream&order_by=asc name' will order the names by ascending order, and 'desc name' will do so in descending order.
I've looked at some of the source code for the REST framework, but I may be looking in the wrong area. Getting stumped on what I should do. Any ideas?
You can do it in your own way. like this:
class DatastreamViewSet(ModelViewSet):
def get_queryset(self):
queryset = super(DatastreamViewSet, self).get_queryset()
order_by = self.request.query_params.get('order_by', '')
if order_by:
order_by_name = order_by.split(' ')[1]
order_by_sign = order_by.split(' ')[0]
order_by_sign = '' if order_by_sign == 'asc' else '-'
queryset = queryset.order_by(order_by_sign + order_by_name)
return queryset
this will look for query parameter order_by if is supplied then will split it by space, the first one will indicate to use + or - sign on order_by filter, and the second will be the name of it. so put it all together and create a text, pass it to order_by and your good to go.
for example:
?order_by=asc name
will be like this in django =>
return queryset.order_by('name')
Django Rest Framework's standard way:
from rest_framework.filters import OrderingFilter
Then on your APIView or ViewSet
filter_backends = (OrderingFilter,)
ordering_fields = ['field_name']
query parameter is ordering and supports reverse ordering as well.
GET https://example.com/?ordering=-field_name
I'm trying to do something that should be very common: add/edit a bunch of related models in a single form. For example:
Visitor Details:
Select destinations and activities:
Miami [] - swimming [], clubbing [], sunbathing[]
Cancun [] - swimming [], clubbing [], sunbathing[]
My models are Visitor, Destination and Activity, with Visitor having a ManyToMany field into Destination through an intermediary model, VisitorDestination, which has the details of the activities to be done on the destination (in itself a ManyToMany field into Activity).
Visitor ---->(M2M though VisitorDestination) -------------> Destination
|
activities ---->(M2M)---> Activity
Note that I don't want to enter new destination / activity values, just choose from those available in the db (but that's a perfectly legit use of M2M fields right?)
To me this looks like an extremely common situation (a many to many relation with additional details which are a FK or M2M field into some other model), and this looks like the most sensible modelling, but please correct me if I'm wrong.
I've spent a few days searching Django docs / SO / googling but haven't been able to work out how to deal with this. I tried several approaches:
Custom Model form for Visitor, where I add multiple choice fields for Destination and Activity. That works ok if Destination and Activity could be selected independently, but here they are correlated, ie I want to choose one or several activities for each destination
Using inlineformset_factory to generate the set of destination / activities forms, with inlineformset_factory(Destination, Visitor). This breaks, because Visitor has a M2M relation to Destination, rather than a FK.
Customizing a plain formset, using formset_factory, eg DestinationActivityFormSet = formset_factory(DestinationActivityForm, extra=2). But how to design DestinationActivityForm? I haven't explored this enough, but it doesn't look very promising: I don't want to type in the destination and a list of activities, I want a list of checkboxes with the labels set to the destination / activities I want to select, but the formset_factory would return a list of forms with identical labels.
I'm a complete newbie with django so maybe the solution is obvious, but I find that the documentation in this area is very weak - if anyone has some pointers to examples of use for forms / formsets that would be also helpful
thanks!
In the end I opted for processing multiple forms within the same view, a Visitor model form for the visitor details, then a list of custom forms for each of the destinations.
Processing multiple forms in the same view turned out to be simple enough (at least in this case, where there were no cross-field validation issues).
I'm still surprised there is no built-in support for many to many relationships with an intermediary model, and looking around in the web I found no direct reference to it. I'll post the code in case it helps anyone.
First the custom forms:
class VisitorForm(ModelForm):
class Meta:
model = Visitor
exclude = ['destinations']
class VisitorDestinationForm(Form):
visited = forms.BooleanField(required=False)
activities = forms.MultipleChoiceField(choices = [(obj.pk, obj.name) for obj in Activity.objects.all()], required=False,
widget = CheckboxSelectMultipleInline(attrs={'style' : 'display:inline'}))
def __init__(self, visitor, destination, visited, *args, **kwargs):
super(VisitorDestinationForm, self).__init__(*args, **kwargs)
self.destination = destination
self.fields['visited'].initial = visited
self.fields['visited'].label= destination.destination
# load initial choices for activities
activities_initial = []
try:
visitorDestination_entry = VisitorDestination.objects.get(visitor=visitor, destination=destination)
activities = visitorDestination_entry.activities.all()
for activity in Activity.objects.all():
if activity in activities:
activities_initial.append(activity.pk)
except VisitorDestination.DoesNotExist:
pass
self.fields['activities'].initial = activities_initial
I customize each form by passing a Visitor and Destination objects (and a 'visited' flag which is calculated outside for convenience)
I use a boolean field to allow the user to select each destination. The field is called 'visited', however I set the label to the destination so it gets nicely displayed.
The activities get handled by the usual MultipleChoiceField (I used I customized widget to get the checkboxes to display on a table, pretty simple but can post it if somebody needs that)
Then the view code:
def edit_visitor(request, pk):
visitor_obj = Visitor.objects.get(pk=pk)
visitorDestinations = visitor_obj.destinations.all()
if request.method == 'POST':
visitorForm = VisitorForm(request.POST, instance=visitor_obj)
# set up the visitor destination forms
destinationForms = []
for destination in Destination.objects.all():
visited = destination in visitorDestinations
destinationForms.append(VisitorDestinationForm(visitor_obj, destination, visited, request.POST, prefix=destination.destination))
if visitorForm.is_valid() and all([form.is_valid() for form in destinationForms]):
visitor_obj = visitorForm.save()
# clear any existing entries,
visitor_obj.destinations.clear()
for form in destinationForms:
if form.cleaned_data['visited']:
visitorDestination_entry = VisitorDestination(visitor = visitor_obj, destination=form.destination)
visitorDestination_entry.save()
for activity_pk in form.cleaned_data['activities']:
activity = Activity.objects.get(pk=activity_pk)
visitorDestination_entry.activities.add(activity)
print 'activities: %s' % visitorDestination_entry.activities.all()
visitorDestination_entry.save()
success_url = reverse('visitor_detail', kwargs={'pk' : visitor_obj.pk})
return HttpResponseRedirect(success_url)
else:
visitorForm = VisitorForm(instance=visitor_obj)
# set up the visitor destination forms
destinationForms = []
for destination in Destination.objects.all():
visited = destination in visitorDestinations
destinationForms.append(VisitorDestinationForm(visitor_obj, destination, visited, prefix=destination.destination))
return render_to_response('testapp/edit_visitor.html', {'form': visitorForm, 'destinationForms' : destinationForms, 'visitor' : visitor_obj}, context_instance= RequestContext(request))
I simply collect my destination forms in a list and pass this list to my template, so that it can iterate over them and display them. It works well as long as you don't forget to pass a different prefix for each one in the constructor
I'll leave the question open for a few days in case some one has a cleaner method.
Thanks!
So, as you've seen, one of the things about inlineformset_factory is that it expects two models - a parent, and child, which has a foreign key relationship to the parent. How do you pass extra data on the fly to the form, for extra data in the intermediary model?
How I do this is by using curry:
from django.utils.functional import curry
from my_app.models import ParentModel, ChildModel, SomeOtherModel
def some_view(request, child_id, extra_object_id):
instance = ChildModel.objects.get(pk=child_id)
some_extra_model = SomeOtherModel.objects.get(pk=extra_object_id)
MyFormset = inlineformset_factory(ParentModel, ChildModel, form=ChildModelForm)
#This is where the object "some_extra_model" gets passed to each form via the
#static method
MyFormset.form = staticmethod(curry(ChildModelForm,
some_extra_model=some_extra_model))
formset = MyFormset(request.POST or None, request.FILES or None,
queryset=SomeObject.objects.filter(something=something), instance=instance)
The form class "ChildModelForm" would need to have an init override that adds the "some_extra_model" object from the arguments:
def ChildModelForm(forms.ModelForm):
class Meta:
model = ChildModel
def __init__(self, some_extra_model, *args, **kwargs):
super(ChildModelForm, self).__init__(*args, **kwargs)
#do something with "some_extra_model" here
Hope that helps get you on the right track.
From django 1.9, there is a support for passing custom parameters to formset forms :
https://docs.djangoproject.com/en/1.9/topics/forms/formsets/#passing-custom-parameters-to-formset-forms
Just add form_kwargs to your FormSet init like this :
from my_app.models import ParentModel, ChildModel, SomeOtherModel
def some_view(request, child_id, extra_object_id):
instance = ChildModel.objects.get(pk=child_id)
some_extra_model = SomeOtherModel.objects.get(pk=extra_object_id)
MyFormset = inlineformset_factory(ParentModel, ChildModel, form=ChildModelForm)
formset = MyFormset(request.POST or None, request.FILES or None,
queryset=SomeObject.objects.filter(something=something), instance=instance,
form_kwargs={"some_extra_model": some_extra_model})
I'm trying to do something that should be very common: add/edit a bunch of related models in a single form. For example:
Visitor Details:
Select destinations and activities:
Miami [] - swimming [], clubbing [], sunbathing[]
Cancun [] - swimming [], clubbing [], sunbathing[]
My models are Visitor, Destination and Activity, with Visitor having a ManyToMany field into Destination through an intermediary model, VisitorDestination, which has the details of the activities to be done on the destination (in itself a ManyToMany field into Activity).
Visitor ---->(M2M though VisitorDestination) -------------> Destination
|
activities ---->(M2M)---> Activity
Note that I don't want to enter new destination / activity values, just choose from those available in the db (but that's a perfectly legit use of M2M fields right?)
To me this looks like an extremely common situation (a many to many relation with additional details which are a FK or M2M field into some other model), and this looks like the most sensible modelling, but please correct me if I'm wrong.
I've spent a few days searching Django docs / SO / googling but haven't been able to work out how to deal with this. I tried several approaches:
Custom Model form for Visitor, where I add multiple choice fields for Destination and Activity. That works ok if Destination and Activity could be selected independently, but here they are correlated, ie I want to choose one or several activities for each destination
Using inlineformset_factory to generate the set of destination / activities forms, with inlineformset_factory(Destination, Visitor). This breaks, because Visitor has a M2M relation to Destination, rather than a FK.
Customizing a plain formset, using formset_factory, eg DestinationActivityFormSet = formset_factory(DestinationActivityForm, extra=2). But how to design DestinationActivityForm? I haven't explored this enough, but it doesn't look very promising: I don't want to type in the destination and a list of activities, I want a list of checkboxes with the labels set to the destination / activities I want to select, but the formset_factory would return a list of forms with identical labels.
I'm a complete newbie with django so maybe the solution is obvious, but I find that the documentation in this area is very weak - if anyone has some pointers to examples of use for forms / formsets that would be also helpful
thanks!
In the end I opted for processing multiple forms within the same view, a Visitor model form for the visitor details, then a list of custom forms for each of the destinations.
Processing multiple forms in the same view turned out to be simple enough (at least in this case, where there were no cross-field validation issues).
I'm still surprised there is no built-in support for many to many relationships with an intermediary model, and looking around in the web I found no direct reference to it. I'll post the code in case it helps anyone.
First the custom forms:
class VisitorForm(ModelForm):
class Meta:
model = Visitor
exclude = ['destinations']
class VisitorDestinationForm(Form):
visited = forms.BooleanField(required=False)
activities = forms.MultipleChoiceField(choices = [(obj.pk, obj.name) for obj in Activity.objects.all()], required=False,
widget = CheckboxSelectMultipleInline(attrs={'style' : 'display:inline'}))
def __init__(self, visitor, destination, visited, *args, **kwargs):
super(VisitorDestinationForm, self).__init__(*args, **kwargs)
self.destination = destination
self.fields['visited'].initial = visited
self.fields['visited'].label= destination.destination
# load initial choices for activities
activities_initial = []
try:
visitorDestination_entry = VisitorDestination.objects.get(visitor=visitor, destination=destination)
activities = visitorDestination_entry.activities.all()
for activity in Activity.objects.all():
if activity in activities:
activities_initial.append(activity.pk)
except VisitorDestination.DoesNotExist:
pass
self.fields['activities'].initial = activities_initial
I customize each form by passing a Visitor and Destination objects (and a 'visited' flag which is calculated outside for convenience)
I use a boolean field to allow the user to select each destination. The field is called 'visited', however I set the label to the destination so it gets nicely displayed.
The activities get handled by the usual MultipleChoiceField (I used I customized widget to get the checkboxes to display on a table, pretty simple but can post it if somebody needs that)
Then the view code:
def edit_visitor(request, pk):
visitor_obj = Visitor.objects.get(pk=pk)
visitorDestinations = visitor_obj.destinations.all()
if request.method == 'POST':
visitorForm = VisitorForm(request.POST, instance=visitor_obj)
# set up the visitor destination forms
destinationForms = []
for destination in Destination.objects.all():
visited = destination in visitorDestinations
destinationForms.append(VisitorDestinationForm(visitor_obj, destination, visited, request.POST, prefix=destination.destination))
if visitorForm.is_valid() and all([form.is_valid() for form in destinationForms]):
visitor_obj = visitorForm.save()
# clear any existing entries,
visitor_obj.destinations.clear()
for form in destinationForms:
if form.cleaned_data['visited']:
visitorDestination_entry = VisitorDestination(visitor = visitor_obj, destination=form.destination)
visitorDestination_entry.save()
for activity_pk in form.cleaned_data['activities']:
activity = Activity.objects.get(pk=activity_pk)
visitorDestination_entry.activities.add(activity)
print 'activities: %s' % visitorDestination_entry.activities.all()
visitorDestination_entry.save()
success_url = reverse('visitor_detail', kwargs={'pk' : visitor_obj.pk})
return HttpResponseRedirect(success_url)
else:
visitorForm = VisitorForm(instance=visitor_obj)
# set up the visitor destination forms
destinationForms = []
for destination in Destination.objects.all():
visited = destination in visitorDestinations
destinationForms.append(VisitorDestinationForm(visitor_obj, destination, visited, prefix=destination.destination))
return render_to_response('testapp/edit_visitor.html', {'form': visitorForm, 'destinationForms' : destinationForms, 'visitor' : visitor_obj}, context_instance= RequestContext(request))
I simply collect my destination forms in a list and pass this list to my template, so that it can iterate over them and display them. It works well as long as you don't forget to pass a different prefix for each one in the constructor
I'll leave the question open for a few days in case some one has a cleaner method.
Thanks!
So, as you've seen, one of the things about inlineformset_factory is that it expects two models - a parent, and child, which has a foreign key relationship to the parent. How do you pass extra data on the fly to the form, for extra data in the intermediary model?
How I do this is by using curry:
from django.utils.functional import curry
from my_app.models import ParentModel, ChildModel, SomeOtherModel
def some_view(request, child_id, extra_object_id):
instance = ChildModel.objects.get(pk=child_id)
some_extra_model = SomeOtherModel.objects.get(pk=extra_object_id)
MyFormset = inlineformset_factory(ParentModel, ChildModel, form=ChildModelForm)
#This is where the object "some_extra_model" gets passed to each form via the
#static method
MyFormset.form = staticmethod(curry(ChildModelForm,
some_extra_model=some_extra_model))
formset = MyFormset(request.POST or None, request.FILES or None,
queryset=SomeObject.objects.filter(something=something), instance=instance)
The form class "ChildModelForm" would need to have an init override that adds the "some_extra_model" object from the arguments:
def ChildModelForm(forms.ModelForm):
class Meta:
model = ChildModel
def __init__(self, some_extra_model, *args, **kwargs):
super(ChildModelForm, self).__init__(*args, **kwargs)
#do something with "some_extra_model" here
Hope that helps get you on the right track.
From django 1.9, there is a support for passing custom parameters to formset forms :
https://docs.djangoproject.com/en/1.9/topics/forms/formsets/#passing-custom-parameters-to-formset-forms
Just add form_kwargs to your FormSet init like this :
from my_app.models import ParentModel, ChildModel, SomeOtherModel
def some_view(request, child_id, extra_object_id):
instance = ChildModel.objects.get(pk=child_id)
some_extra_model = SomeOtherModel.objects.get(pk=extra_object_id)
MyFormset = inlineformset_factory(ParentModel, ChildModel, form=ChildModelForm)
formset = MyFormset(request.POST or None, request.FILES or None,
queryset=SomeObject.objects.filter(something=something), instance=instance,
form_kwargs={"some_extra_model": some_extra_model})