Lazy loading a model field's choices - django

I'm building a Django app to pull in data via an API to track live results of an event with the added ability to override that data before it is displayed.
The first task of the app is to make a request and store the response in the database so I've setup a model;
class ApiData(models.Model):
event = models.CharField(
_("Event"),
max_length=100,
)
key = models.CharField(
_("Data identifier"),
max_length=255,
help_text=_("Something to identify the json stored.")
)
json = JSONField(
load_kwargs={'object_pairs_hook': collections.OrderedDict},
blank=True,
null=True,
)
created = models.DateTimeField()
Ideally I would like it so that objects are created in the admin and the save method populates the ApiData.json field after creating an API request based on the other options in the object.
Because these fields would have choices based on data returned from the API I wanted to lazy load the choices but at the moment I'm just getting a standard Charfield() in my form.
Is this the correct approach for lazy loading model field choices? Or should I just create a custom ModelForm and load the choices there? (That's probably the more typical approach I guess)
def get_event_choices():
events = get_events()
choices = []
for event in events['events']:
choices.append((event['name'], event['title']),)
return choices
class ApiData(models.Model):
# Fields as seen above
def __init__(self, *args, **kwargs):
super(ApiData, self).__init__(*args, **kwargs)
self._meta.get_field_by_name('event')[0]._choices = lazy(
get_event_choices, list
)()

So I went for a typical approach to get this working by simply defining a form for the model admin to use;
# forms.py
from django import forms
from ..models import get_event_choices, ApiData
from ..utils.api import JsonApi
EVENT_CHOICES = get_event_choices()
class ApiDataForm(forms.ModelForm):
"""
Form for collecting the field choices.
The Event field is populated based on the events returned from the API.
"""
event = forms.ChoiceField(choices=EVENT_CHOICES)
class Meta:
model = ApiData
# admin.py
from django.contrib import admin
from .forms.apidata import ApiDataForm
from .models import ApiData
class ApiDataAdmin(admin.ModelAdmin):
form = ApiDataForm
admin.site.register(ApiData, ApiDataAdmin)

Related

Reload choices dynamically when using MultipleChoiceFilter

I am trying to construct a MultipleChoiceFilter where the choices are the set of possible dates that exist on a related model (DatedResource).
Here is what I am working with so far...
resource_date = filters.MultipleChoiceFilter(
field_name='dated_resource__date',
choices=[
(d, d.strftime('%Y-%m-%d')) for d in
sorted(resource_models.DatedResource.objects.all().values_list('date', flat=True).distinct())
],
label="Resource Date"
)
When this is displayed in a html view...
This works fine at first, however if I create new DatedResource objects with new distinct date values I need to re-launch my webserver in order for them to get picked up as a valid choice in this filter. I believe this is because the choices list is evaluated once when the webserver starts up, not every time my page loads.
Is there any way to get around this? Maybe through some creative use of a ModelMultipleChoiceFilter?
Thanks!
Edit:
I tried some simple ModelMultipleChoice usage, but hitting some issues.
resource_date = filters.ModelMultipleChoiceFilter(
field_name='dated_resource__date',
queryset=resource_models.DatedResource.objects.all().values_list('date', flat=True).order_by('date').distinct(),
label="Resource Date"
)
The HTML form is showing up just fine, however the choices are not accepted values to the filter. I get "2019-04-03" is not a valid value. validation errors, I am assuming because this filter is expecting datetime.date objects. I thought about using the coerce parameter, however those are not accepted in ModelMultipleChoice filters.
Per dirkgroten's comment, I tried to use what was suggested in the linked question. This ends up being something like
resource_date = filters.ModelMultipleChoiceFilter(
field_name='dated_resource__date',
to_field_name='date',
queryset=resource_models.DatedResource.objects.all(),
label="Resource Date"
)
This also isnt what I want, as the HTML now form is now a) displaying the str representation of each DatedResource, instead of the DatedResource.date field and b) they are not unique (ex if I have two DatedResource objects with the same date, both of their str representations appear in the list. This also isnt sustainable because I have 200k+ DatedResources, and the page hangs when attempting to load them all (as compared to the values_list filter, which is able to pull all distinct dates out in seconds.
One of the easy solutions will be overriding the __init__() method of the filterset class.
from django_filters import filters, filterset
class FooFilter(filterset.FilterSet):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
try:
self.filters['user'].extra['choices'] = [(d, d.strftime('%Y-%m-%d')) for d in sorted(
resource_models.DatedResource.objects.all().values_list('date', flat=True).distinct())]
except (KeyError, AttributeError):
pass
resource_date = filters.MultipleChoiceFilter(field_name='dated_resource__date', choices=[], label="Resource Date")
NOTE: provide choices=[] in your field definition of filterset class
Results
I tested and verified this solution with following dependencies
1. Python 3.6
2. Django 2.1
3. DRF 3.8.2
4. django-filter 2.0.0
I used following code to reproduce the behaviour
# models.py
from django.db import models
class Musician(models.Model):
name = models.CharField(max_length=50)
def __str__(self):
return f'{self.name}'
class Album(models.Model):
artist = models.ForeignKey(Musician, on_delete=models.CASCADE)
name = models.CharField(max_length=100)
release_date = models.DateField()
def __str__(self):
return f'{self.name} : {self.artist}'
# serializers.py
from rest_framework import serializers
class AlbumSerializer(serializers.ModelSerializer):
artist = serializers.StringRelatedField()
class Meta:
fields = '__all__'
model = Album
# filters.py
from django_filters import rest_framework as filters
class AlbumFilter(filters.FilterSet):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.filters['release_date'].extra['choices'] = self.get_album_filter_choices()
def get_album_filter_choices(self):
release_date_list = Album.objects.values_list('release_date', flat=True).distinct()
return [(date, date) for date in release_date_list]
release_date = filters.MultipleChoiceFilter(choices=[])
class Meta:
model = Album
fields = ('release_date',)
# views.py
from rest_framework.viewsets import ModelViewSet
from django_filters import rest_framework as filters
class AlbumViewset(ModelViewSet):
serializer_class = AlbumSerializer
queryset = Album.objects.all()
filter_backends = (filters.DjangoFilterBackend,)
filter_class = AlbumFilter
Here I've used the django-filter with DRF.
Now, I populated some data through Django Admin console. After that, the album api become as below,
and I got the release_date as
Then, I added new entry through Django admin -- (Screenshot) and I refresh the DRF API endpoint and the possible choices became as below,
I have looked into your problem and I have following suggestions
The Problem
You have got the problem right. Choices for your MultipleChoiceFilter are calculated statically whenever you run server.Thats why they don't get updated dynamically whenever you insert new instance in DatedResource.
To get it working correctly, you have to provide choices dynamically to MultipleChoiceFilter. I searched in documentation but did not find anything regarding this. So here is my solution.
The solution
You have to extend MultipleChoiceFilter and create your own filter class. I have created this and here it is.
from typing import Callable
from django_filters.conf import settings
import django_filters
class LazyMultipleChoiceFilter(django_filters.MultipleChoiceFilter):
def get_field_choices(self):
choices = self.extra.get('choices', [])
if isinstance(choices, Callable):
choices = choices()
return choices
#property
def field(self):
if not hasattr(self, '_field'):
field_kwargs = self.extra.copy()
if settings.DISABLE_HELP_TEXT:
field_kwargs.pop('help_text', None)
field_kwargs.update(choices=self.get_field_choices())
self._field = self.field_class(label=self.label, **field_kwargs)
return self._field
Now you can use this class as replacement and pass choices as lambda function like this.
resource_date = LazyMultipleChoiceFilter(
field_name='dated_resource__date',
choices=lambda: [
(d, d.strftime('%Y-%m-%d')) for d in
sorted(resource_models.DatedResource.objects.all().values_list('date', flat=True).distinct())
],
label="Resource Date"
)
Whenever instance of filter will be created choices will be updated dynamically. You can also pass choices statically (without lambda function) to this field if want default behavior.

Django-ViewFlow: How to add CRUD views to flow

I've recently come across the Viewflow library for Django which I appears to be a very powerful tool for creating complex workflows.
My app is a simple ticketing system were the workflow is started by creating a Ticket, then a user should be able to create zero or more WorkLog's associated with the ticket via a CRUD page(s), similar to the standard Django admin change_list/detail.
What should the template for the list view look like? I would like to have the UI integrated into the library's frontend.
The flow clearly utilises the following views:
1) CreateView for Ticket
2a) ListView of WorkLog's, template has controls 'back', 'add' (goes to step 2b), 'done' (goes to step 3).
2b) CreateView for WorkLog
3) End
Code:
models.py:
class TicketProcess(Process):
title = models.CharField(max_length=100)
category = models.CharField(max_length=150)
description = models.TextField(max_length=150)
planned = models.BooleanField()
worklogs = models.ForeignKey('WorkLog', null=True)
class WorkLog(models.Model):
ref = models.CharField(max_length=32)
description = models.TextField(max_length=150)
views.py:
class WorkLogListView(FlowListMixin, ListView):
model = WorkLog
class WorkLogCreateView(FlowMixin, CreateView):
model = WorkLog
fields = '__all__'
flows.py:
from .views import WorkLogCreateView
from .models import TicketProcess
#frontend.register
class TicketFlow(Flow):
process_class = TicketProcess
start = (
flow.Start(
CreateProcessView,
fields = ['title', 'category', 'description', 'planned']
).Permission(
auto_create=True
).Next(this.resolution)
)
add_worklog = (
flow.View(
WorkLogListView
).Permission(
auto_create=True
).Next(this.end)
)
end = flow.End()
You can handle that in a different view, or in the same view, just don't call activation.done on a worklog adding request. You can do it by checking what button was pressed in the request.POST data.
#flow.flow_view
def worklog_view(request):
request.activation.prepare(request.POST or None, user=request.user)
if '_logitem' in request.POST:
WorkLog.objects.create(...)
elif request.POST:
activation.done()
request.activation.done()
return redirect(get_next_task_url(request, request.activation.process))
return render(request, 'sometemplate.html', {'activation': request.activation})

django model - fetching user data accross multiple tables

I am writing a django (1.10) website and using allauth for authorisations. I don't want to extend the user model in django - because allauth adds a further layer of complexity to what is already a seemingly convoluted process.
I want to create a model (Custom UserManager?) that will have the following methods:
get_all_subscriptions_for_user(user=specified_user)
get_unexpired_subscriptions_for_user(user=specified_user)
Note: unexpired subscriptions are defined by subscriptions whose end_date > today's date.
This is a snippet of my models.py below
from django.db import models
from django.contrib.auth.models import User
#...
class Subscription(models.Model):
token = models.CharKey()
start_date = models.DateTime()
end_date = models.DateTime()
# other attributes
class UserSubscription(models.Model):
user = models.ForeignKey(User)
subscription = models.ForeignKey(Subscription)
# In view
def foo(request):
user = User.objects.get(username=request.user)
# how can I implement the following methods:
# get_all_subscriptions_for_user(user=specified_user)
# get_unexpired_subscriptions_for_user(user=specified_user)
Ideally, I would like to have a custom user manager, which can fetch this data in one trip to the database - but I'm not sure if I can have a custom user manager without having a custom user model.
[[Aside]]
I'm trying to avoid using a custom model, because it wreaks havoc on the other applications (in my project) which have User as a FK. makemigrations and migrate always barf with a message about inconsistent migration history
You can go with a custom Manager, don't need a UserManager since you are fetching related models:
class UserSubscriptionManager(models.Manager):
def for_user(self, user):
return super(UserSubscriptionManager, self).get_queryset().filter(user=user)
def unexpired_for(self, user):
return self.for_user(user).filter(
suscription__end_date__gt=datetime.date.today() # import datetime
)
in your models:
class UserSubscription(models.Model):
user = models.ForeignKey(User)
subscription = models.ForeignKey(Subscription)
user_objects = UserSubscriptionManager()
this way you can do chain filters in the view, for example:
unexpired_suscriptions = UserSubscription.user_objects().unexpired_for(
user=request.user
).exclude(suscription__token='invalid token')
Try this:
response = []
user_sub = UserSubscription.objects.filter(user=user.pk)
for row in user_sub:
subscription = Subscription.objects.get(pk=row.subscription)
end_date = subscription.end_date
if end_date > timezone.now():
response.append(subscription)

Combining unrelated Models in Formset and saving results

These are the models related to my problem:
Models.py
class SequenceDiagram(models.Model):
name = models.TextField(blank=True)
attributeMappingName = models.TextField(blank=True)
class AttributeFilter(models.Model):
seqDiagram = models.ForeignKey(SequenceDiagram)
attributeName = models.TextField(blank=True)
protocol = models.TextField()
isDisplayed = models.BooleanField(default=False)
class AttributeMapping(models.Model):
mappingName = models.TextField()
protocol = models.TextField(blank=True)
nativeName = models.TextField(blank=True)
customName = models.TextField(blank=True
Filters are specific to each SequenceDiagram but mappings are generic and applicable to different Diagrams.
I want a Formset with all AttributeFilters and AttributeMappings linked with the SequenceDiagram.
These are to be displayed in a table where isDisplayed and customName can be edited and then saved to the database.
How can I combine them to a Formset and then save the users changes?
Can a many-to-many relationship help solve my problem? If so, in which end should it be defined?
Please tell me if anything needs to be clarified.
edit
The resulting table should look like this:
Protocol|Native|Custom|Display
prot1 | Nat1 | Cus1 | Chkbx1
prot2 | Nat2 | Cus2 | Chkbx2
.......
So that matching customNames and isDisplayed are aligned.
I have tried using objects.extra() but I can't seem to save the changes to the 'other' model, I also don't know how to get the queryset to a Formfield and back.
AttributeFilter.objects.extra(
select={"protocol":"protocol", "sd":"sdAttributeName"},
where=["customName=nativeName"],
tables=["project_attributemapping"])
You can display both forms in the template and the process the forms separately in the view.
Here, we will be using ModelForms. We will create ModelForm for each model and then save all the models in a single view.
forms.py
from django import forms
from my_app.models import SequenceDiagram, AttributeFilter, AttributeMapping
class SequenceDiagramForm(forms.ModelForm):
class Meta:
model = SequenceDiagram
class AttributeFilterForm(forms.ModelForm):
class Meta:
model = AttributeFilter
exclude = (seqDiagram,)
class AttributeMappingForm(forms.ModelForm):
class Meta:
model = AttributeMapping
views.py
from django.views.generic import View
class MyView(View):
def post(self, request, *args, **kwargs):
sequence_diagram_form = SequenceDiagramForm(request.POST) # create form instance and populate with data
attribute_filter_form = AttributeFilterForm(request.POST) # create form instance and populate with data
attribute_mapping_form = AttributeMappingForm(request.POST) # create form instance and populate with data
sequence_diagram_form_valid = sequence_diagram_form.is_valid() # check if 'SequenceDiagramForm' is valid
attribute_filter_form_valid = attribute_filter_form.is_valid() # check if 'AttributeFilterForm' is valid
attribute_mapping_form_valid = attribute_mapping_form.is_valid() # check if 'AttributeMappingForm' is valid
# Check if all the forms are valid
if sequence_diagram_form_valid and attribute_filter_form_valid and attribute_mapping_form_valid:
sequence_diagram_obj = sequence_diagram_form.save() # save the SequenceDiagram object
attribute_filter_obj = attribute_filter_form.save(commit=False) # not save but get the instance
attribute_mapping_obj = attribute_mapping_form.save() # save the AttributeMapping object
attribute_filter_obj.seqDiagram = sequence_diagram_obj # set the `seqDiagram` to `sequence_diagram_obj`
attribute_filter_obj.save() # Now save the AttributeFilter object
...
# redirect to success page on all all forms being valid
...
# render the page again with errors if any of the form is invalid
In our view, we check if all the forms are valid and then only save all the 3 objects into db. If any of the form is invalid, then we don't save any of them.
For the case when any of the form is invalid, you can add the code to render the page again with the form errors.

Django Rest Framework Update or Create if not exists

I want to use the perform an update or create in django-rest-framework, by passing or not the id field. I've got this model
class Etiqueta(models.Model):
name_tag = models.CharField(max_length=200, blank=False, null=False)
description_tag = models.TextField(max_length=500, blank=False, null=False)
def __unicode__(self):
return self.name_tag
And in django-rest-framework I've got this serializer
from myapp.modulos.estado_1.models import Etiqueta
from rest_framework import serializers, viewsets
# Serializers define the API representation.
class TagSerializer(serializers.ModelSerializer):
class Meta:
model = Etiqueta
fields = (
'id',
'name_tag',
'description_tag'
)
# ViewSets define the view behavior.
class TagViewSet(viewsets.ModelViewSet):
queryset = Etiqueta.objects.all()
serializer_class = TagSerializer
Normally when I create an object, I perform a POST to the URL without the /:id, but if I've got an object with a local id, I want him to be created in the REST with the same id (remote id), django overwrite my local id and creates a new one. Does anybody know how achieve this? Also it is important to mention that I'm working with google-app-engine, google-cloud-datastore and django-dbindexer.
This code should work for your case -
class TagViewSet(viewsets.ModelViewSet):
queryset = Etiqueta.objects.all()
serializer_class = TagSerializer
def get_object(self):
if self.request.method == 'PUT':
obj, created = Etiquetta.objects.get_or_create(pk=self.kwargs.get('pk'))
return obj
else:
return super(TagViewSet, self).get_object()
You should have a look at how Django REST framework does currently and adapts your create method to update whenever you have an id field.
The original ViewSet.create is here and the ViewSet.update is here.
Please note that you will probably end up with two different serializers for /tag/ and /tag/:id since the later should not allow the id field to be writable while the former should.
I've write a drf views mixin for updating an object by id, if no corresponding object, just create it then update.