Django JSONField not receiving cleaned data - django

I have a very simple model which contains a JSONField:
class Thing(models.Model):
title = models.CharField(max_length=1024)
text = JSONField(default=dict)
I've created a custom widget that allows for the input of key-value pairs:
class JsonWidget(forms.widgets.Widget):
template_name = 'json_widget.html'
def get_context(self, name, value, attrs):
context = super().get_context(name, value, attrs)
data = json.loads(value)
if not data:
data = JSON_DEFAULT
context['data'] = data.items()
return context
def value_from_datadict(self, data, files, name):
keys = data.getlist('json-key')
values = data.getlist('json-value')
json_data = {k: v for k, v in zip(keys, values)}
return json_data
I coerce the dict returned by the widget into a string in the field's clean function on the form:
class ThingForm(forms.ModelForm):
class Meta:
model = Thing
fields = ['title', 'text']
widgets = {
'text': JsonWidget(),
}
def clean_text(self):
text = self.cleaned_data.get('text')
return json.dumps(text)
I've inspected the output of JsonWidget.value_from_datadict (dict) and ThingForm.clean_text (str) are the expected types. But when the object goes to save it throws an exception:
TypeError: the JSON object must be str, bytes or bytearray, not 'dict'
This is my first time building a custom widget for Django 1.11, is there something obvious I've missed here?
Thanks!

This was a tricky one but I eventually traced the problem back to Instance construction fails on form data check
While the JSONField is called text on the model, there's no matching field on the ModelForm. Instead, the key/value pairs are compiled & turned into a dict by the widget's value_from_datadict. However, the cleaned_data values are saved back to the instance only if the field name exists in the form's POST data. This caused the widget to throw an error because text could not be blank, and the TypeError exception was raised during the form re-render.
The workaround was to add a hidden input called text to the field and only use it to bypass the field name check during instantion.

Related

How do you save Textarea form data line by line using Django's CreateView?

I have a Model and would like to save data as a batch by using a textarea form in Django. The data shall be save line by line, which I am using splitlines(), each data is comma separated by split(","). I am operating the manipulation in the form_valid() function but I can't seem to get it right.
Only the last line is saved successfully.
forms.py
class DataForm(forms.ModelForm):
textarea_data = forms.CharField(widget=forms.Textarea)
class Meta:
model = Item
exclude = ('part_number','length','height','weight')
views.py
class InsertData(generic.CreateView):
model = Item
form_class = DataForm
def get_success_url(self):
return reverse('item_list')
def form_valid(self, form):
self.object = form.save(commit=False)
textarea_data = form.cleaned_data['textarea_data ']
data_line_list = textarea_data.splitlines()
for each_line in data_line_list:
each_line_list = each_line.split(",")
self.object.part_number = each_line_list[0]
self.object.length = each_line_list[1]
self.object.weight = each_line_list[2]
self.object.height = each_line_list[3]
self.object.save()
May I know where did I go wrong. Should the manipulation be done some where else?
self.object.save() keeps overwriting your object. You want to create separate db records from each line of your textarea input.
Better create a simple form forms.Form for data entry instead of ModelForm.
Then on the its form_valid, iterate through the lines and create objects. Each iteration should create a new object and save.
You need to get rid of the textarea field as well.
If its a lot of data, it’s better to use bulk create function for performance.
I found the solution after reading this https://stackoverflow.com/a/33027228/13152307 .
I should set the primary key to none at the beginning of each iteration.
for each_line in data_line_list:
self.object.pk = None #add this line
each_line_list = each_line.split(",")
self.object.part_number = each_line_list[0]
self.object.length = each_line_list[1]
self.object.weight = each_line_list[2]
self.object.height = each_line_list[3]
self.object.save()
I understand that it may not be a good way to do so, but at least it works now.

Django "ValueError: Can't bulk create a multi-table inherited model"

Problem
I am using the django-model-utils InheritanceManager. I have a super Notification(models.Model) class which I use to create many notification subclasses such as PostNotification(Notification), CommentNotification(Notification), etc., and when trying to run CommentNotification.objects.bulk_create(list_of_comment_notification_objects), i get the following traceback:
File "/home/me/.virtualenvs/project/local/lib/python2.7/site-packages/django/db/models/query.py", line 429, in bulk_create
raise ValueError("Can't bulk create a multi-table inherited model")
ValueError: Can't bulk create a multi-table inherited model
and upon inspecting the query.py file, we get this causes the error:
for parent in self.model._meta.get_parent_list():
if parent._meta.concrete_model is not self.model._meta.concrete_model:
raise ValueError("Can't bulk create a multi-table inherited model")
Environment
Django Model Utils version: 3.1.1
Django version: 1.11.7
Python version: 2.7.3
Example
PostNotification.objects.bulk_create(
[PostNotification(related_user=user, post=instance) for user in users]
)
throws the above exception
What I have tried and though was a success originally:
I though that simply running:
BaseClass.objects.bulk_create(list_of_SubClass_objects) instead of SubClass.objects.bulk_create(list_of_SubClass_objects) would work and return a list of SubClass values, but subsequently running SubClass.objects.all() would return an empty result. The bulk_create() would only create a Notification base class object for each item in the list.
Found a hacky solution. I hope it works in your case. The trick is create a model (which is not an inherited one) dynamically that has some meta (db_table) set. And use this dynamic model to create Child objects in bulk (in other words write into Child's DB table).
class Parent(models.Model):
name = models.CharField(max_length=10)
class Child(Parent):
phone = models.CharField(max_length=12)
# just an example. Should be expanded to work properly.
field_type_mapping = {
'OneToOneField': models.IntegerField,
'CharField': models.CharField,
}
def create_model(Model, app_label='children', module='', options=None):
"""
Create specified model
"""
model_name = Model.__name__
class Meta:
managed = False
db_table = Model._meta.db_table
if app_label:
# app_label must be set using the Meta inner class
setattr(Meta, 'app_label', app_label)
# Update Meta with any options that were provided
if options is not None:
for key, value in options.iteritems():
setattr(Meta, key, value)
# Set up a dictionary to simulate declarations within a class
attrs = {'__module__': module, 'Meta': Meta}
# Add in any fields that were provided
fields = dict()
for field in Model._meta.fields:
if field.attname == 'id':
continue
if field.model.__name__ == model_name:
field_class_name = type(field).__name__
print(field.attname)
fields[field.attname] = field_type_mapping[field_class_name]()
# Create the class, which automatically triggers ModelBase processing
attrs.update(fields)
model = type(f'{model_name}Shadow', (models.Model,), attrs)
return model
mod = create_model(Child)
parents = [Parent(name=i) for i in range(15)]
parents = Parent.objects.bulk_create(parents)
children = [mod(phone=parent.name, parent_ptr_id=parent.id) for parent in parents]
mod.objects.bulk_create(children)
I've done a custom implementation of bulk_create that seems to be working for my case (only one parent relationship and not autoincremented pk):
from django.db import models
class MultiTableChildQueryset(models.QuerySet):
def bulk_create(self, objs, batch_size=None):
assert batch_size is None or batch_size > 0
if not objs:
return objs
self._for_write = True
objs = list(objs)
parent_model = self.model._meta.pk.related_model
parent_objs = []
for obj in objs:
parent_values = {}
for field in [f for f in parent_model._meta.fields if hasattr(obj, f.name)]:
parent_values[field.name] = getattr(obj, field.name)
parent_objs.append(parent_model(**parent_values))
setattr(obj, self.model._meta.pk.attname, obj.id)
parent_model.objects.bulk_create(parent_objs, batch_size=batch_size)
with transaction.atomic(using=self.db, savepoint=False):
self._batched_insert(objs, self.model._meta.local_fields, batch_size)
return objs
A slightly easier to read version of Moises:
from typing import TypeVar
from django.db.models import Model
M = TypeVar('M', bound=Model)
def multi_inheritance_table_bulk_insert(data: List[M]) -> None:
"""
Bulk insert data into a multi-inheritance table.
"""
if not data:
return
model = data[0].__class__
local_fields = model._meta.local_fields
parent_model = model._meta.pk.related_model
parent_fields = parent_model._meta.local_fields
parent_objects = [
parent_model(**{field.name: getattr(obj, field.name) for field in parent_fields})
for obj in data
]
parent_model.objects.bulk_create(parent_objects)
for parent, obj in zip(parent_objects, data):
obj.pk = parent.pk
queryset = QuerySet(model)
queryset._for_write = True
with transaction.atomic(using=queryset.db, savepoint=False):
queryset._batched_insert(
data,
local_fields,
batch_size=None,
)

overriding django formset initial data dynamically

Ok im new to django
So ive got a situation where i want a formset to have dynamic initial data
So basically here is what im looking for.
each form in the formset to have a different UserID
and a set of groups permission which they can choose from based from the initial data
here is my form
class assignGroupPermissionToUser(forms.ModelForm):
UserID = forms.ModelChoiceField(queryset=None)
Groups = forms.ModelMultipleCHoiceField(queryset=None, widget=FilteredSelectMultiple("Groups")
class Meta:
model=User
def __init__(self,*args,**kwargs):
super().__init__(*args,**kwargs)
Userid = kwargs.pop("UserID")
self.fields['UserID'].queryset =User.objects.get(UserID=Userid)
Permissions = kwargs.pop("Groups")
listofPermission = None
for each perm in permission:
listofPermission |= Permissions.objects.filter(GroupID=perm)
self.fields['Groups'].queryset = listofPermission
the data i wanna pass is built into a list like so
it is called
completeList
> completeList =[['13452',{'group1':'Admin','group2':'FrontDesk'}],['3532','group1':'Supervisors','group2':'ReadOnly;}]]
where the first value in each nested loop is the UserID, and the dictionary is the groups they can choose from.
override method in View.py
....
form = assignGroupPermissionToUser()
assignment = formset_factory(form,extra=0)
formset = [ assignment.__init__(completeList[x][0],completeList[x][1]) for x in range(len(completeList))]
then i get an error that str object has no 'is_bound' field line 58 of formset.py
im trytin to get this data to show up on each form and based on the user
it will be all different but everything i try to override it fails for initial form so here i am stuck
note that the Group attribute in the modelform has a widget which is used in the admin section to filter from multiple choices.
settings
Django= 1.8
python 3.5
i erased all this code and just did two loops like so
formset = assignments(initial=[{'UserID': listofUserID[x] } for x in range(len(completeList))])
#then
for form in formset:
form.fields['permissions'].queryset = querysetiwant

How do I save a Django Model from request.POST?

I'm trying to save a new object from a django model using a POST data querydict. This is part of a PISTON handler. I've seen this done in numerous examples, but I just can't get it to work.
Here is my code:
class FestHandler(BaseHandler):
model = Deal
def create(self, request):
"""
Creates a new fest.
"""
attrs = self.flatten_dict(request.POST)
postcopy = request.POST.copy()
if self.exists(**attrs):
return rc.DUPLICATE_ENTRY
else:
loc = Location.objects.get(pk=attrs['location'])
postcopy['location'] = loc
fest = Fest(postcopy)
fest.save()
return fest
Here is the error I receive every time:
Exception was: int() argument must be a string or a number, not 'QueryDict'
I realize what the error means, so basically I am asking how I can save a new "Fest" by passing in the whole POST dictionary without having to type in the keys manually every time like this:
loc = Location.objects.get(pk=attrs['location'])
fest = Fest(
location=loc,
name=attrs['name'],
description=attrs['description'],
details=attrs['details'],
)
Thanks for your help!
First, I think you'll be happier of you explicitly make your PK's into integers.
loc = Location.objects.get(pk=int(attrs['location']))
Second, you should use a Form.
It validates the fields for you.
It will create the Model object from the Form object.
Read this. http://docs.djangoproject.com/en/1.2/topics/forms/modelforms/

Adding data to many-to-many field of a modelform within a view

class Person(models.Model):
name = models.CharField(max_length=100)
class Entry(models.Model):
text = models.CharField(max_length=100)
person = models.ManyToManyField(Person, blank=True, null=True)
class MyModelForm(forms.ModelForm):
class Meta:
model = Entry
In my view, I need to add pk id's to a submitted form before saving it.
data = request.POST.copy()
# 'person' is a ManyToManyField to a 'Person' model
# a form would normally send multiple id's as POST in this format -> u'id': [u'1', u'2']
# u'1,2' (an example) is a str variable accessible to the view
data[u'person'] = u'1,2'.split(",")
form = MyModelForm(data)
if form.is_valid():
form.save()
This gives me:
int() argument must be a string or a
number, not 'list'
Which is fair enough. It does work in case of:
data[u'person'] = u'1'
I also tried this with no success:
new_form = form.save(commit=False)
new_form.person = u'1,2'.split(",")
new_form.save()
form.save_m2m()
How can I save multiple id's to this ManyToManyField?
Must be easy but I am missing the point.
EDIT:
The desired result is one new instance of MyModelForm (in the 'entry' table) with all id's stored for form.person (multiple records in the 'entry_person' table).
UPDATE:
I seem to have isolated the problem.
If I do:
data = {}
data[u'person'] = u'1,2'.split(",")
It does work as the result is:
{u'person': [u'1', u'2'],}
If I do:
data = request.POST.copy()
data[u'person'] = u'1,2'.split(",")
It does NOT work (gives the error described above) as the result is:
<QueryDict: {u'person': [[u'1', u'2']],}>
So all I need is to have
<QueryDict: {u'person': [u'1', u'2'],}>
Any suggestions how?
QueryDict.setlist(key, list_) solves this problem.
Answered in A workaround for Django QueryDict wrapping values in lists?
The split you entered returns the following:
[u'1', u'2']
To create multiple instances in your database you'd need to iterate over the list:
for x in u'1,2'.split(","):
data[u'person'] = x
form = MyModelForm(data)
if form.is_valid():
form.save()
Out of curiosity, why string to list conversion in the first place? Or is this just a general example
Try:
new_form.person = [int(x) for x in u'1,2'.split(",")]
This will set new_form.person to a list of ints, not a list of strings.