How to use timezones in Django Forms - django

Timezones in Django...
I am not sure why this is so difficult, but I am stumped.
I have a form that is overwriting the UTC dateTime in the database with the localtime of the user. I can't seem to figure out what is causing this.
my settings.py timezone settings look like:
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'America/Toronto'
USE_I18N = True
USE_L10N = False
USE_TZ = True
I am in Winnipeg, my server is hosted in Toronto. My users can be anywhere.
I have a modelfield for each user that is t_zone = models.CharField(max_length=50, default = "America/Winnipeg",) which users can change themselves.
with respect to this model:
class Build(models.Model):
PSScustomer = models.ForeignKey(Customer, on_delete=models.CASCADE)
buildStart = models.DateTimeField(null=True, blank=True)
...
I create a new entry in the DB using view logic like:
...
now = timezone.now()
newBuild = Build(author=machine,
PSScustomer = userCustomer,
buildStart = now,
status = "building",
addedBy = (request.user.first_name + ' ' +request.user.last_name),
...
)
newBuild.save()
buildStart is saved to the database in UTC, and everything is working as expected. When I change a user's timezone in a view with timezone.activate(pytz.timezone(self.request.user.t_zone)) it will display the UTC time in their respective timezone.
All is good (I think) so far.
Here is where things go sideways:
When I want a user to change buildStart in a form, I can't seem to get the form to save the date to the DB in UTC. It will save to the DB in whatever timezone the user has selected as their own.
Using this form:
class EditBuild_building(forms.ModelForm):
buildStart = forms.DateTimeField(input_formats = ['%Y-%m-%dT%H:%M'],widget = forms.DateTimeInput(attrs={'type': 'datetime-local','class': 'form-control'},format='%Y-%m-%dT%H:%M'), label = "Build Start Time")
def __init__(self, *args, **kwargs):# for ensuring fields are not left empty
super(EditBuild_building, self).__init__(*args, **kwargs)
self.fields['buildDescrip'].required = True
class Meta:
model = Build
fields = ['buildDescrip', 'buildStart','buildLength'...]
labels = {
'buildDescrip': ('Build Description'),
'buildStart': ('Build Start Time'),
...
}
widgets = {'buildDescrip': forms.TextInput(attrs={'class': 'required'}),
and this view:
class BuildUpdateView_Building(LoginRequiredMixin,UpdateView):
model = Build
form_class = EditBuild_building
template_name = 'build_edit_building.html'
login_url = 'login'
def get(self, request, *args, **kwargs):
proceed = True
try:
instance = Build.objects.get(id = (self.kwargs['pk']))
except:
return HttpResponse("<h2 style = 'margin:2em;'>This build is no longer available it has been deleted, please please return to dashboard</h2>")
if instance.buildActive == False:
proceed = False
if instance.deleted == True:
proceed = False
#all appears to be well, process request
if proceed == True:
form = self.form_class(instance=instance)
timezone.activate(pytz.timezone(self.request.user.t_zone))
customer = self.request.user.PSScustomer
choices = [(item.id, (str(item.first_name) + ' ' + str(item.last_name))) for item in CustomUser.objects.filter(isDevice=False, PSScustomer = customer)]
choices.insert(0, ('', 'Unconfirmed'))
form.fields['buildStrategyBy'].choices = choices
form.fields['buildProgrammedBy'].choices = choices
form.fields['operator'].choices = choices
form.fields['powder'].queryset = Powder.objects.filter(PSScustomer = customer)
context = {}
context['buildID'] = self.kwargs['pk']
context['build'] = Build.objects.get(id = (self.kwargs['pk']))
return render(request, self.template_name, {'form': form, 'context': context})
else:
return HttpResponse("<h2 style = 'margin:2em;'>This build is no longer editable here, or has been deleted, please return to dashboard</h2>")
def form_valid(self, form):
timezone.activate(pytz.timezone(self.request.user.t_zone))
proceed = True
try:
instance = Build.objects.get(id = (self.kwargs['pk']))
except:
return HttpResponse("<h2 style = 'margin:2em;'>This build is no longer available it has been deleted, please please return to dashboard</h2>")
if instance.buildActive == False:
proceed = False
if instance.deleted == True:
proceed = False
#all appears to be well, process request
if proceed == True:
form.instance.editedBy = (self.request.user.first_name)+ " " +(self.request.user.last_name)
form.instance.editedDate = timezone.now()
print('edited date ' + str(form.instance.editedDate))
form.instance.reviewed = True
next = self.request.POST['next'] #grabs prev url from form template
form.save()
build = Build.objects.get(id = self.kwargs['pk'])
if build.buildLength >0:
anticipated_end = build.buildStart + (timedelta(hours = float(build.buildLength)))
print(anticipated_end)
else:
anticipated_end = None
build.anticipatedEnd = anticipated_end
build.save()
build_thres_updater(self.kwargs['pk'])#this is function above, it updates threshold alarm counts on the build
return HttpResponseRedirect(next) #returns to this page after valid form submission
else:
return HttpResponse("<h2 style = 'margin:2em;'>This build is no longer available it has been deleted, please please return to dashboard</h2>")
When I open this form, the date and time of buildStart are displayed in my Winnipeg timezone, so Django converted from UTC to my timezone, perfect, but when I submit this form, the date in the DB has been altered from UTC to Winnipeg Time. Why is this?
I have tried to convert the submitted time to UTC in the form_valid function, but this does not seem like the right approach. What am I missing here?
I simply want to store all times as UTC, but display them in the user's timezone in forms/pages.
EDIT
When I remove timezone.activate(pytz.timezone(self.request.user.t_zone)) from both get and form_valid, UTC is preserved in the DB which is great. But the time displayed on the form is now in the default TIME_ZONE in settings.py. I just need this to be in the user's timezone....
EDIT 2
I also tried to add:
{% load tz %}
{% timezone "America/Winnipeg" %}
{{form}}
{% endtimezone %}
Which displayed the time on the form correctly, but then when the form submits, it will again remove 1 hour from the UTC time in the DB.
If I change template to:
{% load tz %}
{% timezone "Europe/Paris" %}
{{form}}
{% endtimezone %}
The time will be displayed in local Paris time. When I submit the form, it will write this Paris time to the DB in UTC+2. So, in summary:
Time record was created was 11:40 Winnipeg time, which writes
16:40 UTC to database, perfect
I access the form template, and time is displayed as local Paris time, 6:40pm, which is also what I would expect.
I submit form without changing any fields.
Record has been updated with the time as 22:40, which is UTC + 6 hours.
What is happening here!?

Put simply: your activate() call in form_valid() comes too late to affect the form field, so the incoming datetime gets interpreted in the default timezone—which in your case is America/Toronto—before being converted to UTC and saved to the database. Hence the apparent time shift.
The documentation doesn't really specify when you need to call activate(). Presumably, though, it has to come before Django converts the string value in the request to the aware Python datetime in the form dictionary (or vice versa when sending a datetime). By the time form_valid() is called, the dictionary of field values is already populated with the Python datetime object.
The most common place to put activate() is in middleware (as in this example from the documentation), since that ensures that it comes before any view processing. Alternatively, if using generic class-based views like you are, you could put it in dispatch().

Related

Form fields are empty when created using instance = instance

I have two models prodcut_prices and WrongPrice.
In WrongPrice the user can correct report wrong prices - when the price is reported it should also be updated in product_price.
My problem is, even though I instantiate product_price at the very beginning as instance_productprice, all of its required fields returns the "this field has to be filled out" error.
How come those field are not set when im using the instance instance_productprice = product_prices.objects.filter(id=pk)[0] ? Note, that all fields in product_prices are always non-empty since they are being pulled from the product_price model, which is handled in another view, thus that is not the issue.
def wrong_price(request,pk):
#Get the current price object
instance_productprice = product_prices.objects.filter(id=pk)[0]
#Get different values
wrong_link = instance_productprice.link
img_url = instance_productprice.image_url
wrong_price = instance_productprice.last_price
domain = instance_productprice.domain
# Create instances
instance_wrongprice = WrongPrice(
link=wrong_link,
correct_price=wrong_price,
domain = domain)
if request.method == "POST":
form_wrong_price = wrong_price_form(request.POST,instance=instance_wrongprice)
# Update values in product_prices
form_product_price = product_prices_form(request.POST,instance=instance_productprice)
form_product_price.instance.start_price = form_wrong_price.instance.correct_price
form_product_price.instance.last_price = form_wrong_price.instance.correct_price
if form_wrong_price.is_valid() & form_product_price.is_valid():
form_wrong_price.save()
form_product_price.save()
messages.success(request, "Thanks")
return redirect("my_page")
else:
messages.error(request, form_product_price.errors) # Throws empty-field errors,
messages.error(request, form_wrong_price.errors)
return redirect("wrong-price", pk=pk)
else:
form_wrong_price = wrong_price_form(instance=instance_wrongprice)
return render(request, "my_app/wrong_price.html",context={"form":form_wrong_price,"info":{"image_url":img_url}})
I am bit confused about how you implemented it. You have passed a instance of WrongPrice price through the form, which is unnecessary, you could have used initial:
wrong_values = dict(
link=wrong_link,
correct_price=wrong_price,
domain = domain
)
form_wrong_price = wrong_price_form(initial= wrong_values)
Then you are adding values to product_prices_form from instance of form_wrong_price. I don't see why you need a form again here. You can simple use:
form_wrong_price = wrong_price_form(request.POST, initial= wrong_values)
if form_wrong_price.is_valid():
instance = form_wrong_price.save()
instance_productprice.start_price = instance.correct_price
instance_productprice.last_price = instance.correct_price
instance_productprice.save()
Finally, please use PascalCase when defining class names. And you can get the product prices by product_prices.objects.get(id=pk)(instead of filter()[0]).

Flask Admin - is there a way to store current url (with custom filters applied) of table view?

I am working on ticketing system in Flask Admin. The Flask Admin enviroment will be the main one for all the users. For creating or editing tickets I go out from Flask-Admin and use wtforms to implement backend logic. After creation or editing the ticket (validate_on_submit) I want to redirect back to Flask Admin, so I use redirect(url_for(ticket.index_view)). It works fine.
Is there a way to redirect to flask admin, but also with specific filters which were applied before user left Flask admin enviroment? (it is basiccaly GET parameters of url - but in FLASK)
I was trying to use:
referrer = request.referrer
get_url()
But I am probably missing something crucial and don´t know how to implement it (where to put it so I can call the arguments)
Thank you so much.
EDIT : adding more context:
I have a flask admin customized to different roles of users. The main ModelView is the one showing the TICKETS : the specifics of the Class are not vital to my current problem but here its how it looks:
class TicketModelView(ModelView):
column_list = ['id', 'title', 'osoba', 'content', 'povod_vmc_kom', 'dateVMC','zodpovedni', 'deadline', 'odpoved', 'solution', 'is_finished']
column_searchable_list = ['osoba']
column_filters = [ 'povod_vmc_kom', 'dateVMC', 'osoba', 'zodpovedni']
column_labels = dict(povod_vmc_kom='VMČ / Komisia', dateVMC='Dátum VMČ / komisie', zodpovedni = "Zodpovední")
column_display_actions = True
column_filters = [
FilterEqual(column=Ticket.povod_vmc_kom, name='Výbor/komisia', options=(('VMČ Juh','VMČ Juh'), ('UM','UM'), ('Kom dopravy','Kom dopravy'))),
'zodpovedni', 'is_finished',
'dateVMC', 'osoba'
]
def is_accessible(self):
#práva pre vedenie mesta - môže len nazerať
if current_user.is_authenticated and current_user.role == 0:
self.can_export=True
self.can_delete = False
self.can_edit = False
self.can_create = False
self._refresh_form_rules_cache()
self._refresh_forms_cache()
return True
#práva pre super admina (ostatné práva sú defaultne zapnuté)
if current_user.is_authenticated and current_user.role == 1:
self.can_export=True
self.can_delete=True
self.form_edit_rules = ('zodpovedni', 'is_finished' )
self.column_editable_list = ['is_finished']
self._refresh_form_rules_cache()
self._refresh_forms_cache()
return True
#práva pre garantov
if current_user.is_authenticated and current_user.role == 2:
self.can_delete = False
self.can_create = False
self.can_edit = False
self.can_export=True
self.column_searchable_list = ['title']
self._refresh_form_rules_cache()
self._refresh_forms_cache()
return True
#práva pre veducich utvarov
if current_user.is_authenticated and current_user.role == 3:
self.can_create = False
self.can_delete = False
self.can_export=True
self.column_searchable_list = ['title']
self.column_editable_list = ['odpoved', 'date_odpoved', 'solution', 'date_solution' ]
self.form_edit_rules = ('odpoved', 'date_odpoved', 'solution', 'date_solution')
self._refresh_form_rules_cache()
self._refresh_forms_cache()
return True
return False
def _solution_formatter(view, context, model, name):
# Format your string here e.g show first 20 characters
# can return any valid HTML e.g. a link to another view to show the detail or a popup window
if model.solution:
return model.solution[:50]
pass
def _content_formatter(view, context, model, name):
# Format your string here e.g show first 20 characters
# can return any valid HTML e.g. a link to another view to show the detail or a popup window
if len(model.content) > 100:
markupstring = "<a href= '%s'>%s</a>" % (url_for('ticket', ticket_id=model.id), "...")
return model.content[:100] + Markup(markupstring)
return model.content
def _user_formatter(view, context, model, name):
if model.id:
markupstring = "<a href= '%s'>%s</a>" % (url_for('ticket', ticket_id=model.id), model.id)
return Markup(markupstring)
else:
return ""
column_formatters = {
'content': _content_formatter,
'solution': _solution_formatter,
'id': _user_formatter
}
When user viewing the TicketView in Flask Admin, he can apply various filters which is vital to the user experience of the whole web app. The filters work fine and they are stored in URL as GET arguments. When he wants to create or edit a ticket, I am not allowing him to do it in Flask Admin (I edited Flask-Admin layout.html template and added a button to navbar which redirects to my new_ticket url with wtforms.) because of backend logic I want to be applied. For example when he edits field "solution" : I want the value in field "date_of_solution" be generated automatically (date.today()). So I am using wtforms and flask routing : example is bellow:
#app.route("/ticket/<int:ticket_id>/solution", methods = ['GET', 'POST'])
#login_required
def solution(ticket_id):
if current_user.role != 3:
flash("Pre zadanie riešenia alebo odpovede musíte byť prihlásený ako vedúci útvaru", "danger")
return redirect(url_for('ticket', ticket_id=ticket_id))
ticket = Ticket.query.get_or_404(ticket_id)
form = AdminPanelForm()
if form.validate_on_submit():
print("1")
if not ticket.date_solution:
print("2")
ticket.date_solution= datetime.now()
if not ticket.date_odpoved:
print("3")
if form.odpoved.data != ticket.odpoved:
print("4")
ticket.date_odpoved= datetime.now()
ticket.solution = form.solution.data
ticket.odpoved = form.odpoved.data
ticket.is_finished = True
db.session.commit()
flash("Ticket bol updatenutý", "success")
**return redirect(url_for('ticketmod.index_view'))**
elif request.method == 'GET':
form.solution.data = ticket.solution
form.odpoved.data = ticket.odpoved
return render_template("admin_ticket.html", form=form, ticket = ticket)
Now you can see that after succesful updating the ticket, user is redirected to Ticket model View where he came from, return redirect(url_for('ticketmod.index_view')) but without filters applied. I am looking for the solution, how can you store the url GET parameters (the filters) and then use them when redirecting back to ModelView. I tried function get_url() or request.referrer but I wasn´t succesful.
As I said in my original post, maybe I am missing something crucial in web architecture - if you have in mind some learning material I shoul be looking at : thanks for any advice.
Within the formatter method you can get a view's url including the applied filters/sorting criteria using the following:
_view_url = view.get_url('.index_view', **request.args)
Now pass this along to route request, either as a parameter or some other means. For example:
class TicketModelView(ModelView):
# blah blah
def _user_formatter(view, context, model, name):
if model.id:
# This is the current url of the view including filters
_view_url = view.get_url('.index_view', **request.args)
# Pass this as a parameter to your route
markupstring = "<a href= '%s'>%s</a>" % (url_for('ticket', ticket_id=model.id, return_url=_view_url), model.id)
return Markup(markupstring)
At the route you can now pull out the return_url from the request arg and add it as a hidden field in the form. Then in the post back retrieve the value from the form and redirect.
#app.route("/ticket/<int:ticket_id>/solution", methods = ['GET', 'POST'])
#login_required
def solution(ticket_id):
# Get the return_url from the request
_return_url = request.args.get('return_url'):
# Add the return_url to the form as a hidden field
form.return_url.data = _return_url
# blah blah
if form.validate_on_submit():
# get return value from form
_return_url = form.return_url.data
return redirect(_return_url) if _return_url else redirect(url_for('ticketmod.index_view'))

query is getting UTC timestamp from the database but local time is stored in the database

For some reason my queryset is returning a UTC time, however in the database the time it is supposed to be getting is local time. Does anyone know why this is happening? Thanks for your help
The last_checkin_time method is what gets the user's last checkin timestamp from the database, and right now I just have it posting in my toggle method in the time_delta variable so I can see what value its getting. (once i get this timezone thing figured out the time_delta will be an actual time delta)
Heres my model manager
class UserActivityManager(models.Manager):
def current(self, user):
current_obj = self.get_queryset().filter(user=user).order_by('-timestamp').first()
return current_obj
def last_checkin_time(self, user):
last_activity_time = self.get_queryset().order_by('-timestamp').filter(user=user, activity="checkin").first()
return last_activity_time
def toggle(self, user):
last_item = self.current(user)
activity = "checkin"
time_delta = None
last_checkin = self.last_checkin_time(user)
if last_item is not None:
if last_item.timestamp <= tz.localize(datetime.datetime.now()):
pass
if last_item.activity == "checkin":
activity = "checkout"
time_delta = last_checkin.timestamp
obj = self.model(
user=user,
activity=activity,
time_delta = time_delta,
)
obj.save()
return obj
And heres what the table in my database looks like (focusing on the time_delta field in the last few rows)
EDIT:
Also I should mention the timestamp field in my model is set to auto_now_add=True ie.
timestamp = models.DateTimeField(auto_now_add=True)
Not sure if this is causing the problem
Instead of using tz.localize(last_checkin.timestamp) I needed to use tz.normalize(last_checkin.timestamp). It seems since the timestamp was already set to UTC I needed to change it with the normalize method, rather than the localize

Django pagination while objects are being added

I've got a website that shows photos that are always being added and people are seeing duplicates between pages on the home page (last added photos)
I'm not entirely sure how to approach this problem but this is basically whats happening:
Home page displays latest 20 photos [0:20]
User scrolls (meanwhile photos are being added to the db
User loads next page (through ajax)
Page displays photos [20:40]
User sees duplicate photos because the photos added to the top of the list pushed them down into the next page
What is the best way to solve this problem? I think I need to somehow cache the queryset on the users session maybe? I don't know much about caches really so a step-by-step explanation would be invaluable
here is the function that gets a new page of images:
def get_images_paginated(query, origins, page_num):
args = None
queryset = Image.objects.all().exclude(hidden=True).exclude(tags__isnull=True)
per_page = 20
page_num = int(page_num)
if origins:
origins = [Q(origin=origin) for origin in origins]
args = reduce(operator.or_, origins)
queryset = queryset.filter(args)
if query:
images = watson.filter(queryset, query)
else:
images = watson.filter(queryset, query).order_by('-id')
amount = images.count()
images = images.prefetch_related('tags')[(per_page*page_num)-per_page:per_page*page_num]
return images, amount
the view that uses the function:
def get_images_ajax(request):
if not request.is_ajax():
return render(request, 'home.html')
query = request.POST.get('query')
origins = request.POST.getlist('origin')
page_num = request.POST.get('page')
images, amount = get_images_paginated(query, origins, page_num)
pages = int(math.ceil(amount / 20))
if int(page_num) >= pages:
last_page = True;
else:
last_page = False;
context = {
'images':images,
'last_page':last_page,
}
return render(request, '_images.html', context)
One approach you could take is to send the oldest ID that the client currently has (i.e., the ID of the last item in the list currently) in the AJAX request, and then make sure you only query older IDs.
So get_images_paginated is modified as follows:
def get_images_paginated(query, origins, page_num, last_id=None):
args = None
queryset = Image.objects.all().exclude(hidden=True).exclude(tags__isnull=True)
if last_id is not None:
queryset = queryset.filter(id__lt=last_id)
...
You would need to send the last ID in your AJAX request, and pass this from your view function to get_images_paginated:
def get_images_ajax(request):
if not request.is_ajax():
return render(request, 'home.html')
query = request.POST.get('query')
origins = request.POST.getlist('origin')
page_num = request.POST.get('page')
# Get last ID. Note you probably need to do some type casting here.
last_id = request.POST.get('last_id', None)
images, amount = get_images_paginated(query, origins, page_num, last_id)
...
As #doniyor says you should use Django's built in pagination in conjunction with this logic.

Django formset unit test

I can't run a unit test with formset.
I try to do a test:
class NewClientTestCase(TestCase):
def setUp(self):
self.c = Client()
def test_0_create_individual_with_same_adress(self):
post_data = {
'ctype': User.CONTACT_INDIVIDUAL,
'username': 'dupond.f',
'email': 'new#gmail.com',
'password': 'pwd',
'password2': 'pwd',
'civility': User.CIVILITY_MISTER,
'first_name': 'François',
'last_name': 'DUPOND',
'phone': '+33 1 34 12 52 30',
'gsm': '+33 6 34 12 52 30',
'fax': '+33 1 34 12 52 30',
'form-0-address1': '33 avenue Gambetta',
'form-0-address2': 'apt 50',
'form-0-zip_code': '75020',
'form-0-city': 'Paris',
'form-0-country': 'FRA',
'same_for_billing': True,
}
response = self.c.post(reverse('client:full_account'), post_data, follow=True)
self.assertRedirects(response, '%s?created=1' % reverse('client:dashboard'))
and I have this error:
ValidationError: [u'ManagementForm data is missing or has been
tampered with']
My view :
def full_account(request, url_redirect=''):
from forms import NewUserFullForm, AddressForm, BaseArticleFormSet
fields_required = []
fields_notrequired = []
AddressFormSet = formset_factory(AddressForm, extra=2, formset=BaseArticleFormSet)
if request.method == 'POST':
form = NewUserFullForm(request.POST)
objforms = AddressFormSet(request.POST)
if objforms.is_valid() and form.is_valid():
user = form.save()
address = objforms.forms[0].save()
if url_redirect=='':
url_redirect = '%s?created=1' % reverse('client:dashboard')
logon(request, form.instance)
return HttpResponseRedirect(url_redirect)
else:
form = NewUserFullForm()
objforms = AddressFormSet()
return direct_to_template(request, 'clients/full_account.html', {
'form':form,
'formset': objforms,
'tld_fr':False,
})
and my form file :
class BaseArticleFormSet(BaseFormSet):
def clean(self):
msg_err = _('Ce champ est obligatoire.')
non_errors = True
if 'same_for_billing' in self.data and self.data['same_for_billing'] == 'on':
same_for_billing = True
else:
same_for_billing = False
for i in [0, 1]:
form = self.forms[i]
for field in form.fields:
name_field = 'form-%d-%s' % (i, field )
value_field = self.data[name_field].strip()
if i == 0 and self.forms[0].fields[field].required and value_field =='':
form.errors[field] = msg_err
non_errors = False
elif i == 1 and not same_for_billing and self.forms[1].fields[field].required and value_field =='':
form.errors[field] = msg_err
non_errors = False
return non_errors
class AddressForm(forms.ModelForm):
class Meta:
model = Address
address1 = forms.CharField()
address2 = forms.CharField(required=False)
zip_code = forms.CharField()
city = forms.CharField()
country = forms.ChoiceField(choices=CountryField.COUNTRIES, initial='FRA')
In particular, I've found that the ManagmentForm validator is looking for the following items to be POSTed:
form_data = {
'form-TOTAL_FORMS': 1,
'form-INITIAL_FORMS': 0
}
Every Django formset comes with a management form that needs to be included in the post. The official docs explain it pretty well. To use it within your unit test, you either need to write it out yourself. (The link I provided shows an example), or call formset.management_form which outputs the data.
It is in fact easy to reproduce whatever is in the formset by inspecting the context of the response.
Consider the code below (with self.client being a regular test client):
url = "some_url"
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
# data will receive all the forms field names
# key will be the field name (as "formx-fieldname"), value will be the string representation.
data = {}
# global information, some additional fields may go there
data['csrf_token'] = response.context['csrf_token']
# management form information, needed because of the formset
management_form = response.context['form'].management_form
for i in 'TOTAL_FORMS', 'INITIAL_FORMS', 'MIN_NUM_FORMS', 'MAX_NUM_FORMS':
data['%s-%s' % (management_form.prefix, i)] = management_form[i].value()
for i in range(response.context['form'].total_form_count()):
# get form index 'i'
current_form = response.context['form'].forms[i]
# retrieve all the fields
for field_name in current_form.fields:
value = current_form[field_name].value()
data['%s-%s' % (current_form.prefix, field_name)] = value if value is not None else ''
# flush out to stdout
print '#' * 30
for i in sorted(data.keys()):
print i, '\t:', data[i]
# post the request without any change
response = self.client.post(url, data)
Important note
If you modify data prior to calling the self.client.post, you are likely mutating the DB. As a consequence, subsequent call to self.client.get might not yield to the same data, in particular for the management form and the order of the forms in the formset (because they can be ordered differently, depending on the underlying queryset). This means that
if you modify data[form-3-somefield] and call self.client.get, this same field might appear in say data[form-8-somefield],
if you modify data prior to a self.client.post, you cannot call self.client.post again with the same data: you have to call a self.client.get and reconstruct data again.
Django formset unit test
You can add following test helper methods to your test class [Python 3 code]
def build_formset_form_data(self, form_number, **data):
form = {}
for key, value in data.items():
form_key = f"form-{form_number}-{key}"
form[form_key] = value
return form
def build_formset_data(self, forms, **common_data):
formset_dict = {
"form-TOTAL_FORMS": f"{len(forms)}",
"form-MAX_NUM_FORMS": "1000",
"form-INITIAL_FORMS": "1"
}
formset_dict.update(common_data)
for i, form_data in enumerate(forms):
form_dict = self.build_formset_form_data(form_number=i, **form_data)
formset_dict.update(form_dict)
return formset_dict
And use them in test
def test_django_formset_post(self):
forms = [{"key1": "value1", "key2": "value2"}, {"key100": "value100"}]
payload = self.build_formset_data(forms=forms, global_param=100)
print(payload)
# self.client.post(url=url, data=payload)
You will get correct payload which makes Django ManagementForm happy
{
"form-INITIAL_FORMS": "1",
"form-TOTAL_FORMS": "2",
"form-MAX_NUM_FORMS": "1000",
"global_param": 100,
"form-0-key1": "value1",
"form-0-key2": "value2",
"form-1-key100": "value100",
}
Profit
There are several very useful answers here, e.g. pymen's and Raffi's, that show how to construct properly formatted payload for a formset post using the test client.
However, all of them still require at least some hand-coding of prefixes, dealing with existing objects, etc., which is not ideal.
As an alternative, we could create the payload for a post() using the response obtained from a get() request:
def create_formset_post_data(response, new_form_data=None):
if new_form_data is None:
new_form_data = []
csrf_token = response.context['csrf_token']
formset = response.context['formset']
prefix_template = formset.empty_form.prefix # default is 'form-__prefix__'
# extract initial formset data
management_form_data = formset.management_form.initial
form_data_list = formset.initial # this is a list of dict objects
# add new form data and update management form data
form_data_list.extend(new_form_data)
management_form_data['TOTAL_FORMS'] = len(form_data_list)
# initialize the post data dict...
post_data = dict(csrf_token=csrf_token)
# add properly prefixed management form fields
for key, value in management_form_data.items():
prefix = prefix_template.replace('__prefix__', '')
post_data[prefix + key] = value
# add properly prefixed data form fields
for index, form_data in enumerate(form_data_list):
for key, value in form_data.items():
prefix = prefix_template.replace('__prefix__', f'{index}-')
post_data[prefix + key] = value
return post_data
The output (post_data) will also include form fields for any existing objects.
Here's how you might use this in a Django TestCase:
def test_post_formset_data(self):
url_path = '/my/post/url/'
user = User.objects.create()
self.client.force_login(user)
# first GET the form content
response = self.client.get(url_path)
self.assertEqual(HTTPStatus.OK, response.status_code)
# specify form data for test
test_data = [
dict(first_name='someone', email='someone#email.com', ...),
...
]
# convert test_data to properly formatted dict
post_data = create_formset_post_data(response, new_form_data=test_data)
# now POST the data
response = self.client.post(url_path, data=post_data, follow=True)
# some assertions here
...
Some notes:
Instead of using the 'TOTAL_FORMS' string literal, we could import TOTAL_FORM_COUNT from django.forms.formsets, but that does not seem to be public (at least in Django 2.2).
Also note that the formset adds a 'DELETE' field to each form if can_delete is True. To test deletion of existing items, you can do something like this in your test:
...
post_data = create_formset_post_data(response)
post_data['form-0-DELETE'] = True
# then POST, etc.
...
From the source, we can see that there is no need include MIN_NUM_FORM_COUNT and MAX_NUM_FORM_COUNT in our test data:
MIN_NUM_FORM_COUNT and MAX_NUM_FORM_COUNT are output with the rest of the management form, but only for the convenience of client-side code. The POST value of them returned from the client is not checked.
This doesn't seem to be a formset at all. Formsets will always have some sort of prefix on every POSTed value, as well as the ManagementForm that Bartek mentions. It might have helped if you posted the code of the view you're trying to test, and the form/formset it uses.
My case may be an outlier, but some instances were actually missing a field set in the stock "contrib" admin form/template leading to the error
"ManagementForm data is missing or has been tampered with"
when saved.
The issue was with the unicode method (SomeModel: [Bad Unicode data]) which I found investigating the inlines that were missing.
The lesson learned is to not use the MS Character Map, I guess. My issue was with vulgar fractions (¼, ½, ¾), but I'd assume it could occur many different ways. For special characters, copying/pasting from the w3 utf-8 page fixed it.
postscript-utf-8