Related
I am using Stripe in my Django application. I have the following test case: incorrect_cvc leads to an card_error. Now when correcting the CVC and using 4242 4242 4242 4242 what I except is a successful charge. However what I get pack is the following error message:
Request req_auTSTGSGoUVNUa: Keys for idempotent requests can only be
used with the same parameters they were first used with. Try using a
key other than 'k1qjchgqjw' if you meant to execute a different
request.
I am not aware of which parameters I changed. But I think it's not the idea that the checkout process basically doesn't work anymore after an card_error. Does anyone understand which parameters I "changed" that leads to this error message?
def checkout_page(request):
"""
* Check if session and ReservedItem exist.
* Generate order_item dict for every ReservedItem entry, that belongs
to order_reference.
If request.method is 'POST':
* Check if ticket reservation is still valid.
* Create entries in models OrderItem, Order & ReservedItem.
"""
session_order_reference = request.session.get('order_reference')
if request.session.get('order_reference'):
reserved_items = ReservedItem.objects.filter(
order_reference=session_order_reference
)
if not reserved_items:
return redirect('website:index')
else:
return redirect('website:index')
taxes_dict = {}
total_gross = total_tax_amount = 0
order_items_list = []
for item in reserved_items:
event = item.ticket.event
timestamp_of_reservation = item.created
total_gross += item.subtotal
order_item = {
'ticket': item.ticket,
'ticket_name': item.ticket.name,
'quantity': item.quantity,
'subtotal': item.subtotal,
'type': OrderType.ORDER,
}
total_tax_amount += add_tax(
item=item,
taxes_dict=taxes_dict,
order_item=order_item,
)
order_items_list.append(dict(order_item))
total_net = total_gross - total_tax_amount # TODO Marc: Calculate in add_vat func?
if request.method == 'POST':
# TODO Marc: Should live in forms.py or just models?
reservation_expired_redirect = check_if_reservation_expired(
request=request,
timestamp_of_reservation=timestamp_of_reservation,
organizer=event.organizer.slug,
event=event.slug,
)
if reservation_expired_redirect:
return reservation_expired_redirect
# TODO Marc: Should live in forms.py or just models?
ticket_is_on_sale = check_if_ticket_is_on_sale(
order_items_list=order_items_list,
request=request,
organizer=event.organizer.slug,
event=event.slug,
)
if ticket_is_on_sale:
return ticket_is_on_sale
billing = BillingForm(request.POST, prefix='billing')
order = OrderForm(request.POST, prefix='order')
if order.is_valid() and billing.is_valid():
# Charge via Stripe
stripe.api_key = "ABC" # TODO Marc: Change to env
token = request.POST.get('stripeToken')
# https://stripe.com/docs/api#error_handling
paid = False
try:
# Compare with transactions > models copy.py > class ChargeManager(models.Manager):
# Use Stripe's library to make requests...
total_gross_amount_in_smallest_unit = smallest_currency_unit(total_gross, 'eur') #TODO Marc: Replace eur
charge = stripe.Charge.create(
amount=total_gross_amount_in_smallest_unit, # TODO Marc > https://stripe.com/docs/currencies#zero-decimal
application_fee=100, # TODO Marc: Which currency?
currency='eur', # TODO Marc
source=token,
stripe_account="ABC", # TODO Marc: Replace with organizer stripe account
idempotency_key=session_order_reference,
)
new_charge_obj = Charge.objects.create(
amount=charge.amount,
charge_id=charge.id,
livemode=charge.livemode,
paid=charge.paid,
refunded=charge.refunded,
currency=charge.currency,
failure_code=charge.failure_code,
failure_message=charge.failure_message,
fraud_details=charge.fraud_details,
outcome=charge.outcome,
status=charge.status,
application_fee=charge.application_fee,
captured=charge.captured,
created=charge.created,
# TODO Marc: Add refunds:
# amount_refunded=charge.amount_refunded,
# etc.
)
application_fee = stripe.ApplicationFee.retrieve(charge.application_fee)
Fee.objects.create(
fee_id=application_fee.id,
livemode=application_fee.livemode,
currency=application_fee.currency,
amount=application_fee.amount,
charge=new_charge_obj,
# TODO Marc: Add refunds
)
paid = new_charge_obj.paid
except stripe.error.CardError as e:
# Since it's a decline, stripe.error.CardError will be caught
body = e.json_body
err = body.get('error', {})
messages.add_message(
request,
messages.ERROR,
err.get('message')
)
# return redirect(
# 'orders:order-list',
# order_reference=new_order.order_reference,
# access_key=new_order.access_key,
# )
# print("Type is: %s") % err.get('type')
# print("Code is: %s") % err.get('code')
# # param is '' in this case
# print("Param is: %s") % err.get('param')
# print("Message is: %s") % err.get('message')
except stripe.error.RateLimitError as e:
# Too many requests made to the API too quickly
pass
except stripe.error.InvalidRequestError as e:
# Invalid parameters were supplied to Stripe's API
pass
except stripe.error.AuthenticationError as e:
# Authentication with Stripe's API failed
# (maybe you changed API keys recently)
pass
except stripe.error.APIConnectionError as e:
# Network communication with Stripe failed
pass
except stripe.error.StripeError as e:
# Display a very generic error to the user, and maybe send
# yourself an email
pass
except Exception as e:
# Something else happened, completely unrelated to Stripe
pass
if paid:
# Create new attendee
i = 1
attendee_list = []
for item in reserved_items:
for _ in range(item.quantity): # noqa
new_attendee_dict = {
'event': item.ticket.event,
'ticket': item.ticket,
'ticket_name': item.ticket.name,
'ticket_reference': session_order_reference + "-" + str(i),
'ticket_code': get_random_string(length=10),
}
i += 1
attendee_list.append(dict(new_attendee_dict))
# Create new order
new_order_dict = {
'total_gross': total_gross,
'total_tax': total_tax_amount,
'total_net': total_net,
'total_gross_converted': total_gross, # TODO Marc
'event': event,
'order_reference': session_order_reference,
'status': OrderStatus.PENDING,
'access_key': get_random_string(length=10),
}
new_order = order.save(commit=False)
[setattr(new_order, k, v) for k, v in new_order_dict.items()]
new_order.save()
# Create order items
for item in order_items_list:
OrderItem.objects.create(order=new_order, **item)
# Create attendees
for item in attendee_list:
Attendee.objects.create(order=new_order, **item)
# Create billing profile
billing_profile = billing.save(commit=False)
billing_profile.order = new_order
billing_profile.save()
# Delete order_reference session
del request.session['order_reference']
return redirect(
'orders:order-list',
order_reference=new_order.order_reference,
access_key=new_order.access_key,
)
else:
billing = BillingForm(prefix='billing')
order = OrderForm(prefix='order')
context = {
'reserved_items': reserved_items,
'taxes': taxes_dict,
'total_net': total_net,
'total_gross': total_gross,
'currency': event.currency,
'order': order,
'billing': billing,
}
return render(request, 'checkout/checkout.html', context)
The problem is not with anything that you've changed, but rather what you haven't changed :)
On this line you are passing an idempotency_key:
charge = stripe.Charge.create(
...
idempotency_key=session_order_reference,
)
As described in the Stripe docs, you can pass an idempotency key with a request, which allows you to make the same request again in the future, using the same key, and you will get the same result as the first request. This is useful in case you didn't recieve the first response because of a network issue.
In this case, you have changed the CVC, which creates a new token variable. This means that your request is not identical to the previous request that used the same idempotency key. That doesn't make sense as you can only use the same idempotency key with identical requests, so you get this error from Stripe.
To resolve this, you should retry the charge creation using a freshly generated idempotency key. Generally, the key should be generated on each unique request that your application creates.
Had a similar issue where I was passing an indempotency_key and clients were not able to pay after their card was declined because the data that was sent was unique to the charge but not to the card. If for example their CVC was incorrect the subsequent charge will get created with the exact same idempotency key because data pertinent to the actually card was not taken into account.
The fix for this is to make sure your key is unique to the charge AND the card in this case including the card token can fix this.
Some other things to think about are things like partial payments, refunds, same/different ip, other metadata.
Stripe handles this case
when you are submitting wrong cvv
I tested with stripe test credit cards
https://stripe.com/docs/testing#cards
use that one which fails with cvv code than use valid card.
I'm trying to write a custom rest_framework Permission to prevent users from querying information that's not of the same company as them. Unfortunately, I can't seem to access any of the URL's parameters from within has_permission() or has_object_permissions().
Here's the beginning of my router:
# Create a basic router
router = routers.SimpleRouter()
# Establish some variables to assist with nested routes
root_elem = 'companies'
root_elem_id = '/(?P<company_id>[0-9]+)'
loca_elem = '/locations'
loca_elem_id = '/(?P<location_id>[0-9]+)'
# Companies will be the root from which all other relations branch
router.register(r'' + root_elem, views.CompanyViewSet)
router.register(r'' + root_elem + root_elem_id + loca_elem,
views.LocationViewSet)
Here's my custom permission:
# Only permit actions originating from location managers or company admins
class IsLocationManagerOrHigher(BasePermission):
# Checked when displaying lists of records
def has_permission(self, request, *args, **kwargs):
is_correct_level = False
# Admins can see every location if their location_id
# matches a location that's a child of the company
# specified in the URL
if request.employee.is_admin:
is_correct_level = True
return request.user and is_correct_level
# Checked when viewing specific records
def has_object_permission(self, request, view, obj):
is_correct_level = False
# Admins can see location details if their location's company_id
# matches a Location's company_id
if request.employee.is_admin:
is_correct_level = True
# Managers can see location details if it's their location
elif obj.id == request.employee.location_id and request.employee.is_manager:
is_correct_level = True
return request.user and is_correct_level
Right now checking request.employee.is_admin is only half of what I need - I also need to access the company_id from the URL and make sure it matches the admin's location's company_id:
# Pseudocode
try:
user_location = Location.objects.get(id=request.employee.location_id)
return user_location.company_id == kwargs['company_id']
except ObjectDoesNotExist:
pass
I've yet to figure out how to pass these parameters into the Permission so that it can perform this extra step. Or perhaps there's a better way of accomplishing what I'm trying to do?
If you can't pass them in directly (which would be preferable), they are available on the request object:
company_id = request.resolver_match.kwargs.get('company_id')
request.resolver_match.args and request.resolver_match.kwargs contain the positional/keyword arguments captured in your url.
As an alternative to the correct response posted by knbk, you can also get the URL parameters using the view object passed to has_permission method. Like this:
company_id = view.kwargs.get('company_id')
I have set the SOCIAL_AUTH_LINKEDIN_FIELD_OAUTH2_SELECTORS field in my Django Settings per the instructions for LinkedIn configurations here: http://psa.matiasaguirre.net/docs/backends/linkedin.html
But when I run the authentication the additional email selector added to that setting is not added to the list of selectors in the backend call.
When I remove the field SOCIAL_AUTH_LINKEDIN_FIELD_OAUTH2_SELECTORS I get an error that it is missing:
'Settings' object has no attribute
'SOCIAL_AUTH_LINKEDIN_FIELD_OAUTH2_SELECTORS'
So I know I am using the correct settings name.
None of the added params make it to the backend though:
settings.SOCIAL_AUTH_LINKEDIN_FIELD_OAUTH2_SELECTORS = ['id',
'recommendations-received', 'positions', 'email-address', 'headline',
'industry', 'first-name', 'last-name', 'location', 'num-connections',
'skills']
I printed out the result of the backend and always just get the default selector list:
[edited backends/linkedin.py from
https://github.com/omab/python-social-auth/blob/master/social/backends/linkedin.py#L32]
def user_details_url(self):
# use set() since LinkedIn fails when values are duplicated
fields_selectors = list(set(['first-name', 'id', 'last-name'] +
self.setting('FIELD_SELECTORS', [])))
print fields_selectors
# user sort to ease the tests URL mocking
fields_selectors.sort()
fields_selectors = ','.join(fields_selectors)
return self.USER_DETAILS.format(fields_selectors)
#> ['first-name', 'id', 'last-name']
How can I add selectors through DJANGO Settings to expand the data returned when authenticating?
Aamir suggestion worked!:
SOCIAL_AUTH_LINKEDIN_OAUTH2_FIELD_SELECTORS
I ended up adding a print statement to social.strategies.django_strategies and got alisting of all the settings being pulled:
def get_setting(self, name):
print name
return getattr(settings, name)
Listing...
# SOCIAL_AUTH_REDIRECT_IS_HTTPS
# REDIRECT_IS_HTTPS
# SOCIAL_AUTH_LINKEDIN_OAUTH2_KEY
# SOCIAL_AUTH_LINKEDIN_OAUTH2_SECRET
# SOCIAL_AUTH_LINKEDIN_OAUTH2_REQUESTS_TIMEOUT
# SOCIAL_AUTH_REQUESTS_TIMEOUT
# REQUESTS_TIMEOUT
# SOCIAL_AUTH_LINKEDIN_OAUTH2_URLOPEN_TIMEOUT
# SOCIAL_AUTH_URLOPEN_TIMEOUT
# URLOPEN_TIMEOUT
# SOCIAL_AUTH_LINKEDIN_OAUTH2_FIELD_SELECTORS
....
When i try to access to 'tareas' the app returns to me as follows:
'Tareas' object has no attribute '__fields__'
Request Method: GET
Request URL: http://localhost:8000/tareas/
Django Version: 1.4
Exception Type: AttributeError
Exception Value:
'Tareas' object has no attribute '__fields__'
Exception Location: /Users/Tone/Documents/Proyectos/macrotelecom/common/generic.py in view_list, line 240
The problem is in this line:
if not fields:
fields=queryset.__dict__['model']().__fields__(profile)
Which I think is weird because in admin.py i set the attribute fields for my object 'Tareas':
from tareas.models import Tareas
from django.contrib import admin
class TareasAdmin(admin.ModelAdmin):
fields = ['idtarea','idtipotarea','idagente','fechatarea']
list_display = ('idtarea','idagente','fechatarea')
admin.site.register(Tareas,TareasAdmin)
The line is defined in this function 'def_view':
def view_list(request, get_template, **kwargs):
'''
Generic list view with validation included and object transfering support
'''
# Config
default_rows_per_page=100
# Get arguments from the function
object_id=keyarg('object_id',kwargs,None)
action=keyarg('action',kwargs,None)
extra_context=keyarg('extra_context',kwargs,{})
queryset=keyarg('queryset',kwargs,None)
restrictions=keyarg('restrictions',kwargs,None)
permission=keyarg('permission',kwargs,None)
fields=keyarg('fields',kwargs,None)
default_ordering=keyarg('default_ordering',kwargs,None)
compact_rows=keyarg('compact_rows',kwargs,None)
# Get template and profile
namesp=str(queryset.__dict__['model']).replace("class ","").replace(">","").replace("<","").replace("'","").split(".")
appname=namesp[-3].lower()
modelname=namesp[-1].lower()
(profile,template_name)=get_template(keyarg('template_name',kwargs,"%s/%s_list.html" % (appname,modelname)))
# Check permissions
if (permission is not None) and (not request.user.has_perm(permission)):
return HttpResponseRedirect('/not_authorized/')
# Get extra arguments
extra={}
for arg in kwargs:
if arg not in ['object_id','action','template_name','extra_context','queryset','restrictions']:
extra[arg]=kwargs[arg]
# Inicialization
new_extra_context={}
new_extra_context['now']=epochdate(time.time())
# new_extra_context['msglog']=msglog()
# Restrictions fields
new_extra_context['filters']=[]
if restrictions:
for restriction in restrictions:
f={}
f['name']=restriction
f['value']=extra[restriction]
new_extra_context['filters'].append(f)
# Process the filter
new_extra_context['filters_obj']={}
new_extra_context['header_loop']=1
if restrictions:
queryset_obj={}
for rname in restrictions:
# Get name of the field and object
(rfield,robject)=restrictions[rname]
# Get the ID
rid=extra[rname]
# Save the id in extra_context
new_extra_context[rname]=rid
# Save the object in queryset_obj
queryset_obj[rname]=robject(id=rid)
# Filter the queryset
queryset=queryset.filter(eval("Q(%s=queryset_obj['%s'])" % (rfield,rname)))
new_extra_context['filters_obj'][rname]=get_object_or_404(robject,pk=rid)
# Get field list
if not fields:
fields=queryset.__dict__['model']().__fields__(profile)
# Save action if we got one
if action:
new_extra_context['action']=action
# Try to convert object_id to a numeric id
try:
object_id=int(object_id)
except:
pass
# Save GET values
new_extra_context['get']=[]
new_extra_context['getval']={}
for name in request.GET:
if name not in ['filtername','filtervalue']:
struct={}
struct['name']=name
if name=='rowsperpage':
struct['value']=default_rows_per_page
elif name=='page':
struct['value']=1
else:
struct['value']=request.GET[name]
new_extra_context['get'].append(struct)
new_extra_context['getval'][name]=struct['value']
# Filter on limits
limits=queryset.__dict__['model']().__limitQ__(profile,request)
qobjects=None
for name in limits:
if qobjects:
qobjects&=limits[name]
else:
qobjects=limits[name]
if qobjects:
queryset=queryset.filter(qobjects)
# Filters on fields
try:
filters_by_json=request.GET.get('filters','{}')
filters_by_struct=json_decode(str(filters_by_json))
except Exception:
filters_by_struct=[]
filtername=request.GET.get('filtername',None)
filtervalue=request.GET.get('filtervalue',None)
listfilters=queryset.__dict__['model']().__searchF__(profile)
# Process the search
filters_struct={}
for key in filters_by_struct:
# Get the value of the original filter
value=filters_by_struct[key]
# If there is something to filter, filter is not being changed and filter is known by the class
if (key!=filtername) and (key in listfilters) and (value>0):
# Add the filter to the queryset
f=listfilters[key]
fv=f[2][value-1][0]
queryset=queryset.filter(f[1](fv))
# Save it in the struct as a valid filter
filters_struct[key]=value
# Add the requested filter if any
if (filtername in listfilters) and (int(filtervalue)>0):
f=listfilters[filtername]
fv=f[2][int(filtervalue)-1][0]
queryset=queryset.filter(f[1](fv))
filters_struct[filtername]=int(filtervalue)
# Rewrite filters_json updated
filters_json=json_encode(filters_struct)
# Build the clean get for filters
get=new_extra_context['get']
filters_get=[]
for element in get:
if element['name'] not in ['filters']:
struct={}
struct['name']=element['name']
struct['value']=element['value']
filters_get.append(struct)
# Add filter_json
struct={}
struct['name']='filters'
struct['value']=filters_json
filters_get.append(struct)
new_extra_context['filters_get']=filters_get
# Get the list of filters allowed by this class
filters=[]
for key in listfilters:
choice=[_('All')]
for value in listfilters[key][2]:
choice.append(value[1])
# Decide the choosen field
if key in filters_struct.keys():
choose=int(filters_struct[key])
else:
choose=0
filters.append((key,listfilters[key][0],choice,choose))
new_extra_context['filters']=filters
# Search text in all fields
search=request.GET.get('search','')
new_extra_context['search']=search
datetimeQ=None
if len(search)>0:
searchs=queryset.__dict__['model']().__searchQ__(search,profile)
qobjects=None
for name in searchs:
if (searchs[name]=='datetime'):
datetimeQ=name
continue
else:
if qobjects:
qobjects|=searchs[name]
else:
qobjects=searchs[name]
if qobjects:
queryset=queryset.filter(qobjects)
else:
# Look for datetimeQ field
searchs=queryset.__dict__['model']().__searchQ__(search,profile)
for name in searchs:
if (searchs[name]=='datetime'):
datetimeQ=name
continue
# Datetime Q
new_extra_context['datetimeQ']=datetimeQ
if datetimeQ:
# Inicialization
f={}
f['year']=(1900,2100,False)
f['month']=(1,12,False)
f['day']=(1,31,False)
f['hour']=(0,23,False)
f['minute']=(0,59,False)
f['second']=(0,59,False)
date_elements=[None,'year','month','day','hour','minute','second']
# Get configuration of dates and set limits to the queryset
for element in date_elements[1:]:
value=request.GET.get(element,None)
if value:
f[element]=(int(value),int(value),True)
if f['year'][2] and f['month'][2] and not f['day'][2]:
(g,lastday)=calendar.monthrange(f['year'][1],f['month'][1])
f['day']=(f['day'][0],lastday,f['day'][2])
# Limits
date_min=datetime.datetime(f['year'][0], f['month'][0], f['day'][0], f['hour'][0], f['minute'][0], f['second'][0])
date_max=datetime.datetime(f['year'][1], f['month'][1], f['day'][1], f['hour'][1], f['minute'][1], f['second'][1])
queryset=queryset.filter(eval("( Q(%s__gte=date_min) & Q(%s__lte=date_max) ) | Q(%s=None)" % (datetimeQ,datetimeQ,datetimeQ)))
# Find actual deepness
deepness_index=0
for element in date_elements[1:]:
if f[element][2]:
deepness_index+=1
else:
break
# Get results from dates to set the new order
date_results=queryset.values_list(datetimeQ, flat=True) #.dates(datetimeQ,'day')
if f['day'][0]!=f['day'][1]:
if f['month'][0]==f['month'][1]:
date_results=date_results.dates(datetimeQ,'day')
elif f['year'][0]==f['year'][1]:
date_results=date_results.dates(datetimeQ,'month')
else:
date_results=date_results.dates(datetimeQ,'year')
get=new_extra_context['get']
new_extra_context['datefilter']={}
# Save the deepness
if (deepness_index+1==len(date_elements)):
new_extra_context['datefilter']['deepness']=None
else:
new_extra_context['datefilter']['deepness']=date_elements[deepness_index+1]
new_extra_context['datefilter']['deepnessback']=[]
new_extra_context['datefilter']['deepnessinit']=[]
for element in get:
if (not element['name'] in date_elements):
struct={}
struct['name']=element['name']
struct['value']=element['value']
new_extra_context['datefilter']['deepnessinit'].append(struct)
new_extra_context['datefilter']['deepnessback'].append(struct)
elif (element['name']!=date_elements[deepness_index] and f[element['name']][2]):
struct={}
struct['name']=element['name']
struct['value']=element['value']
new_extra_context['datefilter']['deepnessback'].append(struct)
# Build the list of elements
new_extra_context['datefilter']['data']=[]
for element in date_results:
# Save the data
new_extra_context['datefilter']['data'].append(element.timetuple()[deepness_index])
new_extra_context['datefilter']['data']=list(set(new_extra_context['datefilter']['data']))
new_extra_context['datefilter']['data'].sort()
# Prepare the rightnow result
if f['month'][2]:
month=_(month_name(f['month'][0]))
else:
month='__'
if f['hour'][2]:
rightnow="%s/%s/%s %s:%s:%s" % (grv(f,'day'),month,grv(f,'year'),grv(f,'hour'),grv(f,'minute'),grv(f,'second'))
else:
rightnow="%s/%s/%s" % (grv(f,'day'),month,grv(f,'year'))
new_extra_context['datefilter']['rightnow']=rightnow
# Distinct
queryset=queryset.distinct()
# Ordering field autofill
try:
order_by_json=request.GET.get('ordering','[]')
order_by_struct=json_decode(str(order_by_json))
except Exception:
order_by_struct=[]
order_by=[]
position={}
counter=1
for order in order_by_struct:
name=order.keys()[0]
direction=order[name]
if direction=='asc':
order_by.append("%s" % (name))
elif direction=='desc':
order_by.append("-%s" % (name))
position[name]=counter
counter+=1
if order_by:
queryset=queryset.order_by(*order_by)
elif default_ordering:
queryset=queryset.order_by(default_ordering)
else:
queryset=queryset.order_by("pk")
# Check the total count of registers
total_registers=queryset.count()
# Ordering field autofill
sort={}
for value in fields:
# Get values
name=value[0]
publicname=value[1]
if len(value)>2:
size=value[2]
else:
size=None
if len(value)>3:
align=value[3]
else:
align=None
# Process ordering
ordering=[]
found=False
for order in order_by_struct:
subname=order.keys()[0]
direction=order[subname]
if name==subname:
if direction == 'desc':
direction = ''
sort_class='headerSortUp'
elif direction == 'asc':
direction = 'desc'
sort_class='headerSortDown'
else:
sort_class=''
direction = 'asc'
found=True
if direction == 'asc' or direction=='desc':
ordering.append({subname:direction})
if not found:
ordering.append({name:'asc'})
sort_class=''
# Save the ordering method
sort[name]={}
sort[name]['id']=name
sort[name]['name']=publicname
sort[name]['class']=sort_class
sort[name]['size']=size
sort[name]['align']=align
if name:
sort[name]['ordering']=json_encode(ordering).replace('"','\\"')
if name in position:
sort[name]['position']=position[name]
# Pagination
# IMPORTANT: This part is commented because I don't manage to control rowsperpage from urls.py file, it is remembering last query instead
# vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
#if 'rowsperpage' in extra_context:
# rowsperpage=extra_context['rowsperpage']
#else:
# rowsperpage=default_rows_per_page
#total_rows_per_page=request.GET.get('rowsperpage',rowsperpage)
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
total_rows_per_page=request.GET.get('rowsperpage',default_rows_per_page)
if total_rows_per_page:
try:
total_rows_per_page = int(total_rows_per_page)
except Exception:
total_rows_per_page = 'All'
else:
# IMPORTANT: Commented as before coded
# total_rows_per_page = rowsperpage
total_rows_per_page = default_rows_per_page
if total_rows_per_page == 'All':
page_number=1
total_rows_per_page = total_registers
total_rows_per_page_out = _('All')
total_pages=1
else:
total_rows_per_page = int(total_rows_per_page) # By default 10 rows per page
total_rows_per_page_out = total_rows_per_page
total_pages=total_registers/total_rows_per_page
if total_registers%total_rows_per_page:
total_pages+=1
page_number=request.GET.get('page',1) # If no page specified use first page
if page_number=='last':
page_number=total_pages
else:
try:
page_number=int(page_number)
except:
page_number=1
if page_number>total_pages:
page_number=total_pages
# Build the list of page counters allowed
choice=[]
c=default_rows_per_page
chk=1
while total_registers>=c:
choice.append(c)
if chk==1:
# From 5 to 10
c=c*2
# Next level
chk=2
elif chk==2:
# From 10 to 25 (10*2+10/2)
c=c*2+c/2
# Next level
chk=3
elif chk==3:
# From 25 to 50
c*=2
chk=1
# Add all choice in any case
choice.append(_('All'))
# Save the pagination in the structure
new_extra_context['rowsperpageallowed']=choice
new_extra_context['rowsperpage']=total_rows_per_page_out
new_extra_context['pagenumber']=page_number
if type(object_id)==type(u'abc'):
# If object_id is a string, we have a name not an object
new_extra_context['object_name']=object_id
object_obj = None
else:
# If is not an string
if object_id:
# If we got one, load the object
obj=extra_context['obj']
object_obj = get_object_or_404(obj, pk=object_id)
else:
# There is no object
object_obj = None
new_extra_context['object_obj']=object_obj
# Build the columns structure
new_extra_context['columns']=[]
for value in fields:
field=value[0]
new_extra_context['columns'].append(sort[field])
# Get the full number of registers and save it to extra_context
new_extra_context['total_registers']=total_registers
if total_rows_per_page=='All':
# Remove total_rows_per_page if is all
total_rows_per_page=None
new_extra_context['page_before']=None
new_extra_context['page_after']=None
new_extra_context['start_register']=1
new_extra_context['showing_registers']=total_registers
else:
# Page before
if page_number<=1:
new_extra_context['page_before']=None
else:
new_extra_context['page_before']=page_number-1
# Page after
if page_number>=total_pages:
new_extra_context['page_after']=None
else:
new_extra_context['page_after']=page_number+1
# Starting on register number
new_extra_context['start_register']=(page_number-1)*total_rows_per_page+1
new_extra_context['showing_registers']=total_rows_per_page
# Calculate end
new_extra_context['end_register']=min(new_extra_context['start_register']+new_extra_context['showing_registers']-1,total_registers)
# If compact rows
hide_head=[]
hide_tail=[]
hide_subhead=[]
hide_subtail=[]
if compact_rows:
(compact_field,compact_subelements)=compact_rows
lastvalue=None
lastrow=None
total_subelements=0
for row in queryset:
value=eval("row.%s" % (compact_field))
# Count the subelements from this row
if compact_subelements:
count_subelements=eval("row.%s.count()" % (compact_subelements))
else:
count_subelements=1
# If the new row belongs to the same group than the row before
if value==lastvalue:
# Hide the head from this row
hide_head.append(row.id)
# Hide the tail from the last row
hide_tail.append(lastrow.id)
# If there were elements in the group (somebody already opened the subhead, hide the head of the subgroup) or if this row has no elements (no need to open this subhead)
if total_subelements>0 or count_subelements==0:
# Hid the subhead
hide_subhead.append(row.id)
# Hide the tail of the last row, since we want to connect both groups
hide_subtail.append(lastrow.id)
# Add the total count of elements
total_subelements+=count_subelements
# This row doesn't belong to the opened group
else:
# If there was some row already and there are no elements in the group (nobody opened the group, so we don't have to close it either)
if lastrow and total_subelements==0:
# Hide the tail from the group
hide_subtail.append(lastrow.id)
# Startup a new count of elements (Reset the total count of subelements)
total_subelements=0
total_subelements+=count_subelements
# If the new group doesn't have element (we don't think about opening the group)
if total_subelements==0:
# Hide the head from this group
hide_subhead.append(row.id)
# Remember
lastvalue=value
lastrow=row
# Proper closing the group after the bucle if there was some row opened
if lastrow and total_subelements==0:
# Hide the tail from the group if was no element in the group (nobody opened the group)
hide_subtail.append(lastrow.id)
# Save it in the public structure
new_extra_context['hide_head']=hide_head
new_extra_context['hide_tail']=hide_tail
new_extra_context['hide_subhead']=hide_subhead
new_extra_context['hide_subtail']=hide_subtail
# Save extra context
extra_context.update(new_extra_context)
# Empty results are empty
if page_number==0:
total_rows_per_page=0
# Return results
return object_list(request, queryset=queryset, template_name=template_name, extra_context=extra_context, paginate_by=total_rows_per_page, page=page_number)
What it calls in urls.py from 'Tareas':
from django.conf.urls import patterns, include, url
from tareas.models import Tareacobro, Tipotarea, Agentes, Perfil, Tareas
from django.conf import settings
from common.generic import view_list
# Uncomment the next two lines to enable the admin:
info_tasks = {'queryset': Tareas.objects.all()}
urlpatterns = patterns('tareas.views',
# =====TASKS======
# url(r'^$','tareas'),
(r'^', view_list, dict( info_tasks, extra_context={'obj':Tareas} ),'admin/tareas/tareas'),
# (r'^$',view_list, dict(info_tasks),'admin/tareas/tareas'),
#url(r'^$',view_list, dict(info_tasks, extra_context={'obj':Tareas} ),'tareas'),
)
fields=queryset.__dict__['model']() instantiates a model instance. As an aside, you could use queryset.model instead of looking in __dict__.
Django models instances do not have an attribute __fields__, so you get an attribute error.
If you want to access the fields you have defined in the model admin, you can fetch the model admin class from the registry.
from django.contrib.admin import site
model_admin = site._registry[Model]
fields = model_admin.fields
I can't run a unit test with formset.
I try to do a test:
class NewClientTestCase(TestCase):
def setUp(self):
self.c = Client()
def test_0_create_individual_with_same_adress(self):
post_data = {
'ctype': User.CONTACT_INDIVIDUAL,
'username': 'dupond.f',
'email': 'new#gmail.com',
'password': 'pwd',
'password2': 'pwd',
'civility': User.CIVILITY_MISTER,
'first_name': 'François',
'last_name': 'DUPOND',
'phone': '+33 1 34 12 52 30',
'gsm': '+33 6 34 12 52 30',
'fax': '+33 1 34 12 52 30',
'form-0-address1': '33 avenue Gambetta',
'form-0-address2': 'apt 50',
'form-0-zip_code': '75020',
'form-0-city': 'Paris',
'form-0-country': 'FRA',
'same_for_billing': True,
}
response = self.c.post(reverse('client:full_account'), post_data, follow=True)
self.assertRedirects(response, '%s?created=1' % reverse('client:dashboard'))
and I have this error:
ValidationError: [u'ManagementForm data is missing or has been
tampered with']
My view :
def full_account(request, url_redirect=''):
from forms import NewUserFullForm, AddressForm, BaseArticleFormSet
fields_required = []
fields_notrequired = []
AddressFormSet = formset_factory(AddressForm, extra=2, formset=BaseArticleFormSet)
if request.method == 'POST':
form = NewUserFullForm(request.POST)
objforms = AddressFormSet(request.POST)
if objforms.is_valid() and form.is_valid():
user = form.save()
address = objforms.forms[0].save()
if url_redirect=='':
url_redirect = '%s?created=1' % reverse('client:dashboard')
logon(request, form.instance)
return HttpResponseRedirect(url_redirect)
else:
form = NewUserFullForm()
objforms = AddressFormSet()
return direct_to_template(request, 'clients/full_account.html', {
'form':form,
'formset': objforms,
'tld_fr':False,
})
and my form file :
class BaseArticleFormSet(BaseFormSet):
def clean(self):
msg_err = _('Ce champ est obligatoire.')
non_errors = True
if 'same_for_billing' in self.data and self.data['same_for_billing'] == 'on':
same_for_billing = True
else:
same_for_billing = False
for i in [0, 1]:
form = self.forms[i]
for field in form.fields:
name_field = 'form-%d-%s' % (i, field )
value_field = self.data[name_field].strip()
if i == 0 and self.forms[0].fields[field].required and value_field =='':
form.errors[field] = msg_err
non_errors = False
elif i == 1 and not same_for_billing and self.forms[1].fields[field].required and value_field =='':
form.errors[field] = msg_err
non_errors = False
return non_errors
class AddressForm(forms.ModelForm):
class Meta:
model = Address
address1 = forms.CharField()
address2 = forms.CharField(required=False)
zip_code = forms.CharField()
city = forms.CharField()
country = forms.ChoiceField(choices=CountryField.COUNTRIES, initial='FRA')
In particular, I've found that the ManagmentForm validator is looking for the following items to be POSTed:
form_data = {
'form-TOTAL_FORMS': 1,
'form-INITIAL_FORMS': 0
}
Every Django formset comes with a management form that needs to be included in the post. The official docs explain it pretty well. To use it within your unit test, you either need to write it out yourself. (The link I provided shows an example), or call formset.management_form which outputs the data.
It is in fact easy to reproduce whatever is in the formset by inspecting the context of the response.
Consider the code below (with self.client being a regular test client):
url = "some_url"
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
# data will receive all the forms field names
# key will be the field name (as "formx-fieldname"), value will be the string representation.
data = {}
# global information, some additional fields may go there
data['csrf_token'] = response.context['csrf_token']
# management form information, needed because of the formset
management_form = response.context['form'].management_form
for i in 'TOTAL_FORMS', 'INITIAL_FORMS', 'MIN_NUM_FORMS', 'MAX_NUM_FORMS':
data['%s-%s' % (management_form.prefix, i)] = management_form[i].value()
for i in range(response.context['form'].total_form_count()):
# get form index 'i'
current_form = response.context['form'].forms[i]
# retrieve all the fields
for field_name in current_form.fields:
value = current_form[field_name].value()
data['%s-%s' % (current_form.prefix, field_name)] = value if value is not None else ''
# flush out to stdout
print '#' * 30
for i in sorted(data.keys()):
print i, '\t:', data[i]
# post the request without any change
response = self.client.post(url, data)
Important note
If you modify data prior to calling the self.client.post, you are likely mutating the DB. As a consequence, subsequent call to self.client.get might not yield to the same data, in particular for the management form and the order of the forms in the formset (because they can be ordered differently, depending on the underlying queryset). This means that
if you modify data[form-3-somefield] and call self.client.get, this same field might appear in say data[form-8-somefield],
if you modify data prior to a self.client.post, you cannot call self.client.post again with the same data: you have to call a self.client.get and reconstruct data again.
Django formset unit test
You can add following test helper methods to your test class [Python 3 code]
def build_formset_form_data(self, form_number, **data):
form = {}
for key, value in data.items():
form_key = f"form-{form_number}-{key}"
form[form_key] = value
return form
def build_formset_data(self, forms, **common_data):
formset_dict = {
"form-TOTAL_FORMS": f"{len(forms)}",
"form-MAX_NUM_FORMS": "1000",
"form-INITIAL_FORMS": "1"
}
formset_dict.update(common_data)
for i, form_data in enumerate(forms):
form_dict = self.build_formset_form_data(form_number=i, **form_data)
formset_dict.update(form_dict)
return formset_dict
And use them in test
def test_django_formset_post(self):
forms = [{"key1": "value1", "key2": "value2"}, {"key100": "value100"}]
payload = self.build_formset_data(forms=forms, global_param=100)
print(payload)
# self.client.post(url=url, data=payload)
You will get correct payload which makes Django ManagementForm happy
{
"form-INITIAL_FORMS": "1",
"form-TOTAL_FORMS": "2",
"form-MAX_NUM_FORMS": "1000",
"global_param": 100,
"form-0-key1": "value1",
"form-0-key2": "value2",
"form-1-key100": "value100",
}
Profit
There are several very useful answers here, e.g. pymen's and Raffi's, that show how to construct properly formatted payload for a formset post using the test client.
However, all of them still require at least some hand-coding of prefixes, dealing with existing objects, etc., which is not ideal.
As an alternative, we could create the payload for a post() using the response obtained from a get() request:
def create_formset_post_data(response, new_form_data=None):
if new_form_data is None:
new_form_data = []
csrf_token = response.context['csrf_token']
formset = response.context['formset']
prefix_template = formset.empty_form.prefix # default is 'form-__prefix__'
# extract initial formset data
management_form_data = formset.management_form.initial
form_data_list = formset.initial # this is a list of dict objects
# add new form data and update management form data
form_data_list.extend(new_form_data)
management_form_data['TOTAL_FORMS'] = len(form_data_list)
# initialize the post data dict...
post_data = dict(csrf_token=csrf_token)
# add properly prefixed management form fields
for key, value in management_form_data.items():
prefix = prefix_template.replace('__prefix__', '')
post_data[prefix + key] = value
# add properly prefixed data form fields
for index, form_data in enumerate(form_data_list):
for key, value in form_data.items():
prefix = prefix_template.replace('__prefix__', f'{index}-')
post_data[prefix + key] = value
return post_data
The output (post_data) will also include form fields for any existing objects.
Here's how you might use this in a Django TestCase:
def test_post_formset_data(self):
url_path = '/my/post/url/'
user = User.objects.create()
self.client.force_login(user)
# first GET the form content
response = self.client.get(url_path)
self.assertEqual(HTTPStatus.OK, response.status_code)
# specify form data for test
test_data = [
dict(first_name='someone', email='someone#email.com', ...),
...
]
# convert test_data to properly formatted dict
post_data = create_formset_post_data(response, new_form_data=test_data)
# now POST the data
response = self.client.post(url_path, data=post_data, follow=True)
# some assertions here
...
Some notes:
Instead of using the 'TOTAL_FORMS' string literal, we could import TOTAL_FORM_COUNT from django.forms.formsets, but that does not seem to be public (at least in Django 2.2).
Also note that the formset adds a 'DELETE' field to each form if can_delete is True. To test deletion of existing items, you can do something like this in your test:
...
post_data = create_formset_post_data(response)
post_data['form-0-DELETE'] = True
# then POST, etc.
...
From the source, we can see that there is no need include MIN_NUM_FORM_COUNT and MAX_NUM_FORM_COUNT in our test data:
MIN_NUM_FORM_COUNT and MAX_NUM_FORM_COUNT are output with the rest of the management form, but only for the convenience of client-side code. The POST value of them returned from the client is not checked.
This doesn't seem to be a formset at all. Formsets will always have some sort of prefix on every POSTed value, as well as the ManagementForm that Bartek mentions. It might have helped if you posted the code of the view you're trying to test, and the form/formset it uses.
My case may be an outlier, but some instances were actually missing a field set in the stock "contrib" admin form/template leading to the error
"ManagementForm data is missing or has been tampered with"
when saved.
The issue was with the unicode method (SomeModel: [Bad Unicode data]) which I found investigating the inlines that were missing.
The lesson learned is to not use the MS Character Map, I guess. My issue was with vulgar fractions (¼, ½, ¾), but I'd assume it could occur many different ways. For special characters, copying/pasting from the w3 utf-8 page fixed it.
postscript-utf-8