Django raises MultiValueDictKeyError in File Upload - django

I have already consulted lots of forums and I can't get an answer. I have installed a file upload in my Django app to save data into my server. But it does not work. Instead, it raises a MultiValueDictKeyError. I guess the problem is that there is not request.FILES (because it raises an error in request.FILES mentions), so the file upload is not working. This is my views.py:
def list_files(request, phase_id):
phase = get_object_or_404(Phase, pk=int(phase_id))
if request.method == 'POST':
#form = DocumentForm(request.POST, request.FILES)
form = DocumentForm(request.POST, request.FILES)
if form.is_valid():
newdoc = Document(docfile = request.FILES['docfile'], phase = phase_id)
newdoc.save()
doc_to_save = request.FILES['docfile']
filename = doc_to_save._get_name()
fd = open(settings.MEDIA_URL+'documents/'+str(filename),'wb')
for chunk in doc_to_save.chunks():
fd.write(chunk)
fd.close()
return HttpResponseRedirect(reverse('list_files'))
else:
form = DocumentForm()
documents = Document.objects.filter(phase=phase_id)
return render_to_response('teams_test/list_files.html',{'documents': documents, 'form':form, 'phase':phase}, context_instance = RequestContext(request)
)
The document form in forms.py:
class DocumentForm(forms.ModelForm):
docfile = forms.FileField(label='Select a file', help_text='max. 42 megabytes')
class Meta:
model = Document
The class document in models.py:
class Document(models.Model):
docfile = models.FileField(upload_to='documents')
phase = models.ForeignKey(Phase)
Finally, my html code:
{% extends "layouts/app.html" %}
{% load i18n user %}
{% block title %}{% trans "Files list" %}{% endblock %}
{% block robots %}noindex,nofollow{% endblock %}
{% block page%}
<div id="page" class="container">
<div class="header prepend-2 span-20 append-2 last whiteboard">
<h2 style="margin-left:-40px">{{ phase.name }} files</h2>
{% if documents %}
<ul>
{% for document in documents %}
<li><a href="{{ document.docfile.url }}">{{ document.docfile.name }}
{% endfor %}
</ul>
{% else %}
<p>No documents.</p>
{% endif %}
<form action="{% url list_files phase.id %}" method="post" enctype="multipart/form-data">
{% csrf_token %}
<input id="file" type="file" />
<input id="submit" type="submit" value="Upload file" />
</form>
</div>
</div>
{% endblock %}
My traceback says:
Exception Type: MultiValueDictKeyError
Exception Value: "Key 'docfile' not found in <MultiValueDict: {}>"
my_dir/views.py in list_files
newdoc = Document(docfile = request.FILES['docfile'], phase = phase_id)
And my QueryDict is empty:
POST:<QueryDict: {u'csrfmiddlewaretoken': [u'UZSwiLaJ78PqSjwSlh3srGReICzTEWY1']}>
Why? What am I doing wrong?
Thanks in advance.

You need to change multipart/form_data to multipart/form-data - that's why request.FILES is empty: the form isn't sending things in the way Django expects due to the typo. [EDIT: this has now been done]
Update 1: Also, rather than directly access request.FILES, try relying on the modelform's default behaviour, as then it'll have been handled as an upload appropriately. ie, newdoc = form.save() should do all you need, from a quick look at it - is there a particular reason you manually saving the file when the modelform can do that for you?
Update 2: Ah, look: you're not assigning a name to your file upload element
From the docs:
HttpRequest.FILES A dictionary-like object containing all uploaded files. Each key in FILES is the name from the <input type="file" name="" />. Each value in FILES is an UploadedFile
So, you need to change
<input id="file" type="file" />
to
or, for default Django convention
<input id="id_docfile" type="file" name="docfile"/>
Indeed, it's usually better to use the Django form to render the actual field, even if you've moved beyond the whole {{form.as_p}} approach:
{{form.docfile}}
PS. if you've not read them fully, I heartily recommend taking the time to go through all of the forms documentation

Modify Post method to
<form action="" method="post" enctype="multipart/form-data">{% csrf_token %}

For anymore who tried the above and still couldn't figure out a solution. Here's what I did:
views.py
if request.method == 'POST':
doc = request.FILES #returns a dict-like object
doc_name = doc['filename']
...

For anyone who tried the above and still couldn't figure out a solution. Here's what I did (2nd part):
if request.method == 'POST' and 'filename' in request.FILES:
doc = request.FILES #returns a dict-like object
doc_name = doc['filename']
...

Related

Django Return redirect working for one view but not for the other

I cannot understand why it is working in one instance but not the other. I am working with django and output in django template. The only difference between the views/functions are in the second one (the one that is not working) I update the field with time. Time updates, saves in the model and displays updated time correctly. It is just the redirect that is not working.
The working redirect code-
Template, this code takes me to the edit page. Name of the url is "update" -
<td><button>Edit</button></td>
The form on the dit page-
{% block content %}
<div class="wrapper">
<h1 class="ok">Entry Form</h1>
<form action="" method="POST">
{% csrf_token %}
{{form}}
<input type="submit" value="submit">
</form>
</div>
<br>
{% endblock content %}
url-
path('update_entry/<str:pk>/', views.update, name = "update"),
And views.py-
def update(request, pk):
order=Bank1.objects.get(id=pk)
form = Bank1Form(instance=order)
if request.method == 'POST':
form = Bank1Form(request.POST, instance=order)
if form.is_valid():
form.save()
return redirect('/bank1')
context = {'form':form}
return render(request, 'myapp/entry.html', context)
Now here is the non working code. Template, the line that takes me to the update page. Name of the url is "update_enter_workout.-
<td><button>Start Time</button></td>
Form on the Edit page. Didn't add the entire form since I only need to update the time from this page. Just the submit button.-
{% block content %}
<Button>Close this page and go to Home</Button>
<div class="wrapper">
<h1 class="ok">Start/End the set now?</h1>
<form action="" method="post">
{% csrf_token %}
<input type="submit" value="YES!">
</form>
</div>
{% endblock content %}
url-
path('update_enter_workout/<str:pk>/', views.update_workout, name='update_enter_workout'),
Views.py-
def update_workout(request, pk):
order=WorkOut.objects.get(id=pk)
form=WorkOutForm(instance=order)
if request.method=='POST':
form=WorkOutForm(request.POST, instance=order)
time=datetime.now().strftime('%H:%M:%S')
WorkOut.objects.filter(id=pk).update(start=time)
if form.is_valid():
form.save()
return redirect('/bank1')
context={'form':form}
return render(request, 'myapp/enter_workout.html', context)
As you can see they are written the same way, but in the second instance redirect is not working. Any idea what can be changed. It is so simple, couldn't even find a typo or simple mistake like that.
Any advise?
Are you accepting str for id fields? Primary Keys are integers, not strings. You need to cast to an int path converter:
Instead of:
path('update_enter_workout/<str:pk>/', views.update_workout, name='update_enter_workout'),
Use:
path('update_enter_workout/<int:pk>/', views.update_workout, name='update_enter_workout'),

How to validate CSV file upload on the fly in Django using a custom class-based validator?

I have an upload form used to inject CSV file content into the database. So far I'm struggling with validation itself. I switched from function based validator to class based validator so I could pass argument to the validator itself to make it reusable.
views.py
#login_required
def contact_import(request):
if request.method == 'POST' and request.FILES['file']:
form = ContactImportForm(request.POST,request.FILES)
if form.is_valid():
post_result = form.save(commit=True)
return redirect("/contacts/")
else:
form = ContactImportForm()
return render(request,'contact_import_form.html',{
'form': form
})
forms.py
class ContactImportForm(forms.Form):
headers = ["lastname","firstname","email"]
file = forms.FileField(label='Select your CSV File',validators=[CsvFileValidator(headers)])
def clean_file(self):
file = self.cleaned_data['file']
return file
validators.py
class CsvFileValidator(object):
def __init__(self, headers):
self.headers = headers
def __call__(self,file):
# Extension check
file_extension = os.path.splitext(file.name)[1]
valid_extensions = [ ".csv", ".CSV"]
if not file_extension.lower() in valid_extensions:
msg = "Invalid file, select a valid CSV file"
raise ValidationError("{}".format(msg), status='invalid')
# File content check
try:
csv_format = csv.Sniffer().sniff(file.read(1024))
file.seek(0,0)
except csv.Error:
msg = "Invalid file, select a valid CSV file"
raise ValidationError("{}".format(msg), status='invalid')
csv_reader = csv.reader(file.read().splitlines(), dialect=csv_format,delimiter=";")
for r_index, row in enumerate(csv_reader):
# CSV headers check
if r_index == 0:
if sorted(self.headers) != sorted(row):
msg = "Missing or invalid headers"
raise ValidationError("{}".format(msg), status='invalid')
# Skip blank line
if not "".join(str(field) for field in row):
continue
return True
contact_import_form.html
HTML template, relevant parts only:
<div class="messages">
{% if form.errors %}
{% if form.non_field_errors %}
<div class="alert alert-danger" role="alert">
<p class="text-danger"><strong>AN error occured during form validation</strong>
{% for error in form.non_field_errors %}
{{error}}
{% endfor %}
</p>
{% endif %}
{% for field in form %}
{% if field.errors %}
<p class="text-danger"><strong>{{ field.name }} :</strong> {{ field.errors }}</p>
{% endif %}
{% endfor %}
</div>
{% endif %}
</div>
<form id="contact-import-form" method="post" action=" " role="form" enctype="multipart/form-data">
{% csrf_token %}
<div class="form-group">
{{ form.file.label_tag }}
{{ form.file }}
</div>
<div class="form-group">
<label class="col-md-4 control-label" for="submit"></label>
<div class="col-md-8">
<button id="submit" name="submit" class="btn btn-primary" value="Submit">Valider</button>
Annuler
</div>
</div>
</form>
For now I have an error after submitting the form:
TypeError: cannot use a string pattern on a bytes-like object
This error points to file content check in my validator, specifically line above:
csv_format = csv.Sniffer().sniff(file.read(1024))
How to implement this custom class-based validator to work properly? Also I imagine that missing code to update the database should live in the view in if form.is_valid(): code block.
How to validate CSV file upload on the fly with my class-based validator and subsenquently use its data to populate a model?
The issue is with the type of the file, file.read(1024) does not return a string, to fix this you should decode the output as csv_format = csv.Sniffer().sniff(file.read(1024).decode('UTF-8')).
When a Form's clean method is called and you want to check the file contents, you need to know how to properly open the InMemoryUploadedFile type. This answer shows how to open an uploaded file.
As donmelchior pointed out above in a comment, it looks like your CsvFileValidator class is opening the file incorrectly. Try something like
import csv
import io
decoded_file = file.read().decode("utf-8")
io_string = io.StringIO(decoded_file)
csv_reader = csv.DictReader(io_string)

Django Admin Action using intermediate page

I have a model with a lot of fields. I only have a few fields I that I want to be required. So instead of the change list super long, I want to have a short change list then have admin actions that can give predefined subsets of the fields.
The initial action takes me to the correct page but when I submit the form it returns me to whatever page I designate, but doesn't update the fields. I am okay with tearing this down starting over again if needed. I think what I really need to know, what do I put in the action="" portion of the html to have the recursion work properly?
I am using django 1.7. I have to obfuscate a lot of my fields as a cya thing since I am working in a heavily information secure field.
Here is my admin.py
class CredentialAdmin(admin.ModelAdmin):
fields = ['reservedBy','reserveto']
list_display = ['reservedBy','reserveto']
class reserveToFormAdmin(forms.Form):
reservedBy = forms.CharField(widget=forms.Textarea, max_length=50)
reserveto = forms.DateTimeField(widget=forms.DateTimeInput)
def reserveCred(self, request, queryset):
form = None
plural = ''
if 'submit' in request.POST:
form = self.reserveToFormAdmin(request.POST)
for f in form.fields:
print f
print form.is_valid()
print form.errors
if form.is_valid():
reservetos = form.cleaned_data['reserveto']
reservedBys = form.cleaned_data['reservedBy']
print "hello"
count = 0
for cred in queryset:
cred.reserveto = reservetos
cred.reservedBy = reservedByss
cred.save()
count += 1
if count != 1:
plural = 's'
self.message_user(request, "Successfully reserved %s cred%s." % (count, plural))
return HttpResponseRedirect(request.get_full_path(),c)
if not form:
form = self.reserveToFormAdmin(initial={'_selected_action' : request.POST.getlist(admin.ACTION_CHECKBOX_NAME)})
return render(request,'admin/reserveCreds.html',{'creds':queryset, 'form':form, 'path':request.get_full_path()})
reserveCred.short_description = "Reserve Selected Creds"
actions = [check_out_a_cred,check_in_a_cred,audit_creds,CompareAudits,reserveCred]
reserveCreds.html
{% extends "admin/base_site.html" %}
{% block content %}
<p>How long and which department to reserver creds:</p>
<form action="{{ path }}" method="post">{% csrf_token %}
{{ form }}
<input type="submit" name="submit" value="submit" />
<input type="button" value = "Cancel" />
</form>
<h2> reserving: </h2>
<ul>
{% for cred in creds %}
<li> {{ cred.userid }} </li>
{% endfor %}
</ul>
{% endblock %}

Validating a formset

I am getting this error: ValidationError at /screen-many/
[u'ManagementForm data is missing or has been tampered with'] and I think it is due to the folling code in my view...
# e_pk_list is a list of id's that I got from POST
e_students = Student.objects.filter(pk__in=e_pk_list)
my_iterator = iter(e_students) # Each list item will correspond to a form.
SurveyFormset = formset_factory(SurveyForm, extra=len(e_students))
# Is this the tampering that I can't do??
SurveyFormset.form = staticmethod(curry(SurveyForm, item_iterator=my_iterator))
if request.method == 'POST':
survey_formset = SurveyFormset(request.POST)
if survey_formset.is_valid():
for form in survey_formset:
saved = form.save(commit=False)
saved.surveyset = ss
saved.save()
return HttpResponseRedirect('/')
else:
survey_formset = SurveyFormset()
Thanks
EDIT: I guess I should have mentioned that I already have a managementform in my template....
<form action="" method="POST">{% csrf_token %}
{{ survey_formset.management_form }}
{% for form in survey_formset %}
<div class="item">
{% crispy form %}
</div>
{% endfor %}
<input type="submit" value="Submit" class='button' />
</form>
Its seems that you didn't put management_form in your form .
Put this in your html form where your are displaying SurveyFormset
{{ SurveyFormset.management_form }}
A formset has many forms. Django keeps track of number of forms in formset using management form data. You should add management_form in the template too, which should be posted along with other POST data.
So, you should have:
<form method="POST" action=".">
{{survey_formset.management_form}}
{% comment %}Other form fields{% endcomment %}
</form>

problem handling upload file django

I'm trying to do a feature that would allow the user to upload a file. That file would have information to insert into the database, that's why in the function I'm saving data. Not sure if it's the best option.
What I've done so far:
forms.py:
class UploadFileForm(forms.Form):
file = forms.FileField()
views:
def last_step(request, word):
if request.method == 'POST':
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
msg = handle_uploaded_file(word, request.FILES['file'])
return render_to_response('final_insert.html', {'msg': msg})
else:
form = UploadFileForm()
return render_to_response('upload.html',
{
'word': word,
'form': form })
template:
<form enctype="multipart/form-data" action="{% url upload_done 'clinical' %}" method="post">
<div>
{% for field in form %}
{{ field }}
{% endfor %}
<input type="submit" value="Save" />
</div>
</form>
function:
def handle_uploaded_file(word, f):
msg = "first stop"
data = []
for chunk in f.chunks():
data.append(chunk)
msg = "after chunk"
if word == 'clinical':
pat = Patient.objects.values_list('patient', flat=True)
for i in data:
if i[0] not in pat:
b2 = Patient(patient=i[0])
b2.save()
msg = "number was inserted"
else:
msg = "something"
return msg
The problem is when I hit "save" in the template, it redirects well to another template, but I don't see any message, like I suppose to see (final_insert.html shows {{ msg }})
Can someone help me understand what I'm doing wrong?
Any help is welcome!
Thanks for your help!
I was able to understand my mistake.
sorry guys for my silly mistake
so this is the form:
<form enctype="multipart/form-data" action="{% url upload_done 'clinical' %}" method="post">
<div>
{% for field in form %}
{{ field }}
{% endfor %}
<input type="submit" value="Save" />
</div>
</form>
urls:
url(r'^insert/file/(?P<word>clinical)/upload/$', last_step, name="upload"),
url(r'^insert/file/(?P<word>clinical)/upload/result/$', final, name='upload_done'),
so the view last_step corresponds to the url "upload" and not "upload_done"
I wrote into the form action={% url upload_done 'clinical' %}, so when I hit save it will redirect me automatically to the other template. Without running the code!!
So I changed the form to:
<form enctype="multipart/form-data" action="{% url upload 'clinical' %}" method="post">
<div>
{% for field in form %}
{{ field.label_tag }}
{{ field }}
{% endfor %}
<input type="submit" value="Guardar" />
</div>
</form>
and now it works..
sorry guys, I thought I needed to redirect to the other page but when he redirects he doesn't run the code..