I`ve spent a few hours trying to debug this issue. I am trying to import a catalog of product into my app. I am using Django Import-export to import a csv file. I have tried to import it via the admin site and via a simple upload file. In both case, I am encountering some errors. Would you have an idea how to resolve them?
1st Method: Import the csv file via the admin site
Error encountered: Imported file has a wrong encoding: 'charmap' codec can't decode byte 0x9d in position 13365: character maps to
It looks like this method cannot accept some character(""). How can I change my code to accept any character? I`ve already removed the following characters: £, - and try to encode it in UTF-8. I have also done a search to remove every non ASCII characters(128-255)
2eme Method: Import the csv file via the website
Error encountered: 'MultiValueDict' object is not callable
views.py
def simple_upload(request):
if request.method == 'POST':
file_format = request.POST.get('file-format')
product_resource = ProductResource()
dataset = Dataset()
new_products = request.FILES['Price_List']
if file_format == 'CSV':
imported_data = dataset.load(new_products.read().decode('utf-8'),format='csv')
result = product_resource.import_data(dataset, dry_run=True)
elif file_format == 'JSON':
imported_data = dataset.load(new_products.read().decode('utf-8'),format='json')
# Testing data import
result = product_resource.import_data(dataset, dry_run=True)
if not result.has_errors():
# Import now
product_resource.import_data(dataset, dry_run=False)
return render(request, 'catalog/simple_upload.html')
models.py
from import_export import resources
from .models import Product
class ProductResource(resources.ModelResource):
class Meta:
model = Product
skip_unchanged = True
report_skipped = False
simple_upload.html
{% extends 'base.html' %}
{% block body %}
<h3>Import Data</h3>
<p>importing to database</p>
<form method="post" enctype="multipart/form-data">
{% csrf_token %}
<input type="file" name="importData">
<p>Please select format of file.</p>
<select name="file-format" class="form-control my-3">
<option selected>Choose format...</option>
<option>CSV</option>
<option>JSON</option>
</select>
<button class="btn btn-primary" type="submit">Import</button>
</form>
{% endblock %}
Many Thanks,
Related
Recently I've decide to create form creation form in Flask web app. After searching form creation found Formfield, FieldList classes in flask wtf forms and I can create the form with these classes. but it doesn't provide that I want to.
First- I am going to create a form creation form which will be help me to create form and fields on management interface.
Second- I want to be able to add the fields, not the same type of field, all different kind, such as (booleanField, StringField, IntegerField, DateTimeField etc.) because, in the form there could be different type of fields for specific reason.
Third- I want to retreive this form whenever I want to use in my view
On the DB models side;
class Form(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.StringField)
fields = db.relationship('FormFields', backref='forms', lazy=True)
class FormFields(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.StringField)
field_type = db.Column(db.StringField)
form_id = db.Column(db.Integer, db.ForeignKey('forms.id'), nullable=False)
And othr tables for StrinField, BooleanField, TextField, etc. etc. should be connected this field model, and when I save the data over this created form, these data should be saved int the correct tables
The reason I am searching this, because I don't want to hardcode the Forms and fields in the code, when I need to new form or field I don't want to update code itself, it should be dynamically updated on the database.
And I want to use sqlalchemy based form creation from management page. And this will help to create anytime new form and relate the fields to the form. And on the internet still I didn't find the these style form creation for Flask, almost all of them creating dynamic for with same type of fields
Any ideas?
Last a couple of weeks I was search how to create dynamically flask form based on models
And #nick-shebanov has been redirect me to another approach EAV impelemntation, which is really diffucult to implement. I've tried :)
And decide to create form based on dictionary, and intend to populate the related attributes from the model and pass it to form as dictionary.
What I've done so far;
# app.py file
from flask_wtf import Form
from flask import Flask, render_template, request, flash
from flask_wtf import FlaskForm
from flask_migrate import Migrate
from flask_sqlalchemy import SQLAlchemy
from wtforms import TextField, IntegerField, HiddenField, StringField, TextAreaField, SubmitField, RadioField,SelectField
from wtforms import validators, ValidationError
app = Flask(__name__)
app.secret_key = 'secret123'
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///sqlite.db'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = True
db = SQLAlchemy(app)
migrate = Migrate(app, db)
# form class with static fields
class DynamicForm(FlaskForm):
form_type = HiddenField(default='FormType', render_kw={ 'type':'hidden' })
# name = StringField()
#app.route('/', methods=['GET', 'POST'])
def index():
fields = {
'username': 'Username',
'first_name': 'Fisrt Name',
'last_name': 'Last Name',
'email': 'Email',
'mobile_phone': 'Mobile Phone'
}
for key, value in fields.items():
setattr(DynamicForm, key, StringField(value))
form = DynamicForm()
if request.method == 'POST':
# print(dir(request.form))
# print(request.form)
dict = request.form.to_dict()
# print(dict.keys())
print(request.form.to_dict())
return render_template('index.html', form=form)
if __name__ == '__main__':
app.run(debug = True)
# index.html template
<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
<meta charset="utf-8">
<title></title>
</head>
<body>
<form method="POST">
{{ form.csrf_token }}
{{ form.form_type }}
{% for field in form if field.name != 'csrf_token' %}
{% if field.name != 'form_type' %}
<div>
{{ field.label() }}
{{ field() }}
{% for error in field.errors %}
<div class="error">{{ error }}</div>
{% endfor %}
</div>
{% endif %}
{% endfor %}
<input type="submit" value="Go">
</form>
</body>
</html>
Now I can see my all fields has been rendered including hidden field. When I fill the form and post the data, I can capture it.
But still I didn't achieve to implement save the captured data into database like vertical DB modelling style yet
Here is my simple approach of DB modelling
see image here
Is there suggestions?
As you can tell, i'm new to Django, but already love it. I have a functional scraping server that scrapes 7 selects from a diagnostics page on one server. All environments I want to use this will have many of these servers with the same data to monitor performance. I would like to use this function to scrape this data from all entered target servers by input from the html template. Adding each server to monitor that will show up below the input text field from the main html page. I have completed this with a static url, but have been unsuccessful passing the different urls to scrape from the html template to the views url variable I have for the static address.
I've attempted to create forms and pass that to the html template without success, including editing the views file. Reverted the code back to the original working code to not cause more confusion.
html template:
<form method="POST">
{% csrf_token %}
<div class="field has-addons">
<div class="control is-expanded">
<input type="text" class="input"
placeholder="Relay Name">
</div>
<div class="control">
<button type="submit" class="button is-info">
Add Relay
</button>
</div>
</div>
</form>
Views.py:
import requests, bs4
from django.shortcuts import render
from django.http import HttpResponse
from bs4 import BeautifulSoup
from urllib.request import urlopen
from .models import Relay
def index(request):
url = 'hardcoded server url'
page = urlopen(url)
soup = BeautifulSoup(page, 'html.parser')
relay = 'Relay'
dic = requests.get(url.format(relay))
elema = soup.select('body > div:nth-child(13) > div.forminput')
elem1 = elema[0].text.strip()
elemb = soup.select('body > div:nth-child(14) > div.forminput')
elem2 = elemb[0].text.strip()
elemc = soup.select('body > div:nth-child(15) > div.forminput')
elem3 = elemc[0].text.strip()
elemd = soup.select('body > div:nth-child(16) > div.forminput')
elem4 = elemd[0].text.strip()
eleme = soup.select('body > div:nth-child(17) > div.forminput')
elem5 = eleme[0].text.strip()
elemf = soup.select('body > div:nth-child(18) > div.forminput')
elem6 = elemf[0].text.strip()
elemg = soup.select('body > div.versioninfo')
elem7 = elemg[0].text.strip()
#creating dictionary object
dic = {}
dic['relay'] = relay
dic['FFSL'] = elem1
dic['FFCL'] = elem2
dic['FBFQFSL'] = elem3
dic['FBQFCL'] = elem4
dic['TQQ'] = elem5
dic['SQQ'] = elem6
dic['RV'] = elem7
print(dic)
context = {'dic' : dic}
return render(request, 'relchchk/relchck.html', context)
forms.py:
from django import forms
from django.forms import ModelForm, TextInput
from .models import Relay
class RelayForm(ModelForm):
class Meta:
model = Relay
fields = ['Name', 'Relay Version', ]
widgets = {'name' : TextInput(attrs={'class' : 'input',
'placeholder' : 'url'})}
models.py:
from django.db import models
class Relay(models.Model):
name = models.CharField(max_length=45)
def __str__(self):
return self.name
class Meta:
verbose_name_plural = 'Relays'
urls.py
from django.urls import path
from . import views
urlpatterns = [
path('', views.index),
]
The desired result would be to manually enter any of the target servers that could accumulate to all and save in database that exists (but not important now) and have the main page show all selected. I was moving along pretty well and thought this should be simple step and probably is, but I must be missing something. Any guidance would be much appreciated.
You haven't created an instance of your form. In your views.py file, it should look something like this.
if request.method="POST":
#code to handle the form
else:
form = RelayForm()
context = {
"form": form
}
You render the form like this:
<form method="POST">
{% csrf_token %}
{{ form.as_p }}
</form>
Read the docs to learn more about form handling and rendering.
I trying to propose to the users of my site to download a document in either pdf or odt version through radio buttons. How can I get and use the value of the radio button chosen by the user to serve the appropriate file. So far, I can only serve one at a time.
My current work:
models.py
class File(models.Model):
name = models.CharField(max_length=200)
pdf_version = models.FileField()
odt_version = models.FileField()
def __str__(self):
'''String name represents class File'''
return self.name
urls.py
path('files_page/', views.files_page, name='files_page'),
path('download_file/<int:file_id>/', views.download_file, name='download_file'),
views.py
def files_page(request):
files = File.objects.all()
context = {'files':files}
return render (request, 'walk/files_page.html', context)
def download_file(request, file_id):
#No post request; do nothing
if request.method != 'POST':
pass
else:
#fetch the file to download
#file = File.objects.get(id=file_id)
response = FileResponse(open('/home/me/Desktop/super/media_cdn/tog.pdf', 'rb'))
response['Content-Disposition'] = 'attachment; filename="tog.pdf"'
return response
template
{%block content%}
{%for file in files %}
<p>{{file.name}}</p>
<p>{{file.pdf_version}}</p>
<p>{{file.csv_version}}</p>
<form action="{%url 'walk:download_file' file.id%}" method="POST">
{%csrf_token%}
<input type="radio" name="format" value="pdf" checked> pdf
<input type="radio" name="format" value="csv"> csv
<button name="submit">download</button>
</form>
{%endfor%}
{%endblock content%}
Let's start with using forms. Yes, you use django forms in django instead re-implementing everything yourself.
Create forms.py:
from django import forms
FILE_FORMAT_CHOICES = [("csv", "Download PDF"), ("csv", "Download CSV")]
class FileFormatForm(forms.Form):
file_format = forms.ChoiceField(choices=FILE_FORMAT_CHOICES, widget=forms.RadioSelect())
Inside of the template used by files_page (just let django render the fields, don't do it yourself):
<form action="{%url 'walk:download_file' file.id%}" method="POST">
{% csrf_token %}
{{ form }}
<input type="submit" value="Download">
</form>
And finally adjust the views.py:
def files_page(request):
...
context = {
'files': files,
'form': FileFormatForm() # empty / without POST
}
...
def download_file(request, file_id):
assert request.method == 'POST', "users should only come here with POST now"
form = FileFormatForm(request.POST) # populate from POST
if form.data['file_format'] == 'pdf':
return "return PDF file response here"
else:
return "return CSV file response here"
Note: you don't use tab in Python. Use 4x whitespaces instead.
Another Note: Class Based Views to further reduce the amount of boilerplate.
I am trying to make a form to upload a file, but the file data is not being sent with the request. I'm manually navigating to my file and hitting submit. My FileRequired validator fails. (And if I don't include it the data field on form.scan_file is empty.)
Here's my form:
from flask_wtf import FlaskForm
from flask_wtf.file import FileField, FileAllowed, FileRequired
class ScanForm(FlaskForm):
scan_file = FileField(validators=[FileAllowed(['nii', 'nii.gz', 'zip']), FileRequired()])
Here's my views.py:
from flask import Blueprint, render_template, request, flash, redirect, url_for, session
from .models import Scan
from .forms import ScanForm
from .service import ScanService
from cookiecutter_mbam.utils import flash_errors
blueprint = Blueprint('scan', __name__, url_prefix='/scans', static_folder='../static')
#blueprint.route('/add', methods=['GET', 'POST'])
def add():
"""Add a scan."""
form = ScanForm(request.form)
if form.validate_on_submit():
f = form.scan_file.data
service = ScanService()
xnat_uri = service.upload(session['user_id'], session['curr_experiment'], f)
Scan.create(xnat_uri=xnat_uri)
flash('You successfully added a new scan.', 'success')
return redirect(url_for('experiment.experiments'))
else:
flash_errors(form)
return render_template('scans/upload.html',scan_form=form)
Here's my upload.html:
{% extends "layout.html" %}
{% block content %}
<form method="POST" action="{{ url_for('scan.add') }}" enctype="multipart/form-data">
{{ scan_form.csrf_token }}
<input type="file" name="file">
<input class="btn btn-primary" type="submit" value="Submit">
</form>
{% endblock %}
It doesn't look like I'm making the same mistake as this person. What am I doing wrong?
EDIT: Since posting, I have found this question, but on working through the offered solutions, none seem relevant to my situation.
EDIT 2: At one point, I printed request.files in the Werkzeug debugger and it was an empty dict. I can't reconstruct exactly what I did to get that result. Since then, I've inserted some print statements and in fact, request.files has my file object. So I have a way to retrieve my file. But I am supposed to be able to retrieve my file object at form.scan_file.data (see here). Right now this evaluates to None. More specifically, form.scan_file.has_file() evaluates to False. form.data evaluates to {'scan_file': None, 'csrf_token': <long-random-string> }
Even if I have another way of retrieving my file object, a consequence of this problem is that validation isn't working. My form doesn't pass the FileRequired() validation.
EDIT 3: With my new understanding of my problem, I see that it's similar to this question. However, it's at least apparently not duplicative because none of form = ScanForm(request.form), form = ScanForm(), or form = ScanForm(CombinedMultiDict((request.files, request.form))) make any difference to the behavior outlined in Edit 2.
First of all, check if your data gets posted on that route. Second, I think you don't need to pass request.form to ScanForm, you just need to instantiate it like:
def add():
"""Add a scan."""
form = ScanForm()
...
To check what gets posted with form, instead of
if form.validate_on_submit():
you can use, and print form.scan_file.data:
if form.is_submitted():
print(form.scan_file.data)
Lastly, you can render input file with
{{scan_form.scan_file }} or <input type="file" name="scan_file">
(name attribute of input element should be equal to "scan_file")
Here is my example:
Form:
class ArticleForm(FlaskForm):
article_image = FileField('Article_image', validators=[FileRequired()])
Form in template:
<form action="" method="post" enctype="multipart/form-data">
{{ article_form.csrf_token }}
{{ article_form.article_image }}
<input type="submit" value="submit"/>
</form>
Controller (saving file):
article_form = ArticleForm()
if article_form.validate_on_submit():
f = article_form.article_image.data
name = current_user.username + "__" + f.filename
name = secure_filename(name)
f.save(os.path.join("./static/article_images/", name))
I was stupid enough to forget to add enctype="multipart/form-data" to the form tag which caused the form.file.data to contain only the name of the file and not the whole life.
I am trying to build an admin action 'download_selected' which will download selected models. When the action is selected, I redirect to an intermediate page so that users can select a download format. When a user selects a download format and clicks on 'download', it downloads the file. But stays on the same intermediate page. How do I redirect it back to change form admin page? This redirection that I want is similar to django 'download selected file' default admin action. Thanks.
Here is my code.
admin.py
class SelectDownloadFormatForm(forms.Form):
DOWNLOAD_TYPE_CHOICES=[('csv','csv'),
('json', 'json'),
('xml','xml')]
_selected_action = forms.CharField(widget=forms.MultipleHiddenInput)
download_type = forms.ChoiceField(label=_('Select a Download type'), choices=DOWNLOAD_TYPE_CHOICES, widget=forms.RadioSelect())
def download_selected(self, request, queryset):
import csv
from django.http import HttpResponse, HttpResponseRedirect
import StringIO
form = None
if 'download' in request.POST:
form = self.SelectDownloadFormatForm(request.POST)
if form.is_valid():
dtype = form.cleaned_data['download_type']
print dtype
response = HttpResponse(content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename="export.csv"'
writer = csv.writer(response)
writer.writerow(['id', 'name', 'qid' ,'label', 'name', 'field'])
count = 0
for s in queryset:
questions_query = ParentModel.objects.filter(parent_form_id = s.id)
for q in questions_query:
writer.writerow([s.id, s.name, q.id, q.label, q.name, q.field])
count += 1
plural = ''
if count != 1:
plural = 's'
self.message_user(request, "Successfully downloaded %d survey response%s in %s format" % (count, plural, dtype))
return response
if not form:
form = self.SelectDownloadFormatForm(initial={'_selected_action': request.POST.getlist(admin.ACTION_CHECKBOX_NAME)})
return render(request,'admin/download_type.html', {'items': queryset,
'download_type_form': form,
})
download_selected.short_description = "Download selected forms"
download_type.html
{% extends "admin/base_site.html" %}
{% block content %}
<form action="" method="post">
{% csrf_token %}
{{ download_type_form }}
<p>Following survey will be downloaded with corresponding responses:</p>
<ul>{{ items|unordered_list }}</ul>
<input type="hidden" name="action" value="download_selected" />
<input type="submit" name="download" value="Download" />
</form>
{% endblock %}
I added an extra button to go back
Go Back
You'll need javascript for the redirect.
You can use jQuery File Download so you can do:
$.fileDownload('/url/to/download').done(function {
// redirect
})
Not sure if you can combine it with a form post.