AIM
Once the User enters search_text in the HashtagSearch FormView, the function get_tweets() will get from Twitter the locations associated with that hashtag.
Then, use get_or_create to save the search_text in the Hashtag db. Then, go through the txt file line by line and, assuming the regex requirements have been satisfied, add the line as a locations associated with the search_text in the db.
As a summary of the workflow:
A User enters search_text in the HashtagSearch FormView,
A function is run within HashtagSearch FormView that searches Twitter for tweets using the search_text as a hashtag ('the applicable tweets'),
Searches 'the applicable tweets' for whether the tweeter has a location saved on their profile. If so, save that location to a txt file,
Performs regex on the txt file to filter out 'non-genuine' locations,
Saves the locations to the locations object associated with the search_text in the Hashtag model.
Render results.html with the location object associated with the search_text in the Hashtag model.
ERROR
Request Method: POST
Django Version: 2.0
Exception Type: AttributeError
Exception Value: 'tuple' object has no attribute 'locations'
Exception Location: /mnt/project/mapping_twitter/views.py in get_tweets, line 87
Python Executable: /mnt/data/.python-3.6/bin/python
Python Version: 3.6.5
Python Path:
['/mnt/project',
'/mnt/data/.python-3.6/lib/python36.zip',
'/mnt/data/.python-3.6/lib/python3.6',
'/mnt/data/.python-3.6/lib/python3.6/lib-dynload',
'/usr/local/lib/python3.6',
'/mnt/data/.python-3.6/lib/python3.6/site-packages']
CODE
views.py
def get_tweets(self, form):
""" Get tweets from the Twitter API and store them to the db """
consumer_key = '...'
consumer_secret = '...'
access_token = '...'
access_secret = '...'
t = Twython(
app_key=consumer_key,
app_secret=consumer_secret,
oauth_token=access_token,
oauth_token_secret=access_secret,
)
# set to the text entered by User in Form
search_filter = self.request.POST.get('search_text')
# set the filter for hashtag and quantity
search = t.search(q='#{}'.format(search_filter), count=50)
tweets = search['statuses']
f = open('tweet_locations.txt', 'w+')
for tweet in tweets:
if tweet['user']['location']:
tweet_location = tweet['user']['location']
f.write("{}\n".format(tweet_location))
f.close()
data = open('tweet_locations.txt', 'r')
# Regex out 'unmappable' locations
valid_ex = re.compile(r'[A-Z][a-z]+, [A-Za-z]+')
for line in data:
valid_tweet_location = str(valid_ex.search(line))
if valid_tweet_location:
# ISSUE: save the locations_list to the Hashtag model as the locations associated with the search_text entered
tweet_object = Hashtag.objects.get_or_create(search_text=search_filter)
# necessary as locations is a M2M object
tweet_object.locations.create(valid_tweet_location)
data.close()
The get_or_create method returns the tuple (obj, created) instead of only the object. This way you can check whether the object is retrieved or created.
Simply unpack the tuple as follows:
tweet_object, created = Hashtag.objects.get_or_create(search_text=search_filter)
tweet_object.locations.create(valid_tweet_location)
You posted far too much code here - the traceback showed you where the error was happening, you should have just posted the get_tweets method.
The error is indeed happening there. This is because a tuple is what is returned by get_or_create - that is, it returns the object and a boolean showing if it was a create. Since your don't care about that, you can just assign it to a variable that is ignored:
tweet_object, _ = Hashtag.objects.get_or_create(search_text=search_filter)
Related
I am looking to attach a file to an email which includes all the content a user inputs from a contact form. I currently refer a PDF which records their inputs, and I attach that PDF from a file destination. However, I do not know how to attach additional files which the user provides on the contact form. In this case, this is represented by "msg.attach_file(upload_file)." My thoughts are:
Have the file be uploaded to a destination; however, it needs to renamed to a uniform name each time so I can refer to it during the attachment process (msg.attach_file).
Figure out a way to use request.FILES to attach it immediately without having to worry about its file name or upload destination (I am not sure if msg.attach_file is a valid command for this method).
Is there a right way to perform this action? I am attempting to perform method 2 with my views.py file which refers to my forms.py file, but it is giving me an error.
Views.py
def quote_req(request):
submitted = False
if request.method == 'POST':
form = QuoteForm(request.POST, request.FILES)
company = request.POST['company']
contact_person = request.POST['contact_person']
upload_file = request.FILES['upload_file']
description = 'You have received a sales contact form'
if form.is_valid():
data_dict = {
'company_': str(company),
'contact_person_': str(contact_person),
}
write_fillable_pdf(INVOICE_TEMPLATE_PATH, INVOICE_OUTPUT_PATH, data_dict)
form.save()
# assert false
msg = EmailMessage('Contact Form', description, settings.EMAIL_HOST_USER, ['sample#mail.com'])
msg.attach_file('/uploads/file.pdf')
msg.attach_file(upload_file)
msg.send(fail_silently=False)
return HttpResponseRedirect('/quote/?submitted=True')
else:
form = QuoteForm()
if 'submitted' in request.GET:
submitted = True
Error Log
TypeError at /quote/
expected str, bytes or os.PathLike object, not InMemoryUploadedFile
Request Method: POST
Request URL: http://www.mytestingwebsitesample.com/quote/
Django Version: 2.1.3
Exception Type: TypeError
Exception Value:
expected str, bytes or os.PathLike object, not InMemoryUploadedFile
Can you try the following? Since InMemoryUploadedFile doesn't work, might have to process it first
upload_file = request.FILES['upload_file']
content = upload_file.read()
attachment = (upload_file.name, content, 'application/pdf')
# . . .
msg.attach(attachment)
upload_file.read() will return bytes. You might want to try using attach instead of attach_file. attach_file requires the file to be saved to your filesystem, while attach can take data. However, I believe that with attach, you should be able to use request.FILES['upload_file'] directly.
https://docs.djangoproject.com/en/2.2/topics/email/#emailmessage-objects
I have resolved my issue by employing a storage.py file that overwrites files with the same name; in my case, I am uploading each file, renaming it to a uniform name, and then having the storage file overwrite it later on rather than Django adding an extension to a file name with the same title.
I am creating a view function to edit the database using a wtform, I want to populate the form with information held on the database supplied by a differente form, My problem is the query that provides the details
I have read the manual https://wtforms.readthedocs.io/en/stable/crash_course.html
and the following question Python Flask-WTF - use same form template for add and edit operations
but my query does not seem to supply the correct format of data
datatbase model:
class Sensors(db.Model):
id = db.Column(db.Integer, primary_key=True)
sensorID = db.Column(db.String, unique=True)
name = db.Column(db.String(30), unique=True)
form model:
class AddSensorForm(FlaskForm):
sensorID = StringField('sensorID', validators=[DataRequired()])
sensorName = StringField('sensorName', validators=[DataRequired()])
submit = SubmitField('Register')
view function:
#bp.route('/sensors/editsensor/<int:id>', methods=('GET', 'POST'))
#login_required
def editsensor(id):
edit = [(s.sensorID, s.sensorName) for s in db.session.\
query(Sensors).filter_by(id=id).all()]
form = AddSensorForm(obj=edit)
form.populate_obj(edit)
if form.validate_on_submit():
sensors = Sensors(sensorID=form.sensorID.data, sensorName=form.sensorNa$
db.session.add(sensors)
db.session.commit()
shell code for query:
from homeHeating import db
from homeHeating import create_app
app = create_app()
app.app_context().push()
def editsensor(id):
edit = [(s.sensorID, s.sensorName) for s in db.session.query(Sensors).filter_by(id=id).all()]
print(edit)
editsensor(1)
[('28-0000045680fde', 'Boiler input')]
I expect that the two form fields will be populated with the in formation concerning the sensor called by its 'id'
but I get this error
File "/home/pi/heating/homeHeating/sensors/sensors.py", line 60, in
editsensor
form.populate_obj(edit)
File "/home/pi/heating/venv/lib/python3.7/site-
packages/wtforms/form.py", line 96, in populate_obj
Open an interactive python shell in this
framefield.populate_obj(obj, name)
File "/home/pi/heating/venv/lib/python3.7/site-
packages/wtforms/fields/core.py", line 330, in populate_obj
setattr(obj, name, self.data)
AttributeError: 'list' object has no attribute 'sensorID'
The error indicates that it wants 2 parts for each field "framefield.populate_obj(obj, name) mine provides only one the column data but not the column name, "sensorID"
If i hash # out the line "edit = ..." then there are no error messages and the form is returned but the fields are empty. So I want the form to be returned with the information in the database, filled in so that i can modify the name or the sensorID and then update the database.
I hope that this is clear
Warm regards
paul.
ps I have followed the instruction so the ERROR statement is only the part after "field.populate_by".
You are trying to pass a 1-item list to your form.
Typically, when you are selecting a single record based on the primary key of your model, use Query.get() instead of Query.filter(...).all()[0].
Furthermore, you need to pass the request data to your form to validate it on submit, and also to pre-fill the fields when the form reports errors.
Form.validate_on_submit will be return True only if your request method is POST and your form passes validation; it is the step where your form tells you "the user provided syntactically correct information, now you may do more checks and I may populate an existing object with the data provided to me".
You also need to handle cases where the form is being displayed to the user for the first time.
#bp.route('/sensors/editsensor/<int:id>', methods=('GET', 'POST'))
#login_required
def editsensor(id):
obj = Sensors.query.get(id) or Sensors()
form = AddSensorForm(request.form, obj=obj)
if form.validate_on_submit():
form.populate_obj(obj)
db.session.add(obj)
db.session.commit()
# return response or redirect here
return redirect(...)
else:
# either the form has errors, or the user is displaying it for
# the first time (GET)
return render_template('sensors.html', form=form, obj=obj)
I have a web app that needs to do the following:
Present a form to request a client side file for CSV import.
Validate the data in the CSV file or ask for another filename.
At one point, I was doing the CSV data validation in the view, after the form.is_valid() call from getting the filename (i.e. I have the imported CSV file into memory in a dictionary using csv.DictReader). After running into problems trying to pass errors back to the original form, I'm now trying to validate the CONTENTS of the CSV file in the form's clean() method.
I'm currently stumped on how to access the in memory file from clean() as the request.FILES object isn't valid. Note that I have no problems presenting the form to the client browser and then manipulating the resulting CSV file. The real issue is how to validate the contents of the CSV file - if I assume the data format is correct I can import it to my models. I'll post my forms.py file to show where I currently am after moving the code from the view to the form:
forms.py
import csv
from django import forms
from io import TextIOWrapper
class CSVImportForm(forms.Form):
filename = forms.FileField(label='Select a CSV file to import:',)
def clean(self):
cleaned_data = super(CSVImportForm, self).clean()
f = TextIOWrapper(request.FILES['filename'].file, encoding='ASCII')
result_csvlist = csv.DictReader(f)
# first line (only) contains additional information about the event
# let's validate that against its form definition
event_info = next(result_csvlist)
f_eventinfo = ResultsForm(event_info)
if not f_eventinfo.is_valid():
raise forms.ValidationError("Error validating 1st line of data (after header) in CSV")
return cleaned_data
class ResultsForm(forms.Form):
RESULT_CHOICES = (('Won', 'Won'),
('Lost', 'Lost'),
('Tie', 'Tie'),
('WonByForfeit', 'WonByForfeit'),
('LostByForfeit', 'LostByForfeit'))
Team1 = forms.CharField(min_length=10, max_length=11)
Team2 = forms.CharField(min_length=10, max_length=11)
Result = forms.ChoiceField(choices=RESULT_CHOICES)
Score = forms.CharField()
Event = forms.CharField()
Venue = forms.CharField()
Date = forms.DateField()
Div = forms.CharField()
Website = forms.URLField(required=False)
TD = forms.CharField(required=False)
I'd love input on what's the "best" method to validate the contents of an uploaded CSV file and present that information back to the client browser!
I assume that when you want to access that file is in this line inside the clean method:
f = TextIOWrapper(request.FILES['filename'].file, encoding='ASCII')
You can't use that line because request doesn't exist but you can access your form's fields so you can try this instead:
f = TextIOWrapper(self.cleaned_data.get('filename'), encoding='ASCII')
Since you have done super.clean in the first line in your method, that should work. Then, if you want to add custom error message to you form you can do it like this:
from django.forms.util import ErrorList
errors = form._errors.setdefault("filename", ErrorList())
errors.append(u"CSV file incorrect")
Hope it helps.
Hi all as i in learning stage of django so support me.
I have to generate pdf reports in django.I want that the details should be picked from the database and displayed in the pdf document.i am using report lab.
Now have a look at the code
def pdf_view(request):
response = HttpResponse(mimetype='application/pdf')
response['Content-Disposition'] = 'attachment; filename=hello.pdf'
p = canvas.Canvas(response)
details = Data.objects.all()
print details
p.drawString(20, 800, details)
p.drawString(30, 700, "I am a Python Django Professional.")
p.showPage()
p.save()
return response
now as a learning example i have made two fields in models
class Data(models.Model):
first_name = models.CharField(max_length =100,blank=True,null=True)
last_name = models.CharField(max_length =100,blank=True,null=True)
def __unicode__(self):
return self.first_name
and i want that in the pdf document it should display the name s whatever i fill through admin but it is giving me error
'Data' object has no attribute 'decode'
Request Method: GET
Request URL: http://localhost:8000/view_pdf/
Django Version: 1.3
Exception Type: AttributeError
Exception Value:
i want to pik the details from the database and display in the pdf document
'Data' object has no attribute 'decode'
It would have helped if you'd posted the actual traceback.
However I expect the issue is this line:
p.drawString(20, 800, details)
Details is a queryset, that is a list-like container of model instances. It's not a string, and neither does it contain a string. Maybe you want something like:
detail_string = u", ".join(unicode(obj) for obj in details)
which calls the __unicode__ method on every object in your queryset, and joins the resulting list with commas.
I have an extended UserProfile for registering new users. My user_created function connects to signals sent upon registering basic User instance and creates new UserProfile with extended fields from my form. Here's the code :
from registration.signals import user_registered
from accounts.forms import ExtendedRegistrationForm
import accounts
from accounts.models import UserProfile
def user_created(sender, user, request, **kwargs):
form = ExtendedRegistrationForm(request.POST, request.FILES)
data = UserProfile(user=user)
data.is_active = False
data.first_name = form.data['first_name']
data.last_name = form.data['last_name']
data.pid = form.data['pid']
data.image = form.data['image']
data.street = form.data['street']
data.number = form.data['number']
data.code = form.data['code']
data.city = form.data['city']
data.save()
user_registered.connect(user_created)
Problem is that on this form I have an image field for avatar. As you can see from the code, I'm getting data from form's data list. But apparently imageField does not send it's data with POST request(as I'm getting MultiValueDictKeyError at /user/register/, Key 'image' not found in <QueryDict...) so I can't get it from data[] .
alt text http://img38.imageshack.us/img38/3839/61289917.png
If the usual variables are inside 'data', where should I look for files ? Or is the problem more complicated ? Strange thing is that my form doesn't have attribute cleaned_data... I was using dmitko's method here : http://dmitko.ru/?p=546&lang=en . My :
forms : http://paste.pocoo.org/show/230754/
models : http://paste.pocoo.org/show/230755/
You should be validating the form before using it, which will create the "cleaned_data" attribute you're used to. Just check form.is_valid() and the "cleaned_data" attribute will be available, and should contain the file.
The form's "data" attribute is going to be whatever you passed in as its first initalization argument (in this case, request.POST), and files are stored separately in the "files" attribute (whatever you pass in as the second argument, in this case, request.FILES). You don't want to be accessing the form's "data" or "files" attributes directly, as, if you do, you're just reading data straight from the request and not getting any benefit from using forms.
Are you sure the <form enctype="..."> attribute is set to multipart/form-data ? Otherwise the browser is not able to upload the file data.