My flash-pyamf-gae works nice.
Now I would like to create a classic Django form by following the google tutorial : http://code.google.com/appengine/articles/djangoforms.html
I did but when I post the data entered in my form i get the following message from pyamf :
Malformed stream (amfVersion=110)
400 Bad Request The request body was
unable to be successfully decoded.
Traceback:
Traceback (most recent call last):
File
"C:\Users\Giil\Documents\dev\gae\moviesbuilder\pyamf\remoting\gateway\google.py",
line 79, in post
logger=self.logger, timezone_offset=timezone_offset)
File
"C:\Users\Giil\Documents\dev\gae\moviesbuilder\pyamf\remoting_init_.py",
line 640, in decode
msg.amfVersion) DecodeError: Malformed stream (amfVersion=110)Malformed stream (amfVersion=110)
Now that make sens to me because what I send from my form is not amf. How can I deal with this ?
Note : I have the feeling that the problems come from my app.yaml
I have no special handler to tell the application to process this form differently...Malformed stream (amfVersion=110)
I solved the problem my own way :
My Form (the post is directed towards another function and not just "/" as in google example) :
class Projects(webapp.RequestHandler):
def get(self):
self.response.out.write('<html><body>'
'<form method="POST" '
'action="/ProjectsPage">'
'<table>')
self.response.out.write(ProjectForm())
self.response.out.write('</table>'
'<input type="submit">'
'</form></body></html>')
And then what I need to write to the dataStore and display the list :
class ProjectsPage(webapp.RequestHandler):
#getting data and saving
def post(self):
data = ProjectForm(data=self.request.POST)
if data.is_valid():
# Save the data, and redirect to the view page
entity = data.save(commit=False)
entity.added_by = users.get_current_user()
entity.put()
self.redirect('/projects.html')
else:
# Reprint the form
self.response.out.write('<html><body>'
'<form method="POST" '
'action="/">'
'<table>')
self.response.out.write(data)
self.response.out.write('</table>'
'<input type="submit">'
'</form></body></html>')
#display list of projects
def get(self):
query = db.GqlQuery("SELECT * FROM Project WHERE added_by=:1 ORDER BY name",users.get_current_user())
template_values = {
'projects': query,
}
path = os.path.join(os.path.dirname(__file__), 'templates/projects.html')
self.response.out.write(template.render(path, template_values))
Related
How can I download a large CSV file that shows me a 502 bad gateway error?
I get this solution I added in below.
Actually, in this, we use streaming references. In this concept for example we download a movie it's will download in the browser and show status when complete this will give the option to show in a folder same as that CSV file download completely this will show us.
There is one solution for resolving this error to increase nginx time but this is will affect cost so better way to use Django streaming. streaming is like an example when we add a movie for download it's downloading on the browser. This concept is used in Django streaming.
Write View for this in Django.
views.py
from django.http import StreamingHttpResponse
503_ERROR = 'something went wrong.'
DASHBOARD_URL = 'path'
def get_headers():
return ['field1', 'field2', 'field3']
def get_data(item):
return {
'field1': item.field1,
'field2': item.field2,
'field3': item.field3,
}
class CSVBuffer(object):
def write(self, value):
return value
class Streaming_CSV(generic.View):
model = Model_name
def get(self, request, *args, **kwargs):
try:
queryset = self.model.objects.filter(is_draft=False)
response = StreamingHttpResponse(streaming_content=(iter_items(queryset, CSVBuffer())), content_type='text/csv', )
file_name = 'Experience_data_%s' % (str(datetime.datetime.now()))
response['Content-Disposition'] = 'attachment;filename=%s.csv' % (file_name)
except Exception as e:
print(e)
messages.error(request, ERROR_503)
return redirect(DASHBOARD_URL)
return response
urls.py
path('streaming-csv/',views.Streaming_CSV.as_view(),name = 'streaming-csv')
For reference use the below links.
https://docs.djangoproject.com/en/4.0/howto/outputting-csv/#streaming-large-csv-files
GIT.
https://gist.github.com/niuware/ba19bbc0169039e89326e1599dba3a87
GIT
Adding rows manually to StreamingHttpResponse (Django)
I am using the tmdb api to return movie info as well as images.
Steps below of Get logic
Api request is made which provides movie info as well as "backdrop_path"
I then use this path to make another request for the jpg related to that movie.
Blocker
I'm unable to then output that jpg. It currently returns a url path as below.
Views.py
from django.shortcuts import render
from django.views.generic import TemplateView
import requests
import urllib
# Create your views here.
def index(request):
# Query API with user input
if 'movie' in request.GET:
api_key = 'api'
id = request.GET['movie']
url = 'https://api.themoviedb.org/3/search/movie?api_key={}&language=en-US&query={}&include_adult=false'
response = requests.get(url.format(api_key,id))
# successful request
if response.status_code == 200:
# Parse json output for key value pairs
tmdb = response.json()
# save image jpg
backdrop_path = tmdb['results'][0]['backdrop_path']
url = 'https://image.tmdb.org/t/p/original/{}'
gg = urllib.request.urlretrieve(url.format(backdrop_path), 'test.jpg')
context = {
'title': tmdb['results'][0]['original_title'],
'overview': tmdb['results'][0]['overview'],
'release_date': tmdb['results'][0]['release_date'],
'vote_average': tmdb['results'][0]['vote_average'],
'vote_count': tmdb['results'][0]['vote_count'],
'backdrop_path' : tmdb['results'][0]['backdrop_path'],
'jpg' : gg
}
return render(request, 'home.html', {'context': context})
else: # returns homepage if invalid request
return render(request, 'home.html')
else: # Homepage without GET request
return render(request, 'home.html')
urlretrieve doesn't return the image ready to be used, instead it returns a tuple with the local name of the file and the headers (as an HTTPMessage object as you can see in your example), which is what you're seeing.
However, I don't think returning a file in your response is ideal, nor it would work in your scenario. Since you seem to be using templates, what I would do is return the image url and use it in an image HTML tag, like this <img src="{{ jpg }}"/>
So I need to parse a set of pages from an authenticated user's point of view and get some values from there...
So I need scrapy to authenticate itself before starting to parse.
In order to do that I understand that I would need to use InitSpider and init_request.
My problem is that in order to authenticate, using a FormRequest, I must send to the form a certain value that is some kind of a session id (form_key) that is generated automatically.
So first, scrapy must access the self.login_page, get the value of form_key (like in the xpath bellow), then post the LOGIN FORM to self.login_post
The 2 URL's are described bellow in the script section
The HTML containing my form key looks like this:
<input name="form_key" type="hidden" value="2pos7YUekQz6Y9rD">
So in order to obtain my form_key value, I did this (updated scrapy script):
class MyScript(InitSpider):
name = "logged-in"
allowed_domains = ["example.com"]
login_post = 'https://www.example.com/customer/account/loginPost/'
def init_request(self):
""" Called before crawler starts """
return Request(url=self.login_post, callback=self.login, dont_filter = True)
def login(self, response):
""" Generate login request """
login_page = 'https://www.example.com/customer/account/login/'
myformkey= self.pax_key(login_page)
return FormRequest.from_response(response,
formxpath="//div[#class='account-login']/form[#id='login-form']",
formdata={'login[username]':'user',
'login[password]':'pass',
'form_key':myformkey
},
callback=self.check_login_response)
def check_login_response(self,response):
""" Check the response returned by login request to see if we are logged in """
logging.log(logging.INFO,'... checking login response ...')
if "Invalid login or password." in response.body:
logging.log(logging.INFO,'... BAD LOGIN ...')
else:
logging.log(logging.INFO, 'GOOD LOGIN... initialize:'+response.url)
self.initialized()
def pax_key(self, url):
data = urllib.urlopen(url).read()
hxs = Selector(text=data) #HtmlXPath
lista = hxs.xpath('//input[#type="hidden"][#name="form_key"]/#value').extract()
logging.log(logging.INFO,lista)
return lista
def parse(self, response):
...
So the problem now is...:
INFO: do login...
2015-12-04 21:57:18 [root] INFO: [u'yFVS9fR8Jteuo6Co']
2015-12-04 21:57:19 [root] INFO: http://www.example.com/enable-cookies
2015-12-04 21:57:19 [scrapy] ERROR: Spider error processing <GET http://www.example.com/enable-cookies> (referer: None)
Traceback (most recent call last):
File "c:\python27\lib\site-packages\twisted\internet\defer.py", line 588, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "d:\Calin\WorkSpace\example-spider\products\spiders\logged-in.py", line 49, in login
callback=self.check_login_response)
File "c:\python27\lib\site-packages\scrapy\http\request\form.py", line 37, in from_response
form = _get_form(response, formname, formnumber, formxpath)
File "c:\python27\lib\site-packages\scrapy\http\request\form.py", line 81, in _get_form
raise ValueError('No <form> element found with %s' % formxpath)
ValueError: No <form> element found with //div[#class='account-login']/form[#id='login-form']
So first of all... I have no idea why I get redirected towards the enable-cookies page...
Spider error processing <GET http://www.example.com/enable-cookies>
I do not tell it to go there... at all.
I assume that is why there is no form there to xpath it... so I get the error in the end.
ValueError: No <form> element found with
//div[#class='account-login']/form[#id='login-form']
I want to upload files from Backbone to Django File upload system.
First of all I've follow the https://stackoverflow.com/a/10916733/1590377 explanation. I've do a FileModel and with the above indication I have a model with this information:
attributes: Object
data: "data:image/png;base64,iVBORw ..."
file: "image2012-06-12 13:36:45.png"
now I save the model to the URL where I have the upload view in django like this:
def upload_file_64(request):
if request.method == 'POST':
file = cStringIO.StringIO(base64.b64decode(request.POST['data']))
#method to save the file
response_data={"result":"ok"}
return HttpResponse(simplejson.dumps(response_data), mimetype='application/json')
else:
response_data={"success": "No a post request"}
return HttpResponse(simplejson.dumps(response_data), mimetype='application/json')
but the response that the django sistem give me is:
"MultiValueDictKeyError at /api/upload64/↵'Key \'data\' not found in <QueryDict: {u\'base64,iVBORw0KG....
The POST http request is:
POST:
base64,iVBORw0KG ..."} = u''
{"file":"Captura de pantalla de 2012-06-12 13:36:45.png","data":"data:image/png = u''
How I can fix this so that I can upload a file to django. I use a multi-part method to upload files from another platforms how android but with backbone I can't upload a file.
Can someone help me eith this problem?
Thanks!!
I've coded another solution. I've used a jquery upload pluging to upload the file, and get the response.
The plugin is : http://lagoscript.org/jquery/upload/demo?locale=en and the code that I used in my backbone view is:
events : {
'change #file1' : 'upload'
},
upload : function(){
$('#file1').upload('http://192.168.0.195/api/upload/', function(res) {
console.log(res)
//now I use the res to create a model :)
}, 'html');
},
What is the easist way to include username, first and last name and e-amil in django Traceback error.
I know that the way is create a custom error report:
Create a new class that innherit from django.views.debug.SafeExceptionReporterFilter
Set DEFAULT_EXCEPTION_REPORTER_FILTER
But, what method a should overwrite to receive traceback with also this information?
I would like that my treceback look likes:
Traceback (most recent call last):
File "/usr...o/core/handlers/base.py", line 89, in get_response
response = middleware_method(request)
File "/.../g...ap/utils/middleware.py", line 23,...
if elapsedTime.min > 15:
TypeError: can't compare datetime.timedelta to int
Logged user information:
User: pepito
name: Pepito Grillo
e-mail: grillo#peppeto.com
I did it using Custom Middleware. I'm not sure this is the best answer, but it is how I solved it for my project.
settings.py:
MIDDLEWARE_CLASSES = (
...
'utilities.custom_middleware.CustomMiddleware',
...
)
utilities/custom_middleware.py:
from utilities.request import AddRequestDetails
class CustomMiddleware(object):
"""
Adds user details to request context during request processing, so that they
show up in the error emails. Add to settings.MIDDLEWARE_CLASSES and keep it
outermost(i.e. on top if possible). This allows it to catch exceptions in
other middlewares as well.
"""
def process_exception(self, request, exception):
"""
Process the request to add some variables to it.
"""
# Add other details about the user to the META CGI variables.
try:
if request.user.is_authenticated():
AddRequestDetails(request)
request.META['AUTH_VIEW_ARGS'] = str(view_args)
request.META['AUTH_VIEW_CALL'] = str(view_func)
request.META['AUTH_VIEW_KWARGS'] = str(view_kwargs)
except:
pass
utilities/request.py:
def AddRequestDetails(request):
"""
Adds details about the user to the request, so any traceback will include the
details. Good for troubleshooting; this will be included in the email sent to admins
on error.
"""
if request.user.is_anonymous():
request.META['AUTH_NAME'] = "Anonymous User"
request.META['AUTH_USER'] = "Anonymous User"
request.META['AUTH_USER_EMAIL'] = ""
request.META['AUTH_USER_ID'] = 0
request.META['AUTH_USER_IS_ACTIVE'] = False
request.META['AUTH_USER_IS_SUPERUSER'] = False
request.META['AUTH_USER_IS_STAFF'] = False
request.META['AUTH_USER_LAST_LOGIN'] = ""
else:
request.META['AUTH_NAME'] = str(request.user.first_name) + " " + str(request.user.last_name)
request.META['AUTH_USER'] = str(request.user.username)
request.META['AUTH_USER_EMAIL'] = str(request.user.email)
request.META['AUTH_USER_ID'] = str(request.user.id)
request.META['AUTH_USER_IS_ACTIVE'] = str(request.user.is_active)
request.META['AUTH_USER_IS_SUPERUSER'] = str(request.user.is_superuser)
request.META['AUTH_USER_IS_STAFF'] = str(request.user.is_staff)
request.META['AUTH_USER_LAST_LOGIN'] = str(request.user.last_login)
My trivial solution (works in django 1.5)
settings.py:
MIDDLEWARE_CLASSES = (
...
'utilities.custom_middleware.UserTracebackMiddleware',
...
)
custom_middleware.py:
class UserTracebackMiddleware(object):
"""
Adds user to request context during request processing, so that they
show up in the error emails.
"""
def process_exception(self, request, exception):
if request.user.is_authenticated():
request.META['AUTH_USER'] = unicode(request.user.username)
else:
request.META['AUTH_USER'] = "Anonymous User"
hope it helps