I was trying to upload a file to Google Cloud Storage with the help of a django application. I wrote this create_gcs_file function that gets called in my view:
import cloudstorage as gcs
from google.appengine.ext import blobstore
def create_gcs_file(filename, data):
with gcs.open(filename, 'w') as f:
f.write(data)
blobstore_filename = '/gs' + filename
return blobstore.create_gs_key(blobstore_filename)
I call this function in a view and pass it a filename, and some file.read() data as parameters. Here is the view code I have written that makes use of this.
But when I upload the file I get this error:
Expect status [201] from Google Storage. But got status 302
The debug page shows that the error occurs at this line:
with gcs.open(filename, 'w') as f:
P.S: I get this error when running my app with the Google App Engine SDK 1.8.5 locally.
I also had gone though this same situation, In my case the real problem was in model field.
Actual field was like:
photo = models.ImageField(upload_to='../../static/cars')
And later I change the upload to option as;
photo = models.ImageField(upload_to='static/cars')
Now my problem solved and it is working fine now.
Related
i have post request to my python code. in the json body I need to send 1 parameter. and then I need to upload csv file.
I have 2 questions: 1. how to upload the csv from postman side 2. how to get it in my python code.
attached my post request and my python code.
post request screen shot
my code in python.
#cherrypy.tools.json_in()
#cherrypy.tools.json_out()
#cherrypy.tools.accept(media='application/json')
def POST(self):
body = cherrypy.request.json
If you want to use JSON to upload data. You need to convert csv file in to base64 string before you post your data. Because JSON format does not support file.
If you just want to get your data, you can select "form-data" in postman.
Code for 'form-data':
import cherrypy
#cherrypy.expose
#cherrypy.tools.json_out()
def uploadcsv(self, img=None, other=None):
print(img)
print(other)
return 'ok'
cherrypy.quickstart(HelloWorld())
Image of postman setting:
I have a Flask method that gets url and returns the zip file's content. The content should be returned to the axios get method. Flask code:
zip_file = requests.get(url)
if zip_file.ok:
return {
'file': zip_file.content
}
But it doesn't work, it fires exception 'TypeError(repr(o) + " is not JSON serializable").
How to fix it? I saw some solutions with coding-decoding but I'm not sure how to use it, especially after the axios request gets the response (I can't use the direct link because of security reasons).
I want to develop a simple web server using python to handle some simple http request. I have learn how to response the request, such as transferring html pages or transferring some other file. When I transfer a image file, a client use a browser to get the file, the url is like below:
http://114.212.82.104:8080/1.png
I set 'Content-Type = application/x-png'. But the browser directly download the file, and can not display in the browser. Not like the image below
https://www.baidu.com/img/bd_logo1.png
it can display in the browser. How to display the image in the browser?
Can someone help me?
and i know i can encode the image file into html page to fix it. code like below:
class RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_GET(self):
path = os.getcwd()+ self.path
if os.path.isfile(path):
with open(path,'rb') as fileTrans:
content = fileTrans.read().encode('base64').replace('\n','')
#self.sendContent(200, content)
self.send_response(200)
page = "<p>\"fef\"</p><img src=\"data:image/jpg;base64,{0}\"/>"
contentPage = page.format(content)
self.send_header('Content-Type', 'text/html')
self.send_header("Content-Length", str(len(contentPage)))
self.end_headers()
self.wfile.write(contentPage)
else:
self.sendContent(404,"file do not exists")
But I know there must be another way, i see the source code of URL(https://www.baidu.com/)
it just use
<img hidefocus="true" src="//www.baidu.com/img/bd_logo1.png" width="270" height="129"></div><a href="/" id="result_logo" onmousedown="return c({'fm':'tab','tab':'logo'})">
different from my page:
<p>"fef"</p><img src="data:image/jpg;base64,iVBORw0KGgoAAAANSUhEUgAAAcIAAAJYCAIAAAB+b3GqAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAvppVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuNi1jMTMyIDc5LjE1OTI4NCwgMjAxNi8wNC8xOS0xMzoxMzo0MCAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wTU09Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9tbS8iIHhtbG5zOnN0UmVmPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvc1R5cGUvUmVzb3VyY2VSZWYjIiB4bWxuczp4bXA9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8iIHhtcE1NOkRvY3VtZW50SUQ9InhtcC5kaWQ6ODg3NkM1Njg1MzVFMTFFNkE0NkJFNTEzMUFCNzc4RTMiIHhtcE1NOkluc3RhbmNlSUQ9InhtcC5paWQ6ODg3NkM1Njc1MzVFMTFFNkE0NkJFNTEzMUFCNzc4RTMiIHhtcDpDcmVhdG9yVG9vbD0iOC4xLjMiPiA8eG1wTU06RG......
OK,I think I have solved this problem.
Just set the attribute - 'Content-Type' of header as 'image/png' instead of 'application/x-png'.
How can I test a scrapy spider against online data.
I now from this post that it is possible to test a spider against offline data.
My target is to check if my spider still extracts the right data from a page, or if the page changed. I extract the data via XPath and sometimes the page receives and update and my scraper is no longer working. I would love to have the test as close to my code as possible, eg. using the spider and scrapy setup and just hook into the parse method.
Referring to the link you provided, you could try this method for online testing which I used for my problem which was similar to yours. All you have to do is instead of reading the requests from a file you can use the Requests library to fetch the live webpage for you and compose a scrapy response from the response you get from Requests like below
import os
import requests
from scrapy.http import Response, Request
def online_response_from_url (url=None):
if not url:
url = 'http://www.example.com'
request = Request(url=url)
oresp = requests.get(url)
response = TextResponse(url=url, request=request,
body=oresp.text, encoding = 'utf-8')
return response
I'm testing a post request which contains an image, after the post request if succesfull i receive a url like
http://testbed.example.com/_ah/upload/agx0ZXN0YmVkLXRlc3RyGwsSFV9fQmxvYlVwbG9hZFNlc3Npb25fXxgDDA
as i already tried and checked in stackoverflow, this wont work to try to upload the image
i have a handler in a route like "/upload/image"
and the code looks like:
class UploadScreenshot(webapp2.RequestHandler, blobstore_handlers.BlobstoreUploadHandler):
def post(self):
try:
upload_screenshot = self.get_uploads('file')
upload_url = self.request.get('upload_url')
fbkey = self.request.get('fbkey')
screenshotKey = upload_screenshot[0].key()
feedback = N.FeedbackModel.query(N.FeedbackModel.fbkey == fbkey)
feedback.screenshotBlobID = screenshotKey
feedback.put()
except:
self.error(400)
what could i do to upload to the Blobstore, i have my app in appspot also but i wanna test this before deploying
thanks
Sounds like you got things a bit backwards. By the time your UploadScreenshot handler is called the blob should be already uploaded and your handler can access it using the blob key (which you have in screenshotKey). You can't (re)use the upload url at this point (it should have been already used when the user submited the upload form).
You may want to revisit the blob upload procedure/example.
BTW, this can be fully tested on the development server, the upload URL you get will be a localhost one and the blob is stored on localhost as well.