I have a Flask method that gets url and returns the zip file's content. The content should be returned to the axios get method. Flask code:
zip_file = requests.get(url)
if zip_file.ok:
return {
'file': zip_file.content
}
But it doesn't work, it fires exception 'TypeError(repr(o) + " is not JSON serializable").
How to fix it? I saw some solutions with coding-decoding but I'm not sure how to use it, especially after the axios request gets the response (I can't use the direct link because of security reasons).
Related
In tests I am making an api call, which returns 400. It is expected, but I can't find a way to debug this. Does django keep logs in a file somewhere? Or can I enable showing logs please?
res = self.client.post(self.url, data=payload, format='json')
print(res)
// <Response status_code=400, "application/json">
I knot something went wrong but how do I debug the server?
Thanks
You can use response.content to view the final content/error messages that will be rendered on the web-page, as bytestring. docs
>>> response = c.get('/foo/bar/')
>>> response.content
b'<!DOCTYPE html...
If you are returning a json response(which you probably are if using rest framework), you can use response.json() to parse the json. docs
>>> response = client.get('/foo/')
>>> response.json()['name']
'Arthur'
Note: If the Content-Type header is not "application/json", then a ValueError will be raised when trying to parse the response. Be sure to handle it properly.
I am trying to load a web page and monitor it for XHR (XMLHttpRequests). To do this I am using Ghost.py with Python2.7. I can see the XHR being made and can read the URL and response, however I would like to read the request body so that I can recreate these requests at a later date.
from ghost import Ghost, Session
ghost = Ghost()
with ghost.start():
session = Session(ghost, download_images=False, display=False)
page, rs = session.open("https://www.example.com/", timeout=60)
assert page.http_status == 200
result, resources = session.wait_while_selector('[class=loading]:not([style="display: none;"])', timeout=60)
for resource in resources:
print resource.url
print resource.content
I have searched within the documentation, but cannot find any references to XHR request body and I have searched within the returned resources object for references to the request, but can only find request.headers (which does not include the POST body) and the _reply data.
Can you save the POST body of a XHR request as well as the response?
Request is {"name":"vikas","age":20}
I am using following code to return the response from server
URL : "localhost:3000"
status_code, employee_data = webutils.make_request(settings.URL + "get_data/",
method='POST',
body=json.dumps(request.data),
json=True)
return Response(employee_data )
When I am using the postman,accessing the url localhost:3000/get_data/
It is returning the correct format.
But through django framework returning the response as false.
Please let me know the following code,I have written is correct in django
webutils.make_request(settings.URL + "get_data/", method='POST', body=json.dumps(request.data), json=True)
Please try with httpresponse class json dump data.
Try with Python requests library
How can I test a scrapy spider against online data.
I now from this post that it is possible to test a spider against offline data.
My target is to check if my spider still extracts the right data from a page, or if the page changed. I extract the data via XPath and sometimes the page receives and update and my scraper is no longer working. I would love to have the test as close to my code as possible, eg. using the spider and scrapy setup and just hook into the parse method.
Referring to the link you provided, you could try this method for online testing which I used for my problem which was similar to yours. All you have to do is instead of reading the requests from a file you can use the Requests library to fetch the live webpage for you and compose a scrapy response from the response you get from Requests like below
import os
import requests
from scrapy.http import Response, Request
def online_response_from_url (url=None):
if not url:
url = 'http://www.example.com'
request = Request(url=url)
oresp = requests.get(url)
response = TextResponse(url=url, request=request,
body=oresp.text, encoding = 'utf-8')
return response
Background
I have a service A accessible with HTTP requests. And I have other services that want to invoke these APIs.
Problem
When I test service A's APIs with POSTMAN, every request works fine. But when I user python's requests library to make these request, there is one PUT method that just won't work. For some reason, the PUT method being called cannot receive the data (HTTP body) at all, though it can receive headers. On the other side, the POST method called in the same manner receives the data perfectly.
I managed to achieve my goal simply by using httplib library instead, but I am still quite baffled by what exactly happened here.
The Crime Scene
Route 1:
#app.route("/private/serviceA", methods = ['POST'])
#app.route("/private/serviceA/", methods = ['POST'])
def A_create():
# request.data contains correct data that can be read with request.get_json()
Route 2:
#app.route("/private/serviceA/<id>", methods = ['PUT'])
#app.route("/private/serviceA/<id>/", methods = ['PUT'])
def A_update(id):
# request.data is empty, though request.headers contains headers I passed in
# This happens when sending the request with Python requests library, but not when sending with httplib library or with POSTMAN
# Also, data comes in fine when all other routes are commented out
# Unless all other routes are commented out, this happens even when the function body has only one line printing request.data
Route 3:
#app.route("/private/serviceA/schema", methods = ['PUT'])
def schema_update_column():
# This one again works perfectly fine
Using POSTMAN:
Using requests library from another service:
#app.route("/public/serviceA/<id>", methods = ['PUT'])
def A_update(id):
content = request.get_json()
headers = {'content-type': 'application/json'}
response = requests.put('%s:%s' % (router_config.HOST, serviceA_instance_id) + '/private/serviceA/' + str(id), data=json.dumps(content), headers = headers)
return Response(response.content, mimetype='application/json', status=response.status_code)
Using httplib library from another service:
#app.route('/public/serviceA/<id>', methods=['PUT'])
def update_course(id):
content= request.get_json()
headers = {'content-type': 'application/json'}
conn = httplib.HTTPConnection('%s:%s' % (router_config.HOST, serviceA_instance_id))
conn.request("PUT", "/private/serviceA/%s/" % id, json.dumps(content), headers)
return str(conn.getresponse().read())
Questions
1. What am I doing wrong for the route 2?
2. For route 2, the handler doesn't seem to be executed when either handler is commented out, which also confuses me. Is there something important about Flask that I'm not aware of?
Code Repo
Just in case some nice ppl are interested enough to look at the messy undocumented code...
https://github.com/fantastic4ever/project1
The serviceA corresponds to course service (course_flask.py), and the service calling it corresponds to router service (router.py).
The version that was still using requests library is 747e69a11ed746c9e8400a8c1e86048322f4ec39.
In your use of the requests library, you are using requests.post, which is sending a POST request. If you use requests.put then you would send a PUT request. That could be the issue.
Request documentation