python-requests: POST works, PUT fails - "Authentication credentials were not provided." - django

Editing again with more updates:
Trying to troubleshoot python-requests to see if something is wrong with a PUT request, but not sure how to proceed.
Below is a snippet of my code:
def API_request(url=None, headers=None, payload=None, update=False):
r = None
if update and headers and payload:
print "put request to %s with %s of %s" % (url, headers, payload)
r = requests.put(url, headers=headers, data=payload)
if headers and payload and not update:
print "post request to %s with %s of %s" % (url, headers, payload)
r = requests.post(url, headers=headers, data=payload)
print r.status_code
print r.text
When the above sends a POST request to create a record, it works. However, whenever it sends a PUT request, I get a 401 error: "Authentication credentials were not provided." This happens across multiple endpoints.
401
{"detail":"Authentication credentials were not provided."}
If I copy/paste the relevant printed output from the above PUT print function into a direct HTTPie request, it works. The below request results in a successful 200 response and updated record on the server:
http --debug PUT [url] < [file containing payload] Authorization:'Token [token]'
If I hard code a simple script that does nothing more than import python and json and PUT the exact same data to the same url using the same headers (printed form the original statement), it works. The below script results in a successful 200 response and updated record on the server:
import requests, json
url = "[my url"
headers = {'Content-Type': 'application/json', 'Authorization': 'Token [my token]'}
data = {[my data]}
payload = json.dumps(data)
r = requests.put(url, headers=headers, data=payload)
print r.status_code
print r.text
I've sent the information from both scripts to https://requestbin.fullcontact.com/ and they look to be identical.
BIG ISSUE:
After an entire day of debugging I figured out that even the requests that were generating a 401 error were successfully hitting the server and updating the appropriate records. Put simply, this would not be possible without a correct and functional authentication. Given that, why would I be getting a 401 error from the PUT request?
Happy to answer any questions in comments.

The above was sending a follow-up GET request (without the header, so it was failing) after successfully sending the PUT request (which was succeeding). I caught this by logging all requests hitting the server.
I figured out why the follow up GET was being sent and corrected that. I still don't understand why the r.code and r.text response from the successful PUT was never printing and hitting the console. Had I seen that, it would have been much easier to find. However, since the main problem is resolved, I can't spend time troubleshooting that.
I should have been seeing both responses - the success and the fail in the console - problem for another time.

Related

How to see application logs in Django Rest Framework?

In tests I am making an api call, which returns 400. It is expected, but I can't find a way to debug this. Does django keep logs in a file somewhere? Or can I enable showing logs please?
res = self.client.post(self.url, data=payload, format='json')
print(res)
// <Response status_code=400, "application/json">
I knot something went wrong but how do I debug the server?
Thanks
You can use response.content to view the final content/error messages that will be rendered on the web-page, as bytestring. docs
>>> response = c.get('/foo/bar/')
>>> response.content
b'<!DOCTYPE html...
If you are returning a json response(which you probably are if using rest framework), you can use response.json() to parse the json. docs
>>> response = client.get('/foo/')
>>> response.json()['name']
'Arthur'
Note: If the Content-Type header is not "application/json", then a ValueError will be raised when trying to parse the response. Be sure to handle it properly.

How to get request body of XMLHttpRequest (XHR) using Python (Ghost.py)?

I am trying to load a web page and monitor it for XHR (XMLHttpRequests). To do this I am using Ghost.py with Python2.7. I can see the XHR being made and can read the URL and response, however I would like to read the request body so that I can recreate these requests at a later date.
from ghost import Ghost, Session
ghost = Ghost()
with ghost.start():
session = Session(ghost, download_images=False, display=False)
page, rs = session.open("https://www.example.com/", timeout=60)
assert page.http_status == 200
result, resources = session.wait_while_selector('[class=loading]:not([style="display: none;"])', timeout=60)
for resource in resources:
print resource.url
print resource.content
I have searched within the documentation, but cannot find any references to XHR request body and I have searched within the returned resources object for references to the request, but can only find request.headers (which does not include the POST body) and the _reply data.
Can you save the POST body of a XHR request as well as the response?

400 server error if to try to send http Get requests with params and cookies

I'm trying to send https requests with python "requests" lib
payLoad = {'session_id': str(webSessionId), 'Merchant': 'Company'}
resp = requests.get(url, params = payLoad, cookies = cookiesSession, headers = headers)
print resp.url
But I'm getting server error 400. At the same time if to check url from sent request can see following:
https://url/error
Passed parameters are missing.
If to remove cookies following url is returned:
https://url?Merchant=Company&session_id=d1160eda748a5ede015bbe1abe35606138086fc8
But Server returns 403 error(authorization)
Cookies and headers are valid as were successfully used in previous two https request. But here is some strange situation. How can I send "get" request with params and cookies?

Python requests.request ValueError: too many values to unpack

I'm working on Entity Extraction usin an API call to https://dandelion.eu/ . I'm sending text files and automatically i get back a json file as response. It's not the first time i use this service and it worked really good. Now I started to send a new set of text file with the same parameters I always used but i get this: ValueError: too many values to unpack.
Here is my code:
values={"text":" ",
"min_confidence":"0.6",
"include":"types",
"include":"abstract",
"include":"categories"
}
headers = {'X-Target-URI':'https://api.dandelion.eu',
'Host':'api.dandelion.eu',
'Connection': 'keep-alive',
'Server': 'Apache-Coyote/1.1',
'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8',
}
for roots, dirs, files in os.walk(spath): #spath is specified
for file in files:
if file.startswith("A0"):
with open(file, "r") as f:
text = f.read()
values["text"]= " ".join(text.split())
#api call
url = "https://api.dandelion.eu/datatxt/nex/v1/"
data = urllib.urlencode(values, "utf-8")
response = requests.request("POST", url, data=data, headers=headers, params=token_api)
content = response.json()
print content
ErrorValue: too many values to unpack
Can somebody help me on this? I always used the same code for other api calls and it worked good. I don't know what is wrong now.
​
the API returns more than one value.
please refer to API documentation and see what are the return values.
(you did not mentioned what API raised the err in tour question)

Why do I get a 405 GET not allowed when I am sending a POST request?

I am sending the POST request from Android to my server (python/Flask)
URL url = new URL("www.xxx.com/yyy");
HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
httpCon.setRequestMethod("POST");
httpCon.setRequestProperty("Content-Type", "binary/octet-stream");
DataOutputStream out = new DataOutputStream(httpCon.getOutputStream());
out.write(bytes);
out.flush();
out.close();
int status = httpCon.getResponseCode();
...
and I receive this with Flask
cors = CORS(app)
app.config['CORS_HEADERS'] = 'Content-Type'
#app.route('/yyy', methods=['POST'])
#cross_origin(headers=['Content-Type'])
#def get_checkpoint_config():
# ...
First strange thing : sometimes the server logs
WARNING:tornado.access:405 GET /yyy
but it is a POST !
There is a little more strange to that : some other times I don't get the 405 but request.data is empty.
I suspect something is fishy with the CORS config. I replaced #cross_origin by #crossdomain(origin='*') (from this snippet) but I had the same behaviour
How can I fix this ?
Thanks
This is not a full answer but fortunately my requests were not too big so I put the data as a GET parameter and I could remove everything related with cors (Fask's cross origin extension) and it worked. The question still remains.