REST Call using 'requests' library in Python 2.7 - python-2.7

I am trying to execute a REST POST call using 'request' library, and i am using python 2.7
What happening is, somehow the call is not getting executed via python script and and i am getting HTTPS connection error saying cannot connect to proxy.
I have verified in my system, proxy settings has no issue. In the same setup i am able to execute the same REST call using POSTMAN client & cURL.
Below is code snippet: (here the 'hostname' is private IP address and accessible internally only)
import requests
import json
import pprint
import sys
url = "https://<hostname>/rest/login-sessions"
json_data = {'authLoginDomain':'',
'password':'abc',
'userName':'xyz',
'loginMsgAck':'true'}
reqHeaders = {'Content-type':'application/json',
'X-Api-Version':'200'}
try:
reqPost = requests.post(url,data=json_data,headers=reqHeaders,verify = False)
pprint.pprint(resPost.json())
except requests.exceptions.Timeout:
print "Re-Try Again"
except requests.exceptions.TooManyRedirects:
print "Try with a different URL"
except requests.exceptions.RequestException as e:
print e
sys.exit(1)
Error code pasted below:
HTTPSConnectionPool(host='<hostIP>', port=443): Max retries exceeded with url: /rest/login-sessions (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
I tried finding solutions available in the web but none of them are working for me. I am confuse because when i am trying to execute the REST call via a python script then only it happening, via cURL & Postman i am able to do the same.
Please suggest if i am doing some mistake here OR any better way available to this.
I want to do it with python 'requests' library only. Also, i tried executing it with cURL using python subprocess, there also i am getting the same error.

Related

Caller does not have permission error when using GMail API

I get 500 server error on my django website thats running on Google App Engine. When I look at Google App Engine logs I see the following error:
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/discovery/v1/apis/gmail/v1/rest returned "The caller does not have permission">
When I hard refresh the browser this error goes away. Then after some time it pops back up. Happens on mobile(firefox, safari), laptop (firefox,chrome).
UPDATE:
In Django settings.py I have following code. Its last line generates the error :
pickle_path = 'token.pickle' # path to token.pickle
with open(pickle_path, 'rb') as token:
creds = pickle.load(token)
SERVICE = build('gmail', 'v1', credentials=creds) # ERROR LINE
When I run the django server locally: I get following error:
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/discovery/v1/apis/gmail/v1/rest returned "The caller does not have permission">
When I restart cloud sql connection from my terminal - the error is gone. This never used to happen until a few days ago.
Here's a workaround:
Download the discovery_doc directly from google here
Load the json file (you can name it gmail-api.json)
Build from this json file using build_from_document
Before
from googleapiclient.discovery import build
gmail_creds = get_service_account_creds()
gmail_service = build('gmail', 'v1', credentials=gmail_creds)
After
from googleapiclient.discovery import build_from_document
discovery_doc = load_json('config/gmail-api.json')
gmail_creds = get_service_account_creds()
gmail_service = build_from_document(discovery_doc, credentials=gmail_creds)
It seems that it is now a P0 for Google, so hopefully it will be fixed soon.
https://issuetracker.google.com/issues/160441983
I am also started facing this issue. I have been using Google APIs for more than a year, but suddenly this error HttpError 403 when requesting https://www.googleapis.com/discovery/v1/apis/gmail/v1/rest returned "The caller does not have permission. However, couple of times I didn't see this error and API call was successful. I hope this error at the Gmail API server.
Update:
I am able to call Google APIs without any issues. Seems like issue is resolved at google's end.
I am having the same issue with GAM ADV-X. I thought it was an issue on my computer so I tried on 2 different computers and get the same error. It indeed looks like something changed in the Gmail API over the weekend. Will submit a ticket to Google.

Can I get the JSON response from a Flask-api that runs locally using URLLIB or REQUETS?

Is there a way to save the response of a flask api that is running locally on my machine?
It may not make a great sense as I have the logic locally and there is no need to get the response again from the local URL..but in my case, I have another webhook which runs locally which means I need to run flask and my webhook locally.
I am looking to get around this..
You can save your response on your machine through using pickle. But it is not recommended, because website can change their content anytime.
import requests
import pickle
resp = requests.get("https://github.com")
with open("test","wb") as fd:
pickle.dump(resp,fd)
with open("test","rb") as fd:
resp_ = pickle.load(fd)
print(resp_.url)

urllib2.HTTPError: HTTP Error 503: Service Unavailable error while using python google search

I'm using python google search to get the top 3 links for a query.
The following is the code. My list contains 1000 queries and I'm trying to save them into a dictionary with the query as the key and URLs as the value.
from google import search
for each in list:
time.sleep(1)
for URL in search(each,stop=3,num=3):
print URL
When I try to execute this, I get the error:
urllib2.HTTPError: HTTP Error 503: Service Unavailable
Should I use requests or any other libraries to eliminate this error?

cURL vs Requests (SSL issue?)

All, I'm getting some strange behavior trying to use requests for an https call into the gitub api:
print(requests.get('https://api.github.com/gists/bbc56a82f359eccd4bd6').text)
The output looks like printing a binary file (no point in pasting the garbled output here).
An equivalent cURL call ("curl https://api.github.com/gists/bbc56a82f359eccd4bd6") results in the JSON response I'm expecting.
All this started after fixing a pip issue (InsecurePlatformWarning), where a few security-related packages were installed. This fix is required for users of python<2.7.9. I'm on 2.7.3 as it was recommended on some sites not to touch the python build on debian (for dependency-breaking issues).
Note that the issue that i'm having breaks functionality for e.g. github3py python API wrapper, etc.
Is anyone else seeing issues with requests after the upgrade? Any fixes?
This URL clearly responds differently depending on user-agent. I could make the curl command line response differ by simply adding -A moo/1.
You can probably get a curl-like response with Requests from this by using a curl like user-agent.
Or even better: just ask github or read up on their API.
I'm not seeing that behaviour here:
>>> import requests
>>> print(requests.get('https://api.github.com/gists/bbc56a82f359eccd4bd6').text)
Returns a JSON string. You could try debugging this further by changing the User-Agent of your request call to be that of cURL:
headers = {
'User-Agent': 'curl/7.38.0',
}
url = 'https://api.github.com/gists/bbc56a82f359eccd4bd6'
response = requests.get(url, headers=headers)

Gunicorn worker timeout

I have a Django application running in Gunicorn behind Nginx. Everything works fine, exect for one strange thing: I have a "download" view and a RESTful json API. When call the download view I use urllib2 to access the json API to get information. And excactly when I try to do this http get request to the json api, the request times out with an error HTTP Error 504: Gateway Time-out.
When I run the code with ./manage.py runserver everything works fine. The http get request to the json api also only takes a few miliseconds, so no danger of running into a timeout.
Here the Situation in Pseudo code:
myproject/views.py: (accessible as: http://myproject.com/download)
1 def download(request, *args, **kwargs):
2 import urllib2
3 opener = urllib2.build_opener()
4 opener.open('http://myproject.com/api/get_project_stats')
The opener.open() call in line four runs into a timeout when running in Gunicorn, when running with ./manage.py runservereverytihng works fine (and the api call only takes a few miliseconds.
Has anyone had the same problem? And more important: How have you solved it?
I had the same issue using Gunicorn, nGinx, Django and Requests
every time I did:
response = requests.get('http://my.url.com/here')
the workers would timeout
I solved the problem by switching from Syncronous (sync) workers to Asynchronous (eventlet) workers.
if you are launching command line add:
-k 'eventlet'
if you are using a config file add:
worker_class = "eventlet"