migrate from urllib2 to requests python 2.7 - python-2.7

I am trying to take some working code and change from urlib2 to requests.
The original code provides basic login information of username, password and posts the KEY and SECRET in the header of the urllib2 request. The following code is my attempt to change to using the requests module and gain some functionality for making additional API calls. I have tried dozens of combinations and all return a code 400. Apparently, my requests code does not successfully furnish the needed information to return a 200 response and provide the needed authorization token.
## Import needed modules
import urllib2, urllib, base64
import httplib
import requests
import json
## initialize variables
KEY = "7f1xxxx-3xxx-4xxx-9a7f-8be66839dede"
SECRET = "45xxxxxx-45xxx-469a-9ae9-a7927a76cfeb"
userName = "my-email#xxx.com"
passWord = "mypassword"
URL = "https://company.com/auth/token"
token = None
sessionid = None
DATA = urllib.urlencode({"grant_type":"password",
"username":userName,
"password":passWord})
base64string = base64.encodestring('%s:%s' % (KEY, SECRET)).replace('\n', '')
request = urllib2.Request(URL, DATA)
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
token = result.read()
print token
This returns my authorization token and all is well. I can pass the token to the authorization server and have full access to the api for interacting with the database. Below is the attempt to use requests and have the added functions it provides.
client = requests.session()
payload = {"grant_type":"password",
"username":userName,
"password":passWord,
"applicationId": KEY
}
headers = {'content-type':'application/json',
"grant_type":"password",
"username":userName,
"password":passWord,
'applicationsId': KEY,
'Authorization': base64string,
'token': token,
'sessionid': sessionid
}
response = client.post(URL, params = payload, headers=headers)
token = response.content
print token
{"error":"invalid_request"}
print response
<Response [400]>

If you want to use basic auth you should use the method from requests..
Your post should look like
response = client.post(
URL,
params = payload,
headers=headers,
auth=HTTPBasicAuth(
KEY,
SECRET
))

Somewhere in a post a contributor to another question mentioned some items actually needed to be in the body of the request not in the header. I tried various combos and the following solved the 400 response and accomplished my goals.
data = {"grant_type":"password",
"username":userName,
"password":passWord,
"applicationId": KEY
}
headers = {'Authorization': "Basic %s" % base64string,
'token': token
}
response = client.post(URL, data = data, headers=headers)
token = response.text
print token

Related

How to get Superset Token?? (for use Rest api)

I attempted to request a REST request to see the document below. But do not work. https://superset.apache.org/docs/rest-api
request: curl -XGET -L http://[IP:PORT]/api/v1/chart
response: {"msg":"Bad Authorization header. Expected value 'Bearer <JWT>'"}
The Superset installation has been on PIP and was also Helm Chart. But all are the same. helm: https://github.com/apache/superset
How should I order a REST API?
Check the security section of the documentation you have linked. It has this API ​/security​/login, you can follow the JSON parameter format and get the JWT bearer token. Use that token to send in the Header of your other API calls to superset.
open http://localhost:8080/swagger/v1, assuming http://localhost:8080 is your Superset host address
then find this section
the response would be like this
{
"access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJmcmVzaCI6dHJ1ZSwiaWF0IjoxNjU0MzQ2OTM5LCJqdGkiOiJlZGY2NTUxMC0xMzI1LTQ0NDEtYmFmMi02MDc1MzhjZDcwNGYiLCJ0eXBlIjoiYWNjZXNzIiwic3ViIjoxLCJuYmYiOjE2NTQzNDY5MzksImV4cCI6MTY1NDM0NzgzOX0.TfjUea3ycH77xhCWOpO4LFbYHrT28Y8dnWsc1xS_IOY",
"refresh_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJmcmVzaCI6ZmFsc2UsImlhdCI6MTY1NDM0NjkzOSwianRpIjoiNzBiM2EyZDYtNDFlNy00ZDNlLWE0NDQtMTRiNTkyNTk4NjUwIiwidHlwZSI6InJlZnJlc2giLCJzdWIiOjEsIm5iZiI6MTY1NDM0NjkzOSwiZXhwIjoxNjU2OTM4OTM5fQ.OgcctNnO4zTDfTgtHnaEshk7u-D6wOxfxjCsjqjKYyE"
}
Thank #andrewsali commented on this github issue, I finally figure out how to access the superset REST API by python code.
import requests
from bs4 import BeautifulSoup
import json
def get_supetset_session():
"""
# http://192.168.100.120:8088/swagger/v1
url = f'http://{superset_host}/api/v1/chart/'
r = s.get(url)
# print(r.json())
"""
superset_host = '192.168.100.120:8088' # replace with your own host
username = 'YOUR_NAME'
password = 'YOUR_PASSWORD'
# set up session for auth
s = requests.Session()
login_form = s.post(f"http://{superset_host}/login")
# get Cross-Site Request Forgery protection token
soup = BeautifulSoup(login_form.text, 'html.parser')
csrf_token = soup.find('input',{'id':'csrf_token'})['value']
data = {
'username': username,
'password': password,
'csrf_token':csrf_token
}
# login the given session
s.post(f'http://{superset_host}/login/', data=data)
print(dict(s.cookies))
return s
DEMO
# s = get_supetset_session()
base_url = 'http://192.168.100.120:8088'
def get_dashboards_list(s, base_url=base_url):
"""## GET List of Dashboards"""
url = base_url + '/api/v1/dashboard/'
r = s.get(url)
resp_dashboard = r.json()
for result in resp_dashboard['result']:
print(result['dashboard_title'], result['id'])
s = get_supetset_session()
# {'session': '.eJwlj8FqAzEMRP_F5z1Islay8jOLJcu0NDSwm5xK_r0uPQ7DG978lGOeeX2U2_N85VaOz1FuxVK6JIHu1QFhGuEOk5NG8qiYGkJ7rR3_Ym-uJMOzJqySeHhIG8SkNQK6GVhTdLf0ZMmG6sZGQtiQ1Gz0qYiUTVoHhohZthLXOY_n4yu_l0-VKTObLaE13i2Hz2A2rzBmhU7WkkN1cfdH9HsuZoFbeV15_l_C8v4F4nBC9A.Ypn16Q.yz4E-vz0gp3EmJwv-6tYIcOGavU'}
get_dashboards_list(s)
Thanks #Ferris for this visual solution!
To add to this, you can also create the appropriate API call with Python just like following:
import requests
api_url = "your_url/api/v1/security/login"
payload = {"password":"your password",
"provider":"db",
"refresh":True,
"username":"your username"
}
response = requests.post(api_url, json=payload)
# the acc_token is a json, which holds access_token and refresh_token
access_token = response.json()['access_token']
# no get a guest token
api_url_for_guesttoken = "your_url/api/v1/security/guest_token"
payload = {}
# now this is the crucial part: add the specific auth-header
response = request.post(api_url_for_guesttoken , json=payload, headers={'Authorization':f"Bearer {access_token}"})

403 Forbidden when trying to register receiver endpoint using the RISC API

While trying to register my receiver endpoint in order to start receiving RISC indications from google, I constantly get the same reply:
403 Client Error: Forbidden for url:
https://risc.googleapis.com/v1beta/stream:update
I have created the service with the Editor Role and using the json key I created as requested on the integration guide.
This is my provisioning code I use to do that:
import json
import time
import jwt # pip install pyjwt
import requests
def make_bearer_token(credentials_file):
with open(credentials_file) as service_json:
service_account = json.load(service_json)
issuer = service_account['client_email']
subject = service_account['client_email']
private_key_id = service_account['private_key_id']
private_key = service_account['private_key']
issued_at = int(time.time())
expires_at = issued_at + 3600
payload = {'iss': issuer,
'sub': subject,
'aud': 'https://risc.googleapis.com/google.identity.risc.v1beta.RiscManagementService',
'iat': issued_at,
'exp': expires_at}
encoded = jwt.encode(payload, private_key, algorithm='RS256',
headers={'kid': private_key_id})
return encoded
def configure_event_stream(auth_token, receiver_endpoint, events_requested):
stream_update_endpoint = 'https://risc.googleapis.com/v1beta/stream:update'
headers = {'Authorization': 'Bearer {}'.format(auth_token)}
stream_cfg = {'delivery': {'delivery_method': 'https://schemas.openid.net/secevent/risc/delivery-method/push',
'url': receiver_endpoint},
'events_requested': events_requested}
response = requests.post(stream_update_endpoint, json=stream_cfg, headers=headers)
response.raise_for_status() # Raise exception for unsuccessful requests
def main():
auth_token = make_bearer_token('service_creds.json')
configure_event_stream(auth_token, 'https://MY-ENDPOINT.io',
['https://schemas.openid.net/secevent/risc/event-type/sessions-revoked',
'https://schemas.openid.net/secevent/oauth/event-type/tokens-revoked',
'https://schemas.openid.net/secevent/risc/event-type/account-disabled',
'https://schemas.openid.net/secevent/risc/event-type/account-enabled',
'https://schemas.openid.net/secevent/risc/event-type/account-purged',
'https://schemas.openid.net/secevent/risc/event-type/account-credential-change-required'])
if __name__ == "__main__":
main()
Also tested my auth token and it seems as the integration guide suggests.
Could not find 403 forbidden on the error code reference table there.
You can check for error description in the response body and match that against the possible reasons listed here!

Why does my server complain of 'Bad authorization header'?

I am having trouble getting my client and server for a simple AWS Cognito authorization to agree on what the token should look like.
My client uses boto3 and requests. I pass the authorization in the header which I've seen in more than half of tutorials but I've also seen use of the auth parameter which I am not using.
Here's my server:
from flask import request, jsonify, make_response
import flask.json
from flask_jwt_extended import (
JWTManager, jwt_required, create_access_token,
get_jwt_identity
)
app = flask.Flask(__name__)
# from: https://flask-jwt-extended.readthedocs.io/en/stable/basic_usage/
# Setup the Flask-JWT-Extended extension
app.config['JWT_SECRET_KEY'] = 'super-secret' # Change this!
jwt = JWTManager(app)
#app.route("/auth_test")
#jwt_required
def auth_test ():
current_user = get_jwt_identity()
return jsonify({"current_user": "current_user"})
if __name__ == "__main__":
app.run(debug=True)
One clue here is that JWT_SECRET_KEY is copied from a tutorial but I'm unclear on what I need to change it to. Perhaps this is my issue but if it is I don't see the solution or what to change it to.
Here's my client:
import requests
import boto3
host = "http://127.0.0.1:8000"
region = '...'
user_pool_id = '...'
username = '...'
password = '...'
app_client_id = '...'
client = boto3.client('cognito-idp', region_name = region)
auth_response = client.admin_initiate_auth(
UserPoolId = user_pool_id,
ClientId = app_client_id,
AuthFlow = 'ADMIN_NO_SRP_AUTH',
AuthParameters = {
'USERNAME' : username,
'PASSWORD' : password
}
)
id_token = auth_response['AuthenticationResult']['IdToken']
response = requests.get(host + "/auth_test" , headers = { 'Authorization': id_token })
print(response.json())
It prints:
{'msg': "Bad Authorization header. Expected value 'Bearer '"}
This indicates that I should be passing "Bearer " + id_token instead of just id_token but this is contrary to most tutorials I've read and furthermore when I do that I still get the same error.
Anyone know what I'm doing wrong?
EDIT:
I have made some changes due to the comments. I have added
app.config['JWT_ALGORITHM'] = 'RS256'
app.config['JWT_DECODE_ALGORITHMS'] = ['RS256']
#app.config['JWT_PUBLIC_KEY'] = '...' # value that I got from https://cognito-idp.us-east-1.amazonaws.com/<my pool id>/.well-known/jwks.json. There were two entries I tried both values found in kid
RS256 matches Cognito's encoding so I think I need to set that for my server. However I now simply get this error when running the demo:
ValueError: Could not deserialize key data.
Thank you for the help!

Token expire before file was uploaded

I'm using requests session with oauth2 authentication.
Everything works perfectly when I upload small files, but for 4GB file I get token expired error, it looks like the file was uploaded but at the closing session part token was once more validated.
Is there any chance to handle this situation?
Upload large file with token refreshed before the session was closed or something?
a sample of the code is below, Thank You very much for any help. Cheers!
import requests
from io import StringIO
from requests_toolbelt.multipart.encoder import MultipartEncoder
TOKEN_PAYLOAD = {
'grant_type': 'password',
'client_id': '###',
'client_secret': '###',
'username': '###',
'password': '####'
}
def get_token():
response = requests.post(
'https://oauth/token',
params=TOKEN_PAYLOAD)
response_data = response.json()
token = response_data.get('access_token')
return token
# Create test file
MB = 1024 ** 2
GB = MB * 1024
encoded_string = 'x' * 4 * GB
file_test = StringIO()
file_test.write(encoded_string)
# Get token
token = get_token()
# Create form
multipart_data = MultipartEncoder(
fields={
'--': ('4GB_test.txt', file_test, 'text/plain'),
'id': '2217',
'fileFieldDefId': '4258',
}
)
# Create headers
headers = {
"Authorization": "Bearer {}".format(token),
'Content-Type': multipart_data.content_type
}
session = requests.Session()
response = session.post(
'https://oauth2/rest/external/item/multipartUpdate/byId',
headers=headers,
data=multipart_data,
)
print(response)
# <Response [401]>
print(response.content)
# b'{"error":"invalid_token","error_description":"Access token expired: 0f7f6bd9-4e21-407f-4a78347711a9"}'
# response.close() ? with refreshed token
# session.close() ? with refreshed token
If you want to have valid access tokens for more time you can also request for refresh tokens and use them to generate new access tokens whenever the old one expires. Generally access tokens are valid for 1 hour, you can maintain a timer and generate a new access token every time your timer reaches 60 minutes. That way you can have a valid access token for longer sessions.
You have to use grant_type=refresh_token https://www.rfc-editor.org/rfc/rfc6749#section-6

OAuth 2.0 for shortener API

I'am trying to generate unique URLs, and then check this in https://goo.gl/. My code in python:
import requests
import json
def goo_shorten_url(url):
post_url = 'https://www.googleapis.com/urlshortener/v1/url?key=xxxxxxxxxxxxxxxxxxxx'
payload = {'longUrl': url}
headers = {'content-type': 'application/json'}
r = requests.post(post_url, data=json.dumps(payload), headers=headers)
print r.text
goo_shorten_url('http://www.google.com')
However I can't understand how to modifies my code and configure a token in order to get this unique URLs