Token expire before file was uploaded - python-2.7

I'm using requests session with oauth2 authentication.
Everything works perfectly when I upload small files, but for 4GB file I get token expired error, it looks like the file was uploaded but at the closing session part token was once more validated.
Is there any chance to handle this situation?
Upload large file with token refreshed before the session was closed or something?
a sample of the code is below, Thank You very much for any help. Cheers!
import requests
from io import StringIO
from requests_toolbelt.multipart.encoder import MultipartEncoder
TOKEN_PAYLOAD = {
'grant_type': 'password',
'client_id': '###',
'client_secret': '###',
'username': '###',
'password': '####'
}
def get_token():
response = requests.post(
'https://oauth/token',
params=TOKEN_PAYLOAD)
response_data = response.json()
token = response_data.get('access_token')
return token
# Create test file
MB = 1024 ** 2
GB = MB * 1024
encoded_string = 'x' * 4 * GB
file_test = StringIO()
file_test.write(encoded_string)
# Get token
token = get_token()
# Create form
multipart_data = MultipartEncoder(
fields={
'--': ('4GB_test.txt', file_test, 'text/plain'),
'id': '2217',
'fileFieldDefId': '4258',
}
)
# Create headers
headers = {
"Authorization": "Bearer {}".format(token),
'Content-Type': multipart_data.content_type
}
session = requests.Session()
response = session.post(
'https://oauth2/rest/external/item/multipartUpdate/byId',
headers=headers,
data=multipart_data,
)
print(response)
# <Response [401]>
print(response.content)
# b'{"error":"invalid_token","error_description":"Access token expired: 0f7f6bd9-4e21-407f-4a78347711a9"}'
# response.close() ? with refreshed token
# session.close() ? with refreshed token

If you want to have valid access tokens for more time you can also request for refresh tokens and use them to generate new access tokens whenever the old one expires. Generally access tokens are valid for 1 hour, you can maintain a timer and generate a new access token every time your timer reaches 60 minutes. That way you can have a valid access token for longer sessions.
You have to use grant_type=refresh_token https://www.rfc-editor.org/rfc/rfc6749#section-6

Related

403 Forbidden when trying to register receiver endpoint using the RISC API

While trying to register my receiver endpoint in order to start receiving RISC indications from google, I constantly get the same reply:
403 Client Error: Forbidden for url:
https://risc.googleapis.com/v1beta/stream:update
I have created the service with the Editor Role and using the json key I created as requested on the integration guide.
This is my provisioning code I use to do that:
import json
import time
import jwt # pip install pyjwt
import requests
def make_bearer_token(credentials_file):
with open(credentials_file) as service_json:
service_account = json.load(service_json)
issuer = service_account['client_email']
subject = service_account['client_email']
private_key_id = service_account['private_key_id']
private_key = service_account['private_key']
issued_at = int(time.time())
expires_at = issued_at + 3600
payload = {'iss': issuer,
'sub': subject,
'aud': 'https://risc.googleapis.com/google.identity.risc.v1beta.RiscManagementService',
'iat': issued_at,
'exp': expires_at}
encoded = jwt.encode(payload, private_key, algorithm='RS256',
headers={'kid': private_key_id})
return encoded
def configure_event_stream(auth_token, receiver_endpoint, events_requested):
stream_update_endpoint = 'https://risc.googleapis.com/v1beta/stream:update'
headers = {'Authorization': 'Bearer {}'.format(auth_token)}
stream_cfg = {'delivery': {'delivery_method': 'https://schemas.openid.net/secevent/risc/delivery-method/push',
'url': receiver_endpoint},
'events_requested': events_requested}
response = requests.post(stream_update_endpoint, json=stream_cfg, headers=headers)
response.raise_for_status() # Raise exception for unsuccessful requests
def main():
auth_token = make_bearer_token('service_creds.json')
configure_event_stream(auth_token, 'https://MY-ENDPOINT.io',
['https://schemas.openid.net/secevent/risc/event-type/sessions-revoked',
'https://schemas.openid.net/secevent/oauth/event-type/tokens-revoked',
'https://schemas.openid.net/secevent/risc/event-type/account-disabled',
'https://schemas.openid.net/secevent/risc/event-type/account-enabled',
'https://schemas.openid.net/secevent/risc/event-type/account-purged',
'https://schemas.openid.net/secevent/risc/event-type/account-credential-change-required'])
if __name__ == "__main__":
main()
Also tested my auth token and it seems as the integration guide suggests.
Could not find 403 forbidden on the error code reference table there.
You can check for error description in the response body and match that against the possible reasons listed here!

Why does my server complain of 'Bad authorization header'?

I am having trouble getting my client and server for a simple AWS Cognito authorization to agree on what the token should look like.
My client uses boto3 and requests. I pass the authorization in the header which I've seen in more than half of tutorials but I've also seen use of the auth parameter which I am not using.
Here's my server:
from flask import request, jsonify, make_response
import flask.json
from flask_jwt_extended import (
JWTManager, jwt_required, create_access_token,
get_jwt_identity
)
app = flask.Flask(__name__)
# from: https://flask-jwt-extended.readthedocs.io/en/stable/basic_usage/
# Setup the Flask-JWT-Extended extension
app.config['JWT_SECRET_KEY'] = 'super-secret' # Change this!
jwt = JWTManager(app)
#app.route("/auth_test")
#jwt_required
def auth_test ():
current_user = get_jwt_identity()
return jsonify({"current_user": "current_user"})
if __name__ == "__main__":
app.run(debug=True)
One clue here is that JWT_SECRET_KEY is copied from a tutorial but I'm unclear on what I need to change it to. Perhaps this is my issue but if it is I don't see the solution or what to change it to.
Here's my client:
import requests
import boto3
host = "http://127.0.0.1:8000"
region = '...'
user_pool_id = '...'
username = '...'
password = '...'
app_client_id = '...'
client = boto3.client('cognito-idp', region_name = region)
auth_response = client.admin_initiate_auth(
UserPoolId = user_pool_id,
ClientId = app_client_id,
AuthFlow = 'ADMIN_NO_SRP_AUTH',
AuthParameters = {
'USERNAME' : username,
'PASSWORD' : password
}
)
id_token = auth_response['AuthenticationResult']['IdToken']
response = requests.get(host + "/auth_test" , headers = { 'Authorization': id_token })
print(response.json())
It prints:
{'msg': "Bad Authorization header. Expected value 'Bearer '"}
This indicates that I should be passing "Bearer " + id_token instead of just id_token but this is contrary to most tutorials I've read and furthermore when I do that I still get the same error.
Anyone know what I'm doing wrong?
EDIT:
I have made some changes due to the comments. I have added
app.config['JWT_ALGORITHM'] = 'RS256'
app.config['JWT_DECODE_ALGORITHMS'] = ['RS256']
#app.config['JWT_PUBLIC_KEY'] = '...' # value that I got from https://cognito-idp.us-east-1.amazonaws.com/<my pool id>/.well-known/jwks.json. There were two entries I tried both values found in kid
RS256 matches Cognito's encoding so I think I need to set that for my server. However I now simply get this error when running the demo:
ValueError: Could not deserialize key data.
Thank you for the help!

How to pass the user token for API testing in django rest framework?

I'm writing some tests to check the connection to my API.
I've put in place identification via tokens and I am successful with retrieving a token for a specific test user with :
token = Token.objects.get(user__username='testuser')
What I'm struggling with is to use that token to create a successful API request as this one :
client = APIClient(HTTP_AUTHORIZATION='Token ' + token.key)
response = client.get('/patientFull/1/',headers={'Authorization': 'Token ' + token.key})
I have been looking at many ways to make this work and these are some ways I tried to do it :
response = requests.get('http://127.0.0.1:8000/patientFull/1/',headers={'Authorization': 'Token ' + token.key} )
client = APIClient()
client.credentials(HTTP_AUTHORIZATION='Token ' + token.key)
response = client.get('/patientFull/1/')
The test is a simple assert to check response has a 200 OK HTTP answer from the server.
All of these ways above returns a 403 HTTP response.
here's the full code of my test (I'm using fixtures to populate my test database with testing data):
import json
import requests
from rest_framework.authtoken.models import Token
from rest_framework.test import APIRequestFactory, APITestCase, APIClient
class CustomerAPITestBack(APITestCase):
fixtures = ['new-fixtures.json']
def testDE(self):
token = Token.objects.get(user__username='jpmichel')
client = APIClient(HTTP_AUTHORIZATION='Token ' + token.key)
response = client.get('/patientFull/1/',headers={'Authorization': 'Token ' + token.key})
self.assertEqual(200, response.status_code)
I have configured my settings.py file as so for the tokens :
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.BasicAuthentication',
'rest_framework.authentication.SessionAuthentication',
'PHCAdmin.authentication.tokenAuthentication.ExpiringTokenAuthentication',
),
'EXCEPTION_HANDLER': 'PHCAdmin.functions.pxlth_exception_handler',
}
REST_FRAMEWORK_EXPIRY_TIME = 12 # in hours
REST_FRAMEWORK_PASSWORD_RENEWALS = 90 # in days
If I disable the token authentication, this test passes (the GET returns a 200 OK)
How should I do my GET request so that it uses the token to identify as a valid user and returns a 200 OK HTTP response?
Just to close this question:
after some research, I found out the token on the server was not the same as on the local machine, I just needed to update tokens on both side.
The code above works fine.
I came here looking for an answer to force_authenticate not working while using a token. To those in the same case, you need to specify both user and token in this manner:
client = APIClient()
client.force_authenticate(user=UserInstance, token=UserInstanceToken)

How to fetch data from OMS workspace

I read the documentation yesterday and done some coding with python to fetch data in the following way. It's working fine.
import logging as log
import adal
import requests
import json
import datetime
from pprint import pprint
# Details of workspace. Fill in details for your workspace.
resource_group = 'Test'
workspace = 'FirstMyWorkspace'
# Details of query. Modify these to your requirements.
query = "Type=*"
end_time = datetime.datetime.utcnow()
start_time = end_time - datetime.timedelta(hours=24)
num_results = 2 # If not provided, a default of 10 results will be used.
# IDs for authentication. Fill in values for your service principal.
subscription_id = '{subscription_id}'
tenant_id = '{tenant_id}'
application_id = '{application_id}'
application_key = '{application_key}'
# URLs for authentication
authentication_endpoint = 'https://login.microsoftonline.com/'
resource = 'https://management.core.windows.net/'
# Get access token
context = adal.AuthenticationContext('https://login.microsoftonline.com/' + tenant_id)
token_response = context.acquire_token_with_client_credentials('https://management.core.windows.net/', application_id, application_key)
access_token = token_response.get('accessToken')
# Add token to header
headers = {
"Authorization": 'Bearer ' + access_token,
"Content-Type": 'application/json'
}
# URLs for retrieving data
uri_base = 'https://management.azure.com'
uri_api = 'api-version=2015-11-01-preview'
uri_subscription = 'https://management.azure.com/subscriptions/' + subscription_id
uri_resourcegroup = uri_subscription + '/resourcegroups/'+ resource_group
uri_workspace = uri_resourcegroup + '/providers/Microsoft.OperationalInsights/workspaces/' + workspace
uri_search = uri_workspace + '/search'
# Build search parameters from query details
search_params = {
"query": query,
"top": num_results
}
# Build URL and send post request
uri = uri_search + '?' + uri_api
response = requests.post(uri, json=search_params,headers=headers)
# Response of 200 if successful
if response.status_code == 200:
# Parse the response to get the ID and status
data = response.json()
if data.get("__metadata", {}).get("resultType", "") == "error":
log.warn("oms_fetcher;fetch_job;error: " + ''.join('{}={}, '.format(key, val) for key, val in
data.get("error", {}).items()))
else:
print data["value"]
search_id = data["id"].split("/")
id = search_id[len(search_id)-1]
status = data["__metadata"]["Status"]
print status
# If status is pending, then keep checking until complete
while status == "Pending":
# Build URL to get search from ID and send request
uri_search = uri_search + '/' + id
uri = uri_search + '?' + uri_api
response = requests.get(uri, headers=headers)
# Parse the response to get the status
data = response.json()
status = data["__metadata"]["Status"]
print id
else:
# Request failed
print (response.status_code)
response.raise_for_status()
Today I went to the same webpage that I have followed yesterday but there is a different documentation today. So do I need to follow the new documentation? I tried new documentation too but got into an issue
url = "https://api.loganalytics.io/v1/workspaces/{workspace_id}/query"
headers = {
"X-Api-Key": "{api_key}",
"Content-Type": 'application/json'
}
search_param = {
}
res = requests.post(url=url, json=search_param, headers=headers)
print res.status_code
print res.json()
{u'error': {u'innererror': {u'message': u'The given API Key is not
valid for the request', u'code': u'UnsupportedKeyError'}, u'message':
u'Valid authentication was not provided', u'code':
u'AuthorizationRequiredError'}}
Here is the link to documentation
The api_key is not oms primary key on Portal. You could check example in this link. The token should like below:
Authorization: Bearer <access token>
So, you need modify X-Api-Key": "{api_key} to Authorization: Bearer <access token>.
You need create a service principal firstly, please check this link.
Then, you could use the sp to get token, please check this link.
Note: You could your code to get token, but you need modify the resource to https://api.loganalytics.io. Like below:
# Get access token
context = adal.AuthenticationContext('https://login.microsoftonline.com/' + tenant_id)
token_response = context.acquire_token_with_client_credentials('https://api.loganalytics.io', application_id, application_key)
access_token = token_response.get('accessToken')
# Add token to header
headers = {
"Authorization": 'Bearer ' + access_token,
"Content-Type": 'application/json'
}
Working Prototype to Query OMS or Log Analytic workspace.
import adal
import requests
import json
import datetime
from pprint import pprint
# Details of workspace. Fill in details for your workspace.
resource_group = 'xxx'
workspace = 'xxx'
workspaceid = 'xxxx'
# Details of query. Modify these to your requirements.
query = "AzureActivity | limit 10"
# IDs for authentication. Fill in values for your service principal.
subscription_id = 'xxxx'
# subscription_id = 'xxxx'
tenant_id = 'xxxx'
application_id = 'xxxx'
application_key = 'xxxxx'
# Get access token
context = adal.AuthenticationContext('https://login.microsoftonline.com/' + tenant_id)
token_response = context.acquire_token_with_client_credentials('https://api.loganalytics.io', application_id, application_key)
access_token = token_response.get('accessToken')
# Add token to header
headers = {
"Authorization": 'Bearer ' + access_token,
"Content-Type": 'application/json'
}
search_params = {
"query": query
}
url = "https://api.loganalytics.io/v1/workspaces/{workspaceID}/query"
res = requests.post(url=url, json=search_params, headers=headers)
print (res.status_code)
print (res.json())

migrate from urllib2 to requests python 2.7

I am trying to take some working code and change from urlib2 to requests.
The original code provides basic login information of username, password and posts the KEY and SECRET in the header of the urllib2 request. The following code is my attempt to change to using the requests module and gain some functionality for making additional API calls. I have tried dozens of combinations and all return a code 400. Apparently, my requests code does not successfully furnish the needed information to return a 200 response and provide the needed authorization token.
## Import needed modules
import urllib2, urllib, base64
import httplib
import requests
import json
## initialize variables
KEY = "7f1xxxx-3xxx-4xxx-9a7f-8be66839dede"
SECRET = "45xxxxxx-45xxx-469a-9ae9-a7927a76cfeb"
userName = "my-email#xxx.com"
passWord = "mypassword"
URL = "https://company.com/auth/token"
token = None
sessionid = None
DATA = urllib.urlencode({"grant_type":"password",
"username":userName,
"password":passWord})
base64string = base64.encodestring('%s:%s' % (KEY, SECRET)).replace('\n', '')
request = urllib2.Request(URL, DATA)
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
token = result.read()
print token
This returns my authorization token and all is well. I can pass the token to the authorization server and have full access to the api for interacting with the database. Below is the attempt to use requests and have the added functions it provides.
client = requests.session()
payload = {"grant_type":"password",
"username":userName,
"password":passWord,
"applicationId": KEY
}
headers = {'content-type':'application/json',
"grant_type":"password",
"username":userName,
"password":passWord,
'applicationsId': KEY,
'Authorization': base64string,
'token': token,
'sessionid': sessionid
}
response = client.post(URL, params = payload, headers=headers)
token = response.content
print token
{"error":"invalid_request"}
print response
<Response [400]>
If you want to use basic auth you should use the method from requests..
Your post should look like
response = client.post(
URL,
params = payload,
headers=headers,
auth=HTTPBasicAuth(
KEY,
SECRET
))
Somewhere in a post a contributor to another question mentioned some items actually needed to be in the body of the request not in the header. I tried various combos and the following solved the 400 response and accomplished my goals.
data = {"grant_type":"password",
"username":userName,
"password":passWord,
"applicationId": KEY
}
headers = {'Authorization': "Basic %s" % base64string,
'token': token
}
response = client.post(URL, data = data, headers=headers)
token = response.text
print token