Python is inserting newline characters into requests header - django

Some background. This code takes a data from a data collection script and then posts it to a secured REST API in Django. I have an api key generated and it is the only line in the file api.key. I also have the url to post to in the file post.url (it looks like http://example.com/api/ and then I concatenate the proper api node name on the end).
The code is below for my solar data api node (posts data collected from solar panels)
import gather_solar as gs
import requests
import json
import os
def post_solar():
print("DEBUG: start solar")
data = gs.gather_solar()
api_key = None
url = None
try:
here = os.path.dirname(os.path.abspath(__file__))
filename = os.path.join(here, 'api.key')
file = open(filename, "r")
api_key = file.readline()
api_key.replace('\n', '')
except Exception as e:
print("ERROR: " + str(e))
try:
here = os.path.dirname(os.path.abspath(__file__))
filename = os.path.join(here, 'post.url') #server will use different url
file = open(filename, "r")
url = file.readline()
url.replace('\n', '')
except Exception as e:
print("ERROR: " + str(e))
if api_key is not None and url is not None:
authorization = "Token " + api_key
authorization.replace('\n', '')
headers = {
'Content-Type': 'application/json; charset=UTF-8',
'Authorization': authorization
}
json_data = json.dumps(data)
url += "solar/"
url.replace('\n', '')
print(url)
req = requests.Request('POST', url, data=json_data, headers=headers)
prepared = req.prepare()
print("DEBUG: POST Headers: " + str(prepared.headers))
print("DEBUG: POST Body: " + str(prepared.body))
s = requests.Session()
response = s.send(prepared)
print("DEBUG: Response Code: " + str(response.status_code))
print("DEBUG: Response Headers: " + str(response.headers))
print("DEBUG: Response Data: " + str(response.json()))
else:
print("DEBUG: Error with API key")
print("DEBUG: end solar")
I am running code on an ubuntu server via AWS and have Apache 2 setup and running. However, whenever I run this script, I get an error saying that my token is invalid and it shows the token as "Token abcd...abcd\n" . This is especially frustrating because I don't have this issue when I run the script locally (visual studio code on Win10) As you can see I have tried to remove any newlines wherever possible but it doesn't seem to be helping. Any help would be greatly appreciated

replace() does not change the string (strings are immutable); it returns a new string. So:
api_key = file.readline()
api_key.replace('\n', '')
leaves the \n on api_key and you are ignoring the new string returned by replace().
You could just string the line before assignment:
api_key = file.readline().strip()

Related

Not able to connect to the translation API

url: https://translatorappeagle.cognitiveservices.azure.com/
I'm also using my key from my resources page. I get a an error 404: not found. I copied the code from the getting started section for the translation API.
Please let me know what to do.
import os, requests, uuid, json
key_var_name = 'TRANSLATOR_TEXT_SUBSCRIPTION_KEY'
if not key_var_name in os.environ:
raise Exception('Please set/export the environment variable: {}'.format(key_var_name))
subscription_key = os.environ[key_var_name]
endpoint_var_name = 'TRANSLATOR_TEXT_ENDPOINT'
if not endpoint_var_name in os.environ:
raise Exception('Please set/export the environment variable: {}'.format(endpoint_var_name))
endpoint = os.environ[endpoint_var_name]
# If you encounter any issues with the base_url or path, make sure
# that you are using the latest endpoint: https://learn.microsoft.com/azure/cognitive-services/translator/reference/v3-0-translate
path = '/translate?api-version=3.0'
params = '&to=de&to=it'
constructed_url = endpoint + path + params
headers = {
'Ocp-Apim-Subscription-Key': subscription_key,
'Content-type': 'application/json',
'X-ClientTraceId': str(uuid.uuid4())
}
# You can pass more than one object in body.
body = [{
'text' : 'Hello World!'
}]
request = requests.post(constructed_url, headers=headers, json=body)
response = request.json()
print(json.dumps(response, sort_keys=True, indent=4, separators=(',', ': ')))

Why does my server complain of 'Bad authorization header'?

I am having trouble getting my client and server for a simple AWS Cognito authorization to agree on what the token should look like.
My client uses boto3 and requests. I pass the authorization in the header which I've seen in more than half of tutorials but I've also seen use of the auth parameter which I am not using.
Here's my server:
from flask import request, jsonify, make_response
import flask.json
from flask_jwt_extended import (
JWTManager, jwt_required, create_access_token,
get_jwt_identity
)
app = flask.Flask(__name__)
# from: https://flask-jwt-extended.readthedocs.io/en/stable/basic_usage/
# Setup the Flask-JWT-Extended extension
app.config['JWT_SECRET_KEY'] = 'super-secret' # Change this!
jwt = JWTManager(app)
#app.route("/auth_test")
#jwt_required
def auth_test ():
current_user = get_jwt_identity()
return jsonify({"current_user": "current_user"})
if __name__ == "__main__":
app.run(debug=True)
One clue here is that JWT_SECRET_KEY is copied from a tutorial but I'm unclear on what I need to change it to. Perhaps this is my issue but if it is I don't see the solution or what to change it to.
Here's my client:
import requests
import boto3
host = "http://127.0.0.1:8000"
region = '...'
user_pool_id = '...'
username = '...'
password = '...'
app_client_id = '...'
client = boto3.client('cognito-idp', region_name = region)
auth_response = client.admin_initiate_auth(
UserPoolId = user_pool_id,
ClientId = app_client_id,
AuthFlow = 'ADMIN_NO_SRP_AUTH',
AuthParameters = {
'USERNAME' : username,
'PASSWORD' : password
}
)
id_token = auth_response['AuthenticationResult']['IdToken']
response = requests.get(host + "/auth_test" , headers = { 'Authorization': id_token })
print(response.json())
It prints:
{'msg': "Bad Authorization header. Expected value 'Bearer '"}
This indicates that I should be passing "Bearer " + id_token instead of just id_token but this is contrary to most tutorials I've read and furthermore when I do that I still get the same error.
Anyone know what I'm doing wrong?
EDIT:
I have made some changes due to the comments. I have added
app.config['JWT_ALGORITHM'] = 'RS256'
app.config['JWT_DECODE_ALGORITHMS'] = ['RS256']
#app.config['JWT_PUBLIC_KEY'] = '...' # value that I got from https://cognito-idp.us-east-1.amazonaws.com/<my pool id>/.well-known/jwks.json. There were two entries I tried both values found in kid
RS256 matches Cognito's encoding so I think I need to set that for my server. However I now simply get this error when running the demo:
ValueError: Could not deserialize key data.
Thank you for the help!

Amazon Lightsail Oauth2 Google signup generates Error: invalid_request Permission denied

#app.route('/login')
def showLogin():
state = ''.join(random.choice(string.ascii_uppercase + string.digits)
for x in xrange(32))
login_session['state'] = state
# return "The current session state is %s" % login_session['state']
return render_template('login.html', STATE=state)
#app.route('/gconnect', methods=['POST'])
def gconnect():
# Validate state token
if request.args.get('state') != login_session['state']:
response = make_response(json.dumps('Invalid state parameter.'), 401)
response.headers['Content-Type'] = 'application/json'
return response
# Obtain authorization code
code = request.data
try:
# Upgrade the authorization code into a credentials object
oauth_flow = flow_from_clientsecrets('client_secrets.json', scope='')
oauth_flow.redirect_uri = 'postmessage'
credentials = oauth_flow.step2_exchange(code)
except FlowExchangeError:
response = make_response(
json.dumps('Failed to upgrade the authorization code.'), 401)
response.headers['Content-Type'] = 'application/json'
return response
# Check that the access token is valid.
access_token = credentials.access_token
url = ('https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=%s'
% access_token)
h = httplib2.Http()
result = json.loads(h.request(url, 'GET')[1])
# If there was an error in the access token info, abort.
if result.get('error') is not None:
response = make_response(json.dumps(result.get('error')), 500)
response.headers['Content-Type'] = 'application/json'
return response
# Verify that the access token is used for the intended user.
gplus_id = credentials.id_token['sub']
if result['user_id'] != gplus_id:
response = make_response(
json.dumps("Token's user ID doesn't match given user ID."), 401)
response.headers['Content-Type'] = 'application/json'
return response
# Verify that the access token is valid for this app.
if result['issued_to'] != CLIENT_ID:
response = make_response(
json.dumps("Token's client ID does not match app's."), 401)
print "Token's client ID does not match app's."
response.headers['Content-Type'] = 'application/json'
return response
stored_access_token = login_session.get('access_token')
stored_gplus_id = login_session.get('gplus_id')
if stored_access_token is not None and gplus_id == stored_gplus_id:
response = make_response(json.dumps('Current user is connected'), 200)
response.headers['Content-Type'] = 'application/json'
return response
# Store the access token in the session for later use.
login_session['access_token'] = credentials.access_token
login_session['gplus_id'] = gplus_id
# Get user info
userinfo_url = "https://www.googleapis.com/oauth2/v1/userinfo"
params = {'access_token': credentials.access_token, 'alt': 'json'}
answer = requests.get(userinfo_url, params=params)
data = answer.json()
login_session['username'] = data['name']
login_session['picture'] = data['picture']
login_session['email'] = data['email']
# See if a user exists, if it doesn't make a new one
user_id = getUserID(login_session['email'])
if not user_id:
user_id = createUser(login_session)
login_session['user_id'] = user_id
output = ''
output += '<h1>Welcome, '
output += login_session['username']
output += '!</h1>'
output += '<img src="'
output += login_session['picture']
output += ' " style = "width: 300px; height: 300px;border-radius: 150px;'
output += '-webkit-border-radius: 150px;-moz-border-radius: 150px;"> '
print "done!"
return output
I ran this OAuth code on my local machine. It worked but on Amazon Lightsail instance it gave the following error:
*Error: invalid_request Permission denied to generate login hint for target domain.
I even corrected my Redirect Uris and JAVA Script Origins to .com but it still gives the same error even on erasing the cookies.

How to fetch data from OMS workspace

I read the documentation yesterday and done some coding with python to fetch data in the following way. It's working fine.
import logging as log
import adal
import requests
import json
import datetime
from pprint import pprint
# Details of workspace. Fill in details for your workspace.
resource_group = 'Test'
workspace = 'FirstMyWorkspace'
# Details of query. Modify these to your requirements.
query = "Type=*"
end_time = datetime.datetime.utcnow()
start_time = end_time - datetime.timedelta(hours=24)
num_results = 2 # If not provided, a default of 10 results will be used.
# IDs for authentication. Fill in values for your service principal.
subscription_id = '{subscription_id}'
tenant_id = '{tenant_id}'
application_id = '{application_id}'
application_key = '{application_key}'
# URLs for authentication
authentication_endpoint = 'https://login.microsoftonline.com/'
resource = 'https://management.core.windows.net/'
# Get access token
context = adal.AuthenticationContext('https://login.microsoftonline.com/' + tenant_id)
token_response = context.acquire_token_with_client_credentials('https://management.core.windows.net/', application_id, application_key)
access_token = token_response.get('accessToken')
# Add token to header
headers = {
"Authorization": 'Bearer ' + access_token,
"Content-Type": 'application/json'
}
# URLs for retrieving data
uri_base = 'https://management.azure.com'
uri_api = 'api-version=2015-11-01-preview'
uri_subscription = 'https://management.azure.com/subscriptions/' + subscription_id
uri_resourcegroup = uri_subscription + '/resourcegroups/'+ resource_group
uri_workspace = uri_resourcegroup + '/providers/Microsoft.OperationalInsights/workspaces/' + workspace
uri_search = uri_workspace + '/search'
# Build search parameters from query details
search_params = {
"query": query,
"top": num_results
}
# Build URL and send post request
uri = uri_search + '?' + uri_api
response = requests.post(uri, json=search_params,headers=headers)
# Response of 200 if successful
if response.status_code == 200:
# Parse the response to get the ID and status
data = response.json()
if data.get("__metadata", {}).get("resultType", "") == "error":
log.warn("oms_fetcher;fetch_job;error: " + ''.join('{}={}, '.format(key, val) for key, val in
data.get("error", {}).items()))
else:
print data["value"]
search_id = data["id"].split("/")
id = search_id[len(search_id)-1]
status = data["__metadata"]["Status"]
print status
# If status is pending, then keep checking until complete
while status == "Pending":
# Build URL to get search from ID and send request
uri_search = uri_search + '/' + id
uri = uri_search + '?' + uri_api
response = requests.get(uri, headers=headers)
# Parse the response to get the status
data = response.json()
status = data["__metadata"]["Status"]
print id
else:
# Request failed
print (response.status_code)
response.raise_for_status()
Today I went to the same webpage that I have followed yesterday but there is a different documentation today. So do I need to follow the new documentation? I tried new documentation too but got into an issue
url = "https://api.loganalytics.io/v1/workspaces/{workspace_id}/query"
headers = {
"X-Api-Key": "{api_key}",
"Content-Type": 'application/json'
}
search_param = {
}
res = requests.post(url=url, json=search_param, headers=headers)
print res.status_code
print res.json()
{u'error': {u'innererror': {u'message': u'The given API Key is not
valid for the request', u'code': u'UnsupportedKeyError'}, u'message':
u'Valid authentication was not provided', u'code':
u'AuthorizationRequiredError'}}
Here is the link to documentation
The api_key is not oms primary key on Portal. You could check example in this link. The token should like below:
Authorization: Bearer <access token>
So, you need modify X-Api-Key": "{api_key} to Authorization: Bearer <access token>.
You need create a service principal firstly, please check this link.
Then, you could use the sp to get token, please check this link.
Note: You could your code to get token, but you need modify the resource to https://api.loganalytics.io. Like below:
# Get access token
context = adal.AuthenticationContext('https://login.microsoftonline.com/' + tenant_id)
token_response = context.acquire_token_with_client_credentials('https://api.loganalytics.io', application_id, application_key)
access_token = token_response.get('accessToken')
# Add token to header
headers = {
"Authorization": 'Bearer ' + access_token,
"Content-Type": 'application/json'
}
Working Prototype to Query OMS or Log Analytic workspace.
import adal
import requests
import json
import datetime
from pprint import pprint
# Details of workspace. Fill in details for your workspace.
resource_group = 'xxx'
workspace = 'xxx'
workspaceid = 'xxxx'
# Details of query. Modify these to your requirements.
query = "AzureActivity | limit 10"
# IDs for authentication. Fill in values for your service principal.
subscription_id = 'xxxx'
# subscription_id = 'xxxx'
tenant_id = 'xxxx'
application_id = 'xxxx'
application_key = 'xxxxx'
# Get access token
context = adal.AuthenticationContext('https://login.microsoftonline.com/' + tenant_id)
token_response = context.acquire_token_with_client_credentials('https://api.loganalytics.io', application_id, application_key)
access_token = token_response.get('accessToken')
# Add token to header
headers = {
"Authorization": 'Bearer ' + access_token,
"Content-Type": 'application/json'
}
search_params = {
"query": query
}
url = "https://api.loganalytics.io/v1/workspaces/{workspaceID}/query"
res = requests.post(url=url, json=search_params, headers=headers)
print (res.status_code)
print (res.json())

migrate from urllib2 to requests python 2.7

I am trying to take some working code and change from urlib2 to requests.
The original code provides basic login information of username, password and posts the KEY and SECRET in the header of the urllib2 request. The following code is my attempt to change to using the requests module and gain some functionality for making additional API calls. I have tried dozens of combinations and all return a code 400. Apparently, my requests code does not successfully furnish the needed information to return a 200 response and provide the needed authorization token.
## Import needed modules
import urllib2, urllib, base64
import httplib
import requests
import json
## initialize variables
KEY = "7f1xxxx-3xxx-4xxx-9a7f-8be66839dede"
SECRET = "45xxxxxx-45xxx-469a-9ae9-a7927a76cfeb"
userName = "my-email#xxx.com"
passWord = "mypassword"
URL = "https://company.com/auth/token"
token = None
sessionid = None
DATA = urllib.urlencode({"grant_type":"password",
"username":userName,
"password":passWord})
base64string = base64.encodestring('%s:%s' % (KEY, SECRET)).replace('\n', '')
request = urllib2.Request(URL, DATA)
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
token = result.read()
print token
This returns my authorization token and all is well. I can pass the token to the authorization server and have full access to the api for interacting with the database. Below is the attempt to use requests and have the added functions it provides.
client = requests.session()
payload = {"grant_type":"password",
"username":userName,
"password":passWord,
"applicationId": KEY
}
headers = {'content-type':'application/json',
"grant_type":"password",
"username":userName,
"password":passWord,
'applicationsId': KEY,
'Authorization': base64string,
'token': token,
'sessionid': sessionid
}
response = client.post(URL, params = payload, headers=headers)
token = response.content
print token
{"error":"invalid_request"}
print response
<Response [400]>
If you want to use basic auth you should use the method from requests..
Your post should look like
response = client.post(
URL,
params = payload,
headers=headers,
auth=HTTPBasicAuth(
KEY,
SECRET
))
Somewhere in a post a contributor to another question mentioned some items actually needed to be in the body of the request not in the header. I tried various combos and the following solved the 400 response and accomplished my goals.
data = {"grant_type":"password",
"username":userName,
"password":passWord,
"applicationId": KEY
}
headers = {'Authorization': "Basic %s" % base64string,
'token': token
}
response = client.post(URL, data = data, headers=headers)
token = response.text
print token