I am going through the doc
The following code explains the construction of the Authorization Header:
Authorization = "AWS" + " " + AWSAccessKeyId + ":" + Signature;
Signature = Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) );
StringToSign = HTTP-Verb + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Date + "\n" +
CanonicalizedAmzHeaders +
CanonicalizedResource;
CanonicalizedResource = [ "/" + Bucket ] +
<HTTP-Request-URI, from the protocol name up to the query string> +
[ subresource, if present. For example "?acl", "?location", "?logging", or "?torrent"];
CanonicalizedAmzHeaders = <described below>
And, in the Authentication Examples (for Example OBJECT GET), they have showStringToSign like the following:
GET\n
\n
\n
Tue, 27 Mar 2007 19:36:42 +0000\n
/johnsmith/photos/puppy.jpg
My question:
Why they have used three \n after GET ? Is there a reason behind that?
Also, can I write the above code as follows:
GET\n
\n
\n
Tue, 27 Mar 2007 19:36:42 GMT\n
Think of the "\n" as the separator. As the example is sending empty Content-MD5 and Content-Type, you'll see three \n in a row.
I don't know exactly what you are trying to do but please keep in mind that the AWS SDKs can handle the request signature so you don't have to.
Related
i want to play with the Amazon Selling Partner API. From Postman everything works fine. From Powershell not.
I tried to derive my powershell script from the documentation https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html and the python example.
I try it all the day, but i do not find the solution.
What i have done:
Test functionality from Postman -> Works
Read about Signing AWS requests with Signature Version 4 -> OK
Understanding signing key functionality with the test input from
https://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html -> Works
Try to translate the Python example to Powershell: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html -> does not work
Use Google to find a solution from examples and found this: http://www.laurierhodes.info/?q=node/114 -> does not work - maybe because it is for aws lambda
Whatever i do, i receive error Message Sender SignatureDoesNotMatch
I attached a screenshot from Postman, it shows the response from this endpoint. I know it is "Access denied", it is just really near on the example from amazon. Later on i need to make a sts request to receive temp credentials on each call and i also need to sign every call to the amazon selling partner API.
Maybe someone can help me, please? I'm going nuts :D
Here is my Powershell code from the python example: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
I tried both get variants,
Using GET with an authorization header (Python)
Using GET with authentication information in the Query string (Python)
This code is from the 2nd -> Using GET with authentication information in the Query string
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
[cultureinfo]::CurrentCulture = 'de-DE'
#powershell variant with example test input from #https://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html
<#
.Synopsis
HMACSHA256 signing function used in the construction of a "Signature 4 " request
#>
# translated function from Deriving a signing key using .NET (C#)
function hmacSHA256(
[string]$data,
[byte[]]$key)
{
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = $key
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($data))
return $signature
}
<#
.Synopsis
The AWS Signature version 4 creation routine
#>
function getSignatureKey(
[String]$AWSAccessKey,
[String]$dateStamp,
[String]$regionName,
[String]$serviceName)
{
[Byte[]]$kSecret = [System.Text.Encoding]::UTF8.GetBytes("AWS4" + $AWSAccessKey)
$kDate = hmacSHA256 -data $dateStamp -key $kSecret
$kRegion = hmacSHA256 -data $regionName -key $kDate
$kService = hmacSHA256 -data $serviceName -key $kRegion
$kSigningKey = hmacSHA256 -data "aws4_request" -key $kService
return $kSigningKey
}
<#
.Synopsis
Retrieves an SHA hash of a string as required by AWS Signature 4
#>
function hash($request) {
$sha256 = new-object -TypeName System.Security.Cryptography.SHA256Managed
$utf8 = new-object -TypeName System.Text.UTF8Encoding
$hash = [System.BitConverter]::ToString($sha256.ComputeHash($utf8.GetBytes($request)))
return $hash.replace('-','').toLower()
}
# ************* REQUEST VALUES *************
$method = 'GET'
$service = 'iam'
$host1 = 'iam.amazonaws.com'
$region = 'us-east-1'
$endpoint = 'https://iam.amazonaws.com'
$access_key = 'AKIA4KA7FVL7SN2EXAMPLE'
$secret_key = 'EXPL[enter image description here][1]Y59hS5KWKAnfOSnWLjNsiKaK/EXAMPLE'
$now = [DateTime]::UtcNow
$amz_date = $now.ToString('yyyyMMddTHHmmssZ')
$datestamp = $now.ToString('yyyyMMdd')
# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
# Step 2: Create canonical URI--the part of the URI from domain to query
# string (use '/' if no path)
$canonical_uri = '/'
# Step 3: Create the canonical headers and signed headers. Header names
# must be trimmed and lowercase, and sorted in code point order from
# low to high. Note trailing \n in canonical_headers.
# signed_headers is the list of headers that are being included
# as part of the signing process. For requests that use query strings,
# only "host" is included in the signed headers.
$canonical_headers = "host:" + $host1 + "`n"
$signed_headers = "host"
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
$algorithm = 'AWS4-HMAC-SHA256'
$credential_scope = $datestamp + '/' + $region + '/' + $service + '/' + 'aws4_request'
# Step 4: Create the canonical query string. In this example, request
# parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
$canonical_querystring = "Action=CreateUser&UserName=NewUser&Version=2010-05-08"
$canonical_querystring += "&X-Amz-Algorithm=AWS4-HMAC-SHA256 `n"
$canonical_querystring += "&X-Amz-Credential=" + [uri]::EscapeDataString(($access_key + "/" + $credential_scope))
$canonical_querystring += "&X-Amz-Date=" + $amz_date
$canonical_querystring += "&X-Amz-Expires=30"
$canonical_querystring += "&X-Amz-SignedHeaders=" + $signed_headers
# Step 5: Create payload hash. For GET requests, the payload is an
# empty string ("").
$payload_hash = hash ''
# Step 6: Combine elements to create canonical request
$canonical_request = $method + "`n" +
$canonical_uri + "`n" +
$canonical_querystring + "`n" +
$canonical_headers + "`n" +
$signed_headers + "`n" +
$payload_hash
#$canonical_request_hash = hash -request $canonical_request
#write-host $canonical_request_hash
# *************************************************************#
# ************* TASK 2: CREATE THE STRING TO SIGN *************#
$string_to_sign = $algorithm + "`n" +
$amz_date + "`n" +
$credential_scope + "`n" +
$canonical_request_hash
# *************************************************************#
# ************* TASK 3: CALCULATE THE SIGNATURE ***************#
# Create the signing key
$signing_key = getSignatureKey($secret_key, $datestamp, $region, $service)
#write-host "signing-key: $($signing_key)"
# Sign the string_to_sign using the signing_key
$signature = HmacSHA256 -data $string_to_sign -key $signing_key
$signature = [System.BitConverter]::ToString($signature).Replace('-','').ToLower()
# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The auth information can be either in a query string
# value or in a header named Authorization. This code shows how to put
# everything into a query string.
$canonical_querystring += '&X-Amz-Signature=' + $signature
# ************* SEND THE REQUEST *************
# The 'host' header is added automatically by the Python 'request' lib. But it
# must exist as a header in the request.
###
#I am not sure how to work with headers, because i used the Get Variant with authentication information in the Query string
####
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("authorization", "AWS4-HMAC-SHA256 Credential=$($AWSAccessID)/$($shortdate)/$($AWSRegion)/$($AWSService)/aws4_request, SignedHeaders=$($SignedHeadersList), Signature=$($signature)")
$request_url = $endpoint + "/?" + $canonical_querystring
Invoke-RestMethod $request_url -Method 'GET' -Headers $headers
Best regards and many many thanks
Patrick
Postman screenshot
I have a private API gateway with a / endpoint and a /foo with IAM auth enabled.
I created a policy which I attached to my instance's role :
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"execute-api:Invoke"
],
"Resource": [
"arn:aws:execute-api:*:*:*"
]
}
]
}
I have some code to do the AWS signature stuff and also used Postman to create a snippet with the same key/secret/session token. Both give the same result on /foo. It always says :
{"message":"The security token included in the request is invalid"}
I had a concern that the docs do not say you can attach the policy to a role only a user or group.
https://aws.amazon.com/premiumsupport/knowledge-center/iam-authentication-api-gateway/
That whole page doesn't mention roles once... Can I attach a policy to a role to use it with IAM auth'ed API gateway??
The / endpoint returns me a 200 response, and my API GW resource policy denies/allows access to *. If I can get to /, I can get to /foo. (And if I disable the IAM auth, I can get /foo)
The VPC endpoint allows * on *.
In the execution logs I see nothing for the failed attempts.
The attempts to / log : API Key authorized because method 'GET /' does not require API Key. Request will not contribute to throttle or quota limits
And I can see the X-Amz-Security-Token in the payload.
But requests to /foo don't appear there, only the access logs. And I've added some fields but nothing that sheds any light on the problem.
Anything I'm forgetting?? And any ideas why isn't it working?
Here is my signing python, there may be a bug, but it is getting the same error as Postman, which makes me think not! Replace the host/endpoint and path to your own. I added a few print debug lines to show the intermediate steps because I did get some errors about the canonical URL being wrong to start with.
#!/usr/bin/python3
# This is based on the AWS General Reference
# Signing AWS API Requests top available at
# https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
# To use it :
# pip3 install requests
# To use it on instance with instance role :
# sudo yum -y -q install jq
# export AWS_ACCESS_KEY_ID=$(curl 169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance | jq -r .AccessKeyId)
# export AWS_SECRET_ACCESS_KEY=$(curl 169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance | jq -r .SecretAccessKey)
# export AWS_SESSION_TOKEN=$(curl 169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance | jq -r .Token)
# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac
import requests # pip install requests
# ************* REQUEST VALUES *************
method = 'GET'
service = 'execute-api'
host = 'xxx.execute-api.eu-west-2.amazonaws.com'
region = 'eu-west-2'
endpoint = 'https://xxx.execute-api.eu-west-2.amazonaws.com'
path='/stage/foo/'
request_parameters = ''
# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = os.environ.get('AWS_ACCESS_KEY_ID')
secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY')
session_token = os.environ.get('AWS_SESSION_TOKEN')
if access_key is None or secret_key is None:
print('No access key is available.')
sys.exit()
# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope
# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
# Step 1 is to define the verb (GET, POST, etc.)--already done.
# Step 2: Create canonical URI--the part of the URI from domain to query
# string (use '/' if no path)
canonical_uri = path
# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters
# Step 4: Create the canonical headers and signed headers. Header names
# must be trimmed and lowercase, and sorted in code point order from
# low to high. Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amzdate + '\n' 'x-amz-security-token:' + session_token + '\n'
# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-date;x-amz-security-token'
# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").
payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()
# Step 7: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
print ("CANONICAL REQUEST : " + canonical_request)
print ()
# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' + amzdate + '\n' + credential_scope + '\n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
print ("STRING TO SIGN : " + string_to_sign )
# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)
# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()
# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
# The request can include any headers, but MUST include "host", "x-amz-date",
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-date':amzdate, 'Authorization':authorization_header, 'X-Amz-Security-Token':session_token}
# ************* SEND THE REQUEST *************
request_url = endpoint + path + canonical_querystring
print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
print('Headers = ' + str(headers))
r = requests.get(request_url, headers=headers)
print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)
Aha, I found an answer. I was pulling the credentials from the wrong endpoint. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html shows "identity-credentials/ec2/security-credentials/ec2-instance" are "Internal use only". There is an iam/security-credentials/{role} that works a LOT better!!
I am trying to login using social logins like google,github,facebook but when i am trying that in case of LinkedIn i am getting the following error
HTTPError at /social/linkedin_login/
HTTP Error 404: Not Found
I am user v2 for linkedin
views.py
required_info = "id,first-name,last-name,email-address,location,positions,educations,industry,summary,public-profile-url,picture-urls::(original)"
rty = "https://api.linkedin.com/v2/me:(" + required_info + ")" + \
"?format=json&oauth2_access_token="
rty += accesstoken
details = ur.urlopen(rty).read().decode('utf8')
I am getting error in details = ur.urlopen(rty).read().decode('utf8')
else:
rty = "https://www.linkedin.com/uas/oauth2/authorization?response_type=code&client_id=" + \
settings.LN_API_KEY
rty += "&scope=r_liteprofile r_emailaddress w_member_social&state=8897239179ramya"
rty += "&redirect_uri=" + request.scheme + '://' + \
request.META['HTTP_HOST'] + reverse('social:linkedin_login')
return HttpResponseRedirect(rty)
Can someone help me out of this error.
The format you are using resembles LinkedIn's v1 API. You will need to update your request URL to be https://api.linkedin.com/v2/me, with the OAuth 2.0 access token added to the Authorization: Bearer header.
I used this script to check the last redirect url:
set http = CreateObject("Chilkat_9_5_0.Http")
http.FollowRedirects = 0
' resp is a Chilkat_9_5_0.HttpResponse
Set resp = http.QuickGetObj(link)
If (http.LastMethodSuccess = 0) Then
wscript.echo http.LastErrorText
WScript.Quit
End If
status = resp.StatusCode
wscript.echo "HTTP Response Status: " & status
wscript.echo "Redirect URL >>>: " & http.FinalRedirectUrl
nextUrl = http.WasRedirected
loopCount = 0
Do While (status = 302 OR status = 301)
wscript.echo "HTTP Response Status: " & status
wscript.echo "Redirect URL >>>: " & http.FinalRedirectUrl
nextUrl = http.FinalRedirectUrl
' resp is a Chilkat_9_5_0.HttpResponse
Set resp = http.QuickGetObj(nextUrl)
If (http.LastMethodSuccess = 0) Then
wscript.echo http.LastErrorText
WScript.Quit
End If
status = resp.StatusCode
wscript.echo "HTTP Response Status: " & status
' For safety, prevent infinite loops by
' keeping a loopCount and only allows following a max
' of 10 redirects:
loopCount = loopCount + 1
If (loopCount > 10) Then
wscript.echo "Too many redirects."
WScript.Quit
End If
Loop
This code prints the 301 status code correctly, but it doesn't print the http.FinalRedirectURL (it's blank).
I also tried with the url https://www.businessonline.it/articoli/guida-al-nuovo-regime-forfettario-2016-e-2017.html
The output is:
Link: https://www.businessonline.it/articoli/guida-al-nuovo-regime-forfettario-2016-e-2017.html
HTTP Response Status: 301
Redirect URL >>>:
HTTP Response Status: 301
Redirect URL >>>:
ChilkatLog:
QuickGetObj:
DllDate: Oct 28 2019
ChilkatVersion: 9.5.0.80
UnlockPrefix: BRKEVN.XXXXXXXXXXX
Architecture: Little Endian; 32-bit
Language: ActiveX
VerboseLogging: 0
Component successfully unlocked using purchased unlock code.
url:
verb: GET
quickRequestDb:
url:
getHttpConnectionByUrl:
urlObject_loadUrl:
No domain in URL
url:
--urlObject_loadUrl
--getHttpConnectionByUrl
--quickRequestDb
Failed.
--QuickGetObj
--ChilkatLog
I used both chilkat 9.5.0.70 and 9.5.0.80.
If I set FollowRedirects to 1, the first StatusCode is "200" and I can't get the url.
Can you help me?
Thank's
Thanks Lorenzo,
The problem had to do with QuickGetObj and QuickGet. The QuickGetStr method worked fine for the WasRedirected and FinalRedirectUrl properties.
Here's a new 32-bit ActiveX build with the fix:
32-bit: https://chilkatdownload.com/prerelease/chilkatax-9.5.0-win32-1630.zip
I'm currently writing my own C++ HTTP class for a certain project. And I'm trying to find a way to separate the response body from the header, because that's the only part I need to return.
Here's a sample of the raw http headers if you're not familiar with it:
HTTP/1.1 200 OK
Server: nginx/0.7.65
Date: Wed, 29 Dec 2010 06:13:07 GMT
Content-Type: text
Connection: keep-alive
Vary: Cookie
Content-Length: 82
Below that is the HTML/Response body. What would be the best way to do this? I'm only using Winsock library for the requests by the way (I don't even think this matters).
Thanks in advance.
HTTP headers are terminated by the sequence \r\n\r\n (a blank line). Just search for that, and return everything after. (It may not exist of course, e.g. if it was in response to a HEAD request.)
Do you need to roll your own? There are C/C++ libraries out there for doing HTTP, e.g. libcurl. If you need to support the full gamut of HTTP, then it's not always a simple delineation. You might also have to cater, for example, for chunked encoding.
DO IF Socket.IsServerReady(Sock) THEN Text = text + Socket.Read(Sock, 65000) 'print text '' 32000 bytes... whatever they give us Bytes = bytes + Socket.Transferred StatusBar.Panel(0).Caption = "Bytes Read: " + STR$(Bytes)
END IF
'RichEdit.addstrings text zzz=Bytes LOOP UNTIL Socket.Transferred = 0 RichEdit.Clear RichEdit.Text = text Socket.Close(Sock) dim mem as qmemorystream dim S$ as string S$ = text for n=0 to 400 buff$=mid$(S$,n,5)
if buff$="alive" then' found end of headers richedit1.addstrings (buff$) richedit1.addstrings (mid$(S$,n,9)) richedit1.addstrings str$(n+9) zzz=n+8'offset + 8 bit space after headers and before Bof end if next n Mem.WriteStr(S$, LEN(S$))'write entire file to memory Mem.Position = zzz ' use offset as Start position S$ = Mem.ReadStr(LEN(S$)) ' read rest of file into string till Eof Mem.Close' dont forget to close 'PRINT S$ '' print it
Filex.Open("c:/CAP.AVI", fmCreate)'create file on system filex.WriteBinStr(S$,len(S$)-zzz)' write to it filex.close 'dont forget to close