I am trying to match the api signatures with the following as:
API-Sign = Message signature using HMAC-SHA512 of (URI path + SHA256(nonce + POST data)) and base64 decoded secret API key
my code as of now
<cfset x = createobject("java","java.lang.System").currentTimeMillis()>
<cfset sts = "https://api.exmaple.com/records" & hash(x, "SHA-512", "UTF-8")>
<cfset sig = tobase64(hmac(sts, "#secret_key#", "HMACSHA256"))>
but something is wrong, unable to validate, getting internal error
Related
I'm getting "TypeError: Cannot read property 'sigBytes' of undefined" error when running a postman collection with a cryptojs pre-request script. The pre-request script computes a hmac-sha256 signature which is also part of parameters of the main API to be called.
Below is my script:
let mobile = pm.environment.get('mobileNumber');
let value = pm.environment.get('value');
let merchantId = pm.environment.get('merchantId');
let referenceNumber = pm.environment.get('referenceNumber');
let authCode = pm.environment.get('authCode');
let secretKey = pm.environment.get('secretKey');
let string = mobile + value + merchantId + referenceNumber + authCode;
pm.environment.set('string', string);
let hmac = CryptoJS.HmacSHA256(string, secretKey);
pm.environment.set('hmac', hmac);
signature = "hmac256-" + hmac;
pm.environment.set('signature', signature);
I already tried encoding the string and secretKey to UTF-8, base64stringify, JSON stringify but I'm getting errors still.
Here are sample values from each of the variables:
mobile - +639012345678
value - 100
merchantId - TEST_MERCHANT1
referenceNumber - TEST_MERCHANT1-000001
authCode - 000001
secretKey - a2c36909-c9cc-4ed3-9423-ec170e1eb6c2
Looks like at least one of your environment variables isn't defined.
The error message is due to a bug in CryptoJS: https://github.com/brix/crypto-js/issues/85
I have seen pre-signed URL for S3 object. Is it possible to create pre-signed URL for API gateway. I have gone through documentation. I am using .NET. I would like to know if there is .NET library available to create pre-signed request for gateway API.
ISSUE
I have GET API something like this https://xxxxxx.execute-api.us-east-1.amazonaws.com/dev/pets?type=dog&page=1 and our client is going to invoke that API once in a while. The legacy tool that they are using only supports GET. So i wanted to create a pre-signed URL (with short expiry time) and give them when they ask for it. For each client i already have IAM user with their respective accesskey and secretkey
PreSigned URLs are typically signed with AWS SigV4 signing process.
You can generate SigV4 signed Urls for your API Gateway Hosted Endpoints. Typically, you will need to send SigV4 signature in Authorization Request Header. If you are clients are willing to send header, here is one sample Library you can try for .NET which creates a HTTP Request with signed header.
If your clients cannot send Authorization Header or cannot use above library then you can convert the signature to be a Query String Format and provide the pre-signed Urls to them.
This AWS Documentation has example in Python on how to generate Query String URL. Now, you can take python example and convert into .NET based code with following sample.
public string GetSig4QueryString(string host, string service, string region)
{
var t = DateTimeOffset.UtcNow;
var amzdate = t.ToString("yyyyMMddTHHmmssZ");
var datestamp = t.ToString("yyyyMMdd");
var canonical_uri = "/dev/myApigNodeJS";
var canonical_headers = "host:" + host+"\n";
var signed_headers = "host";
var credential_scope = $"{datestamp}/{region}/{service}/aws4_request";
var canonical_querystring = "X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=" + WebUtility.UrlEncode(_access_key + "/" + credential_scope)
+ "&X-Amz-Date=" + amzdate + "&X-Amz-SignedHeaders=" + signed_headers;
Console.WriteLine("canonical_querystring");
Console.WriteLine(canonical_querystring);
var payload_hash = Hash(new byte[0]);//No Payload for GET
var canonical_request = new StringBuilder();
canonical_request.Append("GET\n");
canonical_request.Append(canonical_uri + "\n");
canonical_request.Append(canonical_querystring + "\n");
canonical_request.Append(canonical_headers + "\n");
canonical_request.Append(signed_headers + "\n");
canonical_request.Append(payload_hash);
Console.WriteLine("canonical_request");
Console.WriteLine(canonical_request);
var string_to_sign = $"{algorithm}\n{amzdate}\n{credential_scope}\n" + Hash(Encoding.UTF8.GetBytes(canonical_request.ToString()));
Console.WriteLine("string_to_sign");
Console.WriteLine(string_to_sign);
var signing_key = GetSignatureKey(_secret_key, datestamp, region, service);
var signature = ToHexString(HmacSHA256(signing_key, string_to_sign));
var signed_querystring = canonical_querystring+"&X-Amz-Signature=" + signature;
return signed_querystring;
}
GetSig4QueryString("myApiId.execute-api.us-east-1.amazonaws.com","execute-api","us-east-1");
//Returned String --> X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential= AKIAIOSFODNN7EXAMPLE%2F20190104%2Fus-east-1%2Fexecute-api%2Faws4_request&X-Amz-Date=20190104T190309Z&X-Amz-SignedHeaders=host&X-Amz-Signature=7b830fce28f7800b3879a25850950f6c4247dfdc07775b6952295fa2fff03f7f
Full Endpoint Becomes -
https://myApiId.execute-api.us-east-1.amazonaws.com/dev/myApigNodeJS?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20190104%2Fus-east-1%2Fexecute-api%2Faws4_request&X-Amz-Date=20190104T190309Z&X-Amz-SignedHeaders=host&X-Amz-Signature=7b830fce28f7800b3879a25850950f6c4247dfdc07775b6952295fa2fff03f7f
Note -
This example code refers methods & variables from Github project I gave above.
Also, this example hard coded API Path /dev/myApigNodeJS and signs it and it will be different for you with full absolute path.
AWS recommends to sign all queryStrings, headers which you are planning to send in request. Go through .NET code of library I referred and understand how its doing that.
Let me know if you have questions.
When generating the presigned url a websocket service within the AWS API Gateway, I used the solution by Imran and added the "X-Amz-Security-Token" which is required.
I'm using Python Requests to call an API using GET
I generate a signature using hmac and hashlib which results in a signature that looks like:
4epwTDKhWcIJL6bMM5f2hmBrOoXXIGD9UwX8ErfYzqU%3D
When I use Requests and specify the params as a string, the API call is successful because the signature is used as-is and doesn't change.
However when I use Requests and specify the params as a dictionary, the signature above is somehow URL encoded again, resulting in the % being encoded to %25, causing the signature to display as follows (note the %25):
4epwTDKhWcIJL6bMM5f2hmBrOoXXIGD9UwX8ErfYzqU%253D
For more context, here is the string params:
url = 'https://example.com/API'
payload = '¶meterA=valueA¶meterB=valueB¶meterC=valueC& apikey='+apikey+'&salt='+salt+'&signature='+sig
#payload = {'parameterA': 'valueA', 'parameterB': 'valueB', 'apikey': apikey, 'salt': salt, 'signature': sig}
r = requests.get(url, params=payload)
print r.url
results in a URL of:
https://example.com/API&
¶meterA=valueA
¶meterB=valueB
&apikey=0e1026af-40ce-e354-f1f2-72d280ca122
&salt=12345
&signature=4epwTDKhWcIJL6bMM5f2hmBrOoXXIGD9UwX8ErfYzqU%3D
Now, the dictionary params:
url = 'https://example.com/API'
payload = {'parameterA': 'valueA', 'parameterB': 'valueB', 'apikey': apikey, 'salt': salt, 'signature': sig}
r = requests.get(url, params=payload)
print r.url
results in a URL of:
https://example.com/API
&signature=4epwTDKhWcIJL6bMM5f2hmBrOoXXIGD9UwX8ErfYzqU%253D
&salt=12345
&apikey=0e1026af-40ce-e354-f1f2-72d280ca122
¶meterA=valueA
¶meterB=valueB
Note again that the % in the signature has changed to %25
Thanks!
When passing a dictionary to params it gets url encoded. Your string is url encoded already so it gets double encoded, resulting in a malformed 'signature'.
You can use urllib.unquote to decode sig (or you can just replace '%3D' with '=')
payload = {
'parameterA': 'valueA', 'parameterB': 'valueB',
'apikey': apikey, 'salt': salt, 'signature': urllib.unquote(sig)
}
Note that in python3 unquote is located in urllib.parse.
I am attempting to use a GET request to use the Amazon Mechanical Turk GetFileUploadURL function. However, I get this error code when attempting it.
AWS.NotAuthorized The identity contained in the request is not authorized to use this AWSAccessKeyId
This is the code I'm using to create the request.
now = DateTime.now
#For creating the signature hash
data = "AWSMechanicalTurkRequesterGetFileUploadURL" + now.to_s
sha256 = OpenSSL::Digest::SHA256.new
sig = OpenSSL::HMAC.digest(sha256, Rails.configuration.secret_key, data)
signature = Base64.encode64(sig)
puts "https://mechanicalturk.amazonaws.com", "/?Service=AWSMechanicalTurkRequester&AWSAccessKeyId=#{Rails.configuration.aws_key}&Version=2014-08-15&Operation=GetFileUploadURL&Signature=#{sig}&Timestamp=#{now}&AssignmentId=#{mturk_results[0][:AssignmentId]}&QuestionIdentifier=file1"
puts "\n\n"
uri = URI('https://mechanicalturk.amazonaws.com')
params = {:Service=>"AWSMechanicalTurkRequester", :AWSAccessKeyId=>Rails.configuration.aws_key, :Version=>"2014-08-15", :Operation=>"GetFileUPloadURL", :Signature=>sig, :Timestamp=>now, :AssignmentId=>mturk_results[0][:AssignmentId], :QuestionIdentifier=>"file1"}
uri.query = URI.encode_www_form(params)
res = 0
Net::HTTP.start(uri.host, uri.port,
:use_ssl => uri.scheme == 'https') do |http|
request = Net::HTTP::Get.new uri
res = http.request request # Net::HTTPResponse object
end
Any ideas as to what I'm doing wrong?
You'll need to format your TimeStamp correctly. The time stamp must be in UTC and in the following ISO 8601 format:
YYYYMMDD'T'HHMMSS'Z'. For example, 20150830T123600Z is a valid time stamp. Do not include milliseconds in the time stamp.
It needs to be of the form:
Timestamp=2016-04-23T08:00:05Z
While Ruby's DateTime.now method returns them of the form:
2016-08-05T10:43:27-07:00
You can read more about Timestamp and AWS signatures here:
http://docs.aws.amazon.com/general/latest/gr/sigv4-date-handling.html
I'm writing a wrapper for the Xero API, using oAuth and two-legged authentication. It is a "private" application, as Xero calls it, which requires a RSA-SHA1 signature. The Coldfusion oAuth wrapper doesn't have a function for encrypting in RSA-SHA1, only HMAC-SHA1. We are on CF9.
Consequently, in the course of running a GET request, I get the following error:
signature_method_rejected
Private applications must use the RSA-SHA1 signature method
So, it looks like the GET call is working, but the issue is with the signature method. I found what I thought looked like the solution someone created, as follows:
<cffunction name="rsa_sha1" returntype="string" access="public" descrition="RSA-SHA1 computation based on supplied private key and supplied base signature string.">
<cfargument name="signKey" type="string" required="true" hint="base64 formatted PKCS8 private key">
<cfargument name="signMessage" type="string" required="true" hint="msg to sign">
<cfargument name="sFormat" type="string" required="false" default="UTF-8">
<cfset var jKey = JavaCast("string", arguments.signKey)>
<cfset var jMsg = JavaCast("string",arguments.signMessage).getBytes(arguments.sFormat)>
<cfset var key = createObject("java", "java.security.PrivateKey")>
<cfset var keySpec = createObject("java","java.security.spec.PKCS8EncodedKeySpec")>
<cfset var keyFactory = createObject("java","java.security.KeyFactory")>
<cfset var b64dec = createObject("java", "sun.misc.BASE64Decoder")>
<cfset var sig = createObject("java", "java.security.Signature")>
<cfset var byteClass = createObject("java", "java.lang.Class")>
<cfset var byteArray = createObject("java","java.lang.reflect.Array")>
<cfset byteClass = byteClass.forName(JavaCast("string","java.lang.Byte"))>
<cfset keyBytes = byteArray.newInstance(byteClass, JavaCast("int","1024"))>
<cfset keyBytes = b64dec.decodeBuffer(jKey)>
<!--- keyBytes = 48-111-10345-125-5349-114-581835-28-330-3984120-2848-4384-1-43 --->
<cfset sig = sig.getInstance("SHA1withRSA", "SunJSSE")>
<!--- error occurs on the line below --->
<cfset sig.initSign(keyFactory.getInstance("RSA").generatePrivate(keySpec.init(keyBytes)))>
<cfset sig.update(jMsg)>
<cfset signBytes = sig.sign()>
<cfreturn ToBase64(signBytes)>
</cffunction>
It receives the following arguments:
SFORMAT UTF-8
SIGNKEY 0JxxxxxxxxxxxxxxxxxxxP&
SIGNMESSAGE GET&https%3A%2F%2Fapi.xero.com%2Fapi.xro%2F2.0%2FContacts&contactid%3D%26contactnumber%3D%26name%3D7-Eleven%26oauth_consumer_key%3Dxxxxxxxxxxxxxxxx%26oauth_nonce%3Dxxxxxxxxxxxxxxxx%26oauth_signature_method%3DRSA-SHA1%26oauth_timestamp%3D1339055484%26oauth_version%3D1.0
However, this produces the following error:
Could not read BER data.(ASN1Lengths.determineLengthLen: length greater than 0x7FFF,FFFF.)
ColdFusion cannot determine the line of the template that caused this error. This is often caused by an error in the exception handling subsystem.
Can anyone could shed any light on this?
EDIT
----
There is a link for uploading the public cert, which I did. THere is also a note saying: "Note, For Private applications, the consumer token and secret are also used as the access token and secret.". So I am assuming I need the "Consumer Secret" value shown there in order to sign the request. That being the case, how do I convert that secret key value to RSA-SH1 format for the signature?
(Summary from the comments)
According to their API private applications must generate an RSA public/private pair (one time event). You then upload your public certificate to their server, and use the private key to sign all requests using RSA-SHA1. If you followed the instructions in their link your private key will be in PEM format, which is just the key value encoded in base64, enclosed within a BEGIN/END wrapper:
-----BEGIN RSA PRIVATE KEY-----
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
....
-----END RSA PRIVATE KEY-----
You need to extract the portion between the wrapper before using the key with PKCS8EncodedKeySpec. There are at least three ways to do that. The simplest option is to use string functions:
// read in the key file and remove the wrapper
pathToKey = "c:/path/to/file/privateKey.pem";
rawKey = FileRead( pathToKey );
rawKey = replace( rawKey, "-----BEGIN RSA PRIVATE KEY-----"& chr(10), "" );
rawKey = replace( rawKey, "-----END RSA PRIVATE KEY-----", "" );
yourMessage = "GET&https%3A%2F%2Fapi.xero.com%2Fapi.xro%2F2.0%2F...";
signature = rsa_sha1( rawKey, yourMessage, "utf-8" );
Another option is to use the BouncyCastle PEMReader class. It does it all for you (and also supports password protected keys).
pathToKey = "c:/path/to/file/privateKey.pem";
provider = createObject("java", "org.bouncycastle.jce.provider.BouncyCastleProvider").init();
security = createObject("java", "java.security.Security").addProvider( provider );
fileReader = createObject("java", "java.io.FileReader").init( pathToKey );
keyReader = createObject("java", "org.bouncycastle.openssl.PEMReader").init( fileReader);
privateKey = keyReader.readObject().getPrivate();
rawKey = binaryEncode( privateKey.getEncoded(), "base64" );
yourMessage = "GET&https%3A%2F%2Fapi.xero.com%2Fapi.xro%2F2.0%2FContacts&....";
signature = rsa_sha1( rawKey, yourMessage, "utf-8" );
Yet another option is to convert the key to DER format with openssl
$ openssl pkcs8 -topk8 -in privateKey.pem -outform DER -nocrypt -out privateKey.pk8
pathToKey = "c:/path/to/file/privateKey.pk8";
bytes = binaryEncode( fileReadBinary(pathToKey), "base64");
yourMessage = "GET&https%3A%2F%2Fapi.xero.com%2Fapi.xro%2F2.0%2F...";
signature = rsa_sha1(rawKey, testMessage, "utf-8");
Note If you posted your actual private key here (or on another forum), it is compromised. So I would strongly recommend generating new keys.