How do i generate query url for Amazon SQS - amazon-web-services

I'm going to write a program can post and read messages from SQS with authentication and I've read the document from here
Link: Query Request Authentication
I have successfully written the process which post a message to specified queue follow by the document. But I always get 403 error when I try to receive message from queue. And I found the signature string rules are different for POST and GET methods.
the signature string is:
GET\n
sqs.us-east-1.amazonaws.com\n
/<My Account Id>/<Queue Name>\n
AWSAccessKeyId=<My Access Key>
&Action=ReceiveMessage
&MaxNumberOfMessages=10
&VisibilityTimeout=600
&AttributeName=All
&Expires=2012-04-01T11%3A29%3A24Z
&SignatureMethod=HmacSHA1
&SignatureVersion=2
&Version=2011-10-01
and the url is
https://sqs.us-east-1.amazonaws.com/<My Account Id>/<Queue Name>?
Action=ReceiveMessage
&MaxNumberOfMessages=10
&VisibilityTimeout=600&AttributeName=All
&Version=2011-10-01
&Expires=2012-04-01T11%3A29%3A24Z
&Signature=<BASE64 encoded HmacSHA1 digist with signature string and my security key>
&SignatureVersion=2
&SignatureMethod=HmacSHA1
&AWSAccessKeyId=<My Access Key>
And I always get the 403 forbidden error:
<ErrorResponse xmlns="http://queue.amazonaws.com/doc/2011-10-01/">
<Error>
<Type>Sender</Type>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
</Message>
<Detail/>
</Error>
<RequestId>16f6e910-62e6-4259-8c09-0358b84cbe60</RequestId>
</ErrorResponse>
Is there anyone can tell me how can I deal with it? Thanks a lot

The error message tells you that the signature is being calculated wrong. This is really tough to debug. I spent hours on it the first time I tried it. There's an example signed SQS request at http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/MakingRequests_MakingQueryRequestsArticle.html. You should put those parameters in your program, calculate the signature, and try finding bugs into your program creates the same signature.
Specific problems I had, and fixes for them included:
Sorting the query parameters correctly. They must be in ascending order when creating the string to sign. Your example URL does not show them in order. Did you sort them differently when creating the string to sign?
URI encoding properly. Every parameter must be URI encoded in the string to sign. Your sample URL does have URI encoding, so this probably isn't your issue. But make sure you're not double-encoding.
Padding the base64 signature. At least some AWS services insist that the signature be a multiple of four characters long. Two-thirds of the time a base64 encoding will be too short, and need one or two equal signs appended to it. Most base64 encoding libraries do that for you, but not all.
Of course, the easiest thing is to use somebody else's library to make the requests, but what's the fun in that? Good luck debugging this.

It's most likely the parameter order: when assembling the signature version 2 string, at the last step the Amazon documentation specifies:
Add the query string components (the name-value pairs, not including
the initial question mark (?) as UTF-8 characters which are URL
encoded per RFC 3986 (hexadecimal characters must be uppercased) and
sorted using lexicographic byte ordering. Lexicographic byte ordering
is case sensitive.
I've spent two days debugging this same "SignatureDoesNotMatch" issue by checking my HMAC, BASE64 and URL encoding procedures and it was just a problem of parameter order.
The documentation should emphasize this issue more; if you use unordered parameter strings (e.g. the same one in the request URL, like those found in the documentation examples), you're going to get this non-intuitive error from the server.

Related

C++ Signing message on kucoin API

I'm building a multi-exchange bot in C++ and I'm having one small problem. KuCoin exchange has proven to be frustrating to say the least, one of the headers is a signature header where you encode the string with HMAC sha256 and then encode THAT with base64. However I'm not concerned with the encoding, I can do all that. What is stumping me is the string KuCoins API is expecting, I've scoured their documentation 100 times over and I still can't get it right, here are the instructions
For the header of KC-API-KEY:
Use API-Secret to encrypt the prehash string {timestamp+method+endpoint+body} with sha256 HMAC. The request body is a JSON string and need to be the same with the parameters passed by the API.
After that, use base64-encode to encrypt the result in step 1 again.
I've attempted to craft this string in every way possible, and the documentation provides no examples on what a good string should look like, here are the strings I've crafted BEFORE encoding that DO NOT work.
EXAMPLE 1: 1616096476134POST/api/v1/orders?clientOid=55be5&side=BUY&symbol=BTC-USDT&type=MARKET&funds=0.005000
EXAMPLE 2: 1616099932367POST/api/v1/orders{"clientOid":"55be5","side":"BUY","symbol":"BTC-USDT","type":"MARKET","funds":"0"}
As you can see, in the 2nd example I tried to make the body a JSON string with all the correct parameters, but still, I'm getting a bad signature error returned. Literally all I need is to know what the string is supposed to look like so I can craft it properly. Thanks.
I take the assumption that your code works for a private request without parameters (like getting the balance for instance).
I also struggled a bit for POST requests, but managed to get it right after a few attempts. You need to dump the parameters as a JSON string without spaces, exactly like in your example 2.
Since that time, have you managed to solve it ? I have a code on my application that works if you are interested.
Also, don't forget to add in the headers of the curl request:
Content-Type: application/json
Solved with Kucoin support and pythone example.
The "body" also must be included in POST request.
Was:
reply = netman->post(req, "");
Become:
tradereply = trademan->post(req, data);

Padding in query string of url

In cryptography, starting with:
'http://www.server.com/?tag=xx&uid=99' (for example)
a hash length extension attack attempts to append to the url with
'http://www.server.com/?tag=xx&uid=99%80%00&deposit=100'
where the the '%80%00' represents the percent encoding of the 'padding' characters used to extend the message.
I'm using HTTPConnection.request('Get', url) in python to send the request but I'm getting an invalid uid message. Does this mean that the server is treating the %80%00 as part of the uid value? Or am I encoding the url incorreectly?
Could someone tell me what the problem is and how to fix it?
Thanks.
It depends on the backend code. One possible solution is to append &uid=99 to the url being used for the attack to overwrite the invalid uid.
Ex. http://www.server.com/?tag=xx&uid=99%80%00&deposit=100&uid=99
The server then replaces the invalid uid with the valid one.

List object versions in a S3 bucket via signed URI

In our project we're storing objects in a S3 bucket with versioning enabled. There's no logic on the server besides creating a signed URI for the client to use. We'd like to keep it this way as we want the client to do all the processing.
To the problem. We're successfully able to generate signed URIs for a GET and PUT object for the whole objects, but we're unable to generate a URI for listing all available versions.
This is an example of a GET-url on a object in one of our buckets which works (the 99/2 are folders in the bucket):
https://bucketname.s3.amazonaws.com/99/2?AWSAccessKeyId=ourkey&Signature=signature&Expires=1410784420
According to the docs (GET versions) we're supposed to append ?versions and the different versions. We've tried the following:
https://bucketname.s3.amazonaws.com/99/2?AWSAccessKeyId=ourkey&Signature=signature&Expires=1410784420&versions
This then results in the browser complaining that the signature is wrong, it's missing "?versions". If I read docs I interpret it as it shouldn't be included in the signature unless we append a value to it as well, which we aren't. The problem is then that it doesn't matter if I then add it to the signature creation as it still fails with the error "There is no such thing as the ?versions sub-resource for a key".
Is there someone who has successfully created a signed uri for object to list it's versions? We'd really love to get some pointers on what we're doing wrong!
I'd also like to point out that we're not using the built in URI-generator as we couldn't get it to fit our needs.
Listing object versions is an operation performed "against" the bucket, not against the object... so your path is always going to be /, no matter what keys you want to list.
You specify the key prefix in the query string as prefix=....
The string to sign would then begin with /bucketname/?versions&prefix=....
You sort all of the query string parameters lexically, except for the subresource (versions, in this case), which goes first. If more than one subresource, you also sort them lexically among themselves, but they still go first. Everything is separated by & in the string to sign.
Significant caveat: the list api may not be appropriate to hand over to the client, since you can end up returning the wrong things... "prefix" is just that -- a prefix. If it doesn't match exactly, it can match on substrings, which might not be what you want. You may also need to use delimiter and max-keys and be prepared to handle the pagination through truncated listings that will become necessary when a large number of results is returned.
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETVersion.html
http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html

Vimeo OAuth Error - he consumer key passed was not valid

Am trying to get authenticated with the API and it is saying "The consumer key passed was not valid." msg="Invalid consumer key". Before that it was saying that the oauth_signature is invalid and I have struggled for a day with it and now that error is not showing up. Please find my code in the above gist.
You help is much appreciated on this.
There are a handful of reasons that you might see this error, all of which mean that your oauth 1.0a protocol is likely incorrect.
I highly recommend you test your app alongside this tool : http://hueniverse.com/oauth/guide/authentication/
Below are the important steps to check.
Compare your base string to the Hueniverse base string. If they match, you are likely signing incorrectly. If they do not match make sure you are sorting the parameter keys properly
If the base string matches, check your signature method and your signing process. The secret of your request should be client_secret& OR client_secret&token_secret. Note the ampersand always exists.

How to construct signing request for Amazon EC2 API call with filters?

This question is a follow-up to my earlier question on getting a REST API call to EC2 API working.
Having got that working, I wanted to generalize it and tried a couple of things. For example, I tried to filter by region name and make the request
https://ec2.amazonaws.com/?Action=DescribeRegions
&RegionName.1=us-east-1
&RegionName.2=eu-west-1
which they show in the documentation.
Signing that was easy, stick the RegionName(s) into the signing request in their alphabetical position and sign the whole thing just as I would without them. That worked out just fine.
So I progressed to the second example that they provide in the documentation.
https://ec2.amazonaws.com/?Action=DescribeRegions
&Filter.1.Name=endpoint
&Filter.1.Value.1=*ap*
OK, I tried to put the filters into my signing parameters in the alphabetically sorted location as required ...
char * signing_parameters_template =
"AWSAccessKeyId=%s&"
"Action=DescribeRegions&"
"Filter.1.Name=endpoint&"
"Filter.1.Value=*ap*&"
"SignatureMethod=HmacSHA256&"
"SignatureVersion=2&"
"Timestamp=%s&"
"Version=2013-08-15";
and sign that. I get an error that the signature doesn't match. I've tried a bunch of variations of this, no luck.
How does one sign a request that includes filters?
I have to escape the "*" in the filter
"Filter.1.Value=*ap*&"
That's it!