I am working with Elixir using the JOSE package. I have an ALB that is using the Authentication Listener rule, and the data I am getting in the x-amzn-oidc-data is supposed to be a JWT containing the user claims. However, while the value is mostly in JWT formatting, it is base64 URL encoded, but with padding. As such, JOSE is unable to verify most of the tokens coming in (if a token arrives without padding, JOSE verifies it no problem).
I thought I could fix this by simply stripping the = characters from the token in the header, payload and signature, however no combination of that seems to work.
I've used the jwt.io to try and debug the token, and it complains about the signature becoming invalid if I attempt to strip = characters from the payload (only?). However, I have tried validating the token using the Python JOSE library and the Javascript JOSE library, and both are able to. Again though, if I strip the padding characters from the payload, both those libraries tell me the signature verification fails. I have to leave it in for the signature to go through.
Is there a way to massage this token into a format that doesn't include the padding, and could be verified by the Elixir JOSE package? I've tried to reverse-engineer how these libraries are able to decode the tokens while others fail, but I haven't had any luck.
Just for a visual example, this is how the tokens should look
eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWUsImlhdCI6MTUxNjIzOTAyMn0.tyh-VfuzIxCyGYDlkBA7DfyjrqmSHu6pQ2hoZuFqUSLPNY2N0mpHb3nk5K17HWP_3cYHBw7AhHale5wky6-sVA
but the ALB sends me things that look like (note the added padded characters in the payload and signature)
eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWUsImlhdCI6MTUxNjIzOTAyMn0==.tyh-VfuzIxCyGYDlkBA7DfyjrqmSHu6pQ2hoZuFqUSLPNY2N0mpHb3nk5K17HWP_3cYHBw7AhHale5wky6-sVA==
As you mentioned you have to leave the = in for the signature verification to succeed.
The reason JOSE library fails verification is it reconstructs the signing input by decrypting the header + payload, then encrypting them again (This time omitting = characters).
https://github.com/potatosalad/erlang-jose/blob/main/src/jws/jose_jws.erl#L378
You can get around this by manually constructing the signing input, and calling the alg library directly, something like this:
#data from x-amzn-oidc-data header, key from "https://public-keys.auth.elb." <> aws_region <> ".amazonaws.com/" <> kid
[header, payload, signature] = String.split(data, ".") |> Enum.map(fn part -> :jose_jwa_base64url.decode(part) end)
signing_input = Base.url_encode64(header) <> "." <> Base.url_encode64(payload)
encoded_key = key |> JOSE.JWK.from_pem() |> JOSE.JWK.to_record()
:jose_jws_alg_ecdsa.verify(encoded_key, signing_input, signature, :ES256)
Related
I'm building a multi-exchange bot in C++ and I'm having one small problem. KuCoin exchange has proven to be frustrating to say the least, one of the headers is a signature header where you encode the string with HMAC sha256 and then encode THAT with base64. However I'm not concerned with the encoding, I can do all that. What is stumping me is the string KuCoins API is expecting, I've scoured their documentation 100 times over and I still can't get it right, here are the instructions
For the header of KC-API-KEY:
Use API-Secret to encrypt the prehash string {timestamp+method+endpoint+body} with sha256 HMAC. The request body is a JSON string and need to be the same with the parameters passed by the API.
After that, use base64-encode to encrypt the result in step 1 again.
I've attempted to craft this string in every way possible, and the documentation provides no examples on what a good string should look like, here are the strings I've crafted BEFORE encoding that DO NOT work.
EXAMPLE 1: 1616096476134POST/api/v1/orders?clientOid=55be5&side=BUY&symbol=BTC-USDT&type=MARKET&funds=0.005000
EXAMPLE 2: 1616099932367POST/api/v1/orders{"clientOid":"55be5","side":"BUY","symbol":"BTC-USDT","type":"MARKET","funds":"0"}
As you can see, in the 2nd example I tried to make the body a JSON string with all the correct parameters, but still, I'm getting a bad signature error returned. Literally all I need is to know what the string is supposed to look like so I can craft it properly. Thanks.
I take the assumption that your code works for a private request without parameters (like getting the balance for instance).
I also struggled a bit for POST requests, but managed to get it right after a few attempts. You need to dump the parameters as a JSON string without spaces, exactly like in your example 2.
Since that time, have you managed to solve it ? I have a code on my application that works if you are interested.
Also, don't forget to add in the headers of the curl request:
Content-Type: application/json
Solved with Kucoin support and pythone example.
The "body" also must be included in POST request.
Was:
reply = netman->post(req, "");
Become:
tradereply = trademan->post(req, data);
I am seeing this error when I try to hit endpoint using postman:
{
"errors": [
"token contains an invalid number of segments"
]
}
Console:
TypeError: Cannot read property 'jwt' of undefined
The question is a bit vague, but in this case - TypeError: Cannot read property 'jwt' of undefined.
The object that should have the property jwt, is undefined. So the token itself doesn't exist, and neither does the object that is supposed to hold it.
More generally, this error may occur if the token is malformed.
It should have 3 segments, like this [separated by . ]-
eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJqb2UiLA0KICJleHAiOjEzMDA4MTkzODAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ.dBjftJeZ4CVP-mB92K27uhbUJU1p1r_wW1gFWFOEjXk
Each of those segments can be decoded by base64 to inspect the header, payload, and signature.
Which results in this -
from - https://jwt.io/
More information from https://www.ibm.com/docs/en/cics-ts/6.1_beta?topic=cics-json-web-token-jwt
Header
The header typically consists of two parts: the type of the token, which is JWT, and the algorithm that is used, such as HMAC SHA256 or RSA SHA256. It is Base64Url encoded to form the first part of the JWT.
Payload
The payload contains the claims. There is a set of registered claims, for example: iss (issuer), exp (expiration time), sub (subject), and aud (audience). These claims are not mandatory but recommended to provide a set of useful, interoperable claims. The payload can also include extra attributes that define custom claims, such as employee role. Typically, the subject claim is used to create the OpenID Connect user subject. However, the Liberty JVM server can be configured to use an alternative claim. The payload is Base64Url encoded to form the second part of the JWT.
Signature
To create the signature part, the encoded header and encoded payload are signed by using the signature algorithm from the header. The signature is used to verify that the issuer of the JWT is who it says it is and to ensure that the message wasn't changed along the way.
More information about the spec here -
https://datatracker.ietf.org/doc/html/rfc7519#page-7
In cryptography, starting with:
'http://www.server.com/?tag=xx&uid=99' (for example)
a hash length extension attack attempts to append to the url with
'http://www.server.com/?tag=xx&uid=99%80%00&deposit=100'
where the the '%80%00' represents the percent encoding of the 'padding' characters used to extend the message.
I'm using HTTPConnection.request('Get', url) in python to send the request but I'm getting an invalid uid message. Does this mean that the server is treating the %80%00 as part of the uid value? Or am I encoding the url incorreectly?
Could someone tell me what the problem is and how to fix it?
Thanks.
It depends on the backend code. One possible solution is to append &uid=99 to the url being used for the attack to overwrite the invalid uid.
Ex. http://www.server.com/?tag=xx&uid=99%80%00&deposit=100&uid=99
The server then replaces the invalid uid with the valid one.
So I'm working on a login system in Tornado. I originally tried using websockets (doesn't work) and switched to a regular web server. I'm trying to set a cookie so that I can remember if someone's logged in on the websocket. The problem that's coming up is a
ValuError: Invalid cookie
I looked up the Tornado source code (http://www.tornadoweb.org/en/branch2.0/_modules/tornado/web.html), and this is caused by matching a regular expression for any characters in between 0x00 and 0x20
if re.search(r"[\x00-\x20]", name + value):
# Don't let us accidentally inject bad stuff
raise ValueError("Invalid cookie %r: %r" % (name, value))
The thing is, I'm always going to have these characters, because the Hash generates random bytes. Is there any way I can get around this?
That check actually isn't strict enough; there are a lot of restrictions on what can go in a cookie. In general if you want to put arbitrary binary data in a cookie you'll have to encode it (e.g. in base64). If you use tornado's secure_cookie functions they'll base64 encode the data for you.
Also note that it's not a good idea to store password hashes in a cookie - the best practice is to generate a random session id that is stored somewhere on the server side and put that in the cookie.
I'm going to write a program can post and read messages from SQS with authentication and I've read the document from here
Link: Query Request Authentication
I have successfully written the process which post a message to specified queue follow by the document. But I always get 403 error when I try to receive message from queue. And I found the signature string rules are different for POST and GET methods.
the signature string is:
GET\n
sqs.us-east-1.amazonaws.com\n
/<My Account Id>/<Queue Name>\n
AWSAccessKeyId=<My Access Key>
&Action=ReceiveMessage
&MaxNumberOfMessages=10
&VisibilityTimeout=600
&AttributeName=All
&Expires=2012-04-01T11%3A29%3A24Z
&SignatureMethod=HmacSHA1
&SignatureVersion=2
&Version=2011-10-01
and the url is
https://sqs.us-east-1.amazonaws.com/<My Account Id>/<Queue Name>?
Action=ReceiveMessage
&MaxNumberOfMessages=10
&VisibilityTimeout=600&AttributeName=All
&Version=2011-10-01
&Expires=2012-04-01T11%3A29%3A24Z
&Signature=<BASE64 encoded HmacSHA1 digist with signature string and my security key>
&SignatureVersion=2
&SignatureMethod=HmacSHA1
&AWSAccessKeyId=<My Access Key>
And I always get the 403 forbidden error:
<ErrorResponse xmlns="http://queue.amazonaws.com/doc/2011-10-01/">
<Error>
<Type>Sender</Type>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
</Message>
<Detail/>
</Error>
<RequestId>16f6e910-62e6-4259-8c09-0358b84cbe60</RequestId>
</ErrorResponse>
Is there anyone can tell me how can I deal with it? Thanks a lot
The error message tells you that the signature is being calculated wrong. This is really tough to debug. I spent hours on it the first time I tried it. There's an example signed SQS request at http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/MakingRequests_MakingQueryRequestsArticle.html. You should put those parameters in your program, calculate the signature, and try finding bugs into your program creates the same signature.
Specific problems I had, and fixes for them included:
Sorting the query parameters correctly. They must be in ascending order when creating the string to sign. Your example URL does not show them in order. Did you sort them differently when creating the string to sign?
URI encoding properly. Every parameter must be URI encoded in the string to sign. Your sample URL does have URI encoding, so this probably isn't your issue. But make sure you're not double-encoding.
Padding the base64 signature. At least some AWS services insist that the signature be a multiple of four characters long. Two-thirds of the time a base64 encoding will be too short, and need one or two equal signs appended to it. Most base64 encoding libraries do that for you, but not all.
Of course, the easiest thing is to use somebody else's library to make the requests, but what's the fun in that? Good luck debugging this.
It's most likely the parameter order: when assembling the signature version 2 string, at the last step the Amazon documentation specifies:
Add the query string components (the name-value pairs, not including
the initial question mark (?) as UTF-8 characters which are URL
encoded per RFC 3986 (hexadecimal characters must be uppercased) and
sorted using lexicographic byte ordering. Lexicographic byte ordering
is case sensitive.
I've spent two days debugging this same "SignatureDoesNotMatch" issue by checking my HMAC, BASE64 and URL encoding procedures and it was just a problem of parameter order.
The documentation should emphasize this issue more; if you use unordered parameter strings (e.g. the same one in the request URL, like those found in the documentation examples), you're going to get this non-intuitive error from the server.