I am using CreatePolicy API to create a policy with specific permissions. Initially passed json code as a value to query string parameter "PolicyDocument" but the request failed with code 400 Bad request. While testing through postman found that we have to urlencode given policy document. This solution worked fine on postman but not on my HTTP Client. Error - "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details". Code is working fine for all other APIs even for IAM Get request, but failing when policy doc is being sent as a query string or as a body. Possibly there is something wrong while calculating the signature for IAM api with url encoded policy doc.
Ref - https://docs.aws.amazon.com/IAM/latest/APIReference/API_CreatePolicy.html
Tried passing policy doc as a request body and header - "Content-Type:application/x-www-form-urlencoded". (body is JSON converted to string)
Tried passing policy doc as a query parameter which is url-encoded
Note - Both these methods worked fine when testing them through postman
I am using AWS Cognito federated through Google.
I am trying to use the Authorization Code Grant flow, which requires making two requests.
/oauth2/auth
The first request is to get the authorization code, which is then sent as part of the second request. This request comes back fine.
/oauth2/token
This request accepts the authorization code from the first, and should return the appropriate tokens.
However, this endpoint requires a secret in the header, so I have created an API (to not expose the secret to the client) that accepts the token and makes the request from a different origin as the first request. For example, the first request is made from example.com, while the second is from api.example.com. This fails with the error message: 400: invalid request. But, if I take the exact same code from the API, and put it in the client directly (with the secret), it returns the appropriate tokens. The only difference I can think of is the origin. Is there a way around this or is this just not possible?
Thanks in advance.
Nevermind, forgot to stringify the data as part of the request on the server-side. Thanks.
Why AccessKeyId is included in s3 pre-signed URL? Is it really necessary? The pre-signed URL already includes the Signature field, why it still requires the AccessKeyId as well? Wouldn't be the Signature sufficient?
The signature is used to prove two things:
that the signer authorizes this specific request, and
that the signer was in possession of the secret key associated with the specified access-key-id.
Importantly... the signature does not actually contain any meaningful information. It's either right or it's wrong.
It's an HMAC-based hash of public (the request being made) and private (the secret key) information. The service doesn't "decode" it or interpret it or learn anything from it.
Instead, the service -- using the access-key-id -- looks up the associated secret key,¹ takes the request; and internally generates the signature you should have generated for the same request... then it checks to see if that's what you actually generated.² If not, the error is SignatureDoesNotMatch. The error is not more specific because the signature for any given request at any moment in time has only one possible value. Any other signature is simply the wrong signature.
But the access-key-id must be specified so the service knows who's making the request. The signature does not contain any reversible/decodable/decryptable information.
¹ looks up the associated secret key is probably an oversimplification when using Signature Version 4 because there are layers of (date, region, service, signing) keys derived from the IAM user's secret key... and the structure and nesting implies that individual services have access only to the relevant values they need.
² you generated is an important phrase, since there is some potential for misunderstanding of the source of pre-signed URLs. These are generated entirely in your code, with no interaction with the service. S3 is unaware of the existence of any pre-signed URLs until they are actually used. This has implications that can sometimes be useful; for example, it is entirely possible to generate a pre-signed URL for an object that does not yet exist, and create the object later. Also, disabling or deleting the aws-access-key-id that was used to generate a pre-signed URL immediately invalidates all URLs that key ever generated.
I am writing a Chalice deployment, and experiencing behavior that I cannot explain.
My root endpoint accepts PUT requests, and verifies some basic authorization credentials.
from chalice import Chalice
from base64 import b64decode
app = Chalice(app_name='test-basic-auth-issue')
#app.route('/', methods=['PUT'])
def index():
auth = app.current_request.headers['Authorization'].split()
username, password = b64decode(auth[1]).split(':')
if username == 'test-user' and password == 'test-password':
return {username: password}
else:
raise Exception('Unauthorized')
Using curl to interface with this API:
curl https://test-user:test-password#<API-URL>/dev/ --upload-file test.txt
I get the following response:
{"message":"Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=Basic dGVzdC11c2VyOnRlc3QtcGFzc3dvcmQ="}
However - when including any parameters in the URL:
curl https://test-user:test-password#<API-URL>/dev/?ANYTHING --upload-file test.txt
I get the expected response of:
{"test-user": "test-password"}
I'm not sure why specifying parameters is effecting the Authorization.
I have no inside information, so I have no way of knowing whether the following is actually correct, but this seems like a reasonable explanation for the behavior you're seeing.
AWS APIs generally support two alternatives for supplying your credentials: query string parameters, or the Authorization: header..
To the layer in their stack that checks the Authorization: header, your value seems wrong, so they throw an error, since your supplied credentials are not in the correct format...
...unless it sees a query string in the URI... in which case, it could be choosing to allow the request to proceed, on the assumption that the authorization might be done at that layer.
So the request is handed off to a different layer, which is responsible for query string handling. It doesn't find any credentials in the query string, but it is also aware of no credentials having been found while headers were being processed, previously, so the request is processed as an anonymous request if those are allowed.
So, you're slipping through a hole: by adding a query string, any query string, you prevent the Authorization: header from throwing the error.
It's not a security vulnerability, in my estimation, but rather a case where something in the URI changes how headers are interpreted -- specifically, whether a malformed (for its purposes) authorization header will trigger an exception or be allowed to pass.
I think there's a reasonable case for calling this behavior "broken," but at the same time I suspect it may be out of the hands of the API Gateway developers, who are working behind an unnamed front-end component that is common to multiple AWS services. API Gateway is a bit of an exception, in the AWS ecosystem, in that the customer can define how headers are manipulated... so this may very well be simply a platform limitation.
I disagree -- in part, and on a technicality -- with #LorenzodeLara's assertion that API Gateway is not compliant with RFC-7235. There is no requirement that a server respond with WWW-Authenticate: -- from the RFC:
Upon receipt of a request for a protected resource that omits credentials, contains invalid credentials (e.g., a bad password) or partial credentials (e.g., when the authentication scheme requires more than one round trip), an origin server SHOULD send a 401(Unauthorized) response that contains a WWW-Authenticate header field with at least one (possibly new) challenge applicable to the requested resource.
The words SHOULD and RECOMMENDED in RFCs indicate desired behavior for which there may be valid exceptions. They are not mandatory requirements.
It is, on the other hand, fully accurate that API Gateway does not at all support authenticating requests from its perspective with HTTP basic auth... but you are only trying to get it to pass those credentials through to the code running inside, which does appear to work, given that your user agent submits credentials without a challenge... assuming you prevent the front-end system from thwarting you, with the addition of a query string.
That's my analysis, subject to correction by someone with access to more authoritative information. In that light, using basic auth may not be the best plan, since it appears to work somewhat by accident.
I hate to come to that conclusion. Basic auth gets a bad rap, which is not wholly merited when combined with HTTPS -- unlike request signing, it typically "just works," right out of the box, even for a user who doesn't understand the difference between GET and POST, much less how to generate a hex-encoded HMAC digest.
I am using amazon's itemlookup api to get the details of an book provided the ISBN has been given to it.
I have made an request to Amazon aws with URL as:
http://webservices.amazon.com/onca/xml?AWSAccessKeyId=access_key&Keywords=0439708184&Operation=ItemSearch&ResponseGroup=ItemAttributes%2COffers&SearchIndex=Books&Service=AWSECommerceService&Timestamp=2015-03-23T16%3A04%3A21Z&Version=2009-01-06&Signature=hashed_signature
And all I get from above request is :
<Message>Your request is missing required parameters. Required parameters include AssociateTag.</Message>
Later on, I have generated an AssociateTag for myself and passed it to the the query string but that did not help either. Instead, it gave me another message that "Signature does not match."
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.</Message>
After adding the associate tag, my query string looks like as:
http://webservices.amazon.com/onca/xml?AWSAccessKeyId=access_key&Keywords=0439708184&Operation=ItemSearch&ResponseGroup=ItemAttributes%2COffers&SearchIndex=Books&Service=AWSECommerceService&Timestamp=2015-03-23T16%3A04%3A21Z&Version=2009-01-06&AssociateTag=hemant-20&Signature=hashed_signature
Am I missing anything basic here?