I am writing a Chalice deployment, and experiencing behavior that I cannot explain.
My root endpoint accepts PUT requests, and verifies some basic authorization credentials.
from chalice import Chalice
from base64 import b64decode
app = Chalice(app_name='test-basic-auth-issue')
#app.route('/', methods=['PUT'])
def index():
auth = app.current_request.headers['Authorization'].split()
username, password = b64decode(auth[1]).split(':')
if username == 'test-user' and password == 'test-password':
return {username: password}
else:
raise Exception('Unauthorized')
Using curl to interface with this API:
curl https://test-user:test-password#<API-URL>/dev/ --upload-file test.txt
I get the following response:
{"message":"Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=Basic dGVzdC11c2VyOnRlc3QtcGFzc3dvcmQ="}
However - when including any parameters in the URL:
curl https://test-user:test-password#<API-URL>/dev/?ANYTHING --upload-file test.txt
I get the expected response of:
{"test-user": "test-password"}
I'm not sure why specifying parameters is effecting the Authorization.
I have no inside information, so I have no way of knowing whether the following is actually correct, but this seems like a reasonable explanation for the behavior you're seeing.
AWS APIs generally support two alternatives for supplying your credentials: query string parameters, or the Authorization: header..
To the layer in their stack that checks the Authorization: header, your value seems wrong, so they throw an error, since your supplied credentials are not in the correct format...
...unless it sees a query string in the URI... in which case, it could be choosing to allow the request to proceed, on the assumption that the authorization might be done at that layer.
So the request is handed off to a different layer, which is responsible for query string handling. It doesn't find any credentials in the query string, but it is also aware of no credentials having been found while headers were being processed, previously, so the request is processed as an anonymous request if those are allowed.
So, you're slipping through a hole: by adding a query string, any query string, you prevent the Authorization: header from throwing the error.
It's not a security vulnerability, in my estimation, but rather a case where something in the URI changes how headers are interpreted -- specifically, whether a malformed (for its purposes) authorization header will trigger an exception or be allowed to pass.
I think there's a reasonable case for calling this behavior "broken," but at the same time I suspect it may be out of the hands of the API Gateway developers, who are working behind an unnamed front-end component that is common to multiple AWS services. API Gateway is a bit of an exception, in the AWS ecosystem, in that the customer can define how headers are manipulated... so this may very well be simply a platform limitation.
I disagree -- in part, and on a technicality -- with #LorenzodeLara's assertion that API Gateway is not compliant with RFC-7235. There is no requirement that a server respond with WWW-Authenticate: -- from the RFC:
Upon receipt of a request for a protected resource that omits credentials, contains invalid credentials (e.g., a bad password) or partial credentials (e.g., when the authentication scheme requires more than one round trip), an origin server SHOULD send a 401(Unauthorized) response that contains a WWW-Authenticate header field with at least one (possibly new) challenge applicable to the requested resource.
The words SHOULD and RECOMMENDED in RFCs indicate desired behavior for which there may be valid exceptions. They are not mandatory requirements.
It is, on the other hand, fully accurate that API Gateway does not at all support authenticating requests from its perspective with HTTP basic auth... but you are only trying to get it to pass those credentials through to the code running inside, which does appear to work, given that your user agent submits credentials without a challenge... assuming you prevent the front-end system from thwarting you, with the addition of a query string.
That's my analysis, subject to correction by someone with access to more authoritative information. In that light, using basic auth may not be the best plan, since it appears to work somewhat by accident.
I hate to come to that conclusion. Basic auth gets a bad rap, which is not wholly merited when combined with HTTPS -- unlike request signing, it typically "just works," right out of the box, even for a user who doesn't understand the difference between GET and POST, much less how to generate a hex-encoded HMAC digest.
Related
Ordinarily would not ask this but I have searched high and low for information on "SIGNATURE_REMOVED_BY_GOOGLE" and come up short.
I am running a webapp in firebase using GCP functions as a backend. As part of this work I use access token based authorization where I send a header that is like this:
Authorization: Bearer ${SOME ID TOKEN}
The ID token broadly breaks down into 3 parts:
`${BASE64 encoded header JSON}.${BASE64 encoded body JSON}.${signature}`
These ID tokens are generated by google from the https://securetoken.googleapis.com/v1/token endpoint
As of 2022/08/04 at around 6am GMT the GCP function started throwing errors, it had been working without issue for months.
The errors were a result of the Authorization header being modified in transit by google
from:
Authorization: Bearer ${BASE64 encoded header JSON}.${BASE64 encoded body JSON}.${signature}
to:
Authorization: bearer ${BASE64 encoded header JSON}.${BASE64 encoded body JSON}.SIGNATURE_REMOVED_BY_GOOGLE
Note the word Bearer => bearer, which while technically valid is unusual for the auth header
Primarily, the issue is the removal of the signature from the token. and it's replacement with "SIGNATURE_REMOVED_BY_GOOGLE"
I have found this SO answer: Google Cloud Run masks Bearer token in Authorization header
Which suggests that Cloud Run does this as a security precaution. However I am using GCP functions, not Cloud Run, also, this started happening today after having not happened before. That SO link is several years old so I would have expected it to be occurring before 2022/08/04 if it were caused by the same thing.
So, several questions:
Is there a way to prevent the header from being modified? (I don't like it but if I used something like 'x-custom-auth' as a header would that prevent it?)
If it cannot be stopped, can I assume that a request with this bearer token mapping is valid / secure?
How do I Differentiate between google mapping the auth header vs some malicious actor sending a request without a signature and putting "SIGNATURE_REMOVED_BY_GOOGLE" in themselves?
I wrote the answer that you are referencing in your question.
Note the word Bearer => bearer, which while technically valid is
unusual for the auth header
HTTP headers consist of case-insensitive names. Google often converts HTTP header names (keys) to lowercase.
Is there a way to prevent the header from being modified? (I don't
like it but if I used something like 'x-custom-auth' as a header would
that prevent it?)
No, you cannot prevent Google from modifying the token. The token is modified to prevent the token from being used again (to prevent the service from using the token to impersonate the user to call another service).
Yes, you can use a custom header. However, there have been issues (bugs) reported when doing so.
If it cannot be stopped, can I assume that a request with this bearer
token mapping is valid / secure?
If your service has IAP enabled (the service is not public), then yes, you can safely assume that the token is valid. If the client forges an identity token, IAP will reject their request.
How do I Differentiate between google mapping the auth header vs some
malicious actor sending a request without a signature and putting
"SIGNATURE_REMOVED_BY_GOOGLE" in themselves?
If IAP is enabled, a malicious actor will not get past IAP. The Identity Token signature is created with the identity's private key. The bad actor would need that private key to create requests. The signature is validated with the corresponding public key. Requests with an invalid signature are rejected with an HTTP 40x status code. SIGNATURE_REMOVED_BY_GOOGLE would be an invalid signature.
That SO link is several years old so I would have expected it to be
occurring before 2022/08/04 if it were caused by the same thing.
New features are often released in one service and when found useful added to other services. Cloud Functions has existed for a longer time than Cloud Run. Changes to existing services sometimes takes a long time. First customers are notified that in the future XYZ will be enabled, start working, etc. I am not sure of the exact status of the enablement of OAuth token redaction for Cloud Functions.
I just want to make sure I've got the overall idea down and don't create an implementation that violates basic security best practices. Can somebody please check my understanding?
As I understand it, a user can log in to my application and the authentication server REST API can return a JWT that is signed, but NOT encrypted. Inside that token I can have claims inside the payload that my client application can access, such as features the user can use on the application. That way my client side website can change functionality based on the user privileges and roles. The JWT claims in the payload are NOT sensitive. They will be strings representing categories for images or documents, things like that.
When the user wants to get additional content (like a document, image, or video) from other REST API endpoints, they submit the JWT along with the GET request. My API can then verify the signature of the JWT and grant API access if appropriate.
This last part is what I'm most unsure about. My intent is to use another authorization server API endpoint which takes the JWT in a POST request and returns a simple "valid/invalid" response. My thought is that my Content Delivery Network (CDN) can use this API to verify that the JWT in possession is validly signed. I believe (and maybe here is where I'm goofing up) that the authorization server API can be publicly accessible to ease use by my other microservices. This seems fine because I'm just giving a boolean pass/fail on the validity of the token so I don't see any need to hide or obfuscate the API. I question this because I know AWS has backend stuff to validate and authorize for API calls but I like the simplicity of just using REST APIs for everything for my first implementation; to maintain simplicity.
So in summary:
1.) Signed, unencrypted JWT with non-sensitive user roles/privileges.
2.) Unencrypted so client side webpage can selectively render content based on user.
3.) Public authorization API that anybody could technically use so that my CDN (and other microservices) can validate JWTs.
Any major issue with this approach? Have I committed any technical sins?
Thank you so much in advance for your time on this matter.
Okay, I think I've sorted this out myself after finding a great video tutorial on this stuff. Below is the video I watched:
https://www.youtube.com/watch?v=_XbXkVdoG_0
I had some misconceptions and this video sorted them out. It appears that what I described in my question is precisely how JWT should be used.
I'm working on API that tries to stay in RESTful principles, although one requirement keeps bugging me.
We use JWT based authentication. Inside JWT claims we store roles of the user. Now our main GET endpoint (let's call it ListAllOffers for simplicity) behaves differently depending on what role the user have:
if API recognizes admin via JWT it responds with full list of Offers
if API recognizes ordinary user via JWT it responds with narrowed list of Offers (depending on relation in DB)
My concerns is: is it ok according to REST principles or any unwritten REST practices? I am used to modify response object according to argument from url, params from querystring or alternatively via header values. Altering JSON response basing JWT seems not explicit enough that is feels some kinda strange.
Bonus question: is it against any of REST principles how should this requirement be implemented.
You'll find lots of APIs have resources where the value changes depending on the authenticated user (see GitHub's API).
When it comes to REST the endpoint must always reference the same resource. However, you may choose to represent that resource in any way you wish. Masking some of the offers because the user is not an admin has not changed the resource only the representation you are giving to that user.
While the REST specification doesn't have any specific examples of changing representations due to authorisation it is still a worthwhile read.
If you are concerned about changing the representation implicitly there are a few options available that would make it more explicit whilst still following RESTful standards.
You could add a query parameter that explicitly requests only the current user's offers: /offers?show=mine.
If the offers are 'owner' by a user you could also do something like: /users/{username}/offers. Here a user would be authorised only for their own offers. An admin would of course be authorised for any.
Overall the key point is this: an endpoint must always represent the same resource but how it represents it is up to you.
Hey I am getting started with the serverless framework, apigateway, lambdas and authorizers.
Here my questions:
In order to verify a proper JWT token (which seems nowadays the best solution for serverless authentication), I would need a client id. Since the authorizer lambda is not capable(?) of reaching any other parameters of the request into the lambda except for the token itself, this is a difficult task to do. How can I achieve this?
Since with every authenticated call, an additional lambda is called, my costs are doubled?! May be I am misunderstanding something here, but it seems to me like implementing a method for verifying my token without using the authorizer is cheaper and I don't need to implement the authorizer lambda itself.
The whole point of OAuth2 is that at the point of authorization you don't care who the bearer presenting you with a token is or which client they are using (web app, Postman, etc.), only that the token is valid for whatever the token bearer is trying to do. You trust that the authentication has happened because the token issuer has done the authentication.
When you talk about the aud(audience) (from comments) in JWTs, which may or may not be included dependent on the JWT issuer, this is intended to reflect the service or services that the JWT is valid for.
This could for example have a value of 'myAPIName' or 'myAPIName/test' or even ['myAPIName/test1', 'myAPIName/test2'].
If you need to validate an individual aud claim you have two choices, you can either have different Lambdas authorizing different api routes and methods with hardcoded aud variables or you can get the name of the api being called and map it back to something that would match an aud claim.
For example the method arn for the incoming request can be found in event.methodArn. This looks like arn:aws:execute-api:{regionId}:{accountId}:{apiId}/{stage}/{httpVerb}/ potentially with [{resource}/[{child-resources}]] dependent on your implementation. With a bit of string manipulation, you could map this back to the format of an audience claim (whatever that looks like for you).
If you would rather work an api name instead of an api name you can use the apigateway.getrestapi method for whatever sdk you are using. Documentation on this method for the JavaScript sdk can be found here.
The JWT may have a sub(subject claim), again dependent on implementation of the JWT issuer, this could relate to any of your users (or at least the users the JWT issuer knows about). Validating this beyond checking the signature of the JWT is pointless. The best you could do is check if that user exists and only if you have access to the same user database that the JWT issuer does. Although this can be used to ensure that UserA only has access to UserA's data. Again, you are trusting that the fact a bearer has a token is proof that they have authenticated (proved who they are).
I hope that answers part one of your question.
In regards to part 2, the advantage of using a Lambda Authorizer over doing the authorization in the target Lambda is caching.
So envisage I have a token valid for one hour and I call your API once every second for 30 minutes (1,800 calls). You have a Lambda authorizer with a response cache time of 10 minutes. That means you check the JWT 3 times for 1800 api calls.
But if you do the validation of that token in your target Lambda, you are doing that processing 1,800 times.
There is authentication with AWS Cognito which will allow you to use authentication outside your application and still being able to let your application divide the users into subgroups based on custom roles or similar.
If for some reason you cannot use AWS Cognito (Why not?) there is both the possibility of custom authentication (the same for each function with SLS) or even authenticating with custom code inside each lambda function.
I would recommend to use Cognito, and only custom if you know you need it.
Normally, a csrf token is generated by the server and then sent to the client. When the client submits a form, the token is passed back to the server, which then gets verified.
If I am just using API Gateway and Lambda, how would I ensure that all POST/PUT requests are valid, and protect against csrf attacks? There isn't much written about the subject that I could find, and I'm not sure how to persist a generated csrf token so that all lambda functions can access it anyway.
Is this something that AWS already handles for me, or do I need to specifically configure it in a special way?
While I haven’t done (or even tried) that myself, 2 possible solutions could be:
The obvious one: Persist the data in one of the storages offered by AWS
The less obvious one: use a token that does not need persistence. For instance, JWT (JSON web tokens) can be used statelessly insofar as all servers (in your case: lambda functions) only need to know a shared secret to be able to verify client-side tokens. To prevent re-using a previously generated and used token (in other words: ensure a token is used only once), you could add data to the token payload which describes the form, for instance using an entity identifier plus version number, or simply add a expiration timestamp to the token payload – whatever fits your use case.