I am testing out EG policies for my microservices app. One requirement is that whenever express gateway receives a request, I want to invoke a particular service, parse its result, and based on the result decide to proceed for downstream calls or return an error. It appears to be a very standard requirement. Is there any existing policy (could not find any here) for such scenarios or do I need to write a custom one? Thanks
this is Vincenzo — I am the maintainer of Express Gateway :)
Unfortunately you spotted a lack of Express Gateway, which is "post proxy" policies. Fundamentally right now the proxy policy is the last one to be executed and there's nothing else you can do before sending the request to the downstream client.
This is a limitation that we definitely need to fix, although you're the first one to bring up this use case.
This does not mean that you cannot do it now. I think it'd be kind of easy as well, but unfortunately you'd need to fork the Gateway and add some code.
If you could articulate a little bit more your use case, we might evaluate if there's a way to make it happen in the next release :)
Related
I'm using stripe with DRF and on top of that, I've implemented the library dj-stripe.
Everything works so far but I'm not really sure how to provision my product now. I do have access to the subscription and customer object for every user but these objects are quite complicated / big. I can't really do something like if user.subscription -> do this since the subscription could be e. g. deleted, inactive. I also need a more granular solution since I need to apply limits like:
if subscription.plan.product === "Entry Plan":
# allow user to only create 5 instances
I can't really find information on how to do this elegantly and consistently for an entire app.
If you're looking at the Subscription object for a customer, then yes you will need to check the status of the Subscription to ensure it's still active. dj-stripe appears to have a helper for this.
The implementation of provisioning access to your application/products is up to you, because it depends heavily on your business needs. If you have specific questions about challenges beyond the subscription status, I suggesting asking those clearly so that they can be addressed, but there is no concise way to explain how to generally provision access.
As I side note, I recommended reaching out to the author of dj-stripe with a thank you and ask what you can do to help get the docs for checking subscriptions fleshed out.
Is there a way to chain / consolidate multiple REST calls? For example, expose an api that would accept all order details and then when that api is called, make multiple calls for different steps like add to cart, checkout etc and when it’s all done, send a response back.
unfortunately we're not offering such feature just yet, although it's highly requested, so I feel we will start to work it kind of soon.
In meantime, you can reach almost the same feature through a plugin that you can write by yourself.
Please see this:
http://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html
LogType
You can set this optional parameter to Tail in the request
only if you specify the InvocationType parameter with value
RequestResponse. In this case, AWS Lambda returns the base64-encoded
last 4 KB of log data produced by your Lambda function in the
x-amz-log-result header.
Valid Values: None | Tail
So this means any user with valid credentials for invoking a function can also read the logs this function emits?
If so, this is an obvious vulnerability that can give some attacker useful information regarding processing of invalid input.
How do I configure an Amazon AWS Lambda function to prevent tailing the log in the response?
Update 1
1) Regarding the comment: "If a hacker can call your Lambda function, you have
more problems than seeing log files."
Not true: Lambda functions are also meant to be called directly form client code, using the SDK.
As an example, see the picture below from the book "AWS Lambda in Action":
2) Regarding the comment: "How is this a vulnerability exactly? Only someone you have provided AWS IAM credentials would be able to invoke the Lambda function."
Of course, clients do have some credentials, most of the time (for example,
from having signed in to your mobile app with their Facebook account, through Amazon Cognito). Am I supposed to trust all my users?
3) Regarding the comment: "Only if you have put some secure information to be logged."
Logs may contain sensible information. I'm not talking about secure information like passwords, but simply information to help the development team debugging, or the security team finding out about attacks. Applications may log all kinds of information, including why some invalid input failed, which can help an attacker learn what is the valid input. Also, attackers can see all the information the security team is logging about their attacks. Not good. Even privacy may be at risk depending on what you log.
Update 2
It would also solve my problem if I could somehow detect the Tail parameter in the Lambda code. Then I would just fail with a "Tail now allowed" message. Unfortunately the Context object doesn't seem to contain this information.
I think you can't configure AWS Lambda to prevent tailing the log in the response. However, you could use your own logging component instead of using the one provided by Amazon Lambda to avoid the possibility to expose them via the LogType parameter.
Otherwise, I see your point about adding complexity, but using API Gateway is the most common solution to provide the possibility to invoke Lambdas for clients applications that you do not trust.
You're right, not only it's a bad practice, it's obviously (as you already understood) introducing security vulnerabilities.
If you look carefully in the book you will also find this part:
which explains that in order to be more secure, the client requests should hit Amazon API gateway which will expose a clean API interface and which will call the relevant lambda-function without exposing it to the outer-world.
An example of such API is demo'ed in a previous page:
By introducing a middle-layer between the client and AWS-lambda, we take care of authentication, authorization, access and all other points of potential vulnerability.
This is a comment.
While this should be a comment, I am sorry that I do not have yet enough stackoverflow reputation to do so.
Before commenting on this, please note that lambda Invoke may result in more than one execution of your lambda (per AWS documentation)
Invocations occur at least once in response to an event and functions must be idempotent to handle this.
As the LogType is documented as a valid option, I don't think you can prevent it in your backend. However, you need to have a workaround to handle it. I can think of
1- Generate a junk 4KB tail log (by console.log() for example). Then, the attacker will get a junk info. (incur cost only in case of attacker)
2- Use step functions. This is not only to hide the log but to overcome the problem of 'Invocations occur at least once' and have a predictable execution of your backend. It incurs cost though.
Update: I figured it out, please see the answer post below.
I have an AWS API Gateway api defined with various resources and various GET and POST methods.
Everything works mostly fine.
POSTs are going through.
GETs return a response (JSON payload) except that the returned value seems to be a cached value.
My GET api calls a Lambda function that calls a query to RDS.
I can confirm my responses are stale because:
When I manually Query the RDS, I get the updated value
I have Cloud Watch logs enabled and the lambda function does not get called (I believe I have it set up correctly because when I test invoke the lambda, I can get Cloud Watch logs)
It did refresh once, but I think that was because I crossed some (like 1hr) caching threshold or something.
I understand that API Gateway generates a CloudFront behind the scenes.
And I feel that this is what is doing the caching. But that's just a guess and I have no proof. Maybe some kind of default caching TTL?
I obviously have caching turned off on my API Gateway stage.
I even tried enabling it, setting the TTL to 1, flushing the cache, and disabling cache again.
Each stage of that testing still returned the stale values.
I do not know if it is relevant, but additional details:
I have CORS enabled ("*")
I have Cognito authorizers enabled
I pass in the JWT token via the Authorization header (this is all working fine)
Is there some header I'm supposed to pass to request an uncached value?
I went to CloudFront, but here are no configurations there.
All other posts on API Gateway caching seem to be about caching not working or people asking about cache key specificity.
I haven't seen anything about the value ALWAYS being cached no matter what. So I feel like I'm missing something obvious...
Any help or debugging tips would be much appreciated!
Ok, so I feel like an idiot for answering my own question but hopefully it helps someone one day.
This was not an API Gateway caching issue.
The problem was a pymysql connection & lambda session caching issue.
My Lambda was using pymysql to query the MySQL RDS.
Per recommended performance reasons, I reused the connection across lambdas (meaning I did not close the connection each time).
The solution was to call
conn.commit()
after I did my fetchall()
What was happening was that my subsequent calls were returning a cached query result (termed a consistent read. Thanks! #Michael - sqlbot) I believe I probably had more than one lambda containers or something so when I was inactive for a while (ie busy reading stackoverflow posts), the lambda would unload. Then my next API gateway attempt would reinitialize a fresh lambda handler and a branch new connection would be created (without a cache). So this is why it seems to "sometimes work, then stop".
Apologies if I wasted anyone's time.
Thank you for asking and answering your own question. I had the same issue and was pulling hair out trying to find which API Gateway setting was caching the result.
We want to set-up an existing API as SAAS using AWS
Our code has been deployed via elastic-beanstalk, and we created access to the methods via Gateway to manage permissions.
We're now trying to log the user's activity, for billing purposes
Currently, the best solution we found involves a full logging of the calls (Enabled CloudWatch Logs + Log full requests/responses data), which looks quite heavy, and may even end up beeing expensive.
We reworked the request body in the integration request, by adding a mapping template for the body, but this seems heavy and complicated, whe hope there was a better solution we missed.
Basically, we replaced the default "passthrough" with a generated basic "passthrough" code, and added a value "MyUserArn" : "$context.identity.userArn" in it, which fills the requests body with a large mess, but looks like "The most reliable way to avoid to breaking something".
We'd like to just add the IAM user identifier in a header, or query string parameter, but failed to find if this is even possible. Several posts mention an "Invoke with caller credentials" option, but we didn't find this either.
Is is something related to cognito or something else ?
Are we doing something wrong ?
You have a couple different options for getting this information, both of which have trade offs:
Your current solution pulling the value from $context.identity in a mapping template and sending to your Lambda as part of the body. It seems like you are opposed to this given your "large mess" comment, but ultimately you have control over the content passed to your Lambda.
Enable "user caller credentials" on your method and then use identity value inside your Lambda. Currently this only works if you've used credentials vended from a Cognito authentication flow and does require that Lambda invocation also be part of your role policy, but doesn't require any modification of the template.
UPDATE Apologies, I somehow missed you were using Beanstalk and not Lambda. You can definitely just add a header to your integration request and simply have pull its value from $context.identity.userArn.
UPDATE 2 Double apologies, when using context variables in headers, you omit the $ so you need to use context.identity.userArn.