I want to create a simple AWS lambda that makes a url request to a rest endpoint. The thing I’m not sure about is I know for lambdas the response size limit is 6mb, and the response that comes back from the rest endpoint I intend to have it call fluctuates and sometimes can go beyond that limit.
I don’t necessarily need to return the response from the rest endpoint that the lambda will make a call to, but rather I feel like I can have the lambda write the contents of the response into a database instead of returning it, and then just return a response code at the end. I was wondering if this workaround will work or will I still have issues with the 6mb size limit on some manner?
Still in the prototyping phase, just a general question that I can elaborate more on if necessary
Related
I am wondering if there is a standard way to reject requests with the same body sent within a few seconds at the API gateway itself.
Forex: Reddit rejects if I try to post the same content within few seconds in a different group. Similarly, if I make a credit card payment for the second time, it automatically rejects it.
I am wondering if there is a way to have the same behavior in the AWS API gateway itself so that we are not handling it in lambda functions with dynamoDB and stuff.
Looking forward to efficient ways of doing it.
The API Gateway currently doesn't offer a feature like that, you'd have to implement this yourself.
If I was to implement this, I'd probably use an in-memory cache like ElastiCache for Redis or Memcached as the storage backend for deduplications.
For each incoming request I'd determine what makes it unique and create a hash from that.
Then I check if that hash value is in the cache already. If that's the case it 's a duplicate and I reject the request. If it isn't already in the cache, I'd add it with a time to live of n seconds (The time interval in which I wish to deduplicate).
I'm trying to do ACL by asserting if the item in DynamoDB whose field UserId is really the one logged in which is event.requestContext.identity.cognitoIdentityId.
But, I'm afraid that it can be spoofed just like HTTP headers etc.
My question is, is that safe?
No, this cannot be spoofed in the same way HTTP request headers can. If the request comes in through API Gateway, as a Lambda proxy integration, then there's nothing the browser can do that would allow these values to be overwritten, because this portion of the Lambda event structure is created by API Gateway and not copied from the request. Anything injected into the HTTP request would appear elsewhere in the event structure -- not here. (The HTTP request is in event.input -- which is a sibling object of event.requestContext -- not a parent.)
But then again... yes, this could be spoofed in certain other misconfiguration scenarios -- if, for example, your Lambda function allows itself to be invoked other than by your API Gateway deployment -- then of course the invoker could craft an entire event structure that had nothing to do with any HTTP request and invoke your Lambda function with it. This is perhaps too obvious to mention, since it's implicit from the way you can test a Lambda function from the console, but I mention it for thoroughness. Send a forged test event to your Lambda function using the Lambda console's test function, and naturally the Lambda function processes what you sent it.
So, unsurprisingly, with careless and overly broad permissions, yes, anything is possible... but used as intended behind API Gateway, as a Lambda Proxy Integration, I'd say no.
I have been researching this question for many hours.
I found this post where the author extracts the userId from the Token via:
const userId = await services.getUserIdFromToken(event.headers.Authorization);
This appears to be a safer way to handle setting the userId but all the other examples I have seen use event.requestContext.identity.cognitoIdentityId.
I'm trying to do some AB testing and inserting a Lambda in our AWS CloudFront setup seems a good way to do it. My company has problem with NodeJs. They were burned by some old devs using it and have banned its use.
Lambda supports a bunch of languages so no big deal. But after some research and looking at the source. Only the Javascript seems to be able to interact with CloudFront events and passing the request forward. All the libraries for other languages seem to only emit response events and don't allow for the passing forward of the request after modification.
Am I missing something or is this scenario only supported with JS?
Lambda#Edge supports only Node.js runtimes.
You must create functions with the nodejs6.10 or nodejs8.10 runtime property.
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-requirements-limits.html#lambda-requirements-lambda-function-configuration
Your observation about other runtimes "seem to only emit response events and don't allow for the passing forward of the request after modification" isn't really an accurate representation of what happens. Node.js doesn't do any special "passing forward" magic. CloudFront interprets the response payload from the Lambda function to determine how to proceed. If it's a request trigger and the Lambda function returns something structured like a request, then CloudFront continues processing the request, as modified. Otherwise CloudFront interprets the response payload as a response to be returned to the caller. Everything is handled by CloudFront's interpretation of what the Lambda function returns.
Any Lambda runtime could theoretically have accomplished the same purpose, but Lambda#Egde was designed for -- and currently only supports -- the Node.js runtimes, which are lightweight and fast, as they need to be, because CloudFront invokes the trigger function synchronously and then blocks (suspends its own processing) while waiting for the function to return.
I have used AWS API Gateway with the endpoint as a lambda function. I have enabled the cache functionality provided by API Gateway, to reduce both response time & the number of calls forwarded to my lambda function.
The lambda function queries another data store to return data. If data is not found, an asynchronous call is made to update the data store and "data not found" is returned to the caller. Now the API Gateway is even caching this result, which we do not want to happen. This results in the cache always returning "data not found" for its lifetime (1 hr TTL), even though data is updated asynchronously in the data store.
I'm aware of the request header (Cache-Control: max-age=0) which can invalidate cache and get response directly from Lambda, as mentioned in this documentation page.
But this will not be useful because the caller is unaware whether data is present in data store or not and hence cannot selectively send such request header.
So, my 2 questions are:
Does the API Gateway caches other HTTP response as well, like 404 (apart from 200)?
Is it possible to tell the API Gateway not to cache specific responses?
As you observed, API Gateway caches the result of your endpoint regardless of the status code.
No, not at this time. I can bounce the idea to see if this is something we can support in the future.
Even if API Gateway conditionally did not cache the 404, the caller would need to call the endpoint again anyway, so why not return the result synchronously? This pattern is how most cached APIs that I'm aware of behave and would allow your API to work with what API Gateway offers today.
I'd like to define a lambda. When it receives a POST request, I'd like to make another POST request to an external uri (say, splunk or apigee or anything outside of AWS). Is this possible? Does Lambda allow the internet access? I googled but did not find a good answer for this one.
Yes, you can run pretty much any code that you would run on a normal EC2 instance. For instance, if you write your Lambda in node.js you can use the request library to make HTTP calls out to other webservices. The same is true of Java or Python as long as you include whatever library you want to use to make the call in your Lambda. Just make sure you set the Lambda timeout high enough to allow your call(s) enough time to complete.
I wrote a blog post that shows a simple example of a Lambda calling out to a weather API(HTTP GET) to get weather for a zip code and post it in Slack: http://www.ryanray.me/serverless-slack-integrations