I have a simple C# Lambda function that inserts a record into a table using Entity Framework. When I run the test locally (from my desktop machine) I can connect to the remote database just fine and the record gets inserted into the table at AWS just fine. When I upload the lambda to AWS and then send it data the function times out after 15 seconds. Since the code runs on my (external) desktop machine I am assuming that Lambda does not have permissions to connect to the internal RDS database from inside aws.
I have added AmazonRDSFullAccess to the permissions of the Lambda function. The Lambda function still times out.
What am I missing?
The Lambda function needs to be deployed to the same VPC as the RDS server.
It does not need the AmazonRDSFullAccess IAM policy attached.
The security group for the RDS server needs to allow inbound connections from the security group assigned to the Lambda function.
Related
I would like to use aws sam to setup my serverless application. I have used it with dynamoDB before. This was very easy to since all I had to do was setup a dynamoDB table as a resource and then link it to the lambda functions. AWS SAM seams to know where the table is located. I was even able ot run the functions on my local machine using the sam-cli.
With RDS its a lot harder. The RDS Aurora Instance I am using sits behind a specific endpoint, in a specific subnet with security groups in my vpc protected by specific roles.
Now from what I understand, its aws sams job to use my template.yml to generate the roles and organize access rules for me.
But I don't think RDS is supported by aws sam by default, which means I would either be unable to test locally or need a vpn access to the aws vpc, which I am not a massive fan of, since it might be a real security risk.
I know RDS proxies exist, which can be created in aws sam, but they would also need vpc access, and so they just kick the problem down the road.
So how can I connect my aws sam project to RDS and if possible, execute the lambda functions on my machine?
Here's what I've done:
I setup an Aurora Serverless MySql instance.
Created a security group for Cloud9 which allows me to access Aurora Serverless mysql.
Created a Lambda function which queries my db, I added the custom VPC and added the security group which lets me access Aurora Serverless (same one as Cloud9), the Lambda function works fine and can query my DB.
Before setting up Aurora Serverless, I had a RDS MySql instance which my Lambda functions could query, and had a little deploy script on my local machine which I ran to package my Lambda function's changes and uploaded them to their respective Lambda function. To setup CLI I just used this guide https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html.
Now that I have the security group in my Lambda function, AWS CLI doesn't let me upload the Lambda function, it gets stuck during the upload process. Note: I can still upload Lamda functions using the Lambda GUI in the AWS console.
Does anyone know what I can do to upload my Lambda functions using CLI again?
Here's a picture of where it gets stuck
What I'm trying to do
I am working on a lambda function which will simply register some metadata about files which are uploaded onto an s3 bucket. This is not about actually processing the data in the files yet. To start with, I just want to register the fact that certain files have been uploaded or not. Then I want to connect that metadata to QuickSight just so that we can have a nice visual about which files have been uploaded.
What I've done so far
This part is fairly easy:
Some simply python code with the pymysql module
Chalice to manage the process of creating and updating the lambda function
I created the database
Where I'm stuck
QuickSight is somehow external to AWS in general. So I had to create the RDS (mysql) in the DMZ of our VPC.
I have configured the security group so that the DB is accessible both from QuickSight and from my own laptop.
But the lambda function can't connect.
I configured the right policy for the role, so that the lambda can connect with IAM
I tested that policy with the simulator
But of course the lambda function is going to have some kind of dynamic IP and that needs to be in the security group
Any Ideas ??
I am even thinking about this right?
Two things.
You shouldn't have to put your RDS in a DMZ. See this article about granting QuickSight access to your RDS: https://docs.aws.amazon.com/quicksight/latest/user/enabling-access-rds.html
In order for a lambda to access something in a VPC (like and RDS instance) the lambda must have a VPC configuration. https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html
I have a lambda connected to an API gateway; it's deployed using sls and works great. However, it's datastore is an Aurora that is in the default VPC and is set to public. This is less than ideal, security-wise.
I have, in the past, set up Auroras in their own VPC on private subnets and had ec2s in that VPC easily access it. However, all of the material I have read about getting a lambda to use a VPC RDS states that the lambda itself should also reside in the VPC.
This concerns me because of the cold start issue. So, my questions are:
Is there a way for my 'no vpc' lambda to access an Aurora RDS that
lives in its own VPC without putting the lambda into the VPC itself?
There has been talk for some time that aws will be addressing the lambda VPC 'cold start' issue soon. Do we know when that is anticipated to happen? Will existing lambdas benefit from this change once it is instituted?
Is there some other method of securing a public RDS to restrict access to only my lambda (besides the obvious user/pass credentials)?
Thanks in advance
1. Is there a way for my 'no vpc' lambda to access an Aurora RDS that
lives in its own VPC without putting the lambda into the VPC itself?
No, if your RDS instance is not publicly accessible then your Lambda must be deployed in your VPC.
2. There has been talk for some time that AWS will be addressing the lambda VPC 'cold start' issue soon. Do we know when that is anticipated to happen? Will existing lambdas benefit from this change once it is instituted?
I don't think that a specific timeline has been officially communicated for this feature. Existing Lambda functions will obviously benefit from this change (after all, Lambda functions are just code that is continuously redeployed on containers). Information about the new architecture: AWS Lambda in a VPC Will Soon be Much Faster
3. Is there some other method of securing a public RDS to restrict access to only my Lambda (besides the obvious user/pass credentials)?
You can use IAM Database Authentication with Aurora. With this method, authentication is managed externally using IAM.
I'm trying to insert records into a Postgres database in RDS from a Lambda function. My Node.js lambda function works correctly when run locally, but the database connection times out when run in AWS.
I've read several articles and tutorials which suggest that AWS Lambda functions cannot access RDS instances that are within a VPC. For example: http://ashiina.github.io/2015/01/amazon-lambda-first-impression/
Unfortunately; it seems I am unable to create an RDS instance that exists outside of a VPC. At this dropdown I would expect to be able to select an option for "No VPC" or something along those lines.
Has this option been removed? Perhaps I have missed a step?
You can create a publicly accessible RDS instance. Then you should be able to access it from anywhere, inside or outside AWS. I believe that would get around your issue with Lambda. You are asked if the instances needs to be publicly accessible when you create a new RDS instance via the web console.
Or you could just wait a few weeks, as Lambda within a VPC is supposed to be enabled "later this year".
Edit: Note that newer Amazon accounts are restricted to VPC only resources. You can't create EC2 or RDS instances outside of a VPC anymore. That's why you don't see the "No VPC" option anymore.
Second Edit: VPC access for Lambda functions is now genearally available.
This question is awhile back, but for those of you who are using MySQL, now you can connect AWS Lambda with Aurora Serverless without VPC, utilizing their new Data API. Take a look at this example for details https://coderecipe.ai/architectures/77374273