I have a PostgreSQL instance running on an EC2 instance and I've set up this instance security group to give me access from my IP.
How can I do the same for a Lambda endpoint?
I've included the lambda security group in the EC2 security group (using the same strategy I did for my IP address), but it's not working. I'm getting the error message: Can't reach the database at IP XXXX
What's the missing step here?
Thank you!
You should configure the AWS Lambda function to connect to the same VPC as the Amazon EC2 instance, then configure the security groups as:
One security group on the Lambda function (Lambda-SG) with Outbound rules permitting All Outbound traffic
One security group on the EC2 instance that is running PostgreSQL (DB-SG) with Inbound rules that permit access on port 5432 from Lambda-SG
That is, DB-SG should specifically reference Lambda-SG. This will permit the Lambda function access to the EC2 instance, regardless of which IP address the Lambda function is using.
Alternatively, you could choose to not connect the Lambda function to the VPC, in which case the connection request would come in from the Internet. However, I don't think that there is a fixed range of addresses used by Lambda (it is not listed on AWS IP address ranges - AWS General Reference), so it would not be possible to restrict access.
Related
RDS Database details:
I have an publicly accessible RDS database which is restricted to be accessible to only specific IP address.
The RDS have 2 security groups attached to the RDS:
1st security group allows a specific IP address to access it.
2nd security group allows multiple specific IP addresses to access
it.
Note: I know I can just keep a single security group for this purpose.
This RDS database have the default VPC and 3 public subnets. All these subnets have 2 routes in route tables:
Destination: default vpc's cidr range and Target: local
Destination: 0.0.0.0/0 and Target: internet gateway
Lambda function details:
I have a lambda function, which does not have any vpc configured.
I am trying to access the RDS database from code in the lambda function. Since lambda doesn't have any specific IP address, I couldn't add inbound rule in RDS's security group to allow it. When I simply add inbound rule in RDS's security group to allow access from all IPs then my lambda function code is able to connect to RDS database and work with it.
I wouldn't want my RDS database to be allowed access from anywhere on the internet, what configuration/settings do I need to make to be able to access RDS from lambda.
I also tried configuring the lambda function to be in same vpc and same subnets as that of for RDS instance; but I haven't been successful in making a connection to the RDS database.
Best in this case is to use the same VPC for your lambda as the RDS instance. This is cheaper and is more secure. You don't have to setup a internet gateway and elastic IP. And you can only access your RDS instance from within the VPC.
You can enable this using Lambda management console. Select Lambda function which need access to RDS instance and then go to Configuration -> Advanced settings and select the VPC (which is your RDS instance is in) you need it to access.
For instances attached to the same security group—make the security group the source for the inbound rule and the destination for the outbound rule.
For instances in different security groups—make sure that both security groups allow access to one another.
If this is still not working check the error message that is returned when setting up the connection with RDS. Timeout means issue with network.
find out more here https://aws.amazon.com/premiumsupport/knowledge-center/connect-lambda-to-an-rds-instance/
There are couple of questions around this, but none seem to solve my specific problem.
So i have a publicly accessible RDS instance in a VPC with two Security Groups : one the default SG for VPC & other to allow incoming connections from our developer machines. I am able to successfully connect to this RDS via my developer machines(personal laptops/PCs) as a result of the 2nd Security group.
Now, when i try to connect to this RDS via a Lambda running in same VPC, it fails with this error : "java.net.UnknownHostException: ****.rds.amazonaws.com: Name or service not known "
My lambda is getting launched in the same VPC as RDS and i have chosen all the RDS VPC subnets for the lambda. It also is tied to the default VPC security group.
Below is my understanding(obviously not right as connection is not working), what am i missing here?
The Default VPC SG(security group) associated to the lambda has outgoing rules set as 0.0.0.0/0 , hence it should allow lambda to access the whole world; so at least the RDS in VPC should be accessible.
The same default VPC SG should allow the RDS to accept incoming connections from any ip in the SG subnets(inbound rule mentions SG/default) ; hence lambda should be able to access it.
The error message is indicating that it cannot resolve the DNS Name of the RDS database (Name or service not known).
I suspect that this is due to the fact that the RDS instance has been configured as publicly accessible, which gives it a public DNS Name and IP address.
An AWS Lambda function connected to a VPC will send all network traffic via the VPC. It does not have access to the Internet unless there is a NAT in the VPC and the Lambda function is connected to a Private Subnet. Thus, the connection attempts from the Lambda function are possibly attempting to resolve a Public DNS Name, but is failing to do so.
This could likely be resolved by changing the RDS instance to be Publicly Available = No. This not only makes it safer, but it means that the DNS Name will resolve to a Private IP address, which should work correctly with a Lambda function connected to the VPC.
Alternatively, you could configure the Lambda function to not be connected to the VPC. This gives it Internet access, so it will connect to the RDS database just like any other computer on the Internet.
So the problem was that in my developer machine, I was using the db instance endpoint , but in lambda i was using the cluster endpoint. They look similar, hence i was not able to catch it earlier. On fixing the lambda to use the instance endpoint, both lambda & developer machines can successfully connect to the RDS.
However, I am now curious why the cluster url is not accessible, but i will create another question for that.
I'm not really sure where to start here because i'm not very experienced on this topic. I have a Lambda API and some resources on a EC2 Instance. I want to protect these resource on EC2 in a way it is only accessed by the Lambda.
For that to happen I added the Lamdba to the VPC (I added a role to Lambda so that it could access the VPC), added the lambda to the Security Group and set the VPC subnet addresses to the lambda as well.
I have 2 problems now:
1- How do I open this port on the EC2? I ask that because when I set the Inbound rule, this port becomes open to the external world. Is that open by default, all ports?
2- What address should I use to connect to the EC2 from my Lambda? Because currently I'm using the public addresses (DNS that ends like amazonws.com, but that's certainly wrong, first because the Lambda loses internet connection when we add them to VPCs and because I did not add this port to the Inbound rules.
Could someone lend a hand?
Best practice would be to assign your Lambda function to a different security group than the EC2 instance. No inbound rules needed in the Lambda function's security group.
In the EC2 function's security group you would open the port, with the source as the Lambda function's security group ID. This means that only resources that belong to the Lambda function's security group can access the EC2 instance on the specified port.
Finally, in your Lambda function, access the EC2 instance using its private IP or private DNS name.
How can I create this scenario:
- A private subnet-1
- Lambda in subnet-1
- RDS in subnet-1(same subnet with Lambda)
- Both inside the same VPC.
is the above feasible and is it a good architecture ?
I have tried implementing it only issue is, it's timing out. But, when the Lambda is out of the subnet-1 it's working fine.
My understanding is since both RDS and Lambda are in the same subnet they should easily communicate.
It is perfectly okay to have Amazon RDS and the AWS Lambda function connected to the same private subnet.
Some things to note:
If the Lambda function also requires access to the Internet (eg to make calls to Amazon S3), then the VPC will also require a NAT Gateway in a public subnet.
The Lambda function should refer to the RDS instance by DNS Name. This should resolve to a private IP address local to the VPC.
The Lambda function should be assigned a Security Group (eg Lambda-SG)
The security group associated with the RDS instance (RDS-SG) should permit inbound access on the appropriate port (eg 3306 for MySQL) from Lambda-SG.
That is, RDS-SG should permit inbound connections from Lambda-SG.
I run a service on my EC2 instance and I want to setup an inbound rule that only allows my lambda function to access it. The security group allows me to restrict access by a specific IP, but I don't think that lambda functions have a specific IP assigned. Is there a way to do what I want?
If you are enabling VPC access for your Lambda function, as per this blog post, then you will create a security group for your Lambda functions to use within your VPC. All you need to do at that point is go into the security group your EC2 instance is using and grant access to the security group the Lambda function is using. This is the method I recommend.
If you are not using VPC access then your EC2 instance would need to be publicly accessible and you would basically be going over the internet to access the EC2 instance from the Lambda function. If that's the case then there is no good way to restrict that in the security group. You could (with difficulty) open it up to only requests that originate within AWS, but that still leaves it open to all other users of AWS. If you must go over the internet to access your EC2 instance from Lambda then it would be best to send some sort of security token with each request Lambda sends, and ignore any requests on the EC2 server that don't contain that security token.
What happens if we want our Lambda function to access resources in our VPC? This is a bit of a puzzlement because the Lambda function does not have a stable IP address that we can use as a source in our VPC security groups or in our subnet ACLs.
AWS has overcome these limitations by allowing you to specify a subnet and security group to associate with the Lambda function. You would think that this security group and subnet ACL would control traffic in and out of the Lambda function, but this is wrong. Firstly, Lambda functions do not listen for traffic on any port and so the concept of inbound traffic to a Lambda function is not applicable. Secondly, outbound connections from the Lambda function to VPC resources are not restricted in any way by the Lambda function’s security group. So, what is this mysterious Lambda function security group good for and how does it work?
It turns out that the Lambda function’s security group is just a naming placeholder that we can use in our other EC2 security groups. For example, a Lambda function can make a GET request to an EC2 instance on a private subnet in your VPC. We start out with two security groups. The first one is a normal security group that controls traffic to and from our EC2 instance that will be serving up the files on port 8080. It has a single inbound rule that allows port 8080 traffic with a source being the Lambda function’s security group. Recall that security group rules can specify source or destination as another security group which is a placeholder for all the hosts that are members of that source or destination security group.
The Lambda function’s security group has no rules whatsoever. None are required. It is merely a placeholder for the Lambda function that allows us to specify the Lambda function as source in our other EC2 security groups. Remember that the Lambda function has a random IP address that changes from day to day and hour to hour, so it is not possible to specify it as source for the inbound traffic to the EC2 instance.
Security groups normally control traffic in and out of a network interface but in the case of an AWS Lambda function security group, there is no interface and no rules – it is merely a placeholder.
A Lambda with no VPC association will be on the Internet and assigned with an arbitrary Public IP (obviously from Amazon Registered IPs) which will not be a fix address. So it is almost impossible to whitelist that IP in your EC2's SG since next time that Lambda spins up the IP could change and is not predictable.
However there is a way to Partially limit the Network Access only to the Resources Located in the VPC including Lambda's(since components will be assigned with Internal-IPs). Let's say you want that EC2 server only be accessible from Internal vpc network, therefor you place it in your Internal Subnet with no Public IP assigned to it. Now You can set SG on your EC2 to only accept IP from the Internal subnet CIDR range of your VPC. By Associating Lambda to that VPC and Placing it in the Private Subnet, Lambda will get arbitrary IP from Internal CIDR Range of your VPC which obviously falls in the SG range already configured for your EC2 (If you are going to have lots of Lambda in parallel just make sure you have enough amount of IPs within your defined CIDR range).
If you want your components to Communicate Internally meanwhile be able to access the Internet as well you can add a NAT Gateway Routable to IGW and then you will add Routing rule to the Internal Subnets to point to your NAT Gateway. Therefore all your component within the Internal subnet will be assigned with a Routing Tables pointing to NAT and subsequently the Internet.
Your Lambda will have a temporary IP assigned to it when it runs.
If you configure the IAM role attached to it so it can Allow/Revoke Security Group Ingress, you can make it "let himself in" to your sec group.
Check out this article for an example on adding an IP or CIDR block to your inbound list.
I would clean it up straight away when the function is done.
You can refer below articles to do the same,
1. https://medium.com/#justanotherspyy/how-to-connect-your-lambda-function-securely-to-your-private-rds-instances-in-your-vpc-29789220a33
2. https://medium.com/orchestrated/steps-to-secure-aws-serverless-lambda-part-1-a6e5d1b05f45
The summary would be,
1. Create a role for Lambda using below two policies,
AWSLambdaBasicExecutionRole — provides CloudWatch logging.
AWSLambdaENIManagementAccess — provides ENI description, creation, and deletion.
2. Put lambda in the same VPC of EC2 and create a security group for the lambda function.
3. Put this security group in EC2's security group's inbound rules.