RDS Database details:
I have an publicly accessible RDS database which is restricted to be accessible to only specific IP address.
The RDS have 2 security groups attached to the RDS:
1st security group allows a specific IP address to access it.
2nd security group allows multiple specific IP addresses to access
it.
Note: I know I can just keep a single security group for this purpose.
This RDS database have the default VPC and 3 public subnets. All these subnets have 2 routes in route tables:
Destination: default vpc's cidr range and Target: local
Destination: 0.0.0.0/0 and Target: internet gateway
Lambda function details:
I have a lambda function, which does not have any vpc configured.
I am trying to access the RDS database from code in the lambda function. Since lambda doesn't have any specific IP address, I couldn't add inbound rule in RDS's security group to allow it. When I simply add inbound rule in RDS's security group to allow access from all IPs then my lambda function code is able to connect to RDS database and work with it.
I wouldn't want my RDS database to be allowed access from anywhere on the internet, what configuration/settings do I need to make to be able to access RDS from lambda.
I also tried configuring the lambda function to be in same vpc and same subnets as that of for RDS instance; but I haven't been successful in making a connection to the RDS database.
Best in this case is to use the same VPC for your lambda as the RDS instance. This is cheaper and is more secure. You don't have to setup a internet gateway and elastic IP. And you can only access your RDS instance from within the VPC.
You can enable this using Lambda management console. Select Lambda function which need access to RDS instance and then go to Configuration -> Advanced settings and select the VPC (which is your RDS instance is in) you need it to access.
For instances attached to the same security group—make the security group the source for the inbound rule and the destination for the outbound rule.
For instances in different security groups—make sure that both security groups allow access to one another.
If this is still not working check the error message that is returned when setting up the connection with RDS. Timeout means issue with network.
find out more here https://aws.amazon.com/premiumsupport/knowledge-center/connect-lambda-to-an-rds-instance/
Related
I have a PostgreSQL instance running on an EC2 instance and I've set up this instance security group to give me access from my IP.
How can I do the same for a Lambda endpoint?
I've included the lambda security group in the EC2 security group (using the same strategy I did for my IP address), but it's not working. I'm getting the error message: Can't reach the database at IP XXXX
What's the missing step here?
Thank you!
You should configure the AWS Lambda function to connect to the same VPC as the Amazon EC2 instance, then configure the security groups as:
One security group on the Lambda function (Lambda-SG) with Outbound rules permitting All Outbound traffic
One security group on the EC2 instance that is running PostgreSQL (DB-SG) with Inbound rules that permit access on port 5432 from Lambda-SG
That is, DB-SG should specifically reference Lambda-SG. This will permit the Lambda function access to the EC2 instance, regardless of which IP address the Lambda function is using.
Alternatively, you could choose to not connect the Lambda function to the VPC, in which case the connection request would come in from the Internet. However, I don't think that there is a fixed range of addresses used by Lambda (it is not listed on AWS IP address ranges - AWS General Reference), so it would not be possible to restrict access.
Just created an VPC for EKS Cluster and started RDS PostgreSQL instance with custom VPC.
Custom VPC has Subnets.
My Custom VPC has Internet Gateway attached.
EKS and RDS is in same VPC so they have internal communication.
My problem is that i want to connect to RDS from my local machine and i am unable. Regarding the problem i have created a new Security Group with Inbound Rules for PostgreSQL.
PostgreSQL TCP 5432 0.0.0.0/0 –
Im still unable to connect
UPDATE
RDS is Publicly accessible
Security group allows access to RDS
In order to connect to RDS instances from the internet you need to do these 3 things
Deploy your RDS instance in a "public" subnet. This means the subnet must have an Internet Gateway attached to it so it can respond properly to outbound requests
In your RDS instance under Connectivity, extend the Additional configuration section, and then choose Publicly accessible.
Make sure the security group allows access to your RDS instance.
Note: exposing a database to public access is not secure. What I recommend you to do is create a proxy with haproxy or a VPN.
To be able to connect to the RDS database remotely you need to select "yes" option for the "Public Accessibility" setting for you database. Here are some additional configurations that need to be taken into account (form AWS docs):
If you want your DB instance in the VPC to be publicly accessible, you
must enable the VPC attributes DNS hostnames and DNS resolution.
Your VPC must have a VPC security group that allows access to the DB
instance.
The CIDR blocks in each of your subnets must be large enough to
accommodate spare IP addresses for Amazon RDS to use during
maintenance activities, including failover and compute scaling.
Best,
Stefan
I was running a serverless web application on a lambda inside a VPC, and connecting to a Aurora-MySQL RDS instance, with inbound rules to allow traffic from the security group of the lambda
The connection was working fine, however, quite often the lambda cold start was giving me a timeout.
After some research, I found out that running a lambda on a VPC brings an additional cost on startup and I saw the recommendation in more than 1 place to avoid using lambda on a VPC except if you strictly need to access some resource in the VPC.
So, I decided to move my RDS to a publicly accessible instance, so my lambda can access it over the internet and remove the lambda from the VPC.
So, I changed the RDS Public accessibility option to Yes and edited the security group to allow inbound connection from any IP.
I have also removed the VPC from the lambda, so the lambda is not running on a VPC anymore
I thought it was gonna be enough.
But then my lambda started failing to connect to the database
I tried to connect using my local client, again, failure
tried pinging to the hostname, got request timeouts
After digging a bit into it, I found that my DB instance subnet group having some private subnets might be a problem (?)
So, I have created a new subnet group with only public subnets, and tried to move my db instance to the new subnet group... but got this message:
You cannot move DB instance my-instance to subnet group my-new-group. The specified DB subnet group and DB instance are in the same VPC.
Ok, it seems that I can't move to a different subnet in the same VPC, I started trying to create a new VPC, but it doesn't seem to be right and I'm sure there is something else I am missing here.
I also read about Network ACL, and thought that this might be the problem, but my rules seem to be fine, with the default rule to allow any traffic (and the rule * to DENY)
ALL Traffic ALL ALL 0.0.0.0/0 ALLOW
My RDS Network settings
Subnet group
default
Subnets
subnet-11111111
subnet-22222222
subnet-33333333
subnet-44444444
subnet-55555555
subnet-66666666
Security
VPC security groups
default (sg-111111)
( active )
Public accessibility
Yes
My Security group inbound rules
Type Protocol Port range Source Description - optional
All traffic All All 0.0.0.0/0 -
All traffic All All ::/0 -
Still can't connect, can't connect with my local client, can't even ping it:
Connecting through my local client
Can't connect to MySQL server on 'my-instance.xxxxxxxxxx.us-east-1.rds.amazonaws.com'
ping my-instance.xxxxxxx.us-east-1.rds.amazonaws.com
PING ec2-xx-xx-xx-xx.compute-1.amazonaws.com (xx.xx.xx.xx): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Any idea of what I am missing here?
UPDATE
My VPC has internet access (I can access internet services from it, not an issue), I have an Internet Gateway and NAT Gateway in place.
I'm using Zappa for the lambda deployment, which takes care of creating a keep-warm function... however, I know that concurrent requests could still be an issue
The issue with VPC in lambda is that it can add 10s on the cold start, which is a no-deal for some of my use cases:
https://www.freecodecamp.org/news/lambda-vpc-cold-starts-a-latency-killer-5408323278dd/
Besides enabling "public access" on your RDS instance, you need to enable internet to the VPC using an internet gateway. After you attach it to the VPC, you need to route the data from the database subnets to the internet gateway. Check here
But I would not advise you expose your database like this. If you are having issues with lambda cold start, you should create an event to keep it warm.
Things you need to is :
Create new subnet group with default VPC
assign two subnet for availability zone
then modify your RDS instance
change subnet group to newly created group
mark "Publicly accessibility" to Yes.
Check your VPC is using internet gateway or not.
Check lambda security group whether it's open for outbound connection for Database port is not available or not.
No need to create different VPC for RDS. Use Default VPC.
As recommended by #stargazer try to not to expose publicly or out of VPC. Its works well inside VPC.
I am having issues connecting Elasticache from a Lambda function
I have done the following:
Created a new security group
Assign security group to Lambda and assigned two private subnets of same VPC
Created an Elasticache redis cluster and assigned the above security group
Created a NAT gateway address, assigned and ElasticIP (for the above VPC)
Internet works and I verified this by opening stackoverflow.com URL. However Elasticache auto discovery times out. Is there any additional configuration that needs to be done?
Update
I added inbound rule for Security Group to port 6379 and now still times out.
You will need to attach an IAM policy to the Lambda execution role that grants it the appropriate access permission to ElastiCache. If you are restricting access you will also need to check the VPC setting of the Lambda function and make sure it has the ability to access the subnet the EC cluster is located.
Take a look at the following article for specifics:
https://docs.aws.amazon.com/lambda/latest/dg/services-elasticache-tutorial.html
I run a service on my EC2 instance and I want to setup an inbound rule that only allows my lambda function to access it. The security group allows me to restrict access by a specific IP, but I don't think that lambda functions have a specific IP assigned. Is there a way to do what I want?
If you are enabling VPC access for your Lambda function, as per this blog post, then you will create a security group for your Lambda functions to use within your VPC. All you need to do at that point is go into the security group your EC2 instance is using and grant access to the security group the Lambda function is using. This is the method I recommend.
If you are not using VPC access then your EC2 instance would need to be publicly accessible and you would basically be going over the internet to access the EC2 instance from the Lambda function. If that's the case then there is no good way to restrict that in the security group. You could (with difficulty) open it up to only requests that originate within AWS, but that still leaves it open to all other users of AWS. If you must go over the internet to access your EC2 instance from Lambda then it would be best to send some sort of security token with each request Lambda sends, and ignore any requests on the EC2 server that don't contain that security token.
What happens if we want our Lambda function to access resources in our VPC? This is a bit of a puzzlement because the Lambda function does not have a stable IP address that we can use as a source in our VPC security groups or in our subnet ACLs.
AWS has overcome these limitations by allowing you to specify a subnet and security group to associate with the Lambda function. You would think that this security group and subnet ACL would control traffic in and out of the Lambda function, but this is wrong. Firstly, Lambda functions do not listen for traffic on any port and so the concept of inbound traffic to a Lambda function is not applicable. Secondly, outbound connections from the Lambda function to VPC resources are not restricted in any way by the Lambda function’s security group. So, what is this mysterious Lambda function security group good for and how does it work?
It turns out that the Lambda function’s security group is just a naming placeholder that we can use in our other EC2 security groups. For example, a Lambda function can make a GET request to an EC2 instance on a private subnet in your VPC. We start out with two security groups. The first one is a normal security group that controls traffic to and from our EC2 instance that will be serving up the files on port 8080. It has a single inbound rule that allows port 8080 traffic with a source being the Lambda function’s security group. Recall that security group rules can specify source or destination as another security group which is a placeholder for all the hosts that are members of that source or destination security group.
The Lambda function’s security group has no rules whatsoever. None are required. It is merely a placeholder for the Lambda function that allows us to specify the Lambda function as source in our other EC2 security groups. Remember that the Lambda function has a random IP address that changes from day to day and hour to hour, so it is not possible to specify it as source for the inbound traffic to the EC2 instance.
Security groups normally control traffic in and out of a network interface but in the case of an AWS Lambda function security group, there is no interface and no rules – it is merely a placeholder.
A Lambda with no VPC association will be on the Internet and assigned with an arbitrary Public IP (obviously from Amazon Registered IPs) which will not be a fix address. So it is almost impossible to whitelist that IP in your EC2's SG since next time that Lambda spins up the IP could change and is not predictable.
However there is a way to Partially limit the Network Access only to the Resources Located in the VPC including Lambda's(since components will be assigned with Internal-IPs). Let's say you want that EC2 server only be accessible from Internal vpc network, therefor you place it in your Internal Subnet with no Public IP assigned to it. Now You can set SG on your EC2 to only accept IP from the Internal subnet CIDR range of your VPC. By Associating Lambda to that VPC and Placing it in the Private Subnet, Lambda will get arbitrary IP from Internal CIDR Range of your VPC which obviously falls in the SG range already configured for your EC2 (If you are going to have lots of Lambda in parallel just make sure you have enough amount of IPs within your defined CIDR range).
If you want your components to Communicate Internally meanwhile be able to access the Internet as well you can add a NAT Gateway Routable to IGW and then you will add Routing rule to the Internal Subnets to point to your NAT Gateway. Therefore all your component within the Internal subnet will be assigned with a Routing Tables pointing to NAT and subsequently the Internet.
Your Lambda will have a temporary IP assigned to it when it runs.
If you configure the IAM role attached to it so it can Allow/Revoke Security Group Ingress, you can make it "let himself in" to your sec group.
Check out this article for an example on adding an IP or CIDR block to your inbound list.
I would clean it up straight away when the function is done.
You can refer below articles to do the same,
1. https://medium.com/#justanotherspyy/how-to-connect-your-lambda-function-securely-to-your-private-rds-instances-in-your-vpc-29789220a33
2. https://medium.com/orchestrated/steps-to-secure-aws-serverless-lambda-part-1-a6e5d1b05f45
The summary would be,
1. Create a role for Lambda using below two policies,
AWSLambdaBasicExecutionRole — provides CloudWatch logging.
AWSLambdaENIManagementAccess — provides ENI description, creation, and deletion.
2. Put lambda in the same VPC of EC2 and create a security group for the lambda function.
3. Put this security group in EC2's security group's inbound rules.