I have a few Lambda that are executed from a Jenkins installed on a EC2 instance. I could successfully run a curl command from Jenkins to request any of a my Lambda using the URL.
I'd like to secure this architecture with a VPC. My EC2 instance already under the VPC vpc-001f4912569686a89 so I've put my Lambda under the same one.
I have also created a security group (sg-0de1a07dd602450f7) for my Lambda to allow only traffic from my EC2 instance. See the following (sg-0f62df4def386f83a) is my EC2 instance's security group) :
And here is my Lambda VPC configuration:
From what I understand, this should be enough to get my EC2 to communicate with my Lambda but my curl isn't returning me anything (probably a timeout).
EDIT
Here is my VPC route table :
Your security group setup seems correct.
Please address the following items:
Could you elaborate on the subnet that your EC2 Instance is running in?
Your Lambda function's container(s) can spawn in either of the two subnets
subnet-053af8185c4f6bbc7
subnet-0ac074c5b63e6db9b
Please check the VPC's route table and ensure traffic can flow between the potential subnets
Please check the Lambda function's subnets' ACLs to ensure they're not blocking traffic coming from the Jenkins instance
Related
I have a PostgreSQL instance running on an EC2 instance and I've set up this instance security group to give me access from my IP.
How can I do the same for a Lambda endpoint?
I've included the lambda security group in the EC2 security group (using the same strategy I did for my IP address), but it's not working. I'm getting the error message: Can't reach the database at IP XXXX
What's the missing step here?
Thank you!
You should configure the AWS Lambda function to connect to the same VPC as the Amazon EC2 instance, then configure the security groups as:
One security group on the Lambda function (Lambda-SG) with Outbound rules permitting All Outbound traffic
One security group on the EC2 instance that is running PostgreSQL (DB-SG) with Inbound rules that permit access on port 5432 from Lambda-SG
That is, DB-SG should specifically reference Lambda-SG. This will permit the Lambda function access to the EC2 instance, regardless of which IP address the Lambda function is using.
Alternatively, you could choose to not connect the Lambda function to the VPC, in which case the connection request would come in from the Internet. However, I don't think that there is a fixed range of addresses used by Lambda (it is not listed on AWS IP address ranges - AWS General Reference), so it would not be possible to restrict access.
I have an RDS in one AWS Account - say Acct-1.
The RDS is public (i know it's not a good idea and there are other solutions for that)
I have a lambda in another AWS Account - say Acct-2 which runs in a VPC.
I have setup VPC peering between the 2 accounts, the route table entries are in place as well as the security groups IN/OUT bound policies in place.
In Acct-2 I can verify that I can connect to the RDS instance in Acct-1 using a mysql cient from an EC2 instance. The EC2 instance is in the same subnet as the Lambda and they both have the same security group.
But the Lambda gets a timeout connection. The Lambda has the typical Lambda execution role that Allows logs, and network interfaces.
Thoughts on what could be missing ? Does the RDS need to grant specific access to the Lambda service even if it's running in a VPC ?
Clarification: There is no route to the RDS instance from the internet. Clearly, the ec2 host is able to resolve the Private IP for the RDS instance from the DNS name and connect.
Lambda is unable to resolve the private IP for the RDS instance.
I'm trying to keep the traffic within AWS so as to not pay egress costs.
I am having issues connecting Elasticache from a Lambda function
I have done the following:
Created a new security group
Assign security group to Lambda and assigned two private subnets of same VPC
Created an Elasticache redis cluster and assigned the above security group
Created a NAT gateway address, assigned and ElasticIP (for the above VPC)
Internet works and I verified this by opening stackoverflow.com URL. However Elasticache auto discovery times out. Is there any additional configuration that needs to be done?
Update
I added inbound rule for Security Group to port 6379 and now still times out.
You will need to attach an IAM policy to the Lambda execution role that grants it the appropriate access permission to ElastiCache. If you are restricting access you will also need to check the VPC setting of the Lambda function and make sure it has the ability to access the subnet the EC cluster is located.
Take a look at the following article for specifics:
https://docs.aws.amazon.com/lambda/latest/dg/services-elasticache-tutorial.html
I have a lambda function in a VPC that should be able to connect to SSM via a VPC endpoint. In fact, I have had this working in another VPC (the default), but just can't get it to work in a new VPC that I have built with CloudFormation. In addition, I am able to connect to S3 via a VPC endpoint for the same VPC:
The subnets that Lambda is being deployed in, are associated with the SSM VPC Endpoint.
The VPC endpoint is associated with a security group that accepts incoming requests on port 443 from the application layer security group (which is associated with the lambda function).
The Lambda function's application layer security group has an outgoing rule that allows it to broadcast anywhere 0.0.0.0/0
The NACL for the subnets allows all incoming and outgoing traffic.
The lambda function's role has permission to execute all SSM actions on all resources (just a temporary situation to rule out iam permissions)
I have tried console.log debugging, and it shows the lambda function timing out after it tries to get info from SSM.
I have tried using flow logs, but I can see no connections, or any data flowing to and from SSM.
I have compared the 2 VPC's (the one in which it works and the one in which it doesn't) and I can't see any differences in rules.
Can anyone suggest how I might debug this?
Thanks
Finally found the issue and thought I'd post this just in case anyone else has the same issue.
The VPC Endpoints needed to have private dns enabled.
When creating the VPC endpoint manually via the console, the Enable Private DNS Name checkbox is checked by default.
However, when creating the VPC endpoint using CloudFormation, the PrivateDnsEnabled property needs to be explicitly set to true.
I've created a new environment for my beanstalk app under a vpc.
I created a vpc with private and public subnets.
I've configured the security groups as outlined here:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo-vpc-basic.html
The environment seemed to setup correctly but the status is red.
Clicking on monitor reveals no data for latency or requests.
The website url returns error code 503.
Not sure what to do
This is almost certainly an issue with not being able to communicate with the NAT. The documentation is a little ambiguous when it comes to configuring your .ebextensions config files.
Don't mess with the Network ACLs for the VPC. The infrastructure needs to be able to talk to itself.
The NAT instance & Beanstalk ELB need to be in the Public subnet (10.0.0.0/24)
The Beanstalk instances themselves need to be in the Private subnet (10.0.1.0/24)
The Security Group that you apply to the instances needs to also apply to the NAT. That Security Group also needs to be on its own Inbound list so that everything inside that SG can talk to each other.
Using curl, try to make outbound calls. First on the Bastion Host, then on the NAT, then on a Beanstalk instance. Are any of them unable to make outbound calls?
I just recently spent a couple of weeks trying to develop a CloudFormation template that handles this stuff. I think that most of it is still in my brain. I think.
If these troubleshooting steps don't help, let me know and I'll dig a little deeper.
When you created the VPC, I assume you did it using the wizard. In this situation, the NAT is configured to accept calls from "default" security group.
When you created the Elastic beanstalk environment, did you specify that you want the new instances to be on "default" security group?
Check if the new instance belongs to the VPC's default security group in EC2 management area. If it is not a member of default Security group, add this security group to your NAT's inbound connections.
There is a good AWS CloudFormation template example to create VPC with public and private subnets and the NATs:
https://github.com/awsdocs/elastic-beanstalk-samples/blob/9720e38e9da155752dce132a31d8e13a27364b83/cfn-templates/vpc-privatepublic.yaml
May be very useful for those having issues with such VPC configuration.
To deploy it to your AWS using cli:
aws cloudformation deploy --stack-name mystackname --template-file path/to/vpc-privatepublic.yaml
Or you can deploy from CloudFormation UI dashboard.