I am having issues connecting Elasticache from a Lambda function
I have done the following:
Created a new security group
Assign security group to Lambda and assigned two private subnets of same VPC
Created an Elasticache redis cluster and assigned the above security group
Created a NAT gateway address, assigned and ElasticIP (for the above VPC)
Internet works and I verified this by opening stackoverflow.com URL. However Elasticache auto discovery times out. Is there any additional configuration that needs to be done?
Update
I added inbound rule for Security Group to port 6379 and now still times out.
You will need to attach an IAM policy to the Lambda execution role that grants it the appropriate access permission to ElastiCache. If you are restricting access you will also need to check the VPC setting of the Lambda function and make sure it has the ability to access the subnet the EC cluster is located.
Take a look at the following article for specifics:
https://docs.aws.amazon.com/lambda/latest/dg/services-elasticache-tutorial.html
Related
I have a few Lambda that are executed from a Jenkins installed on a EC2 instance. I could successfully run a curl command from Jenkins to request any of a my Lambda using the URL.
I'd like to secure this architecture with a VPC. My EC2 instance already under the VPC vpc-001f4912569686a89 so I've put my Lambda under the same one.
I have also created a security group (sg-0de1a07dd602450f7) for my Lambda to allow only traffic from my EC2 instance. See the following (sg-0f62df4def386f83a) is my EC2 instance's security group) :
And here is my Lambda VPC configuration:
From what I understand, this should be enough to get my EC2 to communicate with my Lambda but my curl isn't returning me anything (probably a timeout).
EDIT
Here is my VPC route table :
Your security group setup seems correct.
Please address the following items:
Could you elaborate on the subnet that your EC2 Instance is running in?
Your Lambda function's container(s) can spawn in either of the two subnets
subnet-053af8185c4f6bbc7
subnet-0ac074c5b63e6db9b
Please check the VPC's route table and ensure traffic can flow between the potential subnets
Please check the Lambda function's subnets' ACLs to ensure they're not blocking traffic coming from the Jenkins instance
I have an RDS in one AWS Account - say Acct-1.
The RDS is public (i know it's not a good idea and there are other solutions for that)
I have a lambda in another AWS Account - say Acct-2 which runs in a VPC.
I have setup VPC peering between the 2 accounts, the route table entries are in place as well as the security groups IN/OUT bound policies in place.
In Acct-2 I can verify that I can connect to the RDS instance in Acct-1 using a mysql cient from an EC2 instance. The EC2 instance is in the same subnet as the Lambda and they both have the same security group.
But the Lambda gets a timeout connection. The Lambda has the typical Lambda execution role that Allows logs, and network interfaces.
Thoughts on what could be missing ? Does the RDS need to grant specific access to the Lambda service even if it's running in a VPC ?
Clarification: There is no route to the RDS instance from the internet. Clearly, the ec2 host is able to resolve the Private IP for the RDS instance from the DNS name and connect.
Lambda is unable to resolve the private IP for the RDS instance.
I'm trying to keep the traffic within AWS so as to not pay egress costs.
RDS Database details:
I have an publicly accessible RDS database which is restricted to be accessible to only specific IP address.
The RDS have 2 security groups attached to the RDS:
1st security group allows a specific IP address to access it.
2nd security group allows multiple specific IP addresses to access
it.
Note: I know I can just keep a single security group for this purpose.
This RDS database have the default VPC and 3 public subnets. All these subnets have 2 routes in route tables:
Destination: default vpc's cidr range and Target: local
Destination: 0.0.0.0/0 and Target: internet gateway
Lambda function details:
I have a lambda function, which does not have any vpc configured.
I am trying to access the RDS database from code in the lambda function. Since lambda doesn't have any specific IP address, I couldn't add inbound rule in RDS's security group to allow it. When I simply add inbound rule in RDS's security group to allow access from all IPs then my lambda function code is able to connect to RDS database and work with it.
I wouldn't want my RDS database to be allowed access from anywhere on the internet, what configuration/settings do I need to make to be able to access RDS from lambda.
I also tried configuring the lambda function to be in same vpc and same subnets as that of for RDS instance; but I haven't been successful in making a connection to the RDS database.
Best in this case is to use the same VPC for your lambda as the RDS instance. This is cheaper and is more secure. You don't have to setup a internet gateway and elastic IP. And you can only access your RDS instance from within the VPC.
You can enable this using Lambda management console. Select Lambda function which need access to RDS instance and then go to Configuration -> Advanced settings and select the VPC (which is your RDS instance is in) you need it to access.
For instances attached to the same security group—make the security group the source for the inbound rule and the destination for the outbound rule.
For instances in different security groups—make sure that both security groups allow access to one another.
If this is still not working check the error message that is returned when setting up the connection with RDS. Timeout means issue with network.
find out more here https://aws.amazon.com/premiumsupport/knowledge-center/connect-lambda-to-an-rds-instance/
I currently have a Lambda function that performs a series of task, but one part in particular requires connection to an RDS instance in another AWS account. The function spins up a new EC2 instance at each trigger and uses userdata to perform the series of tasks I have written.
The RDS instance has a security group, and I would need to whitelist the EC2 instance's IP to connect. This is infeasible as I am spinning up a new EC2 instance each time the function is triggered. Is there a way to whitelist all EC2 instance's in a particular AWS account in another AWS account's security group rules?
I am assuming you dont want your traffic to go over the internet.
Updating your security groups to reference peer VPC groups
You can update the inbound or outbound rules for your VPC security groups to reference security groups in the peered VPC. Doing so allows traffic to flow to and from instances that are associated with the referenced security group in the peered VPC.
The peer VPC can be a VPC in your account, or a VPC in another AWS account. To reference a security group in another AWS account, include the account number in Source or Destination field; for example, 123456789012/sg-1a2b3c4d.
This is the most elegant way instead of whitelisting all ec2 instances. Traffic always stays on the global AWS backbone, and never traverses the public internet, which reduces threats, such as common exploits, and DDoS attacks.
You still hold the controls what can be accessed between the accounts via many methods
Security groups
Route Tables
Network ACLs.
My AWS Lambda function can't access the hive server running on the master node. It times out -- the same behavior as if you try to access the node from a non white-listed IP. Obviously adding the Lambda function as a whitelisted IP is a non starter.
How do I configure AWS Lambda so that it can access services on the EMR master node?
Make your lambda run in private subnets of the same VPC as that of your EMR cluster.
In the EMR security group, add an inbound rule with the port that needs to be accessed with the lambda security group as source security group id. This will allow all lambda requests.
See if this helps !!