I have a lambda function using the serverless framework. I created a security group and associated it with the lambda function, I also included all 3 subnetIDs that the RDS uses in the serverless.yml. I then associated the new lambda security group with the security group that the RDS is saying it's using. I have it set for MySQL\Aurora on the inbound tab and the source being the new lambda sg.
When I run the lambda I get the handshake inactivity timeout when trying to connect my pool to the instance. I use this same security group to give myself access to the database via IP, so I know it has to be the right instance. What am I missing?
Here is the definition for the function in serverless.yml
consumer:
handler: consumer/handler.testconsumer
timeout: 29
memorySize: 512
vpc:
securityGroupIds:
- sg-lambda
subnetIds:
- subnet-1
- subnet-2
- subnet-3
I also have the following IAM role statement to allow the networking. Though serverless does bark at this:
Warned - iamRoleStatement granting Resource='*'. Wildcard resources in iamRoleStatements are not permitted.
iamRoleStatements:
- Effect: "Allow"
Action:
- "ec2:CreateNetworkInterface"
- "ec2:DescribeNetworkInterfaces"
- "ec2:DeleteNetworkInterface"
Resource: "*"
Is that causing an issue? I'm beating my head against a wall about this.
Related
I am using serverless.yml to deploy lambdas to AWS and I'd like to know how to configure the resource-based policy for my lambda.
I deploy a customised alias to my lambda and need to grant invoke:lambda in the policy of the resouce-based policy. So when you open lambda -> configuration -> permission, the policy should appear as below
when I use the role configure in serverless.yml, it only changes the permission for my lambda execution role. How can I modify the Resource-based policy for my lambda?
I have used before the API Gateway Resource Policy:
https://www.serverless.com/framework/docs/providers/aws/events/apigateway/#resource-policy
For the lambda function association directly you can take a look at that thread:
https://github.com/serverless/serverless/issues/4926
An example serverless.yaml would look like this:
provider:
name: aws
runtime: nodejs8.10
memorySize: 128
stage: dev
apiGateway:
resourcePolicy:
- Effect: Allow
Principal: '*'
Action: execute-api:Invoke
Resource:
- execute-api:/*/*/*
Condition:
IpAddress:
aws:SourceIp:
- 'your ip here'
How to restrict access to a lambda
Please note that the resource policy currently only works for the REST API Gateways. https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-vs-rest.html
HTTP APIs do not support resource policies.
I am using cloudformation to provision lambda and RDS on AWS. But I don't know how to add database proxy on lambda. Below screenshot is from lambda console:
Does cloudformation support adding this? I can't see it in lambda and db proxy template.
The exact configuration I use in CloudFormation template is:
MyLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- rds-db:connect
Resource:
- <rds_proxy_arn>/*
where <rds_proxy_arn> is the ARN of the proxy but service is rds-db instead of rds and resource type is dbuser instead of db-proxy. For example, if your proxy's ARN is arn:aws:rds:us-east-1:123456789012:db-proxy:prx-0123456789abcdef01 the whole line should be arn:aws:rds-db:us-east-1:123456789012:db-proxy:prx-0123456789abcdef01/*.
After deployed, we can see a new link is added in Database Proxies of the Console.
As per the CloudFormation/Lambda documentation there is no option to specify the DB Proxy for a Lambda.
I don't see an option to add an RDS proxy while creating a Lambda function in the low level HTTP API also. Not sure why.
As per the following Github issue, it seems this is not required to connect lambda to RDS proxy. https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/750
You merely need to provide the new connection details to lambda (e.g. using env variables to make it work)
After talking with AWS support, the screenshot in AWS console to add proxy on lambad is only to grant below IAM permission to lambda. That means it is an optional.
Allow: rds-db:connect
Allow: rds-db:*
I'm creating an RDS cluster and, in the same template, a policy with some permissions to allow particular actions over that cluster.
Statement:
- Effect: Allow
Actions:
- 'rds-data:BatchExecuteStatement'
- 'rds-data:ExecuteStatement'
Resource: 'arn:aws:rds:us-east-1:1111111111:cluster:production-mycluster-rdscluster-no1yzvzs29sq'
The problem is that AWS::RDS::DBCluster does not support Fn::GetAtt ARN, and, since RDS add that random string at the end of the ARN, in this example no1yzvzs29sq I don't know how can I use something like a wildcard to "whitelist" part of the name.
I would like something like
arn:aws:rds:us-east-1:1111111111:cluster:production-mycluster-rdscluster-*
But it doesn't work. I will appreciate help!
You can construct the ARN based on the Cluster name yourself.
For example:
Statement:
- Effect: Allow
Actions:
- 'rds-data:BatchExecuteStatement'
- 'rds-data:ExecuteStatement'
Resource: !Sub 'arn:${AWS::Partition}:rds:${AWS::Region}:${AWS::AccountId}:cluster:${MyDBCluster}'
Based on the docs:
Amazon RDS Data API does not support specifying a resource ARN in the Resource element of an IAM policy statement. To allow access to Amazon RDS Data API, specify “Resource”: “*” in your policy.
So at this point it looks like you're stuck with a policy that looks like
Statement:
- Effect: Allow
Actions:
- 'rds-data:BatchExecuteStatement'
- 'rds-data:ExecuteStatement'
Resource: '*'
I would like to perform the following operations in order with CloudFormation.
Start up an EC2 instance.
Give it privileges to access the full internet using security group A.
Download particular versions of Java and Python
Remove its internet privileges by removing security group A and adding a security group B.
I observe that there is a DependsOn attribute for specifying the order in which to create resources, but I was unable to find a feature that would allow me to update the security groups on the same EC2 instance twice over the course of creating a stack.
Is this possible with CloudFormation?
Not in CloudFormation natively, but you could launch the EC2 instance with a configured userdata script that itself downloads Java/Python and the awscli, as necessary, and then uses the awscli to switch security groups for the current EC2 instance.
However, if all you need is Java and Python pre-loaded then why not simply create an AMI with them already installed and launch from that AMI?
The best way out is to utilise a Cloudformation custom resource here. You can create a lambda function that does exactly what you need. This lambda function can then be called as a custom resource function in the cloud formation template.
You can pass your new security group ID and instance ID to the lambda function and code the lambda function to use AWS SDK and do the modifications that you need.
I have leveraged it to post an update to my web server about the progress of the cloud formation template. Below is the sample code of the template.
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
MarkInstanceProfileComplete:
Type: 'Custom::EC2InstanceProfileDone'
Version: '1.0'
DependsOn: EC2InstanceProfile
Properties:
ServiceToken: !Ref CustomResourceArn
HostURL: !Ref Host
LoginType: !Ref LoginType
SecretId: !Ref SecretId
WorkspaceId: !Ref WorkspaceId
Event: 2
Total: 3
Here the resource MarkInstanceProfileComplete is a custom resource that calls a Lambda function. It takes the event count and total count as input and processes them to calculate percentage progress. Based on that it sends out a request to my web server. For all we care, this Lambda function can do potentially anything you want it to do.
I want to connect my RDS Database table with my lambda function, for this, I have created a lambda function and used knex.js and postgres database in rds, I got the knex object, but I cannot work with any query.
To give some more information about the services,
RDS database server security group can be access from anywhere
I have given the vpc in the serverless.yml file in the function.
Region of both lambda and rds are different, but not sure whether it is the problem.
My serverless function
note: this knex code is working when I tried this separately.
module.exports.storeTransaction = async (event) => {
...
knex('Transactions')
.select('*')
.then(response => {
console.log('response is ');
console.log(response);
})
...
};
Serverless.yml file
service: <service-name>
provider:
name: aws
runtime: nodejs8.10
stage: dev
region: us-east-1
package:
exclude:
- node_modules/**
plugins:
- serverless-plugin-include-dependencies
functions:
storeEmail:
handler: handler.storeTransaction
vpc:
securityGroupIds:
- <security-group-id-of-rds>
subnetIds:
- <subnet-id-of-rds>
- <subnet-id-of-rds>
...
region:
- us-east-1a
events:
- http:
path: email/store
method: post
cors: true
So can you identify my issue on why I can't connect my rds db with lambda function, and let me know what I did wrong or what is missing.
I think the problem is that RDS and Lambda are in different regions, which means they are also in different VPCs, as a VPC cannot span across multiple regions. Although you can enable Inter VPC Peering (https://aws.amazon.com/vpc/faqs/#Peering_Connections).
Consider that when you deploy a lambda function in a VPC, it won't have internet access as long as you don't attach a NAT Gateway to that VPC/subnet.
If the RDS is open to the world (and does it really need to be??), you can try to deploy in the same region (without a VPC) and verify if that works.