I am using cloudformation to provision lambda and RDS on AWS. But I don't know how to add database proxy on lambda. Below screenshot is from lambda console:
Does cloudformation support adding this? I can't see it in lambda and db proxy template.
The exact configuration I use in CloudFormation template is:
MyLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- rds-db:connect
Resource:
- <rds_proxy_arn>/*
where <rds_proxy_arn> is the ARN of the proxy but service is rds-db instead of rds and resource type is dbuser instead of db-proxy. For example, if your proxy's ARN is arn:aws:rds:us-east-1:123456789012:db-proxy:prx-0123456789abcdef01 the whole line should be arn:aws:rds-db:us-east-1:123456789012:db-proxy:prx-0123456789abcdef01/*.
After deployed, we can see a new link is added in Database Proxies of the Console.
As per the CloudFormation/Lambda documentation there is no option to specify the DB Proxy for a Lambda.
I don't see an option to add an RDS proxy while creating a Lambda function in the low level HTTP API also. Not sure why.
As per the following Github issue, it seems this is not required to connect lambda to RDS proxy. https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/750
You merely need to provide the new connection details to lambda (e.g. using env variables to make it work)
After talking with AWS support, the screenshot in AWS console to add proxy on lambad is only to grant below IAM permission to lambda. That means it is an optional.
Allow: rds-db:connect
Allow: rds-db:*
Related
I have a SAM template with the resources to create a lambda function and an api gateway. The template is saved along with the code for the lambda function and the buildspec.yaml file. When I run the code through codepipeline without the api gateway resources the SAM template is transformed then deployed successfully. When I include the resources necessary to create the api gateway I am presented with the following error upon creation:
AccessDenied. User doesn't have permission to call apigateway:GetResources
when I look at the policy attached to the cloudformation role I have the following:
Actions:
- apigateway:DELETE
- apigateway:GetResources
- apigateway:GetRestApis
- apigateway:POST
Effect: Allow
Resource: !Sub "arn:${AWS::Partition}:apigateway:*::/*"
The action has apigateway:GetResources defined yet it still fails. When I permit all api gateway actions the template is successfully deployed by codepipeline and cloudformation. That is if I have the following statement:
Actions:
- apigateway:*
Effect: Allow
Resource: !Sub "arn:${AWS::Partition}:apigateway:*::/*"
Question: Is it possible to have codepipeline with cloudformation create an api gaetway without providing the catchall(*) api gateway actions?
There are no such actions in API gateway IAM policies like:
- apigateway:GetResources
- apigateway:GetRestApis
The API gateway permissions have the form of:
apigateway:HTTP_VERB
So you probably need GET:
Actions:
- apigateway:DELETE
- apigateway:GET
- apigateway:POST
I am using serverless framework to provision infrastructure on AWS. And I need to add database proxy on my lambda but I couldn't find how to configure that. I have read the doc https://www.serverless.com/framework/docs/providers/aws/guide/functions/ but it doesn't mention anything relate to database proxy.
Below screenshot is the bottom of lambda in aws console. How can I add the proxy via serverless.yml?
Go to your RDS server and click on the proxy
Copy the Proxy ARN
Edit your serverless.yml and
provider:
name: aws
iam:
role:
statements:
- Effect: "Allow"
Action:
- "rds-db:connect"
Resource: "arn:aws:rds-db:us-east-1:123123123:admin:blah-123abc123abc/*"
Note that "rds" in the ARN was changed to "rds-db" and "db-proxy" in the ARN was changed to "admin" (the admin user of the database). sls deploy and check the lambda. You should see the proxy in the database proxy configuration section.
For example, the Proxy ARN I copied from RDS was
arn:aws:rds:us-east-1:123123123:db-proxy:blah-123abc123abc
and I edited it to be
arn:aws:rds-db:us-east-1:123123123:admin:blah-123abc123abc/*
Also, be sure your lambda is in the same Vpc as the RDS proxy or it will not be able to connect.
I am using:
% sls --version
Framework Core: 2.50.0
Plugin: 5.4.3
SDK: 4.2.3
Components: 3.13.2
I am trying to deploy an AWS Lambda function that gets triggered when an AVRO file is written to an existing S3 bucket.
My serverless.yml configuration is as follows:
service: braze-lambdas
provider:
name: aws
runtime: python3.7
region: us-west-1
role: arn:aws:iam::<account_id>:role/<role_name>
stage: dev
deploymentBucket:
name: serverless-framework-dev-us-west-1
serverSideEncryption: AES256
functions:
hello:
handler: handler.hello
events:
- s3:
bucket: <company>-dev-ec2-us-west-2
existing: true
events: s3:ObjectCreated:*
rules:
- prefix: gaurav/lambdas/123/
- suffix: .avro
When I run serverless deploy, I get the following error:
ServerlessError: An error occurred: IamRoleCustomResourcesLambdaExecution - API: iam:CreateRole User: arn:aws:sts::<account_id>:assumed-role/serverless-framework-dev/jenkins_braze_lambdas_deploy is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::<account_id>:role/braze-lambdas-dev-IamRoleCustomResourcesLambdaExec-1M5QQI6P2ZYUH.
I see some mentions of Serverless needing iam:CreateRole because of how CloudFormation works but can anyone confirm if that is the only solution if I want to use existing: true? Is there another way around it except using the old Serverless plugin that was used prior to the framework adding support for the existing: true configuration?
Also, what is 1M5QQI6P2ZYUH in arn:aws:iam::<account_id>:role/braze-lambdas-dev-IamRoleCustomResourcesLambdaExec-1M5QQI6P2ZYUH? Is it a random identifier? Does this mean that Serverless will try to create a new IAM role every time I try to deploy the Lambda function?
I've just encountered this, and overcome it.
I also have a lambda for which I want to attach an s3 event to an already existing bucket.
My place of work has recently tightened up AWS Account Security by the use of Permission Boundaries.
So i've encountered the very similar error during deployment
Serverless Error ---------------------------------------
An error occurred: IamRoleCustomResourcesLambdaExecution - API: iam:CreateRole User: arn:aws:sts::XXXXXXXXXXXX:assumed-role/xx-crossaccount-xx/aws-sdk-js-1600789080576 is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::XXXXXXXXXXXX:role/my-existing-bucket-IamRoleCustomResourcesLambdaExec-LS075CH394GN.
If you read Using existing buckets on the serverless site, it says
NOTE: Using the existing config will add an additional Lambda function and IAM Role to your stack. The Lambda function backs-up the Custom S3 Resource which is used to support existing S3 buckets.
In my case I needed to further customise this extra role that serverless creates so that it is also assigned the permission boundary my employer has defined should exist on all roles. This happens in the resources: section.
If your employer is using permission boundaries you'll obviously need to know the correct ARN to use
resources:
Resources:
IamRoleCustomResourcesLambdaExecution:
Type: AWS::IAM::Role
Properties:
PermissionsBoundary: arn:aws:iam::XXXXXXXXXXXX:policy/xxxxxxxxxxxx-global-boundary
Some info on the serverless Resources config
Have a look at your own serverless.yaml, you may already have a permission boundary defined in the provider section. If so you'll find it under rolePermissionsBoundary, this was added in I think version 1.64 of serverless
provider:
rolePermissionsBoundary: arn:aws:iam::XXXXXXXXXXXX:policy/xxxxxxxxxxxx-global-boundary
If so, you can should be able to use that ARN in the resources: sample I've posted here.
For testing purpose we can use:
provider:
name: aws
runtime: python3.8
region: us-east-1
iamRoleStatements:
- Effect: Allow
Action: "*"
Resource: "*"
For running sls deploy, I would suggest you use a role/user/policy with Administrator privileges.
If you're restricted due to your InfoSec team or the like, then I suggest you have your InfoSec team have a look at docs for "AWS IAM Permission Requirements for Serverless Framework Deploy." Here's a good link discussing it: https://github.com/serverless/serverless/issues/1439. At the very least, they should add iam:CreateRole and that can get you unblocked for today.
Now I will address your individual questions:
can anyone confirm if that is the only solution if I want to use existing: true
Apples and oranges. Your S3 configuration has nothing to do with your error message. iam:CreateRole must be added to the policy of whatever/whoever is doing sls deploy.
Also, what is 1M5QQI6P2ZYUH in arn:aws:iam::<account_id>:role/braze-lambdas-dev-IamRoleCustomResourcesLambdaExec-1M5QQI6P2ZYUH? Is it a random identifier? Does this mean that serverless will try to create a new role every time I try to deploy the function?
Yes, it is a random identifier
No, sls will not create a new role every time. This unique ID is cached and re-used for updates to an existing stack.
If a stack is destroyed/recreated, it will assign a generate a new unique ID.
I would like to perform the following operations in order with CloudFormation.
Start up an EC2 instance.
Give it privileges to access the full internet using security group A.
Download particular versions of Java and Python
Remove its internet privileges by removing security group A and adding a security group B.
I observe that there is a DependsOn attribute for specifying the order in which to create resources, but I was unable to find a feature that would allow me to update the security groups on the same EC2 instance twice over the course of creating a stack.
Is this possible with CloudFormation?
Not in CloudFormation natively, but you could launch the EC2 instance with a configured userdata script that itself downloads Java/Python and the awscli, as necessary, and then uses the awscli to switch security groups for the current EC2 instance.
However, if all you need is Java and Python pre-loaded then why not simply create an AMI with them already installed and launch from that AMI?
The best way out is to utilise a Cloudformation custom resource here. You can create a lambda function that does exactly what you need. This lambda function can then be called as a custom resource function in the cloud formation template.
You can pass your new security group ID and instance ID to the lambda function and code the lambda function to use AWS SDK and do the modifications that you need.
I have leveraged it to post an update to my web server about the progress of the cloud formation template. Below is the sample code of the template.
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
MarkInstanceProfileComplete:
Type: 'Custom::EC2InstanceProfileDone'
Version: '1.0'
DependsOn: EC2InstanceProfile
Properties:
ServiceToken: !Ref CustomResourceArn
HostURL: !Ref Host
LoginType: !Ref LoginType
SecretId: !Ref SecretId
WorkspaceId: !Ref WorkspaceId
Event: 2
Total: 3
Here the resource MarkInstanceProfileComplete is a custom resource that calls a Lambda function. It takes the event count and total count as input and processes them to calculate percentage progress. Based on that it sends out a request to my web server. For all we care, this Lambda function can do potentially anything you want it to do.
I want to grant vpc access for my lambda function. I use the following aws cli command.
aws lambda update-function-configuration \
--function-name SampleFunction \
--vpc-config SubnetIds=subnet-xxxx,SecurityGroupIds=sg-xxxx
But I receive the following error:
An error occurred (AccessDeniedException) when calling the
UpdateFunctionConfiguration operation: Your access has been denied by
EC2, please make sure your request credentials have permission to
DescribeSecurityGroups for sg-xxxx. EC2 Error Code:
UnauthorizedOperation. EC2 Error Message: You are not authorized to
perform this operation.
I have granted the following permission to both my lambda role and the user who execute the aws command.
- "ec2:CreateNetworkInterface"
- "ec2:DescribeNetworkInterfaces"
- "ec2:DeleteNetworkInterface"
- "ec2:DescribeSecurityGroups"
I further tried to grant full access to both the lambda role and the user. But still received the same error
Can anyone suggest what else I can try?
The trick is to add the pipeline / worker role / user which is deploying the lambda function) have access to network related policies. The lambda function should itself suffice with managed policy - AWSLambdaVPCAccessExecutionRole
arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
Action:
ec2:DescribeSecurityGroups
ec2:DescribeSubnets
ec2:DescribeVpcs
Effect: Allow
Resource: '*'
Your users IAM policy needs further permissions.
For example ec2:CreateSecurityGroup & etc. Have a look at this documentation to add requred permissions.
I experienced the same issue. Despite the IAM policy for the user having the required permissions, I could not use the aws cli to crate a lambda function with a VPC config (aws lambda create-function) or modify an existing function to add a VPC config (aws lambda update-function-configuration).
The only way I could get this to work was to create the lambda function without a VPC config. I then modified the function to add the VPC config information (vpc, subnet and security groups) via the AWS console (in Lambda > Fucntions > My Function > Network). I was only able to use the console to do this, introducing a manual step in an otherwise fully automated process.
To answer some of the questions above about which user needs the ec2:DescribeSecurityGroups and related permissions. It is the user running the cli command or logged in to the console. The function does not need a policy providing these permissions. The only special permissions needed for a function with a VPC config are:
ec2:CreateNetworkInterface
ec2:DescribeNetworkInterfaces
ec2:DeleteNetworkInterface
These allow the function to create ENIs within your VPC using the subnet and security group you provide as described here.
Both the Lambda funtion's role and the user role (either cloudformation or cmline user) must have:
- ec2:CreateNetworkInterface
- ec2:DescribeNetworkInterfaces
- ec2:DeleteNetworkInterface
- ec2:DescribeSecurityGroups
- ec2:DescribeSubnets
or ec2:* if ok for your use case'security
I had the same issue deploying a lambda with a VPC config using SAM/cloudformation and resolved it by adding this above.
on github issue some people say it is because of cloudformation order creation it is not (or maybe not anymore because I tested adding 20 dummy resource and still the same issue only resolved by adding the permissions above)
cheers,