I'm using serverless for AWS deployments and have an issue where we have named one of our Elastic Search resources "prod-logs" on production and "nonprod-logs" on UAT. I have the variable ${opt:stage} available in my serverless.yml to tell if something is on prod or UAT.
How can I use ${opt:stage} within my iamRoleStatements to be able to output arn:aws:es:${aws:region}:${aws:accountId}:domain/nonprod-logs/* or arn:aws:es:${aws:region}:${aws:accountId}:domain/prod-logs/* based on the environment? Or more specifically, how do I omit non when ${opt:stage} is prod?
my yaml looks like this:
provider:
iamRoleStatements:
- Effect: Allow
Resource: arn:aws:es:${aws:region}:${aws:accountId}:domain/nonprod-logs/*
Action:
- es:ESHttpDelete
I've seen that you can use a ternary operator, but i'm not sure how you might use it in the middle of a string like:
Resource: arn:aws:es:${aws:region}:${aws:accountId}:domain/${!e ${opt:stage} = 'prod'? '': 'non'}prod-logs/*
Related
I'm using aws copilot but i think i can generalize this question to apprunner. trying to get envvars set from Parameter Store but having no luck. the left is my aws copilot manifest yml, i saw examples of setting things this way. it results in an apprunner configed on the right. in production it seems these are interpreted as literals and not as parameter store values
any idea on how to properly connect apprunner to parameter store?
Unfortunately currently App Runner doesn't provide an intrinsic way for integrating with SSM parameter store like what ECS does. As a result, Copilot doesn't support secret section for Request-driven service as well (refer to Copilot doc here). As for environment variables, they are what you define in the manifest and will be injected as literals.
However, there is a workaround in Copilot allowing your app to use secrets stored in SSM parameter store. You can specify an addon template (e.g., policy.yaml) and put it in the copilot/${svc name}/addons/ local directory with the following template allowing the App Runner service to be able to retrieve from SSM parameter store:
Parameters:
App:
Type: String
Description: Your application's name.
Env:
Type: String
Description: The environment name your service, job, or workflow is being deployed to.
Name:
Type: String
Description: The name of the service, job, or workflow being deployed.
Resources:
MySSMPolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
PolicyDocument:
Version: 2012-10-17
Statement:
- Sid: SSMActions
Effect: Allow
Action:
- "ssm:GetParameters"
Resource: !Sub 'arn:${AWS::Partition}:ssm:${AWS::Region}:${AWS::AccountId}:parameter/*'
Outputs:
MySSMPolicyArn:
Value: !Ref MySSMPolicy
After that, in your code by using AWS SDK you can call AWS SSM API to retrieve any secrets you defined before. Let me know if you have any more questions!
I am using serverless framework to deploy api gateway. I don't want to apply VPC to every stage in serverless. Is there a way for me to add below configuration based on stage name?
provider:
name: aws
runtime: nodejs12.x
...
endpointType: PRIVATE
vpcEndpointIds:
- 'Fn::ImportValue': 'api-gateway-endpoint'
resourcePolicy:
- Effect: Deny
...
There are a few ways to do conditional deployments in serverless.yml, some are more brittle than others and there are pros/cons to each but here is a list of methods I have collected:
Variables to control conditionals
NOTE: We use a custom regex variable syntax to separate Serverless variables from cloudformation variable syntax. All of the following snippets use this modified syntax:
provider:
name: aws
# Changes serverless variable to ${{}} double curly braces
variableSyntax: "\\${{([ ~:a-zA-Z0-9._\\'\",\\-\\/\\(\\)]+?)}}"
Serverless Custom Variables
custom:
scheduleEnabled:
dev: true
stage: true
prod: true
# Then In lambda function declaration
events:
- schedule:
name: MyScheduleName
description: SomeDescription
rate: cron(0/5 * * * ? *)
# Use custom variable and the serverless `stage` supplied
# via your deployment command to choose whether this feature is enabled
enabled: ${self:custom.scheduleEnabled.${self:custom.stage}}
input: {"_keepwarm": true}
Using Conditional Operations
Setting up CloudFormation Conditionals
resources:
- Conditions:
# True if they are equal ==
MyCondition: [!Equals ["${{env:SOMETHING}}","SOME_STRING"]]
# True if they are not equal !=
MyConditionTwo: !Not [!Equals ["${{env:SOMETHING_ELSE}}","SOME_OTHER_STRING"]]
# Using a custom serverless variable
IsProd: [!Equals ["${{self:provider.stage}}","prod"]]
Using Cloudformation Conditionals
If your conditional has two options, you can write it as:
# If true choose X, if False choose Y
Source:
Type: !If
- MyCondition # Conditional Name
- GITHUB # If condition is true
- GITHUB_ENTERPRISE # if condition is false
If your conditional is something you want to toggle ON or OFF, you can write it as:
# If True do nothing, If False choose X
MyCodebuildSetup:
Type: "AWS::CodeBuild::Project"
VpcConfig: !If
- MyCondition # Condition Name
- !Ref AWS::NoValue # If True, AWS will not attach a VPC Config
- VpcId: <My_VPC_ID> # If False, Use this VPC Config
Subnets:
- <my_VPC_Subnet>
SecurityGroupIds:
- <my_VPC_security_group
Weird Conditional Hacks (document usage well if you decide to implement them)
Conditionally deploying a large series of resources
Using Serverless to selectively deploy entire files of resources using strings set via environment variables.
resources:
- ${{file(some_dir/conditional_file_${{env:MY_CONDITION}}.yml)}}
This is also a way to toggle deployment of many resources via conditionals.
conditional_file_A.yaml can contain all the resources you toggle deployment of, while conditional_file_B.yaml can contain an empty Resources list (If you don't want a "file not found" warning from serverless:
Resources:
Using Serverless Variables in buildspec.yml files or .json files that will be sent to cloudformation (parameter store files, etc)
If you want to conditionally modify a .json or .yml file that will be sent deployed as a value for some CloudFormation key (such as deploying a .json file for parameter store, or submitting buildspec.yml files for CodePipeline), you can't actually use ${} serverless syntax in these external files. This is because serverless gets confused with it's own method of calling Key/Values from external files.
In order to use Serverless Variables in these files, you can make them .txt extensions as long as the actual file is formatted as proper .json or .yml
Source:
Type: CODEPIPELINE
# This file can't have any serverless variables in it
BuildSpec: ${{file(my_buildspec_file.yml)}}
Source:
Type: CODEPIPELINE
# This file CAN have serverless variables because it is
# interpreted by serverless as a .txt, the variables
# are resolved and then it is sent to cloudformation as a string anyway
BuildSpec: ${{file(my_buildspec_file_yaml_formatted.txt)}}
You may check the following answers: Conditional serverless.yml based on stage?
Therefore, you get something like:
resources:
Conditions:
IsProd:
Fn::Equals:
- ${opt:stage}
- prod
Resources:
SomeIAMRole:
Type: AWS::IAM::Role
Condition: IsProd
Properties:
etc
etc
Alternatively, there is a Serverless Plugin Ifelse
Add the plugin:
plugins:
- serverless-plugin-ifelse
and then add your conditions:
custom:
.....
....
serverlessIfElse:
- If: '"${opt:stage}" == "production"'
Set:
functions.helloWorld.cors: true
see more info Serveless plugin if-else
I am using serverless framework for deploying my application on AWS Cloud.
https://serverless.com/
I want to use the value of AWS Account ID in my serverless.yml file and I want to export the acccount ID as the environment variable, so that it can be accessed from Lambda functions.
Based on the value of this lambda function, I want to create some resources ( like IAM roles, etc.), which refer to this accountId variable.
But when I try to deploy the stack, I get the below error,
Trying to populate non string value into a string for variable
${self:custom.accountId}. Please make sure the value of the property
is a string.
My Serverless.yml file is as below
custom:
accountId : !Ref "AWS::AccountId"
provider:
name: aws
runtime: go1.x
stage: dev
region: us-east-1
environment:
ACCOUNT_ID : ${self:custom.accountId}
myRoleArn: arn:aws:iam::${self:custom.accountId}:role/xxxxxxxx
Is there any way to refer to the value of the Account Id in the serverless.yml file?
You can't reference AWS::AccountId in your serverless.yml, because it doesn't quite translate when creating the CloudFormation template.
The workaround is to use the serverless plugin Pseudo Parameters.
You can install the plugin using npm.
npm install serverless-pseudo-parameters
You will also need to add the plugin to the serverless.yml file.
plugins:
- serverless-pseudo-parameters
Then you can reference your AccountId with #{AWS::AccountId}
functions:
helloWorld:
handler: index.handler
events:
- http:
path: /
method: get
environment:
ACCOUNT_ID : #{AWS::AccountId}
Note that the reference begins with a hash instead of a dollar sign.
I have written a service using AWS Step Functions. I would like to integrate this into our applications existing Elastic Beanstalk development process, wherein we have distinct dev, staging, and production applications. Each of these stages have app-specific environment variables, which I would like to pull into my Lambda functions as well.
I am not presently using SAM but I can port over if necessary to accomplish this.
The following is a simplified configuration mirroring my serverless.yml file.
service:
name: small-service
plugins:
- serverless-webpack
- serverless-step-functions
- serverless-pseudo-parameters
provider:
name: aws
runtime: nodejs8.10
stage: dev
region: us-east-2
iamRoleStatements:
- Effect: "Allow"
Action:
- "s3:*"
Resource: { "Fn::Join": ["", ["arn:aws:s3:::S3-bucket-name", "/*" ] ] }
functions:
connect:
handler: handler.connect
stepFunctions:
stateMachines:
smallService:
name: small-service-${self:provider.stage}
definition:
Comment: Service that connects to things
StartAt: Connect
States:
Connect:
Type: Task
Resource: arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:${self:service}-${self:provider.stage}-connect
End: true
How can I dynamically deploy the step functions into different beanstalk applications? How can I access ElasticBeanstalk environment properties from within Step Functions?
Is there a better way to import environment .env variables dynamically into a Serverless application outside of EB? We are integrating the service into a larger AWS applications development workflow, is there is a more "serverless" way of doing this?
Move your environment variables into SSM Parameter Store. Then you can either
reference SSM parameters in your serverless.yaml, or
fetch SSM parameters in the beginning of each Lambda invocation (see e.g. here)
Note that the former method requires re-deploying your Lambda to receive the latest SSM parameters whereas the latter always fetches the latest parameter values.
Does anyone know of a definitive guide for converting cloudformation to yaml in Serverless v1.0?
I can make quite a few things work but I'm completely stuck on how to setup Firehose and allow a lambda to write to it.
I think the resources section would look something like this:
resources:
Resources:
FirehoseBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: my-firehose-bucket
FirehoseDeliveryStream:
Type: AWS::KinesisFirehose::DeliveryStream
Properties:
DeliveryStreamName: "FirehoseDeliveryStream"
S3DestinationConfiguration:
BucketARN:
Fn::Join:
- ''
- - 'arn:aws:s3:::'
- Ref: FirehoseBucket
BufferingHints:
IntervalInSeconds: 60
SizeInMBs: 5
CompressionFormat: GZIP
Prefix: ${prefix}
RoleARN: "arn:aws:iam::${account-number}:role/${project}-${env}-IamRoleLambda"
But I have no idea how to convert the Cloudformation for the IAM section, which is described here: http://docs.aws.amazon.com/firehose/latest/dev/controlling-access.html.
Any pointers, examples or guides on CF->Yaml conversion much appreciated!
The IAM statements for the Lambda Execution Role (ie the role that the Lambda function assumes when executing) is not specified in the resources.
So if you need permissions to do something from inside your Lambda function you need to give the assumed role permission.
This is specified in the provider section. So in your case (I just copied something from your link, you will have to change it to what you need) it will be something like this (assuming nodejs runtime):
provider:
name: aws
runtime: nodejs4.3
iamRoleStatements:
- Effect: Allow
Action:
- firehose:DeleteDeliveryStream
- firehose:PutRecord
- firehose:PutRecordBatch
- firehose:UpdateDestination
Resource:
- arn:aws:firehose:region:account-id:deliverystream/delivery-stream-name
- Effect: Allow
Action:
- logs:PutLogEvents
Resource:
- arn:aws:logs:region:account-id:log-group:log-group-name:log-stream:log-stream-name
Bonus: For general resources specified in cloud formation json format use an online converter from json to yaml. Much easier to get the started that way.