I have following CF Template
{
"Conditions":{
"CreatedProdStage" : {...}
}
...
"Resources":{
"GetMethod": {
...
},
"ApiDeployement":{
...
},
"ProdStage":{
"Type":"AWS::ApiGateway::Stage",
"Condition":"CreatedProdStage",
"Properties": {
"DeploymentId":"...",
"RestApiId":"...",
"MethodSettings":[{
"CachingEnabled":true,
"HttpMethod":{"Ref":"GetMethod"},
"ResourcePath":"/"
}]
}
}
}
}
And I am getting error
Invalid method setting path:
/~1/st-GetMetho-xxxAUMMRWxxx/caching/enabled. Must be one of:
[/deploymentId, /description,
/cacheClusterEnabled/cacheClusterSize/clientCertificateId/{resourcePath}/{httpMethod}/metrics/enabled,
/{resourcePath}/{httpMethod}/logging/dataTrace,
/{resourcePath}/{httpMethod}/logging/loglevel,
/{resourcePath}/{httpMethod}/throttling/burstLimit/{resourcePath}/{httpMethod}/throttling/rateLimit/{resourcePath}/{httpMethod}/caching/ttlInSeconds,
/{resourcePath}/{httpMethod}/caching/enabled,
/{resourcePath}/{httpMethod}/caching/dataEncrypted,
/{resourcePath}/{httpMethod}/caching/requireAuthorizationForCacheControl,
/{resourcePath}/{httpMethod}/caching/unauthorizedCacheControlHeaderStrategy,
///metrics/enabled, ///logging/dataTrace, ///logging/loglevel,
///throttling/burstLimit ///throttling/rateLimit
///caching/ttlInSeconds, ///caching/enabled,
///caching/dataEncrypted,
///caching/requireAuthorizationForCacheControl,
///caching/unauthorizedCacheControlHeaderStrategy, /va
Am I missing something? I thought ResourcePath and HttpMethod are the only required attributes
You first need to enable caching on the stage with the CacheClusterEnabled property. This will allow you to set up caching for methods as you have done in you MethodSettings:
...
"ProdStage":{
"Type":"AWS::ApiGateway::Stage",
"Condition":"CreatedProdStage",
"Properties": {
"DeploymentId":"...",
"RestApiId":"...",
"CacheClusterEnabled": true
"MethodSettings":[{
"CachingEnabled":true,
"HttpMethod":{"Ref":"GetMethod"},
"ResourcePath":"/"
}]
}
}
Then you will need to fix the given error. Your ResourcePath to match one of those listed in the error output. Those are not listed in the documentation, so it's a bit confusing what you need to use. What you currently have is set up for the root path only. If you want all paths use "/*"
APIGateWay::MethodSettings (see ResourcePath) doc:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-apigateway-stage-methodsetting.html
If anyone is still arriving at this, but is NOT using cache, I have provided an example for setting throttling and logging on the whole API. I could not figure it out until I started playing around with the ResourcePath and HttpMethod, and noticed the error changing.
Please note that I used * for both path and method and USED QUOTATIONS. It will fail without quotations.
ProdStage:
Type: AWS::ApiGateway::Stage
Properties:
StageName: Prod
RestApiId: !Ref StunningDisco
DeploymentId: !Ref StunningDiscoDeployment
MethodSettings:
- ResourcePath: '/*'
HttpMethod: '*'
LoggingLevel: INFO
DataTraceEnabled: True
ThrottlingBurstLimit: '10'
ThrottlingRateLimit: '10.0'
StunningDiscoDomainMapping:
Type: 'AWS::ApiGateway::BasePathMapping'
DependsOn: ProdStage
Properties:
DomainName: !Ref StunningDiscoDomain
RestApiId: !Ref StunningDisco
Stage: !Ref ProdStage
StunningDiscoDeployment:
Type: AWS::ApiGateway::Deployment
DependsOn: [StunningDiscoRootEndpoint, LightsInvokeEndpoint]
Properties:
RestApiId: !Ref StunningDisco
Try setting the HttpMethod to a string instead of a reference:
"MethodSettings":[{
"CachingEnabled":true,
"HttpMethod": "GET",
"ResourcePath":"/"
}]
}
Related
We are trying create a AWS::WAFv2::IPSet in our SAM template.
WhitelistedIPAddressesIPSet:
Type: AWS::WAFv2::IPSet
Properties:
Description: 'Merchant IPs'
Scope: REGIONAL
IPAddressVersion: IPV4
Addresses: [0.0.0.0/32, 0.0.10.0/32]
The creation of the IP sets is done successfully.
Once creating the AWS::WAFv2::WebACLAssociation.
WAFApiAssociation:
Type: AWS::WAFv2::WebACLAssociation
DependsOn:
- ApiGateway
- WAFWebAcl
Properties:
ResourceArn: !Sub 'arn:aws:apigateway:${AWS::Region}::/restapis/${ApiGateway}/stages/${EnvType}'
WebACLArn: !GetAtt WAFWebAcl.Arn
The CloudFormation failes and does a rollback. Error displayed is as follows:
Resource handler returned
ion message: "AWS WAF couldn?t
perform the operation
because your resource
doesn?t exist. (Service:
Wafv2, Status Code: 400,
Request ID: e337720a-e32c-
4c29-acde-1896855405c9,
Extended Request ID:
null)" (RequestToken: f24d
0488-3016-4030-3a3b-bbb246
66f130, HandlerErrorCode:
NotFound)
We tried different formatting the SAM template of the IP set, to see if that causes the issues, without any success.
Anyone that could share some helpful insights to this issue?
A) You don't need DependsOn if your resource already directly depends on those other resources. In this case it does, so you can remove this property.
B) You'll need to share your whole stack here, not just what is shared because there is likely a problem with your APIGW configuration. Because that failed to be created, it's possible you get this subsequent problem showing up.
Creating the APIGW isn't enough, you need to make sure to actually attach the WAF after the APIGW stage was created and not just the APIGW. In this case replace the ResourceArn with one that references the APIGW Stage. (And further you might need to wait for the stage deployment to finish.)
This is the APIGW template Warren Parad
CDEAPI:
Type: AWS::Serverless::Api
Properties:
# Domain:
# DomainName: !Ref CDEAPIDomainName
# SecurityPolicy: TLS_1_2
# CertificateArn: !Sub 'arn:aws:acm:us-east-1:${AWS::AccountId}:certificate/${CDEAPICertificateArn}'
# EndpointConfiguration: EDGE
# Route53:
# HostedZoneId: !Ref CDEAPIHostedZoneId
AccessLogSetting:
DestinationArn: !GetAtt CDEAPIAccessLogGroup.Arn
Format: >-
{ "requestId":"$context.requestId",
"ip":"$context.identity.sourceIp",
"caller":"$context.identity.caller",
"user":"$context.identity.user",
"userAgent":"$context.identity.userAgent",
"userArn":"$context.identity.userArn",
"requestTime":"$context.requestTime",
"requestTimeEpoch":"$context.requestTimeEpoch",
"httpMethod":"$context.httpMethod",
"resourcePath":"$context.resourcePath",
"path":"$context.path",
"status":"$context.status",
"protocol":"$context.protocol",
"responseLength":"$context.responseLength",
"responseLatency":"$context.responseLatency",
"authorizerLatency":"$context.authorizer.integrationLatency",
"integrationLatency":"$context.integrationLatency",
"integrationStatus":"$context.integrationStatus",
"xrayTraceId":"$context.xrayTraceId",
"errorMessage":"$context.error.message",
"domainName":"$context.domainName",
"domainPrefix":"$context.domainPrefix",
"tokenScopes":"$context.authorizer.claims.scope",
"tokenIat":"$context.authorizer.claims.iat",
"tokenExp":"$context.authorizer.claims.exp",
"cognitoIdentityId":"$context.identity.cognitoIdentityId",
"awsEndpointRequestId":"$context.awsEndpointRequestId",
"arn":"$context.identity.userArn",
"account":"$context.identity.accountId",
"claims-sub":"$context.authorizer.claims.sub",
"waf-error":"$context.waf.error",
"waf-status":"$context.waf.status",
"waf-latency":"$context.waf.latency",
"waf-response":"$context.waf.wafResponseCode",
"authenticate-error":"$context.authenticate.error",
"authenticate-status":"$context.authenticate.status",
"authenticate-latency":"$context.authenticate.latency",
"integration-error":"$context.integration.error",
"integration-status":"$context.integration.status",
"integration-latency":"$context.integration.latency",
"integration-requestId":"$context.integration.requestId",
"integration-integrationStatus":"$context.integration.integrationStatus",
"response-latency":"$context.responseLatency" }
StageName: !Ref EnvType
Auth:
DefaultAuthorizer: CognitoAuthorizer
AddDefaultAuthorizerToCorsPreflight: false
Authorizers:
CognitoAuthorizer:
AuthType: COGNITO_USER_POOLS
UserPoolArn: !Sub 'arn:aws:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/${CognitoUserPoolArn}'
I'm trying to create a cloudformation template that has default values, and I'm running a few !Sub functions to replace imported parameters into the template.
However, I am passing a list to a nodejs Lambda function that I need to !Sub before sending it.
The code that I'm writing:
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: Creating Athena database and tables
Parameters:
S3DataLocations:
Type: CommaDelimitedList
Description: The S3 locations where the logs are read from (Specify 'Use,Default' to inherit defaults)
Default: Use,Default
Conditions:
CustomS3DataLocations: !Equals ['Use,Default', !Join [",", !Ref S3DataLocations]]
Resources:
# Custom resource for running CreateTableFunction, to create databases
CreateLogTable:
Type: Custom::CreateLogTable
Properties:
ServiceToken: !GetAtt [CreateLogTableFunction, Arn]
S3DataLocations:
Fn::If:
- CustomS3DataLocations
- !Split
- ","
- !Sub
- s3://${LoggingBucket}/data/ApplicationLogs1/,
s3://${LoggingBucket}/data/ApplicationLogs2/,
s3://${LoggingBucket}/data/ApplicationLogs3/
- { LoggingBucket: !ImportValue Parent-LoggingBucket}
- !Ref S3DataLocations
If I pass these as a literal external DataTypes parameter s3://logbucket/data/ApplicationLogs1/,s3://logbucket/data/ApplicationLogs2/,s3://logbucket/data/ApplicationLogs3/ it works fine and translates to ["s3://logbucket/data/ApplicationLogs1/","s3://logbucket/data/ApplicationLogs2/","s3://logbucket/data/ApplicationLogs3/"] and is interpreted by the Lambda without issue.
The parameter gets parsed through the CommaDelimitedList type and is passed to the Lambda without issue.
The issue arises in that I am trying to create a manual default, so I need to !Sub a list, as a string, then !Split to be passed as an actual list to the Custom Lambda. This doesn't seem to be working every way I try it and I cannot figure why.
I've been inspecting the success (manual param) and failure (defaults, without manual param) and I cant see a big difference.
The event of the lambda shows, when working:
{
"RequestType": "Create",
"ServiceToken": "hidden",
"ResponseURL": "hidden",
"StackId": "hidden",
"RequestId": "hidden",
"LogicalResourceId": "CreateLogTable",
"ResourceType": "Custom::CreateLogTable",
"ResourceProperties": {
"S3DataLocations": [
"s3://loggingbucket/data/ApplicationLogs/",
"s3://loggingbucket/data/ApplicationLogs/",
"s3://loggingbucket/data/ApplicationLogs/",
"s3://loggingbucket/data/ApplicationLogs/"
]
}
}
And when NOT working:
...
{
"RequestType": "Create",
"ServiceToken": "hidden",
"ResponseURL": "hidden",
"StackId": "hidden",
"RequestId": "hidden",
"LogicalResourceId": "CreateLogTable",
"ResourceType": "Custom::CreateLogTable",
"ResourceProperties": {
"S3DataLocations": [
"s3://logging/data/ApplicationLogs/",
" s3://loggingbucket/data/ApplicationLogs/",
" s3://loggingbucket/data/ApplicationLogs/",
" s3://loggingbucket/data/ApplicationLogs/"
]
}
}
I'm a little stuck here, I think there might be some Type mismatch but I cant tell the difference between the manual and param.
Does anyone have any idea?
You can break your string into multiple line while preventing the change of \n into space using quotation and slash combo.
To verify that, I used the following surrogate template for your situation:
Resources:
MyBucket:
Type: AWS::S3::Bucket
Properties: {}
Outputs:
Test1:
Value: !Sub
- s3://${LoggingBucket}/data/ApplicationLogs1/,
s3://${LoggingBucket}/data/ApplicationLogs2/,
s3://${LoggingBucket}/data/ApplicationLogs3/
- { LoggingBucket: "Parent-LoggingBucket"}
Test2:
Value: !Sub
- "s3://${LoggingBucket}/data/ApplicationLogs1/,\
s3://${LoggingBucket}/data/ApplicationLogs2/,\
s3://${LoggingBucket}/data/ApplicationLogs3/"
- { LoggingBucket: "Parent-LoggingBucket"}
The Test1 produces string with spaces as in your question:
s3://Parent-LoggingBucket/data/ApplicationLogs1/, s3://Parent-LoggingBucket/data/ApplicationLogs2/, s3://Parent-LoggingBucket/data/ApplicationLogs3/
In contrast, Test2 does not have space:
s3://Parent-LoggingBucket/data/ApplicationLogs1/,s3://Parent-LoggingBucket/data/ApplicationLogs2/,s3://Parent-LoggingBucket/data/ApplicationLogs3/
I'm trying to creating Cognito user pool with a custom domain name through AWS CDK. I manage to get everyting working untill to the point where I needed to create an A record in the Rout53 hosted zone. I searched through all the documents but coudn't find a way to do that. Following is my code. Any help would be much appriciated.
const cfnUserPool = new CfnUserPool(this, 'MyCognitoUserPool', {
userPoolName: 'MyCognitoUserPool',
adminCreateUserConfig: {
allowAdminCreateUserOnly: false
},
policies: {
passwordPolicy: {
minimumLength: 8,
requireLowercase: true,
requireNumbers: true,
requireSymbols: true,
requireUppercase: true,
temporaryPasswordValidityDays: 30
}
},
usernameAttributes: [
UserPoolAttribute.EMAIL
],
schema: [
{
attributeDataType: 'String',
name: UserPoolAttribute.EMAIL,
mutable: true,
required: true
},
{
attributeDataType: 'String',
name: UserPoolAttribute.FAMILY_NAME,
mutable: false,
required: true
},
{
attributeDataType: 'String',
name: UserPoolAttribute.GIVEN_NAME,
mutable: false,
required: true
}
]
});
const cognitoAppDomain = new CfnUserPoolDomain(this, "PigletAuthDomainName", {
domain: authDomainName,
userPoolId: cfnUserPool.ref,
customDomainConfig: {
certificateArn: 'ACM Certificate arn'
}
});
/*
TODO: Create an A record from the created cnfUserPoolDomain
*/
Everything works up untill to this point. Now the question is how to create an A record using the CfnUserPoolDomain
Any help is much appriciated.
Update May 2020
The UserPoolDomain construct has been extended and a UserPoolDomainTarget was added to provide this functionality.
Now, all you need to do is the following:
const userPoolDomain = new cognito.UserPoolDomain(this, 'UserPoolDomain', {
userPool,
customDomain: {
domainName: authDomainName,
certificate,
},
});
new route53.ARecord(this, 'UserPoolCloudFrontAliasRecord', {
zone: hostedZone,
recordName: authDomainName,
target: route53.RecordTarget.fromAlias(new route53_targets.UserPoolDomainTarget(userPoolDomain)),
});
I had the same Problem, It looks like CloudFormation does not have a return parameter for the CfnUserPoolDomain AliasTarget. Which means the cdk can not provide this parameter either.
I ended up implementing it using the AWS SDK (npm install aws-sdk) and getting the value using the APIs:
Update: The better solution is to use the AwsCustomResource. You can see a detailed example in aws/aws-cdk (#6787):
const userPoolDomainDescription = new customResources.AwsCustomResource(this, 'user-pool-domain-description', {
onCreate: {
physicalResourceId: 'user-pool-domain-description',
service: 'CognitoIdentityServiceProvider',
action: 'describeUserPoolDomain',
parameters: {
Domain: userPoolDomain.domain
}
}
});
const dnsName = userPoolDomainDescription.getData('DomainDescription.CloudFrontDistribution').toString();
// Route53 alias record for the UserPoolDomain CloudFront distribution
new route53.ARecord(this, 'UserPoolDomainAliasRecord', {
recordName: userPoolDomain.domain,
target: route53.RecordTarget.fromAlias({
bind: _record => ({
hostedZoneId: 'Z2FDTNDATAQYW2', // CloudFront Zone ID
dnsName: dnsName,
}),
}),
zone,
})
Here's how to get around it. Assuming you have a stack.yaml that you deploy with a CI tool, say through bash:
THE_STACK_NAME="my-cognito-stack"
THE_DOMAIN_NAME="auth.yourveryowndomain.org"
# get the alias target
# notice that it will be empty upon first launch (chicken and the egg problem)
ALIAS_TARGET=$(aws cognito-idp describe-user-pool-domain --domain ${THE_DOMAIN_NAME} | grep CloudFrontDistribution | cut -d \" -f4)
# create/update the deployment CloudFormation stack
# notice the AliasTarget parameter (which can be empty, it's okay!)
aws cloudformation deploy --stack-name ${THE_STACK_NAME} --template-file stack.yaml --parameter-overrides AliasTarget=${ALIAS_TARGET} DomainName=${THE_DOMAIN_NAME}
The stack.yaml minimal version (remember to fill the UserPool config):
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
DomainName:
Type: String
Default: auth.yourveryowndomain.org
Description: The domain name to use to serve this project.
ZoneName:
Type: String
Default: yourveryowndomain.org
Description: The hosted zone name coming along with the DomainName used.
AliasTarget: # no default value, can be empty
Type: String
Description: The UserPoolDomain alias target.
Conditions: # here's "the trick"
HasAliasTarget: !Not [!Equals ['', !Ref AliasTarget]]
Resources:
Certificate:
Type: "AWS::CertificateManager::Certificate"
Properties:
DomainName: !Ref ZoneName
DomainValidationOptions:
- DomainName: !Ref ZoneName
ValidationDomain: !Ref ZoneName
SubjectAlternativeNames:
- !Ref DomainName
UserPool:
Type: AWS::Cognito::UserPool
Properties:
[... fill that with your configuration! ...]
UserPoolDomain:
Type: AWS::Cognito::UserPoolDomain
Properties:
UserPoolId: !Ref UserPool
Domain: !Ref DomainName
CustomDomainConfig:
CertificateArn: !Ref Certificate
DnsRecord: # if AliasTarget parameter is empty, well we just can't do that one!
Condition: HasAliasTarget # and here's how we don't do it when we can't
Type: AWS::Route53::RecordSet
Properties:
HostedZoneName: !Sub "${ZoneName}."
AliasTarget:
DNSName: !Ref AliasTarget
EvaluateTargetHealth: false
# HostedZoneId value for CloudFront is always this one
# see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-route53-aliastarget.html
HostedZoneId: Z2FDTNDATAQYW2
Name: !Ref DomainName
Type: A
Be aware CloudFormation conditions are not "a trick" at all: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/conditions-section-structure.html. We simply use it as a trick along with the "first launch won't do it all" to get around our scenario.
Kinda weird, but only for the first run! Launch it again: everything is fine.
PS: can't wait to avoid all that by simply having the CloudFrontDistribution alias target directly in the AWS::Cognito::UserPoolDomain return values!!
When creating ECS infrastructure we describe our Task Definitions with CloudFormation. We want to be able to dynamically pass environment variables as a parameter to the template. According to the docs, Environment has a KeyValuePair type, but CloudFormation parameters do not have this type.
We can not hardcode Environment variables to the template, because this template is used as a nested stack so environment variables will be dynamically passed inside it.
The only possible way I see so far is to pass all arguments as a CommaDelimitedList, and then somehow parse and map it using CloudFormation functions. I can Fn::Split every entity in key and value, but how to dynamically build an array of KeyValuePair in CloudFormation?
Or maybe there is an easier way, and I'm missing something? Thanks in advance for any ideas.
I know it's late and you have already found a workaround. However, the following is the closest I came to solve this. Still not completely dynamic as expected parameters have to be defined as placeholders. Therefore the maximum number of environment variables expected should be known.
The answer is based on this blog. All credits to the author.
Parameters:
EnvVar1:
Type: String
Description: A possible environment variable to be passed on to the container definition.
Should be a key-value pair combined with a ':'. E.g. 'envkey:envval'
Default: ''
EnvVar2:
Type: String
Description: A possible environment variable to be passed on to the container definition.
Should be a key-value pair combined with a ':'. E.g. 'envkey:envval'
Default: ''
EnvVar3:
Type: String
Description: A possible environment variable to be passed on to the container definition.
Should be a key-value pair combined with a ':'. E.g. 'envkey:envval'
Default: ''
Conditions:
Env1Exist: !Not [ !Equals [!Ref EnvVar1, '']]
Env2Exist: !Not [ !Equals [!Ref EnvVar2, '']]
Env3Exist: !Not [ !Equals [!Ref EnvVar3, '']]
Resources:
TaskDefinition:
ContainerDefinitions:
-
Environment:
- !If
- Env1Exist
-
Name: !Select [0, !Split [":", !Ref EnvVar1]]
Value: !Select [1, !Split [":", !Ref EnvVar1]]
- !Ref "AWS::NoValue"
- !If
- Env2Exist
-
Name: !Select [0, !Split [":", !Ref EnvVar2]]
Value: !Select [1, !Split [":", !Ref EnvVar2]]
- !Ref "AWS::NoValue"
- !If
- Env3Exist
-
Name: !Select [0, !Split [":", !Ref EnvVar3]]
Value: !Select [1, !Split [":", !Ref EnvVar3]]
- !Ref "AWS::NoValue"
You may want to consider using the EC2 Parameter Store to create secured key/value pairs, which is supported in CloudFormation, and can be integrated with ECS environments.
AWS Systems Manager Parameter Store
AWS Systems Manager Parameter Store provides secure, hierarchical
storage for configuration data management and secrets management. You
can store data such as passwords, database strings, and license codes
as parameter values. You can store values as plain text or encrypted
data. You can then reference values by using the unique name that you
specified when you created the parameter. Highly scalable, available,
and durable, Parameter Store is backed by the AWS Cloud. Parameter
Store is offered at no additional charge.
While Parameter Store has great security features for storing application secrets, it can also be used to store nonsensitive application strings such as public keys, environment settings, license codes, etc.
And it is supported directly by CloudFormation, allowing you to easily capture, store and manage application configuration strings which can be accessed by ECS.
This template allows you provide the Parameter store key values at stack creation time via the console or CLI:
Description: Simple SSM parameter example
Parameters:
pSMTPServer:
Description: SMTP Server URL eg [email-smtp.us-east-1.amazonaws.com]:587
Type: String
NoEcho: false
SMTPServer:
Type: AWS::SSM::Parameter
Properties:
Name: my-smtp-server
Type: String
Value: !Ref pSMTPServer
Any AWS runtime environment (EC2, ECS, Lambda) can easily securely retrieve the values. From the console side, there is great Parameter manager interface that maintains parameter version history. Its intergated with IAM, so permissions are controlled with standard IAM policy syntax:
{
"Action": [
"ssm:GetParameterHistory",
"ssm:GetParameter",
"ssm:GetParameters",
"ssm:GetParametersByPath"
],
"Resource": [
"arn:aws:ssm:us-west-2:555513456471:parameter/smtp-server"
],
"Effect": "Allow"
},
{
"Action": [
"kms:Decrypt"
],
"Resource": [
"arn:aws:kms:us-west-2:555513456471:key/36235f94-19b5-4649-84e0-978f52242aa0a"
],
"Effect": "Allow"
}
Finally, this blog article shows a technique to read the permissions into a Dockerfile at runtime. They suggest a secure way to handle environment variables in Docker with AWS Parameter Store. For reference, I am including their Dockerfile here:
FROM grafana/grafana:master
RUN curl -L -o /bin/aws-env https://github.com/Droplr/aws-env/raw/master/bin/aws-env-linux-amd64 && \
chmod +x /bin/aws-env
ENTRYPOINT ["/bin/bash", "-c", "eval $(/bin/aws-env) && /run.sh"]
With that invocation, each of the parameters are available as an environment variable in the container. You app may or may not need a wrapper to read the parameters from the environment variables.
I was facing the same problem ,I needed to create a lambda resource with environment variables.
We decided to fix initial set of environment variable and keys name are also decided in advance.
So I added four parameters , and used Ref for values while keeping fixed keys name.
There is another way too - which may sound overkill but it allows to put whatever env. to the function you wish, no need to "predefine" how many env. variables, only restriction in sample below - can not use :::: or |||| inside value of the key. Key can't have such symbols by AWS docs already.
Gameplan:
Make an inline CF Lambda Function with code which accepts all env in any format you wish as a string and uses any code you want to use inside that function (i use JS with NodeJS env) and while it's your code, parse how you wish that string passing in and use aws-sdk to update the function. Call function once inside the CF template.
In this sample you pass in env as such string:
key1::::value1||||key2::::value2 If you need to use :::: or |||| in your value, of course update to some other divider.
Not big fan of running lambda for such task, yet i want to have option of passing in virtually any env. to the CF template and this works.
LambdaToSetEnvRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/CloudWatchLambdaInsightsExecutionRolePolicy
Policies:
- PolicyName: cloudwatch-logs
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource:
- !Sub "arn:aws:logs:*:${AWS::AccountId}:log-group:*:*"
- !Sub "arn:aws:logs:*:${AWS::AccountId}:log-group:/aws/lambda-insights:*"
- PolicyName: trigger-lambda-by-cloud-events
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'lambda:UpdateFunctionConfiguration'
Resource:
- !GetAtt OriginalLambda.Arn
Tags:
- { Key: managed-by, Value: !Ref AWS::StackId }
LambdaToSetEnv:
Type: AWS::Lambda::Function
DeletionPolicy: Delete
Properties:
Code:
ZipFile: |
const response = require('cfn-response');
const aws = require('aws-sdk');
exports.handler = (event, context) => {
console.log(JSON.stringify({event, context}));
try {
if (event.RequestType === "Delete") {
response.send(event, context, response.SUCCESS, {RequestType: event.RequestType});
} else {
const client = new aws.Lambda({apiVersion: '2015-03-31'});
const Variables = {
"All": process.env.FunctionEnvVariables,
};
console.log('process.env.FunctionEnvVariables: ', process.env.FunctionEnvVariables);
if(process.env.FunctionEnvVariables){
process.env.FunctionEnvVariables.split('||||').forEach((pair) => {
if(pair && pair.trim() !== ''){
Variables[pair.split('::::')[0]] = pair.split('::::')[1];
}
})
}
const result = client.updateFunctionConfiguration({ FunctionName: process.env.LambdaToUpdateArn, Environment: { Variables } }, function (error, data){
console.log('data: ', data);
console.log('error: ', error);
if(error){
console.error(error);
response.send(event, context, response.ERROR, {});
} else {
response.send(event, context, response.SUCCESS, {});
}
});
}
} catch (e) {
response.send(event, context, response.ERROR, e.stack);
}
}
Role: !GetAtt LambdaToSetEnvRole.Arn
Handler: index.handler
Runtime: nodejs14.x
Timeout: '300'
Environment:
Variables:
LambdaToUpdateArn: !GetAtt OriginalLambda.Arn
FunctionEnvVariables: !Ref FunctionEnvVariables
LambdaCall:
DependsOn:
- OriginalLambda
- LambdaToSetEnv
Type: Custom::LambdaCallout
Properties:
ServiceToken: !GetAtt LambdaToSetEnv.Arn
I'm trying to use the Serverless Framework to deploy a Kinesis Firehose that outputs to an ElasticSearch domain.
Since the Firehose needs the ES domain to already exist before it can be created, I am running into this error:
An error occurred: MyFirehoseStream - Domain
arn:aws:es:us-east-1:1234567890:domain/my-elastic-search is still
being created.
Is there a way to make the Firehose creation wait until after the ES domain creation is complete?
Just in case its helpful, here are the relevant parts of my serverless.yml file:
fyi, I'm using the serverless-pseudo-parameters plugin to use #{AWS::Region} and #{AWS::AccountId}
resources:
Resources:
MyFirehoseStream:
Type: "AWS::KinesisFirehose::DeliveryStream"
Properties:
DeliveryStreamName: "MyFirehoseStream"
DeliveryStreamType: "DirectPut"
ElasticsearchDestinationConfiguration:
BufferingHints:
IntervalInSeconds: 300
SizeInMBs: 5
DomainARN: "arn:aws:es:#{AWS::Region}:#{AWS::AccountId}:domain/my-elastic-search"
IndexName: "myindex"
IndexRotationPeriod: "NoRotation"
RetryOptions:
DurationInSeconds: 300
RoleARN: { "Fn::GetAtt": ["FirehoseBackupBucketRole", "Arn" ] }
S3BackupMode: "FailedDocumentsOnly"
S3Configuration:
BucketARN: { "Fn::GetAtt": ["FirehoseBackupBucket", "Arn" ] }
BufferingHints:
IntervalInSeconds: 300
SizeInMBs: 5
CompressionFormat: "GZIP"
RoleARN: { "Fn::GetAtt": ["FirehoseBackupBucketRole", "Arn" ] }
TypeName: "mytype"
MyElasticSearch:
Type: "AWS::Elasticsearch::Domain"
Properties:
AccessPolicies: ${file(./iam_policies/elastic-search.json)}
DomainName: "my-elastic-search"
ElasticsearchVersion: 6.2
ElasticsearchClusterConfig:
InstanceCount: "1"
InstanceType: "t2.small.elasticsearch"
EBSOptions:
EBSEnabled: true
Iops: 0
VolumeSize: 10
VolumeType: "gp2"
UPDATE:
I have this fixed now, so in case the specifics are helpful for anyone:
I changed the DomainARN property to { "Fn::GetAtt": ["MyElasticSearch", "DomainArn" ] }.
The reason I was originally generating the ARN dynamically was because with "Fn::GetAtt" I originally tried to use just Arn instead of DomainArn, which didn't work. Coincidentally though, DomainArn has been deprecated in the latest version, so if you are using the latest version, Arn actually would be correct.
Cloudformation resources support the DependsOn attribute.
resources:
Resources:
MyFirehoseStream:
Type: "AWS::KinesisFirehose::DeliveryStream"
DependsOn: MyElasticSearch
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-dependson.html