I have two nested Cloudformation stacks - the first template needs to define a Kinesis stream, the second needs to use a reference to that stream's ARN, as an argument to a further nested stack.
So it seems I need to "export" the stream from the first template, and "import" it into the second (following AWS docs on importing/exporting values across stacks) -
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html
My export code [truncated] looks like this -
Outputs:
MyKinesisStreamOutput:
Value:
Ref: MyKinesisStream
Export:
Name: my-kinesis-stream
Resources:
MyKinesisStream:
Properties:
Name:
Ref: AppName
ShardCount: 1
Type: AWS::Kinesis::Stream
Whilst my import code [truncated] looks like this -
MyNestedStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub "https://s3.${AWS::Region}.amazonaws.com/my-nested-stack.yaml"
Parameters:
AppName: my-nested-stack
KinesisStream:
Fn::GetAtt:
- Fn::ImportValue:
my-kinesis-stream
- Arn
But then I get the following Cloudformation error -
Template error: every Fn::GetAtt object requires two non-empty parameters, the resource name and the resource attribute
and suspect I am falling foul of this -
For the Fn::GetAtt logical resource name, you cannot use functions. You must specify a string that is a resource's logical ID.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html
Assuming I am exporting and importing the Kinesis stream correctly, how then am I supposed to get its Arn value ?
When you export something in Outputs, it's just a string, you can no longer GetAtt on it in the importing template.
What you need to do is to additionally export ARN:
Outputs:
MyKinesisStream:
Value: !Ref MyKinesisStream
Export:
Name: my-kinesis-stream
MyKinesisStreamArn:
Value: !GetAtt MyKinesisStream.Arn
Export:
Name: my-kinesis-stream-arn
I can think of two potential ways:
1) Fn::GetAtt:[!ImportValue my-kinesis-stream, Arn]
Sorry wasn't reading carefully enough
or what I would prefer
2) Directly export the required value as output: Value: !GetAtt MyKinesisStream.Arn
Hope that helps!
Related
I have the following step function in my AWS SAM template, it was defined using the syntax in the documentation. I'm using intrinsic functions to get some pseudoparameters but something is going wrong with them.
SfdcOrderEventsStepFunction:
Type: AWS::Serverless::StateMachine
Properties:
DefinitionSubstitutions:
Region: !Ref "AWS::Region"
AccountId: !Ref "AWS::AccountId"
EventBusPublishTarget: "order-events"
DefinitionUri: sfn-definition.asl.yml
Events:
EventBridgeEvent:
Type: EventBridgeRule
Properties:
EventBusName: sfdc-events
Pattern:
# TODO: Update pattern when the salesforce service is ready
source:
- prefix: salesforce.com
detail-type:
- Order
detail:
Payload__c:
attributes:
type:
- order
InputPath: $.detail
Name: sfdc-order-events
Role: !Sub 'arn:aws:iam::${AWS::AccountId}:role/stepfunction_execution_role'
Tracing:
Enabled: true
when I try to deploy it shows me the following error:
Resource template validation failed for resource
SfdcOrderEventsStepFunction as the template has invalid properties.
Please refer to the resource documentation to fix the template.
Properties validation failed for resource SfdcOrderEventsStepFunction
with message:
#/De finitionSubstitutions/ AccountId: 2 subschemas matched instead of one
#/DefinitionSubstitutions/AccountId: expected type: Boolean, found: String
At the end it deploys without problems. The step function and all of its components run as expected and without errors, but I wanted to know if there if something I can do to fix the template.
I'm making one main stack and two nested stack.
The first nested stack is a stack which creates Lambda.
And the second nested stack creates Apigateway.
When making the Apigateway I send some Lambda outputs to the Apigateway template.
But some Lambda Outputs don't get created in some conditions.
So when I use Fn::GetAttr to send some outputs to the Apigateway template. I get an error because the output doesn't exist.
[Lambda Template (Nested Stack)]
Because of the condition conditionNeedMock the MockServerArn output wasn't created.
...
Outputs:
MockServerArn:
Condition: conditionNeedMock
Description: lambdaMock function Arn
Value: !GetAttr lambdaMock.Arn
...
[Main Template]
LambdaStack.Ouputs.MockserverArn wasn't created so I get an error when using !GetAttr.
...
Resources:
LambdaStack:
Type: AWS::CloudFormation::Stack
...
ApiGatewayStack:
Type: AWS::CloudFormation::Stack
DependsOn: LambdaStack
Properties:
TemplateURL: apigateway.yml
TimeoutInMinutes: 20
Parameters:
paramMockServerArn: !GetAtt LambdaStack.Outputs.MockServerArn
...
The only way that could possibly work (without introducing custom resources) is if conditionNeedMock is also in the root stack:
paramMockServerArn: !If [conditionNeedMock, !GetAtt LambdaStack.Outputs.MockServerArn, !Ref "AWS::NoValue"]
I'm trying to create a AWS::MSK::Configuration resource, as described here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-msk-configuration.html.
This is my CF template:
Resources:
MSKConfig:
Type: AWS::MSK::Configuration
Properties:
Description: Basic configuration.
Name: test-msk-configuration
ServerProperties: |
auto.create.topics.enable=false
........
It doesn't work failing with the following error:
An error occurred (ValidationError) when calling the CreateChangeSet operation: Template format error: #Unrecognized resource types: [AWS::MSK::Configuration]
I'm not able to find any example with this resource type online. Has anyone ever used it?
Update on 26/03/2022:
I was able to get this deployed. I use the serverless framework for deployment.
MSK-Cluster.yml:
Resources:
ServerlessMSK:
Type: AWS::MSK::Cluster
Properties:
ClusterName: ${self:service}-${self:provider.stage}-msk
KafkaVersion: 2.6.2
BrokerNodeGroupInfo:
InstanceType: kafka.t3.small
ClientSubnets:
- !Ref ServerlessPrivateSubnet1
- !Ref ServerlessPrivateSubnet2
- !Ref ServerlessPrivateSubnet3
SecurityGroups:
- !GetAtt ServerlessMSKSecurityGroup.GroupId
StorageInfo:
EBSStorageInfo:
VolumeSize: 10
NumberOfBrokerNodes: 3
EncryptionInfo:
EncryptionInTransit:
ClientBroker: TLS
InCluster: true
EnhancedMonitoring: PER_TOPIC_PER_BROKER
ConfigurationInfo:
Arn: !GetAtt ServerlessMSKConfiguration.Arn
Revision: 1
MSK-config.yml
Resources:
ServerlessMSKConfiguration:
Type: AWS::MSK::Configuration
Properties:
Description: cluster for msk cluster-${sls:stage}
Name: node-mongo-kafka-experiment-${sls:stage}-config
ServerProperties: ${file('./assets/server.properties')}
server.properties
auto.create.topics.enable=true
default.replication.factor=2
min.insync.replicas=2
num.io.threads=8
num.network.threads=5
num.partitions=10
num.replica.fetchers=2
replica.lag.time.max.ms=30000
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
socket.send.buffer.bytes=102400
unclean.leader.election.enable=true
zookeeper.session.timeout.ms=18000
basically, no base64 was used. I just referenced the file in the deployment and managed to get it up. Hope this helps.
Orginal Answer Below:
I haven't been able to deploy this correctly too. But maybe I can point you in the right direction.I always get back a 400 error with my serverless deployment for this. The only thing I would like to add on here is that serverproperties must be a Base64 encoded string.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-msk-configuration.html
My Template (that results in error):
Resources:
ServerlessMSKConfiguration:
Type: AWS::MSK::Configuration
Properties:
ServerProperties: !Base64 |
auto.create.topics.enable=true
NB: Answering due to the fact I don't have reputation to comment.
I managed to fix the 400 issue, it looks like the AWS::MSK::Configuration resource suspects a name. While the documentation says it is not required.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-msk-configuration.html#cfn-msk-configuration-name
My template:
KafkaConfiguration:
Type: AWS::MSK::Configuration
Properties:
Name: Kafkaconfiguration
ServerProperties: |
auto.create.topics.enable=false
I am working on a proof of concept for deploying a conformance pack via AWS cloudformation and I am stumped by the error "Parameter values specified for a template which does not require them." The config rule I am using does require a parameter. Code is attached. I have also tested the template with cfn-lint and do not receive any feedback/errors.
My template is "simple" and below:
Parameters:
ElbPredefinedSecurityPolicySslCheckParamPredefinedPolicyName:
Default: ELBSecurityPolicy-2016-08
Type: String
Resources:
TestingConformancePack:
Type: AWS::Config::ConformancePack
Properties:
ConformancePackName: TestCP
ConformancePackInputParameters:
-
ParameterName: PredefinedPolicyName
ParameterValue: !Ref ElbPredefinedSecurityPolicySslCheckParamPredefinedPolicyName
TemplateBody: |
Resources:
ElbPredefinedSecurityPolicySslCheck:
Properties:
ConfigRuleName: elb-predefined-security-policy-ssl-check
InputParameters:
predefinedPolicyName:
Ref: ElbPredefinedSecurityPolicySslCheckParamPredefinedPolicyName
Scope:
ComplianceResourceTypes:
- AWS::ElasticLoadBalancing::LoadBalancer
Source:
Owner: AWS
SourceIdentifier: ELB_PREDEFINED_SECURITY_POLICY_SSL_CHECK
Type: AWS::Config::ConfigRule
The cause is that you are passing a parameter (the one specified in ConformancePackInputParameters) to a CloudFormation template (the one specified in TemplateBody) that does not contain a Parameters section and therefore expects no parameters. To solve this, you need to add a parameter to the inner CloudFormation template, which you can then refer to in predefinedPolicyName:
The following template works for me:
Parameters:
ElbPredefinedSecurityPolicySslCheckParamPredefinedPolicyName:
Default: ELBSecurityPolicy-2016-08
Type: String
Resources:
TestingConformancePack:
Type: AWS::Config::ConformancePack
Properties:
ConformancePackName: TestCP
ConformancePackInputParameters:
-
ParameterName: PredefinedPolicyName
ParameterValue: !Ref ElbPredefinedSecurityPolicySslCheckParamPredefinedPolicyName
TemplateBody: |
Parameters:
PredefinedPolicyName:
Type: String
Resources:
ElbPredefinedSecurityPolicySslCheck:
Properties:
ConfigRuleName: elb-predefined-security-policy-ssl-check
InputParameters:
predefinedPolicyName:
Ref: PredefinedPolicyName
Scope:
ComplianceResourceTypes:
- AWS::ElasticLoadBalancing::LoadBalancer
Source:
Owner: AWS
SourceIdentifier: ELB_PREDEFINED_SECURITY_POLICY_SSL_CHECK
Type: AWS::Config::ConfigRule
I was making a test case resource using cloudformation and stumbled upon this same error.
“Parameter values specified for a template which does not require them.”
Since it was a test case, I didn't use any parameters at all. The above answer was helpful for me to understand it has to do something with parameters. Even though you are not using any, there are some parameters passed while deploying the cfn.
By default, cloudformation also sends env as parameter which needs to come under parameters as such. (Below code snippet in JSON)
"Parameters": {
"env": {
"Type": "String"
}
},
Hope this was helpful.
I want to read the URL of my database from parameter store in my CloudFormation template. This is easy enough for a single URL, but I can't figure out how to change the URL with different environments.
I have four environments (development, integration, pre-production and production) and their details are stored on four different paths in Parameter Store:
/database/dev/url
/database/int/url
/database/ppe/url
/database/prod/url
I now want to pick the correct Database URL when deploying via CloudFormation. How can I do this?
Parameters:
Environment:
Type: String
Default: dev
AllowedValues:
- dev
- int
- ppe
- prod
DatabaseUrl:
Type: 'AWS::SSM::Parameter::Value<String>'
# Obviously the '+' operator here won't work - so what do I do?
Default: '/database/' + Environment + '/url'
This feature isn't as neat as one would wish. You have to actually pass name/path of each parameter that you want to look up from the parameter store.
Template:
AWSTemplateFormatVersion: 2010-09-09
Description: Example
Parameters:
BucketNameSuffix:
Type: AWS::SSM::Parameter::Value<String>
Default: /example/dev/BucketNameSuffix
Resources:
Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub parameter-store-example-${BucketNameSuffix}
If you don't pass any parameters to the template, BucketNameSuffix will be populated with the value stored at /example/dev/BucketNameSuffix. If you want to use, say, a prod value (pointed to by /example/prod/BucketNameSuffix), then you should specify value for parameter BucketNameSuffix, but instead of passing the actual value you should pass the alternative name of the parameter to use, so you'd pass /example/prod/BucketNameSuffix.
aws cloudformation update-stack --stack-name example-dev \
--template-body file://./example-stack.yml
aws cloudformation update-stack --stack-name example-prod \
--template-body file://./example-stack.yml \
--parameters ParameterKey=BucketNameSuffix,ParameterValue=/example/prod/BucketNameSuffix
A not so great AWS blog post on this: https://aws.amazon.com/blogs/mt/integrating-aws-cloudformation-with-aws-systems-manager-parameter-store/
Because passing million meaningless parameters seems stupid, I might actually generate an environment-specific template and set the right Default: in the generated template so for prod environment Default would be /example/prod/BucketNameSuffix and then I can update the prod stack without passing any parameters.
You can populate CloudFormation templates with parameters stored in AWS Systems Manager Parameter Store using Dynamic References
In this contrived example we make two lookups using resolve:ssm and replace the environment using !Join and !Sub
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Environment:
Type: String
Default: prod
AllowedValues:
- prod
- staging
Resources:
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
Image: !Join
- ''
- - 'docker.io/bitnami/redis#'
- !Sub '{{resolve:ssm:/app/${Environment}/digest-sha/redis}}'
Environment:
- Name: DB_HOST
Value: !Sub '{{resolve:ssm:/app/${Environment}/db-host}}'
You can make use of Fn::Join here.
Here is some pseudo code.
You will have to take Environment as an parameter, which you are already doing.
Creating the required string in the resources where DatabaseUrl is required.
Resources :
Instance :
Type : 'AWS::Some::Resource'
Properties :
DatabaseURL : !Join [ "", [ "/database/", !Ref "Environment" , "/url ] ]
Hope this helps.
Note: You cannot assign value to a Parameter dynamically using some computation logic. All the values for defined parameters should be given as Input.
I like Fn::Sub, which is much cleaner and easy to read.
!Sub "/database/${Environment}/url"
I am stuck into the same problem, below are my findings:
We can compose values and descriptions while writing into the SSM Parameter Store from CloudFormation like below :
LambdaARN:
Type: AWS::SSM::Parameter
Properties:
Type: String
Description: !Sub "Lambda ARN from ${AWS::StackName}"
Name: !Sub "/secure/${InstallationId}/${AWS::StackName}/lambda-function-arn"
Value: !GetAtt LambdaFunction.Arn
We can not compose values/defaults to looking for in SSM Parameter Store. Like below:
Parameters:
...
LambdaARN:
Type: Type: AWS::SSM::Parameter::Value<String>
Value: !Sub "/secure/${InstallationId}/teststack/lambda-function-arn"
This is not allowed as per AWS Documentation[1]. Both(Sub/Join) Functions won't work. Following is the error which I was getting:
An error occurred (ValidationError) when calling the CreateChangeSet
operation: Template format error: Every Default member must be a
string.
Composing and passing values while creating stacks can be done like this:
Parameters:
...
LambdaARN:
Type: Type: AWS::SSM::Parameter::Value<String>
....
$ aws cloudformation deploy --template-file cfn.yml --stack-name mystack --parameter-overrides 'LambdaARN=/secure/devtest/teststack/lambda_function-arn'
If you add your custom tags while putting the values in the Parameter Store, it will overwrite the default tags added by CFN.
Default Tags:
- aws:cloudformation:stack-id
- aws:cloudformation:stack-name
- aws:cloudformation:logical-id
Every time we update the values in parameter store, it creates a new version, which is beneficial when we are using DynamicResolvers, this can serve as a solution to the problem in the question like
{{ resolve:ssm:/my/value:1}}
The last field is the version. Where different versions can point to different environments.
We are using versions with the parameters, and adding the labels to them[2], this can't be done via CFN[3], only possible way via CLI or AWS Console. This is AWS's way of handling multiple environments.
[1] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html
[2] https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-labels.html
[3] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html