What I want to accomplish is to store the entered parameters from a Cloudformation stack.
For example: Imagine having two parameters param1 and param2.
I want to store the entered values either in DynamoDB, RDS Db, Etc.
I though in SNS notification:
Unfortunately, the notification's payload looks as follow:
StackId='arn:aws:cloudformation:us-east-1:accountId:stack/rfdsf/b6df0100-fd18-11e7-b3ab-500c2893c0d2'
Timestamp='2018-01-19T13:00:24.774Z'
EventId='b6df9d40-fd18-11e7-b3ab-500c2893c0d2'
LogicalResourceId='rfdsf'
Namespace='accountId'
PhysicalResourceId='arn:aws:cloudformation:us-east-1:accountId:stack/rfdsf/b6df0100-fd18-11e7-b3ab-500c2893c0d2'
PrincipalId='accountId'
ResourceProperties='null'
ResourceStatus='CREATE_IN_PROGRESS'
ResourceStatusReason='User Initiated'
ResourceType='AWS::CloudFormation::Stack'
StackName='rfdsf'
ClientRequestToken='Console-CreateStack-774eec95-c976-434c-b43b-ad3d295a0b9b'
As you can see, there is not any entered values.
Is it possible to store the entered parameters into a DB?
As suggested by #Rodrigigo M, You can save the params into SSM parameters store.
Description: "Create SSM Parameter"
Resources:
BasicParameter:
Type: "AWS::SSM::Parameter"
Properties:
Name: "param1"
Type: "String"
Value: "ABCD"
Description: "SSM Parameter for running date command."
AllowedPattern: "^[a-zA-Z]{1,10}$"
Also, if you want to save these into DB, you can create a Lambda to read them and store into DynamoDb or RDS.
In your cloudformation there is an Outputs parameter. You can output any value that was brought into the stack, so long as you explicitly specify which parameters you want to output.
Those values will be visible in the Outputs tab of cloudformation. If you want to move them to a database, such as DynamoDB, you can use the cloudformation:describeStacks api call to get all the output values for any stack.
Related
I am trying to pass a parameter through the aws cloudformation deploy command like this:
aws cloudformation deploy --template-file sam-template.packaged.yaml --parameter-overrides ExcTime=12345
But it fails deploying because it is complaining that the ExcTime parameter is a string and it needs to be a number. Is it the command casting all the parameters as string? And if so, how can I pass a parameter of type number to the YML file through that command?
The yml file:
ApiKey:
Type: AWS::AppSync::ApiKey
Properties:
ApiId: !GetAtt AppSyncApi.ApiId
Expires: ${ExcTime}
Thank You all!
To refer to your parameter, you should use:
Expires: !Ref ExcTime
So i figured out why. The --parameter-overrides Param=value only accepts strings
--parameter-overrides (list) A list of parameter structures that
specify input parameters for your stack template. If you're updating
a stack and you don't specify a parameter, the command uses the
stack's existing value. For new stacks, you must specify parameters
that don't have a default value. Syntax:
ParameterKey1=ParameterValue1 ParameterKey2=ParameterValue2 ...(string).
https://docs.aws.amazon.com/cli/latest/reference/cloudformation/deploy/index.html
To pass a number, You need to specify that the value is a number in the sam template like this:
Parameters:
YourParam:
Type: Number
Description: Bla bla bla
And then You can use it as a number in the template. For example
Expires: !Ref YourParam
After going through the docs and examples I haven't clarified where exactly the parameters from the schema.yaml file are used.
Using the AWS code example here: https://github.com/aws-samples/aws-proton-sample-templates/blob/main/lambda-crud-svc/service/schema/schema.yaml
Pertinent section of the schema.yaml file:
schema:
format:
openapi: "3.0.0"
service_input_type: "CrudServiceInput"
pipeline_input_type: "PipelineInputs"
types:
CrudServiceInput:
type: object
description: "Input properties for a Lambda backed CRUD API. When given a resource name, this input will be used to generate Create, Read, Update and Delete API methods and lambdas."
properties:
resource_name:
type: string
description: "The resource to generate a CRUD API for"
minLength: 1
maxLength: 50
default: "greeting"
resource_handler:
type: string
description: "The handler path to find the CRUD methods for this API"
minLength: 1
maxLength: 50
default: "index"
lambda_memory:
type: number
description: "The size of your Lambda functions in MB"
default: 512
minimum: 1
maximum: 3008
...
I would expect that in the cloudformation.yaml file I would be able to reference {{service_input_type.resource_name}}, but it is referred to as {{service.resource_name}}.
Assume that Proton somehow has service namespace mapped to the values in service_input_type.
HOWEVER
When you use that logic for "lambda_memory" parameter in the same service_input_type object, it doesn't work, because in the template file it refers to this as service_instance.lambda_memory
Can anyone clarify the following:
How are the schema.yaml parameters consumed in the
cloudformation.yaml template?
Further... how do the "xx-spec.yaml"
files play into the mix. I assume they are merged into the service
template when creating the instance, but the parameter naming
convention is also different than the template parameters above.
You should always use service_instance as the namespace, with the exception of
if you are parametrizing a pipeline template, where you can use service;
if you are referring to the name, where you can use the service_name and service_instance_name variables (no namespace).
A Proton service is ultimately a collection of service instances with a pipeline. When Proton provisions infrastructure, it will create a CloudFormation stack for every instance, so the parameters are always applied at the instance level. The pipeline is an exception because it doesn't belong to a single environment, and so in certain cases, you might need to refer to the service as a whole.
As per the xx-spec.yaml files - Those are actually the reflections of the parameters that the developer might provide. If you use the Proton UX to create a new service, Proton will output a file like that and store it to reflect the parameters for that particular service.
According to AWS, when deploying infrastructure that requires secrets, i.e. passwords or similar, with CloudFormation, one popular solution consists in using the Parameter Store's SecureStrings from SSM.
However, despite existing CFN documentation describing step-by-step how to use the Dynamic References within the CFN templates, I can not manage to make use of the actual value of the SecureStrings.
Assuming the following JSON representation of the an existing SecureString stored in the SSM Parameter Store:
{
"MyRedshiftMasterUserPassword": {
"value": "Abcd2019",
"type": "SecureString"
}
}
and a YAML CFN template that uses it as stated in the documentation:
Resources
Redshift:
Type: 'AWS::Redshift::Cluster'
Properties:
NodeType: dc2.large
NumberOfNodes: !Ref RedshiftNodes
ClusterType: multi-node
AutomatedSnapshotRetentionPeriod: !Ref AutomatedSnapshotRetentionPeriod
DBName: datawarehouse_v1
MasterUsername: !Ref RedshiftMasterUsername
MasterUserPassword: '{{resolve:ssm-secure:MyRedshiftMasterUserPassword:1}}'
The above solution does not to work so either I am defining the template incorrectly or the support for this feature is not properly implemented which seems odd to me considering that it comes from AWS.
Particularly, I came across the following errors that all end up as UPDATE_FAILED stack:
Whenever the referenced Parameter Name to be resolved is long enough CloudFormation complains:
The parameter MasterUserPassword is not a valid password because it is longer than 64 characters. (Service: AmazonRedshift; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 7be9bd43-2927-11e9-aa88-29bbdcae859e)
Additionally, even though specifically mentioned that slashes can be used in the template references, e.g. /infrastructure/datawarehouse/redshift/MyRedshiftMasterUserPassword following error is issued:
The parameter MasterUserPassword is not a valid password. Only printable ASCII characters except for '/', '#', '"', ' ', '\', ''' may be used. (Service: AmazonRedshift; Status Code: 400; Error Code: InvalidParameterValue)
thus, as result referenced SecureString do not seem to be compatible with the SSM ParameterStore hierarchy (parameters with slashes).
Moreover, removing any of the previously reported invalid character from the parameter name, then it complains of the following:
The parameter MasterUserPassword must contain at least 1 upper case letter. (Service: AmazonRedshift; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 90a263bd-2929-11e9-80c0-ffcecf297c44)
In the end, although using a basic short non-slashed Parameter name in a template allows the stack to finish the Update operation dynamic reference still does not occur as the actual value used turns out to be supplied Parameter Name rather than the value referenced by this, e.g MyRedshiftMasterUserPassword instead of Abcd2019.
I am aware that AWS Secrets Manager could also be used but it is not free.
Opened support case with AWS requesting guidance for this particular strange behaviour of CloudFormation.
According to the support team, effectively this is indeed a known bug for the CloudFormation service without estimated time to fix though.
The resolution of the a SSM Parameter Store SecureString parameter when used as a Dynamic Reference in the particular case of the RedshiftMasterUserPassword property despite referenced in the documentation does not get properly resolved and parameter name is used instead.
Alternatively, they offer 2 workarounds whilst the issue gets fixed:
Get the 'MasterUserPassword' for Redshift from input parameter with property NoEcho set to true. NoEcho property allows you to mask the password value and you don't need to store the password in a template file. However, each time you update the stack you need to enter the password as input parameter. For your reference, below code snippet will be useful.
Second option, which is more versatile:
Define a Lambda backed Custom Resource in your template file, which queries the SSM service and returns the password to CloudFormation. In this scenario, You need to write a Custom Code for lambda function which uses AWS GetParameter API call to retrieve the value of SSM Secure String Parameter and returns the decrypted value to CloudFormation.
Other supported properties for dynamic reference seem to work fine.
State Manager Documents allow us to define input parameters of type StringList. How can we enumerate each value in a StringList within the document definition?
Eg, imagine a StringList input parameter that defined a list of commands to run. How could we create a new aws:runShellScript action for each command in the list?
The pseudo-document below shows what I'm trying to achieve - creating a a new action for each value in a StringList.
schemaVersion: "2.2"
description: "Updates services configured for the current role"
parameters:
ListOfCommands:
type: "StringList"
description: "A list of commands to execute"
mainSteps:
/* For $C in ListOfCommands: */
- action: "aws:runShellScript"
name: "InstallConsul"
inputs:
runCommand:
- "{{$C}}"
According to AWS support, this is not currently possible. There is no way to enumerate any values in a StringList within the document itself.
I am running a AWS CloudFormation stack that takes in some parameters and launches EC2 instances along with other AWS resources. The parameters are fed into the user data of the EC2 instance and based on that changes are made dynamically to the web application residing on the EC2 instance.
UserData:
Fn::Base64:
Fn::Join:
- ""
-
- "#!/bin/bash \n"
- "sh website-conf/website_mysql_config.sh "
- " -c \""
-
Ref: "CompanyName"
As shown in the example above, CompanyName is one of the many parameters passed to the userdata script. The problem is, when any one or multiple of parameters are updated, CloudFormation does not detect that and instead throws this error.
So, in order to update the stack, I have to edit the stack and make changes to the ASG so that CloudFormation 'sees' the changes and executes the stack update.
Is there a way to force CFN to update the stack when the parameters are updated?
CloudFormation will not update the stack unless there is a change in properties of the resources already created in the stack.
For example:
Consider I have a simple template to create a database where I need to pass 2 parameters:
db-name
region
Assume that I am using db-name passing it as value to DBInstanceIdentifier.
Also assume that I am not using the input parameter region for any purpose in creation of resources (or its properties) of the stack in any way.It is more of a dummy parameter I keep for readability purpose.
I passed (TEST-DB1, us-east-1) as input parameters to the CloudFormation template and successfully created the resources.
Scenario-1:
Now if I update the stack(still using the existing template) and just change the input parameters to (TEST-DB2, us-east-1). ie: changing just the db-name and not the region. Then CloudFormation will detect that, this parameter update, results in change in properties of running resource(s) of the stack and will compute and display the modifications as a change set.
Scenario-2:
Suppose I make another update(still using the existing template) property and just change the input parameters to (TEST-DB1, us-east-2). ie: changing just the region and not the db-name. Then CloudFormation will detect that, this parameter update, result in NO change in properties of running resource(s) of the stack will show the Error creating change set.
Bottomline:
Your change in input parameter must result in an update/replacement of any resources(or its attributes like security-groups,port etc..) of the stack. Then AWS CloudFormation will display them as Change Sets for your review. Also, the method (update or replacement) AWS CloudFormation uses depends on which property you update for a given resource type.
Your parameter "CompanyName" is not making any changes to the running
resources of the stack. Hence it is reporting as Error creating
change set. You need to use it to create any resource/resource properties of the stack. Then CloudFormation will detect the change-sets when you modify it. The same applies for any other input-parameters which you use.
Use the AWS CLI Update-Stack command. If you use the AWS CLI you can inject parameters into your stack so any change to any of the parameters result in a new stack. I do this myself to inject the Git/version commit ID into UserData so simply committing changes to the stack's JSON/Yaml to Git will allow stack updates. Any change to the parameters file will allow stack updates, even just a comment. I reference my Git commit ID in UserData the same way you are referencing Ref:CompanyName so when I change the Git commit ID the userData section is updated on stack updates.
Update Stack Command
aws cloudformation update-stack --stack-name MyStack --template-body file:///Users/Documents/Git/project/cloudformation/stack.json --parameters file:///Users/Documents/Git/project/cloudformation/parameters/stack-parameters.dev.json --capabilities CAPABILITY_IAM
Process
With this approach you make your parameters changes to the parameters json or yaml file then check it into version control. Now if you use a build server you can update your stack by checking out master and just running that one line above. Using AWS CodeBuild makes this easy so you don't need jenkins.
The answer of your problem is already answered with this state, CloudFormation will not update the stack unless there is a change in properties of the resources already created in the stack.
And for the answer for your question, please check the explanation below.
There is a way to force Cloudformation to update the stack using the AWS::CloudFormation::Init.
By using cfn-init, each instance can update itself when it detect the change that made by AWS::CloudFormation::Init in metadata.
There is a concept that we must understand first, that is the difference between UserData and metadata, at least under the AWS::CloudFormation::Init case.
Userdata: Will be only called once when the instance is being launch for the first time (this including update that need the instance to be replaced). So, if you update the stack (not creating a new one), even if you change the parameter value, it won't change anything if you call the parameter under UserData.
Metadata: Can be updated anytime. To make it works, you have to make sure that the daemon that detect the metadata changed is running (the daemon is called the cfn-hup)
If you already use the Metadata and AWS::CloudFormation::Init, the data is not immediately being updated. As far I know, here is the condition the data to be change after change the Metadata value.
Reboot the instance
Run cfn-init command again with it's parameter
Waiting about 15 minutes, because the daemon to check the change in Metadata is checking the change once in 15 minutes.