Use env variables inside cloudformation templates - amazon-web-services

i have an amplify app with multiple branches.
I have added an custom cloudformation template with amplify add custom
It looks like:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": { "env": { "Type": "String" } },
"Resources": {
"db": {
"Type": "AWS::Timestream::Database",
"Properties": { "DatabaseName": "dev_db" }
},
"timestreamtable": {
"DependsOn": "db",
"Type": "AWS::Timestream::Table",
"Properties": {
"DatabaseName": "dev_db",
"TableName": "avg_16_4h",
"MagneticStoreWriteProperties": { "EnableMagneticStoreWrites": true },
"RetentionProperties": {
"MemoryStoreRetentionPeriodInHours": "8640",
"MagneticStoreRetentionPeriodInDays": "1825"
}
}
}
},
"Outputs": {},
"Description": "{\"createdOn\":\"Windows\",\"createdBy\":\"Amplify\",\"createdWith\":\"8.3.1\",\"stackType\":\"custom-customCloudformation\",\"metadata\":{}}"
}
You can see there is a field called DatabaseName. In my amplify app i have written an env variable named TIMESTREAM_DB and i want to use it inside of this cloudformation file.
Is this possible or do i need to write it all by hand in it?

Templates cannot access arbitrary env vars. Instead, CloudFormation injects deploy-time values into a template with Parameters.
Amplify helpfully adds the env variable as a parameter. A la the Amplify docs, use the env value as the AWS::Timestream::Database name suffix:
"DatabaseName": "Fn::Join": [ "", [ "my-timestream-db-name-", { "Ref": "env" } ] ]
The AWS::Timestream::Table resource also requires a DatabaseName parameter. You could repeat the above, but it's more DRY to get the name via the Database's Ref:
"DatabaseName": { "Ref" : "db" }

Related

How we can use same Tags into two AWS::DynamoDB::Table with in Cloudformation Template

I'm trying to create Amazon DynamoDB tables using Cloud Formation Template. So my question is can I Used same Tags in multiple Tables using Ref.
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"Status": {
"Type": "AWS::DynamoDB::Table",
"Properties": {
"AttributeDefinitions": [
{
"AttributeName": "SId",
"AttributeType": "S"
}
],
"KeySchema": [
{
"AttributeName": "SId",
"KeyType": "HASH"
}
],
"ProvisionedThroughput": {
"ReadCapacityUnits": "1",
"WriteCapacityUnits": "1"
},
"TableName": "Statuscf",
"Tags": [
{
"Key": "Application",
"Value": "BFMS"
},
{
"Key": "Name",
"Value": "EventSourcingDataStore"
}
]
}
},
"BMSHSData": {
"Type": "AWS::DynamoDB::Table",
"Properties": {
"TableName": "Billing.FmsDatacf",
"Tags": [{"Ref":"/Status/Tags"}]
}
}
}
Please suggest me how I can used same tags in another table. I am using Like this "Tags": [{"Ref":"/Status/Tags"}].
The only way to do this using plain CloudFormation is by copy-and-paste. So you have to replicate your tags for all tables "manually". The only automated solution would be through development of CloduFormation macro or custom resource. Yet another choice could be through the use of nested stacks.
To resolved this problem, Just need to pass Tags Properties in the Parameter Section of CF.
Then used those tags in DynamoDB like this

How to parse a CloudFormation template programmatically and resolve field?

I want to extract a particular field from a CloudFormation template. Specifically, I am creating a stack using CDK and within it there is a StepFunction resource whose definition inside the template looks like this:
"MyStateMachineE7CD0EAE": {
"Type": "AWS::StepFunctions::StateMachine",
"Properties": {
"RoleArn": {
"Fn::GetAtt": [
"MyStateMachineRoleEC8990B2",
"Arn"
]
},
"DefinitionString": {
"Fn::Join": [
"",
[
"{\"StartAt\": ...part of state machine definition here... ,\"Resource\":\"arn:",
{
"Ref": "AWS::Partition"
},
":states:::lambda:invoke\",\"Parameters\":{\"FunctionName\":\"",
{
"Fn::GetAtt": [
"SingletonLambda2a860ff582bf4d828a504282814af94c7CEF2D65",
"Arn"
]
},
"...remainder of state machine definition here...}"
]
]
},
"LoggingConfiguration": {
"Destinations": [
{
"CloudWatchLogsLogGroup": {
"LogGroupArn": {
"Fn::GetAtt": [
"MyStatemachineLogGroup2DEF8C9E",
"Arn"
]
}
}
}
],
"IncludeExecutionData": true,
"Level": "ALL"
},
"StateMachineType": "STANDARD"
},
"DependsOn": [
"MyStateMachineRoleDefaultPolicy4C064A65",
"MyStateMachineRoleEC8990B2"
],
"Metadata": {
"aws:cdk:path": "some/path/here"
}
}
The DefinitionString is made up of a call to Fn:Join, when there is an array of strings, some of which are references to other CloudFormation resources. There can be many such references, depending on how complicated the StateMachine definition is.
I want to be able to use StepFunctions Local in order to test the flow of data through the StepFunctions state machine, and in particular to check the InputPath and ResultSelector data input and output as part of some integration tests of the state machine as a whole. The StepFunction Local CLI tool requires as input a single string, and that string is the evaluation of the DefinitionString field.
Is there a way of extracting the field from the CloudFormation template, perhaps by a CloudFormation client API? I'd rather not have to try and parse the JSON directly unless I absolutely have to.

AWS Cloudformation "include" transform error

I have some template and other partial definition I'd like to include in main template definition. The sample is below (main template).
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "",
"Parameters": {
"Environment": {
"Type": "String",
"Description": "Specify Environment: prod | dev ",
"AllowedValues": [ "prod", "dev" ],
"Default": "dev"
}
},
"Transform": {
"Name": "AWS::Include",
"Parameters": {
"Location": "s3://some-s3-local-bucket/part-1.json"
}
},
"Resources": {
},
"Outputs": {
}
}
Below is a definition of the part to include in main template
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "",
"Resources": {
"hellobucket": {
"Type": "AWS::S3::Bucket",
"Properties": {
"BucketName": { "Fn::Sub": "testbucket-${Environment}" }
}
}
},
"Outputs": {
}
}
When I try to create stack based on such definitions I receive strange error like "Template parameters modified by transform". I don't know/see any reason any parameter could be considered as "modified".
I don't want to create many "nested" stacks, because there is a limit on aws in number of stacks I can create so the goal is to split stack definition into many (well managable) smaller files and based on them to create ONE stack with all the related resources.
How to properly decompose bigger stack definition into smaller files?
I haven't done this before but it might be because you are using the transform to pull in a template that creates an s3 bucket BUT the template you are pulling into the original one has all it's parameter fields etc. empty. I think this is what the error message - Template parameters modified by transform - is relating to. Try removing the empty parameters entry from the S3 template to see if that helps.

AWS ECS - Unable to specify service name in cloudformation template

I am trying to create a AWS - ECS service using cloudformation template
"service": {
"ServiceName": "XXX",
"Type": "AWS::ECS::Service",
"DependsOn": [
"AutoScalingGroup"
],
"Properties": {
"Cluster": {
"Ref": "ECSCluster"
},
"DesiredCount": "1",
"TaskDefinition": {
"Ref": "taskdefinition"
}
}
},
But I am getting an error.
Failed: Invalid template resource property 'ServiceName'
I had same problem when using Name/serviceName. I can see serviceName is a parameter based on docs. But couldn't figure out why it fails. It works if I don't specify name. But I need to specify name so that I can use the same name in a different system that updates the service.
Could you please help?
This is slightly confusing, but the service name is set by the name of the resource you create. There is no ServiceName or Name property. The following will create an ECS service with the name MyService.
"MyService": {
"Type": "AWS::ECS::Service",
"DependsOn": [
"AutoScalingGroup"
],
"Properties": {
"Cluster": {
"Ref": "ECSCluster"
},
"DesiredCount": "1",
"TaskDefinition": {
"Ref": "taskdefinition"
}
}
}
Obviously if you refer to your service within the CloudFormation template you will also need to update your references.

How to check if specific resource already exists in CloudFormation script

I am using cloudformation to create a stack which inlcudes an autoscaled ec2 instance and an S3 bucket. For the S3 bucket I have DeletionPolicy set to Retain, which works fine, until I want to rerun my cloudformation script again. Since on previous runs, the script created the S3 bucket, it fails on subsequent runs saying my S3 bucket already exists. None of the other resources of course get created as well. My question is how do I check if my S3 bucket exists first inside the cloudformation script, and if it does, then skip creating that resources. I've looked in conditions in the AWS, but it seems all parameter based, I have yet to find a function which checks from existing resources.
There is no obvious way to do this, unless you create the template dynamically with an explicit check. Stacks created from the same template are independent entities, and if you create a stack that contains a bucket, delete the stack while retaining the bucket, and then create a new stack (even one with the same name), there is no connection between this new stack and the bucket created as part of the previous stack.
If you want to use the same S3 bucket for multiple stacks (even if only one of them exists at a time), that bucket does not really belong in the stack - it would make more sense to create the bucket in a separate stack, using a separate template (putting the bucket URL in the "Outputs" section), and then referencing it from your original stack using a parameter.
Update November 2019:
There is a possible alternative now. On Nov 13th AWS launched CloudFormation Resource Import. With that feature you can now creating a stack from existing resources. Currently not many resource types are supported by this feature, but S3 buckets are.
In your case you'd have to do it in two steps:
Create a template that only contains the preexisting S3 bucket using the "Create stack" > "With existing resources (import resources)" (this is the --change-set-type IMPORT flag in the CLI) (see docs)
Update the the template to include all resources that don't already exist.
As they note in their documentation; this feature is very versatile. So it opens up a lot of possibilities. See docs for more info.
Using cloudformation you can use Conditions
I created an input parameter "ShouldCreateBucketInputParameter" and then using CLI you just need to set "true" or "false"
Cloudformation json file:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Description": "",
"Parameters": {
"ShouldCreateBucketInputParameter": {
"Type": "String",
"AllowedValues": [
"true",
"false"
],
"Description": "If true then the S3 bucket that will be proxied will be created with the CloudFormation stack."
}
},
"Conditions": {
"CreateS3Bucket": {
"Fn::Equals": [
{
"Ref": "ShouldCreateBucketInputParameter"
},
"true"
]
}
},
"Resources": {
"SerialNumberBucketResource": {
"Type": "AWS::S3::Bucket",
"Condition": "CreateS3Bucket",
"Properties": {
"AccessControl": "Private"
}
}
},
"Outputs": {}
}
And then (I am using CLI do deploy the stack)
aws cloudformation deploy --template ./s3BucketWithCondition.json --stack-name bucket-stack --parameter-overrides ShouldCreateBucketInputParameter="true" S3BucketNameInputParameter="BucketName22211"
Just add an input parameter to the CloudFormation template to indicate that an existing bucket should be used.... unless you don't already know at the time when you are going to use the template? Then you can either add a new resource or not based on the parameter value.
If you do updates, (potentially of stacks within stacks aka Nested Stacks), the unchanged parts don't get updated.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-stack.html?icmpid=docs_cfn_console_designer
You can then set policies as mentioned to prevent deletion. [remember 'cancel update' permissions for rollbacks]
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/protect-stack-resources.html
There is also Cross-Stack Output to be aware of by adding Export Names to the Stack Outputs.
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html
Walkthrough...
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/walkthrough-crossstackref.html
Then you need to use Fn::ImportValue ...
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html
It implies one could use a network stack name parameter.
Unfortunately you get an error like this when you try them in Conditions.
Template validation error: Template error: Cannot use Fn::ImportValue
in Conditions.
Or in the Parameters?
Template validation error: Template format error: Every Default member
must be a string.
Also this can happen while trying...
Template format error: Output ExportOut is malformed. The Name field
of Export must not depend on any resources, imported values, or
Fn::GetAZs.
So you can't stop it making the existing resource again from the same file. Only when putting it into another stack and using the export import reference.
But if you separate the two then there is a dependency that will stop and rollback for instance a dependency's deletion, thanks to the reference via the ImportValue function.
Example Given here is:
First Make a Group Template
{
"AWSTemplateFormatVersion": "2010-09-09",
"Metadata": {
"AWS::CloudFormation::Designer": {
"6927bf3d-85ec-449d-8ee1-f3e1804d78f7": {
"size": {
"width": 60,
"height": 60
},
"position": {
"x": -390,
"y": 130
},
"z": 0,
"embeds": []
},
"6fe3a2b8-16a1-4ce0-b412-4d4f87e9c54c": {
"source": {
"id": "ac295134-9e38-4425-8d20-2c50ef0d51b3"
},
"target": {
"id": "6927bf3d-85ec-449d-8ee1-f3e1804d78f7"
},
"z": 1
}
}
},
"Resources": {
"TestGroup": {
"Type": "AWS::IAM::Group",
"Properties": {},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "6927bf3d-85ec-449d-8ee1-f3e1804d78f7"
}
},
"Condition": ""
}
},
"Parameters": {},
"Outputs": {
"GroupNameOut": {
"Description": "The Group Name",
"Value": {
"Ref": "TestGroup"
},
"Export": {
"Name": "Exported-GroupName"
}
}
}
}
Then make a User Template that needs the group.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Metadata": {
"AWS::CloudFormation::Designer": {
"ac295134-9e38-4425-8d20-2c50ef0d51b3": {
"size": {
"width": 60,
"height": 60
},
"position": {
"x": -450,
"y": 130
},
"z": 0,
"embeds": [],
"isrelatedto": [
"6927bf3d-85ec-449d-8ee1-f3e1804d78f7"
]
},
"6fe3a2b8-16a1-4ce0-b412-4d4f87e9c54c": {
"source": {
"id": "ac295134-9e38-4425-8d20-2c50ef0d51b3"
},
"target": {
"id": "6927bf3d-85ec-449d-8ee1-f3e1804d78f7"
},
"z": 1
}
}
},
"Resources": {
"TestUser": {
"Type": "AWS::IAM::User",
"Properties": {
"UserName": {
"Ref": "UserNameParam"
},
"Groups": [
{
"Fn::ImportValue": "Exported-GroupName"
}
]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "ac295134-9e38-4425-8d20-2c50ef0d51b3"
}
}
}
},
"Parameters": {
"UserNameParam": {
"Default": "testerUser",
"Description": "Username For Test",
"Type": "String",
"MinLength": "1",
"MaxLength": "16",
"AllowedPattern": "[a-zA-Z][a-zA-Z0-9]*",
"ConstraintDescription": "must begin with a letter and contain only alphanumeric characters."
}
},
"Outputs": {
"UserNameOut": {
"Description": "The User Name",
"Value": {
"Ref": "TestUser"
}
}
}
}
You will get
No export named Exported-GroupName found. Rollback requested by user.
if running User with no Group found Exported.
You could then use the Nested stack approach.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Metadata": {
"AWS::CloudFormation::Designer": {
"66470873-b2bd-4a5a-af19-5d54b11f48ef": {
"size": {
"width": 60,
"height": 60
},
"position": {
"x": -815,
"y": 169
},
"z": 0,
"embeds": []
},
"ed1de011-f1bb-4788-b63e-dcf5494d10d1": {
"size": {
"width": 60,
"height": 60
},
"position": {
"x": -710,
"y": 170
},
"z": 0,
"dependson": [
"66470873-b2bd-4a5a-af19-5d54b11f48ef"
]
},
"c978f2d9-3fb2-4420-b255-74941f10a28a": {
"source": {
"id": "ed1de011-f1bb-4788-b63e-dcf5494d10d1"
},
"target": {
"id": "66470873-b2bd-4a5a-af19-5d54b11f48ef"
},
"z": 1
}
}
},
"Resources": {
"GroupStack": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3-us-west-2.amazonaws.com/cf-templates-x-TestGroup.json"
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "66470873-b2bd-4a5a-af19-5d54b11f48ef"
}
}
},
"UserStack": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3-us-west-2.amazonaws.com/cf-templates-x-TestUserFindsGroup.json"
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "ed1de011-f1bb-4788-b63e-dcf5494d10d1"
}
},
"DependsOn": [
"GroupStack"
]
}
}
}
Unfortunately you can still delete the User stack even though it was made by MultiStack in this example but with deletion policies and other things it just might help.
Then you are only Updating the various stacks it creates, and you won't do the Multi Stack if you're for instance reusing a Bucket.
Otherwise you'll be looking at APIs and scripts in various flavors.
If you're trying to incorporate some existing resources into CF, it is unfortunately not possible. If you just want a set of resources to be part of your template or not depending on the value of some parameters, you can use Conditions. But they don't change the nature of CF itself, and only work to determine which resources are desired, not what actions will be taken, and cannot see whether a resource exists or not beforehand.
Something not explicitly stated. If your first deployment fails, resources will be deleted unless you have at retention policy. In this case it is safe to delete the resource in question manually. Next deployment will recreate it without generating error that resource already exists.