Ansible Cloudformation, how to not break if Resource already exists? - amazon-web-services

I have the following AWS Cloudformation config, which sets up S3, Repositories.
When I run it via an ansible playbook, on the second time running the playbook this happens
AWS::ECR::Repository Repository CREATE_FAILED: production-app-name already exists
etc
How can I make it so that when this is ran multiple times, it will keep the existing s3 and repository instead of just blowing up? (I had assumed the param "DeletionPolicy": "Retain", would do this)
What I'd like to achieve:
If i run this 100x, I want the same resource state as it was after run #1. I do not want any resources deleted/wiped of any data.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Pre-reqs for Elastic Beanstalk application",
"Parameters": {
"BucketName": {
"Type": "String",
"Description": "S3 Bucket name"
},
"RepositoryName": {
"Type": "String",
"Description": "ECR Repository name"
}
},
"Resources": {
"Bucket": {
"Type": "AWS::S3::Bucket",
"DeletionPolicy": "Retain",
"Properties": {
"BucketName": { "Fn::Join": [ "-", [
{ "Ref": "BucketName" },
{ "Ref": "AWS::Region" }
]]}
}
},
"Repository": {
"Type": "AWS::ECR::Repository",
"DeletionPolicy": "Retain",
"Properties": {
"RepositoryName": { "Ref": "RepositoryName" }
}
}
},
"Outputs": {
"S3Bucket": {
"Description": "Full S3 Bucket name",
"Value": { "Ref": "Bucket" }
},
"Repository": {
"Description": "ECR Repo",
"Value": { "Fn::Join": [ "/", [
{
"Fn::Join": [ ".", [
{ "Ref": "AWS::AccountId" },
"dkr",
"ecr",
{ "Ref": "AWS::Region" },
"amazonaws.com"
]]
},
{ "Ref": "Repository" }
]]}
}
}
}
edit:
DB with similar issue when ran twice
{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"DBPassword": {
"MinLength": "8",
"NoEcho": true,
"Type": "String"
},
"Environment": {
"MinLength": "1",
"Type": "String"
},
"DBName": {
"Type": "String",
"Description": "DBName"
},
"DBInstanceIdentifier": {
"Type": "String",
"Description": "DBInstanceIdentifier"
},
"DBPort": {
"Type": "String",
"Description": "DBPort"
},
"DBUsername": {
"Type": "String",
"Description": "DBName"
}
},
"Outputs": {
"Url": {
"Value": {
"Fn::Sub": "postgres://${DBUsername}:${DBPassword}#${Instance.Endpoint.Address}:${Instance.Endpoint.Port}/${DBName}"
}
}
},
"Resources": {
"Instance": {
"Type": "AWS::RDS::DBInstance",
"DeletionPolicy": "Retain",
"Properties": {
"AllocatedStorage": "10",
"DBInstanceClass": "db.t2.micro",
"DBInstanceIdentifier": {"Ref": "DBInstanceIdentifier"},
"DBName": {
"Ref": "DBName"
},
"Engine": "postgres",
"EngineVersion": "9.6.6",
"MasterUsername": {
"Ref": "DBUsername"
},
"MasterUserPassword": {
"Ref": "DBPassword"
},
"MultiAZ": "false",
"Port": {
"Ref": "DBPort"
},
"PubliclyAccessible": "false",
"StorageType": "gp2"
}
}
}
}

The field RepositoryName in AWS::ECR::Repository is actually not required and I would advise against specifying one. By letting CloudFormation dynamically assign a unique name to the repository you'll avoid collision.
If you later want to use the repository name, for exemple: in a task definition, you can use the "Ref" function like so { "Ref": "Repository" } to extract the unique name generated by CloudFormation.
As for the issue with the RDS instance, tt comes down to the same problem of hardcoding resources name.
Using retain will keep the resource alive but it will no longer be managed by CloudFormation which is a big problem.
Just make sure when doing updates to never modify a parameter that require a resource "replacement". The documentation always states what kind of update a parameter change will incur.
Image taken from (here)
If you really need to change a parameter that requires a replacement. Create a new resource with the adapter parameters, migrate whatever data you had in the database or ECR repository, then remove the old resource from the template. If you don't need to migrate anything, make sure you don't have hardcoded names and let CloudFormation perform the replacement.

Related

Why does my subnet and VPC show side by side on CloudFormation

I've been trying to figure out why my VPC and subnet show side by side instead of the subnet inside of the VPC? (I used Atom to generate this.)
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "vpc",
"Metadata": {
},
"Parameters": {"siggyVpcCidr": {
"Description": "vpc cidr",
"Type": "String",
"Default": "10.0.0.0/16"
},
"siggySubnetCidr": {
"Description": "cidr for the subnet",
"Type": "String",
"Default": "10.0.1.0/2"
},
"Subnet1Az": {
"Description": "AZ for siggySubnetCidr",
"Type": "AWS::EC2::AvailabilityZone::Name"
}
},
"Mappings": {
},
"Conditions": {
},
"Resources": {
"siggyVpc": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": { "Ref": "siggyVpcCidr" },
"Tags": [{ "Key": "Name", "Value": "siggyVpc" }]
}
},
"siggyIgw": {
"Type": "AWS::EC2::InternetGateway",
"Properties": {
"Tags": [{ "Key": "Name", "Value": "siggyIgw1" }]
}
},
"AttachGateway": {
"Type": "AWS::EC2::VPCGatewayAttachment",
"Properties": {
"VpcId": { "Ref": "siggyVpc" },
"InternetGatewayId": { "Ref": "siggyIgw" }
}
},
"SubnetSiggy": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"AvailabilityZone": { "Ref": "Subnet1Az" },
"VpcId": { "Ref": "siggyVpc" },
"CidrBlock": { "Ref": "siggySubnetCidr" },
"Tags": [{ "Key": "Name", "Value": "siggySubnetCidr" }]
}
}
},
"Outputs": {
}
}
They are separate resources. CloudFormation templates arrange resources in a flat array. This is pretty much true of most resources. Some resources can be implicitly defined when creating resources, but that probably won't be reflected with an export where you create a template from existing resources.
You would need to inspect the VpcId property to determine the VPC to which the subnet belongs.

Import s3 bucket from one stack to other stack

I have created S3 Bucket with deletepolicy retain using cloud formation, I Have exported the created bucket using Export in outputs in cloudformation.
Now I want to use the same s3 bucket in another stack using import
Cloud formation for s3:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Creates an S3 bucket to be used for static content/website hosting.",
"Parameters": {
"AssetInsightId": {
"Description": "Asset Insight ID",
"Type": "String",
"Default": "206153"
},
"ResourceOwner": {
"Description": "tr:resource-owner",
"Type": "String",
"Default": "####"
},
"EnvironmentType": {
"Description": "tr:environment-type",
"Default": "preprod",
"Type": "String",
"AllowedValues": ["preprod", "prod"],
"ConstraintDescription": "must specify preprod, prod."
}
},
"Resources": {
"S3Bucket": {
"Type": "AWS::S3::Bucket",
"DeletionPolicy": "Retain",
"Properties": {
"BucketName": {
"Fn::Sub": "a${AssetInsightId}-s3bucket-${EnvironmentType}"
},
"Tags": [{
"Key": "tr:application-asset-insight-id",
"Value": {
"Fn::Sub": "${AssetInsightId}"
}
}, {
"Key": "tr:environment-type",
"Value": {
"Fn::Sub": "${EnvironmentType}"
}
}
]
}
}
},
"Outputs": {
"S3Bucket": {
"Description": "Information about the value",
"Description": "Name of the S3 Resource Bucket",
"Value": "!Ref S3Bucket",
"Export": {
"Name": "ExportS3Bucket"
}
}
}
}
cloud formation to use created s3 bucket from another template with import
Second template :
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Creates an S3 apigateway to be used for static content/website hosting.",
"Parameters": {
"AssetInsightId": {
"Description": "Asset Insight ID",
"Type": "String",
"Default": "206153"
},
"ResourceOwner": {
"Description": "tr:resource-owner",
"Type": "String",
"Default": "swathi.koochi#thomsonreuters.com"
},
"EnvironmentType": {
"Description": "tr:environment-type",
"Default": "preprod",
"Type": "String",
"AllowedValues": ["preprod", "prod"],
"ConstraintDescription": "must specify preprod, prod."
},
"endpointConfiguration": {
"Description": "tr:endpoint-configuration",
"Default": "REGIONAL",
"Type": "String",
"AllowedValues": ["REGIONAL", "EDGE"],
"ConstraintDescription": "must specify REGIONAL, EDGE."
}
},
"Resources": {
"S3BucketImport": {
"Type": "AWS::S3::Bucket",
"Properties": {
"BucketName": {"Fn::ImportValue" : "ExportS3Bucket"}
}
},
"APIGateWayRestResourceRestApi": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Name": "MyAPI",
"Description": "API Gateway rest api with cloud formation",
"EndpointConfiguration": {
"Types": [{
"Ref": "endpointConfiguration"
}
]
}
}
},
"APIGateWayResource": {
"Type": "AWS::ApiGateway::Resource",
"Properties": {
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
},
"ParentId": {
"Fn::GetAtt": ["APIGateWayRestResourceRestApi", "RootResourceId"]
},
"PathPart": "test"
}
},
"APIGatewayPostMethod": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"AuthorizationType": "NONE",
"HttpMethod": "POST",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Sub": "arn:aws:apigateway:us-east-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-2:861756181523:function:GreetingLambda/invocations"
}
},
"MethodResponses": [{
"ResponseModels": {
"application/json": {
"Ref": "PostMethodResponse"
}
},
"StatusCode": 200
}
],
"ResourceId": {
"Ref": "APIGateWayResource"
},
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
}
}
},
"PostMethodResponse": {
"Type": "AWS::ApiGateway::Model",
"Properties": {
"ContentType": "application/json",
"Name": "PostMethodResponse",
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
},
"Schema": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "PostMethodResponse",
"type": "object",
"properties": {
"Email": {
"type": "string"
}
}
}
}
},
"RestApiDeployment": {
"DependsOn": "APIGatewayPostMethod",
"Type": "AWS::ApiGateway::Deployment",
"Properties": {
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
}
}
},
"RestAPIStage": {
"Type": "AWS::ApiGateway::Stage",
"Properties": {
"DeploymentId": {
"Ref": "RestApiDeployment"
},
"MethodSettings": [{
"DataTraceEnabled": true,
"HttpMethod": "*",
"ResourcePath": "/*"
}
],
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
},
"StageName": "Latest"
}
},
"APIGateWayDomainName": {
"Type": "AWS::ApiGateway::DomainName",
"Properties": {
"CertificateArn": {
"Ref": "myCertificate"
},
"DomainName": {
"Fn::Join": [".", [{
"Ref": "AssetInsightId"
}, {
"Ref": "EnvironmentType"
}, "api"]]
},
"EndpointConfiguration": {
"Types": [{
"Ref": "endpointConfiguration"
}
]
}
}
},
"myCertificate": {
"Type": "AWS::CertificateManager::Certificate",
"Properties": {
"DomainName": {
"Fn::Join": [".", [{
"Ref": "AssetInsightId"
}, {
"Ref": "EnvironmentType"
}, "api"]]
}
}
}
}
}
when I/m trying to import using Import Value, I'm getting error saying
S3BucketImport
CREATE_FAILED Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 9387EBE0E472E559; S3 Extended Request ID: o8EbE20IOoUgEMwXc7xVjuoyQT03L/nnQ7AsC94Ff1S/PkE100Imeyclf1BxYeM0avuYjDWILxA=)
As #Jarmod correctly pointed out,
In your first template, export the s3 bucket name using { "Ref" : ",S3Bucket" }
In your second template, you don't have to create the bucket again.you can use the exported value from the first template if you want to refer the bucket name from resources. But i don't see any of the resources in the second template refer the S3 bucket name.

Cloud Formation function doesn't assign value to value for key within a Tag

I have the following Cloud Formation template to create a VPC. The VPC name is generated based off of the region and the environment that the template was created in. The VPC creates without any issues, and running aws cloud formation validate-template --template-url https://foo.template doesn't complaing about any of the syntax.
I would expect the VPC to be named:
vpc-uw1-d-fs
What happens instead is the VPC is left with an empty name and the Name tag has an empty value. Am I not using the function correctly? If I remove the Fn::FindInMap function usage, I get the name generated - it's just missing the environment mapped value.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "VPC for a new environment to use.",
"Parameters": {
"EnvironmentName": {
"Description": "Name of the environment that this VPC will be used for.",
"Type": "String",
"MinLength": "2",
"MaxLength": "20",
"AllowedPattern": "[a-zA-Z]*",
"AllowedValues": [
"Development",
"QA",
"Test",
"Production"
]
}
},
"Resources": {
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"EnableDnsSupport": false,
"EnableDnsHostnames": false,
"InstanceTenancy": "default",
"Tags": [ {
"Key": "Name",
"Value": {
"Fn::Join": [
"-",
[
"vpc",
{ "Ref": "AWS::Region" },
{ "Fn::FindInMap": [
"EnvironmentMap", { "Ref": "EnvironmentName" }, "AbbreviatedName"
]},
"fs"
]
]
}
}]
}
}
},
"Mappings": {
"RegionMap": {
"us-east-1": {
"regionName": "ue1"
},
"us-west-1": {
"regionName": "uw1"
}
},
"EnvironmentMap": {
"Development": {
"AbbreviatedName": "d"
},
"QA": {
"AbbreviatedName": "qa"
},
"Test": {
"AbbreviatedName": "t"
},
"Production": {
"AbbreviatedName": "p"
}
}
},
"Outputs": {
}
}
Your template is working perfectly fine for me.
I ran it in the ap-southeast-2 region and it produced the tag:
Name: vpc-ap-southeast-2-d-fs
(The RegionMap is not used in the template given.)

How to pass resources from parent to child stack in nested cloudformation?

My nested stack requires resources located in my main stack. eg: A lambda function in the nested stack requiring DB configs
"ProjectsusgetProjectFinancialsLF": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": "dev",
"S3Key": "test-lamda.zip",
"S3ObjectVersion": "9eNYbcI5EOuuut9igX2xpgbGCtKD1D4K"
},
"Environment": {
"Variables": {
"MYSQLDB_USER": {
"Ref": "DBuser"
},
"MYSQLDB_HOST": {
"Fn::GetAtt": [
"testDB",
"Endpoint.Address"
]
},
"MYSQLDB_DATABASE": {
"Ref": "DBname"
},
"MYSQLDB_PASSWORD": {
"Ref": "DBpass"
}
}
},
"Description": "A get project financials function",
"FunctionName": {
"Fn::Join": [
"-",
[
{
"Ref": "EnvType"
},
"getProjectFinancials"
]
]
},
"Handler": "src/controllers/projects.geFinancials",
"Role": {
"Fn::GetAtt": [
"LambdaExecutionRole",
"Arn"
]
},
"Runtime": "nodejs6.10"
},
"DependsOn": [
"LambdaExecutionRole"
]
},
So I am passing the required params from my main stack to the nested using parameters :
"FinancialStack": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3.amazonaws.com/dev/child-cft.json",
"TimeoutInMinutes": "5",
"Parameters": {
"DBuser": {
"Ref": "DBuser",
"Type": "String"
},
"epmoliteDB": {
"Ref": "testDB",
"Type": "AWS::RDS::DBInstance"
},
"DBname": {
"Ref": "DBname",
"Type": "String"
},
"DBPass": {
"Ref": "DBpass",
"Type": "String"
},
"EnvType": {
"Ref": "EnvType",
"Type": "String"
},
"LambdaExecutionRole": {
"Ref": "LambdaExecutionRole",
"Type": "AWS::IAM::Role"
},
"ApiGatewayRestApi": {
"Ref": "ApiGatewayRestApi",
"Type": "AWS::ApiGateway::RestApi"
}
}
}
}
And this is how I am receiving them in my nested stack :
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "AWS CloudFormation to generate testone shot deployment",
"Parameters": {
"DBuser": {
"Ref": "DBuser",
"Type": "String"
},
"epmoliteDB": {
"Ref": "testDB",
"Type": "AWS::RDS::DBInstance"
},
"DBname": {
"Ref": "DBname",
"Type": "String"
},
"DBPass": {
"Ref": "DBpass",
"Type": "String"
},
"EnvType": {
"Ref": "EnvType",
"Type": "String"
},
"LambdaExecutionRole": {
"Ref": "LambdaExecutionRole",
"Type": "AWS::IAM::Role"
},
"ApiGatewayRestApi": {
"Ref": "ApiGatewayRestApi",
"Type": "AWS::ApiGateway::RestApi"
}
},
Yet when I run the cloudformation script it fails to create the nested stack. Am I passing the resources incorrectly from my main stack to the nested stack?
Should I instead export the parameters in the output of the main stack and import them in my nested stack using "Fn:ImportValue" ?
There's many things preventing these templates to work.
Let's start with the nested stack template. You can't to use the "Ref" intrinsic function inside the input parameters. Just the type is enough. Also not everything is supported as parameter type (here's the list), for exemple, "Type": "AWS::ApiGateway::RestApi" is not a valid parameter type. When something is not directly supported, just use the "String" type. In fact, for nested stacks you can make your life easier and just use the "String" type.
Next thing to fix is the AWS::CloudFormation::Stack resource block. Here you used the "Type" properties to for each passed "Parameters" but you actually can't specify the type there. It's the job of the nested template to dictate which type of input it's expecting.
I highly recommend you to take the time to read the CloudFormation documentation. Even better, read some examples made by AWS. Here's a good example of nested stacks, just have a look at master.yaml.

AWS - Merge cloudformation stack with a VPC

I am trying to create a stack in AWS CloudFormation, My template basically consists of Ec2 instance, RDS instance for DB (MySQL engine) and a S3 bucket. but, its throwing error stating (db.t2.micro) this DB instance class cannot be created without a VPC, then I changed the DB instance class to (db.m1.small) again am getting same error. I even created a VPC too, but not sure how do I create my stack within the VPC which I created. I work in my company's AWS account. where already few other VPCs are available.
Thanks in advance :)
Modified the JSON script after getting answers. This script is in working condition and could create stack. TESTED!
Updated Code
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"DBSubnetGroup": {
"Type": "AWS::RDS::DBSubnetGroup",
"Properties": {
"DBSubnetGroupDescription": "This subnet belongs to Abdul's VPC",
"DBSubnetGroupName": "somename",
"SubnetIds": [
"subnet-f6b15491",
"subnet-b154569e"
]
}
},
"DB": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
"AllocatedStorage": "5",
"StorageType": "gp2",
"DBInstanceClass": "db.m1.small",
"DBName": "wordpress",
"Engine": "MySQL",
"MasterUsername": "wordpress",
"MasterUserPassword": "Word12345",
"DBSubnetGroupName": {
"Ref": "DBSubnetGroup"
}
}
},
"EC2": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-c481fad3",
"InstanceType": "t2.micro",
"SubnetId": "subnet-b154569e"
}
},
"S3": {
"Type": "AWS::S3::Bucket",
"Properties": {
"BucketName": "wp-abdultestbuck"
}
}
}
}
You need to create an AWS::RDS::DBSubnetGroup and then reference in the AWS::RDS::DBInstance
{
"Resources": {
"DBSubnetGroup": {
"Type": "AWS::RDS::DBSubnetGroup",
"Properties": {
"DBSubnetGroupDescription": "",
"SubnetIds": [ "<Subnet ID 1","<Subnet ID 2>" ],
}
},
"DB": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
....
"DBSubnetGroupName": { "Ref": "DBSubnetGroup" }
}
},
"EC2": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-c481fad3",
"InstanceType": "t2.micro",
"SubnetId": "<SubnetID>"
}
}
}
}