I am trying to setup the AWS environment for my SpringBoot application which uses Postgres.
I decided to use CloudFormation to configure all the AWS components needed for my application.
Following is the cloud formation template app.json:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"S3BucketName": {
"Description": "S3 BucketName",
"Type": "String"
},
"S3FileName": {
"Description": "Name of the jar/war file",
"Type": "String"
},
"DBName":{
"Default":"mytododb",
"Description":"The database name",
"Type":"String",
"MinLength":"1",
"MaxLength":"64",
"AllowedPattern":"[a-zA-Z][a-zA-Z0-9]*",
"ConstraintDescription":"must begin with a letter and contain only alphanumeric characters."
},
"DBUser":{
"Description":"The database admin account username",
"Type":"String",
"MinLength":"1",
"MaxLength":"16",
"AllowedPattern":"[a-zA-Z][a-zA-Z0-9]*",
"ConstraintDescription":"must begin with a letter and contain only alphanumeric characters."
},
"DBPassword":{
"NoEcho":"true",
"Description":"The database admin account password",
"Type":"String",
"MinLength":"8",
"MaxLength":"41",
"AllowedPattern":"[a-zA-Z0-9]*",
"ConstraintDescription":"must contain only alphanumeric characters."
},
"DBAllocatedStorage":{
"Default":"5",
"Description":"The size of the database (Gb)",
"Type":"Number",
"MinValue":"5",
"MaxValue":"1024",
"ConstraintDescription":"must be between 5 and 1024Gb."
},
"DBInstanceClass":{
"Default":"db.t2.micro",
"Description":"The database instance type",
"Type":"String",
"ConstraintDescription":"must select a valid database instance type."
},
"MultiAZDatabase":{
"Default":"false",
"Description":"Create a multi-AZ RDS database instance",
"Type":"String",
"AllowedValues":[
"true",
"false"
],
"ConstraintDescription":"must be either true or false."
}
},
"Resources": {
"SpringBootApplication": {
"Type": "AWS::ElasticBeanstalk::Application",
"Properties": {
"Description":"Spring boot and elastic beanstalk"
}
},
"SpringBootApplicationVersion": {
"Type": "AWS::ElasticBeanstalk::ApplicationVersion",
"Properties": {
"ApplicationName":{"Ref":"SpringBootApplication"},
"SourceBundle": {
"S3Bucket": {
"Ref": "S3BucketName"
},
"S3Key": {
"Ref": "S3FileName"
}
}
}
},
"SpringBootBeanStalkConfigurationTemplate": {
"Type": "AWS::ElasticBeanstalk::ConfigurationTemplate",
"Properties": {
"ApplicationName": {"Ref":"SpringBootApplication"},
"Description":"A display of speed boot application",
"OptionSettings": [
{
"Namespace": "aws:rds:dbinstance",
"OptionName": "DBAllocatedStorage",
"Value": {
"Ref": "DBAllocatedStorage"
}
},
{
"Namespace": "aws:rds:dbinstance",
"OptionName": "DBDeletionPolicy",
"Value": "Delete"
},
{
"Namespace": "aws:rds:dbinstance",
"OptionName": "DBEngine",
"Value": "postgres"
},
{
"Namespace": "aws:rds:dbinstance",
"OptionName": "DBEngineVersion",
"Value": "10.4"
},
{
"Namespace": "aws:rds:dbinstance",
"OptionName": "DBInstanceClass",
"Value": { "Ref": "DBInstanceClass" }
},
{
"OptionName": "DBPassword",
"Namespace": "aws:rds:dbinstance",
"Value": { "Ref": "DBPassword" }
},
{
"Namespace": "aws:rds:dbinstance",
"OptionName": "DBUser",
"Value": { "Ref": "DBUser" }
},
{
"Namespace": "aws:rds:dbinstance",
"OptionName": "MultiAZDatabase",
"Value": { "Ref": "MultiAZDatabase" }
}
],
"SolutionStackName": "64bit Amazon Linux 2018.03 v2.9.2 running Java 8"
}
},
"SpringBootBeanstalkEnvironment": {
"Type": "AWS::ElasticBeanstalk::Environment",
"Properties": {
"ApplicationName": {"Ref":"SpringBootApplication"},
"EnvironmentName":"TodoAppEnvironment",
"TemplateName": {"Ref":"SpringBootBeanStalkConfigurationTemplate"},
"VersionLabel": {"Ref": "SpringBootApplicationVersion"}
}
}
},
"Outputs": {
"DevURL": {
"Description": "The URL of the DEV Elastic Beanstalk environment",
"Value": {
"Fn::Join": [
"",
[
{
"Fn::GetAtt": [
"SpringBootBeanstalkEnvironment",
"EndpointURL"
]
}
]
]
},
"Export": {
"Name": {
"Fn::Sub": "${AWS::StackName}-EndpointURL"
}
}
}
}
}
And the parameters file parameters.json is:
[
{
"ParameterKey": "DBUser",
"ParameterValue": "myuser"
},
{
"ParameterKey": "DBPassword",
"ParameterValue": "secret"
},
{
"ParameterKey": "S3BucketName",
"ParameterValue": "todoapp"
},
{
"ParameterKey": "S3FileName",
"ParameterValue": "todo-api-spring-boot-0.0.1.jar"
}
]
I am creating the CloudFormation stack using
aws cloudformation create-stack --template-body file://app.json --parameters file://parameters.json --stack-name=todo-stack
Problem:
The stack is getting created, however the database (RDS) instance is not getting created and when I see in ElasticBeanstalk configuration for the app in Management Console there is no Database configuration associated with the application.
My expectation/assumption is by configuring various aws:rds:dbinstance properties in OptionSettings an RDS instance will be provisioned and associates with ElasticBeanstalk application.
Is my understanding wrong or am I missing any other setting?
PS:
The idea is to use integrated RDS so that I can use RDS_HOST, RDS_PORT etc properties in my application to connect to database.
I am able to configure a separate RDS Resource and connect to it by specifying the connection parameters. But I am expecting aws:rds:dbinstance properties in OptionSettings would create the RDS and associate it to Elasticbeanstalk application. If that is not the case then what is the purpose of configuring those parameters in OptionSettings?
Ref:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options-general.html#command-options-general-rdsdbinstance
https://github.com/metabase/metabase/blob/master/bin/aws-eb-docker/cloudformation-elasticbeanstalk.json.template
RDS creation is not done due to the Namespace Approach. Basically Namespace is used to override the default properties of the resources which you have created.
So the solution is to include the RDS creation template before your "SpringBootBeanStalkConfigurationTemplate"
"Resources": {
"RDS Creation": {
{
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : String,
"AllowMajorVersionUpgrade" : Boolean,
"AssociatedRoles" : [ DBInstanceRole, ... ],
"AutoMinorVersionUpgrade" : Boolean,
"AvailabilityZone" : String,
"BackupRetentionPeriod" : Integer,
"CharacterSetName" : String,
"CopyTagsToSnapshot" : Boolean,
"DBClusterIdentifier" : String,
"DBInstanceClass" : String,
"DBInstanceIdentifier" : String,
"DBName" : String,
"DBParameterGroupName" : String,
"DBSecurityGroups" : [ String, ... ],
"DBSnapshotIdentifier" : String,
"DBSubnetGroupName" : String,
"DeleteAutomatedBackups" : Boolean,
"DeletionProtection" : Boolean,
"Domain" : String,
"DomainIAMRoleName" : String,
"EnableCloudwatchLogsExports" : [ String, ... ],
"EnableIAMDatabaseAuthentication" : Boolean,
"EnablePerformanceInsights" : Boolean,
"Engine" : String,
"EngineVersion" : String,
"Iops" : Integer,
"KmsKeyId" : String,
"LicenseModel" : String,
"MasterUsername" : String,
"MasterUserPassword" : String,
"MonitoringInterval" : Integer,
"MonitoringRoleArn" : String,
"MultiAZ" : Boolean,
"OptionGroupName" : String,
"PerformanceInsightsKMSKeyId" : String,
"PerformanceInsightsRetentionPeriod" : Integer,
"Port" : String,
"PreferredBackupWindow" : String,
"PreferredMaintenanceWindow" : String,
"ProcessorFeatures" : [ ProcessorFeature, ... ],
"PromotionTier" : Integer,
"PubliclyAccessible" : Boolean,
"SourceDBInstanceIdentifier" : String,
"SourceRegion" : String,
"StorageEncrypted" : Boolean,
"StorageType" : String,
"Tags" : [ Tag, ... ],
"Timezone" : String,
"UseDefaultProcessorFeatures" : Boolean,
"VPCSecurityGroups" : [ String, ... ]
}
}
}
}
After this, you can include the RDS parameters in your other Resources as it will be created first.
I was stumbling across the same question and found some sort of an answer here: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.managing.db.html
In that document they say:
You can configure your environment's database instance using configuration files. Use the options in the aws:rds:dbinstance namespace. The following example modifies the allocated database storage size to 100 GB.
So the aws:rds:dbinstance properties in OptionSettings are only meant to modify an existing database instance, not to create one in the first place.
Related
I am using cloud formation to create:
load balancer + target-group w/ instance(from custom AMI)
I used create template in designer and edited their default configs:
I chose Elastic Loadbalancer V2 >> target group
{
"Resources": {
"ELBV3LB29NC1": {
"Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties": {
"IpAddressType" : "ipv4",
"Name" : "quest-loadbalancer2",
"SecurityGroups" : [ "sg-036a1a54caee05af5"],
"Subnets" : [ "subnet-795d4435", "subnet-7d5a5a07" ],
}
}
}
}
// **********************
{
"Resources": {
"ELBTV1G4J8KH": {
"Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
"Properties": {
"Name": "quest-target-group-2",
"Port": 80,
"Protocol": "HTTP",
"Targets": [
{
"Id" : "i-0f5c5714bed60786b",
"Port" : 80
}
],
"TargetType": "i-0f5c5714bed60786b",
"VpcId": "vpc-a28dece9"
}
}
}
}
An EC2 Instance
{
"Resources": {
"EC2I5QYZC": {
"Type": "AWS::EC2::Instance",
"Properties": {
"AvailabilityZone" : "us-east-2",
"InstanceType" : "t2.micro",
"ImageId" :"ami-034e6c68f4dy6d449", // <------------ AMI ID
"SecurityGroups" : [ "sg-036a1a54caee05af5" ],
}
}
}
}
ERROR : (after clicking create stack) ImageId cannot be empty and it stops right there.
The target Instance id is an Instance Id. It should be an AMI ID instead.
You're trying to connect to an already existing instance instead of creating a new one.
In my cf template I have set of conditions defined and those conditions are invoked in the resource section as well however when i try to generate outputs using the conditions its not working as expected.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "CloudFormation to Deploy EMR clusters",
"Parameters": {
"Applications": {
"Default": "Core Hadoop",
"Description": "Installed Applications",
"Type": "String",
"AllowedValues" : ["Core Hadoop","HBase"],
"ConstraintDescription": "Must be valid Applications"
},
"awsRegion": {
"Default" : "<my-region>",
"Description": "awsRegion",
"Type": "String"
},
"Ec2KeyName": {
"Default": "<my-key.pem>",
"Description": "Ec2KeyName",
"Type": "String"
}
},
"Resources": {
"EMRCluster1": {
"Type" : "AWS::EMR::Cluster",
"Condition" : "CH",
"Properties" : {
"Applications" : [
{
"Name": "Hadoop"
},
{
"Name": "Hive"
},
{
"Name": "Hue"
},
{
"Name": "Pig"
},
{
"Name": "Mahout"
},
{
"Name": "Tez"
}
]
}
}
"EMRCluster2": {
"Type" : "AWS::EMR::Cluster",
"Condition" : "HB",
"Properties" : {
"Applications": [
{
"Name": "Hadoop"
},
{
"Name": "Hive"
},
{
"Name": "Hue"
},
{
"Name": "HBase"
},
{
"Name": "Phoenix"
},
{
"Name": "ZooKeeper"
}
]
}
}
}
"Conditions" : {
"CH" : {"Fn::Equals" : [{"Ref" : "Applications"}, "Core Hadoop"]},
"HB" : {"Fn::Equals" : [{"Ref" : "Applications"}, "HBase"]}
},
"Outputs": {
"MasterPublicDnsName": {
"Condition" : "CH",
"Value": {
"Fn::GetAtt": [
"EMRCluster1",
"MasterPublicDNS"
]
},
"Description": "MasterPublicDNS for cluster"
},
"MasterPublicDnsName": {
"Condition" : "HB",
"Value": {
"Fn::GetAtt": [
"EMRCluster2",
"MasterPublicDNS"
]
},
"Description": "MasterPublicDNS for cluster"
}
}
Expected: If i choose "CH" in the parameters then it should give cluster1 masterdns and if i choose "HB" it should provide cluster2's masterdns
Actual: If i choose "HB" which is the last section in output, it will give me the masterdns of cluster2 however if i choose "CH" outputs section in cloudformation gives me "No outputs found.".
Can somebody help me on this please.
The problem probably exists because you are using the same name for both outputs. Since the behaviour of JSON with duplicate keys is undefined, the implementing language can choose how to behave in this situation. Presumably, the second time you are using the MasterPublicDnsName as the output name, you are overwriting the first name, which is consistent with the behaviour you are seeing.
You can either opt to use two different names, but this might makes using cross-stack references difficult, or use an Fn::If statement in the value of the output:
"Fn::If": [condition_name, value_if_true, value_if_false]
Or in your case:
"Fn::If": ["CH", {"Fn::GetAtt": ["EMRCluster1", "MasterPublicDNS"]}, {"Fn::GetAtt": ["EMRCluster2", "MasterPublicDNS"]}]
If you need more than two options, you'll need to nest your Fn::If statements
I have the following Cloud Formation template to create a VPC. The VPC name is generated based off of the region and the environment that the template was created in. The VPC creates without any issues, and running aws cloud formation validate-template --template-url https://foo.template doesn't complaing about any of the syntax.
I would expect the VPC to be named:
vpc-uw1-d-fs
What happens instead is the VPC is left with an empty name and the Name tag has an empty value. Am I not using the function correctly? If I remove the Fn::FindInMap function usage, I get the name generated - it's just missing the environment mapped value.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "VPC for a new environment to use.",
"Parameters": {
"EnvironmentName": {
"Description": "Name of the environment that this VPC will be used for.",
"Type": "String",
"MinLength": "2",
"MaxLength": "20",
"AllowedPattern": "[a-zA-Z]*",
"AllowedValues": [
"Development",
"QA",
"Test",
"Production"
]
}
},
"Resources": {
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"EnableDnsSupport": false,
"EnableDnsHostnames": false,
"InstanceTenancy": "default",
"Tags": [ {
"Key": "Name",
"Value": {
"Fn::Join": [
"-",
[
"vpc",
{ "Ref": "AWS::Region" },
{ "Fn::FindInMap": [
"EnvironmentMap", { "Ref": "EnvironmentName" }, "AbbreviatedName"
]},
"fs"
]
]
}
}]
}
}
},
"Mappings": {
"RegionMap": {
"us-east-1": {
"regionName": "ue1"
},
"us-west-1": {
"regionName": "uw1"
}
},
"EnvironmentMap": {
"Development": {
"AbbreviatedName": "d"
},
"QA": {
"AbbreviatedName": "qa"
},
"Test": {
"AbbreviatedName": "t"
},
"Production": {
"AbbreviatedName": "p"
}
}
},
"Outputs": {
}
}
Your template is working perfectly fine for me.
I ran it in the ap-southeast-2 region and it produced the tag:
Name: vpc-ap-southeast-2-d-fs
(The RegionMap is not used in the template given.)
How can I pass a parameters of type List<AWS::EC2::Subnet::Id> as a comma separated string?
I have the following template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"PrivateSubnets": {
"Description": "The private subnets in which Beanstalk EC2 instances will created",
"Type": "List<AWS::EC2::Subnet::Id>"
},
"PublicSubnets": {
"Description": "The public subnets in which the Beanstalk ELBs will be created",
"Type": "List<AWS::EC2::Subnet::Id>"
}
},
"Resources": {
"MyApp": {
"Type": "AWS::ElasticBeanstalk::Application",
"Properties": {
"ApplicationName": "MyApp",
"Description": "AWS Elastic Beanstalk Application"
}
},
"ConfigTemplate": {
"Type": "AWS::ElasticBeanstalk::ConfigurationTemplate",
"Properties": {
"ApplicationName": { "Ref": "MyApp" },
"Description": "Microsite Beanstalk config template",
"OptionSettings": [
{ "Namespace": "aws:ec2:vpc", "OptionName": "ELBSubnets", "Value": { "Ref": "PublicSubnets" } },
{ "Namespace": "aws:ec2:vpc", "OptionName": "Subnets", "Value": { "Ref": "PrivateSubnets"} }
],
"SolutionStackName": "64bit Amazon Linux 2016.03 v2.1.7 running PHP 5.6"
}
}
}
}
When I attempt to create the stack, I get the following error:
CREATE_FAILED AWS::ElasticBeanstalk::ConfigurationTemplate ConfigTemplate Value of property Value must be of type String
An attempt to use Fn:Join to write the content of the private and public subnets as comma separated strings, e.g.
{ "Namespace": "aws:ec2:vpc", "OptionName": "ELBSubnets", "Value": { "Fn:Join": [",", { "Ref": "PublicSubnets" }]} },
{ "Namespace": "aws:ec2:vpc", "OptionName": "Subnets", "Value": { "Fn:Join": [",", { "Ref": "PrivateSubnets"}]} },
result in
Template validation error: Template Error: Encountered unsupported function: Fn:Join Supported functions are: [Fn::Base64, Fn::GetAtt, Fn::GetAZs, Fn::ImportValue, Fn::Join, Fn::FindInMap, Fn::Select, Ref, Fn::Equals, Fn::If, Fn::Not, Condition, Fn::And, Fn::Or, Fn::Contains, Fn::EachMemberEquals, Fn::EachMemberIn, Fn::ValueOf, Fn::ValueOfAll, Fn::RefAll, Fn::Sub]
According to http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options-general.html#command-options-general-ec2vpc you need to provide a comma delimited list. So use Fn::Join (note the two colons)
How do you set the Redis ReplicationGroup ID when using a CloudFormation template? All the options in the docs show no way to do that and the CLI you can do this easily. My end goal is to have a Redis replication group with 3 cluster members but I want to choose the name rather than AWS set a unique name for me.
Here's a snippet of my template:
"Resources": {
"mqpReplicationGroup": {
"Type": "AWS::ElastiCache::ReplicationGroup",
"Properties": {
"CacheNodeType": {
"Ref": "CacheNodeType"
},
"CacheSubnetGroupName": {
"Ref": "CacheSubnets"
},
"ReplicationGroupDescription": "Redis Replication Group",
"Engine": "redis",
"EngineVersion": {
"Ref": "RedisVersion"
},
"NumCacheClusters": {
"Ref": "NumberOfCacheNodes"
},
"AutoMinorVersionUpgrade": "true",
"AutomaticFailoverEnabled": "true",
"PreferredMaintenanceWindow": "sat:09:25-sat:22:30",
"SnapshotRetentionLimit": "4",
"SnapshotWindow": "00:05-05:30",
"NotificationTopicArn": {
"Fn::Join" :[":",["arn:aws:sns",{ "Ref" : "AWS::Region" },{ "Ref" : "AWS::AccountId" },"service-aws"]]
},
"SecurityGroupIds": [
{
"Fn::Join": [
",",
{
"Ref": "VpcSecurityGroupIds"
}
]
}
]
}
},
"CacheSubnets": {
"Type": "AWS::ElastiCache::SubnetGroup",
"Properties": {
"Description": "mqp-cache-subnet",
"SubnetIds": {
"Ref": "SubnetIds"
}
}
}
}
As of 2017, this is now possible using ReplicationGroupId property.
Since it is optional, AWS CloudFormation will still generate an unique physical ID when not specified.
Restrictions for Name Type:
Must contain from 1 to 20 alphanumeric characters or hyphens.
First character must be a letter.
Cannot end with a hyphen or contain two consecutive hyphens.
This cannot currently be done with CloudFormation.