Cloudformation template properties documentation discrepancy - amazon-web-services

I'm creating my first Cloudformation template using an archived Github project from an AWS Blog:
https://aws.amazon.com/blogs/devops/part-1-develop-deploy-and-manage-for-scale-with-elastic-beanstalk-and-cloudformation-series/
https://github.com/amazon-archives/amediamanager
The template amm-elasticbeanstalk.cfn.json declares an Elastic Beanstalk resource, outlined here:
"Resources": {
"Application": {
"Type": "AWS::ElasticBeanstalk::Application",
"Properties": {
"ConfigurationTemplates": [{...}],
"ApplicationVersions": [{...}]
}
}
}
From the documentation I'm under the impression that AWS::ElasticBeanstalk::ApplicationVersion and AWS::ElasticBeanstalk::ConfigurationTemplate must be defined as separate resources, yet the example I'm working from is using the same AWSTemplateFormatVersion as the documentation. Is this a "shorthand" where namespaces can be nested if they have the same parent (i.e. AWS::ElasticBeanstalk)? Is it documented somewhere?
In the same file AWS::ElasticBeanstalk::Environment is defined as a separate resource - is this just a stylistic choice, perhaps because the environment configuration is so long?

Elastic Beanstalk consists of Applications and Environments components. Basically each environment runs only one application version at a time, however, you can run the same application version in many environments at the same time. Application versions and Saved configurations are part of the Application resource that's why it's possible to define it within the AWS::ElasticBeanstalk::Application resource properties. Environment however is a separate logical component of Elastic Beanstalk so it's impossible to declare it from within the Application resource.
For better readability I would suggest declaring all the resources separately as per this example. Also when using this approach you can directly reference the TemplateName and VersionLabel in the AWS::ElasticBeanstalk::Environment resource.
Alternatively if you want to stick to the github example you can adjust the above example to look like this:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"sampleApplication": {
"Type": "AWS::ElasticBeanstalk::Application",
"Properties": {
"Description": "AWS Elastic Beanstalk Sample Application",
"ApplicationVersions": [{
"VersionLabel": "Initial Version",
"Description": "Initial Version",
"SourceBundle": {
"S3Bucket": {
"Fn::Sub": "elasticbeanstalk-samples-${AWS::Region}"
},
"S3Key": "php-newsample-app.zip"
}
}],
"ConfigurationTemplates": [{
"TemplateName": "DefaultConfiguration",
"Description": "AWS ElasticBeanstalk Sample Configuration Template",
"OptionSettings": [
{
"Namespace": "aws:autoscaling:asg",
"OptionName": "MinSize",
"Value": "2"
},
{
"Namespace": "aws:autoscaling:asg",
"OptionName": "MaxSize",
"Value": "6"
},
{
"Namespace": "aws:elasticbeanstalk:environment",
"OptionName": "EnvironmentType",
"Value": "LoadBalanced"
},
{
"Namespace": "aws:autoscaling:launchconfiguration",
"OptionName": "IamInstanceProfile",
"Value": {
"Ref": "MyInstanceProfile"
}
}
],
"SolutionStackName": "64bit Amazon Linux 2018.03 v2.9.11 running PHP 5.5"
}]
}
},
"sampleEnvironment": {
"Type": "AWS::ElasticBeanstalk::Environment",
"Properties": {
"ApplicationName": {
"Ref": "sampleApplication"
},
"Description": "AWS ElasticBeanstalk Sample Environment",
"TemplateName": "DefaultConfiguration",
"VersionLabel": "Initial Version"
}
},
"MyInstanceRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Description": "Beanstalk EC2 role",
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/AWSElasticBeanstalkWebTier",
"arn:aws:iam::aws:policy/AWSElasticBeanstalkMulticontainerDocker",
"arn:aws:iam::aws:policy/AWSElasticBeanstalkWorkerTier"
]
}
},
"MyInstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Roles": [
{
"Ref": "MyInstanceRole"
}
]
}
}
}
}

Just want to point out that AWS silently phased out the option of having the ApplicationVerions key under an AWS::ElasticBeanstalk::Application's Properties. It was still working in July 2022 but I noticed it stopped some time in August 2022, giving the error in the CloudFormation stack's Event tab:
Properties validation failed for resource TheEBAppResName with message: #: extraneous key [ApplicationVersions] is not permitted
where TheEBAppResName is the name of your AWS::ElasticBeanstalk::Application resource.
The only solution now is to follow the current AWS example and use a separate AWS::ElasticBeanstalk::ApplicationVersion resource.
Interestingly, I can't seem to find any documentation on the obsolete ApplicationVerions property anymore and the AWS blog that you linked to is no longer available, but I did find it cached on the Wayback machine. Even the earliest AWS doc on GitHub for AWS::ElasticBeanstalk::Application doesn't mention the ApplicationVerions property. Seems like AWS silently deprecated it sometime between when the blog was posted in April 2014 and that earliest GitHub doc page in December 2017, but didn't actually remove the option until last month, August 2022.

Related

Where/how do I define a NotificationConfig in an AWS SSM Automation document?

Say I have an SSM document like the below, and I want to be alerted when a run fails or doesn't finish for whatever reason:
{
"description": "Restores specified pg_dump backup to specified RDS/DB.",
"mainSteps": [
{
"action": "aws:runCommand",
"description": "Restores specified pg_dump backup to specified RDS/DB.",
"inputs": {
"DocumentName": "AWS-RunShellScript",
"Parameters": {
"commands": [
"blahblahblah"
],
"executionTimeout": "1800"
},
"Targets": [
{
"Key": "InstanceIds",
"Values": [
"i-xxxxxxxx"
]
}
]
},
"name": "DBRestorer",
"nextStep": "RunQueries"
},
Terraform documents show me that RunCommand documents should support a NotificationConfig where I can pass in my SNS topic ARN and declare what state transitions should trigger a message: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_maintenance_window_task#notification_config
However, I can't find any Amazon docs that actually include the use of a notification configuration in the document itself (not just the maintenance window, which I have set up as automation so it doesn't support it at the window level), so I'm not sure if it belongs as a sub-parameter, or whether to define it with camel case or dash separation.
Try this
{
"description": "Restores specified pg_dump backup to specified RDS/DB.",
"mainSteps": [
{
"action": "aws:runCommand",
"description": "Restores specified pg_dump backup to specified RDS/DB.",
"inputs": {
"DocumentName": "AWS-RunShellScript",
"NotificationConfig": {
"NotificationArn": "<<Replace this with a SNS Topic Arn>>",
"NotificationEvents": ["All"],
"NotificationType": "Invocation"
},
"ServiceRoleArn": "<<Replace this with an IAM role Arn that has access to SNS>>",
"Parameters": {
"commands": [
"blahblahblah"
],
"executionTimeout": "1800"
},
"Targets": [
{
"Key": "InstanceIds",
"Values": [
"i-xxxxxxxx"
]
}
]
},
"name": "DBRestorer",
"nextStep": "RunQueries"
},
...
]
}
Related documentation:
https://docs.aws.amazon.com/systems-manager/latest/userguide/automation-action-runcommand.html
https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_NotificationConfig.html#systemsmanager-Type-NotificationConfig-NotificationType

Developing an app to create a stack from a cloudformation template

I am new to AWS and am currently working on simple tasks.
I have created a free tier EC2 instance using a cloudformation template. Now my next task is to write a simple application that uses respective AWS SDK to call CloudFormation API to create a stack from the template.
Here is the cloudformation template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Ec2 Template",
"Metadata": {
"Copyright":[
"Copyright 2017, All rights reserved"
],
"Comments":[
"Create an EC2"
]
},
"Parameters": {
"KeyName": {
"Type": "AWS::EC2::KeyPair::KeyName",
"Description": "Name of an existing EC2 KeyPair to enable access to join ECS instances."
},
"InstanceTypeParameter":{
"Type": "String",
"Default": "t2.micro",
"AllowedValues": [
"t2.micro",
"m1.small",
"m1.large"
],
"Description": "Enter t2.micro, m1.small, or m1.large. Default is t2.micro."
},
"EcsSecurityGroupLb":{
"Type": "AWS::EC2::SecurityGroup::Id",
"Description":"The ECS ELB Security Group."
},
"vpcid":{
"Type": "AWS::EC2::VPC::Id"
},
"mySubnetIDs": {
"Description":"Subnet IDs",
"Type":"AWS::EC2::Subnet::Id"
}
},
"Resources":{
"Ec2Instance":{
"Type":"AWS::EC2::Instance",
"Properties":{
"ImageId": "ami-bf4193c7",
"KeyName": {
"Ref": "KeyName"
},
"InstanceType":{
"Ref": "InstanceTypeParameter"
},
"NetworkInterfaces":[
{
"AssociatePublicIpAddress":"true",
"DeviceIndex":"0",
"SubnetId":{
"Ref":"mySubnetIDs"
},
"GroupSet":[
{
"Ref": "EcsSecurityGroupLb"
}
]
}
],
"BlockDeviceMappings":[
{
"DeviceName": "/dev/sdc",
"VirtualName":"ephemeral0"
}
]
}
}
},
"Outputs":{
"Ec2Instance":{
"Description": "InstanceId of newly created EC2 instance",
"Value": {
"Ref": "Ec2Instance"
}
},
"InstanceIPAddress":{
"Value":{ "Fn::GetAtt": ["Ec2Instance", "PublicIp"]},
"Description": "Public IP address of instance"
}
}
}
I have gone through a lot of documentation but haven't really understood as to how to proceed. I would like to know if there are any good tutorials on this.
Looking for suggestions with respect to the steps as well.
Thanks!
Since you have to (as a task requirement) write the application yourself, you'll need to use one of the AWS SDKs that are available.
The SDK you choose will depend on what programming language you are most comfortable using (or required by your task).
Roughly, your program will need to do the following:
With the AWS SDK, create an AWS session which uses your IAM user's API keys.
Grab the Cloudformation template off of your local system.
With the AWS SDK, invoke Cloudformation to create the resource stack with your template.
(Optionally) Wait until the stack is complete and output a report on the stack status.
Good luck!

AWS CodePipeLine :Execute deploy action in diffent region than the one codepipeline is triggered in

I'm setting up a pipeline to automate cloudformation stack templates deployment.
The pipeline itself is created in the aws eu-west-1 region, but cloudformation stacks templates would be deployed in any other region.
Actually I know and can execute pipeline action in a different account, but I don't see where to specify the region I would like my template to be deployed in, like we do with aws cli : aws --region cloudformation deploy.....
Is there anyway to trigger a pipeline in one region and execute a deploy action in another region please?
The action configuration properties don't offer such possibility...
A workaround would be to run aws cli deploy command from cli in the codebuild container and speficy the good region, But I would like to know if there is a more elegant way to do it
If you're looking to deploy to multiple regions, one after the other, you could create a Code Pipeline pipeline in every region you want to deploy to, and set up S3 cross-region replication so that the output of the first pipeline becomes the input to a pipeline in the next region.
Here's a blog post explaining this further: https://aws.amazon.com/blogs/devops/building-a-cross-regioncross-account-code-deployment-solution-on-aws/
Since late Nov 2018, CodePipeline supports cross regional deploys. However it still leaves a lot to be desired as you need to create artifact buckets in each region and copy over the deployment artifacts (e.g. in the codebuild container as you mentioned) to them before the Deploy action is triggered. So it's not as automated as it could be, but if you go through the process of setting it up, it works well.
CodePipeline now supports cross region deployment and for to trigger the pipeline in different region we can specify the "Region": "us-west-2" property in the action stage for CloudFormation which will trigger the deployment in that specific region.
Steps to follow for this setup:
Create two bucket in two different region which for example bucket in "us-east-1" and bucket in "us-west-2" (We can also use bucket already created by CodePipeline when you will setup pipeline first time in any region)
Configure the pipeline in such a way that is can use respective bucket while taking action in respective account.
specify the region in the action for CodePipeline.
Note: I have attached the sample CloudFormation template which will help you to do the cross region CloudFormation deployment.
{
"Parameters": {
"BranchName": {
"Description": "CodeCommit branch name for all the resources",
"Type": "String",
"Default": "master"
},
"RepositoryName": {
"Description": "CodeComit repository name",
"Type": "String",
"Default": "aws-account-resources"
},
"CFNServiceRoleDeployA": {
"Description": "CFN service role for create resourcecs for account-A",
"Type": "String",
"Default": "arn:aws:iam::xxxxxxxxxxxxxx:role/CloudFormation-service-role-cp"
},
"CodePipelineServiceRole": {
"Description": "Service role for codepipeline",
"Type": "String",
"Default": "arn:aws:iam::xxxxxxxxxxxxxx:role/AWS-CodePipeline-Service"
},
"CodePipelineArtifactStoreBucket1": {
"Description": "S3 bucket to store the artifacts",
"Type": "String",
"Default": "bucket-us-east-1"
},
"CodePipelineArtifactStoreBucket2": {
"Description": "S3 bucket to store the artifacts",
"Type": "String",
"Default": "bucket-us-west-2"
}
},
"Resources": {
"AppPipeline": {
"Type": "AWS::CodePipeline::Pipeline",
"Properties": {
"Name": {"Fn::Sub": "${AWS::StackName}-cross-account-pipeline" },
"ArtifactStores": [
{
"ArtifactStore": {
"Type": "S3",
"Location": {
"Ref": "CodePipelineArtifactStoreBucket1"
}
},
"Region": "us-east-1"
},
{
"ArtifactStore": {
"Type": "S3",
"Location": {
"Ref": "CodePipelineArtifactStoreBucket2"
}
},
"Region": "us-west-2"
}
],
"RoleArn": {
"Ref": "CodePipelineServiceRole"
},
"Stages": [
{
"Name": "Source",
"Actions": [
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "CodeCommit"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"BranchName": {
"Ref": "BranchName"
},
"RepositoryName": {
"Ref": "RepositoryName"
},
"PollForSourceChanges": true
},
"RunOrder": 1
}
]
},
{
"Name": "Deploy-to-account-A",
"Actions": [
{
"Name": "stage-1",
"InputArtifacts": [
{
"Name": "SourceOutput"
}
],
"ActionTypeId": {
"Category": "Deploy",
"Owner": "AWS",
"Version": 1,
"Provider": "CloudFormation"
},
"Configuration": {
"ActionMode": "CREATE_UPDATE",
"StackName": "cloudformation-stack-name-account-A",
"TemplatePath":"SourceOutput::accountA.json",
"Capabilities": "CAPABILITY_IAM",
"RoleArn": {
"Ref": "CFNServiceRoleDeployA"
}
},
"RunOrder": 2,
"Region": "us-west-2"
}
]
}
]
}
}
}
}

EMR cluster created with CloudFormation not shown

I have added an EMR cluster to a stack. After updating the stack successfully (CloudFormation), I can see the master and slave nodes in EC2 console and I can SSH into the master node. But AWS console does not show the new cluster. Even aws emr list-clusters doesn't show the cluster. I have triple checked the region and I am certain I'm looking at the right region.
Relevant CloudFormation JSON:
"Spark01EmrCluster": {
"Type": "AWS::EMR::Cluster",
"Properties": {
"Name": "Spark01EmrCluster",
"Applications": [
{
"Name": "Spark"
},
{
"Name": "Ganglia"
},
{
"Name": "Zeppelin"
}
],
"Instances": {
"Ec2KeyName": {"Ref": "KeyName"},
"Ec2SubnetId": {"Ref": "PublicSubnetId"},
"MasterInstanceGroup": {
"InstanceCount": 1,
"InstanceType": "m4.large",
"Name": "Master"
},
"CoreInstanceGroup": {
"InstanceCount": 1,
"InstanceType": "m4.large",
"Name": "Core"
}
},
"Configurations": [
{
"Classification": "spark-env",
"Configurations": [
{
"Classification": "export",
"ConfigurationProperties": {
"PYSPARK_PYTHON": "/usr/bin/python3"
}
}
]
}
],
"BootstrapActions": [
{
"Name": "InstallPipPackages",
"ScriptBootstrapAction": {
"Path": "[S3 PATH]"
}
}
],
"JobFlowRole": {"Ref": "Spark01InstanceProfile"},
"ServiceRole": "MyStackEmrDefaultRole",
"ReleaseLabel": "emr-5.13.0"
}
}
The reason is missing VisibleToAllUsers property, which defaults to false. Since I'm using AWS Vault (i.e. using STS AssumeRole API to authenticate), I'm basically a different user every time, so I couldn't see the cluster. I couldn't update the stack to add VisibleToAllUsers either as I was getting Job flow ID does not exist.
The solution was to login as root user and fix things from there (I had to delete the cluster manually, but removing it from the stack template JSON and updating the stack would probably have worked if I hadn't messed things up already).
I then added the cluster back to the template (with VisibleToAllUsers set to true) and updated the stack as usual (AWS Vault).

AWS ECS - Unable to specify service name in cloudformation template

I am trying to create a AWS - ECS service using cloudformation template
"service": {
"ServiceName": "XXX",
"Type": "AWS::ECS::Service",
"DependsOn": [
"AutoScalingGroup"
],
"Properties": {
"Cluster": {
"Ref": "ECSCluster"
},
"DesiredCount": "1",
"TaskDefinition": {
"Ref": "taskdefinition"
}
}
},
But I am getting an error.
Failed: Invalid template resource property 'ServiceName'
I had same problem when using Name/serviceName. I can see serviceName is a parameter based on docs. But couldn't figure out why it fails. It works if I don't specify name. But I need to specify name so that I can use the same name in a different system that updates the service.
Could you please help?
This is slightly confusing, but the service name is set by the name of the resource you create. There is no ServiceName or Name property. The following will create an ECS service with the name MyService.
"MyService": {
"Type": "AWS::ECS::Service",
"DependsOn": [
"AutoScalingGroup"
],
"Properties": {
"Cluster": {
"Ref": "ECSCluster"
},
"DesiredCount": "1",
"TaskDefinition": {
"Ref": "taskdefinition"
}
}
}
Obviously if you refer to your service within the CloudFormation template you will also need to update your references.