Customizing a Prod deployment from a CodeStar project - amazon-web-services

I'm referencing to this documentation: https://docs.aws.amazon.com/codestar/latest/userguide/customize-ec2-multi-endpoints.html#customize-ec2-multi-endpoints-newstage
We currently have a CodeStar project created from the Beanstalk template of type "Java Spring, Web Service". The generated project that uses CloudFormation however only includes a single environment for the deployment of our API.
Question: How does one properly modify this template so that there are two different environments running simultaneously?
In terms of Beanstalk: we need a SingleInstance type for our dev instance, and we want to have a LoadBalanced type for our prod instance (using two EC2 instances for redundancy).
The referenced tutorial mentions awscodestar-<project_name>-infrastructure-prod to be used as the Stack name for the CloudFormation's Sets, but how does that work if there is no actual instance created to be deployed on? I'm confused.
And why is there not already a template file which provides a two-environment CodePipeline/Beanstalk set-up similar to the one described? It seems like a rather mainstream approach to CI/CD.
Here is our template.yml file:
AWSTemplateFormatVersion: 2010-09-09
Transform:
- AWS::CodeStar
Conditions:
UseSubnet: !Not [!Equals [!Ref 'SubnetId', subnet-none]]
Parameters:
ProjectId:
Type: String
Description: AWS CodeStar project ID used to name project resources and create roles.
InstanceType:
Type: String
Description: The type of Amazon EC2 Linux instances that will be launched for this project.
KeyPairName:
Type: String
Description: The name of an existing Amazon EC2 key pair in the region where the project is created, which you can use to SSH into the new Amazon EC2 Linux instances.
VpcId:
Type: String
Description: The ID of the Amazon Virtual Private Cloud (VPC) used for the new Amazon EC2 Linux instances.
SubnetId:
Type: String
Description: The name of the VPC subnet used for the new Amazon EC2 Linux instances launched for this project.
SolutionStackName:
Type: String
Description: The software stack used to launch environments and configure instances in AWS Elastic Beanstalk.
EBTrustRole:
Type: String
Description: The service role in IAM for AWS Elastic Beanstalk to be created for this project.
EBInstanceProfile:
Type: String
Description: The IAM role that will be created for the Amazon EC2 Linux instances.
Stage:
Type: String
Description: The name for a project pipeline stage, such as Staging or Prod, for which resources are provisioned and deployed.
Default: ''
Resources:
EBApplication:
Description: The AWS Elastic Beanstalk application, which is a container used to deploy the correct application configuration.
Type: AWS::ElasticBeanstalk::Application
Properties:
ApplicationName: !Sub '${ProjectId}app${Stage}'
Description: The name of the AWS Elastic Beanstalk application to be created for this project.
EBApplicationVersion:
Description: The version of the AWS Elastic Beanstalk application to be created for this project.
Type: AWS::ElasticBeanstalk::ApplicationVersion
Properties:
ApplicationName: !Ref 'EBApplication'
Description: The application version number.
SourceBundle: 'target/ROOT'
EBConfigurationTemplate:
Description: The AWS Elastic Beanstalk configuration template to be created for this project, which defines configuration settings used to deploy different versions of an application.
Type: AWS::ElasticBeanstalk::ConfigurationTemplate
Properties:
ApplicationName: !Ref 'EBApplication'
Description: The name of the sample configuration template.
OptionSettings:
- Namespace: aws:elasticbeanstalk:environment
OptionName: EnvironmentType
Value: SingleInstance
- Namespace: aws:elasticbeanstalk:environment
OptionName: ServiceRole
Value: !Ref 'EBTrustRole'
- Namespace: aws:elasticbeanstalk:healthreporting:system
OptionName: SystemType
Value: enhanced
SolutionStackName: !Ref 'SolutionStackName'
EBEnvironment:
Description: The AWS Elastic Beanstalk deployment group where the application is deployed, which is made up of the Amazon EC2 Linux instances launched for this project.
Type: AWS::ElasticBeanstalk::Environment
Properties:
ApplicationName: !Ref 'EBApplication'
EnvironmentName: !Ref 'EBApplication'
Description: The application to be deployed to the environment.
TemplateName: !Ref 'EBConfigurationTemplate'
VersionLabel: !Ref 'EBApplicationVersion'
OptionSettings:
- Namespace: aws:autoscaling:launchconfiguration
OptionName: IamInstanceProfile
Value: !Ref 'EBInstanceProfile'
- Namespace: aws:autoscaling:launchconfiguration
OptionName: InstanceType
Value: !Ref 'InstanceType'
- Namespace: aws:autoscaling:launchconfiguration
OptionName: EC2KeyName
Value: !Ref 'KeyPairName'
- Namespace: aws:ec2:vpc
OptionName: VPCId
Value: !Ref 'VpcId'
- !If
- UseSubnet
- Namespace: 'aws:ec2:vpc'
OptionName: Subnets
Value: !Ref 'SubnetId'
- !Ref "AWS::NoValue"

You could create a master template and refer to the production and dev templates as nested stacks. In this case all you would have to do is copy the template.yaml file twice, rename the copies to something appropriate, and upload them to an S3 bucket. Then your template.yaml file would refer to those two templates. Something like this:
AWSTemplateFormatVersion: 2010-09-09
Transform:
- AWS::CodeStar
Description: Master stack which creates all required nested stacks
Resources:
ProdStack:
Type: "AWS::CloudFormation::Stack"
Properties:
TemplateURL: https://your-bucket/templates/production-stack.yml"
Parameters:
InstanceType: t2.micro
EBInstanceProfile: eg
KeyPairName: eg
VpcId: eg
ProjectId: eg
SubnetId: eg
SolutionStackName: eg
EBTrustRole: eg
Tags:
- Key: Environment
Value: Production
DevStack:
Type: "AWS::CloudFormation::Stack"
Properties:
TemplateURL: https://your-bucket/templates/development-stack.yml"
Parameters:
InstanceType: t2.nano
EBInstanceProfile: eg
KeyPairName: eg
VpcId: eg
ProjectId: eg
SubnetId: eg
SolutionStackName: eg
EBTrustRole: eg
Tags:
- Key: Environment
Value: Dev
This will create both stacks and allow you to put in the appropriate parameters for each. Take a look here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-stack.html
Your pipeline would likely remain in the master file, and any other parameters you need for both environments, depending on how you structure the project.

Related

Do I need to create a seperate network stack for this cloudformation yaml file to work?

so I am tring to create Fargate instances into subnets using cloudformation.
I would like the user to be able to choose which vpc id, subnet ids to launch their Fargate instances into which I use as a parameter like this:
Parameters:
VPCSubnets:
Type: List<AWS::EC2::Subnet::Id>
Description: Provide the subnets you wish to deploy into.
VPCInformation:
Type: AWS::EC2::VPC::Id
Description: Provide the VPC ID that resources will be deployed into.
this information is used for the network settings for ECS and task definitions.
If I create network resouces just below parameters like this:
for example:
MyVpc:
Type: AWS::EC2::VPC
Description: VPC for the cluster and fargate instances
Properties:
CidrBlock: 10.0.0.0/26
EnableDnsSupport: true
EnableDnsHostnames: true
Tags:
- Key: interviewchallenge-vpc
Value: !Join ['', [!Ref "AWS::Region", "conversion-challenge-VPC" ]]
PublicSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: myVPC
CidrBlock: 10.0.0.0/28
AvailabilityZone: "us-east-1a"
Tags:
- Key: interviewchallenge-vpc-subnet1
Value: !Join ['', [!Ref "AWS::Region", "conversion-challenge-az1" ]]
At this point in the template, these network resouces havent been created right?
Can this be done in a single stack??
Can this be done in a single stack??
No. You need two templates and the corresponding stacks. The first template creates VPC, subnets and the remaining network resources. Then in the second stack, you use them in for ECS deployment.
Only this way your user will be able to choose the VPC and subnets when creating the ECS stack.

How to reference existing resources in CloudFormation yaml template?

I want to only create the EC2 instance for automation purposes but to point to existing VPC, Subnet, Security groups and Internet Gateway.
Does anyone know what would it look like in the template file?
My current template looks like this. It fails when creating the stack and the instance is deleted automatically.
The error I received is CREATE_FAILED with the description "No default VPC for this user. GroupName is only supported for EC2-Classic and default VPC."
AWSTemplateFormatVersion: 2010-09-09
Parameters:
VPCId:
Type: AWS::EC2::VPC::Id
Resources:
MySubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPCId
EC2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-04ff9e9b51c1f62ca
InstanceType: c5.xlarge
KeyName: CloudFormation
SecurityGroupIds:
- mySecurityGroup
Edited with template
SecurityGroupIds should be ID, not name. The security group IDs have format of sg-xxxxxxxxx, and that's what you have to use.

AWS Code Pipeline fails after adding RDS to Elastic Beanstalk

I have an EBS/Tomcat/Java environment created using the CodeStar template in Jan 2020. Recently I added a RDS within the EBS console following these instructions for creating a dev DB within EBS.
Since adding the RDS, my CodePipeline fails in the CloudFormation step, with the error
Failed Environment update activity. Reason: Configuration validation exception: Missing value for required parameter: AWSEBDBPassword
on my EBS dashboard under recent events.
It seems to be missing a database password or permissions, but I'm not sure how to set that.
If it helps, here's my CodeStar pipeline template file:
AWSTemplateFormatVersion: 2010-09-09
Transform:
- AWS::CodeStar
Conditions:
UseSubnet:
Fn::Not:
- Fn::Equals:
- Ref: SubnetId
- subnet-none
Parameters:
ProjectId:
Type: String
Description: AWS CodeStar project ID used to name project resources and create
roles.
InstanceType:
Type: String
Description: The type of Amazon EC2 Linux instances that will be launched for
this project.
KeyPairName:
Type: String
Description: The name of an existing Amazon EC2 key pair in the region where the
project is created, which you can use to SSH into the new Amazon EC2 Linux instances.
VpcId:
Type: String
Description: The ID of the Amazon Virtual Private Cloud (VPC) used for the new
Amazon EC2 Linux instances.
SubnetId:
Type: String
Description: The name of the VPC subnet used for the new Amazon EC2 Linux instances
launched for this project.
SolutionStackName:
Type: String
Description: The software stack used to launch environments and configure instances
in AWS Elastic Beanstalk.
EBTrustRole:
Type: String
Description: The service role in IAM for AWS Elastic Beanstalk to be created for
this project.
EBInstanceProfile:
Type: String
Description: The IAM role that will be created for the Amazon EC2 Linux instances.
Stage:
Type: String
Description: The name for a project pipeline stage, such as Staging or Prod, for
which resources are provisioned and deployed.
Default: ''
Resources:
EBApplication:
Description: The AWS Elastic Beanstalk application, which is a container used
to deploy the correct application configuration.
Type: AWS::ElasticBeanstalk::Application
Properties:
ApplicationName:
Fn::Sub: ${ProjectId}app${Stage}
Description: The name of the AWS Elastic Beanstalk application to be created
for this project.
EBApplicationVersion:
Description: The version of the AWS Elastic Beanstalk application to be created
for this project.
Type: AWS::ElasticBeanstalk::ApplicationVersion
Properties:
ApplicationName:
Ref: EBApplication
Description: The application version number.
SourceBundle:
S3Bucket: aws-codestar-us-west-2-215674088663-aa5050solnprj3-pipe
S3Key: 810d567534b4cb9ca0ee597128a22b94
EBConfigurationTemplate:
Description: The AWS Elastic Beanstalk configuration template to be created for
this project, which defines configuration settings used to deploy different
versions of an application.
Type: AWS::ElasticBeanstalk::ConfigurationTemplate
Properties:
ApplicationName:
Ref: EBApplication
Description: The name of the sample configuration template.
OptionSettings:
- Namespace: aws:elasticbeanstalk:environment
OptionName: EnvironmentType
Value: SingleInstance
- Namespace: aws:elasticbeanstalk:environment
OptionName: ServiceRole
Value:
Ref: EBTrustRole
- Namespace: aws:elasticbeanstalk:healthreporting:system
OptionName: SystemType
Value: enhanced
SolutionStackName:
Ref: SolutionStackName
EBEnvironment:
Description: The AWS Elastic Beanstalk deployment group where the application
is deployed, which is made up of the Amazon EC2 Linux instances launched for
this project.
Type: AWS::ElasticBeanstalk::Environment
Properties:
ApplicationName:
Ref: EBApplication
EnvironmentName:
Ref: EBApplication
Description: The application to be deployed to the environment.
TemplateName:
Ref: EBConfigurationTemplate
VersionLabel:
Ref: EBApplicationVersion
OptionSettings:
- Namespace: aws:autoscaling:launchconfiguration
OptionName: IamInstanceProfile
Value:
Ref: EBInstanceProfile
- Namespace: aws:autoscaling:launchconfiguration
OptionName: InstanceType
Value:
Ref: InstanceType
- Namespace: aws:autoscaling:launchconfiguration
OptionName: EC2KeyName
Value:
Ref: KeyPairName
- Namespace: aws:ec2:vpc
OptionName: VPCId
Value:
Ref: VpcId
- Fn::If:
- UseSubnet
- Namespace: aws:ec2:vpc
OptionName: Subnets
Value:
Ref: SubnetId
- Ref: AWS::NoValue```

AWS Cloudformation Master Template creation

I am trying to create a master template which calls other templates.My first template is VPC and subnet creation and second one is creating bastion host. Problem i am facing is i am not able to ref. the created VPC in my second template due to which its failing. My master template looks like below:-
Description: >
This template deploys the full agyle time stack as follows, which consists of:
A VPC with with public and private subnets spread across two Availabilty Zones.
It deploys an Internet Gateway and a pair of NAT Gateways, with the relevant routes in each of the subnets.
It then deploys the API ECS cluster distributed across multiple Availability Zones.
Finally, it deploys the API ECS services deployed as containers within the ECS repository
Parameters:
S3TemplateKeyPrefix:
Description: >
An S3 key prefix which will be used to resolve referenced templates
Type: String
Resources:
VPC:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub ${S3TemplateKeyPrefix}/infrastructure/vpc.yaml
Bastion:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub ${S3TemplateKeyPrefix}/infrastructure/bastion.yaml
Parameters:
EnvironmentName: !Ref AWS::StackName
VPC: !GetAtt VPC.Outputs.VPC
Can someone help me here do i have to modify VPC and Bastion host template to reference my VPC in bastion template.
Based on your master template, I believe it fails because CFN starts creating both of them in parallel, whereas Bastion needs to be created after your VPC resource. Just add the DependsOn: VPC for your Bastion resource to have it created only after your VPC has been created.
Bastion:
Type: AWS::CloudFormation::Stack
DependsOn: VPCStack
Properties:
Here's a working example from AWS saas-identity-cognito-master.template.
I was able to resolve the issue with modifying the child templates with Export and Import Function and calling it in master template. below is what I used:-
Outputs:
PubPrivateVPC:
Description: A reference to the created VPC
Value: !Ref PubPrivateVPC
Export:
Name: VPC-PROD
and import
Parameter:-
NetworkStackName:
Description: >-
Name of an active CloudFormation stack that contains the networking
resources, such as the subnet and security group, that will be used in
this stack.
Type: String
MinLength: 1
MaxLength: 255
AllowedPattern: '^[a-zA-Z][-a-zA-Z0-9]*$'
Default: VPC-PROD
and in resources called like below:-
VpcId: !ImportValue VPC-PROD
No i am able to call child templates in master successfully.

AWS Change ECS Service task definition (deploy) without using CloudFormation

Slowly moving all of our resources over to be managed by CloudFormation which so far has been relatively painless.
I'm hitting a bit of a wall with deployments of new task definitions (staging environment only) on ECS services. There's parts which I don't think can be managed with CloudFormation because the ECS Service is referencing a parameter.
The current process:
Push to a develop branch on BitBucket
BitBucket Pipelines then handles (simplified):
creating a ECR image with the tag of latest
creating a new task definition revision via aws ecs register-task-definition --cli-input-json $TASK_JSON
update the ECS service with the new task definition using aws ecs update-service --cluster ${ECS_CLUSTER_NAME} --service ${ECS_SERVICE} --task-definition $TASK_DEFINITION
This all works well!
Move the ECS resources management to CloudFormation with something along the lines of
Parameters:
TaskDefinition:
Type: String
Description: Task Definition ARN
Resources:
HTTPService:
Type: AWS::ECS::Service
DependsOn: ListenerRule
Properties:
Cluster: !Ref ECSCluster
Role: !Ref ServiceRole
DesiredCount: 1
HealthCheckGracePeriodSeconds: 120
TaskDefinition: !Ref TaskDefinition
LoadBalancers:
- ContainerName: http
ContainerPort: 80
TargetGroupArn: !Ref TargetGroup
TargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
VpcId: !Ref VPC
Port: 80
Protocol: HTTP
Matcher:
HttpCode: 200
HealthCheckIntervalSeconds: 15
HealthCheckPath: /
HealthCheckProtocol: HTTP
HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
ListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
Properties:
ListenerArn: !Ref Listener
Priority: 2
Conditions:
- Field: path-pattern
Values:
- /blog
Actions:
- TargetGroupArn: !Ref TargetGroup
Type: forward
If I were to run the aws ecs update-service command manually now the CloudFormation stack would be out of sync with the ECS service.
Had a look into maybe running a aws cloudformation update-stack instead but the IAM permissions for the BitBucket user become quite complex.
Maybe I have gone about setting this all up incorrectly here. Any pointers/suggestions here?