I am attempting to write a Cloud Formation Template that will create a launch configuration, ASG, ECS cluster, and register the ec2 instances in the ASG to said cluster.
I have a long userdata.sh file that is used to setup a launch configuration.
By putting this userdata in the yaml file
i.e.
...
UserData:
Fn::Base64: !Sub |
#!/bin/bash
...
it looks pretty terrible. Also, when i try and plug in the cluster name into the ecs agent ${ECSCluster} with Sub, i get "variable names in Fn::Sub syntax must contain only alphanumeric characters, underscores, periods, and colons". The environment variables in my userdata take on the same form ${ENV_VARIABLE}.
I've tried Passing userdata file to AWS Cloudformation stack but receive "Template format error: YAML not well-formed. "
What is the recommended way for passing in a complex userdata.sh file into a CFT in such a way that you can add the cluster name to the ecs-agent config?
Error is just stating that it doesn't like the way you reference your variable in Fn::Sub. Instead of $(ECSCluster) it needs to be ${ECSCluster}. See the examples in the docs.
Related
I’m creating a generic stack template using CloudFormation, and I’ve hit a rather annoying circular reference.
Overall Requirements:
I want to be able to provision (a lot of other things, but mainly) an ECS Cluster Service that auto-scales using capacity providers, the capacity providers are using auto-scaling groups, and the auto scaling groups are using a launch template.
I don’t want static resource names. This causes issues if a resource has to be re-created due to an update and that particular resource has to have a unique name.
Problem:
Without the launch template “knowing the cluster name” (via UserData) the service tasks get stuck in a PROVISIONING state.
So we have the first dependency chain:
Launch Template <- Cluster (Name)
But the Cluster has a dependency chain of:
Cluster <- Capacity Provider <- AutoScalingGroup <- Launch Template
Thus, we have a circular reference: Cluster <-> Launch Template
——
One way I can think of resolving this is to add a suffix to another resource’s name (one that lives outside of this dependency chain, e.g., the target group) as the Cluster’s name; in that way, it is not static but also removes the circular reference.
My question is: is there a better way?
It feels like there should be a resource that the cluster can subscribe to and the ec2 instance can publish to, which would remove the circular dependency as well as the need to assign resource names.
There is no such resource to break the dependency and the cluster name must be pre-defined. This has already been recognized as a problem and its part of open github issue:
[ECS] Full support for Capacity Providers in CloudFormation.
One of the issues noted is:
Break circular dependency so that unnamed clusters can be created
At the moment one work around noted is to partially predefine the name, e.g.:
ECSCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: !Sub ${AWS::StackName}-ECSCluster
LaunchConfiguration:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
UserData:
Fn::Base64: !Sub |
#!/bin/bash
echo ECS_CLUSTER=${AWS::StackName}-ECSCluster >> /etc/ecs/ecs.config
Alternatively, one could try to solve that by development of some custom resource that would be in the form of a lambda function. So you could probably create your unnamed cluster with launch template (LT) that has some dummy name for cluster. Then once the cluster is running, you would use the custom resource to create new version of LT with updated cluster name and refresh your auto-scaling group to use the new LT version. But I'm not sure if this would work. Nevertheless, its something that can be considered at least.
Sharing an update from the GitHub issue. The circular dependency has been broken by introducing a new resource: Cluster Capacity Provider Associations.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-clustercapacityproviderassociations.html
To use it in my example, you:
Create Cluster (without specifying name)
Create Launch Template (using Ref to get cluster name)
Create Auto Scaling Group(s)
Create Capacity Provider(s)
Create Cluster Capacity Provider Associations <- This is new!
The one gotcha is that you have to wait for the new association to be created before you can create a service on the cluster. So be sure that your service "DependsOn" these associations!
I am new to AWS CloudFormation. I have been testing out automating deployments of containers using this service. However, as of now, I am manually adding the name of my TaskDefinition to the YAML file of the CloudFormation template. Is there a way for me to define this as an input.
Yes, you can use stack output exports, define export value on one stack and import it in any other: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-stack-exports.html
I am new to AWS lambda i have i am using serveless deploy to deploy my lambda function declared in serverless.yml file.
In that file i wrote a lambda function i deployed and it is working fine but problem is that whatever environment variable i gave is not available in console of that lambda function. I might be doing some minor mistake or for deploying environment variable there should be some other syntax.
I can go to lambda function in console and add environment variable manually.
But my doubt is can we do it while deploying it through serverless deploy
You can use versions and aliases provided by AWS Lambda
You can create different versions of the same lambda function and give them an alias. Like when you push your lambda code - create a version (say it's 5) - create an alias this (say TEST).
When you're sure that its ready for production, create a version(or choose an existing version and name that (say PROD).
Now whenever your lambda function executes, it gives lambda ARN
which contains alias, by that you can know which alias(in context.invokedFunctionArn) is
executed i.e. that can be used as the environment variable. While
invoking the function, you can mention which function to execute from
your invocation code.
let thisARN = context.invokedFunctionArn;
// Get the last string in ARN - It's either function name or the alias name
let thisAlias = thisARN.split(":").pop();
Now whenever you deploy a new code, just point the alias to that version.
You can use AWS console or CLI for that.
Take a look at this lambda versioning and aliases
For directly deploying to your alias(PROD), you can do this -
aws lambda update-alias \
--region region \
--function-name helloworld \
--function-version 2 \
--name PROD
serverless deploy
Serverless deploy works fine for deployment on any stage it also deploys environment variable in given stage, my case environment variable was not deployed of indentation problem in yaml file, and even serverless deploy command was not throwing error it was deploying function but environment variables were not deployed
In yaml file we can state the the stage where we want to deploy like this
provider:
name: aws
runtime: nodejs6.10
stage: dev
region: eu-west-2
Hope this will help if someone gets similar issue
I have a CloudFormation template that creates an AWS::ElasticBeanstalk::Environment and an AWS::ElasticLoadBalancingV2::TargetGroup
I would like to associate the AutoScalingGroup that beanstalk creates with the TargetGroup created in my template.
My end goal is doing path-based routing via an ALB to a bunch of beanstalk applications from a single domain (i.e., www.domain.com/foo routes to ebapp1 and www.domain.com/bar routes to ebapp2)
I can actually accomplish what I wish via the CLI:
aws autoscaling attach-load-balancer-target-groups --auto-scaling-group-name "<asg-name>" --target-group-arns "<arn-for-target-group>"
However, I would like to have this association created automatically when I launch my new beanstalk environment via CloudFormation.
I am having trouble figuring out how to translate this into my CloudFormation template. Any pointers?
Your CLI command is related to AWS::AutoScaling::AutoScalingGroup, and if you want to get result as same as your CLI command you can use TargetGroupARNs parameter on AWS::AutoScaling::AutoScalingGroup.
Use the Reference function to get the value of target group arn parameter.
If this is not what you need, and if your command really works, you can call it using UserData or metadata in an instance that you have created on your template to run your CLI command.
I am creating an AWS AMI that is provisioned with Chef using Packer. At the creation of snapshots tags of the AWS AMI, I want it to be tagged with tags that assign it to be deployed with CodeDeploy:
{
"aws:autoscaling:groupName": "Env1"
}
In my JSON configuration for Packer, I am using snapshot_tags to define these.
The problem is that on creating the AMI, Packer fails with:
Build 'amazon-ebs' errored: Error adding tags to Resources ([]*string{(*string)(0xc420107170), (*string)(0xc420478758)}): InvalidParameterValue: Tag keys starting with 'aws:' are reserved for internal use
status code: 400, request id: fef34822-b692-4225-a2eb-a1cfac33a924
Cannot I use CodeDeploy with Packer since I must use aws in the tag?
ANSWER:
There is no need to use a tag for an AMI to deploy with CodeDeploy. My mistake.
Don't know about CodeDeploy, but you can never create a tag which starts with aws:.
Do not use the aws: prefix in your tag names or values because it is reserved for AWS use. You can't edit or delete tag names or values with this prefix. Tags with this prefix do not count against your tags per resource limit.
See AWS Docs: Using Tags - tag restrictions