I have a shell script(userdata file) and wondering is there a CLI command parameter that allows user to launch Cloudformation stack with userdata file?
Inside your template, use a CloudFormation parameter for the instance userdata:
{
"Parameters": {
"UserData": {
"Type": "String"
}
},
"Resources": {
"Instance": {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"UserData" : { "Ref" : "UserData" },
...
}
},
...
}
}
Assuming you're using a Unix-like command line environment, create your stack like this:
aws cloudformation create-stack --stack-name myStack \
--template-body file://myStack.json \
--parameters ParameterKey=UserData,ParameterValue=$(base64 -w0 userdata.sh)
Your user-data must exist in the CloudFormation template when you create the stack. You can write a script to read in your user-data from the file and insert it into the CloudFormation stack prior to creating the stack. Note that you may need to make formatting changes to the userdata (see http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html#cfn-ec2-instance-userdata).
Related
I'm validating a CloudFormation template that was generated via a Troposphere script. The resource in question which seems to be causing the error is a Metric Transformation which is defined as follows in the Troposphere script:
t.add_resource(logs.MetricTransformation(
"planReconciliationFiduciaryStepMetricTransformation",
MetricNamespace=Ref("metricNamespace"),
MetricName=Join("", [Ref("springProfile"), "-", "plan-reconciliation-step-known-to-fiduciary"]),
MetricValue="1"
))
The parameters it depends on are defined beforehand in the script as follows:
t.add_parameter(Parameter(
"metricNamespace",
Type="String",
Default="BATCH-ERRORS",
Description="Metric namespace for CloudWatch filters"
))
t.add_parameter(Parameter(
"springProfile",
Type="String",
Default=" ",
Description="SPRING PROFILE"
))
The exact command that I'm running is
aws cloudformation validate-template --template-body
file://hor-ubshobackgroundtaskdefinition.template --profile saml
with the resulting output
An error occurred (ValidationError) when calling the ValidateTemplate operation:
Invalid template resource property 'MetricName'
My properties for the MetricTransformation appear to be well-defined from the AWS documentation. For visibility, this is also what the metric transformation resource in the template that's being validated looks like too:
"planReconciliationFiduciaryStepMetricTransformation": {
"MetricName": {
"Fn::Join": [
"",
[
{
"Ref": "springProfile"
},
"-",
"plan-reconciliation-step-known-to-fiduciary"
]
]
},
"MetricNamespace": {
"Ref": "metricNamespace"
},
"MetricValue": "1"
}
Any ideas?
UPDATE:
As requested, adding the metric filter resource:
"PlanReconciliationFiduciaryStepMetricFilter": {
"Properties": {
"FilterPattern": "INFO generatePlanReconciliationStepKnownToFiduciary",
"LogGroupName": {
"Ref": "logGroupName"
},
"MetricTransformations": [
{
"Ref": "planReconciliationFiduciaryStepMetricTransformation"
}
]
},
"Type": "AWS::Logs::MetricFilter"
}
As it turns out, the MetricTransformation resource needs to be initialized within the MetricFilter itself in order to produce the correct CloudFormation template from the Troposphere script. If you declare the MetricTransformation as a separate resource and try to reference it within the the MetricFilter resource, the ensuing template will be incorrectly formatted.
The correct format in the Troposphere script will look like the following:
t.add_resource(logs.MetricFilter(
"PlanReconciliationFiduciaryStepMetricFilter",
FilterPattern="INFO generatePlanReconciliationStepKnownToFiduciary",
LogGroupName=Ref("logGroupName"),
MetricTransformations=[logs.MetricTransformation(
"planReconciliationFiduciaryStepMetricTransformation",
MetricNamespace=Ref("metricNamespace"),
MetricName=Join("", [Ref("springProfile"), "-", "plan-reconciliation-fiduciary-step"]),
MetricValue="1")]
))
I'm trying to create a CloudFormation template that'll deploy a Lambda function, And I need the security options to be optional parameters.
I was able to partially accomplish this using the question here:
How to make a whole object in CloudFormation templates optional?
Interestingly, that method worked great to make the VpcConfig property optional in the AWS GUI Console, but it did NOT work to make it optional for the CLI. And unfortunately, I need it to work in the CLI, since I'll be using CodeBuild to call and deploy this template's resources.
Here are the relevant parameters:
"SecurityGroupIds" : {
"Type" : "CommaDelimitedList",
"Description" : "A list of one or more security groups IDs in the VPC that includes the resources to which your Lambda function requires access."
},
"SubnetIds" : {
"Type" : "CommaDelimitedList",
"Description" : "A list of one or more subnet IDs in the VPC that includes the resources to which your Lambda function requires access."
}
And conditions:
"HasVPC": {"Fn::And": [{"Fn::Not": [{"Fn::Equals": [{"Fn::Join": ["", {"Ref": "SubnetIds"}]}, ""]}]}, {"Fn::Not": [{"Fn::Equals": [{"Fn::Join": ["", {"Ref": "SecurityGroupIds"}]}, ""]}]}]}
And here's where that condition is used in the Lambda resource being defined in the Resources section of the template:
"VpcConfig": {
"Fn::If": [
"HasVPC",
{
"SecurityGroupIds" : {"Ref": "SecurityGroupIds"},
"SubnetIds" : {"Ref": "SubnetIds"}
},
{ "Ref":"AWS::NoValue" }
]
},
When I issue the command to deploy this stack in the CLI, I get the following error:
An error occurred (ValidationError) when calling the CreateChangeSet
operation: Parameters: [SecurityGroupIds, SubnetIds] must have values
Here's the AWS CLI command I'm issuing, from the same directory in which the template is located. Note: the ARN values have all been heavily modified to not be real values from my account, but I kept them in the right format so you can see the real format of the command:
aws cloudformation deploy --template-file lambda-template.json --stack-name "CLI-lambda-stack" --parameter-overrides S3BucketName="myBucket" S3FileLocation="lambda_function.zip" S3ObjectVersion="ZuB0iueEghOyh5q00.DiykLNudujdsc5" DeadLetterArn="arn:aws:sns:us-west-2:577898337216:CloudFormationTests" EnvironmentVariable="testing" KmsKeyArn="arn:aws:kms:us-west-2:504398934246:key/b24e7b72-a94d-6a3e-b848-165115c86212" HandlerFunctionName="lambda_function.lambda_handler" MemorySize="128" Role="arn:aws:iam::102893937243:role/serverless-test-default-us-east-1-lambdaRole" FuncName="myCLILambda"
You are not providing SecurityGroupIds neither SubnetIds default values and your are not providing them on your --parameter-overrides. Therefore, CloudFormation doesn't know how to process them if no values are provided.
Adding the Default statement should do the trick:
{
"Parameters" : {
"SecurityGroupIds" : {
"Type" : "CommaDelimitedList",
"Description" : "A list of one or more security groups IDs in the VPC that includes the resources to which your Lambda function requires access.",
"Default" : ""
},
"SubnetIds" : {
"Type" : "CommaDelimitedList",
"Description" : "A list of one or more subnet IDs in the VPC that includes the resources to which your Lambda function requires access.",
"Default" : ""
}
}
Is there a way to tell cfn-init to use a custom JSON file loaded from disk? That way I can quickly troubleshoot problems, otherwise the only way to change the AWS::CloudFormation::Init section is to delete the stack and create it anew.
I'd rather just make changes to my template.json, then tell something like cfn-init -f C:\template.json ... so it uses C:\template.json instead of getting the template from 169.254.169.254 or however it gets it.
Turns out cfn-init does accept a file, but the file needs to have the "Metadata" key as the root key.
So if we save this to template.json:
{
"AWS::CloudFormation::Init": {
"config": {
"files": {
"C:\\Users\\Administrator\\Desktop\\test-file": {
"source": "https://s3.us-east-2.amazonaws.com/example-bucket/example-file"
}
}
}
},
"AWS::CloudFormation::Authentication": {
"S3AccessCreds": {
"type": "S3",
"buckets": ["example-bucket"],
"roleName": "s3access"
}
}
}
Then we can execute cfn-init -v --region us-east-2 template.json.
Note: Do not include the stack or resource, if you use cfn-init -v -s my_stack -r my_instance --region us-east-2 template.json you will get:
Error: You cannot specify more than one input source for metadata
If you put the entire template file instead of just "Metadata" as root, you will get:
Could not find 'AWS::CloudFormation::Init' key in template.json
If I have two cloudformation stacks, how do I references a resource in one stack from the other stack?
In the example below I have a stack that creates an EBS volume and want to reference that via the Ref: key in the second stack for my EC2 instance but I keep getting a rollback since it can't see that resource from the first stack:
"Template format error: Unresolved resource dependencies"
I already tried the DependsOn clause but it didn't work. Do I need to pass information via Parameters?
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"CubesNetworking": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3.amazonaws.com/mybucket/cf_network.json"
}
},
"CubesInstances": {
"DependsOn": ["CubesNetworking"],
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3.amazonaws.com/mybucket/cf_instances.json"
}
}
}
}
In each of your nested stacks, you should have an output section. Then you can get those values in your calling stack (the one you have listed above) with syntax like:
{ "Fn::GetAtt" : [ "CubesNetworking", "Outputs.VolumeID" ] }
You then pass the values into your other nested stacks via Parameters:
"Parameters" : {
"VolumeId" : { "Fn::GetAtt" : [ "CubesNetworking", "Outputs.VolumeID" ] }
You still want the DependsOn since you need the volume created before the instance.
Edit, Mid-2017:
CloudFormation has introduced the ability to export values from one stack, and reference them in other stacks that do not have to be nested.
So your output can specify an export:
Outputs:
Desc:
Value: !Ref CubesNetworking.VolumeID
Export:
Name: some-unique-name
Then in another stack:
Fn::ImportValue: some-unique-name
We wanted to use company specific Tags to the resources that we create in AWS for billing purposes. I am using a cloud formation template to spin up our Elasticbeanstalk instance and other project dependent resources. When I use the CloudFormation console to create a stack it asks me for Tags in the page after parameters. I have to manually input the Tags for that stack. However is there a way to specify those Tags (Tags for the stack) with in the cloud formation template itself? That way the Tag gets propagated to the other resources? I know that the cloud formation automatically tags the resources with the stack name. But we need company specific tags to bill separate departments.
In the template anatomy, you can't set stack-level tags directly. However you can create a wrapper template, having a single resource of AWS::CloudFormation::Stack.
You can define stack-level tags on that resource:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "WrapperTemplate",
"Resources": {
"WrappedStackWithStackLevelTags": {
"Type" : "AWS::CloudFormation::Stack",
"Properties" : {
"Tags" : [ { "Key" : "Stage", "Value" : "QA" } ],
"TemplateURL" : "your-original-template-s3-url"
}
}
}
}
When launching AWS CloudFormation, the tags being requested will be applied to the CloudFormation Stack itself and (where possible) will also be propagated to the resources launched by the Stack.
These tags can be passed to the CreateStack API call, or from the CLI:
See: create-stack CLI documentation
These tags are applied to the whole Stack and aren't included in the CloudFormation template.
However, CloudFormation templates can include tags for specific resources that are being created. For example, when launching Amazon EC2 instances, tags can be included in the template:
"MyInstance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"SecurityGroups" : [{ "Ref" : "MySecurityGroup" }],
"AvailabilityZone" : "us-east-1a",
"ImageId" : "ami-20b65349",
"Volumes" : [{
"VolumeId" : { "Ref" : "MyEBS" },
"Device" : "/dev/sdk"
}],
"Tags" : [{
"Key" : "Stage",
"Value" : "QA"
}]
}
}
Contrary to what #lalyos says, you don't need to use nested stacks for this, just provide the tags that should apply to all resources as stack level tags.
These stack-level tags can be specified whether running the stack on the console or via CLI.
CLI example:
aws cloudformation create-stack --stack-name my-stack-name \
--template-body file://path-to-template-file.yaml \
--parameters ParameterKey=param1key,ParameterValue=param1value \
--tags Key=tag1key,Value=tag1value \
Key=tag2key,Value=tag2value \
Key=tag3key,Value=tag3value
... and generally add as many tags as you need, using the same format and allowing spaces between tag key-value pairs
See:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-console-add-tags.html
You can build and deploy a CloudFormation template using aws-sam-cli. This command reads a samconfig.toml file where you can declare tags for all the resources of the stack (including CloudFormation stack itself)
Your samconfig.toml should look like:
[default.deploy.parameters]
stack_name = "your-application"
s3_bucket = "your-s3-for-cloudformation-stuff"
s3_prefix = "your-folder-name"
...
tags = "Stage=\"QA\""
and then run:
sam build --template <your-cloudformation-template.yml> && sam deploy
You don't need any wrapper..
You can add tags to the stack on when you create/update it:
In Console:
You can also use the aws cli:
aws cloudformation create-stack help
--tags (list)
Key-value pairs to associate with this stack. CloudFormation also
propagates these tags to supported resources in the stack. You can
specify a maximum number of 50 tags.
If you don't specify this parameter, CloudFormation doesn't modify
the stack's tags. If you specify an empty value, CloudFormation re-
moves all associated tags.
(structure)
The Tag type enables you to specify a key-value pair that can be
used to store information about an CloudFormation stack.
Key -> (string)
Required . A string used to identify this tag. You can spec-
ify a maximum of 128 characters for a tag key. Tags owned by
Amazon Web Services (Amazon Web Services) have the reserved
prefix: aws: .
Value -> (string)
Required . A string containing the value for this tag. You
can specify a maximum of 256 characters for a tag value.
Shorthand Syntax:
Key=string,Value=string ...
JSON Syntax:
[
{
"Key": "string",
"Value": "string"
}
...
]