I'm trying to pass in a list of Subnet IDs to a Cloud formation template, but I get an error. What have I done wrong?
Parameter in template:
ClusterSubnets:
Description: Subnets, in the same VPC where cluster ec2 instances reside.
Typically private. Use mutiples, each in a different AZ for HA.
Type: "List<AWS::EC2::Subnet::Id>"
Parameter snippet from json file:
{
"ParameterKey": "ClusterSubnets",
"ParameterValue": [ "subnet-8fc8c4f7" ]
},
Results In:
Parameter validation failed:
Invalid type for parameter Parameters[1].ParameterValue, value: [u'subnet-8fc8c4f7'], type: type 'list', valid types: type 'basestring'
I know this thread is a year old, an I am assuming you were using the aws cli to create the stack, but I was having the same issue and finally stumbled on the answer. Giving credit where its due:
https://github.com/aws/aws-cli/issues/2478#issuecomment-427167512
escape the "," with a backslash and it will solve the issue. In my case this was in a jekins job bashing out so I had to double backslash:
sh "aws cloudformation create-stack --template-url ${TEMPLATE} \
--stack-name $NAME \
--capabilities CAPABILITY_IAM \
--parameters ParameterKey=SecurityGroups,ParameterValue=\"$SG_GROUP2\\,$SG_GROUP1\"; \
aws cloudformation wait stack-create-complete --stack-name $CLUSTER_NAME"
You should specify the ParameterValue using a comma-delimited list of the subnet IDs you want to include, without spaces. See the section under "list" here:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html
Related
Is it possible to fetch instances details like instances id, vpc id, key-pair, sg group etc from cloudformation template from cfn-get-metadata please assist.
If it possible please share ..
cfn-get-metadata only gets info from the metadata section of the cloudformation template. In cfn-init, you can use values from params and maps.
You can use common cloud formation features to refer to the template like !FindInMap or you can use ${}.
Check out this snippet
owner: !FindInMap [ nodes, !Ref nodeType, userName ]
group: !FindInMap [ nodes, !Ref nodeType, userName ]
configure:
commands:
configure_service:
command: /opt/app/configure.sh
test: "test ! -e /etc/systemd/system/${nodeType}.service"
If you need some data from the stack then you could add Outputs for the items you need and run describe stack in your user data.
aws cloudformation describe-stacks --stack-name myteststack
I have a working EKS Cluster with some services running in there. The challenge right now, is to have an API gateways to call those services. That is why I started reading "Integrate Amazon API Gateway with Amazon EKS" explaining that this is possible but there are some steps to do in order to make this work. This is what I did but after some progress, I ran into an issue.
Here is one of the command-line statements I executed.
aws apigatewayv2 get-vpc-links --region $AGW_AWS_REGION
This results in the following output:
Apparently, the VPC link is available, a security group Id is available, two SubnetIds are available (in the documented example it is three but that should not matter so much, feel free to comment about this) and the status message says that the available VPC link is ready to route traffic.
There for I decided to continue with the next documented step:
Create an API with VPC Link integration
cat > apigw-api.yaml<<EOF
apiVersion: apigatewayv2.services.k8s.aws/v1alpha1
kind: API
metadata:
name: apitest-private-nlb
spec:
body: '{
"openapi": "3.0.1",
"info": {
"title": "ack-apigwv2-import-test-private-nlb",
"version": "v1"
},
"paths": {
"/\$default": {
"x-amazon-apigateway-any-method" : {
"isDefaultRoute" : true,
"x-amazon-apigateway-integration" : {
"payloadFormatVersion" : "1.0",
"connectionId" : "$(kubectl get vpclinks.apigatewayv2.services.k8s.aws \
nlb-internal \
-o jsonpath="{.status.vpcLinkID}")",
"type" : "http_proxy",
"httpMethod" : "GET",
"uri" : "$(aws elbv2 describe-listeners \
--load-balancer-arn $(aws elbv2 describe-load-balancers \
--region $AGW_AWS_REGION \
--query "LoadBalancers[?contains(DNSName, '$(kubectl get service authorservice \
-o jsonpath="{.status.loadBalancer.ingress[].hostname}")')].LoadBalancerArn" \
--output text) \
--region $AGW_AWS_REGION \
--query "Listeners[0].ListenerArn" \
--output text)",
"connectionType" : "VPC_LINK"
}
}
},
"/meta": {
"get": {
"x-amazon-apigateway-integration": {
"uri" : "$(aws elbv2 describe-listeners \
--load-balancer-arn $(aws elbv2 describe-load-balancers \
--region $AGW_AWS_REGION \
--query "LoadBalancers[?contains(DNSName, '$(kubectl get service echoserver \
-o jsonpath="{.status.loadBalancer.ingress[].hostname}")')].LoadBalancerArn" \
--output text) \
--region $AGW_AWS_REGION \
--query "Listeners[0].ListenerArn" \
--output text)",
"httpMethod": "GET",
"connectionId": "$(kubectl get vpclinks.apigatewayv2.services.k8s.aws \
nlb-internal \
-o jsonpath="{.status.vpcLinkID}")",
"type": "HTTP_PROXY",
"connectionType": "VPC_LINK",
"payloadFormatVersion": "1.0"
}
}
}
},
"components": {}
}'
EOF
Unfortunately, this resulted in the following errors:
I tried to isolate the problem but then I ran into another issue. In the command-line, I just executed a small fragment of what is shown above.
aws elbv2 describe-load-balancers --load-balancer-arn --region $AGW_AWS_REGION query "LoadBalancers[?contains(DNSName, '$(kubectl get service authorservice -o jsonpath="{.status.loadBalancer.ingress[].hostname}")')].LoadBalancerArn" --output text
but that resulted in another error:
An error occurred (ValidationError) when calling the
DescribeLoadBalancers operation: 'query' is not a valid load balancer
ARN
So how to get this working? All I want is to just to create an API with VPC link integration. But the documented way to do so does not work for me. Please let me know how I can fix this way or just do the same in a different way.
I am exactly on the same step - trying to create API for micro service on EKS following the same tutorial. I also started to break down the executions to the small chunks to check if things are still correct from the time the tutorial was written.
So taking the smallest piece which should be executable
kubectl get service authorservice -o jsonpath="{.status.loadBalancer.ingress[].hostname}"
You must have the "authorservice" already configured and running or it might be called something else for your particular case.
For me that lines give me back the needed hostname. If it is not giving you the hostname you better check with
kubectl get service <the_service_you_have> -o json
whether the service you have is properly configured and having at the end of the output something similar to
"status": {
"loadBalancer": {
"ingress": [
{
"hostname": "k8s-XXXXXXXXXXXXXXXXXXXXXX.elb.us-west-2.amazonaws.com"
}
]
}
My advice is to go over all the "kubectl" executions and check if they are working and giving back some meaningful results for your case and adjust if needed. At first it might be hard to get where the "kubectl" commands ends but later it gets easy.
is there a way to list/describe all the tags in all the services in an aws environment with one command?
E.g. If I wanted to list the tags in ec2 instances, I can do this aws ec2 describe-tags and can see all the tags. Now if I wanted to view, say, eks tags I'd have to do something like aws eks list-tags-for-resource ... or something similar. So my question is if there is a "magic" where I could do aws * describe-tags and see all the tags in all services, instead of going one by one for each service like so: aws <service name> <command to describe/list tags>
You can use : aws resourcegroupstaggingapi get-resources
From the docs:
Returns all the tagged or previously tagged resources that are located
in the specified Region for the AWS account.
Usage:
aws resourcegroupstaggingapi get-resources
Output:
{
"ResourceTagMappingList": [
{
"ResourceARN": " arn:aws:inspector:us-west-2:123456789012:target/0-nvgVhaxX/template/0-7sbz2Kz0",
"Tags": [
{
"Key": "Environment",
"Value": "Production"
}
]
}
]
}
I would like to clone a cloudformation stack in the same region. Is this possible today using the Cloudformation console?
I have a cloudformation template that takes in a big list of parameters. Many times I want to create an identical stack with just a different stack name. Is there a quick way of doing this using the AWS console?
I am thinking of something along the lines of "Launch more like this" option in EC2.
I use the following shell script
#!/bin/sh -x
# debug
if [ -z "$1" ] ; then
STACK=jirithhu-monitorStack-1ALS8UFQP3SRV
else
STACK="$1"
fi
set -e
# parameter 1: stack ID : (example: hhu-monitorStack-1ALS8UFQP3SRV )
aws cloudformation describe-stacks --stack-name $STACK | jq .Stacks[0].Parameters > /tmp/xx.$$.pars
aws cloudformation get-template --stack-name $STACK | jq -rc .TemplateBody > /tmp/xx.$$.body
NEWNAME=`echo "COPY-$STACK"`
aws cloudformation create-stack --stack-name $NEWNAME \
--template-body file:///tmp/xx.$$.body\
--parameters "`cat /tmp/xx.$$.pars`" \
--capabilities '[ "CAPABILITY_NAMED_IAM", "CAPABILITY_AUTO_EXPAND" ]'
rm /tmp/xx.$$*
you can create an input file and quickly launch a new stack.
Ex:
Command:
aws cloudformation create-stack --stackname startmyinstance
--template-body file://startmyinstance.json
--parameters file://startmyinstance-parameters.json
Parameters file:
[
{
"ParameterKey": "KeyPairName",
"ParameterValue": "MyKey"
},
{
"ParameterKey": "InstanceType",
"ParameterValue": "m1.micro"
}
]
Ref: https://aws.amazon.com/blogs/devops/passing-parameters-to-cloudformation-stacks-with-the-aws-cli-and-powershell/
We wanted to use company specific Tags to the resources that we create in AWS for billing purposes. I am using a cloud formation template to spin up our Elasticbeanstalk instance and other project dependent resources. When I use the CloudFormation console to create a stack it asks me for Tags in the page after parameters. I have to manually input the Tags for that stack. However is there a way to specify those Tags (Tags for the stack) with in the cloud formation template itself? That way the Tag gets propagated to the other resources? I know that the cloud formation automatically tags the resources with the stack name. But we need company specific tags to bill separate departments.
In the template anatomy, you can't set stack-level tags directly. However you can create a wrapper template, having a single resource of AWS::CloudFormation::Stack.
You can define stack-level tags on that resource:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "WrapperTemplate",
"Resources": {
"WrappedStackWithStackLevelTags": {
"Type" : "AWS::CloudFormation::Stack",
"Properties" : {
"Tags" : [ { "Key" : "Stage", "Value" : "QA" } ],
"TemplateURL" : "your-original-template-s3-url"
}
}
}
}
When launching AWS CloudFormation, the tags being requested will be applied to the CloudFormation Stack itself and (where possible) will also be propagated to the resources launched by the Stack.
These tags can be passed to the CreateStack API call, or from the CLI:
See: create-stack CLI documentation
These tags are applied to the whole Stack and aren't included in the CloudFormation template.
However, CloudFormation templates can include tags for specific resources that are being created. For example, when launching Amazon EC2 instances, tags can be included in the template:
"MyInstance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"SecurityGroups" : [{ "Ref" : "MySecurityGroup" }],
"AvailabilityZone" : "us-east-1a",
"ImageId" : "ami-20b65349",
"Volumes" : [{
"VolumeId" : { "Ref" : "MyEBS" },
"Device" : "/dev/sdk"
}],
"Tags" : [{
"Key" : "Stage",
"Value" : "QA"
}]
}
}
Contrary to what #lalyos says, you don't need to use nested stacks for this, just provide the tags that should apply to all resources as stack level tags.
These stack-level tags can be specified whether running the stack on the console or via CLI.
CLI example:
aws cloudformation create-stack --stack-name my-stack-name \
--template-body file://path-to-template-file.yaml \
--parameters ParameterKey=param1key,ParameterValue=param1value \
--tags Key=tag1key,Value=tag1value \
Key=tag2key,Value=tag2value \
Key=tag3key,Value=tag3value
... and generally add as many tags as you need, using the same format and allowing spaces between tag key-value pairs
See:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-console-add-tags.html
You can build and deploy a CloudFormation template using aws-sam-cli. This command reads a samconfig.toml file where you can declare tags for all the resources of the stack (including CloudFormation stack itself)
Your samconfig.toml should look like:
[default.deploy.parameters]
stack_name = "your-application"
s3_bucket = "your-s3-for-cloudformation-stuff"
s3_prefix = "your-folder-name"
...
tags = "Stage=\"QA\""
and then run:
sam build --template <your-cloudformation-template.yml> && sam deploy
You don't need any wrapper..
You can add tags to the stack on when you create/update it:
In Console:
You can also use the aws cli:
aws cloudformation create-stack help
--tags (list)
Key-value pairs to associate with this stack. CloudFormation also
propagates these tags to supported resources in the stack. You can
specify a maximum number of 50 tags.
If you don't specify this parameter, CloudFormation doesn't modify
the stack's tags. If you specify an empty value, CloudFormation re-
moves all associated tags.
(structure)
The Tag type enables you to specify a key-value pair that can be
used to store information about an CloudFormation stack.
Key -> (string)
Required . A string used to identify this tag. You can spec-
ify a maximum of 128 characters for a tag key. Tags owned by
Amazon Web Services (Amazon Web Services) have the reserved
prefix: aws: .
Value -> (string)
Required . A string containing the value for this tag. You
can specify a maximum of 256 characters for a tag value.
Shorthand Syntax:
Key=string,Value=string ...
JSON Syntax:
[
{
"Key": "string",
"Value": "string"
}
...
]