Related
I am getting the following error when trying to use the RegisterThings action from AWS IoT. Within the API, call, I am trying to set the thing groups but getting the following error:
An error occurred (ResourceRegistrationFailureException) when calling the RegisterThing operation: Existing ThingGroups: [] conflict with ThingGroups: [mythingggroup] in Template
Here is my provisioning template:
provision_json = json.dumps({
"Parameters": {
"ThingName": {
"Type": "String"
},
"ThingGroup": {
"Type": "String"
},
"CertificateId": {
"Type": "String"
},
"PolicyName": {
"Type": "String"
},
"AssumeRolePolicyName": {
"Type": "String"
}
},
"Resources": {
"Thing": {
"Type": "AWS::IoT::Thing",
"Properties": {
"ThingName": {
"Ref": "ThingName"
},
"ThingGroups": [{
"Ref": "ThingGroup"
}]
},
"OverrideSettings" : {
"ThingGroups" : "REPLACE"
}
},
"Certificate": {
"Type": "AWS::IoT::Certificate",
"Properties": {
"CertificateId": {
"Ref": "CertificateId"
}
}
},
"DevicePolicy": {
"Type": "AWS::IoT::Policy",
"Properties": {
"PolicyName": {
"Ref": "PolicyName"
}
}
},
"TESAssumeRolePolicy": {
"Type": "AWS::IoT::Policy",
"Properties": {
"PolicyName": {
"Ref": "AssumeRolePolicyName"
}
}
}
}
})
then I call the boto3 client like this:
iot = boto3.client("iot")
response = iot.register_thing(
templateBody=provision_json,
parameters={
"CertificateId": "1234",
"ThingName": "mythingname",
"ThingGroup": "mythinggroupname",
"PolicyName": "mypolicyname",
"AssumeRolePolicyName": "anotherpolicyname"
}
)
I thought this might be a permissions error, but even after I give the caller iot:*, I still have the same issue.
AWS isn't providing great documentation on why this error is occurring. Does anyone have any ideas?
I have an arm template to add new virtual network to an existing Event Hub Namespace.
The problem for is that i have to hard code, or ask as paramameter the vnet and subnet addresprefix.
Is there a way to not have to do this or extract those values and use the on the template ?
Vnet exist subnet exist
using reference function get's me the values but i get circular definition error if i use it on resource definition.
Tried to get ip's for vnet and subnet by using reference function but i cant use it in the param or variable and in resource i get Circular dependency error.
Basically i like to to this on a clean template.
"variables": {
"namespaceVirtualNetworkRuleName": "[concat(parameters('eventhubNamespaceName'), concat('/', parameters('vnetRuleName')))]",
"subNetId": "[resourceId('Microsoft.Network/virtualNetworks/subnets/', parameters('vnetRuleName'), parameters('subnetName'))]"
},
resources : [
{
"apiVersion": "2018-01-01-preview",
"name": "[variables('namespaceVirtualNetworkRuleName')]",
"type": "Microsoft.EventHub/namespaces/VirtualNetworkRules",
"properties": {
"virtualNetworkSubnetId": "[variables('subNetId')]"
}
}
]
But i have to add vnet and subnet with correct ip for this to work.
"variables": {
"namespaceVirtualNetworkRuleName": "[concat(parameters('eventhubNamespaceName'), concat('/', parameters('vnetRuleName')))]",
"subNetId": "[resourceId('Microsoft.Network/virtualNetworks/subnets/', parameters('vnetRuleName'), parameters('subnetName'))]"
},
"resources": [
{
"apiVersion": "2018-01-01-preview",
"name": "[parameters('eventhubNamespaceName')]",
"type": "Microsoft.EventHub/namespaces",
"location": "[parameters('location')]",
"sku": {
"name": "Standard",
"tier": "Standard"
},
"properties": { }
},
{
"apiVersion": "2017-09-01",
"name": "[parameters('vnetRuleName')]",
"location": "[parameters('location')]",
"type": "Microsoft.Network/virtualNetworks",
"properties": {
"addressSpace": {
"addressPrefixes": [
"a.b.c.0/24",
"x.y.0.0/16"
]
},
"subnets": [
{
"name": "[parameters('subnetName')]",
"properties": {
"addressPrefix": "x.y.z.w/26",
"serviceEndpoints": [
{
"service": "Microsoft.EventHub"
}
]
}
}
]
}
},
{
"apiVersion": "2018-01-01-preview",
"name": "[variables('namespaceVirtualNetworkRuleName')]",
"type": "Microsoft.EventHub/namespaces/VirtualNetworkRules",
"dependsOn": [
"[concat('Microsoft.EventHub/namespaces/', parameters('eventhubNamespaceName'))]"
],
"properties": {
"virtualNetworkSubnetId": "[variables('subNetId')]"
}
}
]
Any idea on hou to remove the vnet part or at list get the ip's before the template and pass them as param ?
Found out how to make this work.
the solution is this :
"variables": {
},
"resources": [
{
"apiVersion": "2018-01-01-preview",
"name": "[parameters('eventhubNamespaceName')]",
"type": "Microsoft.EventHub/namespaces",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard",
"tier": "Standard"
},
"properties": { }
},
{
"apiVersion": "2018-01-01-preview",
"name": "[concat(parameters('eventhubNamespaceName'), concat('/', parameters('vnetSubscriptioID'), parameters('vnetResorceGroupName'), parameters('vnetRuleName'), parameters('subnetName')[copyIndex('subnetcopy')]))]",
"type": "Microsoft.EventHub/namespaces/VirtualNetworkRules",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.EventHub/namespaces/', parameters('eventhubNamespaceName'))]"
],
"properties": {
"virtualNetworkSubnetId": "[resourceId(parameters('vnetSubscriptioID'), parameters('vnetResorceGroupName'),'Microsoft.Network/virtualNetworks/subnets/', parameters('vnetRuleName'), parameters('subnetName')[copyIndex('subnetcopy')])]"
},
"copy": {
"name": "subnetcopy",
"count": "[length(parameters('subnetName'))]"
}
}
]
}
At the end no network part was needed. Updated my solution to be able to deploy more than one subnet at a time.
I have created S3 Bucket with deletepolicy retain using cloud formation, I Have exported the created bucket using Export in outputs in cloudformation.
Now I want to use the same s3 bucket in another stack using import
Cloud formation for s3:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Creates an S3 bucket to be used for static content/website hosting.",
"Parameters": {
"AssetInsightId": {
"Description": "Asset Insight ID",
"Type": "String",
"Default": "206153"
},
"ResourceOwner": {
"Description": "tr:resource-owner",
"Type": "String",
"Default": "####"
},
"EnvironmentType": {
"Description": "tr:environment-type",
"Default": "preprod",
"Type": "String",
"AllowedValues": ["preprod", "prod"],
"ConstraintDescription": "must specify preprod, prod."
}
},
"Resources": {
"S3Bucket": {
"Type": "AWS::S3::Bucket",
"DeletionPolicy": "Retain",
"Properties": {
"BucketName": {
"Fn::Sub": "a${AssetInsightId}-s3bucket-${EnvironmentType}"
},
"Tags": [{
"Key": "tr:application-asset-insight-id",
"Value": {
"Fn::Sub": "${AssetInsightId}"
}
}, {
"Key": "tr:environment-type",
"Value": {
"Fn::Sub": "${EnvironmentType}"
}
}
]
}
}
},
"Outputs": {
"S3Bucket": {
"Description": "Information about the value",
"Description": "Name of the S3 Resource Bucket",
"Value": "!Ref S3Bucket",
"Export": {
"Name": "ExportS3Bucket"
}
}
}
}
cloud formation to use created s3 bucket from another template with import
Second template :
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Creates an S3 apigateway to be used for static content/website hosting.",
"Parameters": {
"AssetInsightId": {
"Description": "Asset Insight ID",
"Type": "String",
"Default": "206153"
},
"ResourceOwner": {
"Description": "tr:resource-owner",
"Type": "String",
"Default": "swathi.koochi#thomsonreuters.com"
},
"EnvironmentType": {
"Description": "tr:environment-type",
"Default": "preprod",
"Type": "String",
"AllowedValues": ["preprod", "prod"],
"ConstraintDescription": "must specify preprod, prod."
},
"endpointConfiguration": {
"Description": "tr:endpoint-configuration",
"Default": "REGIONAL",
"Type": "String",
"AllowedValues": ["REGIONAL", "EDGE"],
"ConstraintDescription": "must specify REGIONAL, EDGE."
}
},
"Resources": {
"S3BucketImport": {
"Type": "AWS::S3::Bucket",
"Properties": {
"BucketName": {"Fn::ImportValue" : "ExportS3Bucket"}
}
},
"APIGateWayRestResourceRestApi": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Name": "MyAPI",
"Description": "API Gateway rest api with cloud formation",
"EndpointConfiguration": {
"Types": [{
"Ref": "endpointConfiguration"
}
]
}
}
},
"APIGateWayResource": {
"Type": "AWS::ApiGateway::Resource",
"Properties": {
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
},
"ParentId": {
"Fn::GetAtt": ["APIGateWayRestResourceRestApi", "RootResourceId"]
},
"PathPart": "test"
}
},
"APIGatewayPostMethod": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"AuthorizationType": "NONE",
"HttpMethod": "POST",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Sub": "arn:aws:apigateway:us-east-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-2:861756181523:function:GreetingLambda/invocations"
}
},
"MethodResponses": [{
"ResponseModels": {
"application/json": {
"Ref": "PostMethodResponse"
}
},
"StatusCode": 200
}
],
"ResourceId": {
"Ref": "APIGateWayResource"
},
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
}
}
},
"PostMethodResponse": {
"Type": "AWS::ApiGateway::Model",
"Properties": {
"ContentType": "application/json",
"Name": "PostMethodResponse",
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
},
"Schema": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "PostMethodResponse",
"type": "object",
"properties": {
"Email": {
"type": "string"
}
}
}
}
},
"RestApiDeployment": {
"DependsOn": "APIGatewayPostMethod",
"Type": "AWS::ApiGateway::Deployment",
"Properties": {
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
}
}
},
"RestAPIStage": {
"Type": "AWS::ApiGateway::Stage",
"Properties": {
"DeploymentId": {
"Ref": "RestApiDeployment"
},
"MethodSettings": [{
"DataTraceEnabled": true,
"HttpMethod": "*",
"ResourcePath": "/*"
}
],
"RestApiId": {
"Ref": "APIGateWayRestResourceRestApi"
},
"StageName": "Latest"
}
},
"APIGateWayDomainName": {
"Type": "AWS::ApiGateway::DomainName",
"Properties": {
"CertificateArn": {
"Ref": "myCertificate"
},
"DomainName": {
"Fn::Join": [".", [{
"Ref": "AssetInsightId"
}, {
"Ref": "EnvironmentType"
}, "api"]]
},
"EndpointConfiguration": {
"Types": [{
"Ref": "endpointConfiguration"
}
]
}
}
},
"myCertificate": {
"Type": "AWS::CertificateManager::Certificate",
"Properties": {
"DomainName": {
"Fn::Join": [".", [{
"Ref": "AssetInsightId"
}, {
"Ref": "EnvironmentType"
}, "api"]]
}
}
}
}
}
when I/m trying to import using Import Value, I'm getting error saying
S3BucketImport
CREATE_FAILED Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 9387EBE0E472E559; S3 Extended Request ID: o8EbE20IOoUgEMwXc7xVjuoyQT03L/nnQ7AsC94Ff1S/PkE100Imeyclf1BxYeM0avuYjDWILxA=)
As #Jarmod correctly pointed out,
In your first template, export the s3 bucket name using { "Ref" : ",S3Bucket" }
In your second template, you don't have to create the bucket again.you can use the exported value from the first template if you want to refer the bucket name from resources. But i don't see any of the resources in the second template refer the S3 bucket name.
I'm trying to create a nested stack with the root stack looks like this:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"DynamoDBTable": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"Parameters": {
"TableName": {
"Fn::Sub": "${AWS::StackName}"
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/dynamodb.json"
}
},
"S3WebsiteReact": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"Parameters": {
"BucketName": {
"Fn::Sub": "${AWS::StackName}-website"
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/s3-static-website-react.json"
}
},
"S3UploadBucket": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"Parameters": {
"BucketName": {
"Fn::Sub": "${AWS::StackName}-upload"
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/s3-with-cors.json"
}
},
"Cognito": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": "DynamoDBTable",
"Properties": {
"Parameters": {
"CognitoUserPoolName": {
"Fn::Join" : ["",
{
"Fn::Split": ["-", {
"Ref": "AWS::StackName"
}]
}
]
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/cognito.json"
}
},
"ApiGateway": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": ["DynamoDBTable", "Cognito"],
"Properties": {
"Parameters": {
"ApiGatewayName": {
"Fn::Sub": "${AWS::StackName}-api"
},
"CognitoUserPoolArn": {
"Fn::GetAtt": [ "Cognito", "Outputs.UserPoolArn" ]
},
"DynamoDBStack": {
"Fn::GetAtt": [ "DynamoDBTable", "Outputs.DDBStackName" ]
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/api-gateway.json"
}
},
"IdentityPool": {
"Description": "Cognito Identity Pool. Must be created after User Pool and API Gateway.",
"Type": "AWS::Cognito::IdentityPool",
"DependsOn": ["Cognito", "ApiGateway", "S3UploadBucket"],
"Properties": {
"Parameters": {
"AppClientId": {
"Fn::GetAtt": [ "Cognito", "Outputs.AppClientId" ]
},
"UserPoolProviderName": {
"Fn::GetAtt": [ "Cognito", "Outputs.ProviderName" ]
},
"UserPoolName": {
"Fn::GetAtt": [ "Cognito", "Outputs.UserPoolName" ]
},
"UploadBucketName": {
"Fn::GetAtt": [ "S3UploadBucket", "Outputs.UploadBucketName" ]
},
"ApiGatewayId": {
"Fn::GetAtt": [ "ApiGateway", "Outputs.ApiGatewayId" ]
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/identity-pool.json"
}
}
},
"Outputs": {
}
}
And I get this error:
2019-06-19 14:45:14 UTC-0400 IdentityPool CREATE_FAILED Property validation failure: [Encountered unsupported properties in {/}: [TemplateURL, Parameters]]
It looks like my identity pool stack has some issues with the parameters. But the identity pool stack parameters look like this:
"Parameters" : {
"AppClientId": {
"Description": "ID of the App Client of the Cognito User Pool passed into this stack.",
"Type": "String"
},
"UserPoolProviderName": {
"Description": "Cognito User Pool Provider name passed into this stack.",
"Type": "String"
},
"UserPoolName": {
"Description": "Cognito User Pool Name passed into this stack.",
"Type": "String"
},
"UploadBucketName": {
"Description": "Name of the bucket that is used to upload files to.",
"Type": "String"
},
"ApiGatewayId": {
"Description": "ID of the API Gateway created for the stack.",
"Type": "String"
}
},
The funny thing is: I tried creating each stack on its own, then passed the outputs from them as parameters to the stacks that need those parameters and every single stack was created successfully without any problems.
I've tried to look for what is unsupported but was unable to find any answers.
The error:
[Encountered unsupported properties in {/}: [TemplateURL, Parameters]]
Says that those two properties are unsupported. Unlike all the rest of the resources declared in your template which also use those two properties, this resource is a AWS::Cognito::IdentityPool, while the rest are all of type AWS::CloudFormation::Stack.
Those two properties are only valid on the AWS::CloudFormation::Stack type, hence the validation error.
I want to use CloudFormation to trigger Lambda when my CloudWatch function is called. I have the below, but it does not work.
CloudWatch rule created fine
"CloudWatchNewEc2": {
"Type": "AWS::Events::Rule",
"DependsOn": ["LambdaNewEc2"],
"Properties": {
"Description": "Triggered on new EC2 instances",
"EventPattern": {
"source": [
"aws.ec2"
],
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": [
"ec2.amazonaws.com"
],
"eventName": [
"RunInstances"
]
}
},
"Targets": [
{
"Arn": {
"Fn::GetAtt": ["LambdaNewEc2", "Arn"]
},
"Id": "NewEc2AutoTag"
}
]
}
},
Lambda created but is not triggered
"LambdaNewEc2": {
"Type": "AWS::Lambda::Function",
"DependsOn": ["S3Lambda", "IAMRoleLambda"],
"Properties": {
"Code": {
"S3Bucket": {"Ref": "LambdaBucketName"},
"S3Key": "skynet-lambda.zip"
},
"Description": "When new EC2 instances are created, auto tag them",
"FunctionName": "newEc2AutoTag",
"Handler": "index.newEc2_autoTag",
"Role": {"Fn::GetAtt": ["IAMRoleLambda", "Arn"]},
"Runtime": "nodejs6.10",
"Timeout": "30"
}
}
},
It seems like CloudWatch Target is not sufficient?
UPDATE (Full CloudFormation template)
{
"Parameters": {
"Environment": {
"Type": "String",
"Default": "Staging",
"AllowedValues": [
"Testing",
"Staging",
"Production"
],
"Description": "Environment name"
},
"BucketName": {
"Type": "String",
"Default": "skynet-staging",
"Description": "Bucket Name"
},
"LambdaBucketName": {
"Type": "String",
"Default": "skynet-lambda",
"Description": "Lambda Bucket Name"
},
"Owner": {
"Type": "String",
"Description": "Owner"
}
},
"Resources": {
"S3Web": {
"Type": "AWS::S3::Bucket",
"Properties": {
"BucketName": {
"Ref": "BucketName"
},
"WebsiteConfiguration": {
"IndexDocument": "index.html",
"RoutingRules": [
{
"RedirectRule": {
"ReplaceKeyPrefixWith": "#"
},
"RoutingRuleCondition": {
"HttpErrorCodeReturnedEquals": "404"
}
}
]
},
"AccessControl": "PublicRead",
"Tags": [
{
"Key": "Cost Center",
"Value": "Skynet"
},
{
"Key": "Environment",
"Value": {
"Ref": "Environment"
}
},
{
"Key": "Owner",
"Value": {
"Ref": "Owner"
}
}
]
}
},
"S3Lambda": {
"Type": "AWS::S3::Bucket",
"Properties": {
"BucketName": {
"Ref": "LambdaBucketName"
},
"VersioningConfiguration": {
"Status": "Enabled"
},
"Tags": [
{
"Key": "Cost Center",
"Value": "Skynet"
},
{
"Key": "Owner",
"Value": {
"Ref": "Owner"
}
}
]
}
},
"CloudWatchNewEc2": {
"Type": "AWS::Events::Rule",
"DependsOn": ["LambdaNewEc2"],
"Properties": {
"Description": "Triggered on new EC2 instances",
"EventPattern": {
"source": [
"aws.ec2"
],
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": [
"ec2.amazonaws.com"
],
"eventName": [
"RunInstances"
]
}
},
"Targets": [
{
"Arn": {
"Fn::GetAtt": ["LambdaNewEc2", "Arn"]
},
"Id": "NewEc2AutoTag"
}
]
}
},
"IAMRoleLambda": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": "skynet-lambda-role",
"AssumeRolePolicyDocument": {
"Version" : "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [ "lambda.amazonaws.com" ]
},
"Action": [ "sts:AssumeRole" ]
}
]
},
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/AmazonEC2FullAccess",
"arn:aws:iam::aws:policy/AWSLambdaFullAccess",
"arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess",
"arn:aws:iam::aws:policy/CloudWatchLogsFullAccess"
]
}
},
"LambdaNewEc2": {
"Type": "AWS::Lambda::Function",
"DependsOn": ["S3Lambda", "IAMRoleLambda"],
"Properties": {
"Code": {
"S3Bucket": {"Ref": "LambdaBucketName"},
"S3Key": "skynet-lambda.zip"
},
"Description": "When new EC2 instances are created, auto tag them",
"FunctionName": "newEc2AutoTag",
"Handler": "index.newEc2_autoTag",
"Role": {"Fn::GetAtt": ["IAMRoleLambda", "Arn"]},
"Runtime": "nodejs6.10",
"Timeout": "30"
}
}
},
"Outputs": {
"WebUrl": {
"Value": {
"Fn::GetAtt": [
"S3Web",
"WebsiteURL"
]
},
"Description": "S3 bucket for web files"
}
}
}
I managed to deploy your template into a CloudFormation stack (by removing the LambdaBucket and pointing to my own zip file). It seems to create all resource correctly.
It took about 10 minutes for the RunInstances event to appear in CloudTrail. It then successfully triggered the Rule, but the CloudWatch metrics for my rule showed a failed invocation because I faked a Lambda function for your template.
Once I edited the rule to point to a better function and re-tested, it worked fine.
Bottom line: Seems to work!