How to add AWS IoT provisioning template in Cloudformation template / CDK - amazon-web-services

I am using Cloudformation template to create a stack including IoT fleet provisioning template and according to the document the IoT provisioning template body should be string type.
I have the IoT fleet provisioning template like this:
{
"Parameters": {
"SerialNumber": {
"Type": "String"
},
"AWS::IoT::Certificate::Id": {
"Type": "String"
}
},
"Resources": {
"certificate": {
"Properties": {
"CertificateId": {
"Ref": "AWS::IoT::Certificate::Id"
},
"Status": "Active"
},
"Type": "AWS::IoT::Certificate"
},
"policy": {
"Properties": {
"PolicyName": "mypolicy"
},
"Type": "AWS::IoT::Policy"
},
"thing": {
"OverrideSettings": {
"AttributePayload": "MERGE",
"ThingGroups": "REPLACE",
"ThingTypeName": "REPLACE"
},
"Properties": {
"AttributePayload": {
"SerialNumber": {
"Ref": "SerialNumber"
}
},
"ThingName": {
"Ref": "SerialNumber"
}
},
"Type": "AWS::IoT::Thing"
}
}
}
The Cloudformation template is like this:
AWSTemplateFormatVersion: '2010-09-09'
Description: "Template to create iot"
Resources:
FleetProvisioningTemplate:
Type: AWS::IoT::ProvisioningTemplate
Properties:
Description: Fleet provisioning template
Enabled: true
ProvisioningRoleArn: "arn:aws:iam::1234567890:role/IoT-role"
TemplateBody: String
TemplateName: mytemplate
I tried to use the JSON string of the IoT provisioning template for the template body but it didn't work. My question is how I can create an IoT provisioning template using Cloudformation template?
update
It turned out I can add the IoT provisioning template as a 'literal block'
AWSTemplateFormatVersion: '2010-09-09'
Description: "Template to create iot"
Resources:
FleetProvisioningTemplate:
Type: AWS::IoT::ProvisioningTemplate
Properties:
Description: Fleet provisioning template
Enabled: true
ProvisioningRoleArn: "arn:aws:iam::1234567890:role/IoT-role"
TemplateBody: |
{
"Parameters": {
"SerialNumber": {
"Type": "String"
},
"AWS::IoT::Certificate::Id": {
"Type": "String"
}
},
"Resources": {
"certificate": {
"Properties": {
"CertificateId": {
"Ref": "AWS::IoT::Certificate::Id"
},
"Status": "Active"
},
"Type": "AWS::IoT::Certificate"
},
"policy": {
"Properties": {
"PolicyName": "cto-full-function-dev"
},
"Type": "AWS::IoT::Policy"
},
"thing": {
"OverrideSettings": {
"AttributePayload": "MERGE",
"ThingGroups": "DO_NOTHING",
"ThingTypeName": "REPLACE"
},
"Properties": {
"AttributePayload": {},
"ThingGroups": [],
"ThingName": {
"Ref": "SerialNumber"
},
"ThingTypeName": "cto"
},
"Type": "AWS::IoT::Thing"
}
}
}
TemplateName: mytemplate
But as soon as I added the PreProvisioningHook as the cloudformation document says, the template fails with invalid request error.
AWSTemplateFormatVersion: '2010-09-09'
Description: "Template to create iot"
Resources:
LambdaHook:
Type: AWS::Lambda::Function
....
FleetProvisioningTemplate:
Type: AWS::IoT::ProvisioningTemplate
Properties:
Description: Fleet provisioning template
Enabled: true
ProvisioningRoleArn: "arn:aws:iam::1234567890:role/IoT-role"
PreProvisioningHook:
TargetArn: {
"Fn::GetAtt": [
"LambdaHook",
"Arn"
]
}
PayloadVersion: "1.0"
TemplateBody: |
{
"Parameters": {
"SerialNumber": {
"Type": "String"
},
"AWS::IoT::Certificate::Id": {
"Type": "String"
}
},
"Resources": {
"certificate": {
"Properties": {
"CertificateId": {
"Ref": "AWS::IoT::Certificate::Id"
},
"Status": "Active"
},
"Type": "AWS::IoT::Certificate"
},
"policy": {
"Properties": {
"PolicyName": "cto-full-function-dev"
},
"Type": "AWS::IoT::Policy"
},
"thing": {
"OverrideSettings": {
"AttributePayload": "MERGE",
"ThingGroups": "DO_NOTHING",
"ThingTypeName": "REPLACE"
},
"Properties": {
"AttributePayload": {},
"ThingGroups": [],
"ThingName": {
"Ref": "SerialNumber"
},
"ThingTypeName": "cto"
},
"Type": "AWS::IoT::Thing"
}
}
}
TemplateName: mytemplate
I also asked question on here but no luck. Did any one have the same issue and fix it?

I finally figured it out but want to share it in case someone is having the same question.
AWS IoT document doesn't mention this but if you want to add a PreProvisioningHook for your provisioning template, you need to give IoT access to the lambda, AKA PreProvisioningHook, so in the Cloudformation template, add something like this:
LambdaAddPermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName: !GetAtt PreProvisionHook.Arn
Principal: iot.amazonaws.com
In the Provisioning Template resource, make sure you have this:
PreProvisioningHook:
PayloadVersion: '2020-04-01'
TargetArn: {
"Fn::GetAtt": [
"PreProvisionHook",
"Arn"
]
}

in CDK you can opt to use a shorthand too:
preProvisioningHookLambda.grantInvoke(new iam.ServicePrincipal('iot.amazonaws.com')) // allow iot to invoke this function
This is the TS code I am using for everyones reference:
import * as cdk from '#aws-cdk/core';
import * as iam from '#aws-cdk/aws-iam';
import * as lambdaNodeJS from '#aws-cdk/aws-lambda-nodejs';
import * as iot from "#aws-cdk/aws-iot";
const props = {
stage: 'development'
}
const PolicyName = "DevicePolicy";
const templateName = 'DeviceProvisioningTemplateV1';
const templateBody = {
Parameters: {
SerialNumber: {
Type: "String"
},
ModelType: {
Type: "String"
},
"AWS::IoT::Certificate::Id": {
Type: "String"
}
},
Resources: {
certificate: {
Properties: {
CertificateId: {
Ref: "AWS::IoT::Certificate::Id"
},
Status: "Active"
},
Type: "AWS::IoT::Certificate"
},
policy: {
Properties: {
PolicyName
},
Type: "AWS::IoT::Policy"
},
thing: {
OverrideSettings: {
AttributePayload: "MERGE",
ThingGroups: "DO_NOTHING",
ThingTypeName: "REPLACE"
},
Properties: {
ThingGroups: [],
ThingName: {
Ref: "SerialNumber"
}
},
Type: "AWS::IoT::Thing"
}
}
};
const preProvisioningHookLambda = new lambdaNodeJS.NodejsFunction(this, `provisioning-hook-lambda-${props?.stage}`, {
entry: './src/lambda/provisioning/hook.ts',
handler: 'handler',
bundling: {
externalModules: [
]
},
timeout: cdk.Duration.seconds(5)
});
preProvisioningHookLambda.grantInvoke(new iam.ServicePrincipal('iot.amazonaws.com')) // allow iot to invoke this function
// Give the AWS IoT service permission to create or update IoT resources such as things and certificates in your account when provisioning devices
const provisioningRole = new iam.Role(this, `provisioning-role-arn-${props?.stage}`, {
assumedBy: new iam.ServicePrincipal('iot.amazonaws.com'),
});
provisioningRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSIoTThingsRegistration'));
new cdk.CfnOutput(this, 'provisioningRoleArn ', { value: provisioningRole.roleArn || 'undefined' });
const provisioningTemplate = new iot.CfnProvisioningTemplate(this, `provisioning-hook-template-${props?.stage}`, {
provisioningRoleArn: provisioningRole.roleArn,
templateBody: JSON.stringify(templateBody),
enabled: true,
templateName,
preProvisioningHook: {
payloadVersion: '2020-04-01',
targetArn: preProvisioningHookLambda.functionArn,
}
});
new cdk.CfnOutput(this, 'preProvisioningLambdaFunctionName ', { value: preProvisioningHookLambda.functionName || 'undefined' });
new cdk.CfnOutput(this, 'provisioningTemplateName ', { value: provisioningTemplate.templateName || 'undefined' });

Base on the answer by Z Wang, this is how you do it in the AWS CDK:
myLambda.addPermission('InvokePermission', {
principal: new ServicePrincipal('iot.amazonaws.com'),
action: 'lambda:InvokeFunction',
});

Related

Property StageName: not defined for resource of type AWS::Serverless::Api

I have seen alot of post say that you can add OpenApiVersion: '2.0' to fix this problem but it does not change anything in my case. For some reason now that I am trying to add a Stage for the first time and run my function locally with sam local start-api I always get Error: [InvalidResourceException('ApiGateway', 'property StageName: not defined for resource of type AWS::Serverless::Api')] as an error. The reason for adding the new stage is because I need it for my new authorizer I am trying to implement. Any ideas why StageName is not defined? Its clearly needed per AWS documentation
template.json
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Description",
"Transform": [
"AWS::Serverless-2016-10-31"
],
"Resources": {
"RapidApiGateway": {
"Type": "AWS::Serverless::Api",
"Properties": {
"StageName:": "Prod",
"MethodSettings": [
{
"HttpMethod": "*",
"ResourcePath": "/*",
"ThrottlingRateLimit": 10,
"ThrottlingBurstLimit": 10
}
],
"Auth": {
"DefaultAuthorizer": "RapidAuthorizer",
"Authorizers": {
"RapidAuthorizer": {
"Fn::GetAtt": [
"RapidAuthFunction",
"attributeName"
],
"Identity": {
"Headers": [
"X-RapidAPI-Proxy-Secret"
]
}
}
}
}
}
},
"RapidAuthFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"CodeUri": "./authorizer",
"Handler": "handler.authorizer",
"Runtime": "nodejs14.x"
}
}
},
"Outputs": {
"WebEndpoint": {
"Description": "API Gateway endpoint URL for Prod stage",
"Value": {
"Fn::Sub": "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
}
}
}
}

AWS::WAFv2::WebACLAssociation ResourceArn for Application Load Balancer in CloudFormation

I have a CloudFormation template which creates an ElasticBeanstalk environment like this:
"ApplicationEnvironment": {
"Type": "AWS::ElasticBeanstalk::Environment",
"Properties": {
"ApplicationName": {
"Ref": "Application"
},
"SolutionStackName": "64bit Amazon Linux 2018.03 v2.11.2 running Java 8",
"VersionLabel": {
"Ref": "AppVersion"
},
"Tier": {
"Name": "WebServer",
"Type": "Standard"
},
"OptionSettings": [
...
{
"Namespace": "aws:elasticbeanstalk:environment",
"OptionName": "EnvironmentType",
"Value": "LoadBalanced"
},
{
"Namespace": "aws:elasticbeanstalk:environment",
"OptionName": "LoadBalancerType",
"Value": "application"
},
...
---
"WAF": {
"Type": "AWS::WAFv2::WebACL",
"Properties": {
"DefaultAction": {
"Type": "BLOCK"
},
"Scope": "REGIONAL",
"VisibilityConfig": {
"CloudWatchMetricsEnabled": "false",
"MetricName": { "Fn::Join": [ "", [ { "Ref": "AWS::StackName" }, "metric-waf" ] ] },
"SampledRequestsEnabled": "false"
},
"Rules": [
{
"Action" : {
"Type" : "BLOCK"
},
"Priority" : 0,
"Statement" : {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesCommonRuleSet"
}
}
}
]
}
},
"WAFAssociation": {
"Type" : "AWS::WAFv2::WebACLAssociation",
"Properties" : {
"ResourceArn" : ???,
"WebACLArn" : { "Ref": "WAF" }
}
}
I intend to associate the Beanstalk ALB with the WebACL but have no idea how to refer to the application load balancer ARN that the template creates. I cannot just put a hardcoded ARN in since it always changes based on what the template creates.
Is there some way I can refer to the ALB ARN in the ResourceArn field? Or do I need to apply the WebACL somewhere in the Beanstalk Option Settings?
I think the only way would be through a custom resource which takes EB env name, uses describe_environment_resources API call to get the EB env info (including LA arn), and returns back to your stuck.
Below is a working example of such a resource which you could add to your template:
LambdaBasicExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonEC2FullAccess
- arn:aws:iam::aws:policy/AWSElasticBeanstalkFullAccess
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
MyCustomResource:
Type: Custom::GetEBLoadBalancerArn
Properties:
ServiceToken: !GetAtt 'MyCustomFunction.Arn'
EBEnvName: !Ref MyEnv
MyCustomFunction:
Type: AWS::Lambda::Function
Properties:
Handler: index.lambda_handler
Description: "Get ARN of EB Load balancer"
Timeout: 30
Role: !GetAtt 'LambdaBasicExecutionRole.Arn'
Runtime: python3.7
Code:
ZipFile: |
import json
import logging
import cfnresponse
import boto3
logger = logging.getLogger()
logger.setLevel(logging.INFO)
eb = boto3.client('elasticbeanstalk')
ec2 = boto3.client('ec2')
def lambda_handler(event, context):
logger.info('got event {}'.format(event))
try:
responseData = {}
if event['RequestType'] in ["Create"]:
eb_env_name = event['ResourceProperties']['EBEnvName']
response = eb.describe_environment_resources(
EnvironmentName=eb_env_name
)
lb_arn = response['EnvironmentResources']['LoadBalancers'][0]['Name']
logger.info(str(response['EnvironmentResources']['LoadBalancers'][0]['Name']))
responseData = {
"LBArn": lb_arn
}
cfnresponse.send(event, context,
cfnresponse.SUCCESS, responseData)
else:
logger.info('Unexpected RequestType!')
cfnresponse.send(event, context,
cfnresponse.SUCCESS, responseData)
except Exception as err:
logger.error(err)
responseData = {"Data": str(err)}
cfnresponse.send(event,context,
cfnresponse.FAILED,responseData)
return
Having the resource you would just use:
"WAFAssociation": {
"Type" : "AWS::WAFv2::WebACLAssociation",
"Properties" : {
"ResourceArn" : { "GetAtt": ["MyCustomResource", "LBArn"] },
"WebACLArn" : { "Ref": "WAF" }
}
}
Update: This answer is wrong. The resources seem to be available in the other EB config files, but no the template itself
The Wrong Answer:
Looks like Cloudformation gives you access to the resources it creates, even though they're not defined directly in your template.
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-format-resources-eb.html
So to make the association to the load balancer just use "Ref": "AWSEBV2LoadBalancer" or "Ref: "AWSEBLoadBalancer" or whatever
"WAFAssociation": {
"Type" : "AWS::WAFv2::WebACLAssociation",
"Properties" : {
"Ref" : "AWSEBV2LoadBalancer",
"WebACLArn" : { "Ref": "WAF" }
}
}

How to configure Batch job queue as targets for AWS::Events::Rule in AWS cloudformation template

My AWS cloudformation template has this:
"ScheduledRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"Description": "ScheduledRule",
"ScheduleExpression": "cron(0/5 * * * ? *)",
"State": "ENABLED",
"Targets": [{
"Here I want to set batch job queue"
}]
}
}
I have created necessary entities for AWS Batch in the template.
"JobDefinition": {
"Type": "AWS::Batch::JobDefinition",
"Properties": {
"Type": "container",
"ContainerProperties": {
"Image": {
"Ref": "ImageUrl"
},
"Vcpus": 2,
"Memory": 2000,
"Command": ["node", "server.js"]
},
"RetryStrategy": {
"Attempts": 1
}
}
},
"JobQueue": {
"Type": "AWS::Batch::JobQueue",
"Properties": {
"Priority": 1,
"ComputeEnvironmentOrder": [
{
"order": 1,
"ComputeEnvironment": { "Ref": "ComputeEnvironment" }
}
]
}
},
"ComputeEnvironment": {
"Type": "AWS::Batch::ComputeEnvironment",
"Properties": {
"Type": "MANAGED",
"ComputeResourses": {
"Type": "EC2",
"MinvCpus": 2,
"DesiredvCpus": 4,
"MaxvCpus": 64,
"InstanceTypes": [
"optimal"
],
"Subnets" : [{ "Ref" : "Subnet" }],
"SecurityGroupIds" : [{ "Ref" : "SecurityGroup" }],
"InstanceRole" : { "Ref" : "IamInstanceProfile" }
},
"ServiceRole" : { "Ref" : "BatchServiceRole" }
}
}
I came to know that it is possible to submit batch job through aws cloudwatch event. AWS cloudwatch event target
I want to use Batch job queue target to submit my job through cloudformation template. I have seen many examples where batch job submission is done through AWS lambda function but I don't want to use lambda function. I didn't find any cloudformation template where Batch job queue target is configured in "AWS::Events::Rule"
Hey I was trying to find a sample myself but didn't find any. With some testing I figured it out. Thought I'd share it here.
Targets:
- Arn:
Ref: BatchProcessingJobQueue
Id: {your_id}
RoleArn: {your_role_arn}
BatchParameters:
JobDefinition:
Ref: BatchProcessingJobDefinition
JobName: {your_job_name}
Input:
!Sub
- '{"Parameters": {"param1": "--param.name1=${param1}", "param2": "--param.name2=${param2}"}}'
- param1: {param1_value}
param2: {param2_value}

nested cloudformation get vpcid on vpc creation

Following several threads on SO and aws forums, I am trying to get a basic nested cloudformation example working.
The ChildStack01 creates a VPC, then ChildStack02 adds a subnet. but after trying several combinations I get the same type of error, Output 'VpcID' not found in stack
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"ChildStack01": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3-eu-west-1.amazonaws.com/cf-templates-1u1ziwem31f87-eu-west-1/xxx",
"TimeoutInMinutes": "60"
}
},
"ChildStack02": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3-eu-west-1.amazonaws.com/cf-templates-1u1ziwem31f87-eu-west-1/yyy",
"Parameters": {
"VpcId" : { "Fn::GetAtt" : [ "ChildStack01", "Outputs.VpcID" ] }
},
"TimeoutInMinutes": "60"
}
}
}
I have tried adding a parameter with
"VPC" : {
"Description" : "VPC ID",
"Type": "AWS::EC2::VPC::Id"
}
but then generates an error as there is no reference value for VPC listed in http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html
Is there a basic way to get the VPC-id after it has been created?
thanks
Art
You should look at Exports.
In your VPC stack, create an Export section:
Outputs:
VpcId:
Value: !Ref VPC
Export:
Name: Unique-VpcId
You can then import that value in another stack:
VpcId:
Fn::ImportValue: Unique-VpcId
You should of course include some way of generating unique export names (they have to be unique within a region) rather than hard-coding as in my example.
In CF Template create VPC, use as below:
"Resources": {
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
....
}
}
},
"Outputs": {
"VPC": {
"Description": "VPC",
"Value": {
"Ref": "VPC"
},
"Export": {
"Name": {
"Fn::Sub": "${AWS::StackName}-VPC"
}
}
}
}
In CF Template that you want to use output of 1st CF Template, as:
"Parameters": {
"VPCStackName": {
"Description": "Name of VPC CF Stack",
"Type": "String",
"Default": "SOME_NAME"
}
},
"Resources": {
"Subnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Fn::ImportValue": {
"Fn::Sub": "${VPCStackName}-VPC"
}
},
"CidrBlock": {
"Ref": "CidrBlockSubnet1"
},
"AvailabilityZone": {
"Ref": "AZ_NAME"
}
}
}
}

Cloud Formation Output Vars - Returning validation error for user access key id

Thanks in advance!
I'm creating a cloudformation template and would like to output the user access key and secret but to no avail...
It currently is not valid and gives me this error Template contains errors.: Invalid template resource property 'XXXXXXuseraccesskey'.
Here's my template snippet;
"XXXXXXuseraccesskey": {
"Type": "AWS::IAM::AccessKey",
"Properties": {
"UserName": {
"Ref": "XXXXXXuser"
}
}
},
"YYYYYYuseraccesskey": {
"Type": "AWS::IAM::AccessKey",
"Properties": {
"UserName": {
"Ref": "YYYYYYuser"
}
}
},
"outputs" : {
"accesskeyforuser": {
"Ref": "XXXXXXuseraccesskey"
},
"secretkeyforuser": {
"Fn::GetAtt": ["XXXXXXuseraccesskey", "SecretAccessKey"]
},
"accesskeyforotheruser": {
"Ref": "YYYYYYuseraccesskey"
},
"secretkeyforotheruser": {
"Fn::GetAtt": ["YYYYYYuseraccesskey", "SecretAccessKey"]
}
}
Your output structure needs a "Value" section to reflect those mapped entries.
Here is a YAML sample of how to output an access key:
AWSTemplateFormatVersion: '2010-09-09'
Resources:
user:
Type: AWS::IAM::User
XXXXXXuseraccesskey:
Type: AWS::IAM::AccessKey
Properties:
UserName:
Ref: user
Outputs:
accesskeyforuser:
Value:
Ref: XXXXXXuseraccesskey
secretkeyforuser:
Value:
Fn::GetAtt: ["XXXXXXuseraccesskey", "SecretAccessKey"]