Add virtual network to existing Event hub namespace - azure-eventhub

I have an arm template to add new virtual network to an existing Event Hub Namespace.
The problem for is that i have to hard code, or ask as paramameter the vnet and subnet addresprefix.
Is there a way to not have to do this or extract those values and use the on the template ?
Vnet exist subnet exist
using reference function get's me the values but i get circular definition error if i use it on resource definition.
Tried to get ip's for vnet and subnet by using reference function but i cant use it in the param or variable and in resource i get Circular dependency error.
Basically i like to to this on a clean template.
"variables": {
"namespaceVirtualNetworkRuleName": "[concat(parameters('eventhubNamespaceName'), concat('/', parameters('vnetRuleName')))]",
"subNetId": "[resourceId('Microsoft.Network/virtualNetworks/subnets/', parameters('vnetRuleName'), parameters('subnetName'))]"
},
resources : [
{
"apiVersion": "2018-01-01-preview",
"name": "[variables('namespaceVirtualNetworkRuleName')]",
"type": "Microsoft.EventHub/namespaces/VirtualNetworkRules",
"properties": {
"virtualNetworkSubnetId": "[variables('subNetId')]"
}
}
]
But i have to add vnet and subnet with correct ip for this to work.
"variables": {
"namespaceVirtualNetworkRuleName": "[concat(parameters('eventhubNamespaceName'), concat('/', parameters('vnetRuleName')))]",
"subNetId": "[resourceId('Microsoft.Network/virtualNetworks/subnets/', parameters('vnetRuleName'), parameters('subnetName'))]"
},
"resources": [
{
"apiVersion": "2018-01-01-preview",
"name": "[parameters('eventhubNamespaceName')]",
"type": "Microsoft.EventHub/namespaces",
"location": "[parameters('location')]",
"sku": {
"name": "Standard",
"tier": "Standard"
},
"properties": { }
},
{
"apiVersion": "2017-09-01",
"name": "[parameters('vnetRuleName')]",
"location": "[parameters('location')]",
"type": "Microsoft.Network/virtualNetworks",
"properties": {
"addressSpace": {
"addressPrefixes": [
"a.b.c.0/24",
"x.y.0.0/16"
]
},
"subnets": [
{
"name": "[parameters('subnetName')]",
"properties": {
"addressPrefix": "x.y.z.w/26",
"serviceEndpoints": [
{
"service": "Microsoft.EventHub"
}
]
}
}
]
}
},
{
"apiVersion": "2018-01-01-preview",
"name": "[variables('namespaceVirtualNetworkRuleName')]",
"type": "Microsoft.EventHub/namespaces/VirtualNetworkRules",
"dependsOn": [
"[concat('Microsoft.EventHub/namespaces/', parameters('eventhubNamespaceName'))]"
],
"properties": {
"virtualNetworkSubnetId": "[variables('subNetId')]"
}
}
]
Any idea on hou to remove the vnet part or at list get the ip's before the template and pass them as param ?

Found out how to make this work.
the solution is this :
"variables": {
},
"resources": [
{
"apiVersion": "2018-01-01-preview",
"name": "[parameters('eventhubNamespaceName')]",
"type": "Microsoft.EventHub/namespaces",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard",
"tier": "Standard"
},
"properties": { }
},
{
"apiVersion": "2018-01-01-preview",
"name": "[concat(parameters('eventhubNamespaceName'), concat('/', parameters('vnetSubscriptioID'), parameters('vnetResorceGroupName'), parameters('vnetRuleName'), parameters('subnetName')[copyIndex('subnetcopy')]))]",
"type": "Microsoft.EventHub/namespaces/VirtualNetworkRules",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.EventHub/namespaces/', parameters('eventhubNamespaceName'))]"
],
"properties": {
"virtualNetworkSubnetId": "[resourceId(parameters('vnetSubscriptioID'), parameters('vnetResorceGroupName'),'Microsoft.Network/virtualNetworks/subnets/', parameters('vnetRuleName'), parameters('subnetName')[copyIndex('subnetcopy')])]"
},
"copy": {
"name": "subnetcopy",
"count": "[length(parameters('subnetName'))]"
}
}
]
}
At the end no network part was needed. Updated my solution to be able to deploy more than one subnet at a time.

Related

Why does my subnet and VPC show side by side on CloudFormation

I've been trying to figure out why my VPC and subnet show side by side instead of the subnet inside of the VPC? (I used Atom to generate this.)
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "vpc",
"Metadata": {
},
"Parameters": {"siggyVpcCidr": {
"Description": "vpc cidr",
"Type": "String",
"Default": "10.0.0.0/16"
},
"siggySubnetCidr": {
"Description": "cidr for the subnet",
"Type": "String",
"Default": "10.0.1.0/2"
},
"Subnet1Az": {
"Description": "AZ for siggySubnetCidr",
"Type": "AWS::EC2::AvailabilityZone::Name"
}
},
"Mappings": {
},
"Conditions": {
},
"Resources": {
"siggyVpc": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": { "Ref": "siggyVpcCidr" },
"Tags": [{ "Key": "Name", "Value": "siggyVpc" }]
}
},
"siggyIgw": {
"Type": "AWS::EC2::InternetGateway",
"Properties": {
"Tags": [{ "Key": "Name", "Value": "siggyIgw1" }]
}
},
"AttachGateway": {
"Type": "AWS::EC2::VPCGatewayAttachment",
"Properties": {
"VpcId": { "Ref": "siggyVpc" },
"InternetGatewayId": { "Ref": "siggyIgw" }
}
},
"SubnetSiggy": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"AvailabilityZone": { "Ref": "Subnet1Az" },
"VpcId": { "Ref": "siggyVpc" },
"CidrBlock": { "Ref": "siggySubnetCidr" },
"Tags": [{ "Key": "Name", "Value": "siggySubnetCidr" }]
}
}
},
"Outputs": {
}
}
They are separate resources. CloudFormation templates arrange resources in a flat array. This is pretty much true of most resources. Some resources can be implicitly defined when creating resources, but that probably won't be reflected with an export where you create a template from existing resources.
You would need to inspect the VpcId property to determine the VPC to which the subnet belongs.

Invalid template property or properties [Vpcname]

I am trying to create a stack with below templates. Same template worked without parameters.
When I Added parameters I got an error which says -Invalid template property or properties
[Vpcname].
I am not sure what more to add as I have checked every aspect from amazon documentation.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Here are some details about the template.",
"Vpcname": {
"Description": "What do you want your VPC to be called as ?",
"Type": "String",
"Default": "fromCf"
},
"Resources": {
"testvpc": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "192.168.0.0/16",
"EnableDnsHostnames": true,
"EnableDnsSupport": true,
"InstanceTenancy": "default",
"Tags": [
{
"Key": "Name",
"Value": {
"Ref": "Vpcname"
}
}
]
}
},
"WebSubnet": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"AvailabilityZone": "ap-south-1a",
"VpcId": {
"Ref": "testvpc"
},
"CidrBlock": "192.168.0.0/24",
"Tags": [
{
"Key": "Name",
"Value": "fromCfWebSub"
}
]
}
},
"AppSubnet": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"AvailabilityZone": "ap-south-1b",
"VpcId": {
"Ref": "testvpc"
},
"CidrBlock": "192.168.1.0/24",
"Tags": [
{
"Key": "Name",
"Value": "fromCfAppSub"
}
]
}
}
}
}
I got it fixed Guys. I was not including "Parameters" : { } I should have put it this ways - "Parameters" : {"Vpcname": { "Description": "What do you want your VPC to be called as ?", "Type": "String", "Default": "fromCf" }}

AWS Cloudfromation and autoscaling : The requested configuration is currently not supported. Launching EC2 instance failed

I want to replicate the infrastructure from one region(us-east-1) to another(us-east-2). so,I have generated a cloudfromation template of an existing infrastructure with the help of cloudformer tool.
"asgamazonecsclisetupapijoulebugprodEcsInstanceAsg1EIBNOXSXJ7HD": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"AvailabilityZones": [
"us-east-2b",
"us-east-2c"
],
"Cooldown": "300",
"DesiredCapacity": "3",
"HealthCheckGracePeriod": "300",
"HealthCheckType": "ELB",
"MaxSize": "16",
"MinSize": "3",
"VPCZoneIdentifier": [
{
"Ref": "subnet81c8ebab"
},
{
"Ref": "subnet5df40214"
}
],
"LaunchConfigurationName": {
"Ref": "lcamazonecsclisetupapijoulebugprodAMI2017d"
},
"LoadBalancerNames": [
{
"Ref": "elbJBAPILiveCleanbit2016"
}
],
"Tags": [
{
"Key": "Name",
"Value": "Live - Cleanbit2016 - joulebug-api",
"PropagateAtLaunch": true
}
],
"TerminationPolicies": [
"Default"
]
}
},
"lcamazonecsclisetupapijoulebugprodAMI2017d": {
"Type": "AWS::AutoScaling::LaunchConfiguration",
"Properties": {
"AssociatePublicIpAddress": true,
"ImageId": "ami-0e6d83442546978bc",
"InstanceType": "c3.large",
"KeyName": "cleanbit2016_vpc",
"IamInstanceProfile": "amazon-ecs-cli-setup-api-joulebug-prod-EcsInstanceProfile-1M4GOHBP3FP5L",
"InstanceMonitoring": "true",
"SecurityGroups": [
{
"Ref": "sgCleanbit2016WebServerSG"
}
],
"BlockDeviceMappings": [
{
"DeviceName": "/dev/xvda",
"Ebs": {
"SnapshotId": "snap-0b2477be9c863d014",
"VolumeSize": 8
}
},
{
"DeviceName": "/dev/xvdcz",
"Ebs": {
"VolumeSize": 22
}
}
]
}
},
"elbJBAPILiveCleanbit2016": {
"Type": "AWS::ElasticLoadBalancing::LoadBalancer",
"Properties": {
"Policies": [
{
"PolicyName": "AWSConsole-SSLNegotiationPolicy-JB-API-Live-Cleanbit2016-1467998170471",
"PolicyType": "SSLNegotiationPolicyType",
}
],
}
}
"subnet81c8ebab": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.3.0/24",
"AvailabilityZone": "us-east-2b",
"VpcId": {
"Ref": "vpcdcbd08bb"
},
"Tags": [
{
"Key": "Name",
"Value": "Cleanbit2016 - Public 1b"
}
]
}
},
"sgCleanbit2016WebServerSG": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Web server security group for public subnet in vpc.",
"VpcId": {
"Ref": "vpcdcbd08bb"
},
"Tags": [
{
"Key": "Name",
"Value": "Cleanbit2016_ WebServerSG"
}
]
}
},
While launching the template in other region(us-east-2), it is throwing following error:
The requested configuration is currently not supported. Please check the documentation for supported configurations. Launching EC2 instance failed.
You don't have details of the regions you're using, but if you are trying to do this outside of us-east-1 the Availablility Zones won't work. It also looks like you have a number of other parts of the stack hard-coded, which may not work in another region.
And if you are trying to do this in us-east-1, there is the possibility that one of the AZs is unavailable to you - see this question for more details.
You have not provided enough information to be able to diagnose the situation.
I took your template, removed the portions that were incomplete (eg removed the Load Balancer because it was missing Listeners), simplified a few things and it works fine:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"Tags": [
{
"Key": "Name",
"Value": "Lab VPC"
}
]
}
},
"ASG": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"AvailabilityZones": [
"us-east-2b",
"us-east-2c"
],
"Cooldown": "300",
"DesiredCapacity": "1",
"HealthCheckGracePeriod": "300",
"MaxSize": "16",
"MinSize": "1",
"VPCZoneIdentifier": [
{
"Ref": "Subnet1"
},
{
"Ref": "Subnet2"
}
],
"LaunchConfigurationName": {
"Ref": "LaunchConfig"
}
}
},
"LaunchConfig": {
"Type": "AWS::AutoScaling::LaunchConfiguration",
"Properties": {
"AssociatePublicIpAddress": true,
"ImageId": "ami-0b59bfac6be064b78",
"InstanceType": "t2.micro",
"InstanceMonitoring": "true",
"SecurityGroups": [
{
"Ref": "WebServerSG"
}
]
}
},
"Subnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.0.0/24",
"AvailabilityZone": "us-east-2b",
"VpcId": {
"Ref": "VPC"
},
"Tags": [
{
"Key": "Name",
"Value": "Public 1"
}
]
}
},
"Subnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.1.0/24",
"AvailabilityZone": "us-east-2c",
"VpcId": {
"Ref": "VPC"
},
"Tags": [
{
"Key": "Name",
"Value": "Public 2"
}
]
}
},
"WebServerSG": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Web server security group for public subnet in vpc.",
"VpcId": {
"Ref": "VPC"
},
"Tags": [
{
"Key": "Name",
"Value": "WebServerSG"
}
]
}
}
}
}
Therefore, your problem lies in part of the template you did not provide.
You could start with this version, then progressively add back parts of your template until you discover what is causing the error.
If you wish to create a CloudFormation template that can run in multiple regions, you should not refer to specific Availability Zones (eg us-east-2b).
You can use Fn::GetAZs - AWS CloudFormation to obtain a list of AZs in region.
After a lot of debugging, when I started launching the things manually, I found the same error and I got to know that c3.large is causing the error. When I launch the template with c4.large it successfully launched the template from us-east-1 to us-east-2.

Can't send parameters from a child CloudFormation template to another child template

I am crafting a monstrosity of a CloudFormation template that uses the AWS::CloudFormation::Stack resource to spawn other nested stacks. The purpose of this is the classic separation of duties and management.
The overall goal is to build an application environment from the ground up (starting with VPC and ending with the application instances).
So far, most of it works. But I've hit a road block with referencing the outputs of the subnet CF template and the securitygroup CF template for use in creating the EC2 instances.
It works like this:
Master template --> Builds VPC --> Calls child template to build subnets --> Calls child template to build security groups --> Calls child template to build EC2 instances
I need to pass the outputs from the subnet and security group templates to the EC2 instance template so the instances can be provisioned into the correct part of the architecture. The VPC ID ref and KeyPairs pass in fine, but the subnetID and securitygroupID do not.
Here's the portion of the master template that calls the security group/subnet templates:
"DevNetworkStack": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": [
"SNVPC",
"SNIGW"
],
"Properties": {
"TemplateURL": {
"Fn::FindInMap": [
"TemplateURLs",
"DevNetworkStack",
"URL"
]
},
"TimeoutInMinutes": "30",
"Parameters": {
"VPCID": {
"Ref": "SNVPC"
},
"SNIGW": {
"Ref": "SNIGW"
}
}
}
},
"DevSecurityGroupsStack": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": "DevNetworkStack",
"Properties": {
"TemplateURL": {
"Fn::FindInMap": [
"TemplateURLs",
"DevSecurityGroupsStack",
"URL"
]
},
"TimeoutInMinutes": "30",
"Parameters": {
"VPCID": {
"Ref": "SNVPC"
}
}
}
},
These work fine. They create and everything is fine. The templates offer outputs like so:
"Outputs": {
"DevAdminSubnetID": {
"Description": "DevAdminSubnetID",
"Value": {
"Ref": "DevAdminSubnet"
}
},
...
"Outputs": {
"DevAdminSecurityGroupID": {
"Description": "DevAdminSecurityGroupID",
"Value": {
"Ref": "DevAdminSecurityGroup"
}
},
I can see the outputs in the CF console.
Now, the next template is trying to use the Security Group ID and the Subnet ID. But it's not working.
Master template calls the next child as:
"DevAdminStack": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": [
"DevNetworkStack",
"DevSecurityGroupsStack",
"EC2DevRoleInstanceProfile",
"S3DevUserDataBucket",
"S3DevHomeDirsDataBucket"
],
"Properties": {
"TemplateURL": {
"Fn::FindInMap": [
"TemplateURLs",
"DevAdminStack",
"URL"
]
},
"TimeoutInMinutes": "30",
"Parameters": {
"AdminKeyPair": {
"Ref": "AdminServersKeyPair"
},
"VPCID": {
"Ref": "SNVPC"
},
"DevAdminSecurityGroupID": [
{
"Fn::GetAtt": [
"DevSecurityGroupsStack",
"Outputs.DevAdminSecurityGroupID"
]
}
],
"DevAdminSubnetID": [
{
"Fn::GetAtt": [
"DevNetworkStack",
"Outputs.DevAdminSubnetID"
]
}
]
}
}
}
...and the child template looks like this (removed some portions for brevity because I'm just testing right now)
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Dev-Admin",
"Mappings": {
"RegionMap": {
"DevAdminServer": {
"AMI": "ami-6fc9770e",
"InstanceType": "t2.micro"
}
}
},
"Parameters": {
"AdminKeyPair": {
"Type": "AWS::EC2::KeyPair::KeyName"
},
"VPCID": {
"Type": "AWS::EC2::VPC::Id"
},
"DevAdminSecurityGroupID": {
"Type": "AWS::EC2::SecurityGroup::Id"
},
"DevAdminSubnetID": {
"Type": "AWS::EC2::Subnet::Id"
}
},
"Resources": {
"DevAdminServer": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"Comment": "Sets up administrative tools for the server",
"AWS::CloudFormation::Init": {
"config": {
"packages": {
"yum": {}
},
"files": {},
"services": {}
}
}
},
"Properties": {
"ImageId": {
"Fn::FindInMap": [
"RegionMap",
"DevAdminServer",
"AMI"
]
},
"SecurityGroupIds": [
{
"Ref": "DevAdminSecurityGroupID"
}
],
"SubnetId": {
"Ref": "DevAdminSubnetID"
},
"InstanceType": {
"Fn::FindInMap": [
"RegionMap",
"DevAdminServer",
"InstanceType"
]
},
"KeyName": {
"Ref": "AdminKeyPair"
},
"Tags": [
{
"Key": "Application",
"Value": {
"Ref": "AWS::StackId"
}
}
],
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[]
]
}
}
}
}
}
}
But this resource fails on creation with the error:
Value of property Parameters must be an object with String (or simple type) properties
I know it's the last two variables causing the trouble (subnetID and securitygroupID), because I removed them and the provisioning the child template works just fine.
What am I doing wrong?
The values of DevAdminSecurityGroupID and DevAdminSubnetID are defined as JSON arrays [] in your master template, but they should be Strings (after the Fn::GetAtt intrinsic function expansion):
"DevAdminSecurityGroupID":
{
"Fn::GetAtt": [
"DevSecurityGroupsStack",
"Outputs.DevAdminSecurityGroupID"
]
},
"DevAdminSubnetID":
{
"Fn::GetAtt": [
"DevNetworkStack",
"Outputs.DevAdminSubnetID"
]
}

Cloudformation - ElastiCache::SubnetGroup not honouring resource name

I have a weird issue with CloudFormation that seems either to be a bug, or more likely - i've missed something pretty basic.
I have the following template (a snippet) defining two subnets and a subnet group as follow:
...
"redissubnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.8.0/24",
"AvailabilityZone": "us-east-1c",
"VpcId": {
"Ref": "myVPC"
},
"Tags": [
{
"Key": "Name",
"Value": "redissubnet1"
}
]
}
},
"redissubnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.9.0/24",
"AvailabilityZone": "us-east-1c",
"VpcId": {
"Ref": "myVPC"
},
"Tags": [
{
"Key": "Name",
"Value": "redissubnet2"
}
]
}
},
"SubnetGroupName": {
"Type": "AWS::ElastiCache::SubnetGroup",
"Properties": {
"Description": "Subnet group for main application redis elastic cache",
"SubnetIds": [
{
"Ref": "redissubnet1"
},
{
"Ref": "redissubnet2"
}
]
}
}
...
All resources are created, yet the SubnetGroup name - "SubnetGroupName" - is not honoured. AWS auto-assigns a name in the format [a-z]-[a-z]-[a-z0-9]
Has anyone encountered this?
What I'm actually trying to do is reference this subnet group by name in the creation of an ElastiCache::Cluster - however because the resource name is not honoured I can't do so.
Anyone have any ideas? All help gratefully received :)
Answer was to reference the subnet group name in the elastic cache resource, as follows:
{
"subnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.8.0/24",
"AvailabilityZone": "us-east-1c",
"VpcId": {
"Ref": "myVPC"
},
"Tags": [{
"Key": "Name",
"Value": "subnet1"
}]
}
},
"subnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.9.0/24",
"AvailabilityZone": "us-east-1c",
"VpcId": {
"Ref": "myVPC"
},
"Tags": [{
"Key": "Name",
"Value": "subnet2"
}]
}
},
"redis1": {
"Type": "AWS::ElastiCache::SubnetGroup",
"Properties": {
"Description": "Subnet group for main application redis elastic cache",
"SubnetIds": [{
"Ref": "subnet1"
}, {
"Ref": "subnet2"
}]
}
},
"mainredis": {
"Type": "AWS::ElastiCache::CacheCluster",
"Properties": {
"AutoMinorVersionUpgrade": "true",
"CacheNodeType": "cache.t2.small",
"CacheSubnetGroupName": {
"Ref": "redis1"
},
"ClusterName": "mainredis",
"Engine": "redis",
"NumCacheNodes": "1",
"Port": "6379",
"Tags": [{
"Key": "Name",
"Value": "mainredis"
}, {
"Key": "Function",
"Value": "Main redis store"
}],
"VpcSecurityGroupIds": [
"redissecuritygroup"
]
}
}
}