Currently I'm writing a terraform script in which I wanted to set a subnet CIDR-BLOCK dynamically. I know the better approach is to use module instead of resources but this is the limitation at our end to only use resources.
CURRENT PROBLEM
I'had created only VPC in one configuration file. I have another configuration file in which I'm trying to provisioning VPC's configs (subnets,routing) etc.
Now I wanted to know the count of the existing subnets in the current VPC. For this I'm using the datasource to get the count. But while executing my plan I'm getting the following exception.
data.aws_subnet_ids.available-subnets: data.aws_subnet_ids.available-subnets:
no matching subnet found for vpc with id vpc-***************
This is how I'm getting the subnets
data "aws_subnet_ids" "available-subnets" {
vpc_id = "${data.aws_vpc.vpcobject.id}"
}
Do i need to add a depends-on attribute in it? if yes then what will be the condition or test for it? If this is not the case then what would be alternatives?
Thanks
Related
Background: We have an VPC, it has an Internet Gateway attached.
I would like to get the InternetGatewayId of the VPC via aws-cdk
vpc := awsec2.Vpc_FromLookup(stack, jsii.String(viper.GetString(`vpc.id`)), &awsec2.VpcLookupOptions{
VpcId: jsii.String(viper.GetString(`vpc.id`)),
}) //Here it returns awsec2.Ivpc
But according to the code, only awsec2.Vpc has a method InternetGatewayId(). How could I convert awsec2.Ivpc to awsec2.Vpc?
The Ivpc type returned from Vpc_FromLookup is a CDK convenience method to cache a limited set of VPC attributes at synth-time. Unfortunately, the Internet Gateway ID isn't one of them:
Currently you're unable to get the [Internet Gateway] ID for imported VPCs. To do that you'd have to specifically look up the Internet Gateway by name, which would require knowing the name beforehand.
A simple, deterministic workaround is to manually store the ID as a SSM Parameter Store Parameter. At synth-time, StringParameter_ValueFromLookup looks up and caches the IGW ID value as Context in cdk.context.json.:
igwID := awsssm.StringParameter_ValueFromLookup(stack, jsii.String("/my-params/vpc/igw-id"))
A more advanced CDK-only solution is to lookup the ID in a deploy-time CustomResource, which "can do arbitrary lookups or modifications during a CloudFormation deployment" (typically by making SDK calls using a Lambda). Note that this is not necessarily a better solution, because it introduces non-determinacy into the deployment.
I have two terraform projects. For both projects I am using same credentials and state is stored in S3 bucket. In the first project I create the vpc and the subnets. In the 2nd project I am creating an RDS. I want to put the RDS in the vpc and subnets created in the 1st terraform project. How do I do that?
Thanks!
In the RDS project, pull in the VPC and subnets as data resources. You can pull in the VPC like so:
data "aws_vpc" "my_vpc" {
id = var.vpc_id
}
And you can reference that as data.aws_vpc.my_vpc.
Depending on your use case, you might want the subnet_ids or subnets data resources to reference subnets as a group.
Data resources are a powerful way to bridge this kind of gap, but if you're not careful with them it can lead to some nasty dependency issues in your build. Keeping networking away from application resources is a sensible division, but consider merging the projects if you expect the networking to change frequently.
I am new in AWS so have couple of AWS VPC creation using CloudFormation service questions.
1. Scenario: I have created the yaml file and executed that in the stack. the VPC, route table and all the subnets get created successfully. now I have deleted one of the subnet manually (through console). Now I want that subnet back, so I was trying to run the "update" stack using the "current template" (though I have not made any modification in the template). it is showing me that there is no modification in the template error.
Question 1: How to install the deleted resource through template stack without modifying it.
2. Scenario: When we create VPC, we get default route table and NACL created.
Question 2: Why can't we use the default route table and NACL through cloudformation.
Question 3: is there any command from where we can get the default route table and NACL ID in cloudformation. (for eg : there is command where we can associate the subnets to routetable. something like that).
Thanks in advance.
It can be tricky indeed when stuff has changed outside of cloudformations state. Unlike some other IaC tools, it doesn't 'correct' the state of resources when they have deviated from the given state.
Remove the subnet resource from the template, update the stack with the removed resource, add the subnet back and again update the stack.
It's actually best practice to create new route tables and NACLs and associate them with the corresponding subnets, so there is actually no need to modify the default resources.
You can create a cloudformation custom resource to query for the id's and pass them to other resources. However this is not recommended due to answer 2. Ask yourself: what am I trying to achieve here? Is it really necessary?
Goal is to visualise the relationship of resources within AWS account(may have multiple VPC's).
This would help daiy operations. For example: Resources getting affected after modifying the security group
Each resource has ARN assigned in AWS cloud
Below are some example relationsships among resources:
Route table has-many relationship with subnets
NACL has-many relationship with subnets
Availability zone has-many relationship with subnets
IAM resource has-many regions
has-many is something like compose relation
security group has association relation with any resource in VPC
NACL has association relation with subnet only
We also have VPC flow logs to find the relationships
Using AWS SDK's,
1)
For on-prem networks, we take IP range and send ICMP requests to verify existence of devices in the IP range and then we send snmp query to classify the device as (windows/linux/router/gateway etc...)
How to find the list of resources allocated within an AWS account? How to classify resources?
2)
What are the parameters that need to be queried from AWS resources(IAM, VPC, subnet, RTable, NACL, IGW etc...) that help create relationsip view of the resources within an AWS account?
you don't have to stitch your ressources together by your self in your app. you can use the ressourcegrouptagging api from aws. take a look on ressourcegroups in your aws console. there you can group things based on tags. then, you can tag the groups itself. requesting with the boto3 python lib will give you a bunch of information. read about boto3, its huge! another thing which might be intresting for you is "aws config".. there you can have your compliance, config histoty, relationship of ressources and plenty of other stuff!
also, check out aws cloudwatch for health monitoring
From a brief search - there does not seem to be a method to set dynamic hostnames for members of an autoscaling group. The functionality exists within OpenStack Heat using index - but I cannot find anything on doing so with AWS autoscaling groups.
For example, using OpenStack Heat - nodes are automatically given a hostname based on the number of nodes in the autoscaling group:
instance_group:
type: OS::Heat::ResourceGroup
properties:
count: { get_param: instance_number }
resource_def:
type: OS::Nova::Server
properties:
name: instance%index%
Would give me the following if I were to have 3 instances in the autoscaling group
instance0
instance1
instance2
Is there a similar method I can use with the AWS autoscaling groups launch configuration and or cloud-init?
I've found a solution that works pretty well, if you stick to some not-unreasonable conventions.
Every kind of EC2 instance that I launch, whether there are N servers of this kind in an autoscaling group or it's stand-alone instance, I create an Instance Profile for it. This is a good idea anyway in my experience, even if you don't need the instance to access any aws services it doesn't hurt to have a role/profile with empty permissions, it makes it that much easier to give it access to an s3 bucket or whatever else in the future if you need to.
Then at server launch in the user_data script (or your configuration management tool if you're using something like puppet or ansible), I query the instance profile name from the metadata service and append something unique to each server like the private ip and set that as the hostname.
You'll end up with hostnames like webserver-10-0-12-58 which is both human readable and unique to each server.
(The downside of this vs incrementing integers is that these aren't predictable, and can't be used to set up unique behavior for a single server. For example if you had webserver-{0-8} and needed to run some process on exactly one server, you could use logic like if hostname == webserver-0 then run_thing.)