I am using ansible cloudformation to create stack with 20 instances.
Now in the ansible output i can only see the instance ids.
Now after the stack is created i want to connect to them and configure it but i am not sure how can get thos ips or hostnames from instance id.
cloudformation output is like this
{
"last_updated_time": null,
"logical_resource_id": "test2",
"physical_resource_id": "i-24tf97306",
"resource_type": "AWS::EC2::Instance",
"status": "CREATE_COMPLETE",
"status_reason": null
},
{
"last_updated_time": null,
"logical_resource_id": "test1",
"physical_resource_id": "i-6533184348",
"resource_type": "AWS::EC2::Instance",
"status": "CREATE_COMPLETE",
"status_reason": null
}
ec2_remote_facts module is your friend here.
You can retrieve instance meta data when Ansible runs on the instance e.g.
curl http://169.254.169.254/latest/meta-data/public-hostname
ec2-aa-bb-cc-ddd.ap-southeast-2.compute.amazonaws.com
where aa-bb-cc-ddd represents your IP and the full string represents the hostname.
You're using Ansible so you could use Ansible's get_url module: http://docs.ansible.com/ansible/get_url_module.html to perform the HTTP request.
Related
I'm trying to write a script that would allow the instances to update a record set in AWS each time a new one is spun up.
I'm following this guide:
https://aws.amazon.com/premiumsupport/knowledge-center/simple-resource-record-route53-cli/
My sample.json looks like this:
{
"Comment": "CREATE/DELETE/UPSERT a record ",
"Changes": [{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "test.mydomain.com",
"Type": "A",
"TTL": 300,
"ResourceRecords": [{ "Value": "4.4.4.4"}]
}}]
}
I want to replace the 4.4.4.4 but with instance's private IP address.
I tried inserting $IP_ADDRESS there, but obviously, it didn't work.
I also tried entering this manually by doing this:
IP_ADDRESS=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)
aws route53 change-resource-record-sets --hosted-zone-id HKJA837HJS --change-batch {"Comment": "UPSERT a record", "Changes": [{"Action": "UPSERT", "ResourceRecordSet":{"Name":"test.mydomain.com","Type":"A","TTL":300,"ResourceRecords":[{"Value":"$IP_ADDRESS"}]}}]}
When I do this I keep getting the following error:
Unknown options: Changes:, [{Action:, UPSERT,, ResourceRecordSet:Name:test.mydomain.com}]}, ResourceRecordSet:Type:A}]}, ResourceRecordSet:TTL:300}]}, ResourceRecordSet:ResourceRecords:[{Value:}]}]}, UPSERT a record,
I tried re-formatting this numerous times but there's always something wrong.
How can I make sure the instance's IP address is inserted in that recordset each time a new instance in launched?
Trying to pass JSON via the command line is very difficult to get correct because of the need to escape your quote marks.
Instead put your json into a file. Execute a command like this:
sed "s/4.4.4.4/$NEWIP/g" update_rr.json
aws --profile PROD route53 change-resource-record-sets --hosted-zone-id ABCDEFGH012345 --change-batch file://update_rr.json
I am trying to get AWS EC2 instance details using RunInstancesRequest. For that I followed AWS doc https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-ec2-instances.html.
RunInstancesRequest runInstancesRequest = new RunInstancesRequest();
runInstancesRequest.withImageId(imageId).withInstanceType(instanceType).withMinCount(1).withMaxCount(count).withSecurityGroups(securityGroupName);
RunInstancesResult runInstancesResult = amazonEC2.runInstances(runInstancesRequest);
String instance_id = runInstancesResult.getReservation().getReservationId();
//waiting for 2 minute
DescribeInstancesRequest describeInstancesRequest = new DescribeInstancesRequest();
describeInstancesRequest.setInstanceIds(Arrays.asList(instance_id));
DescribeInstancesResult describeInstancesResult = amazonEC2.describeInstances(describeInstancesRequest);
for(Reservation reservation : describeInstancesResult.getReservations()){
for(Instance instance : reservation.getInstances()) {
System.out.println(instance.getPublicDnsName());
}
}
Here I am able to get AWS EC2 instance up and running but the problem I am facing is I am not able to get the EC2 details using the RunInstancesResult object. As per AWS documentation it seems like instance_id is reservation_id but I believe it is not so. As instance_id start with "i-" and reservation_id with "r-".
How I can get the details of only one EC2 which I created using API? As I got RunInstancesResult object as output of the previous API hence the question: How I can get AWS EC2 instance details using RunInstancesRequest?
Reservations are the request to launch instances. For example, you could use one launch request to create two instances. Thus, the Reservation contains multiple Instances.
If you look in the response object, you will see that the Reservation does indeed contain multiple instances, eg:
{
"OwnerId": "123456789012",
"ReservationId": "r-08626e73c547023b1",
"Groups": [
{
"GroupName": "MySecurityGroup",
"GroupId": "sg-903004f8"
}
],
"Instances": [
{
"Monitoring": {
"State": "disabled"
},
"PublicDnsName": null,
"RootDeviceType": "ebs",
"State": {
"Code": 0,
"Name": "pending"
},
"EbsOptimized": false,
"LaunchTime": "2013-07-19T02:42:39.000Z",
"ProductCodes": [],
"StateTransitionReason": null,
"InstanceId": "i-1234567890abcdef0",
"ImageId": "ami-1a2b3c4d",
"PrivateDnsName": null,
"KeyName": "MyKeyPair",
etc.
There is small confusion created in AWS doc. They are refering reservation id as instance_id. After changing following changes in my code, was able to filter out the instances:
String instance_id = runInstancesResult.getReservation().getInstances()..get(0).getInstanceId();
I need to create Elastic Beanstalk Environment from boto3.
For which I guess the API sequence should be:
create_application()
From this API we get the "Application Name":
create_environment(kwargs)
Here i am passing below json as kwargs to api
{
"ApplicationName": "APP-NAME",
"EnvironmentName": "ABC-Nodejs",
"CNAMEPrefix": "ABC-Neptune",
"SolutionStackName": "64bit Amazon Linux 2016.03 v2.1.1 running Node.js"
}
Questions:
How to mention that the Environment EC2 should be attached to which
VPC and subnet
In which Subnet its ELB should be created
Any sample code will be helpful
Please Note: I have one public and one private subnet, we can control the creation of EC2 and ELB creation through subnet IDs
To set up dependent resources with your Environment you would have to use the Elastic Beanstalk Option Settings. Specifically for VPCs you can use the aws:ec2:vpc namespace, I've linked the documentation for those settings with that.
The code example would be something like this:
{
ApplicationName: "APP-NAME",
EnvironmentName: "ABC-Nodejs",
CNAMEPrefix: "ABC-Neptune",
SolutionStackName: "64bit Amazon Linux 2016.03 v2.1.1 running Node.js"
OptionSettings=[
{
'Namespace': 'aws:ec2:vpc',
'OptionName': 'VPCId',
'Value': 'vpc-12345678'
},
{
'Namespace': 'aws:ec2:vpc',
'OptionName': 'ELBSubnets',
'Value': 'subnet-11111111,subnet-22222222'
},
],
}
Thanks nbalas,
I am using below code to create EB.
Despite giving already created security group names in "aws:elb:loadbalancer" and "aws:autoscaling:launchconfiguration" it is creating new security groups and attaching them to EC2 instance and load balancer. So now both of the security groups new and old ones are attached to the resources. I don't want to create new security groups at all and want to use old ones only.
kwargs={
"ApplicationName": "Test",
"EnvironmentName": "ABC-Nodejs",
"CNAMEPrefix": "ABC-ABC",
"SolutionStackName": "64bit Amazon Linux 2016.03 v2.1.1 running Node.js",
"OptionSettings": [
{
"Namespace": "aws:ec2:vpc",
"OptionName": "Subnets",
"Value": "subnet-*******0"
},
{
"Namespace": "aws:ec2:vpc",
"OptionName": "ELBSubnets",
"Value": "subnet-********1"
},
{
"Namespace": "aws:elb:loadbalancer",
"OptionName": "SecurityGroups",
"Value": "sg-*********2"
},
{
"Namespace": "aws:autoscaling:launchconfiguration",
"OptionName": "SecurityGroups",
"Value": "sg-**********3"
}
]
}
response = client.create_environment(**kwargs)
Today when I launch an app using kubernetes over aws it exposes a publicly visible LoadBalancer Ingress URL, however to link that to my domain to make the app accessible to the public, I need to manually go into the aws route53 console in a browser on every launch. Can I update the aws route53 Resource Type A to match the latest Kubernetes LoadBalancer Ingress URL from the command line ?
Kubernetes over gcloud shares this challenge of having to either predefine a Static IP which is used in launch config or manually do a browser based domain linkage post launch. On aws I was hoping I could use something similar to this from the command line
aws route53domains update-domain-nameservers ???
__ OR __ can I predefine an aws kubernetes LoadBalancer Ingress similar to doing a predefined Static IP when over gcloud ?
to show the deployed app's LoadBalancer Ingress URL issue
kubectl describe svc
... output
Name: aaa-deployment-407
Namespace: ruptureofthemundaneplane
Labels: app=bbb
pod-template-hash=4076262206
Selector: app=bbb,pod-template-hash=4076262206
Type: LoadBalancer
IP: 10.0.51.82
LoadBalancer Ingress: a244bodhisattva79c17cf7-61619.us-east-1.elb.amazonaws.com
Port: port-1 80/TCP
NodePort: port-1 32547/TCP
Endpoints: 10.201.0.3:80
Port: port-2 443/TCP
NodePort: port-2 31248/TCP
Endpoints: 10.201.0.3:443
Session Affinity: None
No events.
UPDATE:
Getting error trying new command line technique (hat tip to #error2007s comment) ... issue this
aws route53 list-hosted-zones
... outputs
{
"HostedZones": [
{
"ResourceRecordSetCount": 6,
"CallerReference": "2D58A764-1FAC-DEB4-8AC7-AD37E74B94E6",
"Config": {
"PrivateZone": false
},
"Id": "/hostedzone/Z3II3949ZDMDXV",
"Name": "chainsawhaircut.com."
}
]
}
Important bit used below : hostedzone Z3II3949ZDMDXV
now I craft following using this Doc (and this Doc as well) as file /change-resource-record-sets.json (NOTE I can successfully change Type A using a similar cli call ... however I need to change Type A with an Alias Target of LoadBalancer Ingress URL)
{
"Comment": "Update record to reflect new IP address of fresh deploy",
"Changes": [{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "chainsawhaircut.com.",
"Type": "A",
"TTL": 60,
"AliasTarget": {
"HostedZoneId": "Z3II3949ZDMDXV",
"DNSName": "a244bodhisattva79c17cf7-61619.us-east-1.elb.amazonaws.com",
"EvaluateTargetHealth": false
}
}
}]
}
on command line I then issue
aws route53 change-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV --change-batch file:///change-resource-record-sets.json
which give this error message
An error occurred (InvalidInput) when calling the ChangeResourceRecordSets operation: Invalid request
Any insights ?
Here is the logic needed to update aws route53 Resource Record Type A with value from freshly minted kubernetes LoadBalancer Ingress URL
step 1 - identify your hostedzone Id by issuing
aws route53 list-hosted-zones
... from output here is clip for my domain
"Id": "/hostedzone/Z3II3949ZDMDXV",
... importantly never populate json with hostedzone Z3II3949ZDMDXV its only used as a cli parm ... there is a second similarly named token HostedZoneId which is entirely different
step 2 - see current value of your route53 domain record ... issue :
aws route53 list-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV --query "ResourceRecordSets[?Name == 'scottstensland.com.']"
... output
[
{
"AliasTarget": {
"HostedZoneId": "Z35SXDOTRQ7X7K",
"EvaluateTargetHealth": false,
"DNSName": "dualstack.asomepriorvalue39e7db-1867261689.us-east-1.elb.amazonaws.com."
},
"Type": "A",
"Name": "scottstensland.com."
},
{
"ResourceRecords": [
{
"Value": "ns-1238.awsdns-26.org."
},
{
"Value": "ns-201.awsdns-25.com."
},
{
"Value": "ns-969.awsdns-57.net."
},
{
"Value": "ns-1823.awsdns-35.co.uk."
}
],
"Type": "NS",
"Name": "scottstensland.com.",
"TTL": 172800
},
{
"ResourceRecords": [
{
"Value": "ns-1238.awsdns-26.org. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400"
}
],
"Type": "SOA",
"Name": "scottstensland.com.",
"TTL": 900
}
]
... in above notice value of
"HostedZoneId": "Z35SXDOTRQ7X7K",
which is the second similarly name token Do NOT use wrong Hosted Zone ID
step 3 - put below into your change file aws_route53_type_A.json (for syntax Doc see link mentioned in comment above)
{
"Comment": "Update record to reflect new DNSName of fresh deploy",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"AliasTarget": {
"HostedZoneId": "Z35SXDOTRQ7X7K",
"EvaluateTargetHealth": false,
"DNSName": "dualstack.a0b82c81f47d011e6b98a0a28439e7db-1867261689.us-east-1.elb.amazonaws.com."
},
"Type": "A",
"Name": "scottstensland.com."
}
}
]
}
To identify value for above field "DNSName" ... after the kubernetes app deploy on aws it responds with a LoadBalancer Ingress as shown in output of cli command :
kubectl describe svc --namespace=ruptureofthemundaneplane
... as in
LoadBalancer Ingress: a0b82c81f47d011e6b98a0a28439e7db-1867261689.us-east-1.elb.amazonaws.com
... even though my goal is to execute a command line call I can do this manually by getting into the aws console browser ... pull up my domain on route53 ...
... In this browser picklist editable text box (circled in green) I noticed the URL gets magically prepended with : dualstack. Previously I was missing that magic string ... so json key "DNSName" wants this
dualstack.a0b82c81f47d011e6b98a0a28439e7db-1867261689.us-east-1.elb.amazonaws.com.
finally execute the change request
aws route53 change-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV --change-batch file://./aws_route53_type_A.json
... output
{
"ChangeInfo": {
"Status": "PENDING",
"Comment": "Update record to reflect new DNSName of fresh deploy",
"SubmittedAt": "2016-07-13T14:53:02.789Z",
"Id": "/change/CFUX5R9XKGE1C"
}
}
.... now to confirm change is live run this to show record
aws route53 list-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV
You can also use external-dns project.
AWS specific setup can be found here
After installation it can be used with an annotation e.g.: external-dns.alpha.kubernetes.io/hostname: nginx.external-dns-test.my-org.com.
Note the IAM permissions needs to be set properly.
I am trying to setup an AWS AMI vagrant provision: http://www.packer.io/docs/builders/amazon-ebs.html
I am using the standard .json config:
{
"type": "amazon-instance",
"access_key": "YOUR KEY HERE",
"secret_key": "YOUR SECRET KEY HERE",
"region": "us-east-1",
"source_ami": "ami-d9d6a6b0",
"instance_type": "m1.small",
"ssh_username": "ubuntu",
"account_id": "0123-4567-0890",
"s3_bucket": "packer-images",
"x509_cert_path": "x509.cert",
"x509_key_path": "x509.key",
"x509_upload_path": "/tmp",
"ami_name": "packer-quick-start {{timestamp}}"
}
It connects fine, and I see it create the instance in my AWS account. However, I keep getting Timeout waiting for SSH as an error. What could be causing this problem and how can I resolve it?
As I mentioned in my comment above this is just because sometimes it takes more than a minute for an instance to launch and be SSH ready.
If you want you could set the timeout to be longer - the default timeout with packer is 1 minute.
So you could set it to 5 minutes by adding the following to your json config:
"ssh_timeout": "5m"