My question is that, I have 2 regions in AWS. One is the source region of ec2-instance and other is the target region for that ec2-instances.
In a Disaster Recovery project, when I am doing failover then ec2-instance spin up in target region with new IP (due to application availability hostname is static).
And I need to point the hostname to new IP address, now I want to point old hostname with new IP using any API/SDK, lambda function, basically I want to do this pointing job with automation.
You can use Route 53 for this with AWS CLI (or other programming languages):
Boto3 Python:
import boto3
myNewIP = "X.X.X.X"
client = boto3.client("route53")
client.change_resource_record_sets(
HostedZoneId="your.domain.com",
ChangeBatch={
"Comment": "Updating DNS to new host IP",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "www.your.domain.com",
"Type": "A", #A for IP
"ResourceRecords": [
{
"Value": myNewIP
}
]
"TTL": 60
}
}
]
}
)
For more information check this link Boto3 or AWS CLI.
Related
I’m trying to get an AWS Auto Scaling Group to replace ‘unhealthy’ instances, but I can’t get it to work.
From the console, I’ve created a Launch Configuration and, from there, an Auto Scaling Group with an Application Load Balancer. I've kept all settings regarding the target group and listeners the same as the default settings. I’ve selected ‘ELB’ as an additional health check type for the Auto Scaling Group. I’ve consciously misconfigured the Launch Configuration to result in ‘broken’ instances -- there is no web server to listen to the port configured in the listener.
The Auto Scaling Group seems to be configured correctly and is definitely aware of the load balancer. However, it thinks the instance it has spun up is healthy.
// output of aws autoscaling describe-auto-scaling-groups:
{
"AutoScalingGroups": [
{
"AutoScalingGroupName": "MyAutoScalingGroup",
"AutoScalingGroupARN": "arn:aws:autoscaling:eu-west-1:<accountId>:autoScalingGroup:3edc728f-0831-46b9-bbcc-16691adc8f44:autoScalingGroupName/MyAutoScalingGroup",
"LaunchConfigurationName": "MyLaunchConfiguration",
"MinSize": 1,
"MaxSize": 3,
"DesiredCapacity": 1,
"DefaultCooldown": 300,
"AvailabilityZones": [
"eu-west-1b",
"eu-west-1c",
"eu-west-1a"
],
"LoadBalancerNames": [],
"TargetGroupARNs": [
"arn:aws:elasticloadbalancing:eu-west-1:<accountId>:targetgroup/MyAutoScalingGroup-1/1e36c863abaeb6ff"
],
"HealthCheckType": "ELB",
"HealthCheckGracePeriod": 300,
"Instances": [
{
"InstanceId": "i-0b589d33100e4e515",
// ...
"LifecycleState": "InService",
"HealthStatus": "Healthy",
// ...
}
],
// ...
}
]
}
The load balancer, however, is very much aware that the instance is unhealthy:
// output of aws elbv2 describe-target-health:
{
"TargetHealthDescriptions": [
{
"Target": {
"Id": "i-0b589d33100e4e515",
"Port": 80
},
"HealthCheckPort": "80",
"TargetHealth": {
"State": "unhealthy",
"Reason": "Target.Timeout",
"Description": "Request timed out"
}
}
]
}
Did I just misunderstand the documentation? If not, what else is needed to be done to get the Auto Scaling Group to understand that this instance is not healthy and refresh it?
To be clear, when instances are marked unhealthy manually (i.e. using aws autoscaling set-instance-health), they are refreshed as is expected.
Explanation
If you have consciously misconfigured the instance from the start and the ELB Health Check has never passed, then the Auto Scaling Group does not acknowledge yet that your ELB/Target Group is up and running. See this page of the documentation.
After at least one registered instance passes the health checks, it enters the InService state.
And
If no registered instances pass the health checks (for example, due to a misconfigured health check), ... Amazon EC2 Auto Scaling doesn't terminate and replace the instances.
I configured from scratch and arrived at the same behavior as what you described. To verify that this is indeed the root cause, check the Target Group status in the ASG. It is probably in Added state instead of InService.
[cloudshell-user#ip-10-0-xx-xx ~]$ aws autoscaling describe-load-balancer-target-groups --auto-scaling-group-name test-asg
{
"LoadBalancerTargetGroups": [
{
"LoadBalancerTargetGroupARN": "arn:aws:elasticloadbalancing:us-east-1:xxx:targetgroup/asg-test-1/abc",
"State": "Added"
}
Resolution
To achieve the desired behavior, what I did was
Run a simple web service on port 80. Ensure Security Group is open for the ELB to talk to EC2.
Wait until the ELB status is healthy. Ensure server is returning 200. You may need to create an empty index.html just to pass the health check.
Wait until the target group status has become InService in the ASG.
For example, for Step 3:
[cloudshell-user#ip-10-0-xx-xx ~]$ aws autoscaling describe-load-balancer-target-groups --auto-scaling-group-name test-asg
{
"LoadBalancerTargetGroups": [
{
"LoadBalancerTargetGroupARN": "arn:aws:elasticloadbalancing:us-east-1:xxx:targetgroup/test-asg-1-alb/abcdef",
"State": "InService"
}
]
}
Now that it is in service, turn off the web server and wait. Check often, though, as once ASG detects it is unhealthy it will terminate.
[cloudshell-user#ip-10-0-xx-xx ~]$ aws autoscaling describe-auto-scaling-groups
{
"AutoScalingGroups": [
{
"AutoScalingGroupName": "test-asg",
"AutoScalingGroupARN": "arn:aws:autoscaling:us-east-1:xxx:autoScalingGroup:abc-def-ghi:autoScalingGroupName/test-asg",
...
"LoadBalancerNames": [],
"TargetGroupARNs": [
"arn:aws:elasticloadbalancing:us-east-1:xxx:targetgroup/test-asg-1-alb/abc"
],
"HealthCheckType": "ELB",
"HealthCheckGracePeriod": 300,
"Instances": [
{
"InstanceId": "i-04bed6ef3b2000326",
"InstanceType": "t2.micro",
"AvailabilityZone": "us-east-1b",
"LifecycleState": "Terminating",
"HealthStatus": "Unhealthy",
"LaunchTemplate": {
"LaunchTemplateId": "lt-0452c90319362cbc5",
"LaunchTemplateName": "test-template",
"Version": "1"
},
...
},
...
]
}
I have a requirement, where Kubernetes service is generating an ELB for us with DNS name. I am adding that as an Alias for A, which is giving faster DNS propagation as compared to CNAME. I want to automated adding the entry via shell-script, but it's not working out, I get error as mentioned below. Please note, all e
Error log:
An error occurred (InvalidChangeBatch) when calling the ChangeResourceRecordSets operation: [Tried to create an alias that targets dualstack.AXXXXXXXXX-46346364.eu-central-1.elb.amazonaws.com., type A in zone Z0jshgdjhdg, but the alias target name does not lie within the target zone, Tried to create an alias that targets dualstack.asdgdfhgdfh-56767687.eu-central-1.elb.amazonaws.com., type A in zone Z0XXXXX, but that target was not found]
Command used:
aws route53 change-resource-record-sets --hosted-zone-id /hostedzone/Z0XXXXXXX --change-batch '{"Changes": [ { "Action": "UPSERT", "ResourceRecordSet": { "Name": "test.dev.domain.com", "Type": "A", "AliasTarget":{ "HostedZoneId": "Z0XXXXX","DNSName": "aesdgdfg-46456.eu-central-1.elb.amazonaws.com","EvaluateTargetHealth": false} } } ]}'
What am I doing wrong?
Hosted Zone as asked:
Based on the comments.
The issue was caused by using wrong HostedZoneId in AliasTarget. The correct one is the HostedZoneId associated with your ALB. It can be found in the EC2 console, in Description tab of your ALB under Hosted zone option.
The command should use two HostedZoneIds:
aws route53 change-resource-record-sets \
--hosted-zone-id /hostedzone/<zone-id-from-route53>\
--change-batch \
'{"Changes": [ { "Action": "UPSERT", "ResourceRecordSet": { "Name": "test.dev.domain.com", "Type": "A", "AliasTarget":{ "HostedZoneId": "<zone-id-of-ALB>","DNSName": "aesdgdfg-46456.eu-central-1.elb.amazonaws.com","EvaluateTargetHealth": false} } } ]}'
According the AWS docs at here and here I should be able to automate a certificate creation and validation using cloudformation. Apparently when you specify a HostedZoneId in the DomainValidationOptions, it is supposed to create the required DNS record to complete the validation (at least that is what it seems from the very vague documentation). My CF template for the cert looks like this:
Resources:
MyAPICert:
Type: AWS::CertificateManager::Certificate
Properties:
DomainName: xxxx.dev.mydomain.io
DomainValidationOptions:
- DomainName: mydomain.io
HostedZoneId: /hostedzone/Z03XXXXXXXXXXXX
ValidationMethod: DNS
'mydomain.io' (changed of course) was registered using AWS as registrar as the documents say must be the case for automated validation to work.
This template above is included in a serverless.yml as a resource. However, when I deploy, the stack creation is just stuck waiting for the DNS record - i.e. it does not add the required CNAME entry as I understand it is supposed to do and as such the stack is stuck.
Has anyone gotten this feature to work?
And, yes, I know about the 3rd party custom resources that try to do the same thing, I don't want to use them if CF is supposed to do this natively now.
I hit the same issue. You need to specify the full domain name including the host in the DomainValidationOptions DomainName parameter, and just specify the hosted zone id:
Resources:
MyAPICert:
Type: AWS::CertificateManager::Certificate
Properties:
DomainName: xxxx.dev.mydomain.io
DomainValidationOptions:
- DomainName: xxxx.dev.mydomain.io
HostedZoneId: Z03XXXXXXXXXXXX
ValidationMethod: DNS
In my testing, the Route53 validation record was added about a minute after running the stack, and the domain successfully validated itslef after about 15 minutes.
If this is stuck as in progress for a long time, it could be that you are using a Private Hosted Zone when you need to use the Public one. Probably you don't use a private CA.
That process should take 2-3 minutes, not more than that.
I just deployed the below template to CloudFormation and it successfully created the validation DNS records and authorised the certificate.
If you were to pass the parameters SiteDnsZoneName=mydomain.io. and SiteDnsZoneId=ABCDEFGHIJKLMNOPQRSTU it would create a SAN cert that covers both mydomain.io and *.mydomain.io
{
"Description": "Deploy wildcard SAN cert inc bare domain. (Must deploy cert to us-east-1 for CloudFront)",
"Parameters": {
"SiteDnsZoneName": {
"Type": "String",
"MinLength": 4,
"Description": "DNS Zone",
"Default": "example.com"
},
"SiteDnsZoneId": {
"Type": "String",
"MinLength": 8,
"Description": "DNS Zone Id",
"Default": "ABCDEFGHIJKLMNOPQRSTU"
}
},
"Resources": {
"SiteCertificate": {
"Type": "AWS::CertificateManager::Certificate",
"Properties": {
"DomainName": {
"Fn::Join": [
".",
[
"*",
{
"Ref": "SiteDnsZoneName"
}
]
]
},
"SubjectAlternativeNames": [
{
"Ref": "SiteDnsZoneName"
}
],
"DomainValidationOptions": [
{
"DomainName": {
"Ref": "SiteDnsZoneName"
},
"HostedZoneId": {
"Ref": "SiteDnsZoneId"
}
}
],
"ValidationMethod": "DNS"
}
}
}
}
Note: If you want to use a cert in CloudFront you have to deploy the cert in us-east-1.
Note 2: Route53 needs to be hosting your DNS Zone, but theres no requirement on AWS being the registrar. Your domain can be registered with any provider, so long as you use the AWS name servers provided by Route53 when you add the zone.
I need to create Elastic Beanstalk Environment from boto3.
For which I guess the API sequence should be:
create_application()
From this API we get the "Application Name":
create_environment(kwargs)
Here i am passing below json as kwargs to api
{
"ApplicationName": "APP-NAME",
"EnvironmentName": "ABC-Nodejs",
"CNAMEPrefix": "ABC-Neptune",
"SolutionStackName": "64bit Amazon Linux 2016.03 v2.1.1 running Node.js"
}
Questions:
How to mention that the Environment EC2 should be attached to which
VPC and subnet
In which Subnet its ELB should be created
Any sample code will be helpful
Please Note: I have one public and one private subnet, we can control the creation of EC2 and ELB creation through subnet IDs
To set up dependent resources with your Environment you would have to use the Elastic Beanstalk Option Settings. Specifically for VPCs you can use the aws:ec2:vpc namespace, I've linked the documentation for those settings with that.
The code example would be something like this:
{
ApplicationName: "APP-NAME",
EnvironmentName: "ABC-Nodejs",
CNAMEPrefix: "ABC-Neptune",
SolutionStackName: "64bit Amazon Linux 2016.03 v2.1.1 running Node.js"
OptionSettings=[
{
'Namespace': 'aws:ec2:vpc',
'OptionName': 'VPCId',
'Value': 'vpc-12345678'
},
{
'Namespace': 'aws:ec2:vpc',
'OptionName': 'ELBSubnets',
'Value': 'subnet-11111111,subnet-22222222'
},
],
}
Thanks nbalas,
I am using below code to create EB.
Despite giving already created security group names in "aws:elb:loadbalancer" and "aws:autoscaling:launchconfiguration" it is creating new security groups and attaching them to EC2 instance and load balancer. So now both of the security groups new and old ones are attached to the resources. I don't want to create new security groups at all and want to use old ones only.
kwargs={
"ApplicationName": "Test",
"EnvironmentName": "ABC-Nodejs",
"CNAMEPrefix": "ABC-ABC",
"SolutionStackName": "64bit Amazon Linux 2016.03 v2.1.1 running Node.js",
"OptionSettings": [
{
"Namespace": "aws:ec2:vpc",
"OptionName": "Subnets",
"Value": "subnet-*******0"
},
{
"Namespace": "aws:ec2:vpc",
"OptionName": "ELBSubnets",
"Value": "subnet-********1"
},
{
"Namespace": "aws:elb:loadbalancer",
"OptionName": "SecurityGroups",
"Value": "sg-*********2"
},
{
"Namespace": "aws:autoscaling:launchconfiguration",
"OptionName": "SecurityGroups",
"Value": "sg-**********3"
}
]
}
response = client.create_environment(**kwargs)
Today when I launch an app using kubernetes over aws it exposes a publicly visible LoadBalancer Ingress URL, however to link that to my domain to make the app accessible to the public, I need to manually go into the aws route53 console in a browser on every launch. Can I update the aws route53 Resource Type A to match the latest Kubernetes LoadBalancer Ingress URL from the command line ?
Kubernetes over gcloud shares this challenge of having to either predefine a Static IP which is used in launch config or manually do a browser based domain linkage post launch. On aws I was hoping I could use something similar to this from the command line
aws route53domains update-domain-nameservers ???
__ OR __ can I predefine an aws kubernetes LoadBalancer Ingress similar to doing a predefined Static IP when over gcloud ?
to show the deployed app's LoadBalancer Ingress URL issue
kubectl describe svc
... output
Name: aaa-deployment-407
Namespace: ruptureofthemundaneplane
Labels: app=bbb
pod-template-hash=4076262206
Selector: app=bbb,pod-template-hash=4076262206
Type: LoadBalancer
IP: 10.0.51.82
LoadBalancer Ingress: a244bodhisattva79c17cf7-61619.us-east-1.elb.amazonaws.com
Port: port-1 80/TCP
NodePort: port-1 32547/TCP
Endpoints: 10.201.0.3:80
Port: port-2 443/TCP
NodePort: port-2 31248/TCP
Endpoints: 10.201.0.3:443
Session Affinity: None
No events.
UPDATE:
Getting error trying new command line technique (hat tip to #error2007s comment) ... issue this
aws route53 list-hosted-zones
... outputs
{
"HostedZones": [
{
"ResourceRecordSetCount": 6,
"CallerReference": "2D58A764-1FAC-DEB4-8AC7-AD37E74B94E6",
"Config": {
"PrivateZone": false
},
"Id": "/hostedzone/Z3II3949ZDMDXV",
"Name": "chainsawhaircut.com."
}
]
}
Important bit used below : hostedzone Z3II3949ZDMDXV
now I craft following using this Doc (and this Doc as well) as file /change-resource-record-sets.json (NOTE I can successfully change Type A using a similar cli call ... however I need to change Type A with an Alias Target of LoadBalancer Ingress URL)
{
"Comment": "Update record to reflect new IP address of fresh deploy",
"Changes": [{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "chainsawhaircut.com.",
"Type": "A",
"TTL": 60,
"AliasTarget": {
"HostedZoneId": "Z3II3949ZDMDXV",
"DNSName": "a244bodhisattva79c17cf7-61619.us-east-1.elb.amazonaws.com",
"EvaluateTargetHealth": false
}
}
}]
}
on command line I then issue
aws route53 change-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV --change-batch file:///change-resource-record-sets.json
which give this error message
An error occurred (InvalidInput) when calling the ChangeResourceRecordSets operation: Invalid request
Any insights ?
Here is the logic needed to update aws route53 Resource Record Type A with value from freshly minted kubernetes LoadBalancer Ingress URL
step 1 - identify your hostedzone Id by issuing
aws route53 list-hosted-zones
... from output here is clip for my domain
"Id": "/hostedzone/Z3II3949ZDMDXV",
... importantly never populate json with hostedzone Z3II3949ZDMDXV its only used as a cli parm ... there is a second similarly named token HostedZoneId which is entirely different
step 2 - see current value of your route53 domain record ... issue :
aws route53 list-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV --query "ResourceRecordSets[?Name == 'scottstensland.com.']"
... output
[
{
"AliasTarget": {
"HostedZoneId": "Z35SXDOTRQ7X7K",
"EvaluateTargetHealth": false,
"DNSName": "dualstack.asomepriorvalue39e7db-1867261689.us-east-1.elb.amazonaws.com."
},
"Type": "A",
"Name": "scottstensland.com."
},
{
"ResourceRecords": [
{
"Value": "ns-1238.awsdns-26.org."
},
{
"Value": "ns-201.awsdns-25.com."
},
{
"Value": "ns-969.awsdns-57.net."
},
{
"Value": "ns-1823.awsdns-35.co.uk."
}
],
"Type": "NS",
"Name": "scottstensland.com.",
"TTL": 172800
},
{
"ResourceRecords": [
{
"Value": "ns-1238.awsdns-26.org. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400"
}
],
"Type": "SOA",
"Name": "scottstensland.com.",
"TTL": 900
}
]
... in above notice value of
"HostedZoneId": "Z35SXDOTRQ7X7K",
which is the second similarly name token Do NOT use wrong Hosted Zone ID
step 3 - put below into your change file aws_route53_type_A.json (for syntax Doc see link mentioned in comment above)
{
"Comment": "Update record to reflect new DNSName of fresh deploy",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"AliasTarget": {
"HostedZoneId": "Z35SXDOTRQ7X7K",
"EvaluateTargetHealth": false,
"DNSName": "dualstack.a0b82c81f47d011e6b98a0a28439e7db-1867261689.us-east-1.elb.amazonaws.com."
},
"Type": "A",
"Name": "scottstensland.com."
}
}
]
}
To identify value for above field "DNSName" ... after the kubernetes app deploy on aws it responds with a LoadBalancer Ingress as shown in output of cli command :
kubectl describe svc --namespace=ruptureofthemundaneplane
... as in
LoadBalancer Ingress: a0b82c81f47d011e6b98a0a28439e7db-1867261689.us-east-1.elb.amazonaws.com
... even though my goal is to execute a command line call I can do this manually by getting into the aws console browser ... pull up my domain on route53 ...
... In this browser picklist editable text box (circled in green) I noticed the URL gets magically prepended with : dualstack. Previously I was missing that magic string ... so json key "DNSName" wants this
dualstack.a0b82c81f47d011e6b98a0a28439e7db-1867261689.us-east-1.elb.amazonaws.com.
finally execute the change request
aws route53 change-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV --change-batch file://./aws_route53_type_A.json
... output
{
"ChangeInfo": {
"Status": "PENDING",
"Comment": "Update record to reflect new DNSName of fresh deploy",
"SubmittedAt": "2016-07-13T14:53:02.789Z",
"Id": "/change/CFUX5R9XKGE1C"
}
}
.... now to confirm change is live run this to show record
aws route53 list-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV
You can also use external-dns project.
AWS specific setup can be found here
After installation it can be used with an annotation e.g.: external-dns.alpha.kubernetes.io/hostname: nginx.external-dns-test.my-org.com.
Note the IAM permissions needs to be set properly.