How to get ip address of spot instance created with boto3 - amazon-web-services

I am able to create spot instance with boto3 with the following code
import boto3
import datetime, random, string
client = boto3.client('ec2')
response = client.request_spot_instances(
DryRun=False,
ClientToken=''.join(random.choices(string.ascii_uppercase + string.digits, k=10)),
InstanceCount=1,
Type='one-time',
LaunchSpecification={
'ImageId': 'ami-053ebee0bc662f9b3',
'SecurityGroups': ['default'],
'InstanceType': 't3.medium',
'Placement': {
'AvailabilityZone': 'us-east-1a',
},
'BlockDeviceMappings': [
{
},
],
'EbsOptimized': True,
'Monitoring': {
'Enabled': True
},
'SecurityGroupIds': [
'sg-03432e1f',
]
}
)
print(response)
And the response I got is below. How can I get ip address of all the machines created. I know that the request is asynchronous but if I want to poll for the details of created instances, what is the way?
{
"SpotInstanceRequests": [
{
"CreateTime": "2022-05-22 05:21:54+00:00",
"LaunchSpecification": {
"SecurityGroups": [
{
"GroupName": "default",
"GroupId": "sg-03432e1f"
}
],
"EbsOptimized": true,
"ImageId": "ami-053ebee0bc662f9b3",
"InstanceType": "t3.medium",
"Placement": {
"AvailabilityZone": "us-east-1a"
},
"SubnetId": "subnet-b9f4aadf",
"Monitoring": {
"Enabled": true
}
},
"ProductDescription": "Linux/UNIX",
"SpotInstanceRequestId": "sir-88tef9gh",
"SpotPrice": "0.041600",
"State": "open",
"Status": {
"Code": "pending-evaluation",
"Message": "Your Spot request has been submitted for review, and is pending evaluation.",
"UpdateTime": "2022-05-22 05:21:55+00:00"
},
"Type": "one-time",
"InstanceInterruptionBehavior": "terminate"
}
],
"ResponseMetadata": {
"RequestId": "ac39fdad-cb4d-4e26-82a5-1f55c1f0d5af",
"HTTPStatusCode": 200,
"HTTPHeaders": {
"x-amzn-requestid": "ac39fdad-cb4d-4e26-82a5-1f55c1f0d5af",
"cache-control": "no-cache, no-store",
"strict-transport-security": "max-age=31536000; includeSubDomains",
"content-type": "text/xml;charset=UTF-8",
"content-length": "1696",
"date": "Sun, 22 May 2022 05:21:54 GMT",
"server": "AmazonEC2"
},
"RetryAttempts": 0
}
}

The same as with regular instances. You have to get NetworkInterfaceId of the spot instances which you can get using describe_instances. The outcome of the call gives you InstanceLifecycle equal to spot for spot instances.
Once you have NetworkInterfaceId of your spot instances you can use describe_network_interfaces to get PublicIp associated with the interface.

Related

EKS aws-ebs-csi-driver failed to mount volume

We are using EKS varsion v1.17.17-eks-087e67
With installed aws-ebs-csi-driver components versions:
aws-ebs-csi-driver:v1.1.3
csi-provisioner:v2.1.1
csi-attacher:v3.1.0
csi-snapshotter:v3.0.3
csi-resizer:v1.0.0
When we create PVC driver could not mount volume. As I can see, AWS volume continuously creating and deleting (from cloud trail):
{
"eventVersion": "1.08",
"userIdentity": {
"type": "AssumedRole",
"principalId": "AROAV5QH66QYOM4FMMPFI:1631165222580844502",
"arn": "arn:aws:sts::XXXXXXXXXX:assumed-role/EKSEBSCSIServiceRole-cluster01-eks-external-sandbox/XXXXXXXXXXXXXXXXXXXXXXXX",
"accountId": "XXXXXXXXXX",
"accessKeyId": "ASIAV5QH66QYFCKRZG43",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "AROAV5QH66QYOM4FMMPFI",
"arn": "arn:aws:iam::XXXXXXXXXX:role/eks/EKSEBSCSIServiceRole-cluster01-eks-external-sandbox",
"accountId": "XXXXXXXXXX",
"userName": "EKSEBSCSIServiceRole-cluster01-eks-external-sandbox"
},
"webIdFederationData": {
"federatedProvider": "arn:aws:iam::XXXXXXXXXX:oidc-provider/oidc.eks.eu-central-1.amazonaws.com/id/XXXXXXXXXXXXXXXXXXXXXXXX",
"attributes": {}
},
"attributes": {
"creationDate": "2021-09-09T05:27:03Z",
"mfaAuthenticated": "false"
}
}
},
"eventTime": "2021-09-09T06:11:12Z",
"eventSource": "ec2.amazonaws.com",
"eventName": "CreateVolume",
"awsRegion": "eu-central-1",
"sourceIPAddress": "18.157.68.62",
"userAgent": "aws-sdk-go/1.35.37 (go1.15.6; linux; amd64) exec-env/aws-ebs-csi-driver-v1.1.3",
"requestParameters": {
"size": "8",
"zone": "eu-central-1a",
"volumeType": "gp2",
"encrypted": true,
"tagSpecificationSet": {
"items": [
{
"resourceType": "volume",
"tags": [
{
"key": "ebs.csi.aws.com/cluster",
"value": "true"
},
{
"key": "CSIVolumeName",
"value": "pvc-27fa1e04-c99d-48d2-9efa-0633ee3669d2"
},
{
"key": "kubernetes.io/created-for/pv/name",
"value": "pvc-27fa1e04-c99d-48d2-9efa-0633ee3669d2"
},
{
"key": "kubernetes.io/created-for/pvc/name",
"value": "data-postgres-postgresql-0"
},
{
"key": "kubernetes.io/created-for/pvc/namespace",
"value": "default"
}
]
}
]
}
},
"responseElements": {
"requestId": "5404a63c-a8d6-4bfa-b18f-ce1fba1060ee",
"volumeId": "vol-032b5c6671123cc35",
"size": "8",
"zone": "eu-central-1a",
"status": "creating",
"createTime": 1631167872000,
"volumeType": "gp2",
"iops": 100,
"encrypted": true,
"masterEncryptionKeyId": "arn:aws:kms:eu-central-1:XXXXXXXXXX:key/ef3b2237-00c3-4fd0-b556-91cda7f7db95",
"tagSet": {
"items": [
{
"key": "ebs.csi.aws.com/cluster",
"value": "true"
},
{
"key": "CSIVolumeName",
"value": "pvc-27fa1e04-c99d-48d2-9efa-0633ee3669d2"
},
{
"key": "kubernetes.io/created-for/pv/name",
"value": "pvc-27fa1e04-c99d-48d2-9efa-0633ee3669d2"
},
{
"key": "kubernetes.io/created-for/pvc/name",
"value": "data-postgres-postgresql-0"
},
{
"key": "kubernetes.io/created-for/pvc/namespace",
"value": "default"
}
]
},
"multiAttachEnabled": false
},
"requestID": "5404a63c-a8d6-4bfa-b18f-ce1fba1060ee",
"eventID": "0941702c-119c-45fb-8c9e-6ef8918db6da",
"readOnly": false,
"eventType": "AwsApiCall",
"managementEvent": true,
"recipientAccountId": "XXXXXXXXXX",
"eventCategory": "Management"
}
"eventTime": "2021-09-09T06:11:15Z",
"eventSource": "ec2.amazonaws.com",
"eventName": "DeleteVolume",
"awsRegion": "eu-central-1",
"sourceIPAddress": "x.x.x.x",
"userAgent": "aws-sdk-go/1.35.37 (go1.15.6; linux; amd64) exec-env/aws-ebs-csi-driver-v1.1.3",
"errorCode": "Client.InvalidVolume.NotFound",
"errorMessage": "The volume 'vol-032b5c6671123cc35' does not exist.",
"requestParameters": {
"volumeId": "vol-032b5c6671123cc35"
},
"responseElements": null,
"requestID": "3cf2ce00-5845-436b-8470-3e1918dd24af",
"eventID": "e5fbd13c-fc72-4cc1-9468-2a928d52a186",
"readOnly": false,
"eventType": "AwsApiCall",
"managementEvent": true,
"recipientAccountId": "XXXXXXXXXX",
"eventCategory": "Management"
}
But eventually provisioner could not find this volume
0909 06:11:12.088851 1 controller.go:1332] provision "default/data-postgres-postgresql-0" class "ebs-default": started
I0909 06:11:12.089028 1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"data-postgres-postgresql-0", UID:"27fa1e04-c99d-48d2-9efa-0633ee3669d2", APIVersion:"v1", ResourceVersion:"145344106", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/data-postgres-postgresql-0"
I0909 06:11:15.565942 1 controller.go:1099] Final error received, removing PVC 27fa1e04-c99d-48d2-9efa-0633ee3669d2 from claims in progress
W0909 06:11:15.565962 1 controller.go:958] Retrying syncing claim "27fa1e04-c99d-48d2-9efa-0633ee3669d2", failure 18
E0909 06:11:15.565981 1 controller.go:981] error syncing claim "27fa1e04-c99d-48d2-9efa-0633ee3669d2": failed to provision volume with StorageClass "ebs-default": rpc error: code = Internal desc = Could not create volume "pvc-27fa1e04-c99d-48d2-9efa-0633ee3669d2": failed to get an available volume in EC2: InvalidVolume.NotFound: The volume 'vol-032b5c6671123cc35' does not exist.
status code: 400, request id: a396c26c-71c6-4c88-8f2f-ebb3aa492447
I0909 06:11:15.566164 1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"data-postgres-postgresql-0", UID:"27fa1e04-c99d-48d2-9efa-0633ee3669d2", APIVersion:"v1", ResourceVersion:"145344106", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "ebs-default": rpc error: code = Internal desc = Could not create volume "pvc-27fa1e04-c99d-48d2-9efa-0633ee3669d2": failed to get an available volume in EC2: InvalidVolume.NotFound: The volume 'vol-032b5c6671123cc35' does not exist.
status code: 400, request id: a396c26c-71c6-4c88-8f2f-ebb3aa492447
Here is the policy from AWS Role for annotated CA:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:CreateSnapshot",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteSnapshot",
"ec2:DeleteTags",
"ec2:DeleteVolume",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeInstances",
"ec2:DescribeSnapshots",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DescribeVolumesModifications",
"ec2:DetachVolume",
"ec2:ModifyVolume"
],
"Resource": "*"
}
]
}
Here is StorageClass:
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
storageclass.kubernetes.io/is-default-class: "true"
name: ebs-default
parameters:
csi.storage.k8s.io/fstype: ext4
encrypted: "true"
type: gp2
provisioner: ebs.csi.aws.com
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
~
We are running workers in eu-central-1 region in 3 AZ
I wrote a document. Please do let me know if this helps.
Use this GitHub Page Link!
Follow it from Step 2 - https://github.com/parjun8840/ekscsidriver/blob/main/README.md

How to upgrade AWS EC2 from t2.2xlarge to t3.2xlarge?

I have an t2.2xlarge AWS EC2 instance that i need to change it's type to t3.2xlarge.
But when i try to start it i get an
"Error starting instances The requested configuration is currently not
supported. Please check the documentation for supported
configurations."
When i run the check script everything is fine
https://github.com/awslabs/aws-support-tools/tree/master/EC2/NitroInstanceChecks
OK NVMe Module is installed and available on your instance
OK ENA Module with version is installed and available on your instance
OK fstab file looks fine and does not contain any device names.
And i also did all the checks described here
https://aws.amazon.com/premiumsupport/knowledge-center/boot-error-linux-nitro-instance/
aws ec2 describe-instances --instance-ids my-instance-id --query "Reservations[].Instances[].EnaSupport"
[
true
]
Is there anything else i should change to be able to start it as t3.2xlarge?
To reproduce:
Create an t2.2xlarge instance with default settings
Stop it and change type to t3.2xlarge
Try to start it
More detailed info about instance
aws ec2 describe-instances
{
"Reservations": [
{
"Groups": [],
"Instances": [
{
"AmiLaunchIndex": 0,
"ImageId": "ami-***********",
"InstanceId": "i-***********",
"InstanceType": "t2.2xlarge",
"KeyName": "***********",
"LaunchTime": "2020-11-24T06:11:41+00:00",
"Monitoring": {
"State": "disabled"
},
"Placement": {
"AvailabilityZone": "us-east-1e",
"GroupName": "",
"Tenancy": "default"
},
"PrivateDnsName": "ip-***********.ec2.internal",
"PrivateIpAddress": "***********",
"ProductCodes": [],
"PublicDnsName": "ec2-***********.compute-1.amazonaws.com",
"PublicIpAddress": "***********",
"State": {
"Code": 16,
"Name": "running"
},
"StateTransitionReason": "",
"SubnetId": "subnet-***********",
"VpcId": "vpc-***********",
"Architecture": "x86_64",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"AttachTime": "2020-10-06T05:07:35+00:00",
"DeleteOnTermination": true,
"Status": "attached",
"VolumeId": "vol-***********"
}
}
],
"ClientToken": "",
"EbsOptimized": false,
"EnaSupport": true,
"Hypervisor": "xen",
"NetworkInterfaces": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "***********.compute-1.amazonaws.com",
"PublicIp": "***********"
},
"Attachment": {
"AttachTime": "2020-10-06T05:07:34+00:00",
"AttachmentId": "eni-attach-***********",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"Status": "attached",
"NetworkCardIndex": 0
},
"Description": "",
"Groups": [
{
"GroupName": "launch-wizard-1",
"GroupId": "sg-***********"
}
],
"Ipv6Addresses": [],
"MacAddress": "***********",
"NetworkInterfaceId": "eni-***********",
"OwnerId": "***********",
"PrivateDnsName": "ip-***********.ec2.internal",
"PrivateIpAddress": "***********",
"PrivateIpAddresses": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-***********.compute-1.amazonaws.com",
"PublicIp": "***********"
},
"Primary": true,
"PrivateDnsName": "ip-***********.ec2.internal",
"PrivateIpAddress": "***********"
}
],
"SourceDestCheck": true,
"Status": "in-use",
"SubnetId": "subnet-***********",
"VpcId": "vpc-***********",
"InterfaceType": "interface"
}
],
"RootDeviceName": "/dev/sda1",
"RootDeviceType": "ebs",
"SecurityGroups": [
{
"GroupName": "launch-wizard-1",
"GroupId": "sg-***********"
}
],
"SourceDestCheck": true,
"Tags": [
{
"Key": "Name",
"Value": ""
}
],
"VirtualizationType": "hvm",
"CpuOptions": {
"CoreCount": 8,
"ThreadsPerCore": 1
},
"CapacityReservationSpecification": {
"CapacityReservationPreference": "open"
},
"HibernationOptions": {
"Configured": false
},
"MetadataOptions": {
"State": "applied",
"HttpTokens": "optional",
"HttpPutResponseHopLimit": 1,
"HttpEndpoint": "enabled"
},
"EnclaveOptions": {
"Enabled": false
}
}
],
"OwnerId": "***********",
"ReservationId": "r-***********"
}
]
}
I tried to launch a t3.2xlarge in us-east-1e and got the following error:
Your requested instance type (t3.2xlarge) is not supported in your requested Availability Zone (us-east-1e). Please retry your request by not specifying an Availability Zone or choosing us-east-1a, us-east-1b, us-east-1c, us-east-1d, us-east-1f.
AWS probably doesn't have t3.2xlarge instances available in this AZ.

AWS CLI Describe Target Groups

haii, i have the awscli result describing the target group as json format
{
"TargetHealthDescriptions": [
{
"Target": {
"Id": "1.1.1.1",
"Port": 123,
"AvailabilityZone": "ap-south-1"
},
"HealthCheckPort": "123",
"TargetHealth": {
"State": "healthy"
}
},
{
"Target": {
"Id": "2.2.2.2",
"Port": 123,
"AvailabilityZone": "ap-south-1"
},
"HealthCheckPort": "123",
"TargetHealth": {
"State": "healthy"
}
}
]
}
Im trying to make an awscli script to get a result like this
[
{
"Id": "1.1.1.1",
"Port": 123,
"Health": null
},
{
"Id": "2.2.2.2",
"Port": 123,
"Health": null
}
]
I've tried several query methods but I have problems getting a null value for health, its any errors in the query ?
ex query
--query 'TargetHealthDescriptions[*].Target.{Id:Id, Port:Port, Health:TargetHealth.{state:State}}' --output json
Try the below:
--query 'TargetHealthDescriptions[*].{Id:Target.Id,Port:Target.Port,Health:TargetHealth.State}'

Cloudformation does not support create vpc links in apigateway

In aws api gateway there is a section called API Link and I can manually set that.
The problem is I cannot find any section in cloudformation documentation on how I can create vpc link via cloud formation on api gateway.
Is it sth that cloudformation does not support or am I missing it?
You can use swagger to define an API Gateway using VPC Link. This is a complete CloudFormation template you can deploy to test it out...
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Test backend access via API Gateway. This template provisions a Regional API Gateway proxing requests to a backend via VPC Link and Direct Connect to on-premises resources using private ip addresses.",
"Parameters": {
"VPCId": {
"Description": "VPC Id for API Gateway VPC Link",
"Type": "AWS::EC2::VPC::Id"
},
"NLBSubnetList": {
"Type": "List<AWS::EC2::Subnet::Id>",
"Description": "Subnet Ids for provisioning load balancer supporting the VPC Link"
},
"BackendBaseEndpoint": {
"Description": "The backend service base url including protocol. e.g.: https://<url>",
"Type": "String",
"Default": "https://mybackend.dev.mycompany.com"
},
"TargetIpAddresses": {
"Type": "CommaDelimitedList",
"Description": "Comma separated list of NLB target ip addresses. Specify two entries.",
"Default": "10.78.80.1, 10.79.80.1"
}
},
"Resources": {
"API": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Name": "Test Api",
"Description": "Test Api using VPC_LINK and AWS_IAM authorisation",
"Body": {
"swagger": "2.0",
"info": {
"title": "Test Api"
},
"schemes": [
"https"
],
"paths": {
"/{proxy+}": {
"x-amazon-apigateway-any-method": {
"parameters": [
{
"name": "proxy",
"in": "path",
"required": true,
"type": "string"
}
],
"responses": {},
"security": [
{
"sigv4": []
}
],
"x-amazon-apigateway-integration": {
"responses": {
"default": {
"statusCode": "200"
}
},
"requestParameters": {
"integration.request.path.proxy": "method.request.path.proxy"
},
"uri": {
"Fn::Join": [
"",
[
{
"Ref": "BackendBaseEndpoint"
},
"/{proxy}"
]
]
},
"passthroughBehavior": "when_no_match",
"connectionType": "VPC_LINK",
"connectionId": "${stageVariables.vpcLinkId}",
"httpMethod": "GET",
"type": "http_proxy"
}
}
}
},
"securityDefinitions": {
"sigv4": {
"type": "apiKey",
"name": "Authorization",
"in": "header",
"x-amazon-apigateway-authtype": "awsSigv4"
}
}
},
"EndpointConfiguration": {
"Types": [
"REGIONAL"
]
}
},
"DependsOn": "VPCLink"
},
"APIStage": {
"Type": "AWS::ApiGateway::Stage",
"Properties": {
"StageName": "dev",
"Description": "dev Stage",
"RestApiId": {
"Ref": "API"
},
"DeploymentId": {
"Ref": "APIDeployment"
},
"MethodSettings": [
{
"ResourcePath": "/*",
"HttpMethod": "GET",
"MetricsEnabled": "true",
"DataTraceEnabled": "true",
"LoggingLevel": "ERROR"
}
],
"Variables": {
"vpcLinkId": {
"Ref": "VPCLink"
}
}
}
},
"APIDeployment": {
"Type": "AWS::ApiGateway::Deployment",
"Properties": {
"RestApiId": {
"Ref": "API"
},
"Description": "Test Deployment"
}
},
"VPCLink": {
"Type": "AWS::ApiGateway::VpcLink",
"Properties": {
"Description": "Vpc link to GIS platform",
"Name": "VPCLink",
"TargetArns": [
{
"Ref": "NLB"
}
]
}
},
"NLBTargetGroup": {
"Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
"Properties": {
"Name": "NLBTargetGroup",
"Port": 443,
"Protocol": "TCP",
"TargetGroupAttributes": [
{
"Key": "deregistration_delay.timeout_seconds",
"Value": "20"
}
],
"TargetType": "ip",
"Targets": [
{
"Id": { "Fn::Select" : [ "0", {"Ref": "TargetIpAddresses"} ] },
"Port": 443,
"AvailabilityZone": "all"
},
{
"Id": { "Fn::Select" : [ "1", {"Ref": "TargetIpAddresses"} ] },
"Port": 443,
"AvailabilityZone": "all"
}
],
"VpcId": {
"Ref": "VPCId"
},
"Tags": [
{
"Key": "Project",
"Value": "API and VPC Link Test"
}
]
}
},
"NLB": {
"Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties": {
"Type": "network",
"Scheme": "internal",
"Subnets": {
"Ref": "NLBSubnetList"
}
}
},
"NLBListener": {
"Type": "AWS::ElasticLoadBalancingV2::Listener",
"Properties": {
"DefaultActions": [
{
"Type": "forward",
"TargetGroupArn": {
"Ref": "NLBTargetGroup"
}
}
],
"LoadBalancerArn": {
"Ref": "NLB"
},
"Port": "443",
"Protocol": "TCP"
}
}
},
"Outputs": {
"NetworkLoadBalancerArn": {
"Value": {
"Ref": "NLB"
},
"Description": "The network elastic load balancer Amazon resource name"
}
}
}
Unfortunately, CloudFormation does not support API Gateway's VPC Links at this time.
You can create a Lambda-backed custom resource to manage the VPC Link using CloudFormation.
Here is a Lambda function (using python3.6) for a CloudFormation custom resource I use to manage VPC links:
import copy
import json
import re
import time
import boto3
from botocore.vendored import requests
SUCCESS = "SUCCESS"
FAILED = "FAILED"
FAILED_PHYSICAL_RESOURCE_ID = "FAILED_PHYSICAL_RESOURCE_ID"
class AddOrUpdateTargetArnsError(Exception):
def __init__(self):
self.message = 'Target arns are not allowed to be changed/added.'
super().__init__(self.message)
class FailedVpcLinkError(Exception):
def __init__(self, status_message):
self.message = f'statusMessages: {status_message}'
super().__init__(self.message)
def lambda_handler(event, context):
try:
_lambda_handler(event, context)
except Exception as e:
send(
event,
context,
response_status=FAILED,
# Do not fail on delete to avoid rollback failure
response_data=None,
physical_resource_id=event.get('PhysicalResourceId', FAILED_PHYSICAL_RESOURCE_ID),
reason=e
)
# Must raise, otherwise the Lambda will be marked as successful, and the exception
# will not be logged to CloudWatch logs.
raise
def _lambda_handler(event, context):
print("Received event: ")
print(event)
resource_type = event['ResourceType']
if resource_type != "Custom::ApiGatewayVpcLink":
raise ValueError(f'Unexpected resource_type: {resource_type}')
request_type = event['RequestType']
wait_for = event.get('WaitFor', None)
resource_properties = event['ResourceProperties']
physical_resource_id = event.get('PhysicalResourceId', None)
apigateway = boto3.client('apigateway')
if wait_for:
handle_self_invocation(
wait_for=wait_for,
physical_resource_id=physical_resource_id,
event=event,
context=context,
)
else:
if request_type == 'Create':
kwargs = dict(
name=resource_properties['Name'],
targetArns=resource_properties['TargetArns'],
description=resource_properties.get('Description', None)
)
response = apigateway.create_vpc_link(**kwargs)
event_copy = copy.deepcopy(event)
event_copy['WaitFor'] = 'CreateComplete'
event_copy['PhysicalResourceId'] = response['id']
print('Reinvoking function because VPC link creation is asynchronous')
relaunch_lambda(event=event_copy, context=context)
return
elif request_type == 'Update':
old_resource_properties = event['OldResourceProperties']
current_target_arns = apigateway.get_vpc_link(
vpcLinkId=physical_resource_id,
)['targetArns']
# must compare current_target_arns to resource_properties['TargetArns'], to protect against
# UPDATE created by UPDATE_FAILED. In that particular case, current_target_arns will be the same as
# resource_properties['TargetArns'] but different than old_resource_properties['TargetArns']
if set(current_target_arns) != set(resource_properties['TargetArns']) and \
set(resource_properties['TargetArns']) != set(old_resource_properties['TargetArns']):
raise AddOrUpdateTargetArnsError()
patch_operations = []
if resource_properties['Name'] != old_resource_properties['Name']:
patch_operations.append(dict(
op='replace',
path='/name',
value=resource_properties['Name'],
))
if 'Description' in resource_properties and 'Description' in old_resource_properties:
if resource_properties['Description'] != old_resource_properties['Description']:
patch_operations.append(dict(
op='replace',
path='/description',
value=resource_properties['Description'],
))
elif 'Description' in resource_properties and 'Description' not in old_resource_properties:
patch_operations.append(dict(
op='replace',
path='/description',
value=resource_properties['Description'],
))
elif 'Description' not in resource_properties and 'Description' in old_resource_properties:
patch_operations.append(dict(
op='replace',
path='/description',
value=None,
))
apigateway.update_vpc_link(
vpcLinkId=physical_resource_id,
patchOperations=patch_operations,
)
elif request_type == 'Delete':
delete = True
if physical_resource_id == FAILED_PHYSICAL_RESOURCE_ID:
delete = False
print('Custom resource was never properly created, skipping deletion.')
stack_name = re.match("arn:aws:cloudformation:.+:stack/(?P<stack_name>.+)/.+", event['StackId']).group('stack_name')
if stack_name in physical_resource_id:
delete = False
print(f'Skipping deletion, because VPC link was not created properly. Heuristic: stack name ({stack_name}) found in physical resource ID ({physical_resource_id})')
logical_resource_id = event['LogicalResourceId']
if logical_resource_id in physical_resource_id:
delete = False
print(f'Skipping deletion, because VPC link was not created properly. Heuristic: logical resource ID ({logical_resource_id}) found in physical resource ID ({physical_resource_id})')
if delete:
apigateway.delete_vpc_link(
vpcLinkId=physical_resource_id
)
event_copy = copy.deepcopy(event)
event_copy['WaitFor'] = 'DeleteComplete'
print('Reinvoking function because VPC link deletion is asynchronous')
relaunch_lambda(event=event_copy, context=context)
return
else:
print(f'Request type is {request_type}, doing nothing.')
send(
event,
context,
response_status=SUCCESS,
response_data=None,
physical_resource_id=physical_resource_id,
)
def handle_self_invocation(wait_for, physical_resource_id, event, context):
apigateway = boto3.client('apigateway')
if wait_for == 'CreateComplete':
print('Waiting for creation of VPC link: {vpc_link_id}'.format(vpc_link_id=physical_resource_id))
response = apigateway.get_vpc_link(
vpcLinkId=physical_resource_id,
)
status = response['status']
print('Status of VPC link {vpc_link_id} is {status}'.format(vpc_link_id=physical_resource_id, status=status))
if status == 'AVAILABLE':
send(
event,
context,
response_status=SUCCESS,
response_data=None,
physical_resource_id=physical_resource_id,
)
elif status == 'FAILED':
raise FailedVpcLinkError(status_message=response['statusMessage'])
elif status == 'PENDING':
# Sleeping here to avoid polluting CloudWatch Logs by reinvoking the Lambda too quickly
time.sleep(30)
relaunch_lambda(event, context)
else:
print('Unexpected status, doing nothing')
elif wait_for == 'DeleteComplete':
print('Waiting for deletion of VPC link: {vpc_link_id}'.format(vpc_link_id=physical_resource_id))
try:
response = apigateway.get_vpc_link(
vpcLinkId=physical_resource_id,
)
except apigateway.exceptions.NotFoundException:
print('VPC link {vpc_link_id} deleted successfully'.format(vpc_link_id=physical_resource_id))
send(
event,
context,
response_status=SUCCESS,
response_data=None,
physical_resource_id=physical_resource_id,
)
else:
status = response['status']
assert status == 'DELETING', f'status is {status}'
# Sleeping here to avoid polluting CloudWatch Logs by reinvoking the Lambda too quickly
time.sleep(10)
relaunch_lambda(event, context)
else:
raise ValueError(f'Unexpected WaitFor: {wait_for}')
def relaunch_lambda(event, context):
boto3.client("lambda").invoke(
FunctionName=context.function_name,
InvocationType='Event',
Payload=json.dumps(event),
)
def send(event, context, response_status, response_data, physical_resource_id, reason=None):
response_url = event['ResponseURL']
response_body = {
'Status': response_status,
'Reason': str(reason) if reason else 'See the details in CloudWatch Log Stream: ' + context.log_stream_name,
'PhysicalResourceId': physical_resource_id,
'StackId': event['StackId'],
'RequestId': event['RequestId'],
'LogicalResourceId': event['LogicalResourceId'],
'Data': response_data,
}
json_response_body = json.dumps(response_body)
headers = {
'content-type': '',
'content-length': str(len(json_response_body))
}
try:
requests.put(
response_url,
data=json_response_body,
headers=headers
)
except Exception as e:
print("send(..) failed executing requests.put(..): " + str(e))

Unable to create read replica to aurora cluster using AWS CLI

I'm trying to create read replica using the following command:
aws rds create-db-instance-read-replica --db-instance-identifier dbname-read --source-db-instance-identifier dbname --availability-zone us-east-1c
I'm getting the following error:
A client error (InvalidDBInstanceState) occurred when calling the CreateDBInstanceReadReplica operation: Automated backups are not enabled for this database instance. To enable automated backups, use ModifyDBInstance to set the backup retention period to a non-zero value.
I checked and the cluster is configured with automatic backups:
{
"DBInstances": [
{
"PubliclyAccessible": false,
"MasterUsername": "root",
"LicenseModel": "general-public-license",
"VpcSecurityGroups": [
{
"Status": "active",
"VpcSecurityGroupId": "sg"
}
],
"InstanceCreateTime": "2015-12-20T02:38:26.179Z",
"CopyTagsToSnapshot": false,
"OptionGroupMemberships": [
{
"Status": "in-sync",
"OptionGroupName": "default:aurora-5-6"
}
],
"PendingModifiedValues": {},
"Engine": "aurora",
"MultiAZ": false,
"DBSecurityGroups": [],
"DBParameterGroups": [
{
"DBParameterGroupName": "default.aurora5.6",
"ParameterApplyStatus": "in-sync"
}
],
"AutoMinorVersionUpgrade": true,
"PreferredBackupWindow": "03:44-04:14",
"DBSubnetGroup": {
"Subnets": [
{
"SubnetStatus": "Active",
"SubnetIdentifier": "subnet",
"SubnetAvailabilityZone": {
"Name": "us-east-1a"
}
},
{
"SubnetStatus": "Active",
"SubnetIdentifier": "subnet",
"SubnetAvailabilityZone": {
"Name": "us-east-1c"
}
}
],
"DBSubnetGroupName": "dev-subnet-group",
"VpcId": "vpc",
"DBSubnetGroupDescription": "dev-subnet-group",
"SubnetGroupStatus": "Complete"
},
"ReadReplicaDBInstanceIdentifiers": [],
"AllocatedStorage": 1,
*"BackupRetentionPeriod": 7,*
"PreferredMaintenanceWindow": "mon:10:11-mon:10:41",
"Endpoint": {
"Port": 3306,
"Address": "dbname.us-east-1.rds.amazonaws.com"
},
"DBInstanceStatus": "available",
"EngineVersion": "5.6.10a",
"AvailabilityZone": "us-east-1a",
"DBClusterIdentifier": "dbname",
"StorageType": "aurora",
"DbiResourceId": "db-**********",
"CACertificateIdentifier": "rds-ca-2015",
"StorageEncrypted": false,
"DBInstanceClass": "db.r3.large",
"DbInstancePort": 0,
"DBInstanceIdentifier": "dbname"
}
]
}
Any idea?
Thanks,
Roey
Aurora engine doesn't support
create-db-instance-read-replica
instead just creating another instance using
create-db-instance
with the option --db-cluster-identifier.
So the newly created instance will automatically sync with the writer/master will be promoted to read only automatically.