Getting EC2 instance information created using cloud formation template - python-2.7
I am creating one EC2 instance using below Cloud Formation template.
Named this template as 'dinesh.json'.
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "Dinesh template",
"Resources" : {
"MyEC2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : "ami-775e4f16",
"InstanceType" : "t2.micro",
"KeyName" : "****"
}
}
}
}
Now, using boto3 library, I am launching the above template.
import boto3
cft = boto3.client('cloudformation')
create_cft = cft.create_stack(StackName="Dinesh",TemplateURL=r'https://s3-us-west-2.amazonaws.com/dsp-bucket/dinesh.json')
print create_cft
This is running successfully and getting output as below :
{u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh/5b573240-548a-11e6-90a0-50a68a0bca36', 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '5b507be9-548a-11e6-8405-55192e2be20a', 'HTTPHeaders': {'x-amzn-requestid': '5b507be9-548a-11e6-8405-55192e2be20a', 'date': 'Thu, 28 Jul 2016 06:13:09 GMT', 'content-length': '376', 'content-type': 'text/xml'}}}
Now, I want to get information of above created EC2 instance like public IP, private IP and other info.
So, can any one please suggest me way how to retrieve the information of this specific EC2 instance?
Please let me know the various ways to doing the above thing apart from boto3.
Use the GetAtt function. For example if you have made an ec2 called bob, then adding this to your Output section will show the privateip
"Outputs": {
"AddressOfbob": {
"Description": "Domainame",
"Value": {
"Fn::GetAtt": [
"bob",
"PrivateIp"
]
}
}
}
Thanks #Vorsprung for the pointing towards right direction.
Along with this, I am adding a bit descriptive answer.
Method 1
Getting EC2 instance info when there is "Output" section present in cloud formation template
import boto3
import time
cft = boto3.client('cloudformation')
create_cft = cft.create_stack(StackName="Dinesh-1",TemplateURL=r'https://s3-us-west-2.amazonaws.com/bucket/dinesh.json')
print "Create Stack o/p - ",create_cft
#Just adding sleep so that stack creation come to the status of CREATE_COMPLETE
#More logic can be added to check the status of stack creation programmatically.
time.sleep(120)
des_stack = cft.describe_stacks(StackName="Dinesh-1")
print "Describe Stack o/p - ",des_stack
Output is
Create Stack o/p - {u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-1/a92318a0-54a7-11e6-b050-50d0184f2', 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': 'a91c023-54a7-11e6-ba43-67cc9d6ed45b', 'HTTPHeaders': {'x-amzn-requestid': 'a91c023-54a7-11e6-ba43-67cc9d6ed45b', 'date': 'Thu, 28 Jul 2016 09:42:55 GMT', 'content-length': '378', 'content-type': 'text/xml'}}}
Describe Stack o/p - {u'Stacks': [{u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-1/a92318a0-54a7-11e6-b050-50a0184f2', u'Description': 'Dinesh template', u'Tags': [], u'Outputs': [{u'Description': 'Private IP', u'OutputKey': 'PrivateIP', u'OutputValue': '172.3.28.221'}, {u'Description': 'Public IP', u'OutputKey': 'PublicIP', u'OutputValue': '52.5.203.173'}], u'CreationTime': datetime.datetime(2016, 7, 28, 9, 42, 55, 624000, tzinfo=tzutc()), u'StackName': 'Dinesh-1', u'NotificationARNs': [], u'StackStatus': 'CREATE_COMPLETE', u'DisableRollback': False}], 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': 'f19dc8ce-54a7-11e6-83e8-01451bce0ba', 'HTTPHeaders': {'x-amzn-requestid': 'f19dc8ce-54a7-11e6-83e8-01451b7ce0a', 'date': 'Thu, 28 Jul 2016 09:44:57 GMT', 'content-length': '1158', 'content-type': 'text/xml'}}}
In describe_stack output, you will get Public IP and Private IP of created EC2 instance.
Method 2
Getting EC2 instance info when there is no "Output" section present in cloud formation template
import boto3
import time
cft = boto3.client('cloudformation')
create_cft = cft.create_stack(StackName="Dinesh-2",TemplateURL=r'https://s3-us-west-2.amazonaws.com/dsp-bucket/dinesh.json')
print "Create Stack o/p - ",create_cft
#Just adding sleep so that stack creation come to the status of CREATE_COMPLETE
#More logic can be added to check the status of stack creation programmatically.
time.sleep(120)
list_stack_resp = cft.list_stack_resources(StackName="Dinesh-2")
print list_stack_resp
Output is
Create Stack o/p - {u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-2/7238154a8-11e6-9694-50a686be73f2', 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '7234f160-54a8-11e6-bda6-ef311cece04b', 'HTTPHeaders': {'x-amzn-requestid': '7234f160-54a8-11e6-bda6-ef311cece04b', 'date': 'Thu, 28 Jul 2016 09:48:32 GMT', 'content-length': '378', 'content-type': 'text/xml'}}}
{'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': 'baabaa79-54a8-11e6-90e7-9ba061bfa4c', 'HTTPHeaders': {'x-amzn-requestid': 'baabaa79-54a8-11e6-90e7-9bad061bf4c', 'date': 'Thu, 28 Jul 2016 09:50:33 GMT', 'content-length': '687', 'content-type': 'text/xml'}}, u'StackResourceSummaries': [{u'ResourceType': 'AWS::EC2::Instance', u'PhysicalResourceId': 'i-059f15aa', u'LastUpdatedTimestamp': datetime.datetime(2016, 7, 28, 9, 49, 23, 481000, tzinfo=tzutc()), u'ResourceStatus': 'CREATE_COMPLETE', u'LogicalResourceId': 'MyEC2Instance'}]}
From the output of list_stack_resource, get the 'PhysicalResourceId' which is 'i-059f15aa' in this case.
Then get the output of describe_instance of ec2 to get full info of EC2 instance created above.
import boto3
ec2 = boto3.client('ec2')
ec2_resp = ec2.describe_instances(InstanceIds=['i-059f15aa'])
print ec2_resp
Output is
{u'Reservations': [{u'OwnerId': '089691119308', u'ReservationId': 'r-7245ddb6', u'Groups': [], u'Instances': [{u'Monitoring': {u'State': 'disabled'}, u'PublicDnsName': 'ec2-52-42-17-44.us-west-2.compute.amazonaws.com', u'State': {u'Code': 16, u'Name': 'running'}, u'EbsOptimized': False, u'LaunchTime': datetime.datetime(2016, 7, 28, 9, 48, 37, tzinfo=tzutc()), u'PublicIpAddress': '52.42.10.44', u'PrivateIpAddress': '172.3.29.25', u'ProductCodes': [], u'VpcId': 'vpc-c60a2aa3', u'StateTransitionReason': '', u'InstanceId': 'i-059f15aa', u'ImageId': 'ami-775e4f16', u'PrivateDnsName': 'ip-172-31-29-25.us-west-2.compute.internal', u'KeyName': 'dsp', u'SecurityGroups': [{u'GroupName': 'default', u'GroupId': 'sg-53fdaa37'}], u'ClientToken': 'Dines-MyEC2-DJ1D05Q7A088', u'SubnetId': 'subnet-8d0136e8', u'InstanceType': 't2.micro', u'NetworkInterfaces': [{u'Status': 'in-use', u'MacAddress': '02:9f:ab:4a:3c:0b', u'SourceDestCheck': True, u'VpcId': 'vpc-c60a2aa3', u'Description': '', u'Association': {u'PublicIp': '52.42.170.44', u'PublicDnsName': 'ec2-52-42-170-44.us-west-2.compute.amazonaws.com', u'IpOwnerId': 'amazon'}, u'NetworkInterfaceId': 'eni-d5272ca8', u'PrivateIpAddresses': [{u'PrivateDnsName': 'ip-172-31-29-25.us-west-2.compute.internal', u'Association': {u'PublicIp': '52.42.170.44', u'PublicDnsName': 'ec2-52-42-170-44.us-west-2.compute.amazonaws.com', u'IpOwnerId': 'amazon'}, u'Primary': True, u'PrivateIpAddress': '172.31.29.25'}], u'PrivateDnsName': 'ip-172-31-29-25.us-west-2.compute.internal', u'Attachment': {u'Status': 'attached', u'DeviceIndex': 0, u'DeleteOnTermination': True, u'AttachmentId': 'eni-attach-f33c375f', u'AttachTime': datetime.datetime(2016, 7, 28, 9, 48, 37, tzinfo=tzutc())}, u'Groups': [{u'GroupName': 'default', u'GroupId': 'sg-53fdaa37'}], u'SubnetId': 'subnet-8d0136e8', u'OwnerId': '089691119308', u'PrivateIpAddress': '172.31.29.25'}], u'SourceDestCheck': True, u'Placement': {u'Tenancy': 'default', u'GroupName': '', u'AvailabilityZone': 'us-west-2b'}, u'Hypervisor': 'xen', u'BlockDeviceMappings': [{u'DeviceName': '/dev/sda1', u'Ebs': {u'Status': 'attached', u'DeleteOnTermination': True, u'VolumeId': 'vol-21bddea8', u'AttachTime': datetime.datetime(2016, 7, 28, 9, 48, 37, tzinfo=tzutc())}}], u'Architecture': 'x86_64', u'RootDeviceType': 'ebs', u'RootDeviceName': '/dev/sda1', u'VirtualizationType': 'hvm', u'Tags': [{u'Value': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-2/723ba810-54a8-11e6-9694-50a686be73f2', u'Key': 'aws:cloudformation:stack-id'}, {u'Value': 'MyEC2Instance', u'Key': 'aws:cloudformation:logical-id'}, {u'Value': 'Dinesh-2', u'Key': 'aws:cloudformation:stack-name'}], u'AmiLaunchIndex': 0}]}], 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '8adf8956-0d5a-4d1f-a821-67fec4b5bbf9', 'HTTPHeaders': {'transfer-encoding': 'chunked', 'vary': 'Accept-Encoding', 'server': 'AmazonEC2', 'content-type': 'text/xml;charset=UTF-8', 'date': 'Thu, 28 Jul 2016 09:55:45 GMT'}}}
Related
How to get aws cloudwatch metrics statistics using Boto3 AWS Lambda in Python?
I'm trying to getting a Metric DataPoints or Statics from an AWS/Lambda function, Here is the code i running: import boto3 from datetime import timedelta response = client.get_metric_statistics( Namespace='AWS/Lambda', MetricName='Invocations', Dimensions=[ { 'Name': 'lambda', 'Value': 'my_lambda_name' }, ], StartTime=datetime(2022, 10, 6), EndTime=datetime(2022, 10, 8), Period=300, Statistics=[ 'Sum' ], #Unit='Count/Second' #Unit='Count' ) response Currently I getting an Empty response as follows: {'Label': 'Invocations', 'Datapoints': [], 'ResponseMetadata': {'RequestId': 'd18c70ee-dedd-410f-afc1-4c32f726175c', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'd18c70ee-dedd-410f-afc1-4c32f726175c', 'content-type': 'text/xml', 'content-length': '334', 'date': 'Fri, 07 Oct 2022 21:29:50 GMT'}, 'RetryAttempts': 0}} However, I really sure that the lambda is currently Invoking as the following graph shows:
I just find a solution assigning the Dimensions Name and Value as follows: import boto3 from datetime import timedelta response = client.get_metric_statistics( Namespace='AWS/Lambda', MetricName='Invocations', Dimensions=[ { 'Name': 'FunctionName', 'Value': 'my_function_name' }, ], StartTime=datetime(2022, 10, 6), EndTime=datetime(2022, 10, 8), Period=300, Statistics=[ 'Sum' ], ) response
How to download the current version of a file from an S3 versioned bucket
I have objects with multiple versions and I am trying to compare which versions I can delete. I basically want to delete any version that has the same size of the current version. The problem that I am having is that I can't find out which of the returned versions is the latest/current. If I use the aws cli, it returns a field called 'IsLatest' but apparently, the boto3 version doesn't. The aws cli also always returns the StorageClass while boto3 doesn't in some scenarios apparently. Return from boto3: {'ResponseMetadata': {'RequestId': 'PHQFMDCF3AHQM6R1', 'HostId': 'b7PmgsVm6y30wfA9GExS+Rc659cu1DI4YFec3i7tvDBew8ob5tY0Mtz6q+yC9nTwdmAoykdV7Lo=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'b7PmgsVm6y30wfA9GExS+Rc659cu1DI4YFeR3i7tVDBeu8ab5tY0Mtz6X+yC9nTwdmAoykdV7Lo=', 'x-amz-request-id': 'PHQFMDTB32HQM6R1', 'date': 'Sat, 19 Feb 2022 22:42:14 GMT', 'last-modified': 'Thu, 17 Feb 2022 17:02:54 GMT', 'etag': '"55f146382684970d4970ae31b3d4b310"', 'x-amz-server-side-encryption': 'AES256', 'x-amz-version-id': 'gHm2D2uuosJQS6GpmuySU9uNSXN84cq9', 'accept-ranges': 'bytes', 'content-type': 'text/plain', 'server': 'AmazonS3', 'content-length': '969'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2022, 2, 17, 17, 2, 54, tzinfo=tzutc()), 'ContentLength': 969, 'ETag': '"55f141382684970d4970ae31b3d4b310"', 'VersionId': 'gHa2D2uuosJQS6GpmuySU9uNSXN84cR9', 'ContentType': 'text/plain', 'ServerSideEncryption': 'AES256', 'Metadata': {}, 'Body': <botocore.response.StreamingBody object at 0x10f29e1c0>} Versioning_Test/file1.txt Response from aws cli: { "ETag": "\"55f141382684970d4970ae31b3d4b310\"", "Size": 969, "StorageClass": "STANDARD", "Key": "Versioning_Test/file1.txt", "VersionId": "gHa2D2uuosJQS6GpmuySU9uNSXN84cR9", "IsLatest": true, "LastModified": "2022-02-17T17:02:54+00:00", "Owner": { "ID": "1e5bc34834bec07ae1bc55a5d07adab10d7d58da04ae761769339a914d1ab472" } }, Here is my python script: bucket_name = 'bucket-name' profile_name = 'aws-profile-name' key = '' session = boto3.session.Session(profile_name=profile_name) s3 = session.resource('s3') versions = s3.Bucket(bucket_name).object_versions.filter() for version in versions: print(version.object_key) obj = version.get() print(obj) #print("\t" + obj.get('VersionId'), obj.get('ContentLength'), obj.get('LastModified'), obj.get('IsLatest'), obj.get('StorageClass')) I am missing something?
You can list your object versions from a bucket using list_object_versions API: import boto3 bucket_name = 'bucket-name' profile_name = 'aws-profile-name' if __name__ == "__main__": session = boto3.Session(profile_name=profile_name) client = session.client('s3') response = client.list_object_versions(Bucket=bucket_name) for version in response['Versions']: print(f'Key: {version["Key"]}, Size: {version["Size"]} bytes, Latest: {version["IsLatest"]}' f' LastModified: {version["IsLatest"]}, StorageClass: {version["StorageClass"]}') You can notice that the response from AWS contains an IsLatest property as well.
Can't receive text message using SNS python Lambda
I've read everything possible about SNS and I can't figure out why any text messages are sent from an aws lambda function. My function can't be more simple: def send_sms(message): print ("sending message...") print(message) client = boto3.client("sns") response = client.publish( PhoneNumber="+336xxxxxxxxx", Message=message ) print (response) I'm using an SNSFullAccess for this lambda function. And my log for the print (response) are: {'MessageId': '6e9aa592-4ff7-5cf1-be92-ffe34ca16120', 'ResponseMetadata': {'RequestId': '5d73d693-b584-5ecb-926f-8ce2b6d2831a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '5d73d693-b584-5ecb-926f-8ce2b6d2831a', 'content-type': 'text/xml', 'content-length': '294', 'date': 'Sun, 30 Jan 2022 21:20:22 GMT'}, 'RetryAttempts': 0}}
Why am I getting "datetime.datetime" error while using lambda?
I am currently using AWS Lambda to write this script: import boto3 import json s3 = boto3.client("s3") def lambda_handler(event, context): bucket = "www.onlinecodecompiler.com" key = "aws_cli_bucket.txt" try: data = s3.get_object(Bucket=bucket, Key=key) json_data = data["Body"].read() return{ "response_code ": 200, "data": data } except Exception as e: print(e) raise(e) The txt file "aws_cli_bucket.txt" has the following sentence in it, "list all object in a bucket aws s3 ls s3:// bucket name". Its just a random text I had because I was just testing to see if lambda could read a text file from my S3 bucket but why am I getting an error stating datetime.datetime(2021, 3, 22, 19, 50, 48, tzinfo=tzutc()) is not JSON serializable I searched online and most people were getting this error because their code had something to do with date and time but mine does not. Any help would be greatly appreciated.
The error stems from the response which you are getting from the s3.get_object which contains 3 datetime objects { 'Body': StreamingBody(), 'DeleteMarker': True|False, 'AcceptRanges': 'string', 'Expiration': 'string', 'Restore': 'string', 'LastModified': datetime(2015, 1, 1), ... 'Expires': datetime(2015, 1, 1), ... 'ObjectLockRetainUntilDate': datetime(2015, 1, 1), } s3.get_object Before returning the you also have to remove the Body key as well, which contains StreamingBody() otherwise it will error for that as well. "errorMessage": "Unable to marshal response: <botocore.response.StreamingBody object at 0x7fa2370625d0> is not JSON serializable", Depending on your needs you can simply delete those fields as do as you want. import boto3 s3 = boto3.client('s3') def lambda_handler(event, context): try: data = s3.get_object(Bucket='enrichedobjects1', Key='x.py') json_data = data["Body"].read() del data['Body'] del data['LastModified'] return{ "response_code ": 200, "data": data } except Exception as e: print(e) raise(e) The above code produces the following response { "response_code ": 200, "data": { "ResponseMetadata": { "RequestId": "980JCEDBG0AWZH", "HostId": "Oscwxx/5str0NFoWbnP/f5SAS4euhiJPQUXx7uOs205fowd7ICNtU=", "HTTPStatusCode": 200, "HTTPHeaders": { "x-amz-id-2": "Oscwxx/5Xk3uv7LuFUDx7uOs205fowd7ICNtU=", "x-amz-request-id": "980JC0KEDBWZH", "date": "Mon, 22 Mar 2021 20:43:56 GMT", "last-modified": "Sat, 20 Mar 2021 17:35:25 GMT", "etag": "\"d41d8cd98f00b20498ecf8427e\"", "accept-ranges": "bytes", "content-type": "text/x-python", "content-length": "0", "server": "AmazonS3" }, "RetryAttempts": 1 }, "AcceptRanges": "bytes", "ContentLength": 0, "ETag": "\"d41d8cd98f00b00998ecf8427e\"", "ContentType": "text/x-python", "Metadata": {} } }
aws boto3 paginator of list_images
can someone please help me understand and do paginator codehere in this list_images code: When i run this to get list of images for ec2imagebuilder, in the responce i got the nexttoken so how to use this to list all images in next page/until end. client = boto3.client('imagebuilder') response = client.list_images(owner='Amazon') print(response) Response (Trucated result): {'ResponseMetadata': {'RequestId': 'f4b9e178-b959-4e23-be57-0c234fbec69d', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Sun, 10 May 2020 21:53:55 GMT', 'content-type': 'application/json', 'content-length': '8709', 'connection': 'keep-alive', 'x-amzn-requestid': 'f4b9e178-b959-4e23-be57-0c234fbec69d', 'x-amz-apigw-id': 'MVet5Hp0PHcFeaw=', 'x-amzn-trace-id': 'Root=1-5eb877f2-9bec8ecc200f1394f6b0d340;Sampled=1'}, 'RetryAttempts': 0}, 'requestId': 'f4b9e178-b959-4e23-be57-0c234fbec69d', 'imageVersionList': [{'arn': 'arn:aws:imagebuilder:us-west-2:aws:image/amazon-linux-2-x86/2019.11.21', 'name': 'Amazon Linux 2 x86', 'version': '2019.11.21', 'platform': 'Linux', 'owner': 'Amazon', 'dateCreated': '2019-11-30T07:37:51.495Z'}, {'arn': 'arn:aws:imagebuilder:us-west-2:aws:image/windows-server-2012-r2-rtm-english-core-x86/2019.11.19', 'name': 'Windows Server 2012 R2 RTM English Core x86', 'version': '2019.11.19', 'platform': 'Windows', 'owner': 'Amazon', 'dateCreated': '2019-11-30T07:38:07.177Z'}], 'nextToken': 'eyxxxMS4xOSIsICJBY2NvdW50SWQiOiAiNTgwMDg3NjIzMDA1In0sICJtYXhfcmVzdWx0cyI6IDI1LCAia2V5X2NvbmRpdGlvbnMiOiB7IkFjY291bnRJZCI6IHsiQXR0cmlidXRlVmFsdWVMaXN0IjogWyI1ODAwODc2MjMwdddiOiBmYWxzZSwgInNjYW5faW5kZXhfZm9yd2FyZCI6IHRydWUsICJleHBpcmF0aW9uX2RhdGUiOiAxNTg5MjM0MDM1fQ=='}
Based on the documentation, you can use the following code snippet to list all images owned by Amazon. client = boto3.client('imagebuilder') response = client.list_images(owner='Amazon') print(response['imageVersionList']) while 'nextToken' in response: response = client.list_images(owner='Amazon', nextToken=response['nextToken']) print(response['imageVersionList'])