aws boto3 paginator of list_images - amazon-web-services

can someone please help me understand and do paginator codehere in this list_images code:
When i run this to get list of images for ec2imagebuilder, in the responce i got the nexttoken so how to use this to list all images in next page/until end.
client = boto3.client('imagebuilder')
response = client.list_images(owner='Amazon')
print(response)
Response (Trucated result):
{'ResponseMetadata': {'RequestId': 'f4b9e178-b959-4e23-be57-0c234fbec69d', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Sun, 10 May 2020 21:53:55 GMT', 'content-type': 'application/json', 'content-length': '8709', 'connection': 'keep-alive', 'x-amzn-requestid': 'f4b9e178-b959-4e23-be57-0c234fbec69d', 'x-amz-apigw-id': 'MVet5Hp0PHcFeaw=', 'x-amzn-trace-id': 'Root=1-5eb877f2-9bec8ecc200f1394f6b0d340;Sampled=1'}, 'RetryAttempts': 0}, 'requestId': 'f4b9e178-b959-4e23-be57-0c234fbec69d', 'imageVersionList': [{'arn': 'arn:aws:imagebuilder:us-west-2:aws:image/amazon-linux-2-x86/2019.11.21', 'name': 'Amazon Linux 2 x86', 'version': '2019.11.21', 'platform': 'Linux', 'owner': 'Amazon', 'dateCreated': '2019-11-30T07:37:51.495Z'}, {'arn': 'arn:aws:imagebuilder:us-west-2:aws:image/windows-server-2012-r2-rtm-english-core-x86/2019.11.19', 'name': 'Windows Server 2012 R2 RTM English Core x86', 'version': '2019.11.19', 'platform': 'Windows', 'owner': 'Amazon', 'dateCreated': '2019-11-30T07:38:07.177Z'}], 'nextToken': 'eyxxxMS4xOSIsICJBY2NvdW50SWQiOiAiNTgwMDg3NjIzMDA1In0sICJtYXhfcmVzdWx0cyI6IDI1LCAia2V5X2NvbmRpdGlvbnMiOiB7IkFjY291bnRJZCI6IHsiQXR0cmlidXRlVmFsdWVMaXN0IjogWyI1ODAwODc2MjMwdddiOiBmYWxzZSwgInNjYW5faW5kZXhfZm9yd2FyZCI6IHRydWUsICJleHBpcmF0aW9uX2RhdGUiOiAxNTg5MjM0MDM1fQ=='}


Based on the documentation, you can use the following code snippet to list all images owned by Amazon.
client = boto3.client('imagebuilder')
response = client.list_images(owner='Amazon')
print(response['imageVersionList'])
while 'nextToken' in response:
response = client.list_images(owner='Amazon', nextToken=response['nextToken'])
print(response['imageVersionList'])

Related

How to get aws cloudwatch metrics statistics using Boto3 AWS Lambda in Python?

I'm trying to getting a Metric DataPoints or Statics from an AWS/Lambda function, Here is the code i running:
import boto3
from datetime import timedelta
response = client.get_metric_statistics(
Namespace='AWS/Lambda',
MetricName='Invocations',
Dimensions=[
{
'Name': 'lambda',
'Value': 'my_lambda_name'
},
],
StartTime=datetime(2022, 10, 6),
EndTime=datetime(2022, 10, 8),
Period=300,
Statistics=[
'Sum'
],
#Unit='Count/Second'
#Unit='Count'
)
response
Currently I getting an Empty response as follows:
{'Label': 'Invocations',
'Datapoints': [],
'ResponseMetadata': {'RequestId': 'd18c70ee-dedd-410f-afc1-4c32f726175c',
'HTTPStatusCode': 200,
'HTTPHeaders': {'x-amzn-requestid': 'd18c70ee-dedd-410f-afc1-4c32f726175c',
'content-type': 'text/xml',
'content-length': '334',
'date': 'Fri, 07 Oct 2022 21:29:50 GMT'},
'RetryAttempts': 0}}
However, I really sure that the lambda is currently Invoking as the following graph shows:

I just find a solution assigning the Dimensions Name and Value as follows:
import boto3
from datetime import timedelta
response = client.get_metric_statistics(
Namespace='AWS/Lambda',
MetricName='Invocations',
Dimensions=[
{
'Name': 'FunctionName',
'Value': 'my_function_name'
},
],
StartTime=datetime(2022, 10, 6),
EndTime=datetime(2022, 10, 8),
Period=300,
Statistics=[
'Sum'
],
)
response

How to download the current version of a file from an S3 versioned bucket

I have objects with multiple versions and I am trying to compare which versions I can delete. I basically want to delete any version that has the same size of the current version.
The problem that I am having is that I can't find out which of the returned versions is the latest/current.
If I use the aws cli, it returns a field called 'IsLatest' but apparently, the boto3 version doesn't.
The aws cli also always returns the StorageClass while boto3 doesn't in some scenarios apparently.
Return from boto3:
{'ResponseMetadata': {'RequestId': 'PHQFMDCF3AHQM6R1', 'HostId': 'b7PmgsVm6y30wfA9GExS+Rc659cu1DI4YFec3i7tvDBew8ob5tY0Mtz6q+yC9nTwdmAoykdV7Lo=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'b7PmgsVm6y30wfA9GExS+Rc659cu1DI4YFeR3i7tVDBeu8ab5tY0Mtz6X+yC9nTwdmAoykdV7Lo=', 'x-amz-request-id': 'PHQFMDTB32HQM6R1', 'date': 'Sat, 19 Feb 2022 22:42:14 GMT', 'last-modified': 'Thu, 17 Feb 2022 17:02:54 GMT', 'etag': '"55f146382684970d4970ae31b3d4b310"', 'x-amz-server-side-encryption': 'AES256', 'x-amz-version-id': 'gHm2D2uuosJQS6GpmuySU9uNSXN84cq9', 'accept-ranges': 'bytes', 'content-type': 'text/plain', 'server': 'AmazonS3', 'content-length': '969'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2022, 2, 17, 17, 2, 54, tzinfo=tzutc()), 'ContentLength': 969, 'ETag': '"55f141382684970d4970ae31b3d4b310"', 'VersionId': 'gHa2D2uuosJQS6GpmuySU9uNSXN84cR9', 'ContentType': 'text/plain', 'ServerSideEncryption': 'AES256', 'Metadata': {}, 'Body': <botocore.response.StreamingBody object at 0x10f29e1c0>}
Versioning_Test/file1.txt
Response from aws cli:
{
"ETag": "\"55f141382684970d4970ae31b3d4b310\"",
"Size": 969,
"StorageClass": "STANDARD",
"Key": "Versioning_Test/file1.txt",
"VersionId": "gHa2D2uuosJQS6GpmuySU9uNSXN84cR9",
"IsLatest": true,
"LastModified": "2022-02-17T17:02:54+00:00",
"Owner": {
"ID": "1e5bc34834bec07ae1bc55a5d07adab10d7d58da04ae761769339a914d1ab472"
}
},
Here is my python script:
bucket_name = 'bucket-name'
profile_name = 'aws-profile-name'
key = ''
session = boto3.session.Session(profile_name=profile_name)
s3 = session.resource('s3')
versions = s3.Bucket(bucket_name).object_versions.filter()
for version in versions:
print(version.object_key)
obj = version.get()
print(obj)
#print("\t" + obj.get('VersionId'), obj.get('ContentLength'), obj.get('LastModified'), obj.get('IsLatest'), obj.get('StorageClass'))
I am missing something?

You can list your object versions from a bucket using list_object_versions API:
import boto3
bucket_name = 'bucket-name'
profile_name = 'aws-profile-name'
if __name__ == "__main__":
session = boto3.Session(profile_name=profile_name)
client = session.client('s3')
response = client.list_object_versions(Bucket=bucket_name)
for version in response['Versions']:
print(f'Key: {version["Key"]}, Size: {version["Size"]} bytes, Latest: {version["IsLatest"]}'
f' LastModified: {version["IsLatest"]}, StorageClass: {version["StorageClass"]}')
You can notice that the response from AWS contains an IsLatest property as well.

Can't receive text message using SNS python Lambda

I've read everything possible about SNS and I can't figure out why any text messages are sent from an aws lambda function.
My function can't be more simple:
def send_sms(message):
print ("sending message...")
print(message)
client = boto3.client("sns")
response = client.publish(
PhoneNumber="+336xxxxxxxxx",
Message=message
)
print (response)
I'm using an SNSFullAccess for this lambda function.
And my log for the print (response) are:
{'MessageId': '6e9aa592-4ff7-5cf1-be92-ffe34ca16120',
'ResponseMetadata': {'RequestId':
'5d73d693-b584-5ecb-926f-8ce2b6d2831a', 'HTTPStatusCode': 200,
'HTTPHeaders': {'x-amzn-requestid':
'5d73d693-b584-5ecb-926f-8ce2b6d2831a', 'content-type': 'text/xml',
'content-length': '294', 'date': 'Sun, 30 Jan 2022 21:20:22 GMT'},
'RetryAttempts': 0}}

AWS SES Email Verification RequestId status

Working on AWS SES Email Verification, after verifying an email i get back a RequestId from the response. Im trying to find a way to get an update form that RequestId i cant find a endpoint or method that can give me an update on this RequestId status.
{
ResponseMetadata: { RequestId: '1234567890' }
}
Here is the docs that im using for the Email Verification https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-email-addresses-procedure.html

You can't track the status of email verification using the RequestId received from SES email verification response. Pasting a sample response from SES email verification.
{'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': 'abcd-xyz-123', 'HTTPHeaders': {'date': 'Wed, 26 May 2021 04:44:03 GMT', 'x-amzn-requestid': '98768888-1111-qwaq-2222', 'content-length': '248', 'content-type': 'text/xml', 'connection': 'keep-alive'}}}
To get the status of the email verification, you can try list_verified_email_addresses operation. It list out all the verified email addresses. Check whether your required email address is listing in VerifiedEmailAddresses. If it is not there then it is not yet verified.
import boto3
from botocore.config import Config
my_config = Config(region_name = 'us-west-2')
ses = boto3.client('ses', config=my_config)
response = ses.list_verified_email_addresses()
print(response)
Response:
{'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': 'xxxxx', 'HTTPHeaders': {'date': 'Wed, 26 May 2021 05:25:00 GMT', 'x-amzn-requestid': 'xxxxxx', 'content-length': '412', 'content-type': 'text/xml', 'connection': 'keep-alive'}}, u'VerifiedEmailAddresses': ['example#gmail.com']}

Getting EC2 instance information created using cloud formation template

I am creating one EC2 instance using below Cloud Formation template.
Named this template as 'dinesh.json'.
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "Dinesh template",
"Resources" : {
"MyEC2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : "ami-775e4f16",
"InstanceType" : "t2.micro",
"KeyName" : "****"
}
}
}
}
Now, using boto3 library, I am launching the above template.
import boto3
cft = boto3.client('cloudformation')
create_cft = cft.create_stack(StackName="Dinesh",TemplateURL=r'https://s3-us-west-2.amazonaws.com/dsp-bucket/dinesh.json')
print create_cft
This is running successfully and getting output as below :
{u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh/5b573240-548a-11e6-90a0-50a68a0bca36', 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '5b507be9-548a-11e6-8405-55192e2be20a', 'HTTPHeaders': {'x-amzn-requestid': '5b507be9-548a-11e6-8405-55192e2be20a', 'date': 'Thu, 28 Jul 2016 06:13:09 GMT', 'content-length': '376', 'content-type': 'text/xml'}}}
Now, I want to get information of above created EC2 instance like public IP, private IP and other info.
So, can any one please suggest me way how to retrieve the information of this specific EC2 instance?
Please let me know the various ways to doing the above thing apart from boto3.

Use the GetAtt function. For example if you have made an ec2 called bob, then adding this to your Output section will show the privateip
"Outputs": {
"AddressOfbob": {
"Description": "Domainame",
"Value": {
"Fn::GetAtt": [
"bob",
"PrivateIp"
]
}
}
}

Thanks #Vorsprung for the pointing towards right direction.
Along with this, I am adding a bit descriptive answer.
Method 1
Getting EC2 instance info when there is "Output" section present in cloud formation template
import boto3
import time
cft = boto3.client('cloudformation')
create_cft = cft.create_stack(StackName="Dinesh-1",TemplateURL=r'https://s3-us-west-2.amazonaws.com/bucket/dinesh.json')
print "Create Stack o/p - ",create_cft
#Just adding sleep so that stack creation come to the status of CREATE_COMPLETE
#More logic can be added to check the status of stack creation programmatically.
time.sleep(120)
des_stack = cft.describe_stacks(StackName="Dinesh-1")
print "Describe Stack o/p - ",des_stack
Output is
Create Stack o/p - {u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-1/a92318a0-54a7-11e6-b050-50d0184f2', 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': 'a91c023-54a7-11e6-ba43-67cc9d6ed45b', 'HTTPHeaders': {'x-amzn-requestid': 'a91c023-54a7-11e6-ba43-67cc9d6ed45b', 'date': 'Thu, 28 Jul 2016 09:42:55 GMT', 'content-length': '378', 'content-type': 'text/xml'}}}
Describe Stack o/p - {u'Stacks': [{u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-1/a92318a0-54a7-11e6-b050-50a0184f2', u'Description': 'Dinesh template', u'Tags': [], u'Outputs': [{u'Description': 'Private IP', u'OutputKey': 'PrivateIP', u'OutputValue': '172.3.28.221'}, {u'Description': 'Public IP', u'OutputKey': 'PublicIP', u'OutputValue': '52.5.203.173'}], u'CreationTime': datetime.datetime(2016, 7, 28, 9, 42, 55, 624000, tzinfo=tzutc()), u'StackName': 'Dinesh-1', u'NotificationARNs': [], u'StackStatus': 'CREATE_COMPLETE', u'DisableRollback': False}], 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': 'f19dc8ce-54a7-11e6-83e8-01451bce0ba', 'HTTPHeaders': {'x-amzn-requestid': 'f19dc8ce-54a7-11e6-83e8-01451b7ce0a', 'date': 'Thu, 28 Jul 2016 09:44:57 GMT', 'content-length': '1158', 'content-type': 'text/xml'}}}
In describe_stack output, you will get Public IP and Private IP of created EC2 instance.
Method 2
Getting EC2 instance info when there is no "Output" section present in cloud formation template
import boto3
import time
cft = boto3.client('cloudformation')
create_cft = cft.create_stack(StackName="Dinesh-2",TemplateURL=r'https://s3-us-west-2.amazonaws.com/dsp-bucket/dinesh.json')
print "Create Stack o/p - ",create_cft
#Just adding sleep so that stack creation come to the status of CREATE_COMPLETE
#More logic can be added to check the status of stack creation programmatically.
time.sleep(120)
list_stack_resp = cft.list_stack_resources(StackName="Dinesh-2")
print list_stack_resp
Output is
Create Stack o/p - {u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-2/7238154a8-11e6-9694-50a686be73f2', 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '7234f160-54a8-11e6-bda6-ef311cece04b', 'HTTPHeaders': {'x-amzn-requestid': '7234f160-54a8-11e6-bda6-ef311cece04b', 'date': 'Thu, 28 Jul 2016 09:48:32 GMT', 'content-length': '378', 'content-type': 'text/xml'}}}
{'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': 'baabaa79-54a8-11e6-90e7-9ba061bfa4c', 'HTTPHeaders': {'x-amzn-requestid': 'baabaa79-54a8-11e6-90e7-9bad061bf4c', 'date': 'Thu, 28 Jul 2016 09:50:33 GMT', 'content-length': '687', 'content-type': 'text/xml'}}, u'StackResourceSummaries': [{u'ResourceType': 'AWS::EC2::Instance', u'PhysicalResourceId': 'i-059f15aa', u'LastUpdatedTimestamp': datetime.datetime(2016, 7, 28, 9, 49, 23, 481000, tzinfo=tzutc()), u'ResourceStatus': 'CREATE_COMPLETE', u'LogicalResourceId': 'MyEC2Instance'}]}
From the output of list_stack_resource, get the 'PhysicalResourceId' which is 'i-059f15aa' in this case.
Then get the output of describe_instance of ec2 to get full info of EC2 instance created above.
import boto3
ec2 = boto3.client('ec2')
ec2_resp = ec2.describe_instances(InstanceIds=['i-059f15aa'])
print ec2_resp
Output is
{u'Reservations': [{u'OwnerId': '089691119308', u'ReservationId': 'r-7245ddb6', u'Groups': [], u'Instances': [{u'Monitoring': {u'State': 'disabled'}, u'PublicDnsName': 'ec2-52-42-17-44.us-west-2.compute.amazonaws.com', u'State': {u'Code': 16, u'Name': 'running'}, u'EbsOptimized': False, u'LaunchTime': datetime.datetime(2016, 7, 28, 9, 48, 37, tzinfo=tzutc()), u'PublicIpAddress': '52.42.10.44', u'PrivateIpAddress': '172.3.29.25', u'ProductCodes': [], u'VpcId': 'vpc-c60a2aa3', u'StateTransitionReason': '', u'InstanceId': 'i-059f15aa', u'ImageId': 'ami-775e4f16', u'PrivateDnsName': 'ip-172-31-29-25.us-west-2.compute.internal', u'KeyName': 'dsp', u'SecurityGroups': [{u'GroupName': 'default', u'GroupId': 'sg-53fdaa37'}], u'ClientToken': 'Dines-MyEC2-DJ1D05Q7A088', u'SubnetId': 'subnet-8d0136e8', u'InstanceType': 't2.micro', u'NetworkInterfaces': [{u'Status': 'in-use', u'MacAddress': '02:9f:ab:4a:3c:0b', u'SourceDestCheck': True, u'VpcId': 'vpc-c60a2aa3', u'Description': '', u'Association': {u'PublicIp': '52.42.170.44', u'PublicDnsName': 'ec2-52-42-170-44.us-west-2.compute.amazonaws.com', u'IpOwnerId': 'amazon'}, u'NetworkInterfaceId': 'eni-d5272ca8', u'PrivateIpAddresses': [{u'PrivateDnsName': 'ip-172-31-29-25.us-west-2.compute.internal', u'Association': {u'PublicIp': '52.42.170.44', u'PublicDnsName': 'ec2-52-42-170-44.us-west-2.compute.amazonaws.com', u'IpOwnerId': 'amazon'}, u'Primary': True, u'PrivateIpAddress': '172.31.29.25'}], u'PrivateDnsName': 'ip-172-31-29-25.us-west-2.compute.internal', u'Attachment': {u'Status': 'attached', u'DeviceIndex': 0, u'DeleteOnTermination': True, u'AttachmentId': 'eni-attach-f33c375f', u'AttachTime': datetime.datetime(2016, 7, 28, 9, 48, 37, tzinfo=tzutc())}, u'Groups': [{u'GroupName': 'default', u'GroupId': 'sg-53fdaa37'}], u'SubnetId': 'subnet-8d0136e8', u'OwnerId': '089691119308', u'PrivateIpAddress': '172.31.29.25'}], u'SourceDestCheck': True, u'Placement': {u'Tenancy': 'default', u'GroupName': '', u'AvailabilityZone': 'us-west-2b'}, u'Hypervisor': 'xen', u'BlockDeviceMappings': [{u'DeviceName': '/dev/sda1', u'Ebs': {u'Status': 'attached', u'DeleteOnTermination': True, u'VolumeId': 'vol-21bddea8', u'AttachTime': datetime.datetime(2016, 7, 28, 9, 48, 37, tzinfo=tzutc())}}], u'Architecture': 'x86_64', u'RootDeviceType': 'ebs', u'RootDeviceName': '/dev/sda1', u'VirtualizationType': 'hvm', u'Tags': [{u'Value': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-2/723ba810-54a8-11e6-9694-50a686be73f2', u'Key': 'aws:cloudformation:stack-id'}, {u'Value': 'MyEC2Instance', u'Key': 'aws:cloudformation:logical-id'}, {u'Value': 'Dinesh-2', u'Key': 'aws:cloudformation:stack-name'}], u'AmiLaunchIndex': 0}]}], 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '8adf8956-0d5a-4d1f-a821-67fec4b5bbf9', 'HTTPHeaders': {'transfer-encoding': 'chunked', 'vary': 'Accept-Encoding', 'server': 'AmazonEC2', 'content-type': 'text/xml;charset=UTF-8', 'date': 'Thu, 28 Jul 2016 09:55:45 GMT'}}}