`We are running an AWS Glue job and we believe the following code snippet should return the AWS UserId from which the job is being triggered.
For ex, the following code was run with the user mmohanty
import boto3 client = boto3.client('sts') response = client.get_caller_identity() print('User ID:', response['UserId'])
The output is being shown as **AROA6CNCYWLF5MGCB5DF4:GlueJobRunnerSession **instead of IAM username.
The entire output of client.get_caller_identity() doesn't have any reference to IAM username.
{'UserId': 'AROA6CNCYWLF5MGCB5DF4:GlueJobRunnerSession', 'Account': '1234567', 'Arn': 'arn:aws:sts::12345678:assumed-role/xxx-GlueRole/GlueJobRunnerSession', 'ResponseMetadata': {'RequestId': 'bb43bd2b-4426-46b5-8457-aaaaaaa92116d', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'bb43bd2b-4426-46b5-8457-aaaaaaa2116d', 'content-type': 'text/xml', 'content-length': '459', 'date': 'Fri, 20 Jan 2023 17:31:27 GMT'}, 'RetryAttempts': 0}}
Please let us know how to get IAM username instead of the cryptic userid.
We are getting cryptic Userid instead of actual IAM username.`
I have objects with multiple versions and I am trying to compare which versions I can delete. I basically want to delete any version that has the same size of the current version.
The problem that I am having is that I can't find out which of the returned versions is the latest/current.
If I use the aws cli, it returns a field called 'IsLatest' but apparently, the boto3 version doesn't.
The aws cli also always returns the StorageClass while boto3 doesn't in some scenarios apparently.
Return from boto3:
{'ResponseMetadata': {'RequestId': 'PHQFMDCF3AHQM6R1', 'HostId': 'b7PmgsVm6y30wfA9GExS+Rc659cu1DI4YFec3i7tvDBew8ob5tY0Mtz6q+yC9nTwdmAoykdV7Lo=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'b7PmgsVm6y30wfA9GExS+Rc659cu1DI4YFeR3i7tVDBeu8ab5tY0Mtz6X+yC9nTwdmAoykdV7Lo=', 'x-amz-request-id': 'PHQFMDTB32HQM6R1', 'date': 'Sat, 19 Feb 2022 22:42:14 GMT', 'last-modified': 'Thu, 17 Feb 2022 17:02:54 GMT', 'etag': '"55f146382684970d4970ae31b3d4b310"', 'x-amz-server-side-encryption': 'AES256', 'x-amz-version-id': 'gHm2D2uuosJQS6GpmuySU9uNSXN84cq9', 'accept-ranges': 'bytes', 'content-type': 'text/plain', 'server': 'AmazonS3', 'content-length': '969'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2022, 2, 17, 17, 2, 54, tzinfo=tzutc()), 'ContentLength': 969, 'ETag': '"55f141382684970d4970ae31b3d4b310"', 'VersionId': 'gHa2D2uuosJQS6GpmuySU9uNSXN84cR9', 'ContentType': 'text/plain', 'ServerSideEncryption': 'AES256', 'Metadata': {}, 'Body': <botocore.response.StreamingBody object at 0x10f29e1c0>}
Versioning_Test/file1.txt
Response from aws cli:
{
"ETag": "\"55f141382684970d4970ae31b3d4b310\"",
"Size": 969,
"StorageClass": "STANDARD",
"Key": "Versioning_Test/file1.txt",
"VersionId": "gHa2D2uuosJQS6GpmuySU9uNSXN84cR9",
"IsLatest": true,
"LastModified": "2022-02-17T17:02:54+00:00",
"Owner": {
"ID": "1e5bc34834bec07ae1bc55a5d07adab10d7d58da04ae761769339a914d1ab472"
}
},
Here is my python script:
bucket_name = 'bucket-name'
profile_name = 'aws-profile-name'
key = ''
session = boto3.session.Session(profile_name=profile_name)
s3 = session.resource('s3')
versions = s3.Bucket(bucket_name).object_versions.filter()
for version in versions:
print(version.object_key)
obj = version.get()
print(obj)
#print("\t" + obj.get('VersionId'), obj.get('ContentLength'), obj.get('LastModified'), obj.get('IsLatest'), obj.get('StorageClass'))
I am missing something?
You can list your object versions from a bucket using list_object_versions API:
import boto3
bucket_name = 'bucket-name'
profile_name = 'aws-profile-name'
if __name__ == "__main__":
session = boto3.Session(profile_name=profile_name)
client = session.client('s3')
response = client.list_object_versions(Bucket=bucket_name)
for version in response['Versions']:
print(f'Key: {version["Key"]}, Size: {version["Size"]} bytes, Latest: {version["IsLatest"]}'
f' LastModified: {version["IsLatest"]}, StorageClass: {version["StorageClass"]}')
You can notice that the response from AWS contains an IsLatest property as well.
Working on AWS SES Email Verification, after verifying an email i get back a RequestId from the response. Im trying to find a way to get an update form that RequestId i cant find a endpoint or method that can give me an update on this RequestId status.
{
ResponseMetadata: { RequestId: '1234567890' }
}
Here is the docs that im using for the Email Verification https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-email-addresses-procedure.html
You can't track the status of email verification using the RequestId received from SES email verification response. Pasting a sample response from SES email verification.
{'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': 'abcd-xyz-123', 'HTTPHeaders': {'date': 'Wed, 26 May 2021 04:44:03 GMT', 'x-amzn-requestid': '98768888-1111-qwaq-2222', 'content-length': '248', 'content-type': 'text/xml', 'connection': 'keep-alive'}}}
To get the status of the email verification, you can try list_verified_email_addresses operation. It list out all the verified email addresses. Check whether your required email address is listing in VerifiedEmailAddresses. If it is not there then it is not yet verified.
import boto3
from botocore.config import Config
my_config = Config(region_name = 'us-west-2')
ses = boto3.client('ses', config=my_config)
response = ses.list_verified_email_addresses()
print(response)
Response:
{'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': 'xxxxx', 'HTTPHeaders': {'date': 'Wed, 26 May 2021 05:25:00 GMT', 'x-amzn-requestid': 'xxxxxx', 'content-length': '412', 'content-type': 'text/xml', 'connection': 'keep-alive'}}, u'VerifiedEmailAddresses': ['example#gmail.com']}
can someone please help me understand and do paginator codehere in this list_images code:
When i run this to get list of images for ec2imagebuilder, in the responce i got the nexttoken so how to use this to list all images in next page/until end.
client = boto3.client('imagebuilder')
response = client.list_images(owner='Amazon')
print(response)
Response (Trucated result):
{'ResponseMetadata': {'RequestId': 'f4b9e178-b959-4e23-be57-0c234fbec69d', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Sun, 10 May 2020 21:53:55 GMT', 'content-type': 'application/json', 'content-length': '8709', 'connection': 'keep-alive', 'x-amzn-requestid': 'f4b9e178-b959-4e23-be57-0c234fbec69d', 'x-amz-apigw-id': 'MVet5Hp0PHcFeaw=', 'x-amzn-trace-id': 'Root=1-5eb877f2-9bec8ecc200f1394f6b0d340;Sampled=1'}, 'RetryAttempts': 0}, 'requestId': 'f4b9e178-b959-4e23-be57-0c234fbec69d', 'imageVersionList': [{'arn': 'arn:aws:imagebuilder:us-west-2:aws:image/amazon-linux-2-x86/2019.11.21', 'name': 'Amazon Linux 2 x86', 'version': '2019.11.21', 'platform': 'Linux', 'owner': 'Amazon', 'dateCreated': '2019-11-30T07:37:51.495Z'}, {'arn': 'arn:aws:imagebuilder:us-west-2:aws:image/windows-server-2012-r2-rtm-english-core-x86/2019.11.19', 'name': 'Windows Server 2012 R2 RTM English Core x86', 'version': '2019.11.19', 'platform': 'Windows', 'owner': 'Amazon', 'dateCreated': '2019-11-30T07:38:07.177Z'}], 'nextToken': 'eyxxxMS4xOSIsICJBY2NvdW50SWQiOiAiNTgwMDg3NjIzMDA1In0sICJtYXhfcmVzdWx0cyI6IDI1LCAia2V5X2NvbmRpdGlvbnMiOiB7IkFjY291bnRJZCI6IHsiQXR0cmlidXRlVmFsdWVMaXN0IjogWyI1ODAwODc2MjMwdddiOiBmYWxzZSwgInNjYW5faW5kZXhfZm9yd2FyZCI6IHRydWUsICJleHBpcmF0aW9uX2RhdGUiOiAxNTg5MjM0MDM1fQ=='}
Based on the documentation, you can use the following code snippet to list all images owned by Amazon.
client = boto3.client('imagebuilder')
response = client.list_images(owner='Amazon')
print(response['imageVersionList'])
while 'nextToken' in response:
response = client.list_images(owner='Amazon', nextToken=response['nextToken'])
print(response['imageVersionList'])
I am creating one EC2 instance using below Cloud Formation template.
Named this template as 'dinesh.json'.
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "Dinesh template",
"Resources" : {
"MyEC2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : "ami-775e4f16",
"InstanceType" : "t2.micro",
"KeyName" : "****"
}
}
}
}
Now, using boto3 library, I am launching the above template.
import boto3
cft = boto3.client('cloudformation')
create_cft = cft.create_stack(StackName="Dinesh",TemplateURL=r'https://s3-us-west-2.amazonaws.com/dsp-bucket/dinesh.json')
print create_cft
This is running successfully and getting output as below :
{u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh/5b573240-548a-11e6-90a0-50a68a0bca36', 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '5b507be9-548a-11e6-8405-55192e2be20a', 'HTTPHeaders': {'x-amzn-requestid': '5b507be9-548a-11e6-8405-55192e2be20a', 'date': 'Thu, 28 Jul 2016 06:13:09 GMT', 'content-length': '376', 'content-type': 'text/xml'}}}
Now, I want to get information of above created EC2 instance like public IP, private IP and other info.
So, can any one please suggest me way how to retrieve the information of this specific EC2 instance?
Please let me know the various ways to doing the above thing apart from boto3.
Use the GetAtt function. For example if you have made an ec2 called bob, then adding this to your Output section will show the privateip
"Outputs": {
"AddressOfbob": {
"Description": "Domainame",
"Value": {
"Fn::GetAtt": [
"bob",
"PrivateIp"
]
}
}
}
Thanks #Vorsprung for the pointing towards right direction.
Along with this, I am adding a bit descriptive answer.
Method 1
Getting EC2 instance info when there is "Output" section present in cloud formation template
import boto3
import time
cft = boto3.client('cloudformation')
create_cft = cft.create_stack(StackName="Dinesh-1",TemplateURL=r'https://s3-us-west-2.amazonaws.com/bucket/dinesh.json')
print "Create Stack o/p - ",create_cft
#Just adding sleep so that stack creation come to the status of CREATE_COMPLETE
#More logic can be added to check the status of stack creation programmatically.
time.sleep(120)
des_stack = cft.describe_stacks(StackName="Dinesh-1")
print "Describe Stack o/p - ",des_stack
Output is
Create Stack o/p - {u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-1/a92318a0-54a7-11e6-b050-50d0184f2', 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': 'a91c023-54a7-11e6-ba43-67cc9d6ed45b', 'HTTPHeaders': {'x-amzn-requestid': 'a91c023-54a7-11e6-ba43-67cc9d6ed45b', 'date': 'Thu, 28 Jul 2016 09:42:55 GMT', 'content-length': '378', 'content-type': 'text/xml'}}}
Describe Stack o/p - {u'Stacks': [{u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-1/a92318a0-54a7-11e6-b050-50a0184f2', u'Description': 'Dinesh template', u'Tags': [], u'Outputs': [{u'Description': 'Private IP', u'OutputKey': 'PrivateIP', u'OutputValue': '172.3.28.221'}, {u'Description': 'Public IP', u'OutputKey': 'PublicIP', u'OutputValue': '52.5.203.173'}], u'CreationTime': datetime.datetime(2016, 7, 28, 9, 42, 55, 624000, tzinfo=tzutc()), u'StackName': 'Dinesh-1', u'NotificationARNs': [], u'StackStatus': 'CREATE_COMPLETE', u'DisableRollback': False}], 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': 'f19dc8ce-54a7-11e6-83e8-01451bce0ba', 'HTTPHeaders': {'x-amzn-requestid': 'f19dc8ce-54a7-11e6-83e8-01451b7ce0a', 'date': 'Thu, 28 Jul 2016 09:44:57 GMT', 'content-length': '1158', 'content-type': 'text/xml'}}}
In describe_stack output, you will get Public IP and Private IP of created EC2 instance.
Method 2
Getting EC2 instance info when there is no "Output" section present in cloud formation template
import boto3
import time
cft = boto3.client('cloudformation')
create_cft = cft.create_stack(StackName="Dinesh-2",TemplateURL=r'https://s3-us-west-2.amazonaws.com/dsp-bucket/dinesh.json')
print "Create Stack o/p - ",create_cft
#Just adding sleep so that stack creation come to the status of CREATE_COMPLETE
#More logic can be added to check the status of stack creation programmatically.
time.sleep(120)
list_stack_resp = cft.list_stack_resources(StackName="Dinesh-2")
print list_stack_resp
Output is
Create Stack o/p - {u'StackId': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-2/7238154a8-11e6-9694-50a686be73f2', 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '7234f160-54a8-11e6-bda6-ef311cece04b', 'HTTPHeaders': {'x-amzn-requestid': '7234f160-54a8-11e6-bda6-ef311cece04b', 'date': 'Thu, 28 Jul 2016 09:48:32 GMT', 'content-length': '378', 'content-type': 'text/xml'}}}
{'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': 'baabaa79-54a8-11e6-90e7-9ba061bfa4c', 'HTTPHeaders': {'x-amzn-requestid': 'baabaa79-54a8-11e6-90e7-9bad061bf4c', 'date': 'Thu, 28 Jul 2016 09:50:33 GMT', 'content-length': '687', 'content-type': 'text/xml'}}, u'StackResourceSummaries': [{u'ResourceType': 'AWS::EC2::Instance', u'PhysicalResourceId': 'i-059f15aa', u'LastUpdatedTimestamp': datetime.datetime(2016, 7, 28, 9, 49, 23, 481000, tzinfo=tzutc()), u'ResourceStatus': 'CREATE_COMPLETE', u'LogicalResourceId': 'MyEC2Instance'}]}
From the output of list_stack_resource, get the 'PhysicalResourceId' which is 'i-059f15aa' in this case.
Then get the output of describe_instance of ec2 to get full info of EC2 instance created above.
import boto3
ec2 = boto3.client('ec2')
ec2_resp = ec2.describe_instances(InstanceIds=['i-059f15aa'])
print ec2_resp
Output is
{u'Reservations': [{u'OwnerId': '089691119308', u'ReservationId': 'r-7245ddb6', u'Groups': [], u'Instances': [{u'Monitoring': {u'State': 'disabled'}, u'PublicDnsName': 'ec2-52-42-17-44.us-west-2.compute.amazonaws.com', u'State': {u'Code': 16, u'Name': 'running'}, u'EbsOptimized': False, u'LaunchTime': datetime.datetime(2016, 7, 28, 9, 48, 37, tzinfo=tzutc()), u'PublicIpAddress': '52.42.10.44', u'PrivateIpAddress': '172.3.29.25', u'ProductCodes': [], u'VpcId': 'vpc-c60a2aa3', u'StateTransitionReason': '', u'InstanceId': 'i-059f15aa', u'ImageId': 'ami-775e4f16', u'PrivateDnsName': 'ip-172-31-29-25.us-west-2.compute.internal', u'KeyName': 'dsp', u'SecurityGroups': [{u'GroupName': 'default', u'GroupId': 'sg-53fdaa37'}], u'ClientToken': 'Dines-MyEC2-DJ1D05Q7A088', u'SubnetId': 'subnet-8d0136e8', u'InstanceType': 't2.micro', u'NetworkInterfaces': [{u'Status': 'in-use', u'MacAddress': '02:9f:ab:4a:3c:0b', u'SourceDestCheck': True, u'VpcId': 'vpc-c60a2aa3', u'Description': '', u'Association': {u'PublicIp': '52.42.170.44', u'PublicDnsName': 'ec2-52-42-170-44.us-west-2.compute.amazonaws.com', u'IpOwnerId': 'amazon'}, u'NetworkInterfaceId': 'eni-d5272ca8', u'PrivateIpAddresses': [{u'PrivateDnsName': 'ip-172-31-29-25.us-west-2.compute.internal', u'Association': {u'PublicIp': '52.42.170.44', u'PublicDnsName': 'ec2-52-42-170-44.us-west-2.compute.amazonaws.com', u'IpOwnerId': 'amazon'}, u'Primary': True, u'PrivateIpAddress': '172.31.29.25'}], u'PrivateDnsName': 'ip-172-31-29-25.us-west-2.compute.internal', u'Attachment': {u'Status': 'attached', u'DeviceIndex': 0, u'DeleteOnTermination': True, u'AttachmentId': 'eni-attach-f33c375f', u'AttachTime': datetime.datetime(2016, 7, 28, 9, 48, 37, tzinfo=tzutc())}, u'Groups': [{u'GroupName': 'default', u'GroupId': 'sg-53fdaa37'}], u'SubnetId': 'subnet-8d0136e8', u'OwnerId': '089691119308', u'PrivateIpAddress': '172.31.29.25'}], u'SourceDestCheck': True, u'Placement': {u'Tenancy': 'default', u'GroupName': '', u'AvailabilityZone': 'us-west-2b'}, u'Hypervisor': 'xen', u'BlockDeviceMappings': [{u'DeviceName': '/dev/sda1', u'Ebs': {u'Status': 'attached', u'DeleteOnTermination': True, u'VolumeId': 'vol-21bddea8', u'AttachTime': datetime.datetime(2016, 7, 28, 9, 48, 37, tzinfo=tzutc())}}], u'Architecture': 'x86_64', u'RootDeviceType': 'ebs', u'RootDeviceName': '/dev/sda1', u'VirtualizationType': 'hvm', u'Tags': [{u'Value': 'arn:aws:cloudformation:us-west-2:089691119308:stack/Dinesh-2/723ba810-54a8-11e6-9694-50a686be73f2', u'Key': 'aws:cloudformation:stack-id'}, {u'Value': 'MyEC2Instance', u'Key': 'aws:cloudformation:logical-id'}, {u'Value': 'Dinesh-2', u'Key': 'aws:cloudformation:stack-name'}], u'AmiLaunchIndex': 0}]}], 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '8adf8956-0d5a-4d1f-a821-67fec4b5bbf9', 'HTTPHeaders': {'transfer-encoding': 'chunked', 'vary': 'Accept-Encoding', 'server': 'AmazonEC2', 'content-type': 'text/xml;charset=UTF-8', 'date': 'Thu, 28 Jul 2016 09:55:45 GMT'}}}