autotagging aws instance with boto3 - amazon-web-services

i am trying to retrieve the iam user email and account id who started an instance and use it to tag the instance, the account id works well but the user email return an error
the lamnbda function was trigger by a cloudwatch event rule that returns the instance id to the lambda function when the instance state change to running
import boto3
def lambda_handler(event, context):
print(event)
# Get the EC2 instance ID from the event data
instance_id = event['detail']['instance-id']
# Get the account ID
sts_client = boto3.client('sts')
identity = sts_client.get_caller_identity()
account_id = identity['Account']
# Tag the EC2 instance with the email and account ID
ec2 = boto3.client('ec2')
# Describe the instance to get the IAM role ARN
response = ec2.describe_instances(InstanceIds=[instance_id])
iam_role_arn = response['Reservations'][0]['Instances'][0]['IamInstanceProfile']['Arn']
# Get the IAM client
iam = boto3.client('iam')
# Get the role name from the IAM role ARN
role_name = iam_role_arn.split('/')[1]
# Get the role details
role_details = iam.get_role(RoleName=role_name)
# Get the policy ARN from the role details
policy_arn = role_details['Role']['AssumeRolePolicyDocument']['Statement'][0]['Principal']['AWS'][0]
# Get the policy details
policy_details = iam.get_policy(PolicyArn=policy_arn)
# Get the user ARN from the policy details
user_arn = policy_details['Policy']['UserName']
# Get the user details
user_response = iam.get_user(UserName=user_arn)
# Get the user email from the user details
user_email = user_response['User']['UserName']
ec2.create_tags(
Resources=[instance_id],
Tags=[
{
'Key': 'Email',
'Value': email
},
{
'Key': 'AccountID',
'Value': user_email
}
]
)

You should already have the access key for the user that launched the instance so you can reverse lookup the associated IAM user using GetAccessKeyLastUsed, available in boto3 as get_access_key_last_used and retrieve the IAM user's name (which is not strictly an email address).

Related

Lambda function for creating an EC2 instance

I ran my code to create an EC2 instance but I keep getting this error.
"errorMessage": "'message'",
"errorType": "KeyError",
The full code
import boto3
import os
AMI = os.environ['AMI']
INSTANCE_TYPE = os.environ['INSTANCE_TYPE']
KEY_NAME = os.environ['KEY_NAME']
SUBNET_ID = os.environ['SUBNET_ID']
REGION = os.environ['AWS_REGION']
ec2 = boto3.client('ec2', region_name=REGION)
def lambda_handler(event, context):
message = event['message']
instance = ec2.run_instances(
ImageId=AMI,
InstanceType=INSTANCE_TYPE,
KeyName=KEY_NAME,
SubnetId=SUBNET_ID,
MaxCount=1,
MinCount=1,
InstanceInitiatedShutdownBehavior='terminate',
UserData=init_script
)
instance_id = instance['Instances'][0]['InstanceId']
print instance_id
return instance_id
What could be triggering this key error?
As an environmental variable, am I supposed to use the full key name including its file type.
Ex: "key.pem" instead of "key"
The error indicates that you have an error on this line:
message = event['message']
Most likely the lambda event does not have the 'message' key you are expecting. You should print out the event to CloudWatch Logs and take a look.
I ran the rest of your code and it created an EC2 instance successfully.

How to get the list of Nitro system based EC2 instance type by CLI?

I know this page lists up the instance types which based on Nitro system but I would like to know the list in a dynamic way with CLI. (for example, using aws ec2 describe-instances). Is it possible to get Nitro based instance type other than parsing the static page? If so, could you tell me the how?
You'd have to write a bit of additional code to get that information. aws ec2 describe-instances will give you InstanceType property. You should use a programming language to parse the JSON, extract InstanceType and then call describe-instances like so: https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instance-types.html?highlight=nitro
From the JSON you get back, extract hypervisor. That'll give you Nitro if the instance is Nitro.
Here's a Python code that might work. I have not tested it fully but you can tweak this to get the results you want.
"""List all EC2 instances"""
import boto3
def ec2_connection():
"""Connect to AWS using API"""
region = 'us-east-2'
aws_key = 'xxx'
aws_secret = 'xxx'
session = boto3.Session(
aws_access_key_id = aws_key,
aws_secret_access_key = aws_secret
)
ec2 = session.client('ec2', region_name = region)
return ec2
def get_reservations(ec2):
"""Get a list of instances as a dictionary"""
response = ec2.describe_instances()
return response['Reservations']
def process_instances(reservations, ec2):
"""Print a colorful list of IPs and instances"""
if len(reservations) == 0:
print('No instance found. Quitting')
return
for reservation in reservations:
for instance in reservation['Instances']:
# get friendly name of the server
# only try this for mysql1.local server
friendly_name = get_friendly_name(instance)
if friendly_name.lower() != 'mysql1.local':
continue
# get the hypervisor based on the instance type
instance_type = get_instance_info(instance['InstanceType'], ec2)
# print findings
print(f'{friendly_name} // {instance["InstanceType"]} is {instance_type}')
break
def get_instance_info(instance_type, ec2):
"""Get hypervisor from the instance type"""
response = ec2.describe_instance_types(
InstanceTypes=[instance_type]
)
return response['InstanceTypes'][0]['Hypervisor']
def get_friendly_name(instance):
"""Get friendly name of the instance"""
tags = instance['Tags']
for tag in tags:
if tag['Key'] == 'Name':
return tag['Value']
return 'Unknown'
def run():
"""Main method to call"""
ec2 = ec2_connection()
reservations = get_reservations(ec2)
process_instances(reservations, ec2)
if __name__ == '__main__':
run()
print('Done')
In the above answer , the statement "From the JSON you get back, extract hypervisor. That'll give you Nitro if the instance is Nitro " is not longer accurate.
As per the latest AWS documentation,
hypervisor - The hypervisor type of the instance (ovm | xen ). The value xen is used for both Xen and Nitro hypervisors.
Cleaned up, verified working code below:
# Get all instance types that run on Nitro hypervisor
import boto3
def get_nitro_instance_types():
"""Get all instance types that run on Nitro hypervisor"""
ec2 = boto3.client('ec2', region_name = 'us-east-1')
response = ec2.describe_instance_types(
Filters=[
{
'Name': 'hypervisor',
'Values': [
'nitro',
]
},
],
)
instance_types = []
for instance_type in response['InstanceTypes']:
instance_types.append(instance_type['InstanceType'])
return instance_types
get_nitro_instance_types()
Example output as of 12/06/2022 below:
['r5dn.8xlarge', 'x2iedn.xlarge', 'r6id.2xlarge', 'r6gd.medium',
'm5zn.2xlarge', 'r6idn.16xlarge', 'c6a.48xlarge', 'm5a.16xlarge',
'im4gn.2xlarge', 'c6gn.16xlarge', 'c6in.24xlarge', 'r5ad.24xlarge',
'r6i.xlarge', 'c6i.32xlarge', 'x2iedn.2xlarge', 'r6id.xlarge',
'i3en.24xlarge', 'i3en.12xlarge', 'm5d.8xlarge', 'c6i.8xlarge',
'r6g.large', 'm6gd.4xlarge', 'r6a.2xlarge', 'x2iezn.4xlarge',
'c6i.large', 'r6in.24xlarge', 'm6gd.xlarge', 'm5dn.2xlarge',
'd3en.2xlarge', 'c6id.8xlarge', 'm6a.large', 'is4gen.xlarge',
'r6g.8xlarge', 'm6idn.large', 'm6a.2xlarge', 'c6i.4xlarge',
'i4i.16xlarge', 'm5zn.6xlarge', 'm5.8xlarge', 'm6id.xlarge',
'm5n.16xlarge', 'c6g.16xlarge', 'r5n.12xlarge', 't4g.nano',
'm5ad.12xlarge', 'r6in.12xlarge', 'm6idn.12xlarge', 'g5.2xlarge',
'trn1.32xlarge', 'x2gd.8xlarge', 'is4gen.4xlarge', 'r6gd.xlarge',
'r5a.xlarge', 'r5a.2xlarge', 'c5ad.24xlarge', 'r6a.xlarge',
'r6g.medium', 'm6id.12xlarge', 'r6idn.2xlarge', 'c5n.2xlarge',
'g5.4xlarge', 'm5d.xlarge', 'i3en.3xlarge', 'r5.24xlarge',
'r6gd.2xlarge', 'c5d.large', 'm6gd.12xlarge', 'm6id.2xlarge',
'm6i.large', 'z1d.2xlarge', 'm5a.4xlarge', 'm5a.2xlarge',
'c6in.xlarge', 'r6id.16xlarge', 'c7g.8xlarge', 'm5dn.12xlarge',
'm6gd.medium', 'im4gn.8xlarge', 'm5dn.large', 'c5ad.4xlarge',
'r6g.16xlarge', 'c6a.24xlarge', 'c6a.16xlarge']
"""List all EC2 instances"""
import boto3
def ec2_connection():
"""Connect to AWS using API"""
region = 'us-east-2'
aws_key = 'xxx'
aws_secret = 'xxx'
session = boto3.Session(
aws_access_key_id = aws_key,
aws_secret_access_key = aws_secret
)
ec2 = session.client('ec2', region_name = region)
return ec2
def get_reservations(ec2):
"""Get a list of instances as a dictionary"""
response = ec2.describe_instances()
return response['Reservations']
def process_instances(reservations, ec2):
"""Print a colorful list of IPs and instances"""
if len(reservations) == 0:
print('No instance found. Quitting')
return
for reservation in reservations:
for instance in reservation['Instances']:
# get friendly name of the server
# only try this for mysql1.local server
friendly_name = get_friendly_name(instance)
if friendly_name.lower() != 'mysql1.local':
continue
# get the hypervisor based on the instance type
instance_type = get_instance_info(instance['InstanceType'], ec2)
# print findings
print(f'{friendly_name} // {instance["InstanceType"]} is {instance_type}')
break
def get_instance_info(instance_type, ec2):
"""Get hypervisor from the instance type"""
response = ec2.describe_instance_types(
InstanceTypes=[instance_type]
)
return response['InstanceTypes'][0]['Hypervisor']
def get_friendly_name(instance):
"""Get friendly name of the instance"""
tags = instance['Tags']
for tag in tags:
if tag['Key'] == 'Name':
return tag['Value']
return 'Unknown'
def run():
"""Main method to call"""
ec2 = ec2_connection()
reservations = get_reservations(ec2)
process_instances(reservations, ec2)
if name == 'main':
run()
print('Done')

Programmatically Convert all AWS inline policies to Managed Policies of current IAM Roles

Currently I have several hundred AWS IAM Roles with inline policies.
I would like to somehow convert these inline policies to managed policies.
While AWS Documentation has a way to do this via the Console, this will be very time consuming.
Does anyone know of a way, or have a script to do this via BOTO or AWS CLI...or direct me to some method that I can do this programmatically?
Thanks in advance
boto3 code will be like this.
In this code, inline policies that are embedded in the specified IAM user will be copied to customer managed policies.
Note delete part is commented out.
import json
import boto3
user_name = 'xxxxxxx'
client = boto3.client("iam")
response = client.list_user_policies(UserName=user_name)
for policy_name in response["PolicyNames"]:
response = client.get_user_policy(UserName=user_name, PolicyName=policy_name)
policy_document = json.dumps(response["PolicyDocument"])
response = client.create_policy(
PolicyName=policy_name, PolicyDocument=policy_document_json
)
# response = client.delete_user_policy(
# UserName=user_name,
# PolicyName=policy_name
# )
Updated:
For IAM roles, changing User to Role, user to role (case sensitive) above code works.
Besides, if you execute for multiple roles, use list_roles to get role_name.
response=client.list_roles()
for i in response['Roles']:
role_name = i['RoleName']
# print(role_name)
with #shimo snippet, the following works with added error handling and attaching the newly created managed policy to the IAM role:
import json
import boto3
from botocore.exceptions import ClientError
role_name = 'xxxxxxxx'
account_id = '123456789'
client = boto3.client("iam")
resource = boto3.resource('iam')
response = client.list_role_policies(RoleName=role_name)
for policy_name in response["PolicyNames"]:
response = client.get_role_policy(RoleName=role_name, PolicyName=policy_name)
policy_document = json.dumps(response["PolicyDocument"])
print(policy_document)
try:
response = client.create_policy(
PolicyName=policy_name, PolicyDocument=policy_document
)
print(policy_name + 'Policy Created')
except ClientError as error:
if error.response['Error']['Code'] == 'EntityAlreadyExists':
print(policy_name + ' policy already exists')
else:
print("Unexpected error: %s" % error)
policy_arn = f'arn:aws:iam::{account_id}:policy/{policy_name}'
role = resource.Role(role_name)
role.attach_policy(PolicyArn=policy_arn)
response = client.delete_role_policy(
RoleName=role_name,
PolicyName=policy_name
)

How to retrieve mfa_serial from config file in boto3?

In my ~/.aws/config file I have:
[dev]
region = us-east-1
output = json
mfa_serial = arn:aws:iam::1111:mfa/user
How do I retrieve that mfa_serial from the config in boto3 so I don't have to specify the arn in the py script?
sts = session.client('sts')
mfa_code = input("Enter the MFA code: ")
mfa_session = sts.get_session_token(
DurationSeconds=3600,
SerialNumber=mfa_serial,
TokenCode=mfa_code
)
You can get the mfa_serial defined in the current config file using botocore.session.get_scoped_config
from botocore.session import Session
Session().get_scoped_config().get('mfa_serial')
If you're okay getting username input you can get the MFA arn programmatically using list_mfa_devices.
This snippet requests the User's IAM name and then retrieves the MFA information.
import boto3
iam = session.client('iam')
user = input("Username: ")
response = iam.list_mfa_devices(UserName=user)['MFADevices']
mfa = next(iter(response))['SerialNumber']
print(mfa)
The output will be this format:
arn:aws:iam::123456789123:mfa/amandahugginkiss
References
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.list_mfa_devices

Boto3: Get EC2 images owned by me

I would like to get all ami images owned by me. I tried something like below:
ec2 = boto3.resource('ec2')
owner_id = 'owner_id'
filters = [{'Name': 'owner-id', 'Values': [owner_id]}]
images = ec2.images.filter(Filters=filters).all()
But I need to put owner_id explicid in the code. Is it any solution to do that automatically from aws credentials?
You should be able to use self for the owner. This is what I use.
boto3conn = boto3.resource("ec2", region_name="us-west-2")
images = boto3conn.images.filter(Owners=['self'])
This will help, It will show you ALL AMI thats owned by your aws account
import boto3
client = boto3.client('ec2', region_name='us-east-1')
response = client.describe_images(Owners=['self'])
for ami in response['Images']:
print (ami['ImageId'])
You can use the STS API to get this information.
I found this post talking about it: getting the current user account-id in boto3
So the code you need is:
ec2 = boto3.resource('ec2', region_name='us-east-1')
owner_id = boto3.client('sts').get_caller_identity().get('Account')
filters = [{'Name': 'owner-id', 'Values': [owner_id]}]
images = ec2.images.filter(Filters=filters).all()
Make sure to change the region name to the correct one, or leave it out if you've set it elsewhere in your environment.