I am trying to write a query to return results of any referenced security group not owned by the current account.
This means I am trying to show security groups that are being used as part of a peering connection from another VPC.
There are a couple of restrictions.
Show the entire security group details (security group id, description)
Only show security groups where IpPermissions.UserIdGroupPairs has a Value and where that value is not equal to the owner of the security group
I am trying to write this using a single AWS CLI cmd vs a bash script or python script.
Any thoughts?
Heres what I have so far.
aws ec2 describe-security-groups --query "SecurityGroups[?IpPermissions.UserIdGroupPairs[*].UserId != '`aws sts get-caller-identity --query 'Account' --output text`']"
Following is Python 3.8 based AWS Lambda, but you can change a bit to use as python script file to execute on any supported host machine.
import boto3
import ast
config_service = boto3.client('config')
# Refactor to extract out duplicate code as a seperate def
def lambda_handler(event, context):
results = get_resource_details()
for resource in results:
if "configuration" in resource:
config=ast.literal_eval(resource)["configuration"]
if "ipPermissionsEgress" in config:
ipPermissionsEgress=config["ipPermissionsEgress"]
for data in ipPermissionsEgress:
for userIdGroupPair in data["userIdGroupPairs"]:
if userIdGroupPair["userId"] != "123456789111":
print(userIdGroupPair["groupId"])
elif "ipPermissions" in config:
ipPermissions=config["ipPermissions"]
for data in ipPermissions:
for userIdGroupPair in data["userIdGroupPairs"]:
if userIdGroupPair["userId"] != "123456789111":
print(userIdGroupPair["groupId"])
def get_resource_details():
query = "SELECT configuration.ipPermissions.userIdGroupPairs.groupId,configuration.ipPermissionsEgress.userIdGroupPairs.groupId,configuration.ipPermissionsEgress.userIdGroupPairs.userId,configuration.ipPermissions.userIdGroupPairs.userId WHERE resourceType = 'AWS::EC2::SecurityGroup' AND configuration <> 'null'"
results = config_service.select_resource_config(
Expression=query,
Limit=100
) # you might need to refacor to add support huge list of records using NextToken
return results["Results"]
I want to create a script, and perhaps run it in a cron job every 24 hours, which will list all access keys older than 60 days.
I also want to shove the keys older than 60 days into an array so I can iterate over it and perform other options.
I'm looking at Managing access keys for IAM users - AWS Identity and Access Management and it has a aws iam get-access-key-last-used command but that's not what I want. But it's the closet thing I can find.
What I want to get the key where current date - creation date > 60 days.
I'm imagining my script would look something like this:
# some of this is pseudocode just to
# communicate what I'm envisioning.
# I don't actually know what to put
# here yet; need assistance.
myCommand = "aws cli get key where age > 60"
staleKeys=( $( $myCommand) )
for key in "${staleKeys[#]}"
do
# log "${key}"
# run another aws cli command with ${key} as a value
done
Is this possible from the AWS CLI?
I recommend Getting credential reports for your AWS account - AWS Identity and Access Management. This is an automated process that can generate a CSV file listing lots of information about credentials, including:
The date and time when the user's access key was created or last changed
The date and time when the user's access key was most recently used to sign an AWS API request
The report can be obtained by calling generate-credential-report, waiting a bit, then calling get-credential-report. The response needs to be base64 decoded. The result looks like this:
user,arn,user_creation_time,password_enabled,password_last_used,password_last_changed,password_next_rotation,mfa_active,access_key_1_active,access_key_1_last_rotated,access_key_1_last_used_date,access_key_1_last_used_region,access_key_1_last_used_service,access_key_2_active,access_key_2_last_rotated,access_key_2_last_used_date,access_key_2_last_used_region,access_key_2_last_used_service,cert_1_active,cert_1_last_rotated,cert_2_active,cert_2_last_rotated
user1,arn:aws:iam::111111111111:user/user1,2019-04-08T05:57:22+00:00,true,2020-05-20T10:55:03+00:00,2019-04-18T00:43:43+00:00,N/A,false,true,2019-04-08T05:57:24+00:00,2019-12-05T21:23:00+00:00,us-west-2,iot,true,2019-11-18T09:38:54+00:00,N/A,N/A,N/A,false,N/A,false,N/A
If you decide to generate the information yourself, please note that list_access_keys() only returns information about a single user. Therefore, you would need to iterate through all users, and call list_access_keys() for each user to obtain the CreationDate of the keys.
For an example of usage, see: How to scan your AWS account for old access keys using python - DEV Community
I use the following Python boto3 script, not AWS CLI.
Hope this help those who wanna use boto3:
import boto3
from datetime import datetime, timezone
def utc_to_local(utc_dt):
return utc_dt.replace(tzinfo=timezone.utc).astimezone(tz=None)
def diff_dates(date1, date2):
return abs(date2 - date1).days
resource = boto3.resource('iam')
client = boto3.client("iam")
KEY = 'LastUsedDate'
for user in resource.users.all():
Metadata = client.list_access_keys(UserName=user.user_name)
if Metadata['AccessKeyMetadata']:
for key in user.access_keys.all():
AccessId = key.access_key_id
Status = key.status
CreatedDate = key.create_date
numOfDays = diff_dates(utc_to_local(datetime.utcnow()), utc_to_local(CreatedDate))
LastUsed = client.get_access_key_last_used(AccessKeyId=AccessId)
if (Status == "Active"):
if KEY in LastUsed['AccessKeyLastUsed']:
print("User:", user.user_name, "Key:", AccessId, "Last Used:", LastUsed['AccessKeyLastUsed'][KEY], "Age of Key:", numOfDays, "Days")
else:
print("User:", user.user_name , "Key:", AccessId, "Key is Active but NEVER USED")
else:
print("User:", user.user_name , "Key:", AccessId, "Keys is InActive")
else:
print("User:", user.user_name , "No KEYS for this USER")
My organization is planning for a maintenance window for the next 5 hours. During that time, I do not want Cloud Watch to trigger alarms and send notifications.
Earlier, when I had to disable 4 alarms, I have written the following code in AWS Lambda. This worked fine.
import boto3
import collections
client = boto3.client('cloudwatch')
def lambda_handler(event, context):
response = client.disable_alarm_actions(
AlarmNames=[
'CRITICAL - StatusCheckFailed for Instance 456',
'CRITICAL - StatusCheckFailed for Instance 345',
'CRITICAL - StatusCheckFailed for Instance 234',
'CRITICAL - StatusCheckFailed for Instance 123'
]
)
But now, I was asked to disable all the alarms which are 361 in number. So, including all those names would take a lot of time.
Please let me know what I should do now?
Use describe_alarms() to obtain a list of them, then iterate through and disable them:
import boto3
client = boto3.client('cloudwatch')
response = client.describe_alarms()
names = [[alarm['AlarmName'] for alarm in response['MetricAlarms']]]
disable_response = client.disable_alarm_actions(AlarmNames=names)
You might want some logic around the Alarm Name to only disable particular alarms.
If you do not have the specific alarm arns, then you can use the logic in the previous answer. If you have a specific list of arns that you want to disable, you can fetch names using this:
def get_alarm_names(alarm_arns):
names = []
response = client.describe_alarms()
for i in response['MetricAlarms']:
if i['AlarmArn'] in alarm_arns:
names.append(i['AlarmName'])
return names
Here's a full tutorial: https://medium.com/geekculture/terraform-structure-for-enabling-disabling-alarms-in-batches-5c4f165a8db7
I am trying to use boto v2.32.0 to list the tags on a particular ASG
something simple like this is obviously not working (especially with the lack of a filter system):
import boto.ec2.autoscale
asg = boto.ec2.autoscale.connect_to_region('ap-southeast-2')
tags = asg.get_all_tags('asgname')
print tags
or:
asg = boto.ec2.autoscale.connect_to_region('ap-southeast-2')
group = asg.get_all_groups(names='asgname')
tags = asg.get_all_tags(group)
print tags
or:
asg = boto.ec2.autoscale.connect_to_region('ap-southeast-2')
group = asg.get_all_groups(names='asgname')
tags = group.get_all_tags()
print tags
Without specifying an 'asgname', it's not returning every ASG. Despite what the documentation says about returning a token to see the next page, it doesn't seem to be implemented correctly - especially when you have a large number of ASG's and tags per ASG.
Trying something like this has basically shown me that the token system appears to be broken. it is not "looping" through all ASG's and tags before it returns "None":
asg = boto.ec2.autoscale.connect_to_region('ap-southeast-2')
nt = None
while ( True ):
tags = asg.get_all_tags(next_token=nt)
for t in tags:
if ( t.key == "MyTag" ):
print t.resource_id
print t.value
if ( tags.next_token == None ):
break
else:
nt = str(tags.next_token)
Has anyone managed to achieve this?
Thanks
This functionality is available in AWS using the AutoScaling DescribeTags API call, but unfortunately boto does not completely implement this call.
You should be able to pass a Filter with that API call to only get the tags for a specific ASG, but if you have a look at the boto source code for get_all_tags() (v2.32.1), the filter is not implemented:
:type filters: dict
:param filters: The value of the filter type used
to identify the tags to be returned. NOT IMPLEMENTED YET.
(quote from the source code mentioned above).
I eventually answered my own question by creating a work around using the amazon cli. Since there has been no activity on this question since the day I asked it I am posting this workaround as a solution.
import os
import json
## bash command
awscli = "/usr/local/bin/aws autoscaling describe-tags --filters Name=auto-scaling-group,Values=" + str(asgname)
output = str()
# run it
cmd = os.popen(awscli,"r")
while 1:
# get tag lines
lines = cmd.readline()
if not lines: break
output += lines
# json.load to manipulate
tags = json.loads(output.replace('\n',''))
I am new to boto and trying to get all the metrics that have alarms. Can some one please guide me how to do that? Here is what I am trying to do. I can get all the metrics in the following way.
import boto.ec2.cloudwatch
conn = boto.ec2.cloudwatch.connect_to_region('ap-southeast-1')
metrics = conn.list_metrics()
for metric in metrics:
print metric.name, metric.namespace
I know that there is a function "describe_alarms_for_metric" that returns the alarms for a metric. However it is not working for me and gives me an empty list. Here is what I am trying.
for metric in metrics:
print conn.describe_alarms_for_metric(metric.name, metric.namespace)
I can also see the list of all alarms using "describe_alarms" but I dont know which alarm is for what metric.
alarms = conn.describe_alarms()
for alarm in alarms:
print alarm
describe_alarms() returns a list of boto.ec2.cloudwatch.alarm objects, which can be inspected to find out the metric and other details about the alarm.
alarms = conn.describe_alarms()
for alarm in alarms:
print alarm.name
print alarm.metric
print alarm.namespace
For Boto3 apparently describe_alarms_for_metric() doesn't work unless you also supply a dimension - see the documentation:
Dimensions (list) -- The dimensions associated with the metric. If the
metric has any associated dimensions, you must specify them in order
for the call to succeed.
(dict) -- Expands the identity of a metric.
Name (string) -- [REQUIRED] The name of the dimension.
Value (string) -- [REQUIRED] The value representing the dimension
measurement.
With that requirement I'm not sure what the point of this API is. An alternative is to use describe_alarms() through the paginator then specify a filter.
You can use the example here as a base:
import boto3
# Create CloudWatch client
cloudwatch = boto3.client('cloudwatch')
# List alarms of insufficient data through the pagination interface
paginator = cloudwatch.get_paginator('describe_alarms')
for response in paginator.paginate(StateValue='INSUFFICIENT_DATA'):
print(response['MetricAlarms'])
Then modify it to add a filter:
paginator = cloudwatch.get_paginator('describe_alarms')
page_iterator = paginator.paginate()
filtered_iterator = page_iterator.search("MetricAlarms[?MetricName==`CPUUtilization` && Namespace==`AWS/EC2`]")
for alarm in filtered_iterator:
print(alarm)
More information in the API docs here and here.