I have made script who clean ami id based on from not running instances.
but i want also delete feature this script to clean launch config form AMI ID(actually who are not exist).
good_images = set([instance.image_id for instance in ec2.instances.all()])
#LaunchConfig in use AMI
client = boto3.client('autoscaling', region_name=region)
response = client.describe_launch_configurations()
ls_list=[]
for LC in response['LaunchConfigurations']:
(LC['ImageId'])
print ls_list
but its not working.
Your code:
for LC in response['LaunchConfigurations']:
(LC['ImageId'])
should be:
for LC in response['LaunchConfigurations']:
(ls_list.append(LC['ImageId']))
used_lc = []
all_lc = []
def used_launch_config():
for asg in client.describe_auto_scaling_groups()['AutoScalingGroups']:
launch_config_attached_with_asg = asg['LaunchConfigurationName']
used_lc.append(launch_config_attached_with_asg)
used_launch_config()
print used_lc
def all_spot_lc():
for launch_config in client.describe_launch_configurations(MaxRecords=100,)['LaunchConfigurations']:
lc = launch_config['LaunchConfigurationName']
if str(lc).startswith("string"):
all_lc.append(lc)
all_spot_lc()
print all_lc
I just avoid to delete launch config from AMI.
Now i compared from used or unused it solved the problem.
I was doing wrong in Previous code.
is there way to increase max records
Related
Given the Terraform documentation on AWS Lightsail, I can construct a brand new Lightsail instance as follows.
resource "aws_lightsail_instance" "my_ls_instance" {
name = "my_ls"
availability_zone = "us-east-1b"
blueprint_id = "ubuntu_18_04"
bundle_id = "2xlarge_2_0"
key_pair_name = "MyKeyName"
}
It is possible to create a Lightsail instance from a Lightsail snapshot using Terraform?
No, it's not. Right now Terraform can only create instances based on Lightsail blueprints.
You can, however create an instance from snapshot in python3 /w boto3. Let me include my code:
#######
import boto3
client = boto3.client('lightsail')
response = client.create_instances_from_snapshot(
instanceNames=[
'myitblog',
],
availabilityZone='us-east-1a',
instanceSnapshotName='MYITBLOG_https',
bundleId='nano_2_0',
)
response = client.attach_static_ip(
staticIpName='StaticIp-1',
instanceName='myitblog'
)
I'm updating python function for getting a list of rds instances with 'Backup' tag set to "true" but for some reason the for loop stops at the first instance.
We have 10 rds instances.
Could someone please tell me what I'm doing wrong here?
Here's the code:
def get_instances(region)
rds = boto3.client('rds', region)
instances_to_snapshot = []
response = rds.describe_db_instances()
aurora = "aurora"
oracle = "oracle-se1"
aurora_list = []
instances = response['DBInstances']
for instance in instances:
engine = instance['Engine']
if get_rds_instance_tag(region, instance['DBInstanceIdentifier'], 'Backup'):
print "This instance - %s has engine - %s " % (instance['DBInstanceIdentifier'], engine)
instances_to_snapshot.append(instance['DBInstanceIdentifier'])
return instances_to_snapshot
Thank you
Found it!! The issue was 'Time out' setting in lambda.. haha It was set to "3 seconds" No wonder function didn't work correctly.
I am trying to use boto v2.32.0 to list the tags on a particular ASG
something simple like this is obviously not working (especially with the lack of a filter system):
import boto.ec2.autoscale
asg = boto.ec2.autoscale.connect_to_region('ap-southeast-2')
tags = asg.get_all_tags('asgname')
print tags
or:
asg = boto.ec2.autoscale.connect_to_region('ap-southeast-2')
group = asg.get_all_groups(names='asgname')
tags = asg.get_all_tags(group)
print tags
or:
asg = boto.ec2.autoscale.connect_to_region('ap-southeast-2')
group = asg.get_all_groups(names='asgname')
tags = group.get_all_tags()
print tags
Without specifying an 'asgname', it's not returning every ASG. Despite what the documentation says about returning a token to see the next page, it doesn't seem to be implemented correctly - especially when you have a large number of ASG's and tags per ASG.
Trying something like this has basically shown me that the token system appears to be broken. it is not "looping" through all ASG's and tags before it returns "None":
asg = boto.ec2.autoscale.connect_to_region('ap-southeast-2')
nt = None
while ( True ):
tags = asg.get_all_tags(next_token=nt)
for t in tags:
if ( t.key == "MyTag" ):
print t.resource_id
print t.value
if ( tags.next_token == None ):
break
else:
nt = str(tags.next_token)
Has anyone managed to achieve this?
Thanks
This functionality is available in AWS using the AutoScaling DescribeTags API call, but unfortunately boto does not completely implement this call.
You should be able to pass a Filter with that API call to only get the tags for a specific ASG, but if you have a look at the boto source code for get_all_tags() (v2.32.1), the filter is not implemented:
:type filters: dict
:param filters: The value of the filter type used
to identify the tags to be returned. NOT IMPLEMENTED YET.
(quote from the source code mentioned above).
I eventually answered my own question by creating a work around using the amazon cli. Since there has been no activity on this question since the day I asked it I am posting this workaround as a solution.
import os
import json
## bash command
awscli = "/usr/local/bin/aws autoscaling describe-tags --filters Name=auto-scaling-group,Values=" + str(asgname)
output = str()
# run it
cmd = os.popen(awscli,"r")
while 1:
# get tag lines
lines = cmd.readline()
if not lines: break
output += lines
# json.load to manipulate
tags = json.loads(output.replace('\n',''))
I am new to AWS sdk java. I am trying to write a code through which I want to control the instance and to get all AWS EC2 information.
I am able to start an instance and also stop it. But as you all must be aware that it takes some time to start an instance, so I want to wait there (don't want to use Thread.sleep) till it's up or when I'm stopping an instance it should wait there till I proceed to the next step.
Here's the code:
AmazonEC2 ec2 = = new AmazonEC2Client(credentialsProvider);
DescribeInstancesResult describeInstancesRequest = ec2.describeInstances();
List<Reservation> reservations = describeInstancesRequest.getReservations();
Set<Instance> instances = new HashSet<Instance>();
for (Reservation reservation : reservations) {
instances.addAll(reservation.getInstances());
}
for (Instance instance : instances) {
if ((instance.getInstanceId().equals("myimage"))) {
List<String> instancesToStart = new ArrayList<String>();
instancesToStart.add(instance.getInstanceId());
StartInstancesRequest startr = new StartInstancesRequest();
startr.setInstanceIds(instancesToStart);
ec2.startInstances(startr);
Thread.currentThread().sleep(60*1000);
}
if ((instat.getName()).equals("running")) {
List<String> instancesToStop = new ArrayList<String>();
instancesToStop.add(instance.getInstanceId());
StopInstancesRequest stoptr = new StopInstancesRequest();
stoptr.setInstanceIds(instancesToStop);
ec2.stopInstances(stoptr);
}
Also, I'd like to say that whenever I try to get the list of images it hangs in the below code.
DescribeImagesResult describeImagesResult = ec2.describeImages();
You can get an instance of the class "Instance" every time you want to see the updated status with the same "instance Id".
Instance instance = new Instance(<your instance id that you got previously from describe instances>);
To get the updated status with something like this:
InstanceStatus instat = instance.getStatus();
I think the key here is saving the "instance id" of the instance that you care about.
boto in Python has an nice method instance.update() that can be called on an instance and you can see its status but I can't find it in Java.
Hope this helps.
I am planning to use AWS autoscaling groups for my webservers. As a monitoring solution I am using munin at the moment. In the configuration file on the munin master server, you have to give IP addresses or host names for every host you want to monitor.
Now with autoscaling the number of instances will change frequently, and writing static information in the munin config does not seem to fit well in this environment. I could probably query all server addresses I want to monitor and write the munin master configuration file then, but this seems not like a good approach to me.
What is the preferred way of using munin in such an environment? Does someone use munin with autoscaling?
In general I would like to keep using munin and not switch to another monitoring solution because I wrote quite a lot of specific plugins that I rely on. However if you have another monitoring solution that will probably let me keep my plugins I am also open for that.
One year ago we used munin as alternative monitoring system and I will tell you one: I don't like it at all.
We had some automation for auto scaling system in nagios too, but this is also ugly way to monitor large amount of AWS instances because nagios starts to lag/crash after some amount of monitoring instances.
If you have more that 150-200 instances to monitor I suggest you to use some commercial services like StackDriver or other alternatives.
I stumbled across this old topic because I was looking for a solution to the same problem. Finally I found a way that works for me which I would like to share with you. The tl;dr summary
use AWS Python API to get all instances in the same VPC the munin master is in
test if munin port 4949 is open on the instances found to detect munin nodes
create munin.conf from a munin.base.conf (without nodes) and append entries for all the nodes found
run the script on the munin master all 5 minutes via cron
Finally, here is my Python script which does all the magic:
#! /usr/bin/python
import boto3
import requests
import argparse
import shutil
import socket
socketTimeout = 2
ec2 = boto3.client('ec2')
def getVpcId():
response = requests.get('http://169.254.169.254/latest/meta-data/instance-id')
instance_id = response.text
response = ec2.describe_instances(
Filters=[
{
'Name' : 'instance-id',
'Values' : [ instance_id ]
}
]
)
return response['Reservations'][0]['Instances'][0]['VpcId']
def findNodes(tag):
result = []
vpcId = getVpcId()
response = ec2.describe_instances(
Filters=[
{
'Name' : 'tag-key',
'Values' : [ tag ]
},
{
'Name' : 'vpc-id',
'Values' : [ vpcId ]
}
]
)
for reservation in response['Reservations']:
for instance in reservation['Instances']:
result.append(instance)
return result
def getInstanceTag(instance, tagName):
for tag in instance['Tags']:
if tag['Key'] == tagName:
return tag['Value']
return None
def isMuninNode(host):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(socketTimeout)
try:
s.connect((host, 4949))
s.shutdown(socket.SHUT_RDWR)
return True
except Exception as e:
return False
finally:
s.close()
def appendNodesToConfig(nodes, target, tag):
with open(target, "a") as file:
for node in nodes:
hostname = getInstanceTag(node, tag)
if hostname.endswith('.'):
hostname = hostname[:-1]
if hostname <> None and isMuninNode(hostname):
file.write('[' + hostname + ']\n')
file.write('\taddress ' + hostname + '\n')
file.write('\tuse_node_name yes\n\n')
parser = argparse.ArgumentParser("muninconf.py")
parser.add_argument("baseconfig", help="base munin config to append nodes to")
parser.add_argument("target", help="target munin config")
args = parser.parse_args()
base = args.baseconfig
target = args.target
shutil.copyfile(base, target)
nodes = findNodes('CNAME')
appendNodesToConfig(nodes, target, 'CNAME')
For the API calls to work you have to setup AWS API credentials or assign an IAM role with the required permissions (ec2:DescribeInstances as a bare minimum) to your munin master instance (which is my prefered method).
Some final implementation notes:
I have a tag named CNAME assigned to all my AWS instances which holds the internal DNS host name. Therefore I filter for this tag and use the value as the node name and address for the munin configuration. You probably have to change this for your setup.
Another option would be to assign a specific tag to all the instances you want to monitor with munin. You could then filter for this tag and probably also skip the check for the open munin port.
Hope this is of some help.
Cheers,
Oliver