Is it possible to tag an Elastic IP address with boto3? I know you can tag them through the console and I have tagged many of them this way but I can seem to find any documentation on tagging them with boto3.
I this is not possible with boto3 is there some other library I can use to accomplish this?
The boto3 library offers a generic create_tags method that can be used to apply tags to a number of different types of AWS resources, including Elastic IPs. Here's an example of how to use it:
import boto3
ec2 = boto3.client('ec2', region_name='us-east-1')
response = ec2.create_tags(
Resources=[
'eipalloc-094ca1234de5abcd',
],
Tags=[
{
'Key': 'Description',
'Value': 'Production EIP'
},
]
)
Note: this adds the tag Description=Production EIP to whatever tags are already present on the resource (or it overwrites the Description tag if it already exists).
Related
In AWS, All the instances which do not have the tag "os" with value "linux" should shut down in the region ?.
can someone please help doing this. I tried, but not able to achieve.
I found the possibility to stop instances with specific tag, but unable to create lambda function to stop ec2 instances with out specific tag and value.
Could you get a list of all of your instances, then iterate through them, declining to terminate if they have the tag? In Python, something like:
import boto3
def has_tag(instance):
# check if instance has tag
client = boto3.client('ec2')
response = client.describe_instances(
Filters=[
...
],
...
)
for instance in response['Reservations']['Instances']:
if not has_tag(instance):
#terminate instance
You could use the Resource Group Tagging API to get the list of resource that do not have that value.
This would get you a list of the ARN of all the EC2 instances that do not have "linux" as the value of any tag:
aws resourcegroupstaggingapi get-resources --resource-type-filters ec2:instance --query 'ResourceTagMappingList[?!(Tags[?Value==`linux`])].ResourceARN'
Assuming "linux" is a value only used for the "os" tag to filter out any resources whose tags do not have that value, but you can adjust the JMESPath query as needed.
I'm looking for a programmatic way to retrieve parameters just by giving the name or a part of the complete path (instead of giving the full path with the name).
It's pretty easy using the Parameter Store AWS Systems Manager console, if I type tokens, I retrieve all parameters where the Name contains tokens :
Is there a way to do the same but using AWS CLI or AWS SDK (python or Go preferably) ?
I think this is what you are after:
aws ssm describe-parameters --parameter-filters Key=Name,Values=token,Option=Contains
Or with Python:
import boto3
response = boto3.client("ssm").describe_parameters(
ParameterFilters=[
{
'Key': 'Name',
'Option': 'Contains',
'Values': [
'token',
]
},
]
)
Is there a way to upload file to AWS S3 with Tags(not add Tags to an existing File/Object in S3). I need to have the file appear in S3 with my Tags , ie in a single API call.
I need this because I use a Lambda Function (that uses these S3 object Tags) is triggered by S3 ObjectCreation
You can inform the Tagging attribute on the put operation.
Here's an example using Boto3:
import boto3
client = boto3.client('s3')
client.put_object(
Bucket='bucket',
Key='key',
Body='bytes',
Tagging='Key1=Value1'
)
As per the docs, the Tagging attribute must be encoded as URL Query parameters. (For example, "Key1=Value1")
Tagging — (String) The tag-set for the object. The tag-set must be
encoded as URL Query parameters. (For example, "Key1=Value1")
EDIT: I only noticed the boto3 tag after a while, so I edited my answer to match boto3's way of doing it accordingly.
Tagging directive is now supported by boto3. You can do the following to add tags if you are using upload_file()
import boto3
from urllib import parse
s3 = boto3.client("s3")
tags = {"key1": "value1", "key2": "value2"}
s3.upload_file(
"file_path",
"bucket",
"key",
ExtraArgs={"Tagging": parse.urlencode(tags)},
)
If you're uploading a file using client.upload_file() or other methods that have the ExtraArgs parameter, you specify the tags differently you need to add tags in a separate request. You can add metadata as follows, but this is not the same thing. For an explanation of the difference, see this SO question:
import boto3
client = boto3.client('s3')
client.upload_file(
Filename=path_to_your_file,
Bucket='bucket',
Key='key',
ExtraArgs={"Metadata": {"mykey": "myvalue"}}
)
There's an example of this on the S3 docs, but you have to know that "metadata" corresponds to tags be aware that metadata is not exactly the same thing as tags though it can function similarly.
s3.upload_file(
"tmp.txt", "bucket-name", "key-name",
ExtraArgs={"Metadata": {"mykey": "myvalue"}}
)
rds_client = boto3.client('rds', 'us-east-1')
instance_info = rds_client.describe_db_instances( DBInstanceIdentifier='**myinstancename**')
But the instance_info doesn't contain any tags I set in the RDS instance. I want to fetch the instances that has env='production' in them and want to exclude env='test'. Is there any method in boto3 that fetched the tags as well?
Only through boto3.client("rds").list_tags_for_resource
Lists all tags on an Amazon RDS resource.
ResourceName (string) --
The Amazon RDS resource with tags to be listed. This value is an Amazon Resource Name (ARN). For information about creating an ARN, see Constructing an RDS Amazon Resource Name (ARN) .
import boto3
rds_client = boto3.client('rds', 'us-east-1')
db_instance_info = rds_client.describe_db_instances(
DBInstanceIdentifier='**myinstancename**')
for each_db in db_instance_info['DBInstances']:
response = rds_client.list_tags_for_resource(
ResourceName=each_db['DBInstanceArn'],
Filters=[{
'Name': 'env',
'Values': [
'production',
]
}])
Either use a simple exclusion over the simple filter, or you can dig through the documentation to build complicated JMESPath filter using
paginators.
Notes : AWS resource tags is not a universal implementation. So you must always refer to the boto3 documentation.
Python program will show you how to list all rds instance, there type and status.
list_rds_instances.py
import boto3
#connect ot rds instance
client = boto3.client('rds')
#rds_instance will have all rds information in dictionary.
rds_instance = client.describe_db_instances()
all_list = rds_instance['DBInstances']
print('RDS Instance Name \t| Instance Type \t| Status')
for i in rds_instance['DBInstances']:
dbInstanceName = i['DBInstanceIdentifier']
dbInstanceEngine = i['DBInstanceClass']
dbInstanceStatus = i['DBInstanceStatus']
print('%s \t| %s \t| %s' %(dbInstanceName, dbInstanceEngine, dbInstanceStatus))
Important Note: While working with boto3 you need to setup your credentials in two files ~/.aws/credentials and ~/.aws/config
~/.aws/credentials
[default]
aws_access_key_id=<ACCESS_KEY>
aws_secret_access_key=<SECRET_KEY>
~/.aws/config
[default]
region=ap-south-1
I am planning to use AWS autoscaling groups for my webservers. As a monitoring solution I am using munin at the moment. In the configuration file on the munin master server, you have to give IP addresses or host names for every host you want to monitor.
Now with autoscaling the number of instances will change frequently, and writing static information in the munin config does not seem to fit well in this environment. I could probably query all server addresses I want to monitor and write the munin master configuration file then, but this seems not like a good approach to me.
What is the preferred way of using munin in such an environment? Does someone use munin with autoscaling?
In general I would like to keep using munin and not switch to another monitoring solution because I wrote quite a lot of specific plugins that I rely on. However if you have another monitoring solution that will probably let me keep my plugins I am also open for that.
One year ago we used munin as alternative monitoring system and I will tell you one: I don't like it at all.
We had some automation for auto scaling system in nagios too, but this is also ugly way to monitor large amount of AWS instances because nagios starts to lag/crash after some amount of monitoring instances.
If you have more that 150-200 instances to monitor I suggest you to use some commercial services like StackDriver or other alternatives.
I stumbled across this old topic because I was looking for a solution to the same problem. Finally I found a way that works for me which I would like to share with you. The tl;dr summary
use AWS Python API to get all instances in the same VPC the munin master is in
test if munin port 4949 is open on the instances found to detect munin nodes
create munin.conf from a munin.base.conf (without nodes) and append entries for all the nodes found
run the script on the munin master all 5 minutes via cron
Finally, here is my Python script which does all the magic:
#! /usr/bin/python
import boto3
import requests
import argparse
import shutil
import socket
socketTimeout = 2
ec2 = boto3.client('ec2')
def getVpcId():
response = requests.get('http://169.254.169.254/latest/meta-data/instance-id')
instance_id = response.text
response = ec2.describe_instances(
Filters=[
{
'Name' : 'instance-id',
'Values' : [ instance_id ]
}
]
)
return response['Reservations'][0]['Instances'][0]['VpcId']
def findNodes(tag):
result = []
vpcId = getVpcId()
response = ec2.describe_instances(
Filters=[
{
'Name' : 'tag-key',
'Values' : [ tag ]
},
{
'Name' : 'vpc-id',
'Values' : [ vpcId ]
}
]
)
for reservation in response['Reservations']:
for instance in reservation['Instances']:
result.append(instance)
return result
def getInstanceTag(instance, tagName):
for tag in instance['Tags']:
if tag['Key'] == tagName:
return tag['Value']
return None
def isMuninNode(host):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(socketTimeout)
try:
s.connect((host, 4949))
s.shutdown(socket.SHUT_RDWR)
return True
except Exception as e:
return False
finally:
s.close()
def appendNodesToConfig(nodes, target, tag):
with open(target, "a") as file:
for node in nodes:
hostname = getInstanceTag(node, tag)
if hostname.endswith('.'):
hostname = hostname[:-1]
if hostname <> None and isMuninNode(hostname):
file.write('[' + hostname + ']\n')
file.write('\taddress ' + hostname + '\n')
file.write('\tuse_node_name yes\n\n')
parser = argparse.ArgumentParser("muninconf.py")
parser.add_argument("baseconfig", help="base munin config to append nodes to")
parser.add_argument("target", help="target munin config")
args = parser.parse_args()
base = args.baseconfig
target = args.target
shutil.copyfile(base, target)
nodes = findNodes('CNAME')
appendNodesToConfig(nodes, target, 'CNAME')
For the API calls to work you have to setup AWS API credentials or assign an IAM role with the required permissions (ec2:DescribeInstances as a bare minimum) to your munin master instance (which is my prefered method).
Some final implementation notes:
I have a tag named CNAME assigned to all my AWS instances which holds the internal DNS host name. Therefore I filter for this tag and use the value as the node name and address for the munin configuration. You probably have to change this for your setup.
Another option would be to assign a specific tag to all the instances you want to monitor with munin. You could then filter for this tag and probably also skip the check for the open munin port.
Hope this is of some help.
Cheers,
Oliver