Hi Guys I need to do a script which copies the ips from an aws region AMIs that I have running, after copying this IPs place them into a text file inside. If the Instances are turned off, that IP would get removed, and the text file would change real-time, on it's own automatically, I need this to run across all regions, so any Instance that I have "X" AMI running with, the script would find it, copy its IP keep it if it's running and remove it from the file if they switch to shutdown mode.
stack the IPs in a text like
55.555.555.55
66.123.545.54
.....
.....
real-time.
I've never really used aws cli and I know this is possible to do.
Use the describe-instances command in the AWS CLI. All the information that you need (AMI, instance state, IP address) will be included in the response to that command. Note that you will have to run describe-instances once for each region. (Set the --region flag when running the CLI to set the region.)
You can parse the JSON output of the CLI however you want then write the information you want to the text file.
This command uses a aws cli "describe-instances" command with a filter for only instances that are running.
This outputs a lot of data including the "PublicIp" field. The sed command strips out just the ip address from that line and the uniq removes duplicates
aws ec2 describe-instances --filters 'Name=instance-state-name,Values=running' | sed -n 's/^.*"PublicIp": "\([0-9\.]*\)\",/\1/p'| uniq
See http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html for details on the aws cli describe instances command, including other filters you might want to apply
Related
I am learning how to use the CLI tool for AWS and I come across this command:
aws ec2 describe-instances --filters "Name=ip-address,Values=my.test.ip.address"
The command works fine and I can see the EC2 instance details in the command output, however, it does not have any field with the name ip-address. Instead, it has PublicIp and PublicIpAddress fields that contain the ip address which I am looking for.
How does this filter work?
There is a list of keys which can be used to filter on. This can be found in the documentation for the describe-instances command: --filter. One of them ip-address:
ip-address - The public IPv4 address of the instance.
I did not take a look on the source code for the describe-instances, but I believe it parses the JSON response and it has a preconfigured path for each of the filters.
Update:
After takin a look at the source code if aws-cli on GitHub, all the commands are transformed into API calls and send to AWS. All the filtering is happening in the back-end.
I have 3000+ ebs volumes and I want to list them in aws cli.. however when I run the command
aws ec2 describe-volumes
the result ends up showing only one ebs volume and the terminal keeps saying "skipping".
I want to list all in a text file output maybe? or is there any other way to get all ebs volumes.
I need this to describe and filter out unused ebs volumes.. and then delete them!
Based on the comments.
The solution was to use either (to save output to file):
aws ec2 describe-volumes > myfile.txt
or to display it:
ec2 describe-volumes | less
How can I get the network interface ID by using instance metadata inside an EC2?
I mean I want to get something like eni-0032fc98a9f0a13bk for the current EC2 that I'm in.
curl http://169.254.169.254/latest/meta-data/network/interfaces doesn't work.
The following script works, but it requires jq to parse the response from aws ec2 command.
$ export mac=$(curl http://169.254.169.254/latest/meta-data/mac)
$ aws ec2 describe-network-interfaces \
--region my-region \
--filters Name=mac-address,Values=${mac}
I don't want to install extra tools to get the network interface ID.
My EC2 has only one network interface attached to it.
You can use --query and --output:
aws ec2 describe-network-interfaces --region <my-region> --filters Name=mac-address,Values=${mac} --query "NetworkInterfaces[0].NetworkInterfaceId" --output text
Example outcome:
eni-04b594570017a3b53
The above assumes that you have only one ENI on your instance.
You can use:
curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/"$(curl http://169.254.169.254/latest/meta-data/network/interfaces/macs)"interface-id
Example outcome:
eni-04b594570017a3b53
The above assumes that you have only one ENI on your instance.
$() is called command substitution in bash. "" <-- quotes are always good to use even when you don't need them; because when you forget them when you need them you are left scratching your head.
Here is a answer on the topic --> https://unix.stackexchange.com/questions/118433/quoting-within-command-substitution-in-bash
So the $(curl ...) runs a curl, puts the result in an un-named variable and passes it to the outer curl, and the outer curl gives you the response you want.
Thank you Marcin for providing a template for my answer!
I am trying to set up some easy code to run when trying to spin up an EMR for some ad hoc work I have to do, time to time.
Right now I pass the 'aws emr create-cluster' command and then find the DNS in the console, once the cluster is created to then use ssh to connect.
I'd like to skip having to open the console at all, and use the cluster ID to get the DNS value to create my SSH connection, but I am not seeing a clear command to do this with. I'm new to CLI so I imagine this is a simple task I am merely failing at figuring out myself.
In my mind the solution should be something along the lines of
aws emr create-cluster [config for cluster here] > file.txt
set DNS = aws emr describe-cluster --cluster-id file.txt -MasterPublicDnsName
ssh -i Desktop/AWS/EMRKey.pem -o ServerAliveInterval=15 hadoop#$DNS
probably will have to append 'hadoop#' to the DNS variable before passing it into a command, but I'm more curious at the moment to if the above makes any functional sense, and if so, how I can get the functionality of the describe-cluster command to output the -MasterPublicDnsName, as that is obviously just something I made up and not an actual option that I have found.
The AWS CLI has a query option that lets you query the output of a command. You'll also want to use a waiter to make sure the instance is up before you try to connect to it.
You could simply run
cluster_id="j-2RNBSZZBLXTZ0"
aws emr wait cluster-running --cluster-id $cluster_id
hostname=`aws emr describe-cluster --output text --cluster-id $cluster_id --query Cluster.MasterPublicDnsName`
ssh hadoop#$hostname
That should work!
I am trying to come up with a script to automate the setting up of desired count of AutoScalingGroups based on some kind of profiles e.g., SHUTDOWN profile should set everything to zero.
We have lot of applications under single account itself. So when running below command, it gives all the resources.
aws ecs list-clusters
Is there a way to filter these by either tags or any other means? Apparently --filter is not a valid option for aws ecs or aws autoscaling commands.
I am utilizing the grep command for now.
aws ecs list-clusters | grep string1 | grep string2
Not sure that's exactly what you're asking, but if you want to play with the JSON output of these commands (or filter/transform any JSON string in general), there's no better tool than jq. Takes some time to get into, but this tool might become your best friend.
Once installed, you can issue commands such as:
aws ecs describe-clusters|jq -r '.clusters[]|{clusterName, status}'
To create a cluster name/status list from the info.
aws ecs describe-clusters|jq -r '.clusters[]|if .status == "INACTIVE" then .clusterArn else null end'
To list all the inactive clusters.
Add a delete command this way to delete all the inactive clusters (don't run it!!!):
aws ecs describe-clusters|jq -r '.clusters[]|if .status == "INACTIVE" then .clusterArn else null end'|xargs aws ecs delete-clusters --clusters
I have only one cluster at disposal, I didn't test if these commands still work with many clusters (JSON tables properly parsed), but you get the idea...
jq tutorial: https://stedolan.github.io/jq/tutorial/