I need to upload the updated files into multiple ec2 instace which is under single LB. My problem is I missed some ec2 instance and it broke my webpage.
Is there any tool available to upload the multiple files to multiple EC2 windows server in a single click.
I will update my files weekly or some times daily. I checked with Elastic beanstalk , Amazon Code Deploy and Amazon EFS. But the are hard to use. Anyone please help
I will suggest use AWS S3 and AWS CLI. What you can do is install AWS CLI on all the EC2 instance. Create a Bucket in AWS S3.
Start a Cron Job on each EC2 instance with below syntax.
aws s3 sync s3://bucket-name/folder-on-bucket /path/to/local/folder
So what will happen is when you upload new images to the S3 bucket all images will automatically sync with all the EC2 instances behind your load balancer. And also AWS s3 will be central directory where you upload and delete images.
You could leverage the AWS CLI, you could run something like
aws elb describe-load-balancers --load-balancer-name <name_of_your_lb> --query LoadBalancerDescriptions[].Instances --output text |\
xargs -I {} aws ec2 describe-instances --instance-id {} --query Reservations[].Instances[].PublicIpAddress |\
xargs -I {} scp <name_of_your_file> <your_username>#{}:/some/remote/directory
basically it goes like this:
find out all the ec2 instances connected to your Load Balancer
for each of the ec2 instances, find out the PublicIPAddress (supposedly you have since you can connect to them through scp)
run scp command to copy 1 files somewhere on the ec2 server
you can copy also copy folder if you need to push many files , it might be easier
Amazon ElasticFileSystem would probably now be the easiest option, you would create your file system and attach it to all your ec2 instances that are attached to the Load Balancer, and when you transfer files to the EFS it will be available to all the ec2 instances where the EFS is attached
(the setup to create EFS and mount it to your ec2 instances has to be done once only)
Create a script containing some robocopy commands and run it when you want to update the files on your servers. Something like this:
robocopy Source Destination1 files
robocopy Source Destination2 files
You will also need to share the folder you want to copy to with the user on your machine.
I had an application load balancer (alb), so I had to build on #FredricHenri's answer
EC2_PUBLIC_IPS=`aws elbv2 --profile mfa describe-load-balancers --names c360-infra-everest-dev-lb --query 'LoadBalancers[].LoadBalancerArn' --output text | xargs -n 1 -I {} aws elbv2 --profile mfa describe-target-groups --load-balancer-arn {} --query 'TargetGroups[].TargetGroupArn' --output text | xargs -n 1 -I {} aws elbv2 --profile mfa describe-target-health --target-group-arn {} --query 'TargetHealthDescriptions[*].Target.Id' --output text | xargs -n 1 -I {} aws ec2 --profile mfa describe-instances --instance-id {} --query 'Reservations[].Instances[].PublicIpAddress' --output text`
echo $EC2_PUBLIC_IPS
echo ${EC2_PUBLIC_IPS} | xargs -n 1 -I {} scp -i ${EC2_SSH_KEY_FILE} ../swateek.txt ubuntu#{}:/home/ubuntu/
Points to Note
I have used an AWS profile called "MFA", this is optional
The other environment variables EC2_SSH_KEY_FILE is the name of the .pem file used to access the EC2 instance.
Related
In AWS Backup, I have created a resource assignment to a backup-plan, which targets all EC2 instances.
The ARN prefix looks like this:
arn:aws:ec2:*:*:instance/*
How can I list all instances that match an ARN prefix? Either in AWS Cloudshell or with the aws cli?
I think you can try using ec2's describe-instances cli command and run it over all AWS regions :
for region in `aws ec2 describe-regions --output text | cut -f3`
do
echo -e "\nListing Instances in region:'$region'..."
aws ec2 describe-instances --region $region
done
I need to get information such as VPCs, subnets, security groups, etc for many AWS accounts at once. How can I go about this?
One solution is to use a for loop with the AWS CLI. Check out the CLI Documentation for the service that you're wanting to gather information for and find the appropriate commands then use a for loop to loop over the profiles in your ~/.aws/credentials file.
For example, if you're wanting to get the VPCs, subnets, and security groups, those are all described in the EC2 CLI docs.
Here is an example of getting information about those resources and outputting it into the current directory as .json (this assumes you didn't change the default output format when using aws configure
#!/usr/bin/env bash
region=us-east-1
for profile in `grep [[] ~/.aws/credentials | tr -d '[]'`
do
echo "getting vpcs, subnets, and security groups for $profile"
aws ec2 describe-vpcs --region $region --profile $profile > "$profile"_vpcs.json
aws ec2 describe-subnets --region $region --profile $profile > "$profile"_subnets.json
aws ec2 describe-security-groups --region $region --profile $profile > "$profile"_security_groups.json
done
I have created EC2 instance in the my office PC. It was successfully and I used it well. But when I logged to the AWS console from my home laptom - no one instance exist there and I have the error You do not have any instances in this region. I try to search instance information in other regions but do not found any.
How I can found my created instance or list all instances independently of region?
Quick way would be to use combination of AWS CLI, jq and a simple Bash for loop to iterate through each region and list the instances. Be sure to set your credentials before running
for region in `aws ec2 describe-regions | jq .Regions\[\].RegionName -r`
do
echo -e "\tRegion: ${region}"
aws ec2 describe-instances --query "Reservations[*].Instances[*].{InstanceID:InstanceId}" --output=table --region ${region}
done
You can copy post the code in your Linux shell, or run them in AWS CloudShell which gives you an authenticated shell with aws cli preinstalled
How do I find the name of "this" Lightsail instance. "This" being the instance that the aws command is being executed. My below script isn't working, since I thought Lightsail is just another EC2 instance.
#!/bin/bash
InstanceId=`curl -s http://169.254.169.254/latest/meta-data/instance-id`
echo $InstanceId
Region=`aws configure get region`
echo $Region
InstanceName=$(aws ec2 describe-tags --region $Region --filters "Name=resource-id,Values=$InstanceId" "Name=key,Values=Name" --output text | cut -f5)
echo $InstanceName
The name of a Lightsail instance can be obtained with:
aws lightsail get-instances --query instances[].name
In my case, this was the auto-assigned name when I started the instance via the Lightsail management console. I couldn't see a way to change the name during launch.
Interestingly, I could not find a way to use the AWS CLI to list tags associated with a Lightsail instance. For example, I could not retrieve the Name tag that I manually added to an instance, and which appears in the Lightsail console.
Update:
After discussion in comments, I got this working:
aws lightsail get-instances --query "instances[?contains(supportCode,'`curl -s http://169.254.169.254/latest/meta-data/instance-id`')].name" --output text
I have found a script for starting/stopping a dynamically created ec2 instance, but how do I start any instances in my inventory?
Seems you are talking about scripting, not SDK. So there are two tools to do the job.
1 AWS CLI tools
download aws cli tool and set the API Key in $HOME/.aws/credentials
list all instances on region us-east-1
Confirm which instances you are targeting.
aws ec2 describe-instances --query 'Reservations[].Instances[].InstanceId' --region us-east-1 --output text
2 Amazon EC2 Command Line Interface Tools
download and setup instruction
list all instances on region us-east-1
You should get same output as WAY #1.
ec2-describe-instances --region us-west-2 |awk '/INSTANCE/{print $2}'
With the instance ID list, you can use your command to start them one by one.
for example, the instance name are saved in file instance.list
while read instance
do
echo "Starting instance $instance ..."
ec2-start-instances "$linstance"
done < instance.list
BMW, give you an excellent startup, but you can even summarise the thing like this:
1) First get the id of all the instances and save them into a file
aws ec2 describe-instances --query 'Reservations[].Instances[].InstanceId' --region us-east-1 --output text >> id.txt
2) Then simply run this command to start all the instances
for id in $(awk '{print $1}' id.txt); do echo "starting the following instance $id"; aws ec2 start-instances --instance-ids --region us-east-1 $id; done
Please change the region, I am considering that you have installed and setup the AWS CLI tools properly. Thanks