I have a requirement :
I have 3 ec2 instance A, B, C in A instance i jenkins already installed in that jenkins i need to create a pipeline job which will start and stop instance B & C.
Is it possible ?
You should be able to install the Amazon EC2 Plugin and leverage it to start EC2 instances on demand, and correspondingly terminate them as they get unused.
Yes it is possible.
Install aws cli:
sudo apt-get install awscli
Configure aws credentails for aws cli:
aws configure
Start ec2 instance:
aws ec2 start-instances --instance-ids YOUR_INSTANCE_ID
Stop ec2 instance:
aws ec2 stop-instances --instance-ids YOUR_INSTANCE_ID
Keep in mind that in order to start/stop an instance your instance has to have an Amazon EBS volume as its root device.
Reference:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Stop_Start.html
http://shahzadlinux.blogspot.com/2019/04/how-to-stop-and-start-ec2-instance.html (Specific details for jenkins)
Related
With a human error if an AMI associated to an EC2 got deleted and unrecovered. Is it possible to add new AMI to existing EC2 which is running? Does this destroy the existing EC2 and do we have to create new EC2?
Once an EC2 instance is created it doesn't matter at all if you delete the AMI. The AMI is not "in use" when an EC2 instance is running. The EBS volume(s) that were created when you launched the instance were copied from the AMI, at which point the AMI is no longer involved in the process at all.
You do not need to "add new AMI to existing EC2" which is impossible anyway.
You can create new AMI for that EC2, make sure you enable no reboot option before create AMI, other wise server will be rebooted.
You can use AWS CLI like below
INSTANCE_ID=`/opt/aws/bin/ec2-metadata -i | /usr/bin/awk '{print $2}'`
/usr/bin/aws ec2 create-image --no-reboot --instance-id $INSTANCEID --name "AMINAME" --description "description"
You can also use AWS console.
Creating an AMI will not destroy ANY EC2. it is backup for EC2 for DR, if EC2 fails you can launch new EC2 from updated AMI.
You can also use AWS AMI scheduledr -
https://aws.amazon.com/premiumsupport/knowledge-center/ec2-systems-manager-ami-automation/
I just want to know if there are other ways to approach this problem:
I have an AWS multi account setup. The EC2's are going to be monitored over all the accounts and when alerts are triggered via sns there is a mail going to be sent. For all EC2's with Windows Server 2016 and later, Amazon Linux and Ubuntu 16.04 and 18.04 the SSM agents come pre-installed. That way I can push the CloudWatch agent via System Manager Run Command to the EC2's per AWS account.
I was wondering is there a more simple way that i can force that CloudWatch Agent is installed with every new EC2 deployed in an AWS account, without installing the agent manually on the instance or via Run Command?
I was thinking working with tags, something like: "IsMonitored" and as value true or false. for example everyday at 17hr a Lambda function will go over all the instances in that account and search for IsMonitored = false, Get that instance ID and with a (boto3?) cript push the agent on that instance. This seemed to complicated so i wanted to check if there is maybe other simple solutions that would do the same.
Thanks in advance,
Iman
To install a cloudwatch agent in each instance particular region you can implement by shell script.
The approach is:
Manual work is create some default configuration file in parameter store for both the type of instance a. for windows b. for linux based
In shell script
For particular region
Get the total number of ec2 instance id list
Check the platform which type of machine is using Windows or Linux based
If the platform is Windows then add Windows type configuration file from parameter store else add Linux configuration file
For getting platform name :
platform=$(aws ec2 describe-instances --instance-ids <instance id> --query 'Reservations[*].Instances[*].[Platform]' --output text)
For installing packages :
aws ssm send-command --instance-ids <instance id> --document-name "AWS-ConfigureAWSPackage" --parameters "name=AmazonCloudWatchAgent,action=Install,installationType=Uninstall and reinstall" --comment "Install CloudWatch Agent on EC2 Windows/Linux machine"
For start CWagent :
aws ssm send-command --instance-ids $one_instance --document-name "AmazonCloudWatch-ManageAgent" --parameters "mode=ec2,optionalRestart=yes,optionalConfigurationSource=ssm,action=configure,optionalConfigurationLocation=AmazonLinuxCloudWatchAgentConfig" --comment "Configure CloudWatch Agent on EC2 Linux machine"
For more reference you can use this link.
One simpler approach could be using prebaked AMI. First, spin up an EC2 with the normal AMI you use. Next, install the CloudWatch agent and create an image. From now on, you can spin up EC2's using the new AMI which has CloudWatch agent preinstalled.
If prebaked AMI doesn't work for you, I recommend using an infrastructure-as-code (IaC) tool like Ansible to automate the installation process.
I want to launch an EC2 machine using aws cli. I want several things to take place before I connect, including setting my configuration.
I successfully launch the machine using:
aws ec2 run-instances --image-id ami-062f7200baf2fa504 --count 1 \
--instance-type t2.micro --key-name MyFirstKey --security-group-ids \
launch-wizard-3 --user-data file://aws_setup_script.txt
my aws_setup_script.txt is
sudo yum update -y
aws configure set aws_access_key_id AAAAABBBBBCCCCCDDDDD
aws configure set aws_secret_access_key AAAAABBBBBCCCCCDDDDDEEEEEFFFFFGGGGGHHHHH
aws configure set default.region us-east-1
sudo yum update -y successfully runs, but the aws configure steps do not.
It is insecure passing secrets in user-data.
Your script is failing because it isn't running as ec2-user so it doesn't have aws in the path. Even if it worked, it wouldn't be configuring the CLI tool for the ec2-user account so it isn't going to work the way you want.
Most importantly, there is a much better way to accomplish this. You should be assigning an IAM instance profile to the instance. When you run the aws cli tool on an instance with an IAM role assigned it will automatically use those credentials.
As per best practice, It's always better to use the IAM instance role attached to your Ec2 instead of setting the AWS credentials within Ec2.
Create an IAM instance role (refer AWS Doc) with the required permission want to give to Ec2.
Use --iam-instance-profile in aws cli command to attache the Ec2 with specific Iam role.
aws ec2 run-instances --image-id ami-062f7200baf2fa504 --count 1 \
--instance-type t2.micro --key-name MyFirstKey --security-group-ids \
launch-wizard-3 --iam-instance-profile
I have an EC2 with a role that gives it full control over others EC2.
This role allows calling aws ec2 ... without doing the aws configure step.
However, if I install docker and run a docker container inside that EC2, this container is not able to do the aws ec2 ... without configuring the awscli.
Is there some kind of folder to share of feature to enable in order to run awscli commands inside my container without configuring it with an accesskey/password ?
The aws command is utilizing the IAM instance profile assigned to the EC2 instance, which it is obtaining via the EC2 metadata service. You would need to share that metadata with the Docker container somehow.
Are you using the AWS ECS service? Or are you manually installing and managing docker on an EC2 instance? ECS handles this for you.
Otherwise you might look into something like this Lyft project designed to proxy the EC2 IAM role to the Docker container.
I started an instance based on my AMI (based on Ubuntu 12.04 server) with the following command.
aws ec2 run-instances --image-id MY_AMI_ID --count 1 --instance-type t1.micro
What's surprising is, after I terminated the instance using the following command, it left an volume.
aws ec2 terminate-instances --instance-id MY_INSTANCE_ID
I would like to have the volume destroyed automatically, not sure if there is an easy option in the command line to do it.
Have you attached the volume after launching the instance?
As Amazon EC2 deletes all volumes that were attached during instance launch. Only volumes attached after instance is launched, will not be deleted.
Your AMI probably has the option set to not terminate block devices. You can adjust this behavior in your AMI by using the "delete-on-termination" option in AWS Console or the AWS CLI ec2-register command:
http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-RegisterImage.html
Found that
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/terminating-instances.html has an example
aws ec2 modify-instance-attribute --instance-id i-63893fed --block-device-mappings "[{\"DeviceName\": \"/dev/sda1\",\"Ebs\":{\"DeleteOnTermination\":true}}]"
That solves my problem: now after an instance is terminated, it will not leave a volume behind.