I created multiple (say 16) AWS EC2 ubuntu instances such as:
I want to keep these instances to have the same settings for later jobs. My question is how I could manage them jointly. For example, how could I install Docker in all of them at once and so that I can use docker swarm?
Ideally you would actually configure the server build before you deploy the 16 instances.
You would launch a fresh Ubuntu server and install all of the software on it with its configuration. Once all software is installed you'd create an AMI. When you go to launch the 16 servers you'd go ahead with launching them from your AMI instead of the Ubuntu image.
To follow best practices you'd not do this installation by hand, instead using a configuration automation tool such as Ansible, Chef or Puppet to configure the server to your liking.
You can make use of aws user data to install same software on all the instance during ec2 creation.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
Related
I'm trying to set up a web development environment on Amazon Workspaces running Amazon Linux AMI, but I didn't find a way to install Vagrant on the machine. I would like to have a virtual webdev machine for various practical reasons, but it seems that I can't run vagrant as AWS is already virtualised.
Is that correct, or is there a way to install and run vagrant/virtualbox containers on AWS Workspace?
AWS workspaces only offers a limited number of packages within its repo manager, so you won't find vagrant there. But you can manually install the repo using the CentOS download on their website. For example, this worked for me inside my Linux AMI WorkSpace:
wget https://releases.hashicorp.com/vagrant/2.1.2/vagrant_2.1.2_x86_64.rpm
yum install vagrant_2.1.2_x86_64.rpm
Now a WorkSpace is essentially a virtualized environment, so its unlikely you will be able to run a vm inside it - See this.
However vagrant offers a number of providers other than the default - including aws, which will allow you to spin up a vagrant box on an ec2 instance rather than locally. You can install it as follows:
vagrant plugin install vagrant-aws
And follow the configuration steps here
Is there any way to install exe/MSI agents in AWS EC2 instances in an automated way?? In specific, I am looking for a counterpart of Azure's Custom Script Extension. [Free of cost]
Scenario:
I want to install BigFix and Datadog agents on 1000 Ec2 instances, this is a one time job, so I am not looking for any solution that involves Chef / Puppet, etc.,
Yes, you can pass a script to the instance that will be executed on the first boot (but not thereafter). It is often referred to as a User Data script.
See:
Running Commands on Your Windows Instance at Launch - Amazon Elastic Compute Cloud
Running Commands on Your Linux Instance at Launch - Amazon Elastic Compute Cloud
If you wish to install after the instance has started, use the AWS Systems Manager Run Command.
I have a release of my project. I build a docker image and deploy it on an ec2 instance.
Later, when I have a new release, I would like update the docker on ec2 remotely (without accessing the machine, just executing some service).
Is there a way how to do it without ECS and ElasticBeanstalk?
If it's not possible can I somehow re-run the cfn-init script?
My Research
https://aws.amazon.com/blogs/aws/new-ec2-run-command-remote-instance-management-at-scale/
You can manage your instances remotely (i.e. make changes without manually SSHing into the instance and typing commands) by using any of the many system management services out there. AWS offers Simple Systems Manager (SSM) of which the Run Command you linked is part. AWS also offers the OpsWorks service which uses Chef. You also have other products like Ansible and SaltStack, and you can optionally integrate the use of those services with the AWS SSM service.
I'm seeing so many different sources how to to achieve CI with Jenkins and EC2 and strangely none seem to fit my needs.
I have 2 EC2 ubuntu instances. One is empty and the other has Jenkins installed on it.
I want to perform a build on the Jenkins machine and copy the jar to the other ubuntu machine. Once the jar is there i want to run mvn spring-boot:run
That's is - a very simple flow which i can't find a good source to follow that doesn't include slaves, dockers etc..
AWS Code Deploy lets you use a Jenkins and deploy it on your EC2 instances.
Quick google search gave me this very detailed instruction on how to setup code pipeline with AWS Code Deploy.
The pipeline uses GitHub -> Jenkins -> EC2 flow, as you need it.
Set up jenkins to do a build then scp the artifact to the other machine
There's an answer here how to setup ssh keys for jenkins to publish via ssh about setting up the keys for ssh
In the context of AWS:
AMI is used to package software and can be deployed on EC2.
Docker can also be used to package software and can also be deployed to EC2.
What's the difference between both and how do I choose between them?
An AMI is an image. This is a whole machine that you can start new instances from. A docker container is more lightweight and portable. A docker container should be transportable between providers while an AMI is not (easily).
AMI's are VM images basically.
Docker containers are packaged mini-images that run on some VM in an isolated environment.
Eventhough this doesn't answer the question directly, but gives some background on how they are used.
One approach is you launch EC2 instances with Amazon AMI's (or can be any AMI) then run docker containers (with all dependencies) on top of it. With this approach, the docker image gets bloated over time and there is a container drift over time. Also time taken for the application to be up and running is more as the Ec2 has to be booted and docker has to bring up your app server.
Another approach is "Immutable Ec2 instances". With this approach, you use Amazon AMI as base and install all the dependencies ( use shell scripts or Ansible) and bake them in the AMI. We use Hashicorp Packer which is an amazing tool. Here the time taken for the application to be up and running is greatly reduced as all the dependencies ( java8 , tomcat, war file etc) are already installed in the AMI.
For production use case, use Packer to create AMI and use Terraform to launch cloud resources to use this AMI. Tie all this together in Jenkins pipeline.
This link has details about differences between Docker and AMI:-
https://forums.docker.com/t/how-would-you-differentiate-between-docker-vs-ec2-image/1235/2