I have followed the Docker Docs about setting up Swarm on Virtualbox.
I suppose it is the same procedure to set it up on AWS, Azure or DigitalOcean.
It is a lot to do manually every time .
Is there a tool to automate this?
I would like to use something to set up and scale Swarm like Compose does for Docker .
Maybe I would start with one AWS instance and 2-3 containers and then scale them up to 100 containers and the instances to scale accordingly. Then I would want to scale down to 2 instances and the rest would shut down.
Does something like this exist ?
If you want to avoid manual configurations but still get the required high availability and cost efficiency, try to run Docker Swarm template pre-packaged by Jelastic:
it has built-in automatic clustering and scaling
the installation is performed automatically and you'll get full access to the cluster via intuitive UI
containers are running directly on bare metal, so no need to reserve full VMs for each service (and you can choose the datacenter you want to run your project on)
the payment is done based on actual consumption of RAM and CPU
containers are automatically distributed across different hardware servers that increases high availability
The details about the package and installation steps are in this article.
You can use Ansible for configuring the Swarm master, Swarm nodes, and all the required cluster discovery. Ansible is a general IT automation tool, but it comes with a very powerful Docker module that allows to set up Docker Swarm easily.
This GitHub repository shows a good example how to set up Swarm with Ansible.
You can use Docker Machine for provisioning hosts and configuring swarm easily (example).
The Docker Ecosystem includes also managed solutions like Tutum or Docker Cloud to achieve easily what you want.
Checkout devopsbyte.com blog, which covers how to set up a docker swarm cluster using ansible
Related
Currently we have Jenkins that is running on-premise(VMware), planning to move into the cloud(aws). What would be the best approach to install Jenkins whether on ec2 or ECS?
Best way would be running on EC2. Make sure you have granular control over your instance Security Group and Network ACL's. I would recommend using terraform to build your environment as you can write code and also version control it. https://www.terraform.io/downloads.html
Have you previously containerized your Jenkins? On VMWare itself? If not, and if you are not having experience with containers, go for EC2. It will be as easy as running on any other VM. For reproducing the infrastructure, use Terraform or CloudFormartion.
I would recommend dockerize your on-premise Jenkins first. See how much efforts are required in implementation and administrating/scaling it. Then go for ECS.
Else, shift to EC2 and see how much admin overhead + costs you are billed. Then if required, go for ECS.
Another point you have to consider is how your Jenkins is architected. Are you using master-slave? Are you running builds contentiously so that VMs are never idle? Do you want easy scaling such that build environment is created and destroyed per build execution?
If you have no experience with running containers then create it on EC2. Before running on ECS make sure you really understand containers and container orchestration.
Just want to complement the other answers by providing link to official AWS white paper:
Jenkins on AWS
It might be of special interest as it discusses both options in detail: EC2 and ECS:
In this section we discuss two approaches to deploying Jenkinson AWS. First, you could use the traditional deployment on top of Amazon Elastic Compute Cloud (Amazon EC2). Second, you could use the containerized deployment that leverages Amazon EC2 Container Service (Amazon ECS).Both approaches are production-ready for an enterprise environment.
There is also AWS sample solution for Jenkins on AWS for ECS:
https://github.com/aws-samples/jenkins-on-aws:
This project will build and deploy an immutable, fault tolerant, and cost effective Jenkins environment in AWS using ECS. All Jenkins images are managed within the repository (pulled from upstream) and fully configurable as code. Plugin installation is automated, including versioning, as well as configured through the Configuration as Code plugin.
I am looking at this CloudFormation template:
http://editions-us-east-1.s3.amazonaws.com/aws/stable/18.03.0/Docker.tmpl
This template creates a docker swarm cluster using EC2 instances on AWS. The process is relatively straightforward:
Create a network for the swarm
Create scaling groups for the workers and managers
Initialize the managers and reach quorum
Initialize the workers and join the swarm
I grasp at a high level what is occourng, and have manually created Docker Swarms on a local machine without difficulty. However, I am failing to grasp some key concepts.
How do the workers join the swarm? I see that the manager IP is published to a Dynamo DB table, but where would the workers get the tokens to join the swarm, and where are the commands being run, I don't see anything along the lines of docker swarm join ... in the template file
What is the purpose of the ELB (Elastic Load Balancer)? All the nodes are included in the balancer, so what is it really balancing?
Those two things are quite confusing as it seems that Docker is creating this swarm out of thin air without the use of tokens or even running the necessary docker command! I'd love a clarification on how these work!
Yea that's one of the problems with those depreciated templates, they are no longer getting updates and much of what their doing isn't open source. I haven't seen documentation on what you're asking about.
For Docker EE, Docker has the new Docker Certified Infrastructure templates (Terraform plus Ansible).
For Docker CE, Docker has no currently-supported cloud infrastructure templates. There are talks of doing something with those AWS/Azure templates, but right now it's just ideas.
What is the most cost effective way of running a single docker container on GCP? I have various simple scripts which I've packaged in images and which I'd like to move to GCP and run them as containers. From the docs Google Container engine is:
A Container Engine cluster is a group of Compute Engine instances running Kubernetes. It consists of one or more node instances, and a managed Kubernetes master endpoint. A container cluster is the foundation of a Container Engine application—pods, services, and replication controllers all run on top of a cluster.
This sounds like an overkill as I only need one Compute Engine instance with the docker toolchain installed and easy access to other cloud tools (e.g. SQL). I proceeded to provision a Compute Engine VM but then had to set up docker which felt like reinventing Google Container Engine.
EDIT: I found this which is in alpha stage as of now (2017-09-06): https://cloud.google.com/compute/docs/instance-groups/deploying-docker-containers
The most cost effective way is to run a single VM that runs your container. You can run Google's Container-Optimized OS to run the container and add a startup script to start the container when the machine boots (this OS already has docker installed and is the OS used by default in Google Container Engine).
However, you get other benefits from running on top of Google Container Engine: health checking of your container (and optionally of your VM), the ability to later trivially scale up your application to multiple replicas, the ability to easily deploy new versions of your application, support for logging / monitoring, etc. You may find that the features provided by Google Container Engine are worth the extra overhead it adds to your single node.
I would probably just set up a single node Container engine cluster.
This is pretty much the same thing as a compute engine instance anyway, and it's pretty cost effective. Especially if you find that you aren't fully using the hardware, you can just run a second docker image on the same container engine instance without paying anything extra for it.
I am working on a project using a microservices architecture.
Each service lives in its own docker container and has a separate git repository in order to ensure loose coupling.
It is my understanding that AWS recently announced support for Multi-Container Docker environments in ElasticBeanstalk. This is great for development because I can launch all services with a single command and test everything locally on my laptop. Just like Docker Compose.
However, it seems I only have the option to also deploy all services at once which I am afraid defies the initial purpose of having a micro services architecture.
I would like to be able to deploy/version each service independently to AWS. What would be the best way to achieve that while keeping infrastructure management to a minimum?
We are currently using Amazon ECS to accomplish exactly what you are talking about trying to achieve. You can define your Docker Container as a Task definition and then Create an ECS Service which will handle number of instances, scaling, etc.
One thing to note is Amazon mentions the word container a lot in the documentation. They may be talking about the EC2 instance used for the cluster for your docker instances/containers.
I've been looking into Mesos, Marathon and Chronos combo to host a large number of websites. In my head I should be able to type a few commands into my laptop, and wait about 30 minutes for the thing to build and deploy.
My only issue, is that my resources are scattered across multiple data centers, numerous cloud accounts, and about 6 on premises places. I see no reason why I can't control them all from my laptop -- (I have serious power and control issues when it comes to my hardware!)
I'm thinking that my best approach is to build the brains in the cloud, (zoo keeper and at least one master), and then add on the separate data centers, but I am yet to see any examples of a distributed cluster, where not all the nodes can talk to each other.
Can anyone recommend a way of doing this?
I've got a setup like this, that i'd like to recommend:
Source code, deployment scripts and dockerfiles in GIT
Each webservice has its own directory and comes together with a dockerfile to containerize it
A build script (shell script running docker builds) builds all the docker containers, of which all images are pushed to a docker image repository
A ansible deploy deploys all the containers remotely to a set of VPSes. (You use your own deployment procedure, that fits mesos/marathon)
As part of the process, a activeMQ broker is deployed to the cloud (yep, in a container). While deploying, it supplies each node with the URL of the broker they need to connect to. In your setup you could instead use ZooKeeper or etcd for example.
I am also using jenkins to do automatic rebuilds and to run deploys whenever there has been GIT commits, but they can also be done manually.
Rebuilds are lightning fast, and deploys dont take much time either. I can replicate everything I have in my repository endlessly and have zero configuration.
To be able to do a new deploy, all I need is a set of VPSs with docker daemons, and some datastores for persistence. Im not sure if this is something that you can replace with mesos, but ansible will definitely be able to install a mesos cloud for you onto your hardware.
All logging is being done with logstash, to a central logging server.
i have setup a 3 master, 5 slave, 1 gateway mesos/marathon/docker setup and documented here
https://github.com/debianmaster/Notes/wiki/Mesos-marathon-Docker-cluster-setup-on-RHEL-7-with-three-master
this may help you in understanding the load balancing / scaling across different machines in your data center
1) masters can also be used as slaves
2) mesos haproxy bridge script can be used for service discovery of the newly created services in the cluster
3) gateway haproxy is updated every min with new services that are created
This documentation has
1) master/slave setup
2) setting up haproxy that automatically reloads
3) setting up dockers
4) example service program
You should use Terraform to orchestrate your infrastructure as code.
Terraform has a lot of providers that allows you to manage different resources accross multiples clouds services and/or bare-metal resources such as vSphere.
You can start with the Getting Started Guide.