I am wondering what are benefits of using Docker to deploy my Spring Boot app on AWS EC2/ESB. Are there any?
This is a very broad question but in short the advantages are that if built right, Docker can fully automate your deploy process, allowing you to autoscale and resolve failures quickly (by destroying faulty containers and launching new ones instead - ideally using an automated process).
Related
Currently we have Jenkins that is running on-premise(VMware), planning to move into the cloud(aws). What would be the best approach to install Jenkins whether on ec2 or ECS?
Best way would be running on EC2. Make sure you have granular control over your instance Security Group and Network ACL's. I would recommend using terraform to build your environment as you can write code and also version control it. https://www.terraform.io/downloads.html
Have you previously containerized your Jenkins? On VMWare itself? If not, and if you are not having experience with containers, go for EC2. It will be as easy as running on any other VM. For reproducing the infrastructure, use Terraform or CloudFormartion.
I would recommend dockerize your on-premise Jenkins first. See how much efforts are required in implementation and administrating/scaling it. Then go for ECS.
Else, shift to EC2 and see how much admin overhead + costs you are billed. Then if required, go for ECS.
Another point you have to consider is how your Jenkins is architected. Are you using master-slave? Are you running builds contentiously so that VMs are never idle? Do you want easy scaling such that build environment is created and destroyed per build execution?
If you have no experience with running containers then create it on EC2. Before running on ECS make sure you really understand containers and container orchestration.
Just want to complement the other answers by providing link to official AWS white paper:
Jenkins on AWS
It might be of special interest as it discusses both options in detail: EC2 and ECS:
In this section we discuss two approaches to deploying Jenkinson AWS. First, you could use the traditional deployment on top of Amazon Elastic Compute Cloud (Amazon EC2). Second, you could use the containerized deployment that leverages Amazon EC2 Container Service (Amazon ECS).Both approaches are production-ready for an enterprise environment.
There is also AWS sample solution for Jenkins on AWS for ECS:
https://github.com/aws-samples/jenkins-on-aws:
This project will build and deploy an immutable, fault tolerant, and cost effective Jenkins environment in AWS using ECS. All Jenkins images are managed within the repository (pulled from upstream) and fully configurable as code. Plugin installation is automated, including versioning, as well as configured through the Configuration as Code plugin.
I have a microservice architecture web application that I need to host in AWS in a cheap and optimized manner.
I have 3 spring boot applications and two node applications. My Application used MySql Database.
Following is my plan:
Get 1 EC2 instance.
Get RDS for mySql DB.
Install docker on EC2.
create 2 docker containers.
a. One tomcat container to run all spring boot applications.
b. one container to run node applications.
Q1. Is it possible to deploy my application in this manner, or am I inherently flawed in my understanding of AWS architecture?
Q2. Do I need a 3rd Nginx docker container?
Q3. is there anything else required?
Any Help is welcome. Thanks in advance.
In my opinion, the current design is good to begin with keeping in mind you want to have the economy in mind. You have isolated your datastore by moving it to RDS.
Q1. Yes, I think your approach is fine. But this would mean you will have to take care of the provisioning of the EC2 instance and RDS instance on your own. You can also try to explore Elastic Beanstalk if you want to offload all this to AMZ. The tech stack that you are currently using is supported by Elastic Beanstalk and you may find it a little difficult to begin with but later will prove to be beneficial.
Q2. I would say yes. You should have a separate NGINX container.
Q3. You must also try to containerize each Spring Boot application instead of having just one docker container hosting all of them. And same goes true for your 2 Node applications too. Once you have dockerized all the application you then have complete isolation of the application and can handle the resiliency & scaling part much better than keeping them together.
I hope this answers your query.
I've deployed a couple of websites on AWS Elastic Beanstalk, and then I heard of Docker, so I think I can probably try it this time for a small business e-commerce website (Lumen + Angularjs). I searched all over the Internet, but as having no experience with the Docker, it is still hard to
get a great understanding of the advantages of using Docker on AWS. All I can find are some descriptions like this:
Pros
Separated management of dependencies and server hardware
Development environment is identical (internally) to production environment
Dependency management means that not everyone needs intimate knowledge about every part of your technology stack
Easy custom task and service scheduling with AWS SDK or a third-party tool
Make good use of available resources with ECS assigning tasks to EC2’s with enough free resources Use auto-scaling when tasks need more resources
Cons
Build produces a large file that needs to be uploaded
Docker NAT can increase network latency (use docker run –net=host, for more docker performance info see here)
Some developers have fits when the word docker is mentioned
Some applications need to be fixed to work on Docker
Can someone give me some simple examples or explanations?
I think the primary advantage of Docker on Elastic Beanstalk is the flexibility it gives you when compared to running your app on one of the specific runtime environments Elastic Beanstalk supports.
From the official documentation:
Elastic Beanstalk supports the deployment of web applications from
Docker containers. With Docker containers, you can define your own
runtime environment. You can choose your own platform, programming
language, and any application dependencies (such as package managers
or tools), that aren't supported by other platforms. Docker containers
are self-contained and include all the configuration information and
software your web application requires to run.
For example I've seen lots of people ask how to deploy Java applications that use something other than Tomcat on Elastic Beanstalk. You couldn't do that before they added Docker support.
If you are using one of the officially supported Elastic Beanstalk runtimes then it is difficult for me to recommend using Docker. It would somewhat separate your app from AWS specifics, theoretically allowing you to run your app more easily outside of AWS. If you want to avoid vendor lock-in at all costs, or if you just want to stay up to date on the latest technologies, then Docker is a good choice. Otherwise, if you already have your app running on Elastic Beanstalk there isn't much reason for you to convert it to Docker.
Edit: Note that my reply is related to using Docker specifically with Elastic Beanstalk, as your question title asks. I see in your question that you also refer to the ECS service and the more general use of Docker on AWS. That's a much bigger discussion and there are definitely some advantages to using Docker instead of plain EC2 instances for certain things.
Edit 10/5/2019: AWS seems to be pushing people towards Docker now so that they don't have to maintain updates to the official runtimes. For example the latest Java runtime for Elastic Beanstalk is Java 8. So if you want to run a modern version of Java on Elastic Beanstalk you would have to use Docker.
There is a certain amount of environments considering the elastic beanstalk. In order to add extra configurations and have something custom upon these environments, you have to utilize .ebextensions.
However ebextensions are being executed on the creation of the elastic beanstalk server. Also .ebextensions are not as easy to implement as a docker image.
By using docker on elastic beanstalk you have your image setup ready to be deployed without any extra configuration, and docker is great when you need an immutable architecture.
I'm developing a prototype IoT application which does the following
Receive/Store data from sensors.
Web application with a web-based IDE for users to deploy simple JavaScript/Python scripts which gets executed in Docker Containers.
Data from the sensors gets streamed to these containers.
User programs can use this data to do analytics, monitoring etc.
The logs of these programs are outputted to the user on the webapp
Current Architecture and Services
Using one AWS EC2 instance. I chose EC2 because I was trying to figure out the architecture.
Stack is Node.js, RabbitMQ, Express, MySQl, MongoDB and Docker
I'm not interested in using AWS IoT services like AWS IoT and Greengrass
I've ruled out Heroku since I'm using other AWS services.
Questions and Concerns
My goal is prototype development for a Beta release to a set of 50 users
(hopefully someone else will help/work on a production release)
As far as possible, I don't want to spend a lot of time migrating between services since developing the product is key. Should I stick with EC2 or move to Beanstalk?
If I stick with EC2, what is the best way to handle small-medium traffic? Use one large EC2 machine or many small micro instances?
What is a good way to manage containers? Is it worth it use swarm and do container management? What if I have to use multiple instances?
I also have small scripts which have status of information of sensors which are needed by web app and other services. If I move to multiple instances, how can I make these scripts available to multiple machines?
The above question also holds good for servers, message buses, databases etc.
My goal is certainly not production release. I want to complete the product, show I have users who are interested and of course, show that the product works!
Any help in this regard will be really appreciated!
If you want to manage docker containers with least hassle in AWS, you can use Amazon ECS service to deploy your containers or else go with Beanstalk. Also you don't need to use Swarm in AWS, ECS will work for you.
Its always better to scale out rather scale up, using small to medium size EC2 instances. However the challenge you will face here is managing and scaling underlying EC2's as well as your docker containers. This leads you to use Large EC2 instances to keep EC2 scaling aside and focus on docker scaling(Which will add additional costs for you)
Another alternative you can use for the Web Application part is to use, AWS Lambda and API Gateway stack with Serverless Framework, which needs least operational overhead and comes with DevOps tools.
You may keep your web app on Heroku and run your IoT server in AWS EC2 or AWS Lambda. Heroku is on AWS itself, so this split setup will not affect performance. You may heal that inconvenience of "sitting on two chairs" by writing a Terraform script which provisions both EC2 instance and Heroku app and ties them together.
Alternatively, you can use Dockhero add-on to run your IoT server in a Docker container alongside your Heroku app.
ps: I'm a Dockhero maintainer
I am working on a project using a microservices architecture.
Each service lives in its own docker container and has a separate git repository in order to ensure loose coupling.
It is my understanding that AWS recently announced support for Multi-Container Docker environments in ElasticBeanstalk. This is great for development because I can launch all services with a single command and test everything locally on my laptop. Just like Docker Compose.
However, it seems I only have the option to also deploy all services at once which I am afraid defies the initial purpose of having a micro services architecture.
I would like to be able to deploy/version each service independently to AWS. What would be the best way to achieve that while keeping infrastructure management to a minimum?
We are currently using Amazon ECS to accomplish exactly what you are talking about trying to achieve. You can define your Docker Container as a Task definition and then Create an ECS Service which will handle number of instances, scaling, etc.
One thing to note is Amazon mentions the word container a lot in the documentation. They may be talking about the EC2 instance used for the cluster for your docker instances/containers.