Similarities that I see are:
They are PaaS offerings.
They make AWS more similar to Heroku.
They abstract away load balancing and auto scaling stuff.
The only difference that I see is that App Runner uses docker but Elastic beanstalk may not use it. Correct me if I am wrong, but seems like it is not a requirement to containerize your app first to be able to use it on App Runner as you can just supply the Github Url and App Runner will containerize it for you.
So what is the difference between the two and how do I make a decision to choose one over the other?
It depends. AWS App Runner (AR) is container based only. Not every application nor developer want to use containers, nor their application is suited for container deployments. AR also gives you very little control over your resources and operating system. Many application may require such control (e.g. gpu) Also AWS EB gives you much more control over your resources, including operating system.
As Cloud Guru said that
Behind the scenes, the core of App Builder is that it builds an Amazon
ECS Cluster and uses Fargate to execute your containers
So as you can see
Elastic Beanstalk belonging to PaaS
App Runner is serverless belonging to FaaS
Furthermore, App runner just works with Container only. So it really depends on what kind of your app is.
Can we run an application that is configured to run on multi-node AWS EC2 K8s cluster using kops (project link) into local Kubernetes cluster (setup using kubeadm)?
My thinking is that if the application runs in k8s cluster based on AWS EC2 instances, it should also run in local k8s cluster as well. I am trying it locally for testing purposes.
Heres what I have tried so far but it is not working.
First I set up my local 2-node cluster using kubeadm
Then I modified the installation script of the project (link given above) by removing all the references to EC2 (as I am using local machines) and kops (particularly in their create_cluster.py script) state.
I have modified their application yaml files (app requirements) to meet my localsetup (2-node)
Unfortunately, although most of the application pods are created and in running state, some other application pods are unable to create and therefore, I am not being able to run the whole application on my local cluster.
I appreciate your help.
It is the beauty of Docker and Kubernetes. It helps to keep your development environment to match production. For simple applications, written without custom resources, you can deploy the same workload to any cluster running on any cloud provider.
However, the ability to deploy the same workload to different clusters depends on some factors, like,
How you manage authorization and authentication in your cluster? for example, IAM, IRSA..
Are you using any cloud native custom resources - ex, AWS ALBs used as LoadBalancer Services
Are you using any cloud native storage - ex, your pods rely on EFS/EBS volumes
Is your application cloud agonistic - ex using native technologies like Neptune
Can you mock cloud technologies in your local - ex. Using local stack to mock Kinesis, Dynamo
How you resolve DNS routes - ex, Say you are using RDS n AWS. You can access it using a route53 entry. In local you might be running a mysql instance and you need a DNS mechanism to discover that instance.
I did a google search and looked at the documentation of kOps. I could not find any info about how to deploy to local, and it only supports public cloud providers.
IMO, you need to figure out a way to set up your local EKS cluster, and if there are any usage of cloud native technologies, you need to figure out an alternative way about doing the same in your local.
The true answer, as Rajan Panneer Selvam said in his response, is that it depends, but I'd like to expand somewhat on his answer by saying that your application should run on any K8S cluster given that it provides the services that the application consumes. What you're doing is considered good practice to ensure that your application is portable, which is always a factor in non-trivial applications where simply upgrading a downstream service could be considered a change of environment/platform requiring portability (platform-independence).
To help you achieve this, you should be developing a 12-Factor Application (12-FA) or one of its more up-to-date derivatives (12-FA is getting a little dated now and many variations have been suggested, but mostly they're all good).
For example, if your application uses a database then it should use DB independent SQL or no-sql so that you can switch it out. In production, you may run on Oracle, but in your local environment you may use MySQL: your application should not care. The credentials and connection string should be passed to the application via the usual K8S techniques of secrets and config-maps to help you achieve this. And all logging should be sent to stdout (and stderr) so that you can use a log-shipping agent to send the logs somewhere more useful than a local filesystem.
If you run your app locally then you have to provide a surrogate for every 'platform' service that is provided in production, and this may mean switching out major components of what you consider to be your application but this is ok, it is meant to happen. You provide a platform that provides services to your application-layer. Switching from EC2 to local may mean reconfiguring the ingress controller to work without the ELB, or it may mean configuring kubernetes secrets to use local-storage for dev creds rather than AWS KMS. It may mean reconfiguring your persistent volume classes to use local storage rather than EBS. All of this is expected and right.
What you should not have to do is start editing microservices to work in the new environment. If you find yourself doing that then the application has made a factoring and layering error. Platform services should be provided to a set of microservices that use them, the microservices should not be aware of the implementation details of these services.
Of course, it is possible that you have some non-portable code in your system, for example, you may be using some Oracle-specific PL/SQL that can't be run elsewhere. This code should be extracted to config files and equivalents provided for each database you wish to run on. This isn't always possible, in which case you should abstract as much as possible into isolated services and you'll have to reimplement only those services on each new platform, which could still be time-consuming, but ultimately worth the effort for most non-trival systems.
I'm developing a prototype IoT application which does the following
Receive/Store data from sensors.
Web application with a web-based IDE for users to deploy simple JavaScript/Python scripts which gets executed in Docker Containers.
Data from the sensors gets streamed to these containers.
User programs can use this data to do analytics, monitoring etc.
The logs of these programs are outputted to the user on the webapp
Current Architecture and Services
Using one AWS EC2 instance. I chose EC2 because I was trying to figure out the architecture.
Stack is Node.js, RabbitMQ, Express, MySQl, MongoDB and Docker
I'm not interested in using AWS IoT services like AWS IoT and Greengrass
I've ruled out Heroku since I'm using other AWS services.
Questions and Concerns
My goal is prototype development for a Beta release to a set of 50 users
(hopefully someone else will help/work on a production release)
As far as possible, I don't want to spend a lot of time migrating between services since developing the product is key. Should I stick with EC2 or move to Beanstalk?
If I stick with EC2, what is the best way to handle small-medium traffic? Use one large EC2 machine or many small micro instances?
What is a good way to manage containers? Is it worth it use swarm and do container management? What if I have to use multiple instances?
I also have small scripts which have status of information of sensors which are needed by web app and other services. If I move to multiple instances, how can I make these scripts available to multiple machines?
The above question also holds good for servers, message buses, databases etc.
My goal is certainly not production release. I want to complete the product, show I have users who are interested and of course, show that the product works!
Any help in this regard will be really appreciated!
If you want to manage docker containers with least hassle in AWS, you can use Amazon ECS service to deploy your containers or else go with Beanstalk. Also you don't need to use Swarm in AWS, ECS will work for you.
Its always better to scale out rather scale up, using small to medium size EC2 instances. However the challenge you will face here is managing and scaling underlying EC2's as well as your docker containers. This leads you to use Large EC2 instances to keep EC2 scaling aside and focus on docker scaling(Which will add additional costs for you)
Another alternative you can use for the Web Application part is to use, AWS Lambda and API Gateway stack with Serverless Framework, which needs least operational overhead and comes with DevOps tools.
You may keep your web app on Heroku and run your IoT server in AWS EC2 or AWS Lambda. Heroku is on AWS itself, so this split setup will not affect performance. You may heal that inconvenience of "sitting on two chairs" by writing a Terraform script which provisions both EC2 instance and Heroku app and ties them together.
Alternatively, you can use Dockhero add-on to run your IoT server in a Docker container alongside your Heroku app.
ps: I'm a Dockhero maintainer
I am working on a project using a microservices architecture.
Each service lives in its own docker container and has a separate git repository in order to ensure loose coupling.
It is my understanding that AWS recently announced support for Multi-Container Docker environments in ElasticBeanstalk. This is great for development because I can launch all services with a single command and test everything locally on my laptop. Just like Docker Compose.
However, it seems I only have the option to also deploy all services at once which I am afraid defies the initial purpose of having a micro services architecture.
I would like to be able to deploy/version each service independently to AWS. What would be the best way to achieve that while keeping infrastructure management to a minimum?
We are currently using Amazon ECS to accomplish exactly what you are talking about trying to achieve. You can define your Docker Container as a Task definition and then Create an ECS Service which will handle number of instances, scaling, etc.
One thing to note is Amazon mentions the word container a lot in the documentation. They may be talking about the EC2 instance used for the cluster for your docker instances/containers.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 months ago.
The community reviewed whether to reopen this question 3 months ago and left it closed:
Not suitable for this site
Improve this question
I have developed a .NET MVC application and have started playing around with AWS and deploying it via the Visual Studio Toolkit. I have successfully deployed the application using the Elastic Beanstalk option in the toolkit.
As I was going over the tutorials for deploying .NET apps to AWS with the toolkit, I noticed there are tutorials for deploying with both Elastic Beanstalk and CloudFormation. What is the difference between these two?
From what I can tell, it seems like they both essentially are doing the same thing - making it easier to deploy your application to the AWS cloud (setting up EC2 instances, load balancer, auto-scaling, etc). I have tried reading up on them both, but I can't seem to get anything other than a bunch of buzz-words that sound like the same thing to me. I even found an FAQ on the AWS website that is supposed to answer this exact question, yet I don't really understand.
Should I be using one or the other? Both?
They're actually pretty different. Elastic Beanstalk is intended to make developers' lives easier. CloudFormation is intended to make systems engineers' lives easier.
Elastic Beanstalk is a PaaS-like layer on top of AWS's IaaS services which abstracts away the underlying EC2 instances, Elastic Load Balancers, auto-scaling groups, etc. This makes it a lot easier for developers, who don't want to be dealing with all the systems stuff, to get their application quickly deployed on AWS. It's very similar to other PaaS products such as Heroku, EngineYard, Google App Engine, etc. With Elastic Beanstalk, you don't need to understand how any of the underlying magic works.
CloudFormation, on the other hand, doesn't automatically do anything. It's simply a way to define all the resources needed for deployment in a huge JSON/YAML file. So a CloudFormation template might actually create two Elastic Beanstalk environments (production and staging), a couple of ElasticCache clusters, a DynamoDB table, and then the proper DNS in Route53. I then upload this template to AWS, walk away, and 45 minutes later everything is ready and waiting. Since it's just a plain-text JSON/YAML file, I can stick it in my source control which provides a great way to version my application deployments. It also ensures that I have a repeatable, "known good" configuration that I can quickly deploy in a different region.
For getting started quickly deploying a standard .NET web-application, Elastic Beanstalk is the right service for you.
AWS CloudFormation: "Template-Driven Provisioning"
AWS CloudFormation gives developers and systems administrators an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.
CloudFormation (CFn) is a lightweight, low-level abstraction over existing AWS APIs. Using a static JSON/YAML template document, you declare a set of Resources (such as an EC2 instance or an S3 bucket) that correspond to CRUD operations on the AWS APIs.
When you create a CloudFormation stack, CloudFormation calls the corresponding APIs to create the associated Resources, and when you delete a stack, CloudFormation calls the corresponding APIs to delete them. Most (but not all) AWS APIs are supported.
AWS Elastic Beanstalk: "Web Apps Made Easy"
AWS Elastic Beanstalk is an easy-to-use service for deploying and scaling web applications and services developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker on familiar servers such as Apache, Nginx, Passenger, and IIS.
You can simply upload your code and Elastic Beanstalk automatically handles the deployment, from capacity provisioning, load balancing, auto-scaling to application health monitoring.
Elastic Beanstalk (EB) is a higher-level, managed 'platform as a service' (PaaS) for hosting web applications, similar in scope to Heroku. Rather than deal with low-level AWS resources directly, EB provides a fully-managed platform where you create an application environment using a web interface, select which platform your application uses, create and upload a source bundle, and EB handles the rest.
Using EB, you get all sorts of built-in features for monitoring your application environment and deploying new versions of your application.
Under the hood, EB uses CloudFormation to create and manage the application's various AWS resources. You can customize and extend the default EB environment by adding CloudFormation Resources to an EB configuration file deployed with your application.
Conclusion
If your application is a standard web-tier application using one of Elastic Beanstalk's supported platforms, and you want easy-to-manage, highly-scalable hosting for your application, use Elastic Beanstalk.
If you:
Want to manage all of your application's AWS resources directly;
Want to manage or heavily customize your instance-provisioning or deployment process;
Need to use an application platform not supported by Elastic Beanstalk; or
Just don't want/need any of the higher-level Elastic Beanstalk features
then use CloudFormation directly and avoid the added configuration layer of Elastic Beanstalk.
Cloud Formation is a service that lets you deploy AWS services. You create a template file that describes which services you want. When you deploy that template, Cloud Formation creates the resources for you as a "package". All the resources you defined in your template are started and terminated together. Examples of types of resources that can be created with Cloud Formation are: S3, EC2 instances, AutoScaling, DynamoDb, etc. For EC2, Cloud Formation also gives you the ability to make use of "cfn-init" scripts; which can be used in conjunction with the template to boot strap your instances.
Elastic Beanstalk uses Cloud Formation templates and scipts to: 1. Create a Load Balancer and Auto Scaling Group, 2. Copy your code to S3, 3. Bootstrap an Ec2 instance to Download the code from S3 and deploy it.
Cloud Formation is not as easy to use as EB, but it is much more powerful, because you can create resources other than EC2 instances, control how the cfn-init script, and etc.
There are other differences worth noting. Elastic beanstalk is designed as a container for a single app. I've a set of several websites and services but found it very difficult to deploy multiple websites with beanstalk and was advised, after several attempts, by AWS help to use cloud formation in this situation as it has the extra flexibility.
Theres a really helpful article on bootstrapping AWS cloud formation and updating a running site here thats much clearer than the AWS pages. Still trying to work out if we can deploy from VS straight to the cloud formation template stored on S3 and get it to auto update like beanstalk...
These services are designed to complement each other. AWS Elastic Beanstalk provides an environment to easily deploy and run applications in the cloud. It is integrated with developer tools and provides a one-stop experience for you to manage the lifecycle of your applications. AWS CloudFormation is a convenient provisioning mechanism for a broad range of AWS and third party resources. It supports the infrastructure needs of many different types of applications such as existing enterprise applications, legacy applications, applications built using a variety of AWS resources and container-based solutions (including those built using AWS Elastic Beanstalk).
AWS CloudFormation supports Elastic Beanstalk application environments as one of the AWS resource types. This allows you, for example, to create and manage an AWS Elastic Beanstalk–hosted application along with an RDS database to store the application data. In addition to RDS instances, any other supported AWS resource can be added to the group as well.
Both are for provisioning infrastructure; but they differ in their approach.
Beanstalk: The starting point is the code. I have a NodeJs code I want to upload & run it; please provision the infrastructure for me. (PaaS) Platform as a Service
CloudFormation: The starting point is the infrastructure. Please create an EC2 instance, with one LoadBalancer, Security Group etc so that I can uploaded my NodeJs code to it. Infrastructure as Code (IaC).
Elastic Beanstalk automatically handles the deployment, from capacity provisioning, load balancing, auto-scaling to application health monitoring based on the code you upload to it, where as CloudFormation is an automated provisioning engine designed to deploy entire cloud environments via a JSON script.
Beanstalk: Gives the developer the ability to manage only code and not systems
Cloud Formation: Simplifies and makes everything easier for a Systems Engineer
If a developer or the dev team is looking for a quick MVP testing, the best option is to quickly get deployed with Beanstalk and check.
When a AWS migration happens, systems engineer will get involved in provisioning and Cloud Formation will help a lot and give much more granular control.
Beanstack internally uses cloudformation.
Beanstalk - Basically helpful for software developers.
Example : You want to start the PC quickly and run an application. You don't buy the PC items (harddisk, ram, Processor) separately. You buy a whole CPU or a laptop of a required config. You dont care how its running inside as you want your application to run for you. Beanstalk gives you this feature of everything ready made with no worries.
Cloudformation - Basically helpful for system engineer/ Hardware.
Example : You want to assemble 100's of PC's and give it to the developers then instead of assembling so many PC's you can just give a list of items and the PC is assembled for you by the retailer.
Similarly create a template and send it to cloudformation it will finish your work with no effort.