Does AWS Elastic Kubernetes Service have the same concept as Google Kubernetes Engine apps via a marketplace? I'm looking to deploy RabbitMQ and previously accomplished this on GKE.
Otherwise, it looks like there's a helm chart or I can manually do this via the container on dockerhub.
No, you need top deploy it by your own.
AWS does have a marketplace where you can utilize various AMI's or deploy a certified Bitnami RabbitMQ image. You can see that for yourself here: https://aws.amazon.com/marketplace
The downside is that this isn't something available for AWS EKS and as a result we will have to install/maintain this ourselves. That could look something like using the stable/rabbitmq-ha chart with anti-affinity across AZ's, quorum queues, and EBS.
Learn more about helm here:
https://helm.sh/docs/intro/using_helm/
Learn more about the rabbitmq helm chart here: https://hub.helm.sh/charts/stable/rabbitmq-ha
Related
I want to know a brief explanation or an example of how to migrate a Kubernetes application to GCP from AWS.
What services are implicated like EKS or EC2 and GKE or Compute Engine.
I'm very new to migration, I don't know too much about AWS and I recently started using GCP.
Thanks in advance.
It depends.
At first, AWS -> GCP resources mapping:
At first, you'll want to know the mapping between AWS and GCP resources.
There are several articles:
Cloud Services Mapping For AWS, Azure, GCP ,OCI, IBM and Alibaba provider – Technology Geek
Cloud Terminology Glossary for AWS, Azure, and GCP | Lucidchart:
Cloud Services Terminology Guide: Comparing AWS vs Azure vs Google | CloudHealth by VMware
Migrate AWS EKS to GCP GKE: the hard way
If your cluster is deployed with managed kubernetes service:
from Elastic Kubernetes Service (EKS)
to Google Kubernetes Engine (GKE)
Then it would be hard to migrate. Just due to complexity of kubernetes architecture and differences in the approaches of manage cluster in AWS vs GCP`
Migrate VMs and cluster deployed using your own k8s manifest.
If your kubernetes cluster is deployed on cloud virtual machines with k8s or helm manifests, then it would be easier.
And there are two ways:
Either migrate VMs using GCP Migrate Connector (as #vicente-ayala said in his answer)
Or import your infrastructure to the terraform manifest, change resources definitions step-by-step, and then apply this updated manifest to GCP
Migrating with Migrate Connector
You can found the latest migration manual on migrating VM's here:
Prerequisites
As per GCP manual,
Before you can migrate a source VM to Google Cloud, you must configure the migration environment on your on-premises data center and on Google Cloud. See:
Enabling Migrate for Compute Engine services
Installing the Migrate Connector
Migrating
How-to Guides | Migrate for Compute Engine | Google Cloud
Migrating individual VMs
Migrating VM groups
Migrating using Terraform and Terraformer
There is a great tool for reverse Terraform GoogleCloudPlatform/terraformer. Infrastructure to Code
A CLI tool that generates tf/json and tfstate files based on existing infrastructure (reverse Terraform).
And you can import your infrastructure into terraform manifest:
terraformer import aws --resources=vpc,subnet --connect=true --regions=eu-west-1 --profile=prod
You'll get the terraform manifest declared with aws provider
And you may try to replace every AWS resource to the appropriate GCP resource. There is official terraform GCP provider: hashicorp/google. Unfortunately, there isn't mapping for terraform resources of both cloud providers. But, again, you may some of these mapping lists:
Cloud Services Mapping For AWS, Azure, GCP ,OCI, IBM and Alibaba provider – Technology Geek
Cloud Terminology Glossary for AWS, Azure, and GCP | Lucidchart:
Cloud Services Terminology Guide: Comparing AWS vs Azure vs Google | CloudHealth by VMware
And then apply the new GCP manifest:
terraform init
terraform plan
terraform apply
Additional resources on AWS <-> GCP
GCP to AWS Migration: Why and How to Make the Move
GCP | Google Cloud Migrate for Compute Engine | AWS to GCP Migration using Velostrata - YouTube
Managing a Large and Complex GCP Migration (Cloud Next '19) - YouTube
Lessons Learned Migrating from GCP to AWS | Leverege
How to approach a GCP-to-AWS migration
How Rapyder helped ride-sharing app migrate from GCP to AWS | Rapyder
Cloud Migration Use Case: Moving From AWS to GCP (PDF)
Here is a detailed guide of steps you need to perform to migrate you k8s cluster from AWS to GCP.
https://cloud.google.com/migrate/compute-engine/docs/4.8/how-to/migrate-aws-to-gcp/overview
https://cloud.google.com/migrate/compute-engine/docs/4.8/how-to/migrate-aws-to-gcp/aws-prerequisites
https://cloud.google.com/migrate/compute-engine/docs/4.8/how-to/migrate-aws-to-gcp/configure-aws-as-a-source
I used ECS Fargate and it provides containerization, auto-scaling based on request count, CPU and Memory.
It is working as expected.
I start to explore the AWS EKS feature and I didn't see any advantage in using this as all are provided by ECS Fargate.
Could someone help me understand where to use ECS Fargate and Where to use AWS EKS?
Anyhelp is appreciated.
Thanks,
Harry
You would use AWS EKS if you want to use Kubernetes.
Since Kubernetes is a standard, you could in theory move your application from AWS EKS to other cloud providers like Azure, Google Cloud, or DigitalOcean easily since they all support Kubernetes.
If you don't care about Kubernetes then I find that AWS ECS with the AWS Fargate [Serverless compute for containers] deployment type is currently the easiest method of running Docker containers on AWS.
Note that Amazon is actively working on adding the Fargate deployment type to the EKS service.
I would check back after the AWS re:invent conference next month to see how things have changed in this area.
We hear these questions often and I tried to capture some of the core principles of these comparisons/positioning in this blog post.
I'm looking at the possibly of replacing/moving our existing Apache Kafka set up (version 2.1.0) to Amazon's MSK and for it work on EKS.
I've been looking around to see if this is actually possible and if someone has done this or attempted it but so far I've only seen reference to using Apache Kafka on EKS. Does anyone know if it is possible/makes sense to use MSK on EKS?
Many thanks.
Amazon MSK provides fully-managed Kafka clusters, which means that from your side, you do not have to operate the cluster at all. Broker and Zookeeper nodes are packaged, deployed, created, updated and patched for you.
This step-by-step tutorial illustrates the creation of a cluster.
The answer is not, MSK is a fully managed service provided by AWS, you cannot install managed service :-) but you can run your own Kafka cluster on top of Kubernetes cluster in AWS, eg. on EKS service while installing a Kafka Operator:
https://banzaicloud.com/docs/supertubes/kafka-operator/
I haven’t done it for MSK before but surely done it for AWS Aurora Postgres. Not sure why you can’t define your external persistence (in this case MSK) as a service with no selector then manually register an Endpoint object pointing to the MSK host.
https://kubernetes.io/docs/concepts/services-networking/service/#services-without-selectors
What does AWS' Elastic Kubernetes Service (EKS) do exactly if so much configuration is needed in CloudFormation which is (yet) another AWS service?
I followed the AWS EKS Getting Started in the docs at (https://docs.aws.amazon.com/eks/latest/userguide/eks-ug.pdf) where it seems CloudFormation knowledge is heavily required to run EKS.
Am I mistaken or something?
So in addition to learning the Kubernetes .yaml manifest definitions, to run k8s on EKS, AWS expects you to learn their CloudFormation .yaml configuration manifests as well (which are all PascalCase as opposed to k8s' camelCase i might add)?
I understand that EKS does some management of latest version of k8s and control plane, and is "secure by default" but other than that?
Why wouldn't I just run k8s on AWS using kops then, and deal with the slightly outdated k8s versions?
Or am I supposed to do EKS + CloudFormation + kops at which point GKE looks like a really tempting alternative?
Update:
At this point I'm really thinking EKS is just a thin wrapper over CloudFormation after searching on EKS in detail and how it is so reliant on CloudFormation manifests.
Likely a business response to the alarming popularity of k8s, GKE in general with no substance to back the service.
Hopefully this helps save the time of anyone evaluating the half-baked service that is EKS.
To run Kubernetes on AWS you have basically 2 options:
using kops, it will create Master nodes + workers node under the hood, in plain EC2 machines
EKS + Cloudformation workers stack (you can use also Terraform as an alternative to deploy the workers, or eksctl, that will create both the EKS cluster and the workers. I recommend you to follow this workshop)
EKS alone provides only the master nodes of a kubernetes cluster, in a highly available setup. You still need to add the worker nodes, where your containers will be created.
I tried both kops and EKS + Workers, and I ended up using EKS, because I found it easier to setup and maintain and more fault-tolerant.
I feel the same difficulties earlier, and none of article could give me requirement in a glance for things that need to be done. Lot of people just recommend using eksctl which in my opinion will create a bloated and hard to manage kind of CloudFormation.
Basically both EKS is just a wrapper of Kubernetes, there's some points of integration between Kubernetes and AWS that still need to be done manually.
I've wrote an article that hope could help you understand all the process that need to be inplaces
EKS is the managed control plane for kubernetes , while Cloud-formation is a infrastructure templating service .
Instead of EKS you can run and manage the control plane(master nodes) on top of EC2 machines if you want to optimize for costs.For using EKS you have to pay for the underlying infra(EC2+networking..) and managed service fee(EKS price) .
Cloud-formation provides a nice interface to template and automate your infrastructure.You may use terraform in place of CF
I did not find any topic talking about eucalyptus and Kubernetes. And I find that quite weird because eucalyptus allow you to create an hybrid cloud with a private cloud A.W.S. compatible for the S3, EBS and EC2. And, so I was thinking with just Eucalytpus and Kubernetes, you can easily create an awesome hybrid cloud.
Does some experiments Kubernetes with Eucalyptus? If yes, how did you setup it? And, how works your private and public cloud (working together or are they independent)?
Eucalyptus does not yet officially support AWS ECS service, but there has been some work done on this area recently. For that you will have to install Eucalyptus from source using a custom git branch.
The link below explains how to get AWS ECS like service on Eucalyptus,
http://jeevanullas.in/blog/aws-ec2-container-service-api-in-eucalyptus/
We installed Kubernetes on top of Eucalyptus using Rancher. We had to modify the AWS Docker Machine driver to make it work automatically, but a few months ago the ability to use custom regions with Docker Machine so it should be in Rancher soon. You can also install Rancher hosts manually using the bootstrap script.
For the most part, it works quite well and the bonus with using Rancher is you get Swarm, Mesos, and Cattle orchestration engines which increases the types of container deployments. New versions of Rancher to be released will support EBS volumes and we plan to experiment that for container storage.
We haven't tried connecting private and public, but in theory it should work as long as the UDP ports are open required for Rancher networking. But I'm not even sure this is how container orchestration is designed to work multi-region so try with caution.