In my organization, I need to migrate the Kubernetes cluster from on-premise to AWS EKS. Can someone please tell me how I can do this?
There are many ways and things to consider however you have to ask a board questions so will write an answer accordingly.
You can use the tool velero to migrate the internal components of the Kubernetes cluster.
Velero : https://velero.io/
While other all things you have to consider of migrating Databases using DMS, Storage option like PV and PVC or NFS that used with on-prem K8s.
That’s quite broad question. It can have many approaches & flavours based on organisation needs.
One of the approach is to
Standup the eks cluster based on the needs of the workload you are going to run. Note that EKS can also be deployed in to flavours as managed nodes & other as serverless.
Setup the authentication & authorisation model for users & admins for eks cluster.
Defined strategy to how & when to migrate load to EKS.
Very important note is you must gather all NFRs for your cluster & create a plan to how to full fill those requirements before you start on the above.
Related
I'm trying to create an AWS EKS cluster with Pulumi and it seems two components exists:
#pulumi/eks providing a Cluster component
#pulumi/aws providing an eks/Cluster component
#pulumi/eks seems to be higher level but I cannot find a documentation specifying the concrete difference between those, and if one is preferred depending on use cases.
What's the difference between those two components?
#pulumi/eks/Cluster is a component resource that is built on top of #pulumi/aws/eks/Cluster and other resources to simplify provisioning of EKS clusters. Its goal is to make common scenarios achievable with a handful of lines of code, as opposed to the involved model of raw AWS resources.
You can find some usage examples in
AWS Crosswalk: AWS Elastic Kubernetes Service
Easily Create and Manage AWS EKS Kubernetes Clusters with Pulumi.
I suggest you start with #pulumi/eks and see if it works well for you.
If I want to set up a PostgreSQL-compatible database on AWS, I have 3 choices:
Manual: EC2 (spin up an EC2 and manually set up and maintain a PostgreSQL database on the EC2).
Managed: RDS-PostgreSQL (AWS will set up and maintain the database instances).
Fully managed: RDS Aurora in PostgreSQL-compatible mode (AWS will set up and maintain the database instances, just like with RDS-PostgreSQL?)
My question concerns the difference between "managed" and "fully managed". Many AWS certification training materials highlight the "fully managed" feature as an advantage that RDS Aurora in PostgreSQL-compatible mode has over RDS-PostgreSQL. I don't understand what the distinction is.
AWS documentation lists these items as the advantage of the "managed" feature of RDS Postgres: "hardware provisioning, database setup, patching and backups". To compare, these are the items that AWS documentation lists as the advantage of the "fully managed" feature of RDS Aurora: "hardware provisioning, software patching, setup, configuration, or backups". As far as I can tell, the only difference between these lists is "configuration". What am I missing?
Note that I am not asking about other differences between RDS Aurora and RDS Postgres. I'm specifically asking about the difference between "managed" and "fully managed".
Your concern is to understand fully managed and managed services.
Let me explain fully managed service with an example of DynamoDB, which is a fully managed service. AWS manages all infrastructure, and software updates, and at the end all you need to do is use the service and maybe set up some IAM permissions to access it.
While in case of managed services it works on the shared responsibility, in short you have more control on it, AWS do not manage at Infra level like security patching, updates, scaling etc.
Now the above explanation of managed and fully managed services can be linked with multiple AWS services.
Google Cloud Platform has made hybrid- and multi-cloud computing a reality through Anthos which is an open application modernization platform. How does Anthos work for distributed data platforms?
For example, I have my data in Teradata On-premise, AWS Redshift and Azure Snowflake. Can Anthos joins all datasets and allow users to query or perform reporting with low latency? What is the equivalent of GCP Anthos in AWS and Azure?
Your question is wide. Anthos is designed for managing and distributing container accross several K8S cluster.
For a simpler view, imagine this: you have the Anthos master, and its direct node are K8S masters. If you ask Anthos Master to deploy a pod on AWS for example. Anthos master forward the query to K8S master deployed on EKS, and your pod is deployed on AWS.
Now, rethink your question: what about the data? Nothing magic, if your data are shared across several clusters you have to federate them with a system designed for this. It's quite similar than with only one cluster and with data on different node.
Anyway, you point here the real next challenge of multi-cloud/hybrid deployment. Solutions will emerge from this empty space.
Finally your last point: Azure and AWS equivalent. There isn't.
The newest Azure ARC seems to be light: it only allow to manage VM out of Azure Platform with an agent on it. Nothing as manageable as Anthos. for example: You have 3 VM on GCP and you manage them with Azure ARC. You deployed on each an NGINX and you want to set up a loadbalancer in from of your 3 VM. I don't catch how you can do this with Azure ARC. With Anthos, it's simply a service exposition of K8S -> The Loadbalancer will be deployed according with the cloud platform implementation.
About AWS, outpost is an hardware solution: you have to buy AWS specific hardware and to plug it in your OnPrem infrastructure. Need more investment on prem in your move to cloud strategy? Hard to convince. And not compliant with other cloud provider. BUT ReInvent is coming next month. Maybe an outsider?
What does AWS' Elastic Kubernetes Service (EKS) do exactly if so much configuration is needed in CloudFormation which is (yet) another AWS service?
I followed the AWS EKS Getting Started in the docs at (https://docs.aws.amazon.com/eks/latest/userguide/eks-ug.pdf) where it seems CloudFormation knowledge is heavily required to run EKS.
Am I mistaken or something?
So in addition to learning the Kubernetes .yaml manifest definitions, to run k8s on EKS, AWS expects you to learn their CloudFormation .yaml configuration manifests as well (which are all PascalCase as opposed to k8s' camelCase i might add)?
I understand that EKS does some management of latest version of k8s and control plane, and is "secure by default" but other than that?
Why wouldn't I just run k8s on AWS using kops then, and deal with the slightly outdated k8s versions?
Or am I supposed to do EKS + CloudFormation + kops at which point GKE looks like a really tempting alternative?
Update:
At this point I'm really thinking EKS is just a thin wrapper over CloudFormation after searching on EKS in detail and how it is so reliant on CloudFormation manifests.
Likely a business response to the alarming popularity of k8s, GKE in general with no substance to back the service.
Hopefully this helps save the time of anyone evaluating the half-baked service that is EKS.
To run Kubernetes on AWS you have basically 2 options:
using kops, it will create Master nodes + workers node under the hood, in plain EC2 machines
EKS + Cloudformation workers stack (you can use also Terraform as an alternative to deploy the workers, or eksctl, that will create both the EKS cluster and the workers. I recommend you to follow this workshop)
EKS alone provides only the master nodes of a kubernetes cluster, in a highly available setup. You still need to add the worker nodes, where your containers will be created.
I tried both kops and EKS + Workers, and I ended up using EKS, because I found it easier to setup and maintain and more fault-tolerant.
I feel the same difficulties earlier, and none of article could give me requirement in a glance for things that need to be done. Lot of people just recommend using eksctl which in my opinion will create a bloated and hard to manage kind of CloudFormation.
Basically both EKS is just a wrapper of Kubernetes, there's some points of integration between Kubernetes and AWS that still need to be done manually.
I've wrote an article that hope could help you understand all the process that need to be inplaces
EKS is the managed control plane for kubernetes , while Cloud-formation is a infrastructure templating service .
Instead of EKS you can run and manage the control plane(master nodes) on top of EC2 machines if you want to optimize for costs.For using EKS you have to pay for the underlying infra(EC2+networking..) and managed service fee(EKS price) .
Cloud-formation provides a nice interface to template and automate your infrastructure.You may use terraform in place of CF
I came across an open source Kubernetes project KOPS and AWS Kubernetes service EKS. Both these products allow installation of a Kubernetes cluster. However, I wonder why one would pick EKS over KOPS or vice versa if one has not run any of them earlier.
This question does not ask which one is better, but rather asks for a comparison.
The two are largely the same, at the time of writing, the following are the differences I'm aware of between the 2 offerings
EKS:
Fully managed control plane from AWS - you have no control over the masters
AWS native authentication IAM authentication with the cluster
VPC level networking for pods meaning you can use things like security groups at the cluster/pod level
kops:
Support for more Kubernetes features, such as API server options
Auto provisioned nodes use the built in kops node_up tool
More flexibility over Kubernetes versions, EKS only has a few versions available right now
Other significant difference is that EKS is an AWS product so you require an AWS account but kops allows to run Kubernetes in AWS but also in GCE and DigitalOcean.