I have gone through the cloudbreak documentation and I am still not sure what is the exact purpose of this component.
Is it actually useful only for deploying the cluster in any cloud services and if so can we customise the components that needs to be installed in the cluster.
If it is only for maintaining the deployment of a cluster then is there any cost involved in using cloudbreak?
Cloudbreak main purpose is Hdp or Hdf cluster management. It provides an UI and api to access, create and edit the cluster. It also provides access control management for the clusters. Yes you can customize the components installation via ambari blueprint.
One additional benefit was from its component periscope, which provides autoscaling based on ambari alerts.
Related
I want to migrate Mule applications deployed on Mule standalone (on-Premise) to Anypoint Runtime Fabric (RTF) Self managed Kubernetes on AWS, but I could not find any document on this.
Any ideas or any document available on this please share it.
Thanks in advance
Mule applications run exactly the same on-prem, on CloudHub or in Anypoint Runtime Fabric. It is only if your applications make assumptions about their environment that you are going to need to make adjustments. For example any access to the filesystem (reading a file from some directory) or some network access that is not replicated to the Kubernetes cluster. A common mistake is when developers use Windows as the development environment and are not aware that the execution in a container environment will be different. You may not be aware of those assumptions. Just test the application and see if there are any issues. It is possible it will run fine.
The one exception is if the applications share configurations and/or libraries through domains. Since applications in Runtime Fabric are self isolated, domains are not supported. You need to include the configurations into each separate applications. For example you can not have an HTTP Listener config where several applications share the same TCP Port to listen to incoming requests. That should be replaced by using Runtime Fabric inbound configurations.
About the deployment, when you deploy to a new deployment model, it is considered a completely new application, with no relationship to the previous one. There is no "migration" of deployments. You can deploy using Runtime Manager or Maven. See the documentation. Note that the documentation states that to deploy with Maven you first must publish the application to Exchange.
Yes, you can.
In general, it is an easy exercise. However, things may go a little bit complicated when you have lots of dependencies on the persistent object store. It may require slight code refactoring in the worst case scenario. If you are running on-prem in cluster mode, then you are using HazelCast which is also available in RTF.
Choosing the Self-managed Kubernetes in EKS have some extra responsibilities. If you and your team have good expertise on Kubernetes and AWS then it is a great choice. Keep in mind that the Anypoint runtime console allows at most 8 replicas for each app. However, if you are using CI/CD pipeline, you should be able to scale it more.
There is no straightforward documentation as the majority of work is related to setup your EKS and associated network, ports, ingress, etc.
Can we run an application that is configured to run on multi-node AWS EC2 K8s cluster using kops (project link) into local Kubernetes cluster (setup using kubeadm)?
My thinking is that if the application runs in k8s cluster based on AWS EC2 instances, it should also run in local k8s cluster as well. I am trying it locally for testing purposes.
Heres what I have tried so far but it is not working.
First I set up my local 2-node cluster using kubeadm
Then I modified the installation script of the project (link given above) by removing all the references to EC2 (as I am using local machines) and kops (particularly in their create_cluster.py script) state.
I have modified their application yaml files (app requirements) to meet my localsetup (2-node)
Unfortunately, although most of the application pods are created and in running state, some other application pods are unable to create and therefore, I am not being able to run the whole application on my local cluster.
I appreciate your help.
It is the beauty of Docker and Kubernetes. It helps to keep your development environment to match production. For simple applications, written without custom resources, you can deploy the same workload to any cluster running on any cloud provider.
However, the ability to deploy the same workload to different clusters depends on some factors, like,
How you manage authorization and authentication in your cluster? for example, IAM, IRSA..
Are you using any cloud native custom resources - ex, AWS ALBs used as LoadBalancer Services
Are you using any cloud native storage - ex, your pods rely on EFS/EBS volumes
Is your application cloud agonistic - ex using native technologies like Neptune
Can you mock cloud technologies in your local - ex. Using local stack to mock Kinesis, Dynamo
How you resolve DNS routes - ex, Say you are using RDS n AWS. You can access it using a route53 entry. In local you might be running a mysql instance and you need a DNS mechanism to discover that instance.
I did a google search and looked at the documentation of kOps. I could not find any info about how to deploy to local, and it only supports public cloud providers.
IMO, you need to figure out a way to set up your local EKS cluster, and if there are any usage of cloud native technologies, you need to figure out an alternative way about doing the same in your local.
The true answer, as Rajan Panneer Selvam said in his response, is that it depends, but I'd like to expand somewhat on his answer by saying that your application should run on any K8S cluster given that it provides the services that the application consumes. What you're doing is considered good practice to ensure that your application is portable, which is always a factor in non-trivial applications where simply upgrading a downstream service could be considered a change of environment/platform requiring portability (platform-independence).
To help you achieve this, you should be developing a 12-Factor Application (12-FA) or one of its more up-to-date derivatives (12-FA is getting a little dated now and many variations have been suggested, but mostly they're all good).
For example, if your application uses a database then it should use DB independent SQL or no-sql so that you can switch it out. In production, you may run on Oracle, but in your local environment you may use MySQL: your application should not care. The credentials and connection string should be passed to the application via the usual K8S techniques of secrets and config-maps to help you achieve this. And all logging should be sent to stdout (and stderr) so that you can use a log-shipping agent to send the logs somewhere more useful than a local filesystem.
If you run your app locally then you have to provide a surrogate for every 'platform' service that is provided in production, and this may mean switching out major components of what you consider to be your application but this is ok, it is meant to happen. You provide a platform that provides services to your application-layer. Switching from EC2 to local may mean reconfiguring the ingress controller to work without the ELB, or it may mean configuring kubernetes secrets to use local-storage for dev creds rather than AWS KMS. It may mean reconfiguring your persistent volume classes to use local storage rather than EBS. All of this is expected and right.
What you should not have to do is start editing microservices to work in the new environment. If you find yourself doing that then the application has made a factoring and layering error. Platform services should be provided to a set of microservices that use them, the microservices should not be aware of the implementation details of these services.
Of course, it is possible that you have some non-portable code in your system, for example, you may be using some Oracle-specific PL/SQL that can't be run elsewhere. This code should be extracted to config files and equivalents provided for each database you wish to run on. This isn't always possible, in which case you should abstract as much as possible into isolated services and you'll have to reimplement only those services on each new platform, which could still be time-consuming, but ultimately worth the effort for most non-trival systems.
There is a project migrated from legacy to GCP.
On GCP everything runs on microservices.
May be around 40-50 microservice.
I would like to automate this microservices but there is no endpoint exposed in this project.
How could you automate a microservice where there are no endpoints?
What type of architecture, you could use to test this?
Db: Firestore (nosql)
Thanks
M
In my view you can do it following way:
Use ClusterIP or NodePort to access those POD.
Spin up a new POD which will access your target POD to communicate.
You can enable POD to POD communication based on labels and enabling network policy.
You can use calico as network policy agent.
You can view the log of your testing pod using kubectl logs [pod name] or from logging service of your cloud provider or even using daemonset that you could install.
The testing POD can periodically send traffic. So you can use thread to call the target service and keep the thread in sleep mode for a while or you can use kubernetes cronjob to call the target service. Based on your usecase it will be chosen.
Let me know if it meets your requirements or you have more to elaborate?
In terms of finding out how to test your micro-services on the Google Cloud Platform, I would suggest referencing our documentation on "Microservices Architecture on Google App Engine" as it will explain and guild you how to implement your services onto GCP. You may also look into this document as well as it provides the best practices for designing APIs to communicate between microservices.
Additionally, user "ARINDAM BANERJEE" has a great example you can follow as well.
WSO2 recommends having Puppet & SVN as part of the API Manager architecture. But I have not found documentation that clearly states what this is required for. If someone can help me here with the actual reason for these tools that would be great. Can I use other tools that perform the same functions?
In previous versions, we had SVN based deployment synchronizer mechanism to share API artifacts between gateway nodes. Now we recommend and encourage to use a shared file system between gateway nodes. Another option is to use Rsync between gateway nodes.
https://docs.wso2.com/display/AM260/Distributed+Deployment+of+the+Gateway
Puppet is just a configuration management tool. This is not mandatory and you can use puppet to manage your configuration in a cluster. With Puppet you can automate your deployment creation and it would allow you to easily manage several clusters.
I wanted to get List of containers and their details running on GCP Kubernets,
From API page https://developers.google.com/apis-explorer/#p/container/v1/
We could get Cluster and Node details, but I'm looking for more granular levels like Pods and Container.
Is there any way to get those.
Pod and container details are accessible through the kubernetes API, not using the Google Cloud SDK.
Unfortunately, this means getting that information on a per cluster basis.
Alternatively, the Cloud Console can be used by going to Kubernetes Engine > Workloads which will list all replica sets and individual pods (if not controlled by a replicaset).
You might be able to configure Stackdriver Monitoring to create a group consisting of all the pod and container resources in use in the project and then use the Monitoring API to call on that group, but I haven't tested that.