We have a spring boot application running in physical boxes. Planning to migrate to EKS (AWS). We have hazelcast used for multiple purposes listed below.
Is it possible to use hazelcast itself in AWS to make use of the same features.?
Or should I used any other technology than using hazelcast itself ?
Hazelcast is used for:
1. Master Election (of machines)
2. Caching (To keep some files sometimes and mainly to websocket messages and to transmit those whenever required)
Master Election :
It is used to pick any one machine out of 4 boxes, to do a particular job until next restart/re-deploy.
Hazelcast version used : com.hazelcast.hazelcast : 3.4.2
Reference : what algorithm hazelcast uses to find the master node
Note : Previously I mentioned as leader (actually it's master in the case of Hazelcast)
A. You need to upgrade to a more recent version, preferably 3.12.5, as the later versions have more capabilities in terms of features than their predecessors.
B. From what I could understand, you need a distributed lock so that when acquired, no other application or thread or member can perform the operation defined within the lock boundary. Check out here: https://docs.hazelcast.org/docs/3.12.5/manual/html-single/index.html#lock
Using Hazelcast as a distributed system for its features is agnostic to the underlying platform, EKS or AWS direct or on-prem. Once a cluster is formed, you may want to look at Hazelcast as a service with all the distributed features listed in ref manual.
For caching, definitely yes. A lot of folks use it that way. Hazelcast is very well integrated with all Kubernetes environments.
For leader election, I'm not sure I understand your use case and what you'd like to do in EKS. In Kubernetes, you usually focus more on the container/pod level, not the machine itself. So, to keep the leader election between pods, then yes, you can use Hazelcast, not problem with that.
Related
On joining a business network the node want to get the cordapp installed.How is the procedure?Is there any existing flow for this purpose or can we create a custom flow for this purpose?
It depends on how the network is managed. Also in the Corda Network documentation here (step 7) they do not mention a standard way to do it, it is based on the agreement among parties:
Installation and configuration of your Corda applications must be undertaken by the node operator. [...] Specifics on application usage or installation should be available from your CorDapp provider.
Technically, you can find an interesting reference implementation of a CorDapp Distribution Service here to manage the distribution and management of the updates:
targets scenarios when Business Network members maintain their nodes
by themselves and can decide on what version of a CorDapp they would
like to upgrade to. This design does not target scenarios where
upgrade function is centralised and the whole network can be shut down
to perform stop-upgrade-restart.
My company is currently evaluating hyperledger(fabric) and we're using it for our POC. It looks very promising and we're targeting rolling out to production in next few months.
We're targeting AWS as our production environment.
However, we're struggling to find good tutorial/practices/recommendations about operating hyperledger network in such environment.
I'm aware that Cello is aiming to solve/ease deploying/monitoring hyperledger network but i also read that its not production ready yet. Question is, should we even consider looking at Cello at this point?
If not, what are our alternatives? Docker swarm, kubernetes?
I also didn't find information about recommended instance types. I understand this is application and AWS specific but what are the minimal system requirements
(memory&CPU&network) for example for 'peer' node (our application is not network intensive, nor a lot of transactions will be submitted per hour/day, only few of them per day).
Another question is where to create those instances on AWS from geographical&decentralization point of view. Does it make sense all of them to be created in same region? Or, we must create instances running in different regions?
Tnx a lot.
Igor.
yes, look at Cello.. if nothing else it will help you see the aws deployment model.
really nothing special..
design the desired system, peers, orderer, gateways, etc..
then decide who many ec2 instance u need to support that.
as for WHERE (region).. depends on where the connecting application is and what kind of fault tolerance you need for your business model.
one of the businesses I am working with wants a minimum of 99.99999 % availability. so, multi-region is critical. its just another ec2 instance with sockets open from different hosts..
aws doesn't provide much in terms of support for hyperledger. they have some templates which allow you to setup the VMs initially, but that's stuff you can do yourself as well.
you are right, the documentation is very light and most of the time confusing. I got to the point where I can start from scratch with a brand new VM and got everything ready and deploy my own network definition and chaincode and have the scripts to do that.
IBM cloud has much better support for hyperledger however. you can design your network visually, you can download your connection profiles, deploy and instantiate chaincode, create and join channels, handle certificates, pretty much everything you need to run and support such a network. It's light years ahead of AWS. They even have a full CI / CD pipepline that you could replicate for your own project. if you look at their marbles demo, you'll see what i mean.
Cello is definitely worth looking at, with the caveat that it's incubation meaning, not real yet, not production ready and not really useful until it becomes a fully fledged product.
I am using WSO2 APIM 1.10.0 on a single server deployment and would like to move to a clustering one. Looking at this documentation I could found a lot of information, howevre something is boring me; do I really have to always do all of it?
I mean, I don't want to split all my workers in multiple instances, all I want is configure two full setup configurations (key manager + publisher + store + gateway), each one on its own host and make sure I can put a load balance in front of it.
Thre requiremenst are simple: I would like to share the load on both of them, and guarantee a better availability in case of one of the hosts goes down. Is it a MUST break down the whole installation on both nodes so I have to start each component independently with offset ports configured?
I coud see that on version 2.0.0 a lot have been simplified, any way to reach the same on 1.10.0 one?
Regards
Splitting into profiles is not mandatory. This is designed in this way to scale API Manager based on the TPS. If you have a low TPS count and prefer to have 2 node HA setup, you can do the following.
Cluster the two nodes using wka, aws, etc.
Use dep-sync to share API artifacts between two nodes.
Use one node as the Publisher. You need to handle the publisher node traffic using single node. This is to avoid getting SVN conflicts.
You can serve API requests from both nodes.
You do not want to always use the same deployment pattern mentioned in the docuemtnation that you have pointed there. There are various Other deployment patterns that you can use according to the scalability and the requirement of yours.
Please refer the following documentation [1] for different deployment patterns you can use for WSO2 API Manager and [2] for more information on worker Manager separation and Load balancing.
[1] https://docs.wso2.com/display/CLUSTER44x/API+Manager+Deployment+Patterns
[2] https://docs.wso2.com/display/CLUSTER44x/Separating+the+Worker+and+Manager+Nodes
I have a Tomcat web app running on AWS. I would like to have several instances of it instead of only one, mainly to avoid down-time in case of problems on one instance. I need a concept of a "leader instance" because some operations should not be carried out by all instances but only one of them. Does Spring Cloud support leadership election and quorum out of the box?
P.S. I would like to avoid the obvious ZooKeeper, if possible.
There is some preliminary work going on in spring-cloud-cluster. It is not released (or supported) yet. You have your choice of implementation, currently: zookeeper, redis or hazelcast. Other implementations shouldn't be hard to do either.
I'm trying to install and configure a highly availability setup for the WSO2 API Manager. I've been reading through this document: http://docs.wso2.org/wiki/display/Cluster/Clustering+API+Manager and in there it explains to break up the 4 components of the application into separate folders and that these 4 components can run on a single server. I'm not sure why this is needed. All I really want to do is take 2 servers, install the full application on both of them (without breaking the application up into 4 different pieces) and cluster them together between two servers with an Elastic Load Balancer in front of them.
What is the purpose of splitting up the multiple components on the same server if they all run out of a single installation? I'm looking for the simplest way to provide fail over capability to this application if one server goes down. Any insight into their methodology would be greatly appreciated.
Thanks.
The article you've linked describes on distributing different components of API Manager. If you look at the very end of that article there's a link to clustering configuration doc. In a production deployment usually it is encouraged that the 4 components are run on different nodes rather than having everything in a node and having multiple such nodes. That's why it goes on explaining breaking it down to separate components. The official AM doc below has a page on different deployment patterns.
You can go through the following articles to get a better understanding on clustering API Manager.
http://docs.wso2.org/wiki/display/AM140/Clustered+Deployment
http://sanjeewamalalgoda.blogspot.com/2012/09/how-do-clustering-and-enable-replicate.html
My 2cts:
The documentation mentioned in the remarks, explains how WSO2 sees the world of clustering. Spread the different functionality over different JVM's. This sounds logical from architectural point of view. A dis-advantages is that the diffent applications need to me administrated as well by operations. This makes the technical architecture rather complex.
In our situation, we defined 2 different servers with extra CPU and memory, on these servers we have installed the full WSO2 API Manager and defined the cluster configuration. Everything provisioned via Puppet.
Just a straightforward install, all data-source pointing to one schema in an Oracle database.
And...it is working; Our Developers happy, Operations happy, Architect department happy