I'm trying to write up a tool that requires knowledge of the state of other machines in a cluster (local LAN). This is for a network failover/high availability system similar to VRRP and corosync/openais, but I wish to contain more information (such as near real-time speed/performance characteristics) so devices can make more intelligent choices. This means using a protocol more complicated than a predetermine weight-based mechanism: by allowing all clustered machines to see the state of each other, they can communally agree on which is the most suitable to be the master device.
From my searches, I haven't found any (C, C++ or JavaME) libraries that offer a distributed state mechanism. Ideally, I'm looking for something that broadcasts/multicasts each individual machines state periodically so participating machines can build up a global state table and all can see who the master should be. State in this case is arbitrary key/value pairs.
I'd rather not re-invent any wheels so am curious to know if anyone here can point me in the right direction?
If I were you I'd investigate memcached (memcached.org) or one of the nosql variants.
It sounds like Apache ZooKeeper might be a good match. It's distributed, hierarchical key-value store. To quote their Overview page:
ZooKeeper was designed to store coordination data: status information, configuration, location information, etc.
Here's an example of a simple Leader Election recipie, although it would require adaptation to determine a leader by some weighted criterion.
I'm not sure if there is any application for your purpose or not.
But I know that you can write a simple program with MPI library and broadcast any information that you want.
all client's can send their state to root node, and the root node then broadcast the message.
functions that you need for this are:
MPI_Bcast
MPI_Send
MPI_Recv
there is lots of tutorial on C++/MPI on net, just google it!
Related
I'm working to design a middle layer for an application that will receive up to ~5000 requests every few seconds and need to retrieve information from a database. I've been looking at use the Play Framework (I use scala for my REST api design) as they say its fully async and built on Akka. However, the main bottleneck of any solution seems to happen during read/writes to the database. Many Database cannot support simultaneous read/writes from a database of such a scale. How is such high concurrency achieved then for an app like this? I would guess Facebook/Twitter/ (name other big company) may have achieved this for their Applications as millions of people may be using them concurrently.
As Tim's comment was saying caching may or may not be able to help in your case. If not I would also recommend looking into horizontally scalable databases, for example cockroachdb if you want a transactional SQL db. Otherwise there are many no-sql choices a la mongodb etc. And if you really want to stick to traditional SQL systems you'll have to vertically scale your servers (buy the most expensive hardware) and work with read-replicas.
A huge component is your data model and query access pattern. If each query is incrementing a shared counter that has to be synchronized there will be a ton of contention, but if each query is touch completely separate data on the other end the spectrum than there will be a lot less contention.
I think there are a couple of dimensions I would consider:
Data Schema and Access Patterns (discussed above)
Language Choice
This is important becaues if you were in a web server context and were using prefork by default each process may have its own connection to the database. In an environment like python or ruby you may need hundreds of processes to handle your load. Contrast this with akka or another async networking based runtime (node, python gevent/asyncio, go, etc) where a single instance with a small thread pool can handle a large number of requests. Each have their tradeoffs.
Distributed Systems
Depending on your data schema and access patterns 5000 requests per second to a RDBMS is completely achievable. It would probably require relatively beefy hardware but but I'v personally done it a number of times. Getting to larger scales requires more computers in order to distribute the work/load. If your workload is right heavy and you can support potentially stale reads, a read replica is one option. With another machine in the mix reads are distributed over 2 machines but writes are still directed at a single machine (leader). Caching is another option.
At much higher workloads some sort of partitioning needs to occur in order to overcome the constraints of a single machine. https://github.com/vitessio/vitess
Many of the big contenders have solutions to horizontally scaling their databases. This has many drawbacks as well and will require careful planning.
The one thing I'd recommend is that if 5000 requests per second is projected for the near future, start with the minimal amount of hardware necessary (single instance) query patterns and operation get exponentially more complicated with a distributed database.
I have developed my application with Akka on a single JVM. Now I want to distribute the workload across many machines. I've started to read the documentation and got confused.
There are two ways of making Akka app to be distributed by clustering and remoting.
I don't get the difference between the two.
If I understand properly both are excluding themselves mutually since in configuration one need to use different provider for actor reference:
akka.remote.RemoteActorRefProvider
akka.cluster.ClusterActorRefProvider
So what are the use cases? When I would choose one instead of the other?
Maybe clustering is something like superset of remoting or maybe it is the other way around?
They are not mutually exclusive since clustering is implemented on top of remoting. The main feature of remoting is location transparency for ActorRefs. Clustering adds distributed membership on top of that. It is rarely useful to use remoting directly, clustering is preferred for most of the use cases.
Always best to look to the code
https://github.com/akka/akka/blob/c2983c7225eeaa035af99e0affeb5f5c841668c4/akka-cluster/src/main/scala/akka/cluster/ClusterActorRefProvider.scala
private[akka] class ClusterActorRefProvider ... extends RemoteActorRefProvider
First of all ClusterActorRefProvider is just extension of the RemoteActorRefProvider
Core features of the cluster-akka added to the remoting are:
cluster wide failure detection
routers with rootes group & pool capabilities
you'll find flavour of all this addons in the source code.
Since there's no complete BPM framework/solution in ColdFusion as of yet, how would you model a workflow into a ColdFusion app that can be easily extensible and maintainable?
A business workflow is more then a flowchart that maps nicely into a programming language. For example:
How do you model a task X that follows by multiple tasks Y0,Y1,Y2 that happen in parallel, where Y0 is a human process (need to wait for inputs) and Y1 is a web service that might go wrong and might need auto retry, and Y2 is an automated process; follows by a task Z that only should be carried out when all Y's are completed?
My thoughts...
Seems like I need to do a whole lot of storing / managing / keeping
track of states, and frequent checking with cfscheuler.
cfthread ain't going to help much since some tasks can take days
(e.g. wait for user's confirmation).
I can already image the flow is going to be spread around in multiple UDFs,
DB, and CFCs
any opensource workflow engine in other language that maybe we can port over to CF?
Thank you for your brain power. :)
Study the Java Process Definition Language specification where JBoss has an execution engine for it. Using this Java based engine may be your easiest solution, and it solves many of the problems you've outlined.
If you intend to write your own, you will probably end up modelling states and transitions, vertices and edges in a directed graph. And this as Ciaran Archer wrote are the components of a State Machine. The best persistence approach IMO is capturing versions of whatever data is being sent through workflow via serialization, capturing the current state, and a history of transitions between states and changes to that data. The mechanism probably needs a way to keep track of who or what has responsibility for taking the next action against that workflow.
Based on your question, one thing to consider is whether or not you really need to represent parallel tasks in your solution. Where instead it might be possible to en-queue a set of messages and then specify a wait state for all of those to complete. Representing actual parallelism implies you are moving data simultaneously through several different processes. In which case when they join again you need an algorithm to resolve deltas, which is very much a non trivial task.
In the context of ColdFusion and what you're trying to accomplish, a scheduled task may be necessary if the system you're writing needs to poll other systems. Consider WDDX as a serialization format. JSON, while seductively simple, I recall has some edge cases around numbers and dates that can cause you grief.
Finally see my answer to this question for some additional thoughts.
Off the top of my head I'm thinking about the State design pattern with state persisted to a database. Check out the Head First Design Patterns's Gumball Machine example.
Generally this will work if you have something (like a client / order / etc.) going through a number of changes of state.
Different things will happen to your object depending on what state you are in, and that might mean sitting in a database table waiting for a flag to be updated by a user manually.
In terms of other languages I know Grails has a workflow module available. I don't know if you would be better off porting to CF or jumping ship to Grails (right tool for the job and all that).
It's just a thought, hope it helps.
i have a distributed application. here are set of processes , spread accross mutiple computers , communicating each other. i have a data structure , which is modified among these proceses . and this is not stored in database .
Now the question is how do i maintain the same view of the this data structure , accross all processes
i.e., at any point of time all process should see the same data structure
You say that you don't have a database. That's a shame, because database authors have solved your problem. You would need to incorporate the equivalent technology in your project. And obviously, the fastest and most simple way to incorporate the technology of databases is to incorporate a database.
Redis is designed to solve your problem. It is a key-value store for sharing between programs running on different machines but sharing the data. It is a server you run somewhere, and your programs all connect to this server using the client library it provides.
You can also use a database such as mysql but with in-memory tables.
If your data-structure does not fit into the key-value or relational models very well, you have the same kind of situation as multi-player games. It is non-trivial to sync multi-player games but it can be done and here is an excellent introduction as to how: gafferongames.com
I would recommend something like the Data Distribution Services platform for something like this (open source version is OpenDDS). Their key selling point is that it is designed to propagate changes to data to all interested in such changes. And performance isn't bad either.
Commercial implementations of this protocol are used in a variety of real-time systems, mostly military grade applications.
More options to consider, distributed caches (such as memcached) - though I've not played with this myself - it looks quite straight forward to get up and running.
My goal is to measure the average Bytes received per second using the Windows performance counter API. My problem is that, when there are multiple network adapters, I don't know which one is the "primary" one (i.e. the one used for Internet and LAN traffic).
I can pass "\Network Interface(*)\Bytes Received/sec" to PdhExpandWildCardPath() and get a list back, but that list seems to be in alphabetical order, not routing order. And the names returned by PdhExpandWildCardPath() don't seem to be the same as the ones the WMI functions use -- Pdh seems to add "Packet Scheduler Miniport" -- so even if I went that route I'm not sure how to correlate the information from the different APIs.
Which network adapter is 'Primary' is a qualitative judgment. A computer might have both onboard wireless 802.11, and wired Ethernet adapters, either of which could be primary at any given time (or even at the same time).
You are probably going to have to use the IP Helper API to determine which you prefer and correlate that with the PerfMon ouput. or you could just pick the one with the most activity and call it primary. That's just the kind of value judgment that PerfMon cannot possibly make, as these stats are transparent to it.