I'm a newbie, currently interested in data security & integrity.
I'm quite new to blockchain and distributed system theories, and suffering from some unclear doubts/questions on the fault-tolerant consensus.
May I ask for your kind advice on my dull thoughts regarding on the blockchain's true objective?
It would be a great help for me to step forward on understading better concept of consensus.
Here's the summary of what I understand (please correct me if I'm wrong)
In a synchronous network model, it is assumed that one can guarantee the message being delivered within a certain amount of time.
In an asynchronous network model, there is no certain guarantee on the message delivery.
In a design perspective, it is easier & more efficient to design a system based on the synchronous model.
Even the quorum size requirement can be reduced - synchronous model needs only f+1 votes while asynchronous model needs 2f+1 votes on the consensus.
(This is because that the synchronous model can eliminate the possiblity of message dropout (up to f), while the async model needs to consider both message dropout & possibly malicious messages.)
But in a distributed system based on multiple nodes, it is normally impossible to guarantee the message delivery since there is no central manager who can monitor all nodes whether each receives the message or not.
That is why most of the blockchain-oriented distributed ledgers (non-currency) are based on the asynchronous consensus schemes such as PBFT (Castro, Liskov 99).
In a private blockchain scenario, I believe that the main & final purpose of the consensus is to let all nodes hold a chain of blocks, where each block has a certain amount of agreements (i.e. more than f signatures).
So, based on the facts above, I got curious whether the fault-tolerance model only stands for the standard "client-server" environment.
What happens if we give up the client-server model, and let the client supersede peers' broadcast communications? (For a scenario where the client has enough computation power but is just short on storage, so it wants to manage multiple (but few, e.g. 3) replicas to store data via transactions)
To be more specific with a simple example (an authenticated environment with PKI):
Each replica only performs a "block creation (+signing)" and "block finalization (verifying signatures)" logic, and there may exist some malicious replicas (up to f) which try to spread different outputs, which is not originated from the client's transaction.
Let the client (a "data owner/manager" would be a more suitable term now...) visit all replicas, and it handles each replica as below:
Enforce that all replicas work as a synchronized state machine; assure all 3 replicas are synced up (check the latest block of each, and supplement the lagged ones)
Sets a callback (with a timeout) for each replica to guarantee the message delivery
Send a transaction to all replicas, and receive the block (with a signature) generated from the corresponding transaction (assume that the block contains only one transaction)
If more than f signatures (from replicas) for the same block message are collected, deliver the collected signatures to all replicas and order them to finalize the block.
If we do as above, instead of replicas making consensus on their own (i.e. no view-changes), can we still say that the system suffices a BFT model?
It seems that byzantine replicas cannot breach safety, since the client (and honest replicas) will only order/accept block finalization when the block comes with more than f signatures.
The concern is the liveness - can we say that the liveness still holds, since the system can continue (not a client-server model anymore, but the "system" goes on) because the client will ignore the faulty replica response and order honest replicas to finalize?
This curiosity is just stuck in my head, and it's blocking me from clearly understanding why the private blockchain systems need asynchronous consensus process among peers themselves.
(since they already trust the client transactions, and participants are designated by PKI & signatures)
Could anyone be kind and inform that whether my dominant-client example can still suffice the BFT, or something would go wrong?
Thank you so much!
Related
I know that a peer in a cryptocurrency network can contribute deciding the next block that has to be added to the blockchain. To do that and gain some rewards, such peer has to be the first peer able to resolve some PoW algorithm.
From what I have understood, mining pools use computational power of client machines in order to resolve the PoW as fast as possible.
I guess so that the mining pool server is the only peer that directly participates to the network and it performs entirely the algorithm using the computational power of the clients which perform only some secondary tasks.
How can be splitted this computational task to many clients?
Pool server receives "task" from a current coin node, by request getblocktemplate. Thereafter server, based on received tasks, prepares subtasks for participant miners, and also provide them another getblocktemplate strucutres, with reduced difficulty parameter. When miner solves subtask (with reduced difficulty), he sends his solution to a pool, this partial solution named a share. Pool computes participants contribution by number of submitted shares and shares difficulty.
Difficulty of some shares can be enough to comply coin network difficulty. Such share named solving share, and this is block solution. As result, this solving share added to blockchain as a block, and pool receives block reward.
Technically, miner can directly work with a wallet, without pool. This mode named solo mining.
See spec for getblocktemplate: https://github.com/bitcoin/bips/blob/master/bip-0022.mediawiki
I'm interested in using AKKA for an IoT Device scenario but I'm worried about complicating an individual actor. In most industries, a device is not as simple as a 'temperature sensor' you see in most tutorials. A device represents something more complex that can take on the following characteristics:
Many sensors can be represented (temperatures, electrical/fluid flows, power output, on/off values.....
Each of the values above can be queried for current value, and more likely historical values (trends, histograms....)
Alerting rules can be set up for any one of the sensor values
Each device has a fairly complex configuration that must be managed (what sensors, what unit of measure)
Many different message types can be sent (sensor reading request, alerts, configuration updates....)
So my general question is does anyone have good advice on what the level of complexity an actor should take on?
Thanks
Steve
Below are a few bullet points one might want to keep in mind when determining what level of complexity an actor should take on:
Akka actors are lightweight and loosely-coupled by design, thus scale well in a distributed environment. On the other hand, each actor can be tasked to handle fairly complex business logic using Akka's functionality-rich API. This results in great flexibility in determining how much workload an actor should bear.
In general, quantity of IoT devices and operational complexity in each device are the two key factors in the design of the device actor. If total device quantity is large, one should consider having some group-device actors each of which handles a set of devices using, for instance, a private key-value collection. On the other hand, if each IoT device involves fairly complex computation or state mutation logic, it might be better to make each actor represent an individual device. It's worth noting that the two strategies aren't mutually exclusive.
For historical data, I would recommend having actors periodically fed to a database (e.g. Cassandra, PostgreSQL) for OLAP queries. Actors should be left to answer only simple queries.
Akka actors have a well-defined lifecycle with hooks like preStart(), postRestart(), postStop() for programmatic logic control. Supervisor strategies can be created to manage actors in accordance with specific business rules (send alerts, restart actors, etc).
On customizing attributes (e.g. unit of measure) specific to the type of devices, one could model a device type along with its associated sensor attributes, say, as a case class and make it a parameter of the device actor.
Capability of handling different message types via non-blocking message passing is one of the biggest strengths of Akka actors. The receive partial function in an actor effectively handles various message types via pattern matching. When representing a device with complex state mutation logic, its operational state can be safely hotswapped via context.become.
This blog post about simulating IoT devices as individual actors might be of interest.
I have a question regarding how does Couchbase internally handle concurrency.
I tried researching in their documentation and all I found was that it depends on which locking mechanism you use in your application, the two main being :
Optimistic locking
Pessimistic locking
However, both of the above are related to how we want our strategy to be for saving data , meaning if we prefer to lock it or not.
In our case, IF we are not using either of those locking in our application, how would couchbase serve the document in the scenario below :
If application A writes a document A
At the very same instance application B tries to read Document A
My question is will Application B have to queue up to read the document, or by default it will get served the older version (all of this is not going through sync gateway and we are using .Net DLL directly for writing and reading).
Couchbase version 4.5.0
If you are using the Couchbase SDK and connecting directly to the Data Service, Couchbase is strongly consistent. If application A writes the document and immediately after application B reads it, application B will get that version. The consistency comes from how Couchbase distributes the data and how the client SDK accesses it. Couchbase distributes each object to one of 1024 active shards (Couchbase calls them vBuckets). There are replicas, but I will not get into that here. When the SDK goes to read/write objects directly, it take the object ID you give, passed it into a consistent CRC32 hash. The output of that hash is a number between 0-1023, the vBucket number. The SDK then looks into the cluster map (a JSON document distributed by the cluster) and finds where in the cluster that vBucket lives. The SDK then goes and talks directly to that node and vBucket. That is how application A can write an object and then microseconds later application B reads it. They are both reading and writing to the same place. Couchbase does not scale reads from replicas. Replicas are only for HA.
Because, as Kirk mentioned in his reply, Couchbase is consistent, both the read and write requests in your scenario will go to the same node and access the same object in the server's memory. However, concepts like "at the same time" get fuzzy when talking about distributed systems with network latency and various IO queues involved. Ultimately, the order of execution of the two "simultaneous" requests will depend on the order that the server receives them, but there is no deterministic way to predict what it will be. There are too many variables on the way; what if the CLR of one of the client decides to do garbage collection just then, delaying the request, or one of the client machines experiences momentary network lag, etc. This is one of the reasons that the documentation recommends using explicit locking for concurrent writes, to enforce predictable behavior in the face of unpredictable request ordering.
In your scenario though, there is simply no way to know in which order the write and read will occur, because "at the same time" is not an exact concept. One possible solution in your case might be to use explicit versioning for the data. This can be a timestamp or some sort of revision number that's incremented every time the document is changed. Of course, using a timestamp runs into a similar problem as above, because it's recorded according to the clock of the machine application A runs on, which is most likely different from the clock where application B runs.
I've been tasked with creating an intermediate layer which needs to exchange data (over HTTP) between two independent systems (e.g. Receiver <=> Intermediate Layer (IL) <=> Sender). Receiver and Sender both expose a set of API's via Web Services. Everytime a transaction occurs in the Sender system, the IL should know about it (I'm thinking of creating a Windows Service which constantly pings the Sender), massage the data, then deliver it to the Receiver. The IL can temporarily store the data in a SQL database until it is transferred to the Receiver. I have the following questions -
Can WCF (haven't used it a lot) be used to talk to the Sender and Receiver (both expose web services)?
How do I ensure guaranteed delivery?
How do I ensure security of the messages over the Internet?
What are best practices for handling concurrency issues?
What are best practices for error handling?
How do I ensure reliability of the data (data is not tampered along the way)
How do I ensure the receipt of the data back to the Sender?
What are the constraints that I need to be aware of?
I need to implement this on MS platform using a custom .NET solution. I was told not to use any middleware like BizTalk. The receiver is an SDFC instance, if that matters.
Any pointers are greatly appreciated. Thank you.
A Windows Service that orchestras the exchange sounds fine.
Yes WCF can deal with traditional Web Services.
How do I ensure guaranteed delivery?
To ensure delivery you can use TransactionScope to handle the passing of data between the
Receiver <=> Intermediate Layer and Intermediate Layer <=> Sender but I wouldn't try and do them together.
You might want to consider some sort of queuing mechanism to send the data to the receiver; I guess I'm thinking more of a logical queue rather than an actual queuing component. A workflow framework could also be an option.
make sure you have good logging / auditing in place; make sure it's rock solid, has the right information and is easy to read. Assuming you write a service it will execute without supervision so the operational / support aspects are more demanding.
Think about scenarios:
How do you manage failed deliveries?
What happens if the reciever (or sender) is unavailbale for periods of time (and how long is that?); for example: do you need to "escalate" to an operator via email?
How do I ensure security of the messages over the Internet?
HTTPS. Assuming other existing clients make calls to the Web Services how do they ensure security? (I'm thinking encryption).
What are best practices for handling concurrency issues?
Hmm probably a separate question. You should be able to find information on that easily enough. How much data are we taking? what sort of frequency? How many instances of the Windows Service were you thinking of having - if one is enough why would concurrency be an issue?
What are best practices for error handling?
Same as for concurrency, but I can offer some pointers:
Use an established logging framework, I quite like MS EntLibs but there are others (re-using whatever's currently used is probably going to make more sense - if there is anything).
Remember that execution is unattended so ensure information is complete, clear and unambiguous. I'd be tempted to log more and dial it down once a level of comfort is reached.
use a top level handler to ensure nothing get's lost; but don;t be afraid to log deep in the application where you can still get useful context (like the metadata of the data being sent / recieved).
How do I ensure the receipt of the data back to the Sender?
Include it (sending the receipt) as a step that is part of the transaction.
On a different angle - have a look on CodePlex for ESB type libraries, you might find something useful: http://www.codeplex.com/site/search?query=ESB&ac=8
For example ESBasic which seems to be a class library which you could reuse.
I want to know the technical reasons why the lift webframework has high performance and scalability? I know it uses scala, which has an actor library, but according to the install instructions it default configuration is with jetty. So does it use the actor library to scale?
Now is the scalability built right out of the box. Just add additional servers and nodes and it will automatically scale, is that how it works? Can it handle 500000+ concurrent connections with supporting servers.
I am trying to create a web services framework for the enterprise level, that can beat what is out there and is easy to scale, configurable, and maintainable. My definition of scaling is just adding more servers and you should be able to accommodate the extra load.
Thanks
Lift's approach to scalability is within a single machine. Scaling across machines is a larger, tougher topic. The short answer there is: Scala and Lift don't do anything to either help or hinder horizontal scaling.
As far as actors within a single machine, Lift achieves better scalability because a single instance can handle more concurrent requests than most other servers. To explain, I first have to point out the flaws in the classic thread-per-request handling model. Bear with me, this is going to require some explanation.
A typical framework uses a thread to service a page request. When the client connects, the framework assigns a thread out of a pool. That thread then does three things: it reads the request from a socket; it does some computation (potentially involving I/O to the database); and it sends a response out on the socket. At pretty much every step, the thread will end up blocking for some time. When reading the request, it can block while waiting for the network. When doing the computation, it can block on disk or network I/O. It can also block while waiting for the database. Finally, while sending the response, it can block if the client receives data slowly and TCP windows get filled up. Overall, the thread might spend 30 - 90% of it's time blocked. It spends 100% of its time, however, on that one request.
A JVM can only support so many threads before it really slows down. Thread scheduling, contention for shared-memory entities (like connection pools and monitors), and native OS limits all impose restrictions on how many threads a JVM can create.
Well, if the JVM is limited in its maximum number of threads, and the number of threads determines how many concurrent requests a server can handle, then the number of concurrent requests will be determined by the number of threads.
(There are other issues that can impose lower limits---GC thrashing, for example. Threads are a fundamental limiting factor, but not the only one!)
Lift decouples thread from requests. In Lift, a request does not tie up a thread. Rather, a thread does an action (like reading the request), then sends a message to an actor. Actors are an important part of the story, because they are scheduled via "lightweight" threads. A pool of threads gets used to process messages within actors. It's important to avoid blocking operations inside of actors, so these threads get returned to the pool rapidly. (Note that this pool isn't visible to the application, it's part of Scala's support for actors.) A request that's currently blocked on database or disk I/O, for example, doesn't keep a request-handling thread occupied. The request handling thread is available, almost immediately, to receive more connections.
This method for decoupling requests from threads allows a Lift server to have many more concurrent requests than a thread-per-request server. (I'd also like to point out that the Grizzly library supports a similar approach without actors.) More concurrent requests means that a single Lift server can support more users than a regular Java EE server.
at mtnyguard
"Scala and Lift don't do anything to either help or hinder horizontal scaling"
Ain't quite right. Lift is highly statefull framework. For example if a user requests a form, then he can only post the request to the same machine where the form came from, because the form processeing action is saved in the server state.
And this is actualy a thing which hinders scalability in a way, because this behaviour is inconistent to the shared nothing architecture.
No doubt that lift is highly performant but perfomance and scalability are two different things. So if you want to scale horizontaly with lift you have to define sticky sessions on the loadbalancer which will redirect a user during a session to the same machine.
Jetty maybe the point of entry, but the actor ends up servicing the request, I suggest having a look at the twitter-esque example, 'skitter' to see how you would be able to create a very scalable service. IIRC, this is one of the things that made the twitter people take notice.
I really like #dre's reply as he correctly states the statefulness of lift being a potential problem for horizontal scalability.
The problem -
Instead of me describing the whole thing again, check out the discussion (Not the content) on this post. http://javasmith.blogspot.com/2010/02/automagically-cluster-web-sessions-in.html
Solution would be as #dre said sticky session configuration on load balancer on the front and adding more instances. But since request handling in lift is done in thread + actor combination you can expect one instance handle more requests than normal frameworks. This would give an edge over having sticky sessions in other frameworks. i.e. Individual instance's capacity to process more may help you to scale
you have Akka lift integration which would be another advantage in this.