This is a scalability question for Hyperledger Fabric.
Understand that Hyperledger Fabric leverages couch DB to maintain current state.
Questions:
Is there HA (A/A and/or A/P) configuration for the state DB? How to avoid SPOF?
The chain is a transaction log. How big can this transaction log get? 100GB? without affecting speed of append
1) Fabric can utilize embedded LevelDB or external CouchDB to maintain state. In either case, think of the state database as part of the peer - there is a 1:1 relationship between peer and its state database. HA (A/A) in Fabric, and in blockchain in general, is provided at the node (peer) level. Transactions can be endorsed by any peer(s) and are then ordered and delivered to all peers in the network. There is natural HA across the network, plus most organizations would typically host multiple peers for 'local' HA as well. If a peer goes down you route traffic to your other peers. If a peer or state database becomes corrupt you can rebuild it and it will state transfer blocks from other peers upon joining the channel.
2) The chain is an append-only transaction log on local disk or direct attached storage. It is not a single file but appends to a new file once the prior file reaches a threshold size (64MB by default). Therefore the maximum size of the transaction log is bound only by disk space.
Related
What is peer and node in hyper ledger and also what peer node start will do?
what is the meaning of peer and node and how it can use in hyperledger
unless you have a business requirement to pursue the v0.6 architecture, you should definitely pivot to v1.0 and the current stack.
To answer your question, there are two important entities to consider in a Fabric network (there are many components in the network, but we'll stick with two for now). You have peers and orderers. We tend to use the term peer and peer node interchangeably; they are the same thing. You may also come across orderer and orderer node; again they are the same thing.
All peer nodes in Fabric v1.0 architecture are validating and committing peers (i.e. they maintain a ledger and run validation checks against blocks of transactions before appending to their ledger and updating world state). Ordering nodes do what you expect they might - play a central role in the ordering of transactions. We don't need to delve into the specific role of orderers, but as a quick synopsis: responsible for accepting broadcasts of incoming transactions and responsible for delivering blocks of transactions to peers on a channel; with a kafka implementation, the ordering node(s) will actually relay the txs to the appropriate partition in the kafka cluster. Ok enough on that.
Peer node start is simply the command to launch a peer container. If you inspect a docker-compose file, you'll notice there is a command variable where peer node start is passed.
Correct me if I am wrong, or confirm, is consensus synchronizing history between peers?
Each peer stores all transactions history?
Each peer is running on separated machines or all peers are running on same machine(in docker for example) ?
Simply the consensus or (total-order broadcast, atomic broadcast) done between peers are on same channel.
Ordering service provides a shared communication channel to clients and peers, offering a broadcast service for messages containing transactions. Clients connect to the channel and may broadcast messages on the channel which are then delivered to all peers.
The channel supports atomic delivery of all messages, that is, message communication with total-order delivery and (implementation specific) reliability. In other words, the channel outputs the same messages to all connected peers and outputs them to all peers in the same logical order. This atomic communication guarantee is also called total-order broadcast, atomic broadcast, or consensus in the context of distributed systems. The communicated messages are the candidate transactions for inclusion in the blockchain state.
Consensus is the process of multiple peers determining if their records of transactions match. With Hyperledger Fabric version 0.6, all of the blocks on the ledger will match across peers if the peers are in consensus. In this version, each validating peer does store all transactions.
Hyperledger Fabric version 1.0 which is under development should make it possible for a subset of participants in a blockchain network to process certain transactions. Information on the future consensus architecture has more details.
The peers usually run in multiple Docker containers that are owned by one entity when using Hyperledger Fabric version 0.6. With upcoming Hyperledger Fabric version 1.0 it should be possible for one entity to start a blockchain network and then invite other parties (such as other companies) to join. In this case, peers would be distributed across multiple entities.
I am evaluating whether to use Akka and akka persistent as key toolkit of certain project, in which there is a complex background running process (might be triggered by Quartz at a fixed time per day).
The back-ground running process will communicate with many different external services via HTTP communication, will generates many encrypted files locally, and transfer them via SFTP.
From business perspective:
The service is mission critical, roughly it will charge N million users' money from their bank cards automatically and help them purchase some fund product.
From technical perspective:
Each of the external service might not be available with whatever reason, such as network issues, the external service might running out of their resources(i.e. jdbc connections).
Our service might be killed, restarted, re-deployed due to urgent reason or crashed with some unexpected errors.
Once the process was restarted with an incomplete job, then it needs to gracefully complete them with different strategies, such as redo, confirm external system business state, and resume from certain check point.
I was reading from official AkkaScala.PDF, and some youtube conference videos, all of them were mentioning, actor's state can be restored by replaying the events from journal after JVM crash.
But it must be a stupid question, since i did not find it was being discussed:
Imagine there were 1000 persistent actors living in the service, and the service's JVM crashed and restarted, who should be in charge of triggering re-create those 1000 persistent actors in the newly created actor system in both single process mode and clustered mode? And how? Or what articles should I read first?
You should read basics of Akka Persistence and Akka Persistence Query. But probably, first thing that comes to my mind is to use Akka Persistence Query AllPersistenceIdsQuery or CurrentPersistenceIdsQuery. It will give you all persistence id's which you can use to reignite your persistent actors. Persistent actors by specific persistent id will replay all events from event store journal. You can take snapshots to speed up recovery. Your event store will probably be some kind of database (e.g. Cassandra). Considering that your persistent actor has specific mutable state, it will be brought back to its last state after the recovery. Recovery might take some time.
I created a network with 4 peers using docker-compose and docker for Mac.
I deploy my blockchain on this network successfully.
Now I'm launching a 5th peer using another yml file using the details of one of the previous peer as discovery node.
It appears in the list returned by http://localhost:7050/network/peers however my blockchain is not deployed on this peer and I cannot use it to process transactions.
Do I have to deploy the chaincode again on this peer? Did I miss something?
This is limitation in Fabric’s versions 0.5 and 0.6
Network configuration cannot be changed in realtime. In case If you use PBFT consensus, network configuration is hardcoded in:
“fabric/consensus/pbft/config.yaml"
# Maximum number of validators/replicas we expect in the network
# Keep the "N" in quotes, or it will be interpreted as "false".
"N": 4
The challenge is in updating configuration on all peers synchronously, otherwise they will not be able to reach consensus.
In one of next Fabric versions this configuration’s parameter will be moved to blockchain and it will be possible to add new peers and modify consensus configuration on the fly.
Update for question in comment:
Saw only this high level Roadmap proposal:
In Bitcoin blockchain a node/peer will order transactions, create a block, does the PoW and "announces" this block to the other miners.
Once the other miners agree (by hashing the block+nounce+etc...) that the block is valid it is part of the blockchain.
But in Hyperledger (as far as I understood) the VPs don't do mining (and hence don't spread the mined block). So how does the individual VPs order them so that all VPs have the same ordered transactions of blocks?
For the Fabric implementation the goal is to have (link):
Consensus is a method for validating the order of network requests, or
transactions (deploy and invoke), on a blockchain network. The correct
ordering of transactions is critical, because many types of network
transactions have a dependency on one or more prior transactions
(account debits often have a dependency on prior credits, for
example). On a blockchain network, there is no single authority that
determines the transaction order; instead, each blockchain node (or
peer) has an equal say in establishing the order, by implementing the
network consensus protocol. Consensus, therefore, ensures that a
quorum of nodes agree on the order in which transactions are appended
to the shared ledger. By resolving any discrepancies in the proposed
transaction order, consensus guarantees that all network nodes are
operating on an identical blockchain. In other words, consensus
guarantees the integrity and consistency of blockchain network
transactions.
Nevertheless in current version there is a leader (one of validation peers) who is responsible for ordering transactions before they are executed by other peers.
In next Fabric’s version this behavior can be changed (Next Consensus Architecture Proposal):
…
A peer communicates with the consensus service and maintain the
blockchain state and the ledger. Such peers receive ordered state
updates from the consensus service and apply them to the locally held
state
…
Peers are clients of the consensus service, to which the consensus
service provides a shared communication channel offering a broadcast
service for messages containing transactions. Peers connect to the
channel and may send and receive messages on the channel. The channel
supports atomic delivery of all messages, that is, message
communication with total-order delivery and (implementation specific)
reliability. In other words, the channel outputs the same messages to
all connected peers and outputs them to all peers in the same logical
order. This atomic communication guarantee is also called total-order
broadcast, atomic broadcast, or consensus in the context of
distributed systems. The communicated messages are the candidate
transactions for inclusion in the blockchain state.