I am building decentralized application which grabs data from blockchain to mysql database.
I'm not sure, but I guess it is possible that one part of Ethereum network accepts newly mined transaction X and another part accepts mined transaction Y. Some time later one of this transactions should be accepted by full chain and other transaction should fail.
If my node gets in wrong chain I will have incorrect data on my mysql database. And it will be hard to revert database back.
How to deal correctly with these types of conflicts? Should I grab data only after certain number of confirmations (for example 5 or 10)? Or there is another approach?
Related
Since bitcoin is a blockchain and blockchain has been described as a kind of database, how would the data schema of bitcoin look like? Is it a single table database? If yes, which columns are inside this table?
The data is stored in an application-specific format optimized for compact storage, and wasn't really intended to be easily parsed by other applications.
See https://bitcoin.stackexchange.com/q/10814
For this custom format, see https://en.bitcoin.it/wiki/Protocol_documentation#block
There are various databases for various usages. As a reference client I would use bitcoin-core and describe its standard structure that is stored via the client. It actually uses "leveldb" and "berkleydb-4.8" for storing all kind of data.
Wallet database
Saves your transactions, generated public/private keys. That is usually encrypted ;)
Source: Wallets
Index Database
It's usually OPTIONAL, but usually stores a list of all transactions and in which block they occurred
Block Database
It's the most important db which locally stored and share via the network to communicate about newly created blocks and verify them. Every client has a copied version of it.
They usually store all blocks that ever occurred and also include fork-off blocks and also obsolete blocks.
Source: Blockchain / Transactions
Peers Database
Obviously there also is a database for all peers you have seen in the past. It rates each peer by giving it a ban-score, stores their IP addresses, ports and last seen status.
Conclusion:
That would be all databases. They mostly have "one table" which includes exactly the previously described data structures.
More information about the p2p network structure can be found right here.
I'm not sure how to achieve consistent read across multiple SELECT queries.
I need to run several SELECT queries and to make sure that between them, no UPDATE, DELETE or CREATE has altered the overall consistency. The best case for me would be something non blocking of course.
I'm using MySQL 5.6 with InnoDB and default REPEATABLE READ isolation level.
The problem is when I'm using RDS DataService beginTransaction with several executeStatement (with the provided transactionId). I'm NOT getting the full result at the end when calling commitTransaction.
The commitTransaction only provides me with a { transactionStatus: 'Transaction Committed' }..
I don't understand, isn't the commit transaction fonction supposed to give me the whole (of my many SELECT) dataset result?
Instead, even with a transactionId, each executeStatement is returning me individual result... This behaviour is obviously NOT consistent..
With SELECTs in one transaction with REPEATABLE READ you should see same data and don't see any changes made by other transactions. Yes, data can be modified by other transactions, but while in a transaction you operate on a view and can't see the changes. So it is consistent.
To make sure that no data is actually changed between selects the only way is to lock tables / rows, i.e. with SELECT FOR UPDATE - but it should not be the case.
Transactions should be short / fast and locking tables / preventing updates while some long-running chain of selects runs is obviously not an option.
Issued queries against the database run at the time they are issued. The result of queries will stay uncommitted until commit. Query may be blocked if it targets resource another transaction has acquired lock for. Query may fail if another transaction modified resource resulting in conflict.
Transaction isolation affects how effects of this and other transactions happening at the same moment should be handled. Wikipedia
With isolation level REPEATABLE READ (which btw Aurora Replicas for Aurora MySQL always use for operations on InnoDB tables) you operate on read view of database and see only data committed before BEGIN of transaction.
This means that SELECTs in one transaction will see the same data, even if changes were made by other transactions.
By comparison, with transaction isolation level READ COMMITTED subsequent selects in one transaction may see different data - that was committed in between them by other transactions.
How to get Ethereum internal transaction list / contract transaction list by RPC ?
Contract: 0xa5025faba6e70b84f74e9b1113e5f7f4e7f4859f
How to get the contract address transaction list by RPC ?
enter image description here
What you are trying to do is iterate through the blockchain and check each transaction in each block for a transaction that you are describing. While this will work (with a bit of code), you are better off reading the data from an existing block explorer. By doing it on your own, you will be creating a limited-functionality block explorer, when you should be using other resources.
Take a look at the Etherscan APIs, as they contain all the data you need.
I'm exploring moving an application built on top of MySQL into Spanner and am not sure if I can replicate certain functionality from our MySQL db.
basically a simplified version of our mysql schema would look like this
users idnamebalanceuser_transactionsiduser_idexternal_idamountuser_locksuser_iddate
when the application receives a transaction for a user the app starts a mysql transaction, updates the user_lock for that user, checks if the user has sufficient balance for the transaction, creates a new transaction, and then updates the balance. It is possible the application receive transactions for a user at the same time and so the lock forces them to be sequential.
Is it possible to replicate this in Spanner? How would I do so? Basically If the application receives two transactions at the same time I want to ensure that they are given an order and that the changed data from the first transaction is propagated to the second transaction.
Cloud Spanner would do this by default since it provides serializability which means that all transactions appear to have occurred in serial order. You can read more about the transaction semantics here:
https://cloud.google.com/spanner/docs/transactions#rw_transaction_semantics
I have setup the Fabric as per the instruction using docker and everything works fine. I have a chaincode which stores a value in the world state which I can read afterwards using a query method.
My scenario is something like this: I submit multiple separate requests within a short period of time to store different data in the world state. Within each request I need to read the data just submitted previously. However, I am unable to read the most recently submitted data.
My understanding is that it might be because those data might not be stored in the blockchain yet and hence they cannot be read. With this understanding, I introduced a sleep function to sleep for a few seconds to give enough time for the previously submitted data to be included in the blockchain. However, this approach was not successful.
So I am wondering if there is any way to read the previous data just after storing the subsequent data.
Thanks,
Ripul
Waiting a few seconds in the chaincode would not be sufficient. Data that is 'written' in chaincode is not yet committed to the database, it is only a proposal to write something to the database at that point. Only committed data is read back in chaincode. Therefore after you make an update in chaincode and get the proposal response, you must submit the transaction to ordering. It may take a few seconds for orderer to cut the block, distribute it to peers, and have peers commit the data. Only then can the data be read back in chaincode.
If you must read the data that you just wrote within the same chaincode function, then you will need to keep a map of the data that has been written and retrieve the value from the map rather than from the committed database.