I need a blocksci parser for ethereum. I am using https://github.com/citp/BlockSci for bitcoin . Is any similar blocksci parsing tool for ethereum?.
if i understode the question correctly i thinks this is what you are locking for :
https://github.com/alex-miller-0/Ethereum_Blockchain_Parser
This is a project to parse the Ethereum blockchain from a local geth node. Blockchains are perfect data sets because they contain every transaction ever made on the network. This is valuable data if you want to analyze the network, but Ethereum stores its blockchain in RLP encoded binary blobs within a series of LevelDB files and these are surprisingly difficult to access, even given the available tools. This project takes the approach of querying a local node via JSON-RPC, which returns unencoded transactional data, and then moves that data to a mongo database.
Related
I wish to download every transaction that happened on blockchain since the genesis. In particular, I want to download data for BSC.
How do I download blockchain data?, e.g. Is there an API or a protocol I can use?
Once I've dowloaded the data to date... How do I subscribe to new transactions?
You can run a full node (or connect to an external provider that runs a full node, but most of them have some rate limits and some don't allow subscriptions) and then query the JSON-RPC API.
A widely used wrapper for the JSON-RPC API calls is web3js. You can use their web3.eth library because of the compatilibity between Ethereum and BSC.
If you want to download all transactions, you'll need to loop through all blocks from 1 to latest using the getBlock() method, which also returns all transactions in this particular block.
For subscribing to new transactions, you'll subscribe to the newBlockHeaders event, from which you can get the block number and query the block (see previous paragraph).
You can use BlockchainSpider to collect the data in the blockchain without a full node.
In particular, Blockchainspider also supports collecting the data on BSC.
Run this command for collecting the metadata, external transactions, and internal transactions of all blocks in BSC:
scrapy crawl blocks.bsc -a types=external,internal
In addition, you can even use BlockchainSpider to collect label data on the blockchain and deeply mine the transaction network of a specific address.
Enjoy it.
Let's say I have few dapps installed on my smartphone. Are they storing full blockchain they are using on my device, or where it's stored?
No, your device doesn't hold the whole blockchain on it's storage (a blockchain is typically counted in GBs, while your app is probably a few MBs).
The key is IFPS (from How do dApps store data ? on Hackernoon)
Store only the content hashes on the blockchain. So, while the
blockchain stores only the hashes, the hashes themselves provide an
easy way to retrieve the large files being stored.
How it works (from ipfs.io):
Each file and all of the blocks within it are given a unique fingerprint called a cryptographic hash.
IPFS removes duplications across the network.
Each network node stores only content it is interested in, and some indexing information that helps figure out who is storing what
When looking up files, you're asking the network to find nodes
storing the content behind a unique hash.
Every file can be found by human-readable names using a decentralized naming system called IPNS.
I suggest you to read the whole Hackernoon article if you want to understand it deeper.
It is highly unlikely that the entirety of a blockchain is being stored on your device, nor is it likely that the blockchain data needs to exist on your device especially if you are simply using a DApp. The only real necessity for having the entirety of a blockchain or even snapshots of a blockchain stored on your mobile device would be if you are running a full or light node directly on your mobile device.
DApps are simply a user interface for subsequent interaction with a blockchain network. The blockchain network data is "stored" on a series of nodes which the DApp is interacting with via APIs. Think of this in terms of the Stack Overflow application. Your machine does not store the entire database of Stack Overflow, it simply requests and updates data via a series of APIs.
I just got a chance to work on the block-chain based platform. Could anybody help me understand the differences between IPFS and Ethereum?
You cannot store large blobs like images, pdf, videos so on the Ethereum. The Ethereum Dapp can hold a small amount of data, whereas for
saving anything more or bigger such as images, word, PDF files, and so on,
we can rely on Interplanetary File System (IPFS).
Basically, you store any file in IPFS, and then you store the IPFS hash in
the Ethereum contract. Any user with an IPFS node will be able to access
the file using that hash.
You can store data, images, the front end, and whatever you want.
IPFS is a protocol that can be used independently and not necessarily
in a blockchain. However, in real life, IPFS and the blockchain are a
perfect match! With the support of IPFS, data can remain immutable
and permanent, and just like any other content database, you can link
the address of the file stored to the Ethereum blockchain. With IPFS, the
Ethereum user has to focus only on the contract without having to put the
data on the chain itself.
Blockchain and IPFS are based on similar concepts of decentralized networks, but that's where their similarities end.
While blockchain shares a ledger with its nodes, IPFS is a peer to peer file-sharing system that hashes files (similar to the way blockchain hashes transactions), and then allows users to search for files based on those hashes.
IPFS and Blockchain are very different, and in fact you can use IPFS to store files while the hashes are kept on the blockchain. It would be like comparing Stack Overflow to Facebook. Both are websites living on the internet, but they accomplish very different tasks for their users.
I am trying to build a Decentralized application on ethereum using the solidity language.
How can i use Ethereum as database?
I mean i just want store image file on Ethereum using decentralized application.
Is that possible?
And how will it be in solidity?
you can use IPFS (checkout here : https://ipfs.io) to store data like images on the chain. Also you can checkout BigchainDB, put I assume that the first will be more suitable for your case
Storing the data on Blockchain is a costly affair.It can cost you approx.5 dollars even for the smallest kb(e.g.1 kb),calculated upon the basis of gas price.We normally store hashes of the data.
You can use 2 techniques depends upon your problem:
Storing hashes of data.
Combination of data and hashes.
We can use IPFS (https://ipfs.io/) for storing hashes to the blockchain and we can use GPG encryption to make it secure.
I understand that the BigChainDB is a distributed DB at the basic level.
It claims that is solves the problem of Scalability in the BlockChain world. What i don't understand is that how it fits into the overall block chain architecture.
In the typical Block Chain world, each node has the full copy of the
data and validates any new transactions? How does this work in the
BigChain DB when the data is distributed? Each node validates only the
blocks that it holds? Even if thats the case, it needs the entire
chain? Not clear on the processing scalability here.
Can i build a block chain network just with the BigChainDB or do i
need something like Ethereum or Hyperledger to build the block chain
network itself?
If i can build the blockchain network with BigChainDB, then what is
the equivalent of smart contract in BigChainDB?
BigchainDB internally uses RethinkDB as a datastore. Technically, the blockchain is stored as JSON strings inside RethinkDB. And BigchainDB is a kind of wrapper on top of this storage which provides the needed cryptography, techniques for building the blocks, parsing the blocks and provides utility methods to access the database. RethinkDB provides clustered storage with possible shard based architecture. This makes BigchainDB scalable when RethinkDB is configured accordingly.
Yes. It is possible to build a blockchain network just with BigchainDB.
BigchainDB library provides APIs to be called from our code. So we can build any application(preferably in Python) and integrate BigchainDB into it. This means, the business logic stays inside our application code. This is totally different from Smart contracts used in Ethereum.
You can have a look at my github code for more insights. It is my college project and not much documentation is available :-D It is a voucher transfer application where company like Sodexo issue vouchers and users can redeem it in private companies like Starbucks.