I wish to download every transaction that happened on blockchain since the genesis. In particular, I want to download data for BSC.
How do I download blockchain data?, e.g. Is there an API or a protocol I can use?
Once I've dowloaded the data to date... How do I subscribe to new transactions?
You can run a full node (or connect to an external provider that runs a full node, but most of them have some rate limits and some don't allow subscriptions) and then query the JSON-RPC API.
A widely used wrapper for the JSON-RPC API calls is web3js. You can use their web3.eth library because of the compatilibity between Ethereum and BSC.
If you want to download all transactions, you'll need to loop through all blocks from 1 to latest using the getBlock() method, which also returns all transactions in this particular block.
For subscribing to new transactions, you'll subscribe to the newBlockHeaders event, from which you can get the block number and query the block (see previous paragraph).
You can use BlockchainSpider to collect the data in the blockchain without a full node.
In particular, Blockchainspider also supports collecting the data on BSC.
Run this command for collecting the metadata, external transactions, and internal transactions of all blocks in BSC:
scrapy crawl blocks.bsc -a types=external,internal
In addition, you can even use BlockchainSpider to collect label data on the blockchain and deeply mine the transaction network of a specific address.
Enjoy it.
Related
I would like to get USD price of DeFi tokens from Chainlink.
However, it looks like, I can get the price of an asset through either the Chainlink API calls, or Chainlink price feeds.
What are the costs associated with each and the differences and tradeoffs?
With Chainlink price feeds, each data feed is updated by a decentralized oracle network. Each oracle in the set publishes data during an aggregation round. That data is validated and aggregated by a smart contract, which forms the feed's latest and trusted answer.
Each oracle operator is rewarded for publishing data. On mainnet, sponsors are paying the LINK associated to keep those feeds live, decentralized, and secure, so they are not free. This allows the network to be a shared resource where everyone chips in a tiny bit and makes them even cheaper than running even your own centralized feed.
That being said, the price feeds are currently a simple view function, and anyone can technically use them as access controls are on the roadmap to be built at some point. It’s not really economical or fair for those backers who are paying to support them for everyone else. Until access controls are built, please reach out if you are using them on mainnet and learn how you can contribute to keep them reliable honest, and secure.
With API calls, Chainlink enables your contracts to access any external data source, through their decentralized oracle network. Making a HTTP GET request will fail unless your deployed contract has enough LINK to pay for it, therefore one needs to fund that smart contract.
If your contract is calling a public API endpoint, an Oracle job may already exist for it. To check available jobs and GET request prices, visit https://market.link/
Can you elaborate on the differences between Pub/Sub and Firestore and provide some scenarios or use cases on which one to choose?
I'm not sure which one to use for building an app for a food delivery service that services real-time updates reflected as soon as they are added or changed to the database, ensuring that customers and drivers are aware of when food is ready for pickup and when food is in transit to their end destination such as UberEats.
The difference is quite simple:
Firestore (RealtimeDB) is for backend to frontend (customers/users) communication and realtime updates
Pubsub is a backend to backend message bus for async processing.
In your use case, you won't use PubSub to send notification to your users! Use realtimeDB to perform these updates.
Pub/Sub is like a notification system wherein you receive updates when something is added, changed or removed.
Firestore, on the other hand, is a NoSQL database for mobile (Android, iOS) and other web apps that can be directly access via native SDK. It can support many data types, from simple strings to complex objects. It also supports whatever data structure that works best for your app.
It is best to use Firestore for your app as it provides realtime updates.
You can check for the detailed documentation of Pub/Sub and Firestore.
For Firestore, you can either use either mobile/web client library or server client library.
Here's the link for Firestore, containing its benefits and key features.
Is there a REST or Node.JS library API that provides global network metadata for Bitcoin and/or Etherum ?
The metadata I'm looking for is -
Average wait time for a transaction confirmation on the network
Average fee cost per transaction on the network
I know I could crawl/parse one of the many sites that provide this data, but that's not ideal. Hence I'm looking for a dedicated API to obtain this information.
Eventually I found out of places like CryptoCompare that provide that data
I need to get the List of Partitions for an EventHub. I am trying to use EventProcessorClient from the Latest SDK. This does not seem to have a getRuntimeInformation method.
Is there any way I can get the list of partitions foa an EventHub using any API for a EventProcessorClient client.
The processor is intended to manage partitions on your behalf and without any explicit action on your part, so it does not expose any means to inspect the Event Hub or its properties.
To inspect your Event Hub and its partitions, you'll want to use the EventHubClientBuilder to create either a producer or consumer client to do so. I'd recommend taking a peek at GetEventHubMetadata.java from the client library samples, which demonstrates doing so.
I was looking here if is possible to insert/update a large quantity of rows from an as400 system.
I have a website stored on another server online and that website must be updated with the new stocks for each article. But this data only exists in the as400 system.
I would like to be the as400 system linking the web-server instead of the web-server link to as400 for security reasons.
A better system would be to update/insert everytime a change has been made in the as400, but if this is not possible it could be making an update every 3 hours in order to mantain consistency between the 2 servers.
Thanks
Yes it can be done.
You can attach a database trigger to your stock table that pushes the key fields to a data queue anytime an insert, update or delete is performed. You can then process the data queue to send the updates to the web site using an HTTP POST or other means.
IBM Redbook: Stored Procedures, Triggers, and User-Defined Functions
on DB2 Universal Database for
iSeries
IBM i 7.1 Information Center: Data Queue APIs
Scott Klement's RPG IV Sockets Tutorial