I'm wondering whether I could replicate the forceKeyspaceFlush() function found in the nodetool utility from the C/C++ driver of Cassandra.
The nodetool function looks like this:
public class Flush extends NodeToolCmd
{
#Arguments(usage = "[<keyspace> <tables>...]", description = "The keyspace followed by one or many tables")
private List<String> args = new ArrayList<>();
#Override
public void execute(NodeProbe probe)
{
List<String> keyspaces = parseOptionalKeyspace(args, probe);
String[] tableNames = parseOptionalTables(args);
for (String keyspace : keyspaces)
{
try
{
probe.forceKeyspaceFlush(keyspace, tableNames);
} catch (Exception e)
{
throw new RuntimeException("Error occurred during flushing", e);
}
}
}
}
What I would like to replicate in my C++ software is this line:
probe.forceKeyspaceFlush(keyspace, tableNames);
Is it possible?
That's an unusual request, primarily because Cassandra is designed to be distributed, so if you're executing a query, you'd need to perform that blocking flush on each of the (potentially many) replicas. Rather than convince you that you don't really need this, I'll attempt to answer your question - however, you probably don't really need this.
Nodetool is using the JMX interface (on tcp/7199) to force that flush - Your c/c++ driver talks over the native protocol (on tcp/9042). At this time, flush is not possible via the native protocol.
Work around the limitation, you'd need to either exec a jmx-capable commandline utility (nodetool or other), implement a JMX client in c++ (it's been done), or extend the native protocol. None of those are particularly pleasant options, but I imagine executing a jmx CLI utility is significantly easier than the other two.
Related
I have created simple blockchain application using NodeJS. The blockchain data file is getting stored on local File System. There is no mining blocks, no difficulty level involved in this blockchain.
Please suggest, if I can host this application on private ethereum / hyperledge, and what all changes I would need to do for this? Below code I'm using for creating blocks.
Sample Genesis Block
[{"index":0,"previousHash":"0","timestamp":1465154705,"transaction":{"id":"0","transactionHash":"0","type":"","data":{"StudInfo":[{"id":"","studentId":"","parenterId":"","schemeId":"","batchId":"","instructorId":"","trainingId":"","skillId":""}]},"fromAddress":""},"hash":"816534932c2b7154836da6afc367695e6337db8a921823784c14378abed4f7d7"}]
Sample Code(NodeJS)
var generateNextBlock = (blockData) => {
var previousBlock = getLatestBlock();
var nextIndex = previousBlock.index + 1;
var nextTimestamp = new Date().getTime() / 1000;
var nextHash = calculateHash(nextIndex, previousBlock.hash, nextTimestamp, blockData);
return new Block(nextIndex, previousBlock.hash, nextTimestamp, blockData, nextHash);
};
var calculateHashForBlock = (block) => {
return calculateHash(block.index, block.previousHash, block.timestamp, block.transaction);
};
var calculateHash = (index, previousHash, timestamp, transaction) => {
return CryptoJS.SHA256(index + previousHash + timestamp + transaction).toString();
};
var addBlock = (newBlock) => {
if (isValidNewBlock(newBlock, getLatestBlock())) {
blockchain.push(newBlock);
blocksDb.write(blockchain);
}
};
var isValidNewBlock = (newBlock, previousBlock) => {
if (previousBlock.index + 1 !== newBlock.index) {
console.log('invalid index');
return false;
} else if (previousBlock.hash !== newBlock.previousHash) {
console.log('invalid previoushash');
return false;
} else if (calculateHashForBlock(newBlock) !== newBlock.hash) {
console.log(typeof (newBlock.hash) + ' ' + typeof calculateHashForBlock(newBlock));
console.log('invalid hash: ' + calculateHashForBlock(newBlock) + ' ' + newBlock.hash);
return false;
}
return true;
};
Congratulations if you have gotten this far you have successfully setup Geth on AWS. Now we will go over how to configure an Ethereum node. Make sure you are in your home directory on your cloud server with the pwd command and then create a new folder called whatever you want to create the genesis block of your Ethereum blockchain. You can do this with the following commands. The first command is to create the folder, change directory into the folder and then create a file called genesis.json.
mkdir mlg-ethchain cd mlg-ethchain nano genesis.json
To create a private blockchain you need to define the genesis block. Genesis blocks are usually embedded in the client but with Ethereum you are able to configure a genesis block using a json object. Paste the following JSON object into your genesis.json file and we explain each variable in the following section.
{
"nonce": "0xdeadbeefdeadbeef",
"timestamp": "0x0",
"parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
"extraData": "0x0",
"gasLimit": "0x8000000",
"difficulty": "0x400",
"Mixhash":
"0x0000000000000000000000000000000000000000000000000000000000000000", "coinbase": "0x3333333333333333333333333333333333333333",
"alloc": {
}
}
The coinbase is the default address of the mining address. Because you have not created a wallet yet this can be set to whatever you want provided that it is a valid Ethereum address. Difficulty is how hard it is for a miner to create a new block. For testing and development purposes it is recommended that you start with a low difficulty and then increase it. The parentHash is the hash of the parent block which doesnt exist because this is the first block in the chain. The gasLimit is the maximum amount of gas that is required to execute a transaction or send tokens from one account to another. The nonce is a random number for this block. This is the number that every miner has to guess to define the block but for the first block it must be hardcoded. You are able to provide any extraData is the extraData section. In the alloc section you can allocate a number of premined tokens or ether to certain addresses at the beginning of the blockchain. We do not want to do this so we will keep it blank.
After you have confirmed this you can check the file to make sure it has been configured correctly with the cat command. From the same directory input this command.
cat genesis.json
From what I understand, your JS code does some sort of low-level block generation (albeit in a centralised/standalone way). I'm not sure of the purpose of that code, but it's not an app that you could "port to" Ethereum or Hyperledger [Fabric], since the role of those blockchain engines is precisely to implement the block generation logic for you.
A JS app designed to work with Ethereum wouldn't do any of the block management. At the contrary, it would perform high-level and client-level interaction with a smart contract, which is basically a class (similar to Java classes) available on the whole network and whose methods are guaranteed by the network to do what they're meant to. Your JS app would essentially call the smart contract's methods as a client without caring about any block production issues.
In other, more vague but maybe more familiar terms:
Client: the JS app
Server/backend: the smart contract
Infrastructure/engine: Ethereum
On Ethereum, the way you get to the smart contract as a client is by sending RPC calls in JSON (hence the name JSON-RPC) to the contract's methods. The communication is done by embedding that JSON over HTTP to an Ethereum node, preferably your own. In Javascript, a few libraries such as web3 give you a higher-level abstraction view so that you don't need to care about JSON-RPC and you can think of your contract's methods as normal Javascript functions.
Also, since you're asking about private Ethereum: another consequence of that distribution of layers is that client code and smart contracts don't need to care about whether the Ethereum network is a public or a private one, i.e. what consensus protocol is in place. To make a bold analogy, it's similar to how SQL queries or schemas stay the same no matter how the database is persisted on disk.
Interaction with Hyperledger Fabric is similar in concept, except that you do plain REST calls to an HTTP endpoint exposed by the network. I'm not sure what client-level abstraction libraries are available.
I want to be able to replace and add some classes to an already running JVM. I read that I need to use CreateRemoteThread, but I don't completely get it. I read this post on how to do it (Software RnD), but I can't figure out what it does and why. Besides that, it only introduces new classes, but doesn't change existing ones. How can I do it with C++?
You don't even need CreateRemoteThread - there is an official way to connect to remote JVM and replace loaded classes by using Attach API.
You need a Java Agent that calls Instrumentation.redefineClasses.
public static void agentmain(String args, Instrumentation instr) throws Exception {
Class oldClass = Class.forName("org.pkg.MyClass");
Path newFile = Paths.get("/path/to/MyClass.class");
byte[] newData = Files.readAllBytes(newFile);
instr.redefineClasses(new ClassDefinition(oldClass, newData));
}
You'll have to add MANIFEST.MF with Agent-Class attribute and pack the agent into a jar file.
Then use Dynamic Attach to inject the agent jar into the running VM (with process ID = pid).
import com.sun.tools.attach.VirtualMachine;
...
VirtualMachine vm = VirtualMachine.attach(pid);
try {
vm.loadAgent(agentJarPath, options);
} finally {
vm.detach();
}
A bit more details in the article.
If you insist on using C/C++ instead of Java API, you may look at my jattach utility.
I have an emulator program written in C++ running on Ubuntu 12.04. There are some settings and options needed for running the program, which are given by the main's arguments. I need to query these options via HTTPS from a remote machine/mobile device (so basically imagine I want to return main's arguments). I was wondering if someone can help me with that.
There should probably be some libraries for the ease, for example Poco. I'm not sure how suitable it is for my case, but here is any example of connection setup in poco. It's not a must to use any libraries though; just the most efficient/simplest way.
Mongoose (or the non-GPL fork Civetweeb) are embedded web servers. Very easy to set up and add controllers for (typically a half dozen lines of code)
Just add the project file (1 c file) to your project and build, add a line to start the server listening and give it what options you like, and add a callback function to handle requests. It does ssl out of the box (though IIRC you'll need to have openssl installed too)
There was another SO answer with some comparisons. I used civetweb at work, and was impressed at how easy it all was. There's not too much documentation though.
Here's a stripped-down POCO version, for full code see HTTPSTimeServer example.
struct MyRequestHandler: public HTTPRequestHandler
{
void handleRequest(HTTPServerRequest& request, HTTPServerResponse& response)
{
response.setContentType("text/html");
// ... do your work here
std::ostream& ostr = response.send();
ostr << "<html><head><title>HTTPServer example</title>"
<< "<body>Success!</body></html>";
}
};
struct MyRequestHandlerFactory: public HTTPRequestHandlerFactory
{
HTTPRequestHandler* createRequestHandler(const HTTPServerRequest& request)
{
return new MyRequestHandler;
}
};
// ...
// set-up a server socket
SecureServerSocket svs(port);
// set-up a HTTPServer instance (you may want to new the factory and params
// prior to constructing object to prevent the possibility of a leak in case
// of exception)
HTTPServer srv(new MyRequestHandlerFactory, svs, new HTTPServerParams);
// start the HTTPServer
srv.start();
I am working on a blackberry application. I need to call soap webservices, but I am unable to do so, and am getting null as a response. Following is my code:
private static final String CONNECTION_PARAMS = ";deviceside=true";
SoapObject request = new SoapObject("http://service.action.com/",
"findActiveSecurities");
SoapSerializationEnvelope envelope = new SoapSerializationEnvelope(SoapEnvelope.VER11);
envelope.bodyOut = request;
HttpTransportBasicAuth ht =
new HttpTransportBasicAuth("http://myurl.com/ebclient/services/MobileClientService?wsdl"+CONNECTION_PARAMS,
"myusername",
"mypassword");
PropertyInfo propInfo=new PropertyInfo();
propInfo.type=PropertyInfo.INTEGER_CLASS;
//adding parameters
request.addProperty("arg0","NSE");
request.addProperty("arg1","0");
request.addProperty("arg2","100");
envelope.setOutputSoapObject(request);
try {
ht.call(SOAP_ACTION, envelope);
result = (SoapObject)envelope.getResponse();
System.out.println(result);
} catch (IOException e) {
e.printStackTrace();
} catch (XmlPullParserException e) {
e.printStackTrace();
}
I am getting null as a result.Please have a look at the code and help me to correct it.
Thanks in advance
Actually the problem was instead of passing 0 and 100 as String ...
request.addProperty("arg0","NSE");
request.addProperty("arg1","0");
request.addProperty("arg2","100");
I use
request.addProperty("arg0","NSE");
request.addProperty("arg1",new Integer(0));
request.addProperty("arg2",new Integer(1000));
also this link helped me.
also before asking this question I was facing some problem that the Simulator was not recognizing a Library. It shows error message something like "there is no library Ksoap2_j2me.jar" - resolved from this link.
Sorry for poor English but I think this can save time of some other developer.
It's hard to tell from what you're posted, but my guess is that you're having some kind of network problem. I'm guessing that you initialize result = null;, and then your call to ht.call() throws an IOException, leaving result null.
You're using ksoap2, which is a library written for generic J2ME clients. However, BlackBerry networking doesn't work exactly like all other J2ME platforms.
You are controlling the BlackBerry network transport with your connection params string, which is hardcoded:
private static final String CONNECTION_PARAMS = ";deviceside=true";
Unfortunately, this string suffix may not be right for all network conditions (or any, if you don't have device APN settings correct).
I think you have a couple choices:
1. Connection Suffix Strings
You can try dynamically choosing the right suffix string, depending on conditions when your app runs. This can allow the device, for example, to connect via Wi-Fi if it's available, or via BES if that's available. Developers new to BlackBerry may be surprised that app code needs to worry about this (read here for more, or watch this).
If you want to simply replace CONNECTION_PARAMS with a dynamic string, you might check out the implementation here.
2. ConnectionFactory
In OS 5.0, BlackBerry added the ConnectionFactory class, which was a big improvement over the old way of having to assemble connection strings. If you only need to support OS 5.0 and greater, you might choose to rewrite the code to use ConnectionFactory.
Since you have access to the ksoap source code, you could change it. It looks like the connection code is in ServiceConnectionMidp.java:
public ServiceConnectionMidp(String url) throws IOException {
connection = (HttpConnection) Connector.open(url, Connector.READ_WRITE, true);
}
Instead of attaching connection parameters to the url passed to this class, you could change the class to get the connection from a ConnectionFactory, customized to support the network transports you want.
Doing this means that if you ever want to update your code to use a new version of ksoap2, you'll need to make these modifications again. However, given the future of BlackBerry Java, that seems like a reasonable compromise to make.
What is the best approach to write unit tests for code that persists data to nosql data store, in our case cassandra?
=> We are using embedded server approach using a utility from git hub (https://github.com/hector-client/hector/blob/master/test/src/main/java/me/prettyprint/hector/testutils/EmbeddedServerHelper.java). However I have been seeing some issues with this. 1) It persists data across multiple test cases making it hard for us to make sure data is different in test cases of a test class. I tried calling cleanUp #After each test case, but that doesn't seem to cleanup data. 2) We are running out of memory as we add more tests and this could be because of 1, but I am not sure yet on that. I currently have 1G heap size to run my build.
=> The other approach I have been thinking is to mock the cassandra storage. But that might leak some issues in the cassandra schema as we often found the above approach catching issues with the way data is stored into cassandra.
Please let me know you thoughts on this and if anyone has used EmbeddedServerHelper and are familiar with the issues I have mentioned.
Just an update. I was able to resolve 2) running out of java heap space issue when running builds by changing the in_memory_compaction_limit_in_mb parameter to 32 in the cassandra.yaml used by the test embedded server. The below link helped me http://www.datastax.com/docs/0.7/configuration/storage_configuration#in-memory-compaction-limit-in-mb. It was 64 and started to fail consistently during compaction.
We use an embedded cassandra server, and I think that is the best approach when testing cassandra, mocking the cassandra API is too error prone.
EmbeddedServerHelper.cleanup() just removes files rom the file system, but data may still exist in memory.
There is a teardown() method in EmbeddedServerHelper, but I a not sure how effective that is, as cassandra has a lot of static singletons whose state is not cleaned up by teardown()
What we do is we have a method that calls truncate on each column family between tests. That will remove all data.
I think you can take a look at cassandra-unit : https://github.com/jsevellec/cassandra-unit/wiki
I use the Mojo Cassandra maven plugin.
Here's an example plugin configuration that I use to spin up a Cassandra server for use by my unit tests:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cassandra-maven-plugin</artifactId>
<version>1.1.0-1</version>
<executions>
<execution>
<goals>
<goal>start</goal>
<goal>flush</goal>
<goal>cleanup</goal>
</goals>
<phase>compile</phase>
</execution>
</executions>
</plugin>
<plugins>
<build>
I did manage to get Hector's embedded server helper class working which can be very useful, however I ran into classloader conflicts due to this bug.
You cannot restart Cassandra instance within one VM - Cassandra has "shutdown per kill policy" due to singeltons that they are using.
You also do not need to restart Casandra, just only remove all column families (CFs). In order to remove CF you need first to flush data, compact it and after that finally you can drop it.
This code will connect to embedded Cassandra and execute required cleaup:
private void cleanAndCompact() throws Exception {
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
ObjectName ssn = new ObjectName("org.apache.cassandra.db:type=StorageService");
StorageServiceMBean ssmb = JMX.newMBeanProxy(mbs, ssn, StorageServiceMBean.class);
List<String> keyspaces = ssmb.getKeyspaces();
if (keyspaces == null) {
LOG.info("No keysaces to cleanup");
return;
}
for (String keyspace : keyspaces) {
if (keyspace.equalsIgnoreCase("system")) {
continue;
}
execCleanup(ssmb, keyspace);
}
}
private void execCleanup(StorageServiceMBean ssmb, String keyspace) throws Exception {
LOG.info("Cleaning up keyspace: " + keyspace);
ssmb.invalidateKeyCaches(keyspace, new String[0]);
ssmb.invalidateRowCaches(keyspace, new String[0]);
ssmb.forceTableFlush(keyspace, new String[0]);
ssmb.forceTableCompaction(keyspace, new String[0]);
ssmb.forceTableCleanup(keyspace, new String[0]);
}
Now execute CLI drop CF script:
CliMain.main(new String[] { "-host", host, "-port", Integer.toString(rpcPort), "-f", "/my/script/path/script.txt","-username", "myUser", "-password", "123456" });
and script.txt could have:
use ExampleTestSpace;
drop column family ExampleCF;
By "doesn't seem to clean up data" what exactly do you mean? That you still see your data in the database?
That problem might be due to Cassandra that doesn't delete the "values" instantly, but only after the gc_grace_seconds seconds are passed (that usually defaults to 10 days). Cassandra marks the values to be deleted.
In addition to what's been posted, there are cases when you want to test error handling - how does your app behave when a Cassandra query fails.
There are a few libraries that can help you with this:
Stubbed Cassandra
Cassandra-Spy
I'm the author of cassandra-spy and wrote to it help me test these cases.