Ive been struggling to find a fast/performant way to pass data between C# and Java.
My initial and unique approach up to now was using JNIEnv.
A simple test case of a Java class Adder:
void add(float[] arr1, float[] arr2, float[] res)
{
for (int i=0; i<arr1.length; i++)
{
res[i] = arr1[i]+arr2[i];
}
}
And on C# side:
public void Add(float[] arr1, float[] arr2, float[] res)
{
var methodHandle = JNIEnv.GetMethodID(class_ref, "add","([F[F[F)V");
IntPtr arr1Handle = JNIEnv.NewArray(arr1);
IntPtr arr2Handle = JNIEnv.NewArray(arr2);
IntPtr resHandle = JNIEnv.NewArray(res);
JNIEnv.CopyArray (arr1, arr1Handle);
JNIEnv.CopyArray (arr2, arr2Handle);
JNIEnv.CallVoidMethod(Handle, methodHandle, new JValue(arr1Handle), new JValue(arr2Handle), new JValue(resHandle));
JNIEnv.CopyArray (resHandle, res);
}
For the same physical device: With float[] arrays of size 1million, profiling this Add() method in c# holds values between 200-500ms! A note to say that if i run the same test in an Android native Eclipse project, the call to the function takes 20-50ms.
If you need to handle large arrays of data like images, matrixes, for real time requirements ... then this becomes unusable.
Can anyone advise another way of binding data to Java or another approach within JNIEnv?
Thank you for your time.
Related
I was recently developing a NodeJs module for electron using C++ and the Node addon C++ API.
I wanted to create an ArrayBuffer object containing the data of an image I read using c++ iostream functions like ifstream::read(). I allocated the buffer holding the image data using the "new char[]" operator and then tried to pass the pointer to the function Napi::ArrayBuffer::New(napi_env, void*, size_t) but when I try to run the electron application I get the following error:
[45256:1208/161314.044:ERROR:node_bindings.cc(149)] Fatal error in V8: v8_ArrayBuffer_NewBackingStore When the V8 Sandbox is enabled, ArrayBuffer backing stores must be allocated inside the sandbox address space. Please use an appropriate ArrayBuffer::Allocator to allocate these buffers, or disable the sandbox.
[45256:1208/161314.074:ERROR:crashpad_client_win.cc(844)] not connected
I have no idea of what it means nor did I find anything on the internet about fixing this issue.
Here is the code:
addon.cc:
Napi::ArrayBuffer loadImage(const Napi::CallbackInfo& info) {
std::ifstream image("img.jpg");
Napi::ArrayBuffer imageBuffer;
int bufferSize;
image.seekg(0, image.end);
bufferSize = image.tellg();
image.seekg(0, image.beg);
char *data = new char[bufferSize];
image.read(data, bufferSize);
imageBuffer = Napi::ArrayBuffer::New(info.Env(), (void*)data, bufferSize);
return imageBuffer;
}
addon.js:
const testAddon = require('./build/Release/testaddon.node')
const electron = require("electron")
const path = require('path')
const { ipcMain, BrowserWindow, app } = require('electron');
const createWindow = () => {
const win = new BrowserWindow({width: 800, height: 600, webPreferences: {
preload: path.join(__dirname, 'preload.js')
}, contextIsolation: true})
win.loadFile("test.html")
}
app.whenReady().then(() => {createWindow()})
var eddu = testAddon.loadImage();
Electron broke this in 21 as you noticed, there has been some discussion with them at https://github.com/electron/electron/issues/35801.
But the ultimate answer is that you can't do Napi::ArrayBuffer::New() with an existing pointer anymore in Electron. As you are already doing a copy into data you could instead do:
imageBuffer = Napi::ArrayBuffer::New(info.Env(), bufferSize);
image.read(imageBuffer.Data(), bufferSize);
Or in other cases where a copy is not being done then Napi::ArrayBuffer::Copy() could be used instead, or the upcoming Napi::ArrayBuffer::NewOrCopy() method.
Ref: https://github.com/nodejs/node-addon-api/blob/main/doc/array_buffer.md
[RESOLVED]
I'm building a game engine that uses LuaBridge in order to read components for entities. In my engine, an entity file looks like this, where "Components" is a list of the components that my entity has and the rest of parameters are used to setup the values for each individual component:
-- myEntity.lua
Components = {"MeshRenderer", "Transform", "Rigidbody"}
MeshRenderer = {
Type = "Sphere",
Position = {0,300,0}
}
Transform = {
Position = {0,150,0},
Scale = {1,1,1},
Rotation = {0,0,0}
}
Rigidbody = {
Type = "Sphere",
Mass = 1
}
I'm currently using this function (in C++) in order to read the value from a parameter (given its name) inside a LuaRef.
template<class T>
T readParameter(LuaRef& table, const std::string& parameterName)
{
try {
return table.rawget(parameterName).cast<T>();
}
catch (std::exception e) {
// std::cout ...
return NULL;
}
}
For example, when calling readVariable<std::string>(myRigidbodyTable, "Type"), with myRigidbodyTable being a LuaRef with the values of Rigidbody, this function should return an std::string with the value "Sphere".
My problem is that when I finish reading and storing the values of my Transform component, when I want to read the values for "Ridigbody" and my engine reads the value "Type", an unhandled exception is thrown at Stack::push(lua_State* L, const std::string& str, std::error_code&).
I am pretty sure that this has to do with the fact that my component Transform stores a list of values for parameters like "Position", because I've had no problems while reading components that only had a single value for each parameter. What's the right way to do this, in case I am doing something wrong?
I'd also like to point out that I am new to LuaBridge, so this might be a beginner problem with a solution that I've been unable to find. Any help is appreciated :)
Found the problem, I wasn't reading the table properly. Instead of
LuaRef myTable = getGlobal(state, tableName.c_str());
I was using the following
LuaRef myTable = getGlobal(state, tableName.c_str()).getMetatable();
i want to develop an IOTA application, but not a messaging application or coin based system. I want an simple example of how to store data in IOTA. For example i want to build an SCM or even an simple login/registration app. Can anyone guide me? Any sample application? i try to run https://github.com/domschiener/leaderboard-example But getting same error like https://github.com/domschiener/leaderboard-example/issues/6 How to run this.
Storing text data on the tangle is not that difficult. The following are snippets from my tangle-based app. I used IOTA's API Java wrapper library Jota.
1) Connect to IOTA node. You can find a list of nodes here https://nodes.iota.works. Also you can set up your own full node and use it instead of an external one.
final String protocol = "https";
final String url = "tuna.iotasalad.org";
final String port = "14265";
IotaAPI iotaServer = new IotaAPI.Builder().protocol(protocol).host(host).port(port).build();
2) Covert your text into trytes
String trytes = TrytesConverter.toTrytes("my text");
3) Prepare and send transaction to tangle
private static final String SEED = "IHDEENZYITYVYSPKAURUZAQKGVJERUZDJMYTANNZZGPZ9GKWTEOJJ9AAMXOGZNQLSNMFDSQOTZAEETA99";//just a random one
private static final int MIN_WEIGHT_MAGNITUDE = 14;
private static final int DEPTH = 9;
private static final int TAG = "mytag"; //optional
String tangleHash = prepareTransfer(createAddress(), trytes);
public String createAddress() throws ArgumentException {
GetNewAddressResponse res = iotaServer.getNewAddress(SEED, 2, 0, false, 1, false);
return res.getAddresses().get(0);
}
public String prepareTransfer(String address_seclevel_2, String trytes) throws ArgumentException {
List<Transfer> transfers = new ArrayList<Transfer>();
transfers.add(new Transfer(address_seclevel_2, 0, trytes, TAG));
SendTransferResponse str = iotaServer.sendTransfer(SEED, 2, DEPTH, MIN_WEIGHT_MAGNITUDE, transfers, null,
null, false, false);
if(str.getSuccessfully()!=null){
//Transfer successfully!
for(Transaction tx: str.getTransactions()) {
return tx.getHash();
}
}
return "Handle error here. Something went wrong!";
}
Writing a WP8 Silverlight app. Is there a standard .NET technique available in this environment I can use to serialize an object like this
private static List<MemoryStream> MemoryStreamList = new List<MemoryStream>();
to save it to a file and restore it later?
I tried to use DataContractJsonSerializer for this which is good to serialize a List of simple custom objects, but it fails for List (I get System.Reflection.TargetInvocationException).
I would suggest converting your list to a list of byte arrays before persisting and then you should be able to serialize. Of course this comes with some overhead at deserialization as well.
Serialization part:
byte[] bytes = null;
var newList = MemoryStreamList.Select(x => x.ToArray()).ToList();
XmlSerializer ser = new XmlSerializer(newList.GetType());
using (var ms = new MemoryStream())
{
ser.Serialize(ms, newList);
//if you want your result as a string, then uncomment to lines below
//ms.Seek(0, SeekOrigin.Begin);
//using (var sr = new StreamReader(ms))
//{
//string serializedStuff = sr.ReadToEnd();
//}
//else you can call ms.ToArray() here and persist the byte[]
bytes = ms.ToArray();
}
Deserialization part:
using (var ms = new MemoryStream(bytes))
{
var result = ser.Deserialize(ms) as List<byte[]>;
}
I have an object that reads data from an Excel file using, which takes a IDbConnection, IDbDataAdapter and an IDbCommand. I use the adapters fill method to populate a table with data, and this is how I am currently mocking it:
[TestCase]
public void TestReadCellsFromSpreadsheetReadsSuccessfully()
{
var cells = new List<ReportData>
{
new ReportData { CellId = 1, ExcelCellLocation = "A1"},
new ReportData { CellId = 2, ExcelCellLocation = "A2"},
new ReportData { CellId = 3, ExcelCellLocation = "A3"},
new ReportData { CellId = 4, ExcelCellLocation = "A4"}
};
_mockAdapter.Setup(a => a.Fill(It.IsAny<DataSet>()))
.Callback((DataSet ds) =>
{
if (ds.Tables["Table"] == null)
{
ds.Tables.Add("Table");
ds.Tables["Table"].Columns.Add(new DataColumn());
}
var row = ds.Tables["Table"].NewRow();
row[0] = "Test";
ds.Tables["Table"].Rows.Add(row);
});
var excelReader = new ExcelReader(_mockConnection.Object, _mockAdapter.Object, _mockCommand.Object);
excelReader.ReadCellsFromSpreadsheet("Deal Summary", cells);
_mockCommand.VerifySet(c => c.CommandText = It.IsAny<string>(), Times.Exactly(cells.Count));
_mockAdapter.VerifySet(a => a.SelectCommand = _mockCommand.Object, Times.Exactly(cells.Count));
_mockAdapter.Verify(a => a.Fill(It.IsAny<DataSet>()), Times.Exactly(cells.Count));
}
This implementation works, but I feel like I'm doing too much to Mock the adapter... is there a better way to do this?
Do not pass those 3 objects as parameters. Instead pass IDataReader, IDataProvider or sth like that that returns data. Then You just mock this object. And you don't need reference to System.Data in project containing ExcellReader.
And two other things I don't like about your code.
Why TestCase instead of Test?
Are you sure you want to create command and fill dataset for each column separately? (but maybe I don't understand your code)
In general, I have some rules about data access:
Write a simple class that wraps all the data access logic, so that other classes don't have to deal with DataAdapters and all that crap.
When you write unit tests, don't mock out DataAdapters; instead just mock out the wrapper classes that you just created.
Make the data access wrapper logic so simple that it doesn't really need to be unit tested. If it DOES need to be tested, then write integration tests that hit a small sample database.