Lua RPC and userdata - c++

I'm currently using luarpc in my program to make interprocess communication. The problem now is that due to my tolua++ binding which stores class instances as userdata im unable to use any of those functions cause luarpc cant handle userdata. My question now is if would be possible (and how) to transmit userdata if you know that its always only a pointer (4 Bytes) and has a metatable attached for call and indexing operations.

You can't.
It doesn't matter if the userdata is a pointer or an object. The reason you can't arbitrarily RPC through them is because the data is not stored in Lua. And therefore LuaRPC cannot transmit it properly.
A pointer into your address space is absolutely worthless for some other process; even moreso if it's running on another machine. You have to actually transmit the data itself to make the RPC work. LuaRPC can do this transmission, but only for data that it can understand. And the only data it understands is data stored in Lua.

Ok i got it working now. What i did is for userdata args/returns i send the actual ptr + metatable name(typename) to the client. the client then attaches a metatable with an __index method that creates a new helper with the typename and appends a helper with the field you want to access. when you then call or read a field from that userdata the client sends the data for calling a field of the typetable and the userdata to the server.
ReadVariable:
lua_pushlightuserdata(L,msg.read<void*>());
#ifndef RPC_SERVER
luaL_getmetatable(L,"rpc.userdata");
int len = msg.read<int>();
char* s = new char[len];
msg.read((uint8*)s,len);
s[len] = '\0';
lua_pushlstring(L,s,len);
lua_setfield(L,-2,"__name");
lua_pushlightuserdata(L,TlsGetValue(transporttls));
lua_setfield(L,-2,"__transport");
lua_setmetatable(L,-2);
#endif
Write Variable:
else
{
msg.append<RPCType>(RPC_USERDATA);
msg.append<void*>(lua_touserdata(L,idx));
#ifdef RPC_SERVER
lua_getmetatable(L,idx);
lua_rawget(L,LUA_REGISTRYINDEX);
const char* s = lua_tostring(L,-1);
int len = lua_strlen(L,-1);
msg.append<int>(len);
msg.append(s,len);
#endif
lua_settop(L,stack_at_start);
}
userdata indexing:
checkNumArgs(L,2);
ASSERT(lua_isuserdata(L,1) && isMetatableType(L,1,"rpc.userdata"));
if(lua_type(L,2) != LUA_TSTRING)
return luaL_error( L, "can't index a handle with a non-string" );
const char* s = lua_tostring(L,2);
if(strlen(s) > MAX_PATH - 1)
return luaL_error(L,"string to long");
int stack = lua_gettop(L);
lua_getmetatable(L,1);
lua_getfield(L,-1,"__name");
const char* name = lua_tostring(L,-1);
if(strlen(name) > MAX_PATH - 1)
return luaL_error(L,"string to long");
lua_pop(L,1); // remove name
lua_getfield(L,-1,"__transport");
Transport* t = reinterpret_cast<Transport*>(lua_touserdata(L,-1));
lua_pop(L,1);
Helper* h = Helper::create(L,t,name);
Helper::append(L,h,s);
return 1;
well i more or less rewrote the complete rpc library to work with named pipes and windows but i think the code should give anyone enough information to implement it.
this allows code like:
local remote = rpc.remoteobj:getinstance()
remote:dosmthn()
on the clientside. it currently doesnt allow to add new fields but well this is all i need for now :D

Related

Adobe String Memory Leak - Where to Call External Library Entry Point to Free Memory?

I've created an After Effects script that extracts data from JSON files downloaded from an HTTPS URL. The problem is with the C++ DLL I've coded to download it and pass it back to the script. Even though it has been working fine, there was one instance of memory leak - After Effects issued a popup saying, "STRING MEMORY LEAK".
I'm new to C++ but I've managed to compose a DLL that downloads the files based on the examples provided with the After Effects installation (samplelib and basicexternalobject) as well as by Microsoft's C++ documentation. The Adobe JavaScript Tools Guide says that the method "ESFreeMem()" must be "called to free memory allocated for a null-terminated string passed to or from library functions". The problem is I don't know how or where to use it. I'm using After Effects CC 15.0.0 (build 180) on Windows 7.
This is the C++ function that gets some parameters from the javascript caller and returns a string with the JSON contents. If it fails it returns a bool (FALSE) so that the script can do what is necessary in this case.
extern "C" TvgAfx_Com_API long DownloadJson(TaggedData* argv, long argc, TaggedData * result)
{
//... first I check the arguments passed
// The returned value type
result->type = kTypeString;
//Converts from string into LPCWSTR ---------------------------------------------------
std::wstring stemp = s2ws(argv[0].data.string);
LPCWSTR jsonLink = stemp.c_str();
std::wstring stemp02 = s2ws(argv[1].data.string);
LPCWSTR jsonHeader = stemp02.c_str();
//--------------------------------------------------------------------------------------
//Class that does the HTTP request
WinHttpClient client(jsonLink, jsonHeader);
//Synchronous request
if (client.SendHttpsRequest())
{
string httpResponse = client.GetHttpResponse();
if (httpResponse.length() > 0)
{
//Sends response string back to javascript
result->data.string = getNewBuffer(httpResponse);
}
else
{
//Sends FALSE back to javascript
result->type = kTypeBool;
result->data.intval = 0;
}
}
else
{
//Sends FALSE back to javascript
result->type = kTypeBool;
result->data.intval = 0;
}
return kESErrOK;
}
The class WinHttpClient that does the actual request frees the memory allocated to the buffer that holds the response. Here's a piece of code:
// Read the data.
ZeroMemory(pszOutBuffer, dwSize + 1);
if (!WinHttpReadData(hRequest, (LPVOID)pszOutBuffer, dwSize, &dwDownloaded))
{
//Log error
}
else
{
resource.append(pszOutBuffer).c_str();
}
// Free the memory allocated to the buffer.
delete[] pszOutBuffer;
This is the function that the Adobe example uses to hold the string that will be returned to javascript:
//brief Utility function to handle strings and memory clean up
static char* getNewBuffer(string& s)
{
// Dynamically allocate memory buffer to hold the string
// to pass back to JavaScript
char* buff = new char[1 + s.length()];
memset(buff, 0, s.length() + 1);
strcpy(buff, s.c_str());
return buff;
}
Now, the manual says this method must be implemented:
/**
* \brief Free any string memory which has been returned as function result.
* JavaScipt calls this function to release the memory associated with the string.
* Used for the direct interface.
*
* \param *p Pointer to the string
*/
extern "C" SAMPLIB void ESFreeMem (void* p)
{
if (p)
free (p);
}
What I understand from this is that the memory associated with the json string returned must be released. But didn't the request class already do it? I just don't know where to call this method and what to pass on to it. I would appreciate any help. Thanks a lot!
When you create a DLL ExternalObject for After Effects, it's really just an interface that you call via some ExtendScript script. You have to load the ExternalObject in the AE Script and once loaded, the methods / functions you create in the C++ class can be called from the script.
You have to know how to load the DLL in an ExtendScript script file. Then, you can call the methods of the DLL.

BerkeleyDB Db->get not working when using custom comparison function

In a C++ program I am trying to set a custom comparison function for a Berkeley DB, using the Db::set_bt_function member function (the DB is opened as a BTREE type). My code works fine when I'm not changing the comparison function; I can put and get keys/values using Db::put and Db::get.
To try the set_bt_function method, I defined my own "lexicographic comparison" as follows:
int compkeys(Db *db, const Dbt *dbt1, const Dbt *dbt2, size_t *locp) {
size_t s = dbt1->get_size() > dbt2->get_size() ? dbt2->get_size() : dbt1->get_size();
int c = std::memcmp(dbt1->get_data(), dbt2->get_data(), s);
if(c != 0) return c;
if(dbt1->get_size() < dbt2->get_size()) return -1;
if(dbt1->get_size() > dbt2->get_size()) return 1;
return 0;
}
So this should lead to exactly the same behavior as my reference code, when the comparison function isn't changed, since by default Berkeley DB uses lexicographical order.
Yet, when using this comparison function, Db::get doesn't work anymore. It returns -30999 (DB_BUFFER_SMALL).
Here is what I am doing to get the value associated with a given key:
Db* _dbm = ... /* DB is open */
std::vector<char> mykey;
... /* mykey is set to some content */
Dbt db_key((void*)(mykey.data()), uint32_t(mykey.size()));
Dbt db_data;
db_key.set_flags(DB_DBT_USERMEM);
db_data.set_flags(DB_DBT_MALLOC);
int status = _dbm->get(NULL, &db_key, &db_data, 0);
... /* check status, do something with db_data */
free(db_data.get_data());
Any idea why this code works when I'm not setting the comparison function, and doesn't when I am?
Note: if I access key/values using a cursor (Dbc::get) I don't have this issue.
The DB_BUFFER_SMALL error in this case is complaining about your db_key Dbt. You need to call db_key.set_ulen(uint32_t(mykey.size())) to tell BDB how much space you've allocated to hold the keys that come out of the database.
Things get a little weirder when you're using a custom comparison function. You can have data in the key that's not part of the compare - and not in the key that you passed in to get(). For this reason, BDB returns the key it found in the database in your db_key.
When setting the ulen, make it large enough to hold any key that can come back from the database. You may find that it's saner to just keep a char array on the stack to deal with this key in/out behavior.

BluetoothGATTSetCharacteristicValue returns E_INVALIDARG or ERROR_INVALID_FUNCTION

I have build a set C++ containing classes on top of the BluetoothAPIs apis.
I can enumerate open handles to services, characteristics and descriptors. I can read characteristic values. The issue that I have is that I cannot write to a characteristic value.
Below is the code use to write the characteristic value
void BleGattCharacteristic::setValue(UCHAR * data, ULONG size){
if (pGattCharacteristic->IsSignedWritable || pGattCharacteristic->IsWritable || pGattCharacteristic->IsWritableWithoutResponse)
{
size_t required_size = sizeof(BTH_LE_GATT_CHARACTERISTIC_VALUE) + size;
PBTH_LE_GATT_CHARACTERISTIC_VALUE gatt_value = (PBTH_LE_GATT_CHARACTERISTIC_VALUE)malloc(required_size);
ZeroMemory(gatt_value, required_size);
gatt_value->DataSize = (ULONG)size;
memcpy(gatt_value->Data, data, size);
HRESULT hr = BluetoothGATTSetCharacteristicValue(bleDeviceContext.getBleServiceHandle(), pGattCharacteristic, gatt_value, NULL, BLUETOOTH_GATT_FLAG_NONE);
free(gatt_value);
if (HRESULT_FROM_WIN32(S_OK) != hr)
{
stringstream msg;
msg << "Unable to write the characeristic value. Reason: ["
<< Util.getLastError(hr) << "]";
throw BleException(msg.str());
}
}
else
{
throw BleException("characteristic is not writable");
}}
The call to bleDeviceContext.getBleServiceHandle() returns the open handle to the device info service.
pGattCharacteristics is the pointer to the characteristic to write too. It was opened with a call to BluetoothGATTGetCharacteristics.
I have tried different combinations of the flags with no difference in the return code.
I have also tried using the handle to the device not to the service. In that case I get an ERROR_INVALID_FUNCTION return error code.
I would appreciate any pointers as to what I am doing wrong or what other possible options I could try.
1- You have to use the Service Handle, right.
2- I don't know how you designed your class, and then how you allocate some memory for the Characteristic's Value itself.
What I do (to be sure to have enough and proper memory for Value's data):
a) at init of the Value object, call ::BluetoothGATTGetCharacteristicValue twice, to get the needed size and then actually allocate some internal memory for it.
b) when using it, set the inner memory to what it may , then call ::BluetoothGATTSetCharacteristicValue
hr=::BluetoothGATTSetCharacteristicValue(
handle,
(PBTH_LE_GATT_CHARACTERISTIC)Characteristic,
value,//actually a (PBTH_LE_GATT_CHARACTERISTIC_VALUE) to allocated memory
0,//BTH_LE_GATT_RELIABLE_WRITE_CONTEXT ReliableWriteContext,
BLUETOOTH_GATT_FLAG_NONE)
So a few things:
typedef struct _BTH_LE_GATT_CHARACTERISTIC_VALUE {
ULONG DataSize;
UCHAR Data[];
} BTH_LE_GATT_CHARACTERISTIC_VALUE, *PBTH_LE_GATT_CHARACTERISTIC_VALUE;
is how the data structure used in the parameter CharacteristicValue is defined. Please note that Data is NOT an allocated array, but rather a pointer. So accessing Data[0] is undefined behavior and could be accessing anywhere in memory. Rather you need to do gatt_value.Data = &data; setting the pointer to the address of the input parameter.
Secondly the documentation is quite clear as to why you might get ERROR_INVALID_FUNCTION; if another reliable write is already pending then this write will fail. You should consider retry logic in that case.
As for E_INVALIDARG I'd assume it's related to the undefined behavior but I'd check after fixing the other issues previously mentioned.

In lua is there a way to bind an upvalue to a userdata value instead of a function?

In the following example a userdata value is created of type MyType and a table is created with a metafunction __tostring which calls LI_MyType__tostring. The code creates a closure-based lua OOP. My gripe with the example provided is it appears as though there is only one way to associate userdata with a method call, via upvalues. In and of itself, this isn't problematic unless I want to share the same metatable across instances.
In an ideal world - and what I'm hoping to unearth with this question - is there a way to associate an upvalue with a value (e.g. userdata) without associating it with a function call via an upvalue? I'm hoping there is a trick that will let me continue to use closure-based lua OOP and share the same metatable across instances. I'm not optimistic, but I figured I'd ask to see if someone has a suggestion or a non-obvious trick.
using FuncArray = std::vector<const ::luaL_Reg>;
static const FuncArray funcs = {
{ "__tostring", LI_MyType__tostring },
};
int LC_MyType_newInstance(lua_State* L) {
auto userdata = static_cast<MyType*>(lua_newuserdata(L, sizeof(MyType)));
new(userdata) MyType();
// Create the metatable
lua_createtable(L, 0, funcs.size()); // |userdata|table|
lua_pushvalue(L, -2); // |userdata|table|userdata|
luaL_setfuncs(L, funcs.data(), 1); // |userdata|table|
lua_setmetatable(L, -2); // |userdata|
return 1;
}
int LI_MyType__tostring(lua_State* L) {
// NOTE: Blindly assume that upvalue 1 is my userdata
const auto n = lua_upvalueindex(1);
lua_pushvalue(L, n); // |userdata|
auto myTypeInst = static_cast<MyType*>(lua_touserdata(L, -1));
lua_pushstring(L, myTypeInst->str()); // |userdata|string|
return 1; // |userdata|string|
}
I'm hoping there's a way of performing something like (this is pseudo-code!):
// Assume that arg 1 is userdata
int LI_MyType__tostring(lua_State* L) {
const int stackPosition = -1;
const int upvalueIndex = 1;
const auto n = lua_get_USERDATA_upvalue(L, stackPosition, upvalueIndex);
lua_pushvalue(L, n); // |userdata|
auto myTypeInst = static_cast<MyType*>(lua_touserdata(L, -1));
lua_pushstring(L, myTypeInst->str()); // |userdata|string|
return 1; // |userdata|string|
}
I know this is similar to how things would be for the "normal" metatable style of OOP, but I want to keep things closure based and avoid introducing the colon syntax.
Another way of asking this question would be, is there a way to share metatables across userdata instances while using a closure-based OOP? Using lua's syntax from the scripting side of things, I don't think it's possible, but I'm hoping there's something that can be done on the C side of things.
UPDATE (2013-10-10): Based on #lhf's answer to use lua_setuservalue() and lua_getuservalue() the protocol I've settled on which allows me to reuse metatables is this:
Register a single metatable object using luaL_newmetatable(). This metatable can now be shared across userdata instances because no upvalues are used when registering the metatable.
Create a userdata value (lua_newuserdata()).
Assign the correct metatable to the userdata value (lua_setmetatable()).
Create and populate an instance method calls/attributes table with one upvalue, the userdata.
Use lua_setuservalue() on userdata to store a reference to the per-instance attribute/method table.
Change various metamethods (e.g. __index) to use the userdata's uservalue table.
As a consequence:
upvalues are never used in metamethods
upvalues are only used in a value's instance methods
there is only one extra table per instance of a given class
It's still not possible to escape creating a method/attribute table per userdata, but that overhead is nominal. It would be nice if obj.myMethod() would pass obj to function myMethod() somehow without using :, but that's exactly what : does because this isn't possible another way (unless you do make use of an upvalue).
lua_setuservalue seems to be exactly what you need. There is also of course lua_getuservalue.
(I'm skipping the C++ code and answering the question in the title.)
I don't think you should be trying to do it exactly, for a few reasons.
If you call object.method(), and you try to infer the object from the instance that was created, you're blocking your ability to pass function pointers around that behave on any object given.
You have cyclic references to objects that will never get garbage collected (each instance's function pointing back to the instance).
Just get the object from slot 1, and check its type matches your userdata. (luaL_checkudata)
If its not an object and tostring is called for example, just output that its a class of object name, instead of the instance details. It makes far more sense, and may well make debugging simpler if the object reports what it actually is, rather than trying to be too clever and misleading you.

DTrace - How do I correctly retrieve initial parameters in the return probe

I'm trying to read the initial arguments that was passed into the function in my return probe. Unlike the entry probe, the argument variables(arg0,arg1,...) in the return probe do not contain the initial parameters, and I'm not sure how I would be able to retrieve those values.
Also, I would like to avoid storing the values in global variables because of the concurrency issues.
You can save the parameters in thread-local storage, e.g.
pid$target:foo:bar:entry
{
self->arg0 = arg0;
self->arg1 = arg1;
/*
* Avoid the race in which dtrace(1) attaches to the victim during
* the window between the two probes.
*/
self->trace = 1;
}
pid$target:foo:bar:return
/self->trace/
{
printf("arg0 = 0x%x, arg1 = 0x%x\n", self->arg0, self->arg1);
/* Deallocate the thread-local storage. */
self->arg0 = 0;
self->arg1 = 0;
}
As rmh answered - using local variables is the way to do it. Otherwise, dtrace would have to save the values for you on entry - and it doesnt know anything about the incoming arguments or your expectations, and would have to garbage collect. (Technically, it does know whats going to happen - eventually - but that would add complex overhead vs the local variables approach which is mapped to a simple set of virtual D instructions).