Accessing value from callback in ctypes - c++

I'm using Python/ctypes to write an app based on a commercial DLL. This DLL reads a flatfile and returns data through structs. The relevant C structs looks like this:
struct System{
unsigned short user;
unsigned short version;
};
struct Message {
unsigned short header;
unsigned short data;
unsigned int messageType;
};
The API that came with the DLL provides a pointer to a function with the following prototype. This function loops through the flatfile and calls Callback defined next:
typedef int (__stdcall *Process) (const char* filename, const char* base, unsigned int flags, int user, Callback stdcallback);
and the Callback prototype (the fifth argument above) is defined as:
typedef int (__stdcall *Callback) (const System* Sys, const Message* Msg);
In my python file:
from ctypes import *
# using WinDLL for stdcall
lib = WinDLL('CVO.dll')
class System(Structure):
_fields_ = [('user', c_ushort),
('version', c_ushort)]
class Message(Structure):
_fields_ = [('header', c_ushort),
('data', c_ushort),
('messageType', c_uint)]
PROTO = WINFUNCTYPE(c_int, POINTER(System), POINTER(Message))
def py_callbck(Sys, Msg):
print Msg._type_.messageType
# return 0 to read the next line in the flatfile
return 0
Callback = PROTO(py_callbck)
pProcess lib.Process
pProcess.argtypes = [c_char_p,c_char_p,c_uint,c_int,PROTO]
pProcess.restype = c_int
pProcess(c_char_p('flat.ccf'),c_char_p(None),c_uint(0),c_int(0),Callback)
My problem lies in accessing the value the DLL passes to the Msg struct. The code should be printing an c_int (0, 1, 2) but instead is returning a Field type like such:
<Field type=c_ulong, ofs=136, size=4>
I know the flatfile is being processed, because my Python code prints out hundreds of the <Field ...> statements, which it is supposed to do.
My question is how I can access the value the DLL is pushing to the Msg Structure instead of returning <Field ...> statements?
Note I'm relatively new to ctypes and have very little C so if you notice issues with defining anything, please let me know. For example, I know there is a difference between ctypes.pointer which returns and object and ctypes.POINTER which returns a new type. When I use ctypes.pointer the code errors out.

For being relatively new to ctypes you've done a pretty good job here. The _type_ attribute of a pointer is a reference to the pointed at data type. In this case it's Message. The class attribute messageType is the CField data descriptor used by an instance.
What you want to do instead is dereference the pointer to get a Message instance. You can either use a [0] subscript or the contents attribute:
def py_callbck(Sys, Msg):
print Msg[0].messageType # Msg.contents.messageType
# return 0 to read the next line in the flatfile
return 0
Remember to retain a reference to Callback to keep it from being garbage collected. Currently you reference it as a global. That's fine.
Regarding the way you're calling pProcess, it's generally unnecessary to manually create ctypes objects for simple types when you've defined argtypes. You can more simply use the following:
pProcess('flat.ccf', None, 0, 0, Callback)
But pay attention to whether the function needs a writable string buffer. In that case use create_string_buffer. It's unnecessary here since the char * parameters are both const.

Related

Import C++ function returning pointer of structs into VB.NET

I have created a simple DLL in C++, for the sole purpose of isolating the issue and testing on a small scale the importing of a function that returns a list of structs with members of different types.
dll_header.h
#ifdef MY_DLL_EXPORTS
#define MY_DLL_API __declspec(dllexport)
#else
#define MY_DLL_API __declspec(dllimport)
#endif
enum color_type
{
RGB = 1,
MONO = 2
};
struct my_struct
{
unsigned char *name;
char *value;
color_type type;
my_struct* next;
};
extern "C" struct my_struct MY_DLL_API * get_list(void);
dll_header.cpp
#include "dll_header.h"
MY_DLL_API my_struct * get_list(void)
{
my_struct my_list[2];
unsigned char name1[] = "name1";
unsigned char name2[] = "name2";
char val1[] = "val1";
char val2[] = "val2";
my_list[0].name = name1;
my_list[0].value = val1;
my_list[0].type = RGB;
my_list[0].next = &my_list[1];
my_list[0].name = name2;
my_list[0].value = val2;
my_list[0].type = MONO;
my_list[0].next = NULL;
return my_list;
}
Like I said, I have to use these data types (cannot change them to strings or anything else because I am testing something and I need them like this)
Now, in my VB.NET application I import and try to retrieve the list like this from the DLL
Form1.vb
Imports System.Runtime.InteropServices
Imports System.Text
Public Class Form1
Public Enum color_type
RGB = 1
COLOR = 2
End Enum
Private Structure my_struct
Public name As Byte()
Public value As Char()
Public type As color_type
Public next As IntPtr
End Structure
Private Declare Function get_list Lib "my_lib.dll" () As IntPtr
Private my_list As List(Of my_struct)
Private Sub get_list()
Dim my_list_pointer As IntPtr = get_list()
my_list = New List(Of my_struct)
Dim my_item As my_struct
While my_list_pointer <> IntPtr.Zero
my_item = CType(Marshal.PtrToStructure(my_list_pointer, GetType(my_struct)), my_struct)
my_list.Add(my_item)
my_list_pointer = my_item.next
End While
End Sub
I have tried many other methods, specially changing data types but so far here I am, stuck with this exception when trying to run the code:
System.AccessViolationException: 'Attempted to read or write protected
memory. This is often an indication that other memory is corrupt.'
Trying to find a way to make these two understand each other
Either use mixed mode C++ to do the translation between native and managed types or use types that are compatible with P/Invoke.
Looking how Win32 define structure (and function that use those) is a good start to help you understand how to define compatible structures as one of the main intent of P/Invoke is to be able to use Win32 API from managed code.
If you go for mixed mode, then you can do whatever required translation between both world as you can mix native C++ and C++/CLI in the same assembly. So essentially, you would write code that convert the native structure into a managed one and then VB.NET will be able to use that managed code (assuming that you use types available in VB.NET for example).
Well, if you go for the mixed mode route, you generally ends up with 3 DLLs/Assemblies/Executable as you would have the original C++ DLL (native code), the VB.NET code (managed) and the mixed mode C++ assembly in between.
Update
While it could be relatively easy to provide code if the list is read only, if this is not the case, it could be much harder depending how the code is expected to be used and what are the availaible function in the existing DLL.
The starting point would be to create a ref class in C++/CLI.
Some similar questions
https://bytes.com/topic/c-sharp/answers/674468-passing-linked-list-c-dll-c-pinvoke
Return list of points (x,y,z) from C to C# using PInvoke
PInvoke of self referential struct from C++
Problem with your code
Your exported function retuns a pointer to a local variable which is undefined behavior.
Additional observation
If your original list is an array, the why making it a list also? An array would be easier to marshal and use. And probably perform better too.
When defining you structure, you have be explicit on how strings are marshalled and the alignment that should be used. You shoull validate that everything is as expected.

How to use custom type function pointer in C++

Condition
I use a framework that has an custom type as bellow:
typedef log (*CustomType) (
int timeStamp,
const char* data,
int dataSize,
void* userData,
int dataType,
int viewId
)
and MyClass init method as bellow:
MyClass_Init (void **output, CustomType video, CustomType audio, void* userData)
Question
I used init method like bellow but always receive error (error content is not displayed because i use a framework). pls point me what is missed.
CustomType videoInput;
CustomType audioInput;
void *output = malloc(sizeof(void*);
void *userData = malloc(sizeof(void*));
long result = MyClass_Init(&output, videoInput, audioInput, userData);
A number of things wrong with this code:
You can't intermix function pointers and method pointers. What it boils down to is that the this for a method has to be included in the method call signature. Since the function pointer doesn't include the this pointer (it is a function, not a method pointer), the two can not match.
Most C-based API includes some sort of reference value (most frameworks call those refCon, context or userData), so what you can do is create an adapter function that calls your method. The userData parameter in your CustomType parameter list looks like it is one of those (consult the docs to be sure).
You can probably provide a userData wherever you set MyClass_Init as your callback now. So, if that function to provide a callback to the library was called set_callback( MyCustomType callback, void* userData ), do something like
MyClass *obj = new MyClass; // Or however you create your object
set_callback( MyClassCallbackAdapterFunction, obj );
with an adapter function like:
log MyClassCallbackAdapterFunction( int timeStamp, const char* data, int dataSize, void* userData, int dataType, int viewId )
{
MyClass *myThis = (MyClass*) userData;
// Here you can now call myThis->MyClass_Init( ... ) however you want to.
}
The malloc( sizeof(void*) ) statements look like you're misunderstanding return parameters (also called "side effects" by some teachers). I don't have the docs to whatever API/library you're using, but I'm pretty certain you're supposed to not just pass in buffers the size of a pointer. Either you'd just provide a pointer on the stack in which a buffer will be returned, or you provide a whole buffer (e.g. an array) and its size, and that is where the callback will write to or so.

creating a callback struct to pass into the LuaJIT FFI

So first I load in a DLL I need
local ffi = require("ffi")
local theDLL = ffi.load("thisDLL")
in the ffi cdef I have two different kinds of structs
ffi.cdef [[
typedef struct StructSession StructSession;
typedef struct {
/*
* begin_proj callback
*/
bool (__cdecl *begin_proj)(char *proj);
/*
* save_proj_state
*/
bool (__cdecl *save_proj_state)(unsigned char **buffer, int *len);
} StructCallbacks;
I also have this function in the cdef
__declspec(dllexport) int __cdecl start_session(StructSession **session,
StructCallbacks *cb);
Now I would like to call this function
print(theDLL.start_session(a,b))
vars a and b are obviously placeholders, the question is how can I pass the structs the function needs? And say we get StructSession working, is making a callback to a function WITHIN LuaJIT even going to be possible for StructCallbacks?
Creating the StructCallbacks is easy; you can create it with ffi.new and create FFI callbacks for the fields (see the FFI semantics for information on callbacks).
Creating the StructSession is trickier since it's an opaque type, but it's not much different from how you would do it in C.
Here is how you would create one in C:
StructSession* S = NULL;
start_session(*S, foo);
Notice how you are not directly allocating a StructSession. Instead, you allocate a pointer to one, and let start_session allocate the actual struct.
So we now translate this to LuaJIT code:
local S = ffi.new("StructSession*")
lib.start_session(getPointer(S), foo) -- getPointer should take the pointer of S, but...
...the FFI doesn't provide any way to take the pointer of an object (This is intentional; it allows for optimizations).
So how do we get a pointer to a StructSession? Well, recall that arrays are convertible to pointers, and we can access those through the FFI. So we instead create a single-slot array of pointers and pass that to start_session:
local S_slot = ffi.new("StructSession*[1]")
lib.start_session(S_slot, foo)
local S = S_slot[0]
And now you have a StructSession object.

Correct __bridge usage with ARC and C++ interop? (how to avoid memory leaks?)

I have a bit of pure C++ code which is reading from Objective-C data structures with the help of a function pointer to a method in an Objective C class. I'm treating the Objective-C class instance to read from as an opaque pointer. For example, the C++ method that does the reading has a signature like this:
typedef void(*DataGetterFunc)(void * dataSource, int key, int * outValue);
...
void readData(void * dataSource, DataGetterFunc dataReadingFunc);
When I call the C++ method from Objective-C, I do the following:
MYDataStructure * objectiveCData;
cppObject->readData((__bridge void*)objectiveCData, DataGetterFuncImpl);
Finally, DataGetterFuncImpl dereferences the Objective-C class like so:
void DataGetterFuncImpl(void * dataSource, int key, int * outValue)
{
MYDataStructure * objCData = (__bridge MYDataStructure*)dataSource;
...
}
Originally in DataGetterFuncImpl I was using __bridge_transfer, but then I was getting EXC_BAD_ACCESS the next time ARC called retain on MYDataStructure, So I assumed it was being over-released by the use of __bridge_transfer and changed it to just __bridge.
Are there any memory leaks I should look for by just using __bridge, or do I need to use some combination of __bridge_retain and __bridge_transfer in this case?
When you're using __bridge to convert to or from objc, owership is just not affected. That means, that while you're using the object in C++ you must make sure that there's still a strong reference around.
If you, on the other hand, use __bridge_retain to convert to void* and __bridge_transfer to convert back to id (or any other retainable object type), you must make sure that each __bridge_retain is matched by exactly one __bridge_transfer later.

How can I bind a C/C++ structure to Ruby?

I need some advice how can I bind a C/C++ structure to Ruby. I've read some manuals and I found out how to bind class methods to a class, but I still don't understand how to bind structure fields and make them accessible in Ruby.
Here is the code I'm using:
myclass = rb_define_class("Myclass", 0);
...
typedef struct nya
{
char const* name;
int age;
} Nya;
Nya* p;
VALUE vnya;
p = (Nya*)(ALLOC(Nya));
p->name = "Masha";
p->age = 24;
vnya = Data_Wrap_Struct(myclass, 0, free, p);
rb_eval_string("def foo( a ) p a end"); // This function should print structure object
rb_funcall(0, rb_intern("foo"), 1, vnya); // Here I call the function and pass the object into it
The Ruby function seems to assume that a is a pointer. It prints the numeric value of the pointer instead of it's real content (i.e., ["Masha", 24]). Obviously the Ruby function can't recognize this object —I didn't set the object's property names and types.
How can I do this? Unfortunately I can't figure it out.
You have already wrapped your pointer in a Ruby object. Now all you have to do is define how it can be accessed from the Ruby world:
/* Feel free to convert this function to a macro */
static Nya * get_nya_from(VALUE value) {
Nya * pointer = 0;
Data_Get_Struct(value, Nya, pointer);
return pointer;
}
VALUE nya_get_name(VALUE self) {
return rb_str_new_cstr(get_nya_from(self)->name);
}
VALUE nya_set_name(VALUE self, VALUE name) {
/* StringValueCStr returns a null-terminated string. I'm not sure if
it will be freed when the name gets swept by the GC, so maybe you
should create a copy of the string and store that instead. */
get_nya_from(self)->name = StringValueCStr(name);
return name;
}
VALUE nya_get_age(VALUE self) {
return INT2FIX(get_nya_from(self)->age);
}
VALUE nya_set_age(VALUE self, VALUE age) {
get_nya_from(self)->age = FIX2INT(age);
return age;
}
void init_Myclass() {
/* Associate these functions with Ruby methods. */
rb_define_method(myclass, "name", nya_get_name, 0);
rb_define_method(myclass, "name=", nya_set_name, 1);
rb_define_method(myclass, "age", nya_get_age, 0);
rb_define_method(myclass, "age=", nya_set_age, 1);
}
Now that you can access the data your structure holds, you can simply define the high level methods in Ruby:
class Myclass
def to_a
[name, age]
end
alias to_ary to_a
def to_s
to_a.join ', '
end
def inspect
to_a.inspect
end
end
For reference: README.EXT
This is not a direct answer to your question about structures, but it is a general solution to the problem of porting C++ classes to Ruby.
You could use SWIG to wrap C/C++ classes, structs and functions. In the case of a structure, it's burning a house to fry an egg. However, if you need a tool to rapidly convert C++ classes to Ruby (and 20 other languages), SWIG might be useful to you.
In your case involving a structure, you just need to create a .i file which includes (in the simplest case) the line #include <your C++ library.h>.
P.S. Once more, it's not a direct answer to your question involving this one struct, but maybe you could make use of a more general solution, in which case this may help you.
Another option is to use RubyInline - it has limited support for converting C and Ruby types (such as int, char * and float) and it also has support for accessing C structurs - see accessor method in the API.