cython create string with nogil - c++

I would like to create a c++ string in a nogil function in a file that would be cimported via pxd. If I define, string output = "" or string output = string("blah"), this uses the python interpreter. Is there a way to define the a string so that the compiler writes in the cython cpp file:
std::string val = "blah";
Basically have this:
from libcpp.string cimport string
cdef string my_func() nogil:
cdef:
string output = "blah"
....
return output

%%cython -a
#distutils: language = c++
from libcpp.string cimport string
cdef string my_func() nogil:
cdef:
char* c_str = 'blah'
string output = <string>(c_str)
return output
def py_call():
return my_func()
Then calling py_call() gives b'blah', i.e. a bytes object.
EDIT: Here's the generated C++ code:
+08: char* c_str = 'blah'
__pyx_v_c_str = ((char *)"blah");
+09: string output = <string>(c_str)
__pyx_v_output = ((std::string)__pyx_v_c_str);
So it literally casts char* to std::string.
An alternative is then to invoke the constructor from char*:
cdef:
char* c_str = 'blah'
string output = string(c_str)
which generates
+08: char* c_str = 'blah'
__pyx_v_c_str = ((char *)"blah");
+09: string output = string(c_str, 4)
try {
__pyx_t_1 = std::string(__pyx_v_c_str, 4);
} catch(...) {
#ifdef WITH_THREAD
PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();
#endif
__Pyx_CppExn2PyErr();
#ifdef WITH_THREAD
PyGILState_Release(__pyx_gilstate_save);
#endif
__PYX_ERR(0, 9, __pyx_L1_error)
}
__pyx_v_output = __pyx_t_1;
which looks better.

Related

How to pass a callback to EM_ASM for c++?

I do not know how to pass the value asynchronously to EM_ASM, here's how I try to do it (but JS says it's not a function):
const auto testFunc = [] (const char* data)
{
printf("data: %s\n", data);
}
EM_ASM_(
{
var funcResult = ($0);
var text = "data";
var lengthBytes = lengthBytesUTF8(text) + 1;
var stringOnWasmHeap = _malloc(lengthBytes);
stringToUTF8(text, stringOnWasmHeap, lengthBytes);
// exception thrown: TypeError: funcResult is not a function
funcResult(stringOnWasmHeap);
}, testFunc);
The documentation says that you can use a function (em_str_callback_func) of the em_str_callback_func type. But it doesn't say how to use it.
https://emscripten.org/docs/api_reference/emscripten.h.html
I don't know about passing callbacks, but if what you want to do is to return a value from JS, then the docs have such an example: you have to use EM_ASM_INT instead:
int x = EM_ASM_INT({
console.log('I received: ' + $0);
return $0 + 1;
}, 100);
printf("%d\n", x);
The API reference has another example for returning a string:
char *str = (char*)EM_ASM_INT({
var jsString = 'Hello with some exotic Unicode characters: Tässä on yksi lumiukko: ☃, ole hyvä.';
var lengthBytes = lengthBytesUTF8(jsString)+1;
// 'jsString.length' would return the length of the string as UTF-16
// units, but Emscripten C strings operate as UTF-8.
var stringOnWasmHeap = _malloc(lengthBytes);
stringToUTF8(jsString, stringOnWasmHeap, lengthBytes);
return stringOnWasmHeap;
});
printf("UTF8 string says: %s\n", str);
free(str); // Each call to _malloc() must be paired with free(), or heap memory will leak!
Once you have the value in C you can just call your testfunc directly.

How to convert PyObject to UTF-8 string?

I'm trying to pass a string value to a Python function and get a utf-8 string value. The function retrieves Japanese characters and returns characters in another language.
I used ctypes.windll.user32.MessageBoxW in the Python script and made sure there is no problem with the Python function. And I tried a simple function like return 'hello world' to test. I also checked every function with assert. I guess the problem starts with PyBytes_AsString. It always returns DB DB DB DB DB DB DB... (in hex). But I don't know how to fix this.
char* result;
PyObject* module, *func, *arg, *ret, *value;
Py_Initialize();
PyObject* sysPath = PySys_GetObject("path");
PyObject* path = PyUnicode_FromString(".");
PyList_Append(sysPath, path);
module = PyImport_ImportModule("test");
if (module != 0)
{
const wchar_t* w = L"翻訳テスト";
func = PyObject_GetAttrString(module, "translate");
arg = PyTuple_New(1);
value = PyUnicode_FromWideChar(w, wcslen(w));
PyTuple_SetItem(arg, 0, value);
ret = PyObject_CallObject(func, arg);
PyObject* repr = PyObject_Repr(ret);
PyObject* str = PyUnicode_AsEncodedString(repr, "utf-8", "strict");
result = PyBytes_AsString(str);
Py_DECREF(repr);
Py_DECREF(str);
Py_DECREF(module);
Py_DECREF(func);
Py_DECREF(arg);
Py_DECREF(ret);
fstream file("text.txt", std::ios::out);
file << result;
file.close();
}
Py_Finalize();
result = PyBytes_AsString(str) returns a pointer to the internal buffer of str, so don't Py_DECREF(str) before you write result to the file.

How to get a Java string from a char pointer in C++

I am porting the openvr sample to jogl, after we created the binding with jna.
Almost at the end (before rendering the controllers and the tracking base stations), I got stuck trying to translate a char pointer in C to a String in Java.
C++ code here:
//-----------------------------------------------------------------------------
// Purpose: Helper to get a string from a tracked device property and turn it
// into a std::string
//-----------------------------------------------------------------------------
std::string GetTrackedDeviceString( vr::IVRSystem *pHmd, vr::TrackedDeviceIndex_t unDevice, vr::TrackedDeviceProperty prop, vr::TrackedPropertyError *peError = NULL )
{
uint32_t unRequiredBufferLen = pHmd->GetStringTrackedDeviceProperty( unDevice, prop, NULL, 0, peError );
if( unRequiredBufferLen == 0 )
return "";
char *pchBuffer = new char[ unRequiredBufferLen ];
unRequiredBufferLen = pHmd->GetStringTrackedDeviceProperty( unDevice, prop, pchBuffer, unRequiredBufferLen, peError );
std::string sResult = pchBuffer;
delete [] pchBuffer;
return sResult;
}
GetStringTrackedDeviceProperty here:
/** Returns a string property. If the device index is not valid or the property is not a string type this function will
* return 0. Otherwise it returns the length of the number of bytes necessary to hold this string including the trailing
* null. Strings will generally fit in buffers of k_unTrackingStringSize characters. */
virtual uint32_t GetStringTrackedDeviceProperty( vr::TrackedDeviceIndex_t unDeviceIndex, ETrackedDeviceProperty prop, VR_OUT_STRING() char *pchValue, uint32_t unBufferSize, ETrackedPropertyError *pError = 0L ) = 0;
Where VR_OUT_STRING() is defined here as:
# define VR_CLANG_ATTR(ATTR)
#define VR_OUT_STRING() VR_CLANG_ATTR( "out_string: ;" )
I have already done something similar where I had to call a function that expect the pointer to an array of TrackedDevicePose_t structures:
private TrackedDevicePose_t.ByReference trackedDevicePosesReference = new TrackedDevicePose_t.ByReference();
public TrackedDevicePose_t[] trackedDevicePose
= (TrackedDevicePose_t[]) trackedDevicePosesReference.toArray(VR.k_unMaxTrackedDeviceCount);
I created first the reference and then from it the actual array.
But here I can't have a class extending the char array..
private String getTrackedDeviceString(IVRSystem hmd, int device, int prop, IntBuffer propError) {
int requiredBufferLen = hmd.GetStringTrackedDeviceProperty.apply(device, prop, Pointer.NULL, 0, propError);
if(requiredBufferLen == 0) {
return "";
}
CharArray.ByReference charArrayReference = new CharArray.ByReference();
char[] cs = charArrayReference.toArray(requiredBufferLen);
return null;
}
Where apply (here) is:
public interface GetStringTrackedDeviceProperty_callback extends Callback {
int apply(int unDeviceIndex, int prop, Pointer pchValue, int unBufferSize, IntBuffer pError);
};
CharArray class, crap attempt here
Any ideas?
I've done some porting of C and C++ code to Java, and while it's probably horribly hacky, the best I've come up with to solve cases where a pointer to an int primitive or a char*/String is needed for a function call, is to create a small wrapper class with a single property, pass that object into the function, change the property as needed, and retrieve the new value after the function call. So something like:
public class StringPointer {
public String value = "";
}
StringPointer pchBuffer = new StringPointer();
unRequiredBufferLen = pHmd.GetStringTrackedDeviceProperty( unDevice, prop, pchBuffer, unRequiredBufferLen, peError );
String sResult = pchBuffer.value;
and inside GetStringTrackedDeviceProperty()
...
pchValue.value = "some string";
...
In this case, you can use a String, since that's what your code is doing with the char* after the function call, but if it actually really needs to be a char[], you can just create char[] pchBuffer = new char[unRequiredBufferLen]; and pass that into the function. It will be just like you were using a char* in C++, and any changes you make inside the array will be visible after the function ends, and you can even do String sResult = new String(pchBuffer);.

Passing reference from Python to c++ function wrapped with swig for return value

Disclaimer, I am a swig and python noob
I have my own c++ library and I am wrapping it to use in python with swig.
My c++ class is like this:
public MyCppClass()
{
public:
void MyFunction(char* outCharPtr, string& outStr, int& outInt, long& outLong)
{
outCharPtr = new char[2];
outCharPtr[0] = "o";
outCharPtr[1] = "k";
outStr = "This is a result";
outInt = 1;
outLong = (long)12345;
}
}
Now I wrap this class using swig and say the module is called MyClass.
What I want to achieve in python is the following code (OR SAY PSEUDO CODE because if it was the code it would be working) and output:
import module MyClass
from MyClass import MyCppClass
obj = MyCppClass();
outCharPtr = "";
outStr = "";
outInt = 0;
outLong = 0;
obj.MyFunction(outCharPtr, outStr, outInt, outLong);
print(outCharPtr);
print(outStr);
print(outInt);
print(outLong);
The output I want is:
>>>Ok
>>>This is a result
>>>1
>>>12345
I am using python 3.4
I am really apologetic if this is something basic but I have already spent around 8 hours on the resolution and can't come up with anything.
Any help will be very appreciated.
Thanks.
Here's one way to do it. I made some modifications to your non-working example:
example.i
%module example
%include <std_string.i>
%include <cstring.i>
%cstring_bounded_output(char* outCharPtr, 256)
%apply std::string& OUTPUT {std::string& outStr};
%apply int& OUTPUT {int& outInt};
%apply long& OUTPUT {long& outLong};
%inline %{
class MyCppClass
{
public:
void MyFunction(char* outCharPtr, std::string& outStr, int& outInt, long& outLong)
{
outCharPtr[0] = 'o';
outCharPtr[1] = 'k';
outCharPtr[2] = '\0';
outStr = "This is a result";
outInt = 1;
outLong = (long)12345;
}
};
%}
Example use:
>>> import example
>>> c=example.MyCppClass()
>>> c.MyFunction()
['ok', 'This is a result', 1, 12345]

embedded python passing unicode strings

note everything works great if i use Ascii, but:
in my utf8-encoded script i have this:
print "frøânçïé"
in my embedded C++ i have this:
PyObject* CPython_Script::print(PyObject *args)
{
PyObject *resultObjP = NULL;
const char *utf8_strZ = NULL;
if (PyArg_ParseTuple(args, "s", &utf8_strZ)) {
Log(utf8_strZ, false);
resultObjP = Py_None;
Py_INCREF(resultObjP);
}
return resultObjP;
}
Now, i know that my Log() can print utf8 (has for years, very well debugged)
but what it actually prints is this:
print "frøânçïé"
frøânçïé
another method i use looks like this:
kj_commands.menu("控件", "同步滑帧", "全局无滑帧")
or
kj_commands.menu(u"控件", u"同步滑帧", u"全局无滑帧")
and in my C++ i have:
SuperString ScPyObject::GetAs_String()
{
SuperString str;
if (PyUnicode_Check(i_objP)) {
#if 1
// method 1
{
ScPyObject utf8Str(PyUnicode_AsUTF8String(i_objP));
str = utf8Str.GetAs_String();
}
#elif 0
// method 2
{
UTF8Char *uniZ = (UTF8Char *)PyUnicode_AS_UNICODE(i_objP);
str.assign(&uniZ[0], &uniZ[PyUnicode_GET_DATA_SIZE(i_objP)], kCFStringEncodingUTF16);
}
#else
// method 3
{
UTF32Vec charVec(32768); CF_ASSERT(sizeof(UTF32Vec::value_type) == sizeof(wchar_t));
PyUnicodeObject *uniObjP = (PyUnicodeObject *)(i_objP);
Py_ssize_t sizeL(PyUnicode_AsWideChar(uniObjP, (wchar_t *)&charVec[0], charVec.size()));
charVec.resize(sizeL);
charVec.push_back(0);
str.Set(SuperString(&charVec[0]));
}
#endif
} else {
str.Set(uc(PyString_AsString(i_objP)));
}
Log(str.utf8Z());
return str;
}
for the string, "控件", i get:
控件
for the unicode string, u"控件", Methods 1, 2, and 3, i get the same thing:
控件
okay so what am i doing wrong???