For me, the following all result in a segfault:
my_array->descr->subarray->shape;
my_array->dimensions;
PyArray_SHAPE(my_array);
PyArray_DIMS(my_array);
PyArray_ITEMSIZE(my_array);
PyArray_NBYTES(my_array);
My function looks like this:
static PyObject* exterior(PyObject* self, PyArrayObject* old_simplices_array)
{//code here
The rest of my cpp file looks like this:
#include "Python.h"
#include "numpy/arrayobject.h"
/* function */
static PyMethodDef compiled_methods[] =
{
{"_exterior",(PyCFunction)exterior , METH_VARARGS},
{NULL, NULL} /* Sentinel */
};
PyMODINIT_FUNC init_alto(void)
{
(void) Py_InitModule("_alto", compiled_methods);
import_array();
}
The python code that passes the array to "exterior" just passes an NxM uint array. That part works. I can access the array's strides and data. I just cannot determine the bounds of iteration. I am working from within sage if that makes any difference.
How am I supposed to iterate over an array without segfaulting? If the answer is obvious, please idiotproof your answer.
For a better idea of what the function looks like, see here.
In the past I have done the following to iterate over a PyArrayObject:
static PyObject *func1(PyObject *self, PyObject *args) {
PyArrayObject *X;
int ndX;
npy_intp *shapeX;
PyArray_Descr *dtype;
NpyIter *iter;
NpyIter_IterNextFunc *iternext;
PyArg_ParseTuple(args, "O!", &PyArray_Type, &X);
ndX = PyArray_NDIM(X);
shapeX = PyArray_SHAPE(X);
dtype = PyArray_DescrFromType(NPY_DOUBLE);
iter = NpyIter_New(X, NPY_ITER_READONLY, NPY_KEEPORDER, NPY_NO_CASTING, dtype);
if (iter==NULL) {
return NULL;
}
iternext = NpyIter_GetIterNext(iter, NULL);
dataptr = (double **) NpyIter_GetDataPtrArray(iter);
do {
cout << **dataptr << endl;
} while (iternext(iter));
NpyIter_Deallocate(iter);
return Py_BuildValue(something);
}
To find out more information check out this link: http://docs.scipy.org/doc/numpy/reference/c-api.iterator.html
Related
I'm trying to get an array of tensorflow box predictions from C++ to golang, but I'm not able to do it no matter what I do. I have a GO program that calls a function that does tensorflow detections in C++ using cgo. This all works and I'm able to get the predictions in C++. The problem is to transfer these predictions into GO as an array of 100 structs that each hold one prediction.
I'm able to set a pointer in GO and use this pointer address to set one struct in C++. The code for this is seen below.
I want to set an array of structs in C++ and retreive this array in GO. I thought it should be easy to just use the same pointer address as earlier and use this as the address for my C++ array. Then I could restore the struct from the pointer in GO. Does anyone have a solution for this?
GO
type PredictResult struct {
Loc [4]float32
Score int
Label int
}
var predictions PredictResult
predictions_ptr := unsafe.Pointer(&predictions)
C.LIB_predict(predictions_ptr)
fmt.Println("GO predictions; ", predictions)
bridge.hpp
struct PredictResult{
float Loc[4];
int64_t Score;
int64_t Label;
};
void LIB_predict(void* predictions);
bridge.cpp
void LIB_predict(void* predictions){
PredictResult *p = (PredictResult*)predictions;
p->Score = 6;
p->Label = 95;
}
Prints:
GO predictions; {[0 0 0 0] 6 95}
Assuming your C function returns the array as PredictResult* and assuming you know the length of the returned array (in the example below I assume 10, but you can replace it by whatever works), this approach should work:
// #include <stdio.h>
// #include <stdlib.h>
//
// typedef struct PredictResult {
// float Loc[4];
// int64_t Score;
// int64_t Label;
// } PredictResult;
//
// PredictResult* getOneResult() {
// PredictResult* p = (PredictResult*)calloc(1, sizeof(PredictResult));
// p->Score = 10;
// p->Label = 99;
// p->Loc[1] = 2.5;
// p->Loc[3] = 3.5;
// return p;
// }
//
// PredictResult* getTenResults() {
// PredictResult* parr = (PredictResult*)calloc(10, sizeof(PredictResult));
// parr[0].Score = 10;
// parr[0].Label = 99;
// parr[0].Loc[1] = 2.5;
// parr[0].Loc[3] = 3.5;
//
// parr[4].Score = 44;
// parr[4].Label = 123;
// parr[4].Loc[1] = 12.25;
// parr[4].Loc[3] = -40.5;
// return parr;
// }
//
//
import "C"
type PredictResult C.struct_PredictResult
func main() {
p := C.getOneResult()
if p == nil {
log.Fatal("got nil")
}
pp := (*PredictResult)(p)
fmt.Println(pp)
parr := C.getTenResults()
if parr == nil {
log.Fatal("got nil")
}
pslice := (*[1 << 28]PredictResult)(unsafe.Pointer(parr))[:10:10]
fmt.Println(pslice)
}
What you'll be most interested in is how the result of getTenResults is converted to a Go slice of the appropriate struct type. This is employing the technique recommended on the Go wiki.
Depending on the exact signature of your C function you may need to write a "bridge" function in the import "C" part to provide the data as convenient to Go, but this is the basic gist of it.
As an alternative, if you wish to allocate the slice on the Go side and pass in a pointer to C to populate, you can do this:
// void PopulateTenResults(void* arr) {
// PredictResult* parr = (PredictResult*)arr;
// parr[1].Score = 210;
// parr[1].Label = 299;
// parr[1].Loc[1] = 22.5;
// parr[1].Loc[3] = 23.5;
//
// parr[8].Score = 344;
// parr[8].Label = 3123;
// parr[8].Loc[1] = 312.25;
// parr[8].Loc[3] = -340.5;
// }
//
//
import "C"
And then in Go do:
prslice := make([]PredictResult, 10)
C.PopulateTenResults(unsafe.Pointer(&prslice[0]))
fmt.Println(prslice)
Of course the hard-coded 10 is just for simplicity here; you could pass the length of arr as a parameter to C.
You can pass a pointer to the first element in a slice and the length of the slice to C++ and treat it like a C-style array.
Is there a good way to handle type errors when working with blobs in SQLite? For example, the following code registers two functions create_vector and display_vector. Basically, create_vector stores a std::vector as a blob and display_vector converts this blob into text, so that we can see it:
/* In order to use
sqlite> .load "./blob.so"
sqlite> select display_vector(create_vector());
[ 1.200000, 3.400000, 5.600000, 7.800000, 9.100000 ]
*/
#include <string>
#include <sqlite3ext.h>
SQLITE_EXTENSION_INIT1
extern "C" {
int sqlite3_blob_init(
sqlite3 * db,
char ** err,
sqlite3_api_routines const * const api
);
}
// Cleanup handler that deletes an array
template <typename T>
void array_cleanup(void * v) {
delete [] static_cast <T *> (v);
}
// Creates and returns a std::vector as a blob
static void create_vector(
sqlite3_context *context,
int argc,
sqlite3_value **argv
){
// Create a dummy vector
auto * v = new double[5] {1.2,3.4,5.6,7.8,9.10};
// Either cleanup works
sqlite3_result_blob(context,v,sizeof(double[5]),array_cleanup <double>);
}
// Converts a std::vector into text
static void display_vector(
sqlite3_context *context,
int argc,
sqlite3_value **argv
){
// Grab the vector. Note, if this is not a vector, then sqlite will
// almost certainly segfault.
auto const * const v =static_cast <double const * const> (
sqlite3_value_blob(argv[0]));
// Assuming we have a vector, convert it into a string
auto s = std::string("[ ");
for(unsigned i=0;i<5;i++) {
// If we're not on the first element, add a comma
if(i>0) s += ", ";
// Add the number
s += std::to_string(v[i]);
}
s += " ]";
// Return the text
sqlite3_result_text(
context,sqlite3_mprintf("%s",s.c_str()),s.size(),sqlite3_free);
}
// Register our blob functions
int sqlite3_blob_init(
sqlite3 *db,
char **err,
sqlite3_api_routines const * const api
){
SQLITE_EXTENSION_INIT2(api)
// Register the create_vector function
if( int ret = sqlite3_create_function(
db, "create_vector", 0, SQLITE_ANY, 0, create_vector, 0, 0)
) {
*err=sqlite3_mprintf("Error registering create_vector: %s",
sqlite3_errmsg(db));
return ret;
}
// Register the display_vector function
if( int ret = sqlite3_create_function(
db, "display_vector", 1, SQLITE_ANY, 0, display_vector, 0, 0)
) {
*err=sqlite3_mprintf("Error registering display_vector: %s",
sqlite3_errmsg(db));
return ret;
}
// If we've made it this far, we should be ok
return SQLITE_OK;
}
We can compile this with:
$ make
g++ -g -std=c++14 blob.cpp -shared -o blob.so -fPIC
Now, if we use these functions as advertised, everything works fine:
sqlite> .load "./blob.so"
sqlite> select display_vector(create_vector());
[ 1.200000, 3.400000, 5.600000, 7.800000, 9.100000 ]
However, if we try to use display_vector on a non-vector, we segfault:
sqlite> .load "./blob.so"
sqlite> select display_vector(NULL);
Segmentation fault
Really, the issue is that the static_cast in display_vector vector is not correct. In any case, is there a good way check the type of the blob or even guarantee that we have a blob? Is there a good way to prevent a segfault when a new extension requires an input of a certain type?
A blob is just a bunch of bytes, and not every value is a blob.
Your function should check the value's type with sqlite3_value_type(), and check the length with sqlite3_value_bytes().
I have a real simple Python function:
def myfunc(x):
return 2.0 * x
I want to send this function to a C++ program and call it so I have done this:
#include "Python.h"
static PyObject *execMyPyFunc(PyObject *self, PyObject *args) {
PyObject *Fx, *pyresult;
double x;
PyArg_ParseTuple(args, "dO", &x, &Fx);
pyresult = PyObject_CallFunction(Fx, "d", x);
return pyresult;
}
static PyMethodDef C_API_TestMethods[] = {
{"execMyPyFunc", execMyPyFunc, METH_VARARGS, "Add documentation here.."},
{NULL, NULL}
};
PyMODINIT_FUNC initC_API_Test(void) {
Py_InitModule("C_API_Test", C_API_TestMethods);
}
My Python program works correctly:
from C_API_Test import execMyPyFunc
def myfunc(x):
return 2.0 * x
fx = execMyPyFunc(1.28,myfunc)
print fx
What I would like to do though is to somehow get the pointer from my Python function (PyObject *Fx) and pass this to a C++ function expecting: double(*fx)(double). Does anyone know how to do this (if possible)? I tried to initialize double(*cFx)(double) and cast my Python function as cFx = (double(*)(double))Fx but this does not work. Any ideas?
You aren't going to be able to simply cast a Python function to C like that.
Instead pass the PyObject function pointer, call the function, and convert to C double. This code will return -1 on failure.
static double cFx(PyObject *fx, double x){
PyObject *pyresult = PyObject_CallFunction(fx, "d", x);
if (pyresult == NULL) return -1;
double cppresult = PyFloat_AsDouble(pyresult);
Py_DECREF(pyresult);
if (PyErr_Occurred()) return -1;
return cppresult;
}
Important part of this is to decrement the reference count to the return value of PyObject_CallFunction since you aren't passing it off to the Python interpreter to deal with.
I have a situation in Visual C++ 2008 that I have not seen before. I have a class with 4 STL objects (list and vector to be precise) and integers.
It has a method:
inline int id() { return m_id; }
The return value from this method is corrupt, and I have no idea why.
debugger screenshot http://img687.imageshack.us/img687/6728/returnvalue.png
I'd like to believe its a stack smash, but as far as I know, I have no buffer over-runs or allocation issues.
Some more observations
Here's something that puts me off. The debugger prints right values in the place mentioned // wrong ID.
m_header = new DnsHeader();
assert(_CrtCheckMemory());
if (m_header->init(bytes, size))
{
eprintf("0The header ID is %d\n", m_header->id()); // wrong ID!!!
inside m_header->init()
m_qdcount = ntohs(h->qdcount);
m_ancount = ntohs(h->ancount);
m_nscount = ntohs(h->nscount);
m_arcount = ntohs(h->arcount);
eprintf("The details are %d,%d,%d,%d\n", m_qdcount, m_ancount, m_nscount, m_arcount);
// copy the flags
// this doesn't work with a bitfield struct :(
// memcpy(&m_flags, bytes + 2, sizeof(m_flags));
//unpack_flags(bytes + 2); //TODO
m_init = true;
}
eprintf("Assigning an id of %d\n", m_id); // Correct ID.
return
m_header->id() is an inline function in the header file
inline int id() { return m_id; }
I don't really know how best to post the code snippets I have , but here's my best shot at it. Please do let me know if they are insufficient:
Class DnsHeader has an object m_header inside DnsPacket.
Main body:
DnsPacket *p ;
p = new DnsPacket(r);
assert (_CrtCheckMemory());
p->add_bytes(buf, r); // add bytes to a vector m_bytes inside DnsPacket
if (p->parse())
{
read_packet(sin, *p);
}
p->parse:
size_t size = m_bytes.size(); // m_bytes is a vector
unsigned char *bytes = new u_char[m_bytes.size()];
copy(m_bytes.begin(), m_bytes.end(), bytes);
m_header = new DnsHeader();
eprintf("m_header allocated at %x\n", m_header);
assert(_CrtCheckMemory());
if (m_header->init(bytes, size)) // just set the ID and a bunch of other ints here.
{
size_t pos = DnsHeader::SIZE; // const int
if (pos != size)
; // XXX perhaps generate a warning about extraneous data?
if (ok)
m_parsed = true;
}
else
{
m_parsed = false;
}
if (!ok) {
m_parsed = false;
}
return m_parsed;
}
read_packet:
DnsHeader& h = p.header();
eprintf("The header ID is %d\n", h.id()); // ID is wrong here
...
DnsHeader constructor:
m_id = -1;
m_qdcount = m_ancount = m_nscount = m_arcount = 0;
memset(&m_flags, 0, sizeof(m_flags)); // m_flags is a struct
m_flags.rd = 1;
p.header():
return *m_header;
m_header->init: (u_char* bytes, int size)
header_fmt *h = (header_fmt *)bytes;
m_id = ntohs(h->id);
eprintf("Assigning an id of %d/%d\n", ntohs(h->id), m_id); // ID is correct here
m_qdcount = ntohs(h->qdcount);
m_ancount = ntohs(h->ancount);
m_nscount = ntohs(h->nscount);
m_arcount = ntohs(h->arcount);
You seem to be using a pointer to an invalid class somehow. The return value shown is the value that VS usually uses to initialize memory with:
2^32 - 842150451 = 0xCDCDCDCD
You probably have not initialized the class that this function is a member of.
Without seeing more of the code in context.. it might be that the m_id is out of the scope you expect it to be in.
Reinstalled VC++. That fixed everything.
Thank you for your time and support everybody! :) Appreciate it!
I'm the lead dev for Bitfighter, and am adding user-scripted bots using Lua. I'm working with C++ and Lua using Lunar to glue them together.
I'm trying to do something that I think should be pretty simple: I have an C++ object in Lua (bot in the code below), and I call a method on it that (findItems) which causes C++ to search the area around the robot and return a list of objects it finds (TestItems and others not shown here). My question is simply how do I assemble and return the list of found items in C++, and then iterate over them in Lua?
Basically, I want to fill in the <<<< Create list of items, return it to lua >>>> block below, and make any corrections I may need in the Lua code itself, included below that.
I've tried to keep the code simple but complete. Hope there's not too much here! Thanks!
C++ Header file
class TestItem : public LuaObject
{
public:
TestItem(); // C++ constructor
///// Lua Interface
TestItem(lua_State *L) { } ; // Lua constructor
static const char className[];
static Lunar<TestItem>::RegType methods[];
S32 getClassID(lua_State *L) { return returnInt(L, TestItemType); }
};
class LuaRobot : public Robot
{
LuaRobot(); // C++ constructor
///// Lua Interface
LuaRobot(lua_State *L) { } ; // Lua constructor
static const char className[];
static Lunar<LuaRobot>::RegType methods[];
S32 findItems(lua_State *L);
}
C++ .cpp file
const char LuaRobot::className[] = "Robot"; // Class name in Lua
// Define the methods we will expose to Lua
Lunar<LuaRobot>::RegType LuaRobot::methods[] =
{
method(LuaRobot, findItems),
{0,0} // End method list
};
S32 LuaRobot::findItems(lua_State *L)
{
range = getIntFromStack(L, 1); // Pop range from the stack
thisRobot->findObjects(fillVector, range); // Put items in fillVector
<<<< Create list of items, return it to lua >>>>
for(int i=0; i < fillVector.size(); i++)
do something(fillVector[i]); // Do... what, exactly?
return something;
}
/////
const char TestItem::className[] = "TestItem"; // Class name in Lua
// Define the methods we will expose to Lua
Lunar<TestItem>::RegType TestItem::methods[] =
{
// Standard gameItem methods
method(TestItem, getClassID),
{0,0} // End method list
};
Lua Code
bot = LuaRobot( Robot ) -- This is a reference to our bot
range = 10
items = bot:findItems( range )
for i, v in ipairs( items ) do
print( "Item Type: " .. v:getClassID() )
end
So you need to fill a vector and push that to Lua.
Some example code follows. Applications is a std::list.
typedef std::list<std::string> Applications;
I create a table and fill it with the data in my list.
int ReturnArray(lua_State* L) {
lua_createtable(L, applications.size(), 0);
int newTable = lua_gettop(L);
int index = 1;
Applications::const_iterator iter = applications.begin();
while(iter != applications.end()) {
lua_pushstring(L, (*iter).c_str());
lua_rawseti(L, newTable, index);
++iter;
++index;
}
return 1;
}
This leaves me with an array in the stack. If it were returned to Lua, then I could write the following:
for k,v in ipairs( ReturnArray() ) do
print(v)
end
Of course so far, this just gets me a Lua array of strings. To get an array of Lua objects we just tweak your method a bit:
S32 LuaRobot::findItems(lua_State *L)
{
range = getIntFromStack(L, 1); // Pop range from the stack
thisRobot->findObjects(fillVector, range); // Put items in fillVector
// <<<< Create list of items, return it to lua >>>>
lua_createtable(L, fillVector.size(), 0);
int newTable = lua_gettop(L);
for(int i=0; i < fillVector.size(); i++) {
TestItem* item = fillVector[i];
item->push(L); // put an object, not a string, in Lua array
lua_rawseti(L, newTable, i + 1);
}
return 1;
}
This works perfectly. To clarify to others who are reading this, the method
item->push(L)
is
void push(lua_State *L) { Lunar<TestItem>::push(L, this); }
By encapsulating this in a method, it's possible to make the findItems agnostic to what it's finding.
Thank you for the help!