Unbuffered communication between C++ and Python - c++

I want to pass data from C++ to Python in Linux and I do this through a pipe.
My C++ program looks like this:
#include<iostream>
#include<stdio.h>
#include<sstream>
#include<math.h>
int main() {
FILE *cmd;
std::ostringstream ss;
double val;
cmd = popen("python plot.py","w");
for (int i=0; i<100; i++) {
val = cos(3.14159*i/40);
ss.str("");
ss << val << std::endl;
fputs(ss.str().c_str(),cmd);
fflush(cmd);
}
fputs("\n",cmd);
fflush(cmd);
pclose(cmd);
return 0;
}
The corresponding python function plot.py looks like:
fig, = plt.plot([],[])
index = 0
plt.ion()
while True:
data = sys.stdin.readline().strip()
if not data:
plt.ioff()
break
index = index + 1
val = [float(n) for n in self.data.split()]
fig.set_xdata(numpy.append(fig.get_xdata(),index))
fig.set_ydata(numpy.append(fig.get_ydata(),val))
axis = list(plt.axis())
min_x = 0
max_x = max(self.fig.get_xdata())
min_y = min(self.fig.get_ydata())
max_y = max(self.fig.get_ydata())
axis[0] = min_x
axis[1] = max_x
axis[2] = min_y
axis[3] = max_y
plt.axis(axis)
plt.draw()
plt.show()
Now my problem is that there seems to be a buffering of data from my c++ function before data is being passed to the python function, which I am not able to get rid of. I have searched for a solution but have not been able to find any. The best suggestion I could find was to use setvbuf in the c++ function, so I also tried to include the following line to the c++ function
setvbuf(cmd,NULL,_IONBUF,0)
But that did not have any effects.
Does anyone know how I can pass data from my c++ function to python in an unbuffered manner?

Related

Building a css grid with javascript: creating elements, storing them in arrays and append them to other elements

I recently build an masonry gallery with html and css. I used the display: grid; property to make it look like so. Now I am trying to create randomized layouts. Therefor I want to create elements with classes. I want to append / elements to the elements, store those elements inside an array and later append them inside my "masonry" . I put in different console.logs to see whats happening, but I get some either weird or undefined returns. Javascript can be as tricky as it can be fun for beginners, so I hope you guys can help me out.
Thank you all. :)
//arrays and variables
const figures = [];
let newFigures = "";
const divs = [];
let newDivs = "";
function makeCells(){
for(let i = 0; i < 33; i++){
newFigure = document.createElement("figure");
figures[i] = newFigure.classList.add("cell", "cell--" + i);
newDiv = document.createElement("div");
divs[i] = newDiv.setAttribute("id", i);
console.log("log1: " + newFigure.classList);
console.log("log2: " + figures[i]);
}
console.log("log3: " + divs);
console.log("log4: " + figures);
console.log("log5: " + divs);
for(let i=0; i<3; i++){
figures[i] = newFigure.appendChild(newDiv);
}
console.log("figures = " + figures);
console.log("divs = " + divs);
let z = {};
for(let i=0; i<3; i++){
z = figures[i];
document.getElementById("masonry").appendChild(z);
}
console.log(document.getElementById("masonry"));
}
Here is a picture of the corresponding console.logs.
Console Logs
This works as intended:
const figures = [];
let newFigure = "";
const divs = [];
let newDiv = "";
function makeCells(){
for(let i = 0; i < 33; i++){
newFigure = document.createElement("figure");
newDiv = document.createElement("div");
newFigure.appendChild(newDiv);
document.getElementById("masonry").appendChild(newFigure);
figures[i] = newFigure.classList.add("cell", "cell--" + i);
}
console.log(document.getElementById("masonry"));
}
Sorry to bother you guys.
I don't like this line of code, mainly because the add() method does not have a return value.
figures[i] = newFigure.classList.add("cell", "cell--" + i);
I would split it into two statements.
newFigure.classList.add("cell", "cell--" + i);
figures.push(newFigure);

Parameter passing from Python script to C++ with boost-python

I am currently embedding Python in C++ using boost-python and boost-numpy.
I have the following Python test script:
import numpy as np
import time
def test_qr(m,n):
print("create numpy array")
A = np.random.rand(m, n)
print("Matrix A is {}".format(A))
print("Lets QR factorize this thing! Mathematics is great !!")
ts = time.time()
Q, R = np.linalg.qr(A)
te = time.time()
print("It took {} seconds to factorize A".format(te - ts))
print("The Q matrix is {}".format(Q))
print("The R matrix is {}".format(R))
return Q,R
def sum(m,n):
return m+n
I am able to execute a part of the code in C++ like this:
namespace p = boost::python;
namespace np = boost::python::numpy;
int main() {
Py_Initialize(); //initialize python environment
np::initialize(); //initialize numpy environment
p::object main_module = p::import("__main__");
p::object main_namespace = main_module.attr("__dict__");
// execute code in the main_namespace
p::exec_file("/Users/Michael/CLionProjects/CythonTest/test_file.py",main_namespace); //loads python script
p::exec("m = 100\n"
"n = 100\n"
"Q,R = test_qr(m,n)", main_namespace);
np::ndarray Q_matrix = p::extract<np::ndarray>(main_namespace["Q"]); // extract results as numpy array types
np::ndarray R_matrix = p::extract<np::ndarray>(main_namespace["R"]);
std::cout<<"C++ Q Matrix: \n" << p::extract<char const *>(p::str(Q_matrix)) << std::endl; // extract every element as a
std::cout<<"C++ R Matrix: \n" << p::extract<char const *>(p::str(R_matrix)) << std::endl;
std::cout<<"code also works with numpy, ask for a raise" << std::endl;
p::object sum = main_namespace.attr("sum")(10,10);
int result = p::extract<int>(main_namespace.attr("sum")(10,10));
std::cout<<"sum result works " << result << std::endl;
return 0;}
Now I am trying to use the sum function in the Python script but I do not always want to write a string like:
p::exec("m = 100\n"
"n = 100\n"
"Q,R = test_qr(m,n)", main_namespace);}
How can this be done without using the exec function?
I have tried things like:
p::object sum = main_namespace.attr("sum")(10,10);
int result = p::extract<int>(main_namespace.attr("sum")(10,10));
std::cout<<"sum result works " << result << std::endl;
As mentioned in the documentation of boost.
I also tried using the call_method function, but it didn't work.
I get either boost::python::error_already_set exception which mean there is something wrong in Python, but I do not know what.
Or an exit code 11.
The issue is rather trivial. Let's look at the tutorial you mention:
object main_module = import("__main__");
object main_namespace = main_module.attr("__dict__");
object ignored = exec("result = 5 ** 2", main_namespace);
int five_squared = extract<int>(main_namespace["result"]);
Notice how they extract the result object in the last line: main_namespace["result"]
The main_namespace object is a Python dictionary, and rather than extracting it's attribute, you're just looking for a value stored with the particular key. Hence, indexing with [] is the way to go.
C++ code:
#define BOOST_ALL_NO_LIB
#include <boost/python.hpp>
#include <boost/python/numpy.hpp>
#include <iostream>
namespace bp = boost::python;
int main()
{
try {
Py_Initialize();
bp::object module = bp::import("__main__");
bp::object globals = module.attr("__dict__");
bp::exec_file("bpcall.py", globals);
bp::object sum_fn = globals["sum"];
int result = bp::extract<int>(sum_fn(1,2));
std::cout << "Result (C++) = " << result << "\n";
} catch (bp::error_already_set) {
PyErr_Print();
}
Py_Finalize();
}
Python script:
def sum(m,n):
return m+n
Output:
Result (C++) = 3

VARIANT export/import C++ DLL SAFEARRAY

I know that this topic is quite popular. However, I'd like to solve a simple problem... My target is to create a DLL in C++ able to import and export data matrices from VB (Excel or other programs). I read in some blogs that a common way to do that is to use the SAFEARRAYs.
So, I created the following C++ code:
VARIANT _stdcall ReadArrayVBA_v1(VARIANT *Input0) {
double dm = 0, rm = 0;
//OUTPUT DATA DEFINITION
VARIANT v;
SAFEARRAYBOUND Dim[1];
Dim[0].lLbound = 0; Dim[0].cElements = 4;
v.vt = VT_R8;
v.parray = SafeArrayCreate(VT_R8, 1, Dim);
//==============================================
SAFEARRAY* pSafeArrayInput0 = NULL; //Define a pointer SAFEARRAY Type
pSafeArrayInput0 = *V_ARRAYREF(Input0);
long lLBound = -1, lUBound = 1; //Preset dimension
SafeArrayGetLBound(pSafeArrayInput0, 1, &lLBound);
SafeArrayGetUBound(pSafeArrayInput0, 1, &lUBound);
long CntElements = lUBound - lLBound + 1;
long Index[1];
for (int i=0; i<CntElements; i++){
Index[0] = i;
SafeArrayGetElement(pSafeArrayInput0, Index, &dm);
rm = dm + 1;
SafeArrayPutElement(v.parray, Index, &rm);
}
return v;
}
It compiles and I call from VBA Excel in the following manner:
Private Declare Function ReadArrayVBA_v1 Lib "[PATH]\VARIANTtest.dll" (ByRef Input0 As Variant) As Variant
Sub VBACall()
Dim InputR0(1 To 4) As Variant
Dim vResult1 As Variant
InputR0(1) = 2
InputR0(2) = 20
InputR0(3) = 200
InputR0(4) = 240
vResult1 = ReadArrayVBA_v1(InputR0)
End Sub
The function gives me back values such as 1.2E-305 and similar. Why?

(c++) get data after executing mysql query group_concat

i have a question. i have a complex query and it contains group_concat.
MYSQL_RES* res = GetDBManager()->Query("select ls.time, count(ws.id) , group_concat(wc.id) , group_concat(ws.SIGNAL_STRENGTH) , ul.LATITUDE , ul.LONGITUDE , ul.ALTITUDE from user_location_scan ls, wifi_scan ws, wifi_cell wc, user_location ul where ls.id = ws.user_scan and ws.wifi_cell = wc.id and ls.time = ul.time group by ls.id order by ls.id");
how can i assign the data in group_concat to a struct such as
struct {
int APid;
double rssi;
}* sigstr;
i already tried a loop for() but it didn't work.
while (row = mysql_fetch_row(res)) {
int j = 0;
inf.time.FromSQL(row[j++]);
inf.numAP = atoi(row[j++]);
for (int i = 0; i < inf.numAP; i++) {
inf.sigstr[i].APid = atoi(row[j++]);
inf.sigstr[i].rssi = atof(row[j++]);
}
...
please help me.
UPDATE:
after some searching, i found that the data returned by group_concat actually cannot be assigned to an array. are there any ways to convert it?

What am I doing wrong? (multithreading)

Here s what I'm doing in a nutshell.
In my class's cpp file I have:
std::vector<std::vector<GLdouble>> ThreadPts[4];
The thread proc looks like this:
unsigned __stdcall BezierThreadProc(void *arg)
{
SHAPETHREADDATA *data = (SHAPETHREADDATA *) arg;
OGLSHAPE *obj = reinterpret_cast<OGLSHAPE*>(data->objectptr);
for(unsigned int i = data->start; i < data->end - 1; ++i)
{
obj->SetCubicBezier(
obj->Contour[data->contournum].UserPoints[i],
obj->Contour[data->contournum].UserPoints[i + 1],
data->whichVector);
}
_endthreadex( 0 );
return 0;
}
SetCubicBezier looks like this:
void OGLSHAPE::SetCubicBezier(USERFPOINT &a,USERFPOINT &b, int &currentvector )
{
std::vector<GLdouble> temp;
if(a.RightHandle.x == a.UserPoint.x && a.RightHandle.y == a.UserPoint.y
&& b.LeftHandle.x == b.UserPoint.x && b.LeftHandle.y == b.UserPoint.y )
{
temp.clear();
temp.push_back((GLdouble)a.UserPoint.x);
temp.push_back((GLdouble)a.UserPoint.y);
ThreadPts[currentvector].push_back(temp);
temp.clear();
temp.push_back((GLdouble)b.UserPoint.x);
temp.push_back((GLdouble)b.UserPoint.y);
ThreadPts[currentvector].push_back(temp);
}
}
The code that calls the threads looks like this:
for(int i = 0; i < Contour.size(); ++i)
{
Contour[i].DrawingPoints.clear();
if(Contour[i].UserPoints.size() < 2)
{
break;
}
HANDLE hThread[4];
SHAPETHREADDATA dat;
dat.objectptr = (void*)this;
dat.start = 0;
dat.end = floor((Contour[i].UserPoints.size() - 1) * 0.25);
dat.whichVector = 0;
dat.contournum = i;
hThread[0] = (HANDLE)_beginthreadex(NULL,0,&BezierThreadProc,&dat,0,0);
dat.start = dat.end;
dat.end = floor((Contour[i].UserPoints.size() - 1) * 0.5);
dat.whichVector = 1;
hThread[1] = (HANDLE)_beginthreadex(NULL,0,&BezierThreadProc,&dat,0,0);
dat.start = dat.end;
dat.end = floor((Contour[i].UserPoints.size() - 1) * 0.75);
dat.whichVector = 2;
hThread[2] = (HANDLE)_beginthreadex(NULL,0,&BezierThreadProc,&dat,0,0);
dat.start = dat.end;
dat.end = Contour[i].UserPoints.size();
dat.whichVector = 3;
hThread[3] = (HANDLE)_beginthreadex(NULL,0,&BezierThreadProc,&dat,0,0);
WaitForMultipleObjects(4,hThread,true,INFINITE);
}
Is there something wrong with this?
I'd expect it to fill ThreadPts[4]; ... There should never be any conflicts the way I have it set up. I usually get error writing at... on the last thread where dat->whichvector = 3. If I remove:
dat.start = dat.end;
dat.end = Contour[i].UserPoints.size();
dat.whichVector = 3;
hThread[3] = (HANDLE)_beginthreadex(NULL,0,&BezierThreadProc,&dat,0,0);
Then it does not seem to crash, what could be wrong?
Thanks
The problem is that you're passing the same dat structure to each thread as the argument to the threadproc.
For example, When you start thread 1, there's no guarantee that it will have read the information in the dat structure before your main thread starts loading that same dat structure with the information for thread 2 (and so on). In fact, you're constantly directly using that dat structure throughout the thread's loop, so the thread won't be finished with the structure passed to it until the thread is basically done with all its work.
Also note that currentvector in SetCubicBezier() is a reference to data->whichVector, which is referring to the exact same location in a threads. So SetCubicBezier() will be performing push_back() calls on the same object in separate threads because of this.
There's a very simple fix: you should use four separate SHAPETHREADDATA instances - one to initialize each thread.