I am trying to speeden up my Matlab program by writing a few functions in C++ and using the mex interface to integrate them. I got my results in a vector in c++. I want to transfer it to an array in MATLAB.
I know i should redirect
plhs[0] to the vector
but i am not getting how exactly should I do it.
When I've done similar things, I manually marshal the data so that it won't be freed when the C++ routine is completed. Here's a basic outline:
#include <vector>
#include "mex.h"
mxArray * getMexArray(const std::vector<double>& v){
mxArray * mx = mxCreateDoubleMatrix(1,v.size(), mxREAL);
std::copy(v.begin(), v.end(), mxGetPr(mx));
return mx;
}
void mexFunction(int nlhs, mxArray *plhs[ ], int nrhs, const mxArray *prhs[ ]) {
std::vector<double> v;
v.push_back(0);
v.push_back(1);
v.push_back(2);
v.push_back(3);
plhs[0] = getMexArray(v);
}
If I save this as test.cpp and then open matlab in that directory, I do the following:
>> mex test.cpp
>> test
ans =
0 1 2 3
which is the expected output. Hopefully that is a good starting point - you may want to inline it, but I'm not sure of the benefit. Btw, if you haven't checked out the matlab mex help, it is a great resource.
Related
The title summarizes the goal that is more exactly to dynamically retrieve the number of dimensions of MATLAB arrays passed to armadillo matrices.
I would like to change the second and third arguments of mY() and mD() to parametric ones below.
// mat(ptr_aux_mem, n_rows, n_cols, copy_aux_mem = true, strict = false)
arma::mat mY(&dY[0], 2, 168, false);
arma::mat mD(&dD[0], 2, 168, false);
This must be definitely a common use case, but I still could not find a nice way of achieving it for the general case when the number of dimensions of the arrays feeding from MATLAB could be arbitrary (n > 2).
For the matrix (two dimensional) case, I could possibly hack my way around but I feel like that is not elegant enough (probably not efficient either).
IMHO, the way to go must be:
matlab::data::TypedArray<double> has getDimensions() member function which retrieves matlab::data::ArrayDimensions that is fundamentally a std::vector<size_t>.
Indexing the first and second element of the vector retrieved by getDimensions() one can retrieve the number of rows and columns, for instance like below.
unsigned int mYrows = matrixY.getDimensions()[0];
unsigned int mYcols = matrixY.getDimensions()[1];
However, with my current setup, I cannot get to call getDimensions() through pointers/references in the foo() function of sub.cpp. If it is feasible, I would neither like to create additional temporary objects nor passing other arguments to foo(). How it possible that way?
Intuition keeps telling me that there must be an elegant solution that way too. Maybe using multiple indirection?
I would highly appreciate any help, hints or constructive comments from more knowledgeable SO members. Thank you in advance.
Setup:
Two C++ source files and a header file:
main.cpp
contains the general IO interface between MATLAB and C++
feeds two double arrays and two double const doubles into C++
it does some Armadillo based looping (this part is not that important therefore omitted) by calling foo()
returns outp which is a “just a plain” scalar double
Nothing fancy or complicated.
sub.cpp
This is only for the foo() looping part.
sub.hpp
Just a simple header file.
// main.cpp
// MATLAB API Header Files
#include "mex.hpp"
#include "mexAdapter.hpp"
// Custom header
#include "sub.hpp"
// Overloading the function call operator, thus class acts as a functor
class MexFunction : public matlab::mex::Function {
public:
void operator()(matlab::mex::ArgumentList outputs,
matlab::mex::ArgumentList inputs){
matlab::data::ArrayFactory factory;
// Validate arguments
checkArguments(outputs, inputs);
matlab::data::TypedArray<double> matrixY = std::move(inputs[0]);
matlab::data::TypedArray<double> matrixD = std::move(inputs[1]);
const double csT = inputs[2][0];
const double csKy = inputs[3][0];
buffer_ptr_t<double> mY = matrixY.release();
buffer_ptr_t<double> mD = matrixD.release();
double* darrY = mY.get();
double* darrD = mD.get();
// data type of outp is "just" a plain double, NOT a double array
double outp = foo(darrY, darrD, csT, csKy);
outputs[0] = factory.createScalar(outp);
void checkArguments(matlab::mex::ArgumentList outputs, matlab::mex::ArgumentList inputs){
// Create pointer to MATLAB engine
std::shared_ptr<matlab::engine::MATLABEngine> matlabPtr = getEngine();
// Create array factory, allows us to create MATLAB arrays in C++
matlab::data::ArrayFactory factory;
// Check input size and types
if (inputs[0].getType() != ArrayType::DOUBLE ||
inputs[0].getType() == ArrayType::COMPLEX_DOUBLE)
{
// Throw error directly into MATLAB if type does not match
matlabPtr->feval(u"error", 0,
std::vector<Array>({ factory.createScalar("Input must be double array.") }));
}
// Check output size
if (outputs.size() > 1) {
matlabPtr->feval(u"error", 0,
std::vector<Array>({ factory.createScalar("Only one output is returned.") }));
}
}
};
// sub.cpp
#include "sub.hpp"
#include "armadillo"
double foo(double* dY, double* dD, const double T, const double Ky) {
double sum = 0;
// Conversion of input parameters to Armadillo types
// mat(ptr_aux_mem, n_rows, n_cols, copy_aux_mem = true, strict = false)
arma::mat mY(&dY[0], 2, 168, false);
arma::mat mD(&dD[0], 2, 168, false);
// Armadillo calculations
for(int t=0; t<int(T); t++){
// some armadillo based calculation
// each for cycle increments sum by its return value
}
return sum;
}
// sub.hpp
#ifndef SUB_H_INCLUDED
#define SUB_H_INCLUDED
double foo(double* dY, double* dD, const double T, const double Ky);
#endif // SUB_H_INCLUDED
One way is to convert it to arma matrix using a function
template<class T>
arma::Mat<T> getMat( matlab::data::TypedArray<T> A)
{
matlab::data::TypedIterator<T> it = A.begin();
matlab::data::ArrayDimensions nDim = A.getDimensions();
return arma::Mat<T>(it.operator->(), nDim[0], nDim[1]);
}
and call it by
arma::mat Y = getMat<double>(inputs[0]);
arma::mat D = getMat<double>(inputs[1]);
...
double outp = foo(Y,D, csT, csKy);
and change foo() to
double foo( arma::mat& dY, arma::mat& dD, const double T, const double Ky)
Recently, I am trying to work on the Rcpp package to improve efficiency of computation in my work. However,I am not deep knowledged about C++, there are some strange behavoirs I can not understand. The below example show a simple tasks about derving weight of NumericVector, there are several questions:
When I use WAP=rev(WAP), it results in an incorrect output, I have to introduce a new variable to store the result so that I get the right output. I do not know why, should it NEVER use a 'x=f(x)' operation in C++ and Rcpp (must copy by clone instead) ?
About the CharacterVector method="eq", exactly I want to use a char or string type, however, it does not work with strncmp function (now I have to use method[0]), but I do not know how to look up the API of Rcpp functions in Rstudio?
I wonder whether there is a R-style grep, tolower function for conditions in Rcpp, I do not know which document I should refer to except for Rcpp suger, so that I can find the availiable base functions. Otherwise, I am thinking about calling R functions with Rcpp::function R_grep("grep"), but I do know whether this is the best way and recommended.
Any suggestions would be greatly appreciated.
#include <Rcpp.h>
#include <string>
#include <math.h>
#include <algorithm>
using namespace std;
using namespace Rcpp;
// [[Rcpp::export]]
NumericVector cppweight(int N, CharacterVector method="eq", const bool reverse=false, const bool test=false){
NumericVector W(N);
NumericVector WAP(N);
NumericVector revWAP(N);
//method=tolower(method); //function not exists
if(strncmp(method[0],"eq",2)==0){
W=rep(1,N)/1.0*N;//convert int to float by multiplying 1.0
WAP=W/sum(W);
Rcout<< sum(W) << "\n";
} else if(strncmp(method[0],"ln",2)==0){
W=rev(seq(1,N))/1.0*N;
WAP=W/sum(W);
}
if(reverse){
if(test){
WAP=rev(WAP);//Why this result in incorrect result
revWAP=WAP;
}else{
revWAP=rev(WAP);
}
}else{
revWAP=WAP;
}
return(round(revWAP,3));
}
/*** R
cppweight(6,"ln",reverse=F,test=F)
cppweight(6,"ln",reverse=T,test=F)
cppweight(6,"ln",reverse=T,test=T)
*/
I'm looking into using some C++ for simple parts of my R package using the Rcpp package. I'm a C++ novice (but keen to learn!). I've implemented a few simple cpp programs using the excellent Rcpp - in fact that package has motivated me to learn C++...
Anyway, I've got stuck with a simple problem, which if I can fix would help lots. I have a NumericVector I want to subset and then sort. The code below sorts the whole vector (and would also deal with NAs, which is what I need).
My question is, say I want to extract a part of this vector, sort and have it available for other processing - how can I do that? For example, for a vector of length 10, how do I extract and sort the elements 5:10?
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
RcppExport SEXP rollP(SEXP x) {
NumericVector A(x); // the data
A = sort_unique(A);
return A;
}
which I call from R:
sourceCpp( "rollP.cpp")
rollP(10:1)
# [1] 1 2 3 4 5 6 7 8 9 10
Here are 3 variants:
include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
NumericVector rollP(NumericVector A, int start, int end) {
NumericVector B(end-start+1) ;
std::copy( A.begin() + start-1, A.begin() + end, B.begin() ) ;
return B.sort() ;
}
// [[Rcpp::export]]
NumericVector rollP2(NumericVector A, int start, int end) {
NumericVector B( A.begin() + start-1, A.begin() + end ) ;
return B.sort() ;
}
// [[Rcpp::export]]
NumericVector rollP3(NumericVector A, int start, int end) {
NumericVector B = A[seq(start-1, end-1)] ;
return B.sort() ;
}
start and end are meant as 1-based indices, as if you were passing A[start:end] from R.
You need to look into C++ indexing, iterators and the whole bit. At a minimum, you need to change your interface (vector, fromInd, toInd) and figure out what you want to return.
One interpretation of your question would be to copy the subset from [fromInd, toInd) into a new vector, sort it and return it. All that is standard C++ fare, and a good text like the excellent (and free!!) C++ Annotations will be of help. It has a pretty strong STL section too.
You can use std::slice on a std::valarray. But if you want to use std::vector specifically then you can use std::copy to extract a portion of the vector and then use std::sort to sort the extracted slice of the vector.
You can do this quite easily by using the std::sort implementation that receives two iterators:
#include <vector>
#include <cinttypes>
#include <algorithm>
template <typename SeqContainer>
SeqContainer slicesort(SeqContainer const& sq, size_t begin, size_t end) {
auto const b = std::begin(sq)+begin;
auto const e = std::begin(sq)+end;
if (b <= std::end(sq) && e <= std::end(sq)) {
SeqContainer copy(b,e);
std::sort(copy.begin(),copy.end());
return copy;
}
return SeqContainer();
}
Which can be invoked like
std::vector<int> v = {3,1,7,3,6,-2,-8,-7,-1,-4,2,3,9};
std::vector<int> v2 = slicesort(v,5,10);
I'm using boost for matrix and vector operations in a code and one of the libraries I am using (CGNS) has an array as an argument. How do I copy the vector into double[] in a boost 'way', or better yet, can I pass the data without creating a copy?
I'm a bit new to c++ and am just getting going with boost. Is there a guide I should read with this info?
Contents between any two input iterators can be copied to an output iterator using the copy algorithm. Since both ublas::vector and arrays have iterator interfaces, we could use:
#include <boost/numeric/ublas/vector.hpp>
#include <algorithm>
#include <cstdio>
int main () {
boost::numeric::ublas::vector<double> v (3);
v(0) = 2;
v(1) = 4.5;
v(2) = 3.15;
double p[3];
std::copy(v.begin(), v.end(), p); // <--
printf("%g %g %g\n", p[0], p[1], p[2]);
return 0;
}
Depends on the types involved. For std::vector you just make sure that it's non-empty and then you can pass &v[0]. Most likely the same holds for the Boost types you're using.
I am writing some C++ code in Linux where I have declared a few 2D arrays like so:
double x[5000][500], y[5000][500], z[5000][500];
During compilation there is no error. When I execute it says "segmentation fault".
Wen I reduce the size of the array from 5000 to 50, the program runs fine. How can I protect myself against this problem?
If your program looks like this ...
int main(int, char **) {
double x[5000][500],y[5000][500],z[5000][500];
// ...
return 0;
}
... then you are overflowing the stack. The fastest way to fix this is to add the word static.
int main(int, char **) {
static double x[5000][500],y[5000][500],z[5000][500];
// ...
return 0;
}
The second fastest way to fix this is to move the declaration out of the function:
double x[5000][500],y[5000][500],z[5000][500];
int main(int, char **) {
// ...
return 0;
}
The third fastest way to fix this is to allocate the memory on the heap:
int main(int, char **) {
double **x = new double*[5000];
double **y = new double*[5000];
double **z = new double*[5000];
for (size_t i = 0; i < 5000; i++) {
x[i] = new double[500];
y[i] = new double[500];
z[i] = new double[500];
}
// ...
for (size_t i = 5000; i > 0; ) {
delete[] z[--i];
delete[] y[i];
delete[] x[i];
}
delete[] z;
delete[] y;
delete[] x;
return 0;
}
The fourth fastest way is to allocate them on the heap using std::vector. It is fewer lines in your file but more lines in the compilation unit, and you must either think of a meaningful name for your derived vector types or tuck them into an anonymous namespace so they won't pollute the global namespace:
#include <vector>
using std::vector
namespace {
struct Y : public vector<double> { Y() : vector<double>(500) {} };
struct XY : public vector<Y> { XY() : vector<Y>(5000) {} } ;
}
int main(int, char **) {
XY x, y, z;
// ...
return 0;
}
The fifth fastest way is to allocate them on the heap, but use templates so the dimensions are not so remote from the objects:
include <vector>
using namespace std;
namespace {
template <size_t N>
struct Y : public vector<double> { Y() : vector<double>(N) {} };
template <size_t N1, size_t N2>
struct XY : public vector< Y<N2> > { XY() : vector< Y<N2> > (N1) {} } ;
}
int main(int, char **) {
XY<5000,500> x, y, z;
XY<500,50> mini_x, mini_y, mini_z;
// ...
return 0;
}
The most performant way is to allocate the two-dimensional arrays as one-dimensional arrays, and then use index arithmetic.
All the above assumes that you have some reason, a good one or a poor one, for wanting to craft your own multidimensional array mechanism. If you have no reason, and expect to use multidimensional arrays again, strongly consider installing a library:
A plays-nicely-with-STL way is to
use the Boost Multidimensional
Array.
A speed way is to use Blitz++.
These arrays are on the stack. Stacks are quite limited in size. You probably run into a ... stack overflow :)
If you want to avoid this, you need to put them on the free store:
double* x =new double[5000*5000];
But you better start the good habit of using the standard containers, which wrap all this for you:
std::vector< std::vector<int> > x( std::vector<int>(500), 5000 );
Plus: even if the stack fits the arrays, you still need room for functions to put their frames on it.
You may want to try and use Boost.Multi_array
typedef boost::multi_array<double, 2> Double2d;
Double2d x(boost::extents[5000][500]);
Double2d y(boost::extents[5000][500]);
Double2d z(boost::extents[5000][500]);
The actual large memory chunk will be allocated on the heap and automatically deallocated when necessary.
Your declaration should appear at top level, outside any procedure or method.
By far the easiest way to diagnose a segfault in C or C++ code is to use valgrind. If one of your arrays is at fault, valgrind will pinpoint exactly where and how. If the fault lies elsewhere, it will tell you that, too.
valgrind can be used on any x86 binary but will give more information if you compile with gcc -g.
One reservation about always using vector: as far as I understand it, if you walk off the end of the array it just allocates a larger array and copies everything over which might create subtle and hard to find errors when you are really tying to work with a fixed size array. At least with a real array you'll segfault if you walk off the end making the error easier to catch.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
typedef double (*array5k_t)[5000];
array5k_t array5k = calloc(5000, sizeof(double)*5000);
// should generate segfault error
array5k[5000][5001] = 10;
return 0;
}
Looks to me like you have an honest-to-Spolsky stack overflow!
Try compiling your program with gcc's -fstack-check option. If your arrays are too big to allocate on the stack, you'll get a StorageError exception.
I think it's a good bet, though, as 5000*500*3 doubles (8 bytes each) comes to around 60 megs - no platform has enough stack for that. You'll have to allocate your big arrays on the heap.
Another solution to the previous ones would be to execute a
ulimit -s stack_area
to expand the maximum stack.
You may want to try the Multi library for multidimensional arrays (C++17).
#include<multi/array.hpp>
#include<cassert>
namespace multi = boost::multi;
int main() {
using Double2D = multi::array<double, 2>;
Double2D X({5000, 500}, 999.0);
Double2D Y({5000, 500});
Double2D Z({5000, 500});
assert( X.size() == 5000 );
auto [m, n] = X.extensions();
assert( m == 5000 );
assert( n == 500 );
Y = X;
assert( Y[0][0] == 999.0 );
}
https://godbolt.org/z/rh5M463Y1
Similarly to the Boost.MultiArray library (other answer), allocates memory in the heap instead of using (overflowing) the stack.
It provides other features, like assignment and iteration.