Thread-safe parallel sparse-sparse matrix multiplication with Eigen and OpenMP - c++

Question

According to the docs, Eigen supports multi-threaded dense v. dense, or row-major sparse v. dense, but not sparse v. sparse matrix multiplications. In my application I have 2 sparse matrices and I want to multiply them in parallel, i.e. obtain C = A * B where C and B are column major.
[Edit:] Specific to my application I also know the sparsity pattern of all matrices in advance, and I need to perform such matrix multiplication many times (each time updating the non-zero values of C using the updated A and B). This leads me to think that there should be a thread-safe way to perform this operation in this case since the memory allocation of C is fixed [end edit]
Given that Eigen gives fast read/write on column blocks for column-major sparse matrices, I can perform the operation by subsetting B into column blocks (suppose A, B and C have all been initialized and are of the correct size):
int nchunks = 7;
int chunksize = B.cols() / nchunks;
#pragma omp parallel for
for(int i=0; i<nchunks; i++){
if(i == nchunks-1){
C.rightCols(B.cols() - (nchunks-1)*chunksize) = A * B.rightCols(B.cols() - (nchunks-1)*chunksize);
} else {
C.middleCols(i*chunksize, chunksize) = A * B.middleCols(i*chunksize, chunksize);
}
}
In my application, I have to perform A.transpose() * S * A thousands of times where S is symmetric positive definite, and I can split this operation into 2 steps each done in parallel. The result must be symmetric. By doing so I get a significant speedup.
To my non-expert eyes, there should be no issue because all threads share C but each operates on a different set of columns. In fact I expect my application to be extremely sensitive to even tiny issues because it needs to maintain symmetry after running 2 such sparse matrix multiplications.
However, this code works on Linux (Ubuntu 18.04, with R 3.6.1 linked to Intel MKL), but not on Windows (I've tried with both vanilla R 3.6.1 and MRO 3.5.3). My application fails in some cases on Windows due to a failed Cholesky, in turn caused by C not being created correctly, which I've traced to the above chunk of code. Simply removing OMP solves the issue, of course at the cost of performance.
I have not been able to reproduce this consistently on Windows: in isolation, the product works correctly. But it is definitely an OMP issue because removing the parallel clause solves the issue.
So what's going on?
Here's a working piece of code
//[[Rcpp::export]]
Rcpp::List spmat_mult(const Eigen::SparseMatrix<double, Eigen::RowMajor>& A,
const Eigen::SparseMatrix<double>& B){
int nchunks = 7;
int chunksize = B.cols() / nchunks;
std::chrono::steady_clock::time_point start = std::chrono::steady_clock::now();
Eigen::SparseMatrix<double> Cs(A.rows(), B.cols());
Cs = A * B;
std::chrono::steady_clock::time_point end = std::chrono::steady_clock::now();
clog << "Standard "
<< std::chrono::duration_cast<std::chrono::microseconds>(end - start).count()
<< "us.\n";
Eigen::SparseMatrix<double> C(A.rows(), B.cols());
C = Cs;
start = std::chrono::steady_clock::now();
#pragma omp parallel for
for(int i=0; i<nchunks; i++){
if(i == nchunks-1){
C.rightCols(B.cols() - (nchunks-1)*chunksize) = A * B.rightCols(B.cols() - (nchunks-1)*chunksize);
} else {
C.middleCols(i*chunksize, chunksize) = A * B.middleCols(i*chunksize, chunksize);
}
}
end = std::chrono::steady_clock::now();
clog << "OMP "
<< std::chrono::duration_cast<std::chrono::microseconds>(end - start).count()
<< "us.\n";
return Rcpp::List::create(
Rcpp::Named("Cs") = Cs,
Rcpp::Named("C") = C
);
}
and on R
library(Matrix)
set.seed(1)
n <- 5000 # choose size
A <- B <- matrix(0, ncol=n, nrow=n)
nnz <- 250
A_rows <- sample(1:n, nnz, replace=F)
A_cols <- sample(1:n, nnz, replace=F)
B_rows <- sample(1:n, nnz, replace=F)
B_cols <- sample(1:n, nnz, replace=F)
A[A_rows, A_cols] <- apply(A[A_rows, A_cols], 1:2, function(x) runif(1))
B[B_rows, B_cols] <- apply(B[B_rows, B_cols], 1:2, function(x) runif(1))
A <- as(A, "dgRMatrix")
B <- as(B, "dgCMatrix")
C <- spmat_mult(A, B)