armadillo requested size is too large - c++

Question

I am using armadillo4.300.0. I am operating on a dense matrix of size 2840260x103. I am loading this matrix from a .csv file of size approximately 3.7GB. I have enabled "ARMA_64BIT_WORD" in my application as well as config.hpp under armadillo_bits directory.
#if !defined(ARMA_64BIT_WORD)
#define ARMA_64BIT_WORD
#endif
I am compiling with gcc49 and running on ubuntu 12.04. When I run I am getting the following error. Interestingly, the application occasionally runs too. For eg., if I keep trying for some 10 times, it runs sometime.
error: Mat::init(): requested size is too large
terminate called after throwing an instance of 'std::logic_error'
what(): Mat::init(): requested size is too large
Do I need to take care of something else?
Ramki.

Answer 1

This problem is solved with the Intel MKL library, when we compile with the -DMKL_ILP64 -m64. Typically we focus only on link flags. But it is important to note that these flags must be enabled during compile phase on the gcc command as well. I am not sure how to enable this on openmpi library. Also the lib armadillo.so must link with mkl_ilp64 instead of mkl_lp64. Follow the instruction below.
Building and installing armadillo :
export CXX=icpc
export CC=icpc
export PATH=$PATH:/home/ramki/intel/bin:
Edit $armadillo_root/cmake_aux/Modules/ARMA_FindMKL.cmake, include the PATHS correctly.
Edit $armadillo_root/cmake_aux/Modules/ARMA_FindMKL.cmake, change mkl_lp64 to mkl_ilp64
Edit $armadillo_root/CMakeLists.txt and (1) Change CMAKE_SHARED_LINKER_FLAGS to include the link line by intel link advisor and (2) Change CMAKE_CXX_FLAGS as given by intel link advisor
Run ./configure and make sure MKL library is used for blas and lapack, icpc to be the compiler and the rest to be alright.
Run make .
Verify the linked libraries by running ldd libarmadillo.so. Mainly verify whether it is linked with mkl_ilp64 library and mkl blas and lapack libraries.
Now run make install DESTDIR=local path.
This should work.