Speeding up large file write to disk in c++ - c++

Question

I need to write a large csv file to disk. I've reduced the problem to the below code. When I compile with VS 2017 run on my Windows 7 box it runs on average in 26 seconds. Could someone suggest a way to speed this up without changing the data container or output format? Any help will be appreciated.
PS:Probably obvious, but the speedup should be to the base case on your hardware
I tried using fopen and fprintf but got worse results. I also played around with setting the buffer size with no success.
#include <iostream>
#include <iomanip>
#include <fstream>
#include <chrono>
#include <vector>
#include <string>
typedef std::chrono::high_resolution_clock Clock;
typedef std::vector<double> VecD;
typedef std::vector<VecD> VecVecD;
void test_file_write_stream() {
VecVecD v(10000, VecD(2000, 1.23456789));
const std::string delimiter(",");
const std::string file_path("c:\\junk\\speedtest.csv");
auto t1_stream = Clock::now();
std::ofstream ostream(file_path.c_str());
if (!ostream.good())
return;
ostream << std::setprecision(12);
for (const auto & row : v) {
for (const auto & col : row) {
ostream << col << delimiter;
}
ostream << std::endl;
}
auto t2_stream = Clock::now();
std::cout << "Stream test: " << std::chrono::duration_cast<std::chrono::microseconds>(t2_stream - t1_stream).count() / 1.0e6 << " seconds" << std::endl;
}
void main(int argc, char * argv[]) {
test_file_write_stream();
}
Stream test: 26.2086 seconds

Answer 1

What you wan't to use is memory mapped files according to wikipedi:
The benefit of memory mapping a file is increasing I/O performance,
especially when used on large files.
Why ? because the data is not needed to be copied around an extra time - and you should start seeing increases in the ballpark of 50%-100% or maybe more.
boost has a very neat interface in boost-interprocess. I do not have a testbench for this atm., but something ala:
boost::interprocess::file_mapping fm(filename, ...);
boost::interprocess::mapped_region region(fm, ...);
//mapped_region is a memory mapped file
Otherwise, you can of course use the interface for your platform:
https://learn.microsoft.com/en-us/dotnet/standard/io/memory-mapped-files

Answer 2

I tried your code and made some minor hipshot-optimizations - but got consistant results. The baseline was ~14 s for me, and it stayed close to that all the time. Good work -O3.
I replaced your std::vector<std::vector<double>> with a one-dimensional version and std::copyed it to disk with only a minor improvement.
Only when I gave up and dumped the memory as-is (not as a .csv) to disk did I get down to 1-3 seconds.
The big drawback is that what I dumped isn't portable in any way. It can hopefully be read back to the same computer on a good day, but I wouldn't recommend dumping doubles this way for real:
#include <chrono>
#include <fstream>
#include <iomanip>
#include <iostream>
#include <iterator>
#include <string>
#include <vector>
typedef std::chrono::high_resolution_clock Clock;
// custom 2D array with a 1D memory layout
template<typename T>
class array2d {
public:
array2d(size_t h, size_t w, const T& value = T{}) : data_(h * w, value), w_(w) {}
inline double* operator[](size_t y) { return &data_[y * w_]; }
inline double const* operator[](size_t y) const { return &data_[y * w_]; }
inline size_t width() const { return w_; }
T const* data() const { return data_.data(); }
size_t size() const { return data_.size(); }
private:
std::vector<T> data_;
size_t w_;
};
using VecVecD = array2d<double>;
void test_file_write_stream() {
VecVecD v(10000, 2000, 1.23456789);
const std::string delimiter(",");
const std::string file_path("c:\\junk\\speedtest.csv");
auto t1_stream = Clock::now();
std::ofstream ostream(file_path.c_str(), std::ios::binary);
if(!ostream) return;
ostream << std::setprecision(12);
/* this may give a somewhat better performance than yours, but not much:
std::copy(v.data(), v.data() + v.size(),
std::ostream_iterator<double>(ostream, delimiter));
*/
// non-portable binary dump
ostream.write(reinterpret_cast<const char*>(v.data()),
static_cast<std::streamsize>(v.size() * sizeof(double)));
auto t2_stream = Clock::now();
auto elapsed_s =
std::chrono::duration_cast<std::chrono::seconds>(t2_stream - t1_stream);
std::cout << "Stream test: " << elapsed_s.count() << " seconds" << std::endl;
}
int main(int, char**) {
test_file_write_stream();
}