win32 I/O performance issues - c++

Question

I have a problem with win32 I/O performance:
I'm trying to achieve a decent writing speed using OpenFile/WriteFile.
Using Resource Monitor (it comes with windows) I measured the writing speed of the following piece of code and I found that it is writing at 2MB/sec...
HANDLE hFile = INVALID_HANDLE_VALUE;
hFile = CreateFile(
L"test",
(GENERIC_READ | GENERIC_WRITE),
FILE_SHARE_READ,
NULL,
OPEN_ALWAYS,
(FILE_ATTRIBUTE_NORMAL |
FILE_FLAG_WRITE_THROUGH |
FILE_FLAG_NO_BUFFERING),
NULL);
if (hFile != INVALID_HANDLE_VALUE)
{
//OK
unsigned long bytesWritten = 0;
unsigned long* Buffer = (unsigned long*)malloc(4096*sizeof(unsigned long));
ZeroMemory(Buffer, 4096); //thanks to 'bash.d'
while (true)
{
/*the infinite loop is intentional
because I wanted to see if the writing speed of 2MB/sec
was right */
WriteFile(hFile,
Buffer,
4096,
&bytesWritten,
NULL);
if (bytesWritten <= 0)
{
break;
}
}
}
I tried with the following and it's the same...
hFile = CreateFile(
L"test",
(GENERIC_READ | GENERIC_WRITE),
FILE_SHARE_READ,
NULL,
OPEN_ALWAYS,
(FILE_ATTRIBUTE_NORMAL);
What am I doing wrong(about the writing speed) ? and how can I improve the writing speed ?
Thank you and sorry for my english
Edit:
I'm writing on a local disk

Answer 1

This is very interesting, and similar to an issue I have, and can reproduce on 2 different servers with Windows Server 2003 SP2 64-bit (single hard drives, not RAID). Simply doing a WriteFile() of 36 bytes and then 99964 bytes in a loop produces similar behavior (I'm guessing it would be the same with a single write, and some other versions of Windows; that's just what I happened to be using). CPU usage starts off very low, and then increases gradually -- on one server, the test was around 50% CPU usage at around 175GB (about 95% of that is kernel time; 60% in my program and 40% in 'System').
You may also try async IO to get the test performance. That is opening the file with FILE_FLAG_OVERLAPPED and using the LPOVERLAPPED argument of WriteFile. You may or may not get better performance with FILE_FLAG_NO_BUFFERING. You will have to test to see.
FILE_FLAG_NO_BUFFERING will generally give you more consistent speeds and better streaming behavior, and it avoids polluting your disk cache with data that you may not need again, but it isn't necessarily faster overall.
You should also test to see what the best size is for each block of IO. In my experience There is a huge performance difference between copying a file 4k at a time and copying it 1Mb at a time.
In my past testing of this (a few years ago) I found that block sizes below about 64kB were dominated by overhead, and total throughput continued to improve with larger block sizes up to about 512KB. I wouldn't be surprised if with today's drives you needed to use block sizes larger than 1MB to get maximum throughput.
The numbers you are currently using appear to be reasonable, but may not be optimal. Also I'm fairly certain that FILE_FLAG_WRITE_THROUGH prevents the use of the on-disk cache and thus will cost you a fair bit of performance.
It would be of worth to try below thing...
1) Enabling FILE_FLAG_SEQUENTIAL_SCAN flag
2) "Enable advanced performance" in "Disk Policies" in the Device Manager
3) Varying disk chunk size from 64 KB to 4096 ...
4) Try FILE_FLAG_NO_BUFFERING

Answer 2

Use async IO bound to a completion port
Pre-grow the file using SetFileValidData
Open the handle with FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH
A consumer grade drive (even 5400RPM) should be able to write ~130MB/sec (single spindle, no raid). No other IO should occur at the same time (no head movement).
See https://github.com/rusanu/writing-a-binary-file-in-c-very-fast for an example.