Faster way to read file than boost::file_mapping? - c++

Question

I'm writing a latency-sensitive app which reads a text file upon initialisation. I have profiled and re-written all my algorithms such that 85% of my execution time is from the lines:
boost::interprocess::file_mapping file(Path, read_only);
boost::interprocess::mapped_region data(file, read_only);
I am writing this on windows- is there any faster way to map a file into memory? Portability is not a concern.

Answer 1

You could just use the native functions of Win32, but I think you won't save alot, because boost will not add alot of overhead:
OFSTRUCT ofStruct;
ofStruct.cBytes=sizeof (OFSTRUCT);
HANDLE file=(HANDLE)OpenFile(fileName, &ofStruct, OF_READ);
if (file==INVALID_HANDLE_VALUE)
handle errors
else {
HANDLE map=CreateFileMapping(file, NULL, PAGE_READONLY, 0, 0, 0);
if (map==INVALID_HANDLE_VALUE)
handle errors
else {
const char *p=(const char *)MapViewOfFile(map, FILE_MAP_READ, 0, 0, 0));
if (p) {
// enjoy using p to read access file contents.
}
// close all that handles now...
}

Answer 2

I would suggest dropping the idea of file mapping.
FM is a complicated construct and adds some overhead. Plain cached read also involves non-trivial interaction with the physical device. You could do unbuffered reads. Probably the next thing to ask is what kind of IO you actually want - how big is the file? is it sequential? Is it on the network? Do you have a choice of hardware, or is it on the customers' machine?

Answer 3

If the files are small, just open and read them into memory using standard Win32 CreateFile()/ReadFile() APIs.
If you're consuming each file sequentially (or could arrange your code in such a way that you do), you should specify FILE_FLAG_SEQUENTIAL_SCAN. This is a hint for the file/caching subsystem to read-ahead aggressively. For small files, the file might be read into cache before your first call to ReadFile() is issued.
Edit: As requested, Here's a snippet that illustrates reading the contents of a file into a vector of bytes using the Win32 API:
void ReadFileIntoBuffer( const std::wstring& fileName, std::vector< uint8_t >& output )
{
HANDLE hFile( INVALID_HANDLE_VALUE );
try
{
// Open the file.
hFile = CreateFile( filename.c_str(),
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
FILE_FLAG_SEQUENTIAL_SCAN,
NULL );
if( INVALID_HANDLE_VALUE != hFile )
throw std::runtime_error( "Failed to open file." );
// Fetch size
LARGE_INTEGER fileSize;
if( !GetFileSizeEx( hFile, &fileSize ) );
throw std::runtime_error( "GetFileSizeEx() failed." );
// Resize output buffer.
output.resize( fileSize.LowPart );
// Read the file contents.
ULONG bytesRead;
if( !ReadFile( hFile, &output[0], fileSize.LowPart, &bytesRead, NULL ) )
throw std::runtime_error( "ReadFile() failed." );
// Recover resources.
CloseHandle( hFile );
}
catch( std::exception& )
{
// Dump the error.
std::cout << e.what() << " GetLastError() = " << GetLastError() << std::endl;
// Recover resources.
if( INVALID_HANDLE_VALUE != hFile )
CloseHandle( hFile );
throw;
}
}