Add/Remove bytes from end of file on Windows - c++

So, I've looked around, but couldn't find a way to remove bytes from the end of a file without rewriting the entire file. I found that a truncate function works for linux, but didn't find anything for windows. Now, obviously, to expand a file, I can just pad the end with null bytes, but for reducing a file's size, is it literally necessary to rewrite the whole file on windows? or is there a function, maybe in windows.h, that allows me, like truncate on linux, to reassign a file's size?
EDIT: I did just find the function _chdir(int,long), and I'm reading on how to use it.
EDIT: And, why exactly did fstream leave out this vital function?
EDIT: Ok, so it appears that _chdir() will not work (I forgot to mention this, btw), because the function must support files larger than 4 GB - i.e., I'm using 64bit file pointers. I thought that would be inherent, but after reading the arguments to chsize, the length is not size_t.

You truncate a file by calling SetFilePointer or SetFilePointerEx to the desired location followed by SetEndOfFile. The following shows how a truncate function can be implemented:
bool truncate( HANDLE hFile, LARGE_INTEGER NewSize ) {
LARGE_INTEGER Size = { 0 };
if ( GetFileSizeEx( hFile, &Size ) ) {
LARGE_INTEGER Distance = { 0 };
// Negative values move the pointer backward in the file
Distance.QuadPart = NewSize.QuadPart - Size.QuadPart;
return ( SetFilePointerEx( hFile, Distance, NULL, FILE_END ) &&
SetEndOfFile( hFile ) );
}
return false;
}
// Helper function taking a file name instead of a HANDLE
bool truncate( const std::wstring& PathName, LARGE_INTEGER NewSize ) {
HANDLE hFile = CreateFileW( PathName.c_str(), GENERIC_WRITE, FILE_SHARE_READ,
NULL, OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, NULL );
if ( hFile == INVALID_HANDLE_VALUE ) {
return false;
}
bool Success = truncate( hFile, NewSize );
CloseHandle( hFile );
return Success;
}
EDIT: Shorter Version
The truncate function can be shortened to the following:
bool truncate( HANDLE hFile, LARGE_INTEGER NewSize ) {
return ( SetFilePointerEx( hFile, NewSize, NULL, FILE_BEGIN ) &&
SetEndOfFile( hFile ) );
}
If you would rather want to pass the amount of bytes by which to shrink the file, the following implementation can be used:
bool truncate( HANDLE hFile, LARGE_INTEGER ShrinkBy ) {
ShrinkBy.QuadPart = -ShrinkBy.QuadPart;
return ( SetFilePointerEx( hFile, ShrinkBy, NULL, FILE_END ) &&
SetEndOfFile( hFile ) );
}
To grow a file, open the file using CreateFile with a dwDesiredAccess that contains FILE_APPEND_DATA. Using SetFilePointer again to set the file pointer to the end of file you can then write new data calling WriteFile. For an example, see Appending One File to Another File.
EDIT: Growing a file without writing to it
If you don't care about the file contents beyond the original file size you can apply the same sequence as shown for truncating a file to extend it:
bool SetFileSize( HANDLE hFile, LARGE_INTEGER NewSize ) {
return ( SetFilePointerEx( hFile, NewSize, NULL, FILE_BEGIN ) &&
SetEndOfFile( hFile ) );
}
This is documented behavior for SetEndOfFile:
The SetEndOfFile function can be used to truncate or extend a file. If the file is extended, the contents of the file between the old end of the file and the new end of the file are not defined.

You probably want the SetEndOfFile function.
EDIT: This should work with files larger than 4GB. Use the SetFilePointerEx function for that.

To remove bytes from end of file on Windows:
FSUTIL file seteof <filename> <new size>
To add (null)bytes to end of (existing)file:
FSUTIL file seteof <filename> <new size>
No need to copy/"rewrite the whole file on windows"

Related

Read 'Binary' files with ReadFile WinAPI

When I try to open '.exe' files with ReadFile() Windows API, It's just return the 2 first character of file like : MZ
Here is my Code:
#define BUFFERSIZE 5000
VOID CALLBACK FileIOCompletionRoutine(
__in DWORD dwErrorCode,
__in DWORD dwNumberOfBytesTransfered,
__in LPOVERLAPPED lpOverlapped
);
VOID CALLBACK FileIOCompletionRoutine(
__in DWORD dwErrorCode,
__in DWORD dwNumberOfBytesTransfered,
__in LPOVERLAPPED lpOverlapped)
{
_tprintf(TEXT("Error code:\t%x\n"), dwErrorCode);
_tprintf(TEXT("Number of bytes:\t%x\n"), dwNumberOfBytesTransfered);
g_BytesTransferred = dwNumberOfBytesTransfered;
}
HANDLE hFile;
DWORD dwBytesRead = 0;
char ReadBuffer[BUFFERSIZE] = { 0 };
OVERLAPPED ol = { 0 };
hFile = CreateFile(fullFilePath.c_str(), // file to open
GENERIC_READ, // open for reading
FILE_SHARE_READ, // share for reading
NULL, // default security
OPEN_EXISTING, // existing file only
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED, // normal file
NULL); // no attr. template
ReadFileEx(hFile, ReadBuffer, BUFFERSIZE - 1, &ol, FileIOCompletionRoutine);
When I print ReadBuffer It's just MZ(exe file).
But Using:
std::ifstream file(argv[1], std::ios::in | std::ios::binary);
It's work perfectly.
How Can I Read Binary files With ReadFile?
The problem is not with reading, the problem is with printing.
You're not showing your code, but you're likely trying to print with printf or something similar. IOW, you're printing it as C string.
Well, binary data includes 0s, and in this case the first 3 bytes are 'M', 'Z', '\0' - and that prints as a null-terminated string "MZ".
You'd have to write a converter to per-byte hex numbers if you want to see meaningful printing of binary data: 4D 5A 00 and so on
How Can I Read Binary files With ReadFile?
ReadFile (and ReadFileEx ) works 'in binary mode'. You get exact file contents byte by byte without any translation.
You have problem with writing/printing. This mostly depends where you want to write, but for outputing (binary) data potentially containing nulls in C++ choose write method
some_output_stream.write( buffer_ptr, num_bytes_in_buffer );
some_output_stream should be set to binary mode (std::ios::binary). Without this flag all bytes with value 10 could be translated to pairs 13,10.
If C FILE functions are used
fwrite( buffer_ptr, 1, num_bytes_in_buffer, some_output_file );
Again some_output_file has to be in binary mode.
In some scenarios WriteFile can be used complementary to your usage of ReadFile.

how to create own cache for file loading from disk to memory in c++ framwork?

how to create own cache for file loading from disk to memory in c++ framwork,I do not want to use windows cache ,because some of case windows cache does not gives the good results ?
Is there any plugin in c++ I can directly use for multiImport of file from disk to memory .
thanks in advance
What do you mean by "windows cache does not gives the good results"? What version of Windows are we talking about here?
The Windows file cache is actually quite efficient, but there are things a developer can do to their own data (if they fully control it) which can greatly improve the performance of file I/O. In particular, if you ensure your files are organized into multiples of 4096 bytes (aka 4k), you can make use of "overlapped" I/O which avoids both blocking behavior and the need to do additional copies of memory data.
An example of this is the DirectX Tool Kit and the WaveBankReader class. The xwbtool command-line utility is used to pack a number of audio .wav files into a single file where each individual sound file is aligned to the 4096 boundary, a xwb file.
At runtime, the xwb reader then sets up the target memory and issues asynchronous reads. Ideally the application sets up a number of other reads, and then at some later time ensures that all async I/O is complete before using the data.
struct handle_closer { void operator()(HANDLE h) { if (h) CloseHandle(h); } };
typedef public std::unique_ptr<void, handle_closer> ScopedHandle;
inline HANDLE safe_handle( HANDLE h ) { return (h == INVALID_HANDLE_VALUE) ? 0 : h; }
ScopedHandle m_event;
#if (_WIN32_WINNT >= _WIN32_WINNT_VISTA)
m_event.reset( CreateEventEx( nullptr, nullptr, CREATE_EVENT_MANUAL_RESET, EVENT_MODIFY_STATE | SYNCHRONIZE ) );
#else
m_event.reset( CreateEvent( nullptr, TRUE, FALSE, nullptr ) );
#endif
if ( !m_event )
{
return HRESULT_FROM_WIN32( GetLastError() );
}
#if (_WIN32_WINNT >= _WIN32_WINNT_WIN8)
CREATEFILE2_EXTENDED_PARAMETERS params = { sizeof(CREATEFILE2_EXTENDED_PARAMETERS), 0 };
params.dwFileAttributes = FILE_ATTRIBUTE_NORMAL;
params.dwFileFlags = FILE_FLAG_OVERLAPPED | FILE_FLAG_SEQUENTIAL_SCAN;
ScopedHandle hFile( safe_handle( CreateFile2( szFileName,
GENERIC_READ,
FILE_SHARE_READ,
OPEN_EXISTING,
&params ) ) );
#else
ScopedHandle hFile( safe_handle( CreateFileW( szFileName,
GENERIC_READ,
FILE_SHARE_READ,
nullptr,
OPEN_EXISTING,
FILE_FLAG_OVERLAPPED | FILE_FLAG_SEQUENTIAL_SCAN,
nullptr ) ) );
#endif
if ( !hFile )
{
return HRESULT_FROM_WIN32( GetLastError() );
}
// Read and verify header
OVERLAPPED request;
memset( &request, 0, sizeof(request) );
request.hEvent = m_event.get();
bool wait = false;
if( !ReadFile( hFile.get(), &m_header, sizeof( m_header ), nullptr, &request ) )
{
DWORD error = GetLastError();
if ( error != ERROR_IO_PENDING )
return HRESULT_FROM_WIN32( error );
wait = true;
}
DWORD bytes;
#if (_WIN32_WINNT >= _WIN32_WINNT_WIN8)
BOOL result = GetOverlappedResultEx( hFile.get(), &request, &bytes, INFINITE, FALSE );
#else
if ( wait )
(void)WaitForSingleObject( m_event.get(), INFINITE );
BOOL result = GetOverlappedResult( hFile.get(), &request, &bytes, FALSE );
#endif
if ( !result || ( bytes != sizeof( m_header ) ) )
{
return HRESULT_FROM_WIN32( GetLastError() );
}
// ... code here to verify and parse header cut for readability ...
m_waveData.reset( new (std::nothrow) uint8_t[ waveLen ] );
if ( !m_waveData )
return E_OUTOFMEMORY;
dest = m_waveData.get();
memset( &m_request, 0, sizeof(OVERLAPPED) );
m_request.Offset = m_header.Segments[HEADER::SEGIDX_ENTRYWAVEDATA].dwOffset;
m_request.hEvent = m_event.get();
if ( !ReadFile( hFile.get(), dest, waveLen, nullptr, &m_request ) )
{
DWORD error = GetLastError();
if ( error != ERROR_IO_PENDING )
return HRESULT_FROM_WIN32( error );
}
else
{
m_prepared = true;
memset( &m_request, 0, sizeof(OVERLAPPED) );
}
// ...
// At some later point we need to check to see if the data is ready
// or wait if the data is not yet ready
if ( !m_prepared )
{
WaitForSingleObjectEx( m_request.hEvent, INFINITE, FALSE );
m_prepared = true;
}
This code makes use of the buffering hint FILE_FLAG_SEQUENTIAL_SCAN that the file will be read sequentially. You can also use the hint FILE_FLAG_RANDOM_ACCESS if the file will really be access randomly instead, but it is more efficient if you can arrange your data for a sequential scan.
The complexity here is that this code builds for Windows Vista, Windows 7, Windows 8.x, Windows 10, Xbox One, Windows phone 8, Windows 8 Store, and universal Windows apps. Namely I'm using the improved GetOverlappedResultEx on Windows 8 or later, but have to emulate it on older versions of the OS with WaitForSingleObject and GetOverlappedResult.
Having a few dozen outstanding read-requests of a reasonable size can also help optimize disk seek behavior but it is important not to flood the system with lots of small requests. Generally prefer to make read requests of 32k or greater at a time.
If you want to bypass the file cache for some reason (say you are doing streaming of audio and don't want any extra copies of it in memory anywhere because you know the data will only get used once before you read it again) you can use FILE_FLAG_NO_BUFFERING--be sure you aren't opening another handle to the same file without this flag or it will get buffered anyhow:
#if (_WIN32_WINNT >= _WIN32_WINNT_WIN8)
CREATEFILE2_EXTENDED_PARAMETERS params2 = { sizeof(CREATEFILE2_EXTENDED_PARAMETERS), 0 };
params2.dwFileAttributes = FILE_ATTRIBUTE_NORMAL;
params2.dwFileFlags = FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING;
m_async = CreateFile2( szFileName,
GENERIC_READ,
FILE_SHARE_READ,
OPEN_EXISTING,
&params2 );
#else
m_async = CreateFileW( szFileName,
GENERIC_READ,
FILE_SHARE_READ,
nullptr,
OPEN_EXISTING,
FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING,
nullptr );
#endif
As with all optimizations, be sure to profile both with and without FILE_FLAG_NO_BUFFERING under real world loads to make sure you aren't actually making things slower by using it.

CreateFile failing with errorcode 2, while file exists

I try to open existing file via CreateFile, but it is always failing with errorcode 2 - like file doesn't exist, but it exists - it's in folder with executable.
hFile = CreateFile( argv[ 1 ], GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL );
if ( hFile == INVALID_HANDLE_VALUE )
{
printf( "\nError: Unable to open file (%d)\n", GetLastError( ) );
return -1;
}
it fails even if I replace argv[1] with hardcoded filename string.
app is running as an admin.
The error code is accurate. The file cannot be found. Possible explanations include:
You used the wrong file name.
You used a relative path and the process working directory is not what you expect it to be.
If you wish to interpret the file name as being relative to the directory in which the executable resides then do that. Form an absolute path from the directory containing the executable and the specified file name.
There is no reason to expect a process working directory to be the directory in which the executable resides.
You are attempting to open a file using a relative pathname. Relative pathnames are relative to the current working directory (see GetCurrentDirectory). The current working directory is not necessarily the directory, where the executable image resides. It can be different for a number of reasons, for example:
The application was launched through a shortcut that explicitly sets the working directory.
The application called SetCurrentDirectory.
The application was launched through the command interpreter from a directory other than the executable's directory.
If you want to open a file located relative to the application's executable image, you need to construct a fully qualified pathname, based on the executable's location and the desired filename. The following code retrieves the executable's directory1):
#include <windows.h>
#include <Shlwapi.h>
#pragma comment(lib, "Shlwapi.lib")
#include <string>
#include <vector>
std::wstring GetExePath() {
// Retrieve fully qualified module pathname
std::vector<wchar_t> buffer( MAX_PATH );
DWORD cbSize = ::GetModuleFileNameW( nullptr, buffer.data(),
static_cast<DWORD>( buffer.size() ) );
while ( cbSize == buffer.size() ) {
buffer.resize( buffer.size() + MAX_PATH );
cbSize = ::GetModuleFileNameW( nullptr, buffer.data(),
static_cast<DWORD>( buffer.size() ) );
}
if ( cbSize == 0 ) {
throw ::GetLastError();
}
// Remove filename from fully qualified pathname
if ( ::PathRemoveFileSpecW( buffer.data() ) ) {
::PathAddBackslashW( buffer.data() );
}
// Construct string object from character buffer
std::wstring str( &buffer[0] );
return str;
}
This can be used as follows:
int wmain( int argc, const wchar_t* argv[] ) {
if ( argc <= 1 ) {
return -1;
}
std::wstring pathname = GetExePath();
pathname += argv[1];
HANDLE hFile = ::CreateFileW( pathname.c_str(), GENERIC_READ,
FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL, OPEN_EXISTING, 0, NULL );
if ( hFile == INVALID_HANDLE_VALUE )
{
wprintf( L"\nError: Unable to open file (%d)\n", GetLastError() );
return -1;
}
// ...
::CloseHandle( hFile );
return 0;
}
1) Code targeting Windows 8 and later should use PathCchRemoveFileSpec and PathCchAddBackslash instead.

Last write FILETIME always returning current time

I need to compare a file's last modified time to a date time stored in a database. I initially looked at this question to get started.
I am currently getting the FILETIME for the last write of the file, converting it to a SYSTEMTIME. Then I use that SYSTEMTIME to create a TDateTime object that I can use for my comparison. However, the FileModifiedDT variable, is always coming out to be the current time, despite the file having been modified previously.
FILETIME lastWriteTime;
String * FileNamePtr = new String( FileName );
GetFileTime( FileNamePtr, NULL, NULL, &lastWriteTime );
SYSTEMTIME systemTime;
FileTimeToSystemTime( &lastWriteTime, &systemTime );
TDateTime * FileModifiedDT = new TDateTime( systemTime.wYear, systemTime.wMonth,
systemTime.wDay, systemTime.wHour,
systemTime.wMinute, systemTime.wSecond,
systemTime.wMilliseconds );
Am I missusing GetFileTime in some way? Is there a better way I should go about this?
The error is
String * FileNamePtr = new String( FileName );
GetFileTime( FileNamePtr, NULL, NULL, &lastWriteTime );
According to the documentation, the first argument has to be a file handle created by CreateFile and nothing else.
Thus you need something like this:
HANDLE fileHandle = CreateFile(
FileName, //LPCTSTR
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL
);
if ( fileHandle != INVALID_HANDLE )
{
GetFileTime( fileHandle, NULL, NULL, &lastWriteTime );
CloseHandle( fileHandle );
}
else
{
// error, do something else...
}

c++ check if file is empty

I got a project in C++ which I need to edit. This is a declaration of variable:
// Attachment
OFSTRUCT ofstruct;
HFILE hFile = OpenFile( mmsHandle->hTemporalFileName , &ofstruct , OF_READ );
DWORD hFileSize = GetFileSize( (HANDLE) hFile , NULL );
LPSTR hFileBuffer = (LPSTR)GlobalAlloc(GPTR, sizeof(CHAR) * hFileSize );
DWORD hFileSizeReaded = 0;
ReadFile( (HANDLE) hFile , hFileBuffer, hFileSize, &hFileSizeReaded, NULL );
CloseHandle( (HANDLE) hFile );
I need to check if the file is attached (I suppose I need to check if hFile has any value), but don't know how. I tried with hFile == NULL but this doesn't do the job.
Thanks,
Ile
Compare hFile with HFILE_ERROR (not with NULL!). Also, you should change OpenFile to CreateFile and call it properly, OpenFile has long been deprecated. In fact MSDN clearly states:
OpenFile Function
Only use this function with 16-bit
versions of Windows. For newer
applications, use the CreateFile
function.
When you make this change, you will get a HANDLE back, which you should compare with INVALID_HANDLE_VALUE.
Update: Correct way to get a file's size:
LARGE_INTEGER fileSize={0};
// You may want to use a security descriptor, tweak file sharing, etc...
// But this is a boiler plate file open
HANDLE hFile=CreateFile(mmsHandle->hTemporalFileName,GENERIC_READ,0,NULL,
OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,NULL);
if (hFile!=INVALID_HANDLE_VALUE && GetFileSizeEx(hFile,&fileSize) &&
fileSize.QuadPart!=0)
{
// The file has size
}
else
{
// The file is missing or size==0 (or an error occurred getting its size)
}
// Do whatever else and don't forget to close the file handle when done!
if (hFile!=INVALID_HANDLE_VALUE)
CloseHandle(hFile);
Before you open the file you can try this:
WIN32_FIND_DATA wfd;
HANDLE h = FindFirstFile(filename, &wfd);
if (h != INVALID_FILE_HANDLE)
{
// file exists
if (wfd.nFileSizeHigh != 0 || wfd.nFileSizeLow != 0)
{
// file is not empty
}
FindClose(h)
}