I would like to read the data with fread from file. However, I encounter the problem of setting of NULL terminator.I think the the line (fileMem[fileSize] = 0;) should have been solved. However, I still get rubbish at the with I check the value of "fileMem". Would anyone help me to find the problem?
I followed other posts about setting the NULL terminator but just does not work
File *input = fopen(filePath, "r");
fseek(input, 0, SEEK_END);
auto fileSize = ftell(input);
fseek(input, 0, SEEK_SET);
char* fileMem = new char[fileSize+1];
fileMem[fileSize] = 0;// the NULL terminator problem should have been solved here
clearerr(input);
fread(fileMem, fileSize,1, input);
What is the problem with my code?
fread is reading more bytes than fileSize, because you are specifying a record size of fileSize, and asking it to read only one text record. It then overwrites the 0 at the end with actual data, so you get garbage.
fread returns the number of bytes it actually read, so you can allocate a bigger buffer, then use the return value from fread to determine how much of it is valid (and to set a null-terminator).
Since it is updating your data in this way, I also suggest changing the file type to binary ("rb" rather than "r" in call to fopen).
The reason this is happening is because fread performs translation of text when in text mode ("r" rather than "rb"), such as carriage returns and line feeds.
Assuming you are on Windows, I think the problem is that you are opening the file in text mode and using fread which is meant for binary mode. In text mode, what you read may not be exactly what is in the file. Windows text files have "\r\n" at the end of the file, but in text mode this two character combination is converted to a single character, "\n". So the fileSize you computed will be too large and so your null terminator will be in the wrong place.
To verify this, change your fread to be:
int nr = fread( fileMen, 1, fileSize, input);
Swapping the middle args, will have fread return the number of bytes read. If you look at the value of nr, it will be smaller than fileSize because of the line end translation.
This wouldn't be a problem on a *nix system since there is no translation there in text mode.
When you want to use fread to read the contents of a file, you must open the file in binary mode.
FILE *input = fopen(filePath, "rb");
^^
Otherwise, the size of the file you get by using
fseek(input, 0, SEEK_END);
auto fileSize = ftell(input);
will be greater than the number of characters that what can be read by fread.
If you have a CR and a LF, they will count as two characters by the above method but fread will read only one character. Hence, fread will read less than fileSize characters. You can also change the fread line to:
// Swap the middle arguments.
// The first argument is supposed to be the size of each object.
// The second argument is supposed to be the number of objects to read.
auto n = fread(fileMem, 1, fileSize, input);
if ( n != fileSize )
{
// Surprise
}
fileMem[n] = '\0';
Related
I'm reading a binary uint32_t data from a file that indicates the size of the next binary block, after that I read that block but the reading pointer is "moving" wrong.
FILE* file = fopen("file.zip", "r");
long pointerA = ftell(file);
uint32_t streamSize = 0;
fread(reinterpret_cast<char*>(&streamSize), sizeof streamSize,1,file);
long pointerB = ftell(file);
char* zipData = new char[streamSize];
fread(zipData, sizeof(char),streamSize,file);
long pointerC = ftell(file);
fseek( file, pointerA + 4 + streamSize, SEEK_SET );
long pointerD = ftell(file);
qDebug()<<"streamSize"<<streamSize<<"Positions"<<pointerA<<pointerB<<pointerC<<pointerD;
PointerA is the original position, PointerB the position after reading that uint32_t, PointerC is the pointer after reading all that binary data and
PointerD is just a check about what I suppose that should be the right behaviour.
Now let's see the debug:
streamSize 2653 Positions 151 156 4627 2808
Why the stream read position has moved too 4627 instead 2808?
Thank you in advance for any tip!
Both users #alan-birtles and #remy-lebeau were right, I opened it as text instead binary that was the issue.
Unfortunatly I cannot mark this as solved.
PS. For begginers this means open file with "rb" instead "r".
You need to open your file in binary mode. When a file is opened in text mode some characters are changed as you read them. For example on Windows when reading '\n' "\r\n" is returned. To open in binary mode add 'b' to your open mode, e.g:
FILE * file = fopen("file.txt", "rb");
Note you need to do the same when writing binary files otherwise the same transformations take place.
std::fstream also needs std::ios_base::binary to be passed to the constructor/open to avoid the same issue.
RESOLVED
I'm trying to make a simple file loader.
I aim to get the text from a shader file (plain text file) into a char* that I will compile later.
I've tried this function:
char* load_shader(char* pURL)
{
FILE *shaderFile;
char* pShader;
// File opening
fopen_s( &shaderFile, pURL, "r" );
if ( shaderFile == NULL )
return "FILE_ER";
// File size
fseek (shaderFile , 0 , SEEK_END);
int lSize = ftell (shaderFile);
rewind (shaderFile);
// Allocating size to store the content
pShader = (char*) malloc (sizeof(char) * lSize);
if (pShader == NULL)
{
fputs ("Memory error", stderr);
return "MEM_ER";
}
// copy the file into the buffer:
int result = fread (pShader, sizeof(char), lSize, shaderFile);
if (result != lSize)
{
// size of file 106/113
cout << "size of file " << result << "/" << lSize << endl;
fputs ("Reading error", stderr);
return "READ_ER";
}
// Terminate
fclose (shaderFile);
return 0;
}
But as you can see in the code I have a strange size difference at the end of the process which makes my function crash.
I must say I'm quite a beginner in C so I might have missed some subtilities regarding the memory allocation, types, pointers...
How can I solve this size issue?
*EDIT 1:
First, I shouldn't return 0 at the end but pShader; that seemed to be what crashed the program.
Then, I change the type of reult to size_t, and added a end character to pShader, adding pShdaer[result] = '/0'; after its declaration so I can display it correctly.
Finally, as #JamesKanze suggested, I turned fopen_s into fopen as the previous was not usefull in my case.
First, for this sort of raw access, you're probably better off
using the system level functions: CreateFile or open,
ReadFile or read and CloseHandle or close, with
GetFileSize or stat to get the size. Using FILE* or
std::filebuf will only introduce an additional level of
buffering and processing, for no gain in your case.
As to what you are seeing: there is no guarantee that an ftell
will return anything exploitable as a numeric value; it could
very well be just a magic cookie. On most current systems, it
is a byte offset into the physical file, but on any non-Unix
system, the offset into the physical file will not map directly
to the logical file you are reading unless you open the file in
binary mode. If you use "rb" to open the file, you'll
probably see the same values. (Theoretically, you could get
extra 0's at the end of the file, but practically, the OS's
where that happened are either extinct, or only used on legacy
mainframes.)
EDIT:
Since the answer stating this has been deleted: you should loop
on the fread until it returns 0 (setting errno to 0 before
each call, and checking it after the return to see whether the
function returned because of an error or because it reached the
end of file). Having said this: if you're on one of the usual
Windows or Unix systems, and the file is local to the machine,
and not too big, fread will read it all in one go. The
difference in size you are seeing (given the numerical values
you posted) is almost certainly due to the fact that the two
byte Windows line endings are being mapped to a single '\n'
character. To avoid this, you must open in binary mode;
alternatively, if you really are dealing with text (and want
this mapping), you can just ignore the extra bytes in your
buffer, setting the '\0' terminator after the last byte
actually read.
My function gets a FILE* to read from, and it needs to read starting from some non-negative offset.
I could use fseek(file, offset, SEEK_SET), but it fails on stdin, for instance.
How can I determine if fseek works? If it doesn't, I could read and discard offset bytes.
And is there a way to read (and discard) from FILE without allocating read buffer?
You can test if fseek works on that stream, by calling fseek( file, offset, SEEK_SET) and on error, checking that errno == EBADF which is returned to say "The stream specified is not a seekable stream".
I think you need to read and discard, with a buffer, but if it can just be pagesize bytes and you keep a count of bytes read, reading till you did the equivalent of a seek. If it were a memory mappable file, then you can read without reading, but then the seek would have worked.
The return value of fseek() tells you if it worked or not:
Return Value
...Upon successful completion, fgetpos(), fseek(), fsetpos() return 0, ...Otherwise, -1 is returned and errno is set to indicate the error.
So attempt to fseek() from the file and check the return result, and handle your failure case accordingly. Ex:
ret = fseek(stdin, 0, SEEK_SET);
if(ret < 0)
printf("failed because: %s\n", strerror(errno));
will give you something like:
failed because: Illegal seek
So that failed because you can't seek stdin, where as:
FILE * fp = fopen("word.txt", "r");
ret = fseek(fp, 0, SEEK_SET);
if(ret < 0)
printf("failed because: %s\n", strerror(errno));
Wouldn't print anything because you got back 0 indicating success (assuming of course that "word.txt" exists, is readable, was opened successfully, etc).
I don't understand this part of your question:
is there a way to read (and discard) from FILE without allocating read buffer
You can just fseek() to the point you want to read, or you can read into an array to a buffer and then overwrite the results. The answer depends on your goals, but using things like fread() or read() will require a non-null pointer to store data into.
Just this:
int size = getFileSize(path); //Listed below
ifstream fs(path, ios::in);
ofstream os(path2, ios::out);
//Check - both streams are valid
char buff[CHUNK_SIZE]; //512
while (size > CHUNK_SIZE)
{
fs >> buff;
os << buff;
size -= CHUNK_SIZE;
}
char* dataLast = new char[size];
fs>>dataLast;
os<<dataLast;
fs.close();
os.close();
//Found on SO, works fine
int getFileSize(string path)
{
FILE *pFile = NULL;
if (fopen_s( &pFile, path.c_str(), "rb" ))
{
return 0;
}
fseek( pFile, 0, SEEK_END );
int Size = ftell( pFile );
fclose( pFile );
return Size;
}
File at path2 is corrupted and less then 1 Kb. (initial file is 30Kb);
I don't need advices how to copy file, I am curios what is wrong about this example.
First an important warning: Never (as in really never) use the formatted input operator for char* without setting the width()! You open yourself up to a buffer overrun. This is basically the C++ version of writing gets() which was bad enough to be removed (not just deprecated) from the C standard! If you insist in using formatted input with char* (normally you are much better off using std::string), set the width, e.g.:
char buffer[512];
in >> std::setw(sizeof(buffer) >> buffer;
OK, with this out of the way: it seems you actually want to change two important things:
You probably don't want to use formatted input, i.e., operator>>(): the formatted input operators start off with skipping whitespace. When reading into char* it also stops when reaching a whitespace (or when the width() is non-zero when having read a much characters and still space to store a terminating zero; note that the set width() will be reset to 0 after each of these reads). That is you probably want to use unformatted input, e.g., in.read(buffer, sizeof(buffer)) which sets in.gcount() to the number of characters actually read which may be less then size parameter, e.g., at the end of the stream.
You probably should open the file in std::ios_base::binary mode. Although it doesn't matter on some systems (e.g., POSIX systems) on some systems reading in text mode merges a line end sequence, e.g. \r\n on Windows, into the line end character \n. Likewise, when writing a \n in text mode, it will be replaced by a line end sequence on some system, i.e., you probably also want to open the output stream in text mode.
Th input and output operators, when used with strings (like buff is from the libraries point of view), reads space-delimited words, only.
If you want to read chunks, then use std::istream::read, and use std::istream::gcount to get the number of bytes actually read. Then write with std::ostream::write.
And if the data in the file is binary, you should use the binary open mode.
I am writing a C library that reads a file into memory. It skips the first 54 bytes of the file (header) and then reads the remainder as data. I use fseek to determine the length of the file, and then use fread to read in the file.
The loop runs once and then ends because the EOF is reached (no errors). At the end, bytesRead = 10624, ftell(stream) = 28726, and the buffer contains 28726 values. I expect fread to read 30,000 bytes and the file position to be 30054 when EOF is reached.
C is not my native language so I suspect I've got a dumb beginner mistake somewhere.
Code is as follows:
const size_t headerLen = 54;
FILE * stream;
errno_t ferrno = fopen_s( &stream, filename.c_str(), "r" );
if(ferrno!=0) {
return -1;
}
fseek( stream, 0L, SEEK_END );
size_t bytesTotal = (size_t)(ftell( stream )) - headerLen; //number of data bytes to read
size_t bytesRead = 0;
BYTE* localBuffer = new BYTE[bytesTotal];
fseek(stream,headerLen,SEEK_SET);
while(!feof(stream) && !ferror(stream)) {
size_t result = fread(localBuffer+bytesRead,sizeof(BYTE),bytesTotal-bytesRead,stream);
bytesRead+=result;
}
Depending on the reference you use, it's quite apparent that adding a "b" to the mode flag is the answer. Seeking nominations for the bonehead-badge. :-)
This reference talks about it in the second paragraph, second sentence (though not in their table).
MSDN doesn't discuss the binary flag until halfway down the page.
OpenGroup mentions the existance of the "b" tag, but states that it "shall have no effect".
perhaps it's a binary mode issue. Try opening the file with "r+b" as the mode.
EDIT: as noted in a comment "rb" is likely a better match to your original intent since "r+b" will open it for read/write and "rb" is read-only.
Also worth noting that simply including binmode.obj into your link command will do this for you for all file opens.
A solution, based on the previous answers:
size_t bytesRead = 0;
BYTE* localBuffer = new BYTE[bytesTotal];
fseek(stream,headerLen,SEEK_SET);
while(!feof(stream) && !ferror(stream)) {
size_t result = fread(localBuffer+bytesRead,sizeof(BYTE),bytesTotal-
bytesRead,stream);
bytesRead+=result;
}