I'm reading a binary uint32_t data from a file that indicates the size of the next binary block, after that I read that block but the reading pointer is "moving" wrong.
FILE* file = fopen("file.zip", "r");
long pointerA = ftell(file);
uint32_t streamSize = 0;
fread(reinterpret_cast<char*>(&streamSize), sizeof streamSize,1,file);
long pointerB = ftell(file);
char* zipData = new char[streamSize];
fread(zipData, sizeof(char),streamSize,file);
long pointerC = ftell(file);
fseek( file, pointerA + 4 + streamSize, SEEK_SET );
long pointerD = ftell(file);
qDebug()<<"streamSize"<<streamSize<<"Positions"<<pointerA<<pointerB<<pointerC<<pointerD;
PointerA is the original position, PointerB the position after reading that uint32_t, PointerC is the pointer after reading all that binary data and
PointerD is just a check about what I suppose that should be the right behaviour.
Now let's see the debug:
streamSize 2653 Positions 151 156 4627 2808
Why the stream read position has moved too 4627 instead 2808?
Thank you in advance for any tip!
Both users #alan-birtles and #remy-lebeau were right, I opened it as text instead binary that was the issue.
Unfortunatly I cannot mark this as solved.
PS. For begginers this means open file with "rb" instead "r".
You need to open your file in binary mode. When a file is opened in text mode some characters are changed as you read them. For example on Windows when reading '\n' "\r\n" is returned. To open in binary mode add 'b' to your open mode, e.g:
FILE * file = fopen("file.txt", "rb");
Note you need to do the same when writing binary files otherwise the same transformations take place.
std::fstream also needs std::ios_base::binary to be passed to the constructor/open to avoid the same issue.
Related
I would like to read the data with fread from file. However, I encounter the problem of setting of NULL terminator.I think the the line (fileMem[fileSize] = 0;) should have been solved. However, I still get rubbish at the with I check the value of "fileMem". Would anyone help me to find the problem?
I followed other posts about setting the NULL terminator but just does not work
File *input = fopen(filePath, "r");
fseek(input, 0, SEEK_END);
auto fileSize = ftell(input);
fseek(input, 0, SEEK_SET);
char* fileMem = new char[fileSize+1];
fileMem[fileSize] = 0;// the NULL terminator problem should have been solved here
clearerr(input);
fread(fileMem, fileSize,1, input);
What is the problem with my code?
fread is reading more bytes than fileSize, because you are specifying a record size of fileSize, and asking it to read only one text record. It then overwrites the 0 at the end with actual data, so you get garbage.
fread returns the number of bytes it actually read, so you can allocate a bigger buffer, then use the return value from fread to determine how much of it is valid (and to set a null-terminator).
Since it is updating your data in this way, I also suggest changing the file type to binary ("rb" rather than "r" in call to fopen).
The reason this is happening is because fread performs translation of text when in text mode ("r" rather than "rb"), such as carriage returns and line feeds.
Assuming you are on Windows, I think the problem is that you are opening the file in text mode and using fread which is meant for binary mode. In text mode, what you read may not be exactly what is in the file. Windows text files have "\r\n" at the end of the file, but in text mode this two character combination is converted to a single character, "\n". So the fileSize you computed will be too large and so your null terminator will be in the wrong place.
To verify this, change your fread to be:
int nr = fread( fileMen, 1, fileSize, input);
Swapping the middle args, will have fread return the number of bytes read. If you look at the value of nr, it will be smaller than fileSize because of the line end translation.
This wouldn't be a problem on a *nix system since there is no translation there in text mode.
When you want to use fread to read the contents of a file, you must open the file in binary mode.
FILE *input = fopen(filePath, "rb");
^^
Otherwise, the size of the file you get by using
fseek(input, 0, SEEK_END);
auto fileSize = ftell(input);
will be greater than the number of characters that what can be read by fread.
If you have a CR and a LF, they will count as two characters by the above method but fread will read only one character. Hence, fread will read less than fileSize characters. You can also change the fread line to:
// Swap the middle arguments.
// The first argument is supposed to be the size of each object.
// The second argument is supposed to be the number of objects to read.
auto n = fread(fileMem, 1, fileSize, input);
if ( n != fileSize )
{
// Surprise
}
fileMem[n] = '\0';
For some reason (that I've yet to figure out), the following code doesn't quite work as expected. I'm trying to write an array of shorts to a file and while the file is generated successfully, the values are just not right. The example code is
short* sarray = new short[2000];
for(int i=0;i<2000;i++)
sarray[i]=i*2;
FILE* oFile;
oFile = fopen("E:\\audio_rec\\test_1.raw", "w");
int shortsRead =fwrite(sarray, sizeof(short), 2000, oFile);
fclose(oFile);
The file content (as displayed in a hex editor). It's clear that only the first 5 values (up to 8) are as expected.
Any ideas?
Thanks
In Windows you need to tell fopen that the data is binary or it messes up "\n" to "\r\n".
fopen("E:\\audio_rec\\test_1.raw", "wb");
I am actually writing a c++ program that reads any kind of file and saves it as a bmp file, but first I need to read the file, and thats were the issue is
char fileName[] = "test.jpg";
FILE * inFileForGettingSize;//This is for getting the file size
fopen_s(&inFileForGettingSize, fileName, "r");
fseek(inFileForGettingSize, 0L, SEEK_END);
int fileSize = ftell(inFileForGettingSize);
fclose(inFileForGettingSize);
ifstream inFile;//This is for reading the file
inFile.open(fileName);
if (inFile.fail()) {
cerr << "Error Opening File" << endl;
}
char * data = new char[fileSize];
inFile.read(data, fileSize);
ofstream outFile;//Writing the file back again
outFile.open("out.jpg");
outFile.write(data, fileSize);
outFile.close();
cin.get();
But when I read the file, lets say its a plainttext file it allways outputs some wierd charactes at the end, for example:
assdassaasd
sdaasddsa
sdadsa
passes to:
assdassaasd
sdaasddsa
sdadsaÍÍÍ
So when I do this with a jpg, exe, etc. It corrupts it.
I am not trying to COPY a file, I know there are other ways for that, Im just trying to read a complete file byte per byte. Thanks.
EDIT:
I found out that those 'Í' are equal to the number of end lines the file has, but this doesn't help me much
This is caused by newline handling.
You open the files in text mode (because you use "r" instead of "rb" for fopen and because you don't pass ios::binary to your fstream open calls), and on Windows, text mode translates "\r\n" pairs to "\n" on reading and back to "\r\n" when writing. The result is that the in-memory size is going to be shorter than the on-disk size, so when you try to write using the on-disk size, you go past the end of your array and write whatever random stuff happens to reside in memory.
You need to open files in binary mode when working with binary data:
fopen_s(&inFileForGettingSize, fileName, "rb");
inFile.open(fileName, ios::binary);
outFile.open("out.jpg", ios::binary);
For future reference, your copy routine could be improved. Mixing FILE* I/O with iostream I/O feels awkward, and opening and closing the file twice is extra work, and (most importantly), if your routine is ever run on a large enough file, it will exhaust memory trying to load the entire file into RAM. Copying a block at a time would be better:
const int BUFFER_SIZE = 65536;
char buffer[BUFFER_SIZE];
while (source.good()) {
source.read(buffer, BUFFER_SIZE);
dest.write(buffer, source.gcount());
}
It's a binary file, so you need to read and write the file as binary; otherwise it's treated as text, and assumed to have newlines that need translation.
In your call to fopen(), you need add the "b" designator:
fopen_s(&inFileForGettingSize, fileName, "rb");
And in your fstream::open calls, you need to add std::fstream::binary:
inFile.open(fileName, std::fstream::binary);
// ...
outFile.open("out.jpg", std::fstream::binary);
RESOLVED
I'm trying to make a simple file loader.
I aim to get the text from a shader file (plain text file) into a char* that I will compile later.
I've tried this function:
char* load_shader(char* pURL)
{
FILE *shaderFile;
char* pShader;
// File opening
fopen_s( &shaderFile, pURL, "r" );
if ( shaderFile == NULL )
return "FILE_ER";
// File size
fseek (shaderFile , 0 , SEEK_END);
int lSize = ftell (shaderFile);
rewind (shaderFile);
// Allocating size to store the content
pShader = (char*) malloc (sizeof(char) * lSize);
if (pShader == NULL)
{
fputs ("Memory error", stderr);
return "MEM_ER";
}
// copy the file into the buffer:
int result = fread (pShader, sizeof(char), lSize, shaderFile);
if (result != lSize)
{
// size of file 106/113
cout << "size of file " << result << "/" << lSize << endl;
fputs ("Reading error", stderr);
return "READ_ER";
}
// Terminate
fclose (shaderFile);
return 0;
}
But as you can see in the code I have a strange size difference at the end of the process which makes my function crash.
I must say I'm quite a beginner in C so I might have missed some subtilities regarding the memory allocation, types, pointers...
How can I solve this size issue?
*EDIT 1:
First, I shouldn't return 0 at the end but pShader; that seemed to be what crashed the program.
Then, I change the type of reult to size_t, and added a end character to pShader, adding pShdaer[result] = '/0'; after its declaration so I can display it correctly.
Finally, as #JamesKanze suggested, I turned fopen_s into fopen as the previous was not usefull in my case.
First, for this sort of raw access, you're probably better off
using the system level functions: CreateFile or open,
ReadFile or read and CloseHandle or close, with
GetFileSize or stat to get the size. Using FILE* or
std::filebuf will only introduce an additional level of
buffering and processing, for no gain in your case.
As to what you are seeing: there is no guarantee that an ftell
will return anything exploitable as a numeric value; it could
very well be just a magic cookie. On most current systems, it
is a byte offset into the physical file, but on any non-Unix
system, the offset into the physical file will not map directly
to the logical file you are reading unless you open the file in
binary mode. If you use "rb" to open the file, you'll
probably see the same values. (Theoretically, you could get
extra 0's at the end of the file, but practically, the OS's
where that happened are either extinct, or only used on legacy
mainframes.)
EDIT:
Since the answer stating this has been deleted: you should loop
on the fread until it returns 0 (setting errno to 0 before
each call, and checking it after the return to see whether the
function returned because of an error or because it reached the
end of file). Having said this: if you're on one of the usual
Windows or Unix systems, and the file is local to the machine,
and not too big, fread will read it all in one go. The
difference in size you are seeing (given the numerical values
you posted) is almost certainly due to the fact that the two
byte Windows line endings are being mapped to a single '\n'
character. To avoid this, you must open in binary mode;
alternatively, if you really are dealing with text (and want
this mapping), you can just ignore the extra bytes in your
buffer, setting the '\0' terminator after the last byte
actually read.
I am writing a C library that reads a file into memory. It skips the first 54 bytes of the file (header) and then reads the remainder as data. I use fseek to determine the length of the file, and then use fread to read in the file.
The loop runs once and then ends because the EOF is reached (no errors). At the end, bytesRead = 10624, ftell(stream) = 28726, and the buffer contains 28726 values. I expect fread to read 30,000 bytes and the file position to be 30054 when EOF is reached.
C is not my native language so I suspect I've got a dumb beginner mistake somewhere.
Code is as follows:
const size_t headerLen = 54;
FILE * stream;
errno_t ferrno = fopen_s( &stream, filename.c_str(), "r" );
if(ferrno!=0) {
return -1;
}
fseek( stream, 0L, SEEK_END );
size_t bytesTotal = (size_t)(ftell( stream )) - headerLen; //number of data bytes to read
size_t bytesRead = 0;
BYTE* localBuffer = new BYTE[bytesTotal];
fseek(stream,headerLen,SEEK_SET);
while(!feof(stream) && !ferror(stream)) {
size_t result = fread(localBuffer+bytesRead,sizeof(BYTE),bytesTotal-bytesRead,stream);
bytesRead+=result;
}
Depending on the reference you use, it's quite apparent that adding a "b" to the mode flag is the answer. Seeking nominations for the bonehead-badge. :-)
This reference talks about it in the second paragraph, second sentence (though not in their table).
MSDN doesn't discuss the binary flag until halfway down the page.
OpenGroup mentions the existance of the "b" tag, but states that it "shall have no effect".
perhaps it's a binary mode issue. Try opening the file with "r+b" as the mode.
EDIT: as noted in a comment "rb" is likely a better match to your original intent since "r+b" will open it for read/write and "rb" is read-only.
Also worth noting that simply including binmode.obj into your link command will do this for you for all file opens.
A solution, based on the previous answers:
size_t bytesRead = 0;
BYTE* localBuffer = new BYTE[bytesTotal];
fseek(stream,headerLen,SEEK_SET);
while(!feof(stream) && !ferror(stream)) {
size_t result = fread(localBuffer+bytesRead,sizeof(BYTE),bytesTotal-
bytesRead,stream);
bytesRead+=result;
}