cpp file reading error with stat and read() [closed] - c++

Question

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I am running into this error not regularly, can't reproduce it.
File being read is a read-only file and can't be deleted or modified.
Code is not exactly the same because it is part of something bigger that I am writing but this is the important part of the code which is causing the problem.
This code is for explaining purposes and not to reproduce the problem because of point 1
I am trying to read the file using
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <memory>
#include <exception>
#include <iostream>
#include <glog/logging.h>
using namespace std;
int main() {
string fileName="blah";
struct stat fileStat;
int status = ::stat(fileName.c_str(), &fileStat);
if (status != 0) {
LOG(ERROR) << "Error stating the file";
}
size_t fileSize = fileStat.st_size;
// fileSize is 79626240. I am trying to read block starting from
// 67108864 bytes, so there will be 1251736
size_t fileBlockSize = 16 * 1024 * 1024;
size_t numBlocks = fileSize / fileBlockSize;
size_t offset = numBlocks;
size_t actualSize = fileSize - offset * fileBlockSize;
if (actualSize == 0) {
LOG(INFO) << "You read the entire file";
return 1;
}
int fd = ::open(fileName.c_str(), O_RDONLY);
if (fd < 0) {
throw std::runtime_error("Error opening the file");
} else if (offset > 0 && lseek(fd, offset, SEEK_SET) < 0) {
throw std::runtime_error("Error seeking the file");
}
uint64_t readBlockSize = 256 * 1024;
char *data = new char[readBlockSize + 1];
uint64_t totalRead = 0;
while (totalRead < actualSize) {
ssize_t numRead = ::read(fd, data, readBlockSize);
// Use the data you read upto numRead
if (numRead == 0) {
LOG(ERROR) << "Reached end of file";
break;
} else if (numRead < 0) {
throw std::runtime_error("read unsuccessful");
}
totalRead += numRead;
}
if (totalRead != actualSize) {
LOG(ERROR) << "Error reading the file";
}
}
If you imagine me slicing the file into blocks of size 16 mybtes and then reading the last block. I am reading the block in a loop with a smaller size, however I get EOF before I can finish reading the entire block. Can it ever happen the size reported by stat is greater than the size of data in the file ?
The output I see :
Reached end of file
Error reading the file
I don't need alternative solutions, I can do other things such as lseek to END however I wanna know why this is happening ?
PS It is not because of number of blocks on the disk. I am using st_size and nothing more

Answer 1

You must take care using stat over a file, it is better to use fstat to avoid TOCTOU race conditions.
int fileDescriptor = -1;
struct stat fileStat;
std::vector<char> fileContent;
std::string filename("test.txt");
fileDescriptor = open(filename.c_str(),O_RDONLY);
// Do error check of fileDescriptor
fstat(fileDescriptor,&fileStat);
// Do error check of fstat
fileContent.resize(fileStat.st_size);
::read(fileDescriptor,fileContent.data(),fileStat.st_size);
close(fileDescriptor);
Additionally, consider that read may return a value lesser than fileStat.st_size and you must read the remaining bytes(pretty hard in file I/O, quite common with sockets though), the code is just a small example.
Edit
I have copied your code and modified to load a local 9MB file, after compilation with g++ -g -std=c++11 -lglog main.cpp, I have setup a breakpoint in the line 51
if (totalRead != actualSize)
This is the result from my debug session:
(gdb) b main.cpp:51 Breakpoint 1 at 0x4013fc: file main.cpp, line 51.
(gdb) r Starting program: /home/jpalma/Documents/functionTest/a.out
[Thread debugging using libthread_db enabled] Using host libthread_db
library "/lib64/libthread_db.so.1".
Breakpoint 1, main () at main.cpp:51
51 if (totalRead !=> actualSize) {
(gdb) p totalRead
$1 = 9000032
(gdb) p actualSize
$2 = 9000032
So basically your program works flawless for me. Maybe you have a problem in your filesystem or something not related with this.
I'm using ext4 as filesystem.
ll reports this size from the file I'm reading 9000032 abr 29 16:10 WebDev.pdf, so as you can see it is actually correct. My page size is
$ getconf PAGESIZE
4096

Answer 2

So you question is : can the size reported by fstat, or by stat on a read only non modified file be greater that what will be read from the file ?
First some elements on read return value (from man page) :
On success, the number of bytes read is returned (zero indicates end of file), and the file position is advanced by this number. It is not an error if this number is smaller than the number of bytes requested ... On error, -1 is returned, and errno is set appropriately
So the return value is at max the requested size, can be less, with 0 as end of file indication, and -1 as error indication.
Man page says also that a read of less bytes than requested size may happen for example because fewer bytes are actually available right now (maybe because we were close to end-of-file, or because we are reading from a pipe, or from a terminal), or because read() was interrupted by a signal.
So even if I could never see that, nothing in the documentation guarantees that even reading on the file, you will get as much data as requested unless end of file has been reached.
But it is clearly stated than a return value of 0 means that you are at end of file. As you test end of file with 0 == read(...), all is fine.
For your question : can size reported by stat be different from the number of bytes that can be read from a file, the answer is no, except if the file system is broken, or there are physical read errors. That is the definition of the size member : The st_size field gives the size of the file (if it is a regular file or a symbolic link) in bytes (from stat man page)
But I really cannot understand your code. First I see :
size_t fileSize = fileStat.st_size;
// fileSize is 79626240. I am trying to read block starting from
// 67108864 bytes, so there will be 1251736
size_t fileBlockSize = 16 * 1024 * 1024;
size_t numBlocks = fileSize / fileBlockSize;
size_t offset = numBlocks;
size_t actualSize = fileSize - offset * fileBlockSize;
So actualSize is now 1251736 when file is 79626240 bytes long.
And later, without actualSize has changed :
uint64_t totalRead = 0;
while (totalRead < actualSize) {
ssize_t numRead = ::read(fd, data, readBlockSize);
...
totalRead += numRead;
}
if (totalRead != actualSize) {
LOG(ERROR) << "Error reading the file";
}
As actualSize despite of its name is not the actual file size, you can go into the Error reading the file branch. But if it happens with the true file size, double check your file system.