I did a sample project to read a file into a buffer.
When I use the tellg() function it gives me a larger value than the
read function is actually read from the file. I think that there is a bug.
here is my code:
EDIT:
void read_file (const char* name, int *size , char*& buffer)
{
ifstream file;
file.open(name,ios::in|ios::binary);
*size = 0;
if (file.is_open())
{
// get length of file
file.seekg(0,std::ios_base::end);
int length = *size = file.tellg();
file.seekg(0,std::ios_base::beg);
// allocate buffer in size of file
buffer = new char[length];
// read
file.read(buffer,length);
cout << file.gcount() << endl;
}
file.close();
}
main:
void main()
{
int size = 0;
char* buffer = NULL;
read_file("File.txt",&size,buffer);
for (int i = 0; i < size; i++)
cout << buffer[i];
cout << endl;
}
tellg does not report the size of the file, nor the offset
from the beginning in bytes. It reports a token value which can
later be used to seek to the same place, and nothing more.
(It's not even guaranteed that you can convert the type to an
integral type.)
At least according to the language specification: in practice,
on Unix systems, the value returned will be the offset in bytes
from the beginning of the file, and under Windows, it will be
the offset from the beginning of the file for files opened in
binary mode. For Windows (and most non-Unix systems), in text
mode, there is no direct and immediate mapping between what
tellg returns and the number of bytes you must read to get to
that position. Under Windows, all you can really count on is
that the value will be no less than the number of bytes you have
to read (and in most real cases, won't be too much greater,
although it can be up to two times more).
If it is important to know exactly how many bytes you can read,
the only way of reliably doing so is by reading. You should be
able to do this with something like:
#include <limits>
file.ignore( std::numeric_limits<std::streamsize>::max() );
std::streamsize length = file.gcount();
file.clear(); // Since ignore will have set eof.
file.seekg( 0, std::ios_base::beg );
Finally, two other remarks concerning your code:
First, the line:
*buffer = new char[length];
shouldn't compile: you have declared buffer to be a char*,
so *buffer has type char, and is not a pointer. Given what
you seem to be doing, you probably want to declare buffer as
a char**. But a much better solution would be to declare it
as a std::vector<char>& or a std::string&. (That way, you
don't have to return the size as well, and you won't leak memory
if there is an exception.)
Second, the loop condition at the end is wrong. If you really
want to read one character at a time,
while ( file.get( buffer[i] ) ) {
++ i;
}
should do the trick. A better solution would probably be to
read blocks of data:
while ( file.read( buffer + i, N ) || file.gcount() != 0 ) {
i += file.gcount();
}
or even:
file.read( buffer, size );
size = file.gcount();
EDIT: I just noticed a third error: if you fail to open the
file, you don't tell the caller. At the very least, you should
set the size to 0 (but some sort of more precise error
handling is probably better).
In C++17 there are std::filesystem file_size methods and functions, so that can streamline the whole task.
std::filesystem::file_size - cppreference.com
std::filesystem::directory_entry::file_size - cppreference.com
With those functions/methods there's a chance not to open a file, but read cached data (especially with the std::filesystem::directory_entry::file_size method)
Those functions also require only directory read permissions and not file read permission (as tellg() does)
void read_file (int *size, char* name,char* buffer)
*buffer = new char[length];
These lines do look like a bug: you create an char array and save to buffer[0] char. Then you read a file to buffer, which is still uninitialized.
You need to pass buffer by pointer:
void read_file (int *size, char* name,char** buffer)
*buffer = new char[length];
Or by reference, which is the c++ way and is less error prone:
void read_file (int *size, char* name,char*& buffer)
buffer = new char[length];
...
fseek(fptr, 0L, SEEK_END);
filesz = ftell(fptr);
will do the file if file opened through fopen
using ifstream,
in.seekg(0,ifstream::end);
dilesz = in.tellg();
would do similar
Related
I am still learning Cpp, so please advise if I am misunderstanding here.
Using an ESP32, I am trying to read / write files to Flash / FFat. This is the method I have created which should read a file from flash and load it into PSRAM:
unsigned char* storage_read(char* path) {
File file = FFat.open(path);
if(!file) {
Serial.println("no file");
return 0x00;
}
int count = file.size();
unsigned char* buffer = (unsigned char*)ps_malloc(count);
Serial.printf("Bytes: %d\n", count);
Serial.printf("Count: %d\n", sizeof(buffer));
for (int i = 0; i < count; i++) {
buffer[i] = (unsigned char)file.read();
}
file.close();
return buffer;
}
The problem is that I get the contents of my b64 data file, with the addition of several extra bytes of data globbed on the end.
Calling the method with:
Serial.printf("Got: %s", storage_read("/frame/testframe-000.b64"));
I get the output:
Bytes: 684
Count: 4
Got: <myb64string> + <68B of garbage>
Why would sizeof not be returning the proper size?
What would be the proper way of loading this string into a buffer?
Why would sizeof not be returning the proper size?
That's because sizeof() has a very specific function (not very intuitive). It is used - compile time - to query the size of the data type passed to it. Calling sizeof(buffer) returns the size, in bytes, of the type of variable buffer. It's an unsigned char*, so a 4-byte memory address. So that's what you get.
What would be the proper way of loading this string into a buffer?
What I noticed is that you're expecting to load string data from your file, but you don't explicitly terminate it with a zero byte. As you probably know, all C strings must be terminated with a zero byte. Data that you load from the file most likely doesn't have one (unless you took extra care to add it while saving). So when you read a string from a file sized N bytes, allocate a buffer of N+1 bytes, load the file into it and terminate it with a zero. Something like this:
unsigned char* storage_read(char* path) {
File file = FFat.open(path);
if(!file) {
Serial.println("no file");
return 0x00;
}
int count = file.size();
unsigned char* buffer = (unsigned char*)ps_malloc(count + 1); //< Updated
Serial.printf("Bytes: %d\n", count);
Serial.printf("Count: %d\n", sizeof(buffer));
for (int i = 0; i < count; i++) {
buffer[i] = (unsigned char)file.read();
}
buffer[count] = 0; //< Added
file.close();
return buffer;
}
And since you're returning a heap-allocated buffer from your function, take extra care to remember to delete it in caller when finished. This line in your code will leak the memory:
Serial.printf("Got: %s", storage_read("/frame/testframe-000.b64"));
In the following code the read method doesn't seem to fill the given buffer:
ifstream pkcs7_file(file_name, std::ios::binary);
if ( pkcs7_file.fail() )
{
std::cout << "File failed before reading!\n";
}
pkcs7_file.seekg(0, pkcs7_file.end);
size_t len = pkcs7_file.tellg();
char * buffer = new char[len];
pkcs7_file.read(buffer, len);
pkcs7_file.close();
When debugging with VS 2012 and printing, the Len variable is as expected (and not zero) but the buffer doesn't change after the read function - it remains with the same value from before the read.
What am I doing wrong?
You seek to end-of-file, and then try to read. Of course it fails - the file is positioned at EOF, there's no data to read.
I wrote a simple program to read a TXT file. The problem is the file contains some '\0' characters. Here's a sample :
And here's the solution I've found to solve my problem :
FILE *pInput = fopen("Encoded.txt", "rb");
fseek(pInput, 0, SEEK_END);
size_t size = ftell(pInput);
fseek(pInput, 0, SEEK_SET);
char *buffer = new char[size];
for (int i = 0; i < size; i++)
buffer[i] = fgetc(pInput);
I would like to replace the following code :
for (int i = 0; i < size; i++)
buffer[i] = fgetc(pInput);
By just a simple function call. Is there a function which can do this job ?
I tried with fread, fgets but they stop to read at the first '\0' character.
Thanks a lot in advance for your help.
fread is fine for reading arbitrary binary; it returns the number of elements read, which is a value you should store and use in all dealings with your buffer. (Read some documentation on fread to find out how it works.)
(On the other hand, with fgets you won't be able to find out how many characters were read because a pointer to a [assumedly null-terminated] C-string is all you get out of it.)
You need to ensure that your handling of your resultant buffer is zero-safe. That means no strlen or the like, which are all designed to work on ASCII input (more or less).
Quoting cplusplus.com and removing the plumbering that you'll find in the link:
// Open the file with the pointer at the end
ifstream file("example.bin", ios::in|ios::binary|ios::ate);
// Get the file size
streampos size = file.tellg();
// Allocate a block
char* memblock = new char [size];
// We were at the end go to the begining
file.seekg 0, ios::beg);
// Read the whole file
file.read(memblock, size);
Et voilĂ !
None of the posted answers I've read work, so I'm asking again.
I'm trying to copy the string data pointed to by a char pointer into a char array.
I have a function that reads from a ifstream into a char array
char* FileReader::getNextBytes(int numberOfBytes) {
char *buf = new char[numberOfBytes];
file.read(buf, numberOfBytes);
return buf;
}
I then have a struct :
struct Packet {
char data[MAX_DATA_SIZE]; // can hold file name or data
} packet;
I want to copy what is returned from getNextBytes(MAX_DATA_SIZE) into packet.data;
EDIT: Let me show you what I'm getting with all the answers gotten below (memcpy, strcpy, passing as parameter). I'm thinking the error comes from somewhere else. I'm reading a file as binary (it's a png). I'll loop while the fstream is good() and read from the fstream into the buf (which might be the data array). I want to see the length of what I've read :
cout << strlen(packet.data) << endl;
This returns different sizes every time:
8
529
60
46
358
66
156
After that, apparently there are no bytes left to read although the file is 13K + bytes long.
This can be done using standard library function memcpy, which is declared in / :
strcpy(packet.data, buf);
This requires file.read returns proper char series that ends with '\0'. You might also want to ensure numberOfBytes is big enough to accommodate the whole string. Otherwise you could possibly get segmentation fault.
//if buf not properly null terminated added a null char at the end
buf[numberofbytes] = "\0"
//copy the string from buf to struc
strcpy(packet.data, buf);
//or
strncpy(packet.data, buf);
Edit:
Whether or not this is being handled as a string is a very important distinction. In your question, you referred to it as a "string", which is what got us all confused.
Without any library assistance:
char result = reader.getNextBytes(MAX_DATA_SIZE);
for (int i = 0; i < MAX_DATA_SIZE; ++MAX_DATA_SIZE) {
packet.data[i] = result[i];
}
delete [] result;
Using #include <cstring>:
memcpy(packet.data, result, MAX_DATA_SIZE);
Or for extra credit, rewrite getNextBytes so it has an output parameter:
char* FileReader::getNextBytes(int numberOfBytes, char* buf) {
file.read(buf, numberOfBytes);
return buf;
}
Then it's just:
reader.getNextBytes(MAX_DATA_SIZE, packet.data);
Edit 2:
To get the length of a file:
file.seekg (0, ios::end);
int length = file.tellg();
file.seekg (0, ios::beg);
And with that in hand...
char* buffer = new char[length];
file.read(buffer, length);
Now you have the entire file in buffer.
strlen is not a valid way to determine the amount of binary data. strlen just reads until it finds '\0', nothing more. If you want to read a chunk of binary data, just use a std::vector, resize it to the amount of bytes you read from the file, and return it as value. Problem solved.
I am trying to read data from binary file, and having issues. I have reduced it down to the most simple case here, and it still won't work. I am new to c++ so I may be doing something silly but, if anyone could advise I would be very grateful.
Code:
int main(int argc,char *argv[]) {
ifstream myfile;
vector<bool> encoded2;
cout << encoded2 << "\n"<< "\n" ;
myfile.open(argv[2], ios::in | ios::binary |ios::ate );
myfile.seekg(0,ios::beg);
myfile.read((char*)&encoded2, 1 );
myfile.close();
cout << encoded2 << "\n"<< "\n" ;
}
Output
00000000
000000000000000000000000000011110000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Compression_Program(58221) malloc: * error for object 0x10012d: Non-aligned pointer being freed
* set a breakpoint in malloc_error_break to debug
Thanks in advance.
Do not cast a vector<bool>* to a char*. It is does not do anything predictable.
You are reading on encoded2: myfile.read((char*)&encoded2, 1 );. this is wrong. you can to read a bool and then put it in encoded2
bool x;
myfile.read( &x, 1 );
encoded2[0] = x;
Two mistakes here:
you assume the address of a vector is the address of the first element
you rely on vector<bool>
Casting a vector into a char * is not really a good thing, because a vector is an object and stores some state along with its elements.
Here you are probably overwriting the state of the vector, thus the destructor of fails.
Maybe you would like to cast the elements of the vector (which are guaranteed to be stored contiguously in memory). But another trap is that vector<bool> may be implementation-optimized.
Therefore you should do a encoded2.reserve(8) and use myfile.read(reinterpret_cast<char *>(&encoded2[0])).
But probably you want to do something else and we need to know what the purpose is here.
You're overwriting a std::vector, which you shouldn't do. A std::vector is actually a pointer to a data array and an integer (probably a size_t) holding its size; if you overwrite these with practically random bits, data corruption will occur.
Since you're only reading a single byte, this will suffice:
char c;
myfile.read(&c, 1);
The C++ language does not provide an efficient I/O method for reading bits as bits. You have to read bits in groups. Also, you have to worry about Endianess when reading int the bits.
I suggest the old fashioned method of allocating a buffer, reading into the buffer then operating on the buffer.
Allocating a buffer
const unsigned int BUFFER_SIZE = 1024 * 1024; // Let the compiler calculate it.
//...
unsigned char * const buffer = new unsigned char [BUFFER_SIZE]; // The pointer is constant.
Reading in the data
unsigned int bytes_read = 0;
ifstream data_file("myfile.bin", ios::binary); // Open file for input without translations.
data_file.read(buffer, BUFFER_SIZE); // Read data into the buffer.
bytes_read = data_file.gcount(); // Get actual count of bytes read.
Reminders:
delete the buffer when you are
finished with it.
Close the file when you are finished
with it.
myfile.read((char*) &encoded2[0], sizeof(int)* COUNT);
or you can use push_back();
int tmp;
for(int i = 0; i < COUNT; i++) {
myfile.read((char*) &tmp, 4);
encoded2.push_back(tmp);
}