I read a text file and it works very well.
std::string str_buf = "";
FILE *file = fopen("/home/pi/log_2019-03-07.txt", "r");
if (file != NULL)
{
while (true)
{
char buffer[MAX_BUFFER_SIZE] = { 0x00, };
size_t rSize = fread(buffer, MAX_BUFFER_SIZE, sizeof(char), file);
str_buf += buffer;
if (rSize == 0)
break;
}
printf("%s", str_buf.data());
fclose(file);
}
Then I try to write it to same path, another name. This is the code:
FILE *writefile = fopen("/home/pi/WriteTest.txt", "wb");
if (writefile != NULL)
{
int offset = 0;
while (true)
{
size_t wSize = fwrite(&str_buf.data()[offset], sizeof(char), strlen(str_buf.data()) - 1, writefile);
offset += (int)wSize;
if (offset >= strlen(str_buf.data()))
break;
}
fclose(writefile);
}
If I try to execute this code, it works. I open WriteTest.txt, it has same string. It's perfectly same.
But I found WriteTest.txt's volume is almost 2 twice read text.
Why it happens?
size_t wSize = fwrite(&str_buf.data()[offset], sizeof(char), strlen(str_buf.data()) - 1, writefile);
you start writing the text at offset &str_buf.data()[offset] but you write the length of a string starting at position 0. You are writing offset bytes too much. You should
size_t wSize = fwrite(&str_buf.data()[offset], sizeof(char),
strlen(str_buf.data()) - offset, writefile);
Also, you don't write string length nor a NUL terminator. So you'd have trouble figuring out how much to read unless, like in your simple example, it is at file's end.
And last, it's better to use str_buf.length() rather than strlen. It's faster and works with strings that have NUL in the middle.
Related
Bellow you can find a code snippet that I used to write an string_length with it to binary file but the code does not works as expected. After it writes I opened the output file and the string was located there but when I read the string from file it reads the string partially. It seems that after reading the string_length the file pointer seeks more than what it should and then it missed the first 8 characters of the string!
#include <iostream>
#include <string>
FILE* file = nullptr;
bool open(std::string mode)
{
errno_t err = fopen_s(&file, "test.code", mode.c_str());
if (err == 0) return true;
return false;
}
void close()
{
std::fflush(file);
std::fclose(file);
file = nullptr;
}
int main()
{
open("wb"); // open file in write binary mode
std::string str = "blablaablablaa";
auto sz = str.size();
fwrite(&sz, sizeof sz, 1, file); // first write size of string
fwrite(str.c_str(), sizeof(char), sz, file); // second write the string
close(); // flush the file and close it
open("rb"); // open file in read binary mode
std::string retrived_str = "";
sz = -1;
fread(&sz, sizeof(size_t), 1, file); // it has the right value (i.e 14) but it seems it seeks 8 bytes more!
retrived_str.resize(sz);
fread(&retrived_str, sizeof(char), sz, file); // it missed the first 8 char
close(); // flush the file and close it
std::cout << retrived_str << std::endl;
return 0;
}
PS: I removed checks in the code in order to makes it more readable.
You're clobbering the retrieved_str object with the file contents rather than reading the file contents into the buffer controlled by retrieved_str.
fread(&retrived_str[0], 1, sz, file);
Or, if you're using C++17 with its non-const std::string::data method:
fread(retrived_str.data(), 1, sz, file);
Change
fread(&retrived_str, sizeof(char), sz, file); // it missed the first 8 char
To
fread((void*)( retrived_str.data()), sizeof(char), sz, file); // set the data rather than the object
I am trying to read a binary file's data sadly opening in C++ is a lot different than in python for these things as they have byte mode. It seems C++ does not have that.
for (auto p = directory_iterator(path); p != directory_iterator(); p++) {
if (!is_directory(p->path()))
byte tmpdata;
std::ifstream tmpreader;
tmpreader.open(desfile, std::ios_base::binary);
int currentByte = tmpreader.get();
while (currentByte >= 0)
{
//std::cout << "Does this get Called?" << std::endl;
int currentByte = tmpreader.get();
tmpdata = currentByte;
}
tmpreader.close()
}
else
{
continue;
}
I want basically a clone of Python's methods of opening a file in 'rb' mode. To have to actual byte data of all of the contents (which is not readable as it has nonprintable chars even for C++. Most of which probably cant be converted to signed chars just because it contains zlib compressed data that I need to feed in my DLL to decompress it all.
I do know that in Python I can do something like this:
file_object = open('[file here]', 'rb')
turns out that replacing the C++ Code above with this helps. However fopen is depreciated but I dont care.
What the Code above did not do was work because I was not reading from the buffer data. I did realize later that fopen, fseek, fread, and fclose was the functions I needed for read bytes mode ('rb').
for (auto p = directory_iterator(path); p != directory_iterator(); p++) {
if (!is_directory(p->path()))
{
std::string desfile = p->path().filename().string();
byte tmpdata;
unsigned char* data2;
FILE *fp = fopen("data.d", "rb");
fseek(fp, 0, SEEK_END); // GO TO END OF FILE
size_t size = ftell(fp);
fseek(fp, 0, SEEK_SET); // GO BACK TO START
data2 = new unsigned char[size];
tmpdata = fread(data2, 1, size, fp);
fclose(fp);
}
else
{
continue;
}
int currentByte = tmpreader.get();
while (currentByte >= 0)
{
//std::cout << "Does this get Called?" << std::endl;
int currentByte = tmpreader.get();
//^ here!
You are declaring a second variable hiding the outer one. However, this inner one is only valid within the while loop's body, so the while condition checks the outer variable which is not modified any more. Rather do it this way:
int currentByte;
while ((currentByte = tmpreader.get()) >= 0)
{
I am Trying to read 64000 bytes from file in binary mode in buffer at one time till end of the file. My problem is tellg() returns position in hexadecimal value, How do I make it return decimal value?
because my if conditions are not working, it is reading more than 64000 and when I am relocating my pos and size_stream(size_stream = size_stream - 63999;
pos = pos + 63999;), it is pointing to wrong positions each time.
How do I read 64000 bytes from file into buffer in binary mode at once till the end of file?
Any help would be appreciated
std::fstream fin(file, std::ios::in | std::ios::binary | std::ios::ate);
if (fin.good())
{
fin.seekg(0, fin.end);
int size_stream = (unsigned int)fin.tellg(); fin.seekg(0, fin.beg);
int pos = (unsigned int)fin.tellg();
//........................<sending the file in blocks
while (true)
{
if (size_stream > 64000)
{
fin.read(buf, 63999);
buf[64000] = '\0';
CString strText(buf);
SendFileContent(userKey,
(LPCTSTR)strText);
size_stream = size_stream - 63999;
pos = pos + 63999;
fin.seekg(pos, std::ios::beg);
}
else
{
fin.read(buf, size_stream);
buf[size_stream] = '\0';
CString strText(buf);
SendFileContent(userKey,
(LPCTSTR)strText); break;
}
}
My problem is tellg() returns position in hexadecimal value
No, it doesn't. It returns an integer value. You can display the value in hex, but it is not returned in hex.
when I am relocating my pos and size_stream(size_stream = size_stream - 63999; pos = pos + 63999;), it is pointing to wrong positions each time.
You shouldn't be seeking in the first place. After performing a read, leave the file position where it is. The next read will pick up where the previous read left off.
How do I read 64000 bytes from file into buffer in binary mode at once till the end of file?
Do something more like this instead:
std::ifstream fin(file, std::ios::binary);
if (fin)
{
unsigned char buf[64000];
std::streamsize numRead;
do
{
numRead = fin.readsome(buf, 64000);
if ((!fin) || (numRead < 1)) break;
// DO NOT send binary data using `LPTSTR` string conversions.
// Binary data needs to be sent *as-is* instead.
//
SendFileContent(userKey, buf, numRead);
}
while (true);
}
Or this:
std::ifstream fin(file, std::ios::binary);
if (fin)
{
unsigned char buf[64000];
std::streamsize numRead;
do
{
if (!fin.read(buf, 64000))
{
if (!fin.eof()) break;
}
numRead = fin.gcount();
if (numRead < 1) break;
// DO NOT send binary data using `LPTSTR` string conversions.
// Binary data needs to be sent *as-is* instead.
//
SendFileContent(userKey, buf, numRead);
}
while (true);
}
My program read bytes from file and try convert it to string. Head of file is a text in CP866.
iconv_t cd = iconv_open("UTF-8","CP866");
char* iconv_in = bytes.data(); //Bytes is a char vector
char* iconv_out = (char *)malloc(counter * sizeof(char)); //counter is a length of bytes array (char vector)
size_t iconv_in_bytes =counter;
size_t iconv_out_bytes = counter;
size_t ret = iconv(cd, &iconv_in, &iconv_in_bytes, &iconv_out, &iconv_out_bytes);
if ((size_t) -1 == ret) {
cout << "Error convert";
return NULL;
}
Program ends with failure (Error convert). And this way is not simple and beautiful. There is exist more simple solution?
I am trying to write a large binary file into a buffer in a C++ program. GDB always gets a segfault after trying to create a buffer the same size as the file read. It either fails on fclose(pf), rewind, or f(open) which leads me to believe that there is something wrong when I am trying to create the buffer. My code segment is as follows.
static int fileTransfer(struct mg_connection *conn, char * filename){
FILE *fp = fopen(filename, "r");
fseek(fp, 0, SEEK_END);
int size = ftell(fp);
char buf[size];
fclose(fp);
// This is an attempt to stop a segment fault from rewind.
fp = fopen(filename, "r");
conn->connection_param = (void *) fp;
size_t n = 0;
if(fp != NULL)
{
n = fread(buf, 1, sizeof(buf), fp);
mg_send_data(conn, buf, n);
if(n < sizeof(buf) || conn->wsbits != 0)
{
fclose(fp);
conn->connection_param = NULL;
}
}
return 1;
}
I have tried putting print statements in this code but they don't print to the console as they are running in a separate thread. Can someone give me some insight on why this segfault is happening, or some suggestions on how to make this code more efficient.
I should note that this code works properly on 1 and 10 MB files but not on anything larger.
never do this:
int size = ftell(fp);
char buf[size];
You are creating size on the STACK, not on the heap.... 100MB on the stack will not work.
AND... size must be a constant number, not a number coming from ftell(). I even don't know how it is compiling...
What you have to to is to allocate memory using malloc() or new operator.
static int fileTransfer(struct mg_connection *conn, char * filename){
FILE *fp = fopen(filename, "r");
fseek(fp, 0, SEEK_END);
int size = ftell(fp);
char * buf = new char[size]; // fix also here!
fclose(fp);
// This is an attempt to stop a segment fault from rewind.
fp = fopen(filename, "r");
conn->connection_param = (void *) fp;
size_t n = 0;
if(fp != NULL)
{
n = fread(buf, 1, size, fp); // fix also here!
mg_send_data(conn, buf, n);
if(n < size || conn->wsbits != 0)
{
fclose(fp);
conn->connection_param = NULL;
}
}
delete [] buf; // and you have to deallocate your buffer
return 1;
}
You are creating a buffer with automatic storage duration, which means it will be put on the stack by g++. The default stack size for any OS known to me is below 100 MB, meaning it will cause a segfault on system supporting them.
Try allocating your buffer with dynamic storage duration, which will place it on the heap.
What's going on is actually the namesake of this site! Basically, what is happening is your program is created it has a set amount of memory allocated for the stack.
When you create char buf[size], you are using a C99 feature called a variable length array (VLA). This allocates space on the stack for buf. However, buf is too large for the stack, so your program fails.
In order to fix this problem, you should use char * buf; and then do buf = malloc(size). This will place buf on the heap, which is larger than the stack. It also lets you check if you do not have enough memory, by checking if malloc() returns NULL. You need to be sure to free(buf) before you exit though!
As a side note, you can check how much space you have on the stack by using the ulimit -s command.
That seems like a lot to allocate on the stack. What if you put it on the heap instead?
char *buf = new char[size];
Use std::vector. Then you don't have the issues of stack space, or the other issue of writing non-standard C++ code:
#include <vector>
//...
static int fileTransfer(struct mg_connection *conn, char * filename)
{
FILE *fp = fopen(filename, "r");
fseek(fp, 0, SEEK_END);
int size = ftell(fp);
std::vector<char> buf(size);
fclose(fp);
fp = fopen(filename, "r");
conn->connection_param = (void *) fp;
size_t n = 0;
if(fp != NULL)
{
n = fread(&buf[0], 1, buf.size(), fp);
mg_send_data(conn, &buf[0], n);
if(n < buf.size() || conn->wsbits != 0)
{
fclose(fp);
conn->connection_param = NULL;
}
}
return 1;
}