Problems reading/writing to binary file - c++

I wrote the code below to save a vector of struct to a binary file. The problem is the reading stops before all the written data is read.
Changing the reinterpret_cast<const char*> with (char*) or (const char*) does not help or change the output.
Seeing as it does read the right number for the data it does read the code is at least partially right.
Is there something obvious i'm missing? Or why does this not work?
#include <iostream>
#include <fstream>
#include <vector>
struct Pixel {
long long data;
Pixel() {
data = 0;
}
Pixel(long long data) :
data(data)
{}
};
void saveToBin(std::vector<Pixel>* pixels) {
std::ofstream binFile;
binFile.open("test.bin", std::ios::binary);
if (!binFile.is_open()) {
perror("Error open");
exit(EXIT_FAILURE);
}
for (int i = 0; i < pixels->size(); i++) {
binFile.write(reinterpret_cast<const char*>(&pixels->at(i)),sizeof(Pixel));
}
binFile.close();
}
void readBin(std::vector<Pixel>* pixels) {
std::ifstream binFile;
binFile.open("test.bin", std::ifstream::in);
if (!binFile.is_open()) {
perror("Error open");
exit(EXIT_FAILURE);
}
Pixel p;
while (binFile.read(reinterpret_cast<char*>(&p), sizeof(Pixel))) {
pixels->push_back(p);
}
binFile.close();
}
int main()
{
std::vector<Pixel> vecOrig;
for (int i = 0; i < 1000; i++) {
vecOrig.push_back(Pixel(i*7));
}
std::vector<Pixel> vecRead;
saveToBin(&vecOrig);
readBin(&vecRead);
std::cout << "starting length : " << vecOrig.size() << " , read length : " << vecRead.size() << std::endl;
for (int i = 0; i < std::min(vecOrig.size(), vecRead.size()); i++) {
std::cout << vecOrig[i].data << " -> " << vecRead[i].data << std::endl;
}
}

binFile.open("test.bin", std::ios::binary);
The output file was opened in binary mode.
binFile.open("test.bin", std::ifstream::in);
The input file was not.
This only matters on operating systems that trace their lineage to MS-DOS, and if your binary file happens to have a 0x0D byte it will be gratuously deleted, when read. Open the input file in binary mode, too.

Related

Using read() to store data into a char array

I'm working on a project where I need to read a binary file and then operate on the data(count how many times 3 appears). My approach was to use read() to store data into a char buffer. Then operate on the buffer. My biggest issue is using read() correctly, I don't believe it is storing any data into char buffer. char buffer remains empty after read() is called.
// Program reads how many times the integer 3 occurs in a binary file
#include <iostream>
#include <fstream> // needed to use files
using namespace std;
int main()
{
int count = 0;
char buffer[32];
int size = sizeof(buffer) / sizeof(buffer[0]);
// open file "example.bin"
fstream file("example.bin", ios::in | ios::ate | ios::binary);
if(!file)
{
cout << "Error opening file" << endl;
return 0;
}
cout << "opened file" << endl;
// reading the data into memory
file.read(reinterpret_cast<char *>(buffer), sizeof(buffer));
for(int i = 0; i < size; i++)
{
if (buffer[i] == '3')
count++;
}
cout << "There exists " << count << " threes in this file." << endl;
// close file
file.close();
return 0;
}

Trouble when writing to a file in a loop

Here I have two pieces of C++ code that is supposed to write to a file some data. The first one is below and it works:
void ParameterManager::Save()
{
std::ofstream saveFile;
saveFile.open(path, std::ios::trunc | std::ios::out);
if (saveFile.is_open())
{
saveFile << "File opened. Begin saving.\n";
for (int i = 0; i < 4; ++i)
{
saveFile << "Hoppa" << std::endl;
}
}
saveFile.close();
}
The result in output file is:
File opened. Begin saving.
Hoppa
Hoppa
Hoppa
Hoppa
Like expected.
The second one is below and it don't work:
void ParameterManager::Save()
{
std::ofstream saveFile;
saveFile.open(path, std::ios::trunc | std::ios::out);
if (saveFile.is_open())
{
saveFile << "File opened. Begin saving.\n";
for (auto item : map)
{
std::cout << "Hoppa" << std::endl;
saveFile << "Hoppa" << std::endl;
}
}
saveFile.close();
}
Where the map is a hash map containing 4 entries and it is a member of the class where the function Save is implememnted. The result in output file is:
File opened. Begin saving.
The Hoppa lines are printed in terminal but never written to file. I verified in debug mode and the execution write 4 times but the content is not written to file.
I test it on a Virtual machine windows 7 pro. The host is MacBookPro. I use Visual Studio 2013 Pro.
Can you help me please to understand why the second version of code don't work like expected.
A big thanks for all of you.
Welp, I dunno, but it works when I use std::map<int, int> and input 4 std::pair<int, int> and use the same for loop:
http://coliru.stacked-crooked.com/a/e694252e96aebab5
#include <iostream>
#include <fstream>
#include <string>
#include <map>
void save()
{
std::map<int, int> mappa;
for (size_t i = 0; i < 4; ++i) {
mappa.insert(mappa.begin(), std::pair<int, int>(i,i));
}
std::ofstream saveFile;
saveFile.open("test.txt", std::ios::trunc | std::ios::out);
if (saveFile.is_open())
{
saveFile << "File opened. Begin saving.\n";
for (auto it : mappa)
{
saveFile << "Hoppa" << std::endl;
std::cout << "PRINTED LINE" << std::endl;
}
}
saveFile.close();
}
int main() {
save();
}
you can try:
for (auto &item : map)
{
std::cout << item << std::endl;
saveFile << "Hoppa" << std::endl;
}

How to tail a log file in C++? [duplicate]

I want to create a small code in C++ with the same functionality as "tail-f": watch for new lines in a text file and show them in the standard output.
The idea is to have a thread that monitors the file
Is there an easy way to do it without opening and closing the file each time?
Have a look at inotify on Linux or kqueue on Mac OS.
Inotify is Linux kernel subsystem that allows you to subscribe for events on files and it will report to your application when the even happened on your file.
Just keep reading the file. If the read fails, do nothing. There's no need to repeatedly open and close it. However, you will find it much more efficient to use operating system specific features to monitor the file, should your OS provide them.
Same as in https://stackoverflow.com/a/7514051/44729 except that the code below uses getline instead of getc and doesn't skip new lines
#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
using namespace std;
static int last_position=0;
// read file untill new line
// save position
int find_new_text(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
// check if the new file started
if(filesize < last_position){
last_position=0;
}
// read file from last position untill new line is found
for(int n=last_position;n<filesize;n++) {
infile.seekg( last_position,ios::beg);
char test[256];
infile.getline(test, 256);
last_position = infile.tellg();
cout << "Char: " << test <<"Last position " << last_position<< endl;
// end of file
if(filesize == last_position){
return filesize;
}
}
return 0;
}
int main() {
for(;;) {
std::ifstream infile("filename");
int current_position = find_new_text(infile);
sleep(1);
}
}
I read this in one of Perl manuals, but it is easily translated into standard C, which, in turn, can be translated to istreams.
seek FILEHANDLE,POSITION,WHENCE
Sets FILEHANDLE's position, just like the "fseek" call of
"stdio".
<...>
A WHENCE of 1 ("SEEK_CUR") is useful for not moving the file
position:
seek(TEST,0,1);
This is also useful for applications emulating "tail -f". Once
you hit EOF on your read, and then sleep for a while, you might
have to stick in a seek() to reset things. The "seek" doesn't
change the current position, but it does clear the end-of-file
condition on the handle, so that the next "<FILE>" makes Perl
try again to read something. We hope.
As far as I remember, fseek is called iostream::seekg. So you should basically do the same: seek to the end of the file, then sleep and seek again with ios_base::cur flag to update end-of-file and read some more data.
Instead of sleeping, you may use inotify, as suggested in the other answer, to sleep (block while reading from an emulated file, actually) exactly until the file is updated/closed. But that's Linux-specific, and is not standard C++.
I needed to implement this too, I just wrote a quick hack in standard C++. The hack searches for the last 0x0A (linefeed character) in a file and outputs all data following that linefeed when the last linefeed value becomes a larger value. The code is here:
#include <iostream>
#include <string>
#include <fstream>
using namespace std;
int find_last_linefeed(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
for(int n=1;n<filesize;n++) {
infile.seekg(filesize-n-1,ios::beg);
char c;
infile.get(c);
if(c == 0x0A) return infile.tellg();
}
}
int main() {
int last_position=-1;
for(;;) {
ifstream infile("testfile");
int position = find_last_linefeed(infile);
if(position > last_position) {
infile.seekg(position,ios::beg);
string in;
infile >> in;
cout << in << endl;
}
last_position=position;
sleep(1);
}
}
#include <iostream>
#include <fstream>
#include <string>
#include <list>
#include <sys/stat.h>
#include <stdlib.h>
#define debug 0
class MyTail
{
private:
std::list<std::string> mLastNLine;
const int mNoOfLines;
std::ifstream mIn;
public:
explicit MyTail(int pNoOfLines):mNoOfLines(pNoOfLines) {}
const int getNoOfLines() {return mNoOfLines; }
void getLastNLines();
void printLastNLines();
void tailF(const char* filename);
};
void MyTail::getLastNLines()
{
if (debug) std::cout << "In: getLastNLines()" << std::endl;
mIn.seekg(-1,std::ios::end);
int pos=mIn.tellg();
int count = 1;
//Get file pointer to point to bottom up mNoOfLines.
for(int i=0;i<pos;i++)
{
if (mIn.get() == '\n')
if (count++ > mNoOfLines)
break;
mIn.seekg(-2,std::ios::cur);
}
//Start copying bottom mNoOfLines string into list to avoid I/O calls to print lines
std::string line;
while(getline(mIn,line)) {
int data_Size = mLastNLine.size();
if(data_Size >= mNoOfLines) {
mLastNLine.pop_front();
}
mLastNLine.push_back(line);
}
if (debug) std::cout << "Out: getLastNLines()" << std::endl;
}
void MyTail::printLastNLines()
{
for (std::list<std::string>::iterator i = mLastNLine.begin(); i != mLastNLine.end(); ++i)
std::cout << *i << std::endl;
}
void MyTail::tailF(const char* filename)
{
if (debug) std::cout << "In: TailF()" << std::endl;
int date = 0;
while (true) {
struct stat st;
stat (filename, &st);
int newdate = st.st_mtime;
if (newdate != date){
system("#cls||clear");
std::cout << "Print last " << getNoOfLines() << " Lines: \n";
mIn.open(filename);
date = newdate;
getLastNLines();
mIn.close();
printLastNLines();
}
}
if (debug) std::cout << "Out: TailF()" << std::endl;
}
int main(int argc, char **argv)
{
if(argc==1) {
std::cout << "No Extra Command Line Argument Passed Other Than Program Name\n";
return 0;
}
if(argc>=2) {
MyTail t1(10);
t1.tailF(argv[1]);
}
return 0;
}

Binary input/output in C++

I am trying a reasonably simple program to test binary input/output. I am basically writing a file with a header (string) and some data (doubles). The code is as follows:
#include <iostream>
#include <iomanip>
#include <fstream>
#include <string>
#include <vector>
#include <iterator>
#include <algorithm>
int main() {
typedef std::ostream_iterator<double> oi_t;
typedef std::istream_iterator<double> ii_t;
std::ofstream ofs("data.bin", std::ios::in);
//-If file doesn't exist, create a new one now
if(!ofs) {
ofs.open("data.bin", std::ios::out|std::ios::binary|std::ios::app);
}
else {
ofs.close();
ofs.open("data.bin", std::ios::out|std::ios::binary|std::ios::app);
}
//-Write a header consisting of length of grid subdomain and its name
///*
const std::string grid = "Header";
unsigned int olen = grid.size();
ofs.write(reinterpret_cast<const char*>(&olen), sizeof(olen));
ofs.write(grid.c_str(), olen);
//*/
//-Now write the data
///*
std::vector<double> data_out;
//std::vector<std::pair<int, int> > cell_ids;
for(int i=0; i<100; ++i) {
data_out.push_back(5.0*double(i) + 100.0);
}
ofs << std::setprecision(4);
std::copy(data_out.begin(), data_out.end(), oi_t(ofs, " "));
//*/
ofs.close();
//-Now read the binary file; first header then data
std::ifstream ifs("data.bin", std::ios::binary);
///*
unsigned int ilen;
ifs.read(reinterpret_cast<char*>(&ilen), sizeof(ilen));
std::string header;
if(ilen > 0) {
char* buf = new char[ilen];
ifs.read(buf,ilen);
header.append(buf,ilen);
delete[] buf;
}
std::cout << "Read header: " << header << "\n";
//*/
///*
std::vector<double> data_in;
ii_t ii(ifs);
std::copy(ii, ii_t(), std::back_inserter(data_in));
std::cout << "Read data size: " << data_in.size() << "\n";
//*/
ifs.close();
//-Check the result
///*
for(int i=0; i < data_out.size(); ++i) {
std::cout << "Testing input/output element #" << i << " : "
<< data_out[i] << " " << data_in[i] << "\n";
}
std::cout << "Element sizes: " << data_out.size() << " " << data_in.size() <<
"\n";
//*/
return 0;
}
The problem is that when I try to write and read (and then print) both the header and the data it fails (I confirmed that it doesn't read the data then, but displays the header correctly). But when I comment out one of the write sections (header and/or data), it displays that part correctly indicating the read worked. I am sure I am not doing the read properly. Perhaps I am missing the usage of seekg somewhere.
The code runs fine for me. However you never check if the file is successfully opened for writing, so it could be silently failing on your system. After you open ofs you should add
if (!ofs) {
std::cout << "Could not open file for writing" << std::endl;
return 1;
}
And the same thing after you open ifs
if (!ifs) {
std::cout << "Could not open file for reading" << std::endl;
return 1;
}
Or something along those lines. Also I do not understand why you check if the file exists first since you do the same whether it exists or not.
This should work
#include <iostream>
using std::cout;
using std::cerr;
using std::cin;
using std::endl;
#include <fstream>
using std::ifstream;
#include <cstdint>
int main() {
ifstream fin;
fin.open("input.dat", std::ios::binary | std::ios::in);
if (!fin) {
cerr << "Cannot open file " << "input.dat" << endl;
exit(1);
}
uint8_t input_byte;
while (fin >> input_byte) {
cout << "got byte " << input_byte << endl;
}
return 0;
}

Implement "tail -f" in C++

I want to create a small code in C++ with the same functionality as "tail-f": watch for new lines in a text file and show them in the standard output.
The idea is to have a thread that monitors the file
Is there an easy way to do it without opening and closing the file each time?
Have a look at inotify on Linux or kqueue on Mac OS.
Inotify is Linux kernel subsystem that allows you to subscribe for events on files and it will report to your application when the even happened on your file.
Just keep reading the file. If the read fails, do nothing. There's no need to repeatedly open and close it. However, you will find it much more efficient to use operating system specific features to monitor the file, should your OS provide them.
Same as in https://stackoverflow.com/a/7514051/44729 except that the code below uses getline instead of getc and doesn't skip new lines
#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
using namespace std;
static int last_position=0;
// read file untill new line
// save position
int find_new_text(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
// check if the new file started
if(filesize < last_position){
last_position=0;
}
// read file from last position untill new line is found
for(int n=last_position;n<filesize;n++) {
infile.seekg( last_position,ios::beg);
char test[256];
infile.getline(test, 256);
last_position = infile.tellg();
cout << "Char: " << test <<"Last position " << last_position<< endl;
// end of file
if(filesize == last_position){
return filesize;
}
}
return 0;
}
int main() {
for(;;) {
std::ifstream infile("filename");
int current_position = find_new_text(infile);
sleep(1);
}
}
I read this in one of Perl manuals, but it is easily translated into standard C, which, in turn, can be translated to istreams.
seek FILEHANDLE,POSITION,WHENCE
Sets FILEHANDLE's position, just like the "fseek" call of
"stdio".
<...>
A WHENCE of 1 ("SEEK_CUR") is useful for not moving the file
position:
seek(TEST,0,1);
This is also useful for applications emulating "tail -f". Once
you hit EOF on your read, and then sleep for a while, you might
have to stick in a seek() to reset things. The "seek" doesn't
change the current position, but it does clear the end-of-file
condition on the handle, so that the next "<FILE>" makes Perl
try again to read something. We hope.
As far as I remember, fseek is called iostream::seekg. So you should basically do the same: seek to the end of the file, then sleep and seek again with ios_base::cur flag to update end-of-file and read some more data.
Instead of sleeping, you may use inotify, as suggested in the other answer, to sleep (block while reading from an emulated file, actually) exactly until the file is updated/closed. But that's Linux-specific, and is not standard C++.
I needed to implement this too, I just wrote a quick hack in standard C++. The hack searches for the last 0x0A (linefeed character) in a file and outputs all data following that linefeed when the last linefeed value becomes a larger value. The code is here:
#include <iostream>
#include <string>
#include <fstream>
using namespace std;
int find_last_linefeed(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
for(int n=1;n<filesize;n++) {
infile.seekg(filesize-n-1,ios::beg);
char c;
infile.get(c);
if(c == 0x0A) return infile.tellg();
}
}
int main() {
int last_position=-1;
for(;;) {
ifstream infile("testfile");
int position = find_last_linefeed(infile);
if(position > last_position) {
infile.seekg(position,ios::beg);
string in;
infile >> in;
cout << in << endl;
}
last_position=position;
sleep(1);
}
}
#include <iostream>
#include <fstream>
#include <string>
#include <list>
#include <sys/stat.h>
#include <stdlib.h>
#define debug 0
class MyTail
{
private:
std::list<std::string> mLastNLine;
const int mNoOfLines;
std::ifstream mIn;
public:
explicit MyTail(int pNoOfLines):mNoOfLines(pNoOfLines) {}
const int getNoOfLines() {return mNoOfLines; }
void getLastNLines();
void printLastNLines();
void tailF(const char* filename);
};
void MyTail::getLastNLines()
{
if (debug) std::cout << "In: getLastNLines()" << std::endl;
mIn.seekg(-1,std::ios::end);
int pos=mIn.tellg();
int count = 1;
//Get file pointer to point to bottom up mNoOfLines.
for(int i=0;i<pos;i++)
{
if (mIn.get() == '\n')
if (count++ > mNoOfLines)
break;
mIn.seekg(-2,std::ios::cur);
}
//Start copying bottom mNoOfLines string into list to avoid I/O calls to print lines
std::string line;
while(getline(mIn,line)) {
int data_Size = mLastNLine.size();
if(data_Size >= mNoOfLines) {
mLastNLine.pop_front();
}
mLastNLine.push_back(line);
}
if (debug) std::cout << "Out: getLastNLines()" << std::endl;
}
void MyTail::printLastNLines()
{
for (std::list<std::string>::iterator i = mLastNLine.begin(); i != mLastNLine.end(); ++i)
std::cout << *i << std::endl;
}
void MyTail::tailF(const char* filename)
{
if (debug) std::cout << "In: TailF()" << std::endl;
int date = 0;
while (true) {
struct stat st;
stat (filename, &st);
int newdate = st.st_mtime;
if (newdate != date){
system("#cls||clear");
std::cout << "Print last " << getNoOfLines() << " Lines: \n";
mIn.open(filename);
date = newdate;
getLastNLines();
mIn.close();
printLastNLines();
}
}
if (debug) std::cout << "Out: TailF()" << std::endl;
}
int main(int argc, char **argv)
{
if(argc==1) {
std::cout << "No Extra Command Line Argument Passed Other Than Program Name\n";
return 0;
}
if(argc>=2) {
MyTail t1(10);
t1.tailF(argv[1]);
}
return 0;
}