Binary input/output in C++ - c++

I am trying a reasonably simple program to test binary input/output. I am basically writing a file with a header (string) and some data (doubles). The code is as follows:
#include <iostream>
#include <iomanip>
#include <fstream>
#include <string>
#include <vector>
#include <iterator>
#include <algorithm>
int main() {
typedef std::ostream_iterator<double> oi_t;
typedef std::istream_iterator<double> ii_t;
std::ofstream ofs("data.bin", std::ios::in);
//-If file doesn't exist, create a new one now
if(!ofs) {
ofs.open("data.bin", std::ios::out|std::ios::binary|std::ios::app);
}
else {
ofs.close();
ofs.open("data.bin", std::ios::out|std::ios::binary|std::ios::app);
}
//-Write a header consisting of length of grid subdomain and its name
///*
const std::string grid = "Header";
unsigned int olen = grid.size();
ofs.write(reinterpret_cast<const char*>(&olen), sizeof(olen));
ofs.write(grid.c_str(), olen);
//*/
//-Now write the data
///*
std::vector<double> data_out;
//std::vector<std::pair<int, int> > cell_ids;
for(int i=0; i<100; ++i) {
data_out.push_back(5.0*double(i) + 100.0);
}
ofs << std::setprecision(4);
std::copy(data_out.begin(), data_out.end(), oi_t(ofs, " "));
//*/
ofs.close();
//-Now read the binary file; first header then data
std::ifstream ifs("data.bin", std::ios::binary);
///*
unsigned int ilen;
ifs.read(reinterpret_cast<char*>(&ilen), sizeof(ilen));
std::string header;
if(ilen > 0) {
char* buf = new char[ilen];
ifs.read(buf,ilen);
header.append(buf,ilen);
delete[] buf;
}
std::cout << "Read header: " << header << "\n";
//*/
///*
std::vector<double> data_in;
ii_t ii(ifs);
std::copy(ii, ii_t(), std::back_inserter(data_in));
std::cout << "Read data size: " << data_in.size() << "\n";
//*/
ifs.close();
//-Check the result
///*
for(int i=0; i < data_out.size(); ++i) {
std::cout << "Testing input/output element #" << i << " : "
<< data_out[i] << " " << data_in[i] << "\n";
}
std::cout << "Element sizes: " << data_out.size() << " " << data_in.size() <<
"\n";
//*/
return 0;
}
The problem is that when I try to write and read (and then print) both the header and the data it fails (I confirmed that it doesn't read the data then, but displays the header correctly). But when I comment out one of the write sections (header and/or data), it displays that part correctly indicating the read worked. I am sure I am not doing the read properly. Perhaps I am missing the usage of seekg somewhere.

The code runs fine for me. However you never check if the file is successfully opened for writing, so it could be silently failing on your system. After you open ofs you should add
if (!ofs) {
std::cout << "Could not open file for writing" << std::endl;
return 1;
}
And the same thing after you open ifs
if (!ifs) {
std::cout << "Could not open file for reading" << std::endl;
return 1;
}
Or something along those lines. Also I do not understand why you check if the file exists first since you do the same whether it exists or not.

This should work
#include <iostream>
using std::cout;
using std::cerr;
using std::cin;
using std::endl;
#include <fstream>
using std::ifstream;
#include <cstdint>
int main() {
ifstream fin;
fin.open("input.dat", std::ios::binary | std::ios::in);
if (!fin) {
cerr << "Cannot open file " << "input.dat" << endl;
exit(1);
}
uint8_t input_byte;
while (fin >> input_byte) {
cout << "got byte " << input_byte << endl;
}
return 0;
}

Related

How to read data from AVRO file using C++ interface?

I'm attempting to write a simple program to extract some data from a bunch of AVRO files. The schema for each file may be different so I would like to read the files generically (i.e. without having to pregenerate and then compile in the schema for each) using the C++ interface.
I have been attempting to follow the generic.cc example but it assumes a separate schema where I would like to read the schema from each AVRO file.
Here is my code:
#include <fstream>
#include <iostream>
#include "Compiler.hh"
#include "DataFile.hh"
#include "Decoder.hh"
#include "Generic.hh"
#include "Stream.hh"
const std::string BOLD("\033[1m");
const std::string ENDC("\033[0m");
const std::string RED("\033[31m");
const std::string YELLOW("\033[33m");
int main(int argc, char**argv)
{
std::cout << "AVRO Test\n" << std::endl;
if (argc < 2)
{
std::cerr << BOLD << RED << "ERROR: " << ENDC << "please provide an "
<< "input file\n" << std::endl;
return -1;
}
avro::DataFileReaderBase dataFile(argv[1]);
auto dataSchema = dataFile.dataSchema();
// Write out data schema in JSON for grins
std::ofstream output("data_schema.json");
dataSchema.toJson(output);
output.close();
avro::DecoderPtr decoder = avro::binaryDecoder();
auto inStream = avro::fileInputStream(argv[1]);
decoder->init(*inStream);
avro::GenericDatum datum(dataSchema);
avro::decode(*decoder, datum);
std::cout << "Type: " << datum.type() << std::endl;
return 0;
}
Everytime I run the code, no matter what file I use, I get this:
$ ./avrotest twitter.avro
AVRO Test
terminate called after throwing an instance of 'avro::Exception'
what(): Cannot have negative length: -40 Aborted
In addition to my own data files, I have tried using the data files located here: https://github.com/miguno/avro-cli-examples, with the same result.
I tried using the avrocat utility on all of the same files and it works fine. What am I doing wrong?
(NOTE: outputting the data schema for each file in JSON works correctly as expected)
After a bunch more fooling around, I figured it out. You're supposed to use DataFileReader templated with GenericDatum. With the end result being something like this:
#include <fstream>
#include <iostream>
#include "Compiler.hh"
#include "DataFile.hh"
#include "Decoder.hh"
#include "Generic.hh"
#include "Stream.hh"
const std::string BOLD("\033[1m");
const std::string ENDC("\033[0m");
const std::string RED("\033[31m");
const std::string YELLOW("\033[33m");
int main(int argc, char**argv)
{
std::cout << "AVRO Test\n" << std::endl;
if (argc < 2)
{
std::cerr << BOLD << RED << "ERROR: " << ENDC << "please provide an "
<< "input file\n" << std::endl;
return -1;
}
avro::DataFileReader<avro::GenericDatum> reader(argv[1]);
auto dataSchema = reader.dataSchema();
// Write out data schema in JSON for grins
std::ofstream output("data_schema.json");
dataSchema.toJson(output);
output.close();
avro::GenericDatum datum(dataSchema);
while (reader.read(datum))
{
std::cout << "Type: " << datum.type() << std::endl;
if (datum.type() == avro::AVRO_RECORD)
{
const avro::GenericRecord& r = datum.value<avro::GenericRecord>();
std::cout << "Field-count: " << r.fieldCount() << std::endl;
// TODO: pull out each field
}
}
return 0;
}
Perhaps an example like this should be included with libavro...

Cannot open text file using ifstream

ifstream fin;
fin.open("‪C:\\Users\\Zach\\Desktop\\input.txt");
if (!fin)
{
cout << "e";
}
e is printing whether I use the full pathway or just input.txt from a resource file
If the file exists, make sure that you have got the path specified correctly. Since you're running on Windows, you can verify the full path to your executable with the following code.
#include <iostream>
#include <fstream>
#include <string>
#include <windows.h>
#define BUFSIZE 4096
std::string getExePath()
{
char result[BUFSIZE];
return std::string(result, GetModuleFileName(NULL, result, BUFSIZE));
}
int main()
{
std::ifstream infile("input.txt");
if (infile.is_open())
{
std::cout << "Success!" << std::endl;
infile.close();
}
else
{
std::cout << "Failed to open input.txt!" << std::endl;
std::cout << "Executable path is ->" << getExePath() << "<-" << std::endl;
}
return 0;
}
This will allow you to verify that your path to the input file is correct, assuming that it's collocated with your executable.
You need to direct output into the ifstream object by using fin << "string"; and not directing to standard out via cout.

How to check if csv file has no data?

I am reading data from a comma delimited csv file. I would like to verify that the file has data before reading and return an error if the file doesn't have any data.
const char* sample_data_file = "sample_data1.csv" ;
std::ifstream file(sample_data_file);
Thanks!
A simple call to stat will tell you if the file is empty. That should be enough to solve your problem.
check the size of the file when you open it?
// reading a text file
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main ()
{
ifstream myfile ("C:/temp/sample1.csv");
// this gives you the number of bytes in the file.
if (myfile.is_open())
{
long begin, end;
begin = myfile.tellg();
myfile.seekg (0, ios::end);
end = myfile.tellg();
if(end-begin == 0)
{
cout << "file is empty \n";
}
else
{
cout << "size: " << (end-begin) << " bytes." << endl;
}
myfile.close();
}
else cout << "Unable to open file \n";
return 0;
}

Trouble when writing to a file in a loop

Here I have two pieces of C++ code that is supposed to write to a file some data. The first one is below and it works:
void ParameterManager::Save()
{
std::ofstream saveFile;
saveFile.open(path, std::ios::trunc | std::ios::out);
if (saveFile.is_open())
{
saveFile << "File opened. Begin saving.\n";
for (int i = 0; i < 4; ++i)
{
saveFile << "Hoppa" << std::endl;
}
}
saveFile.close();
}
The result in output file is:
File opened. Begin saving.
Hoppa
Hoppa
Hoppa
Hoppa
Like expected.
The second one is below and it don't work:
void ParameterManager::Save()
{
std::ofstream saveFile;
saveFile.open(path, std::ios::trunc | std::ios::out);
if (saveFile.is_open())
{
saveFile << "File opened. Begin saving.\n";
for (auto item : map)
{
std::cout << "Hoppa" << std::endl;
saveFile << "Hoppa" << std::endl;
}
}
saveFile.close();
}
Where the map is a hash map containing 4 entries and it is a member of the class where the function Save is implememnted. The result in output file is:
File opened. Begin saving.
The Hoppa lines are printed in terminal but never written to file. I verified in debug mode and the execution write 4 times but the content is not written to file.
I test it on a Virtual machine windows 7 pro. The host is MacBookPro. I use Visual Studio 2013 Pro.
Can you help me please to understand why the second version of code don't work like expected.
A big thanks for all of you.
Welp, I dunno, but it works when I use std::map<int, int> and input 4 std::pair<int, int> and use the same for loop:
http://coliru.stacked-crooked.com/a/e694252e96aebab5
#include <iostream>
#include <fstream>
#include <string>
#include <map>
void save()
{
std::map<int, int> mappa;
for (size_t i = 0; i < 4; ++i) {
mappa.insert(mappa.begin(), std::pair<int, int>(i,i));
}
std::ofstream saveFile;
saveFile.open("test.txt", std::ios::trunc | std::ios::out);
if (saveFile.is_open())
{
saveFile << "File opened. Begin saving.\n";
for (auto it : mappa)
{
saveFile << "Hoppa" << std::endl;
std::cout << "PRINTED LINE" << std::endl;
}
}
saveFile.close();
}
int main() {
save();
}
you can try:
for (auto &item : map)
{
std::cout << item << std::endl;
saveFile << "Hoppa" << std::endl;
}

How to write to and read from a file simultaneously in C++

I need to write two programs write.cpp & read.cpp to run simultaneously. One of them write(overwrite) to a file and the other one reads from it.
Basically, there is always only one line in the file.
write.cpp performs the operation successfully but read.cpp doesn't show anything. Using tail -f also shows incorrect result.
write.cpp:
#include <stdio.h>
#include <ctime>
#include <unistd.h>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main () {
ofstream myfile;
int i = 70;
char c;
while(i <85)
{
myfile.open ("example.txt");
c = i++;
myfile << c << endl;
myfile.close();
sleep(1);
}
return 0;
}
read.cpp:
#include <iostream>
#include <fstream>
#include <string>
#include <unistd.h>
using namespace std;
int main () {
string line;
ifstream myfile ("example.txt");
if (myfile.is_open())
{
while ( myfile.good() )
{
sleep(1);
getline (myfile,line);
cout << line << endl;
}
myfile.close();
}
else cout << "Unable to open file";
return 0;
}
May I know which part of both programs causes the problem and how may I solve it?
You're doing the right thing in the writer, but once you've read to end of file, the input stream becomes unusable until the fail condition is set. The best solution is probably to do exactly what you're doing in the writer: open and close the file each time in the read loop.
Be aware that there will be a moment when the file is empty; when you open the file for writing in the writer, it will be truncated, and if the reader happens to try to read at precisely this moment, it will find an empty file. (It's no big problem; just be aware of it, maybe skipping the sleep if you find an empty line.)
To add some detail to my answer to your previous question, here is how you could use Boost's interprocess communication to achieve this if you insist on using a file for ipc.
A writer may look like this:
#include <boost/interprocess/sync/file_lock.hpp>
#include <boost/interprocess/sync/scoped_lock.hpp>
#include <fstream>
#include <iostream>
int main()
{
using namespace boost::interprocess;
std::string line, shared_filename = "shared";
{
std::ofstream create_shared_file(shared_filename.c_str());
}
for (;;)
{
std::cout << "Enter some text: ";
std::cin >> line;
try
{
file_lock lock(shared_filename.c_str());
scoped_lock<file_lock> lock_the_file(lock);
std::ofstream shared_file(shared_filename.c_str(), std::ofstream::trunc);
shared_file << line << std::endl;
shared_file.flush();
}
catch (interprocess_exception const& e)
{
std::cerr << e.what() << std::endl;
}
}
}
The corresponding reader:
#include <boost/interprocess/sync/file_lock.hpp>
#include <boost/interprocess/sync/sharable_lock.hpp>
#include <fstream>
#include <iostream>
#include <unistd.h>
int main()
{
using namespace boost::interprocess;
std::string line, shared_filename = "shared";
for (;;)
{
try
{
file_lock lock(shared_filename.c_str());
std::cout << "Waiting for file lock..." << std::endl;
sharable_lock<file_lock> lock_the_file(lock);
std::cout << "Acquired file lock..." << std::endl;
std::ifstream shared_file(shared_filename.c_str());
shared_file >> line;
if (line.empty())
{
std::cout << "Empty file" << line << std::endl;
}
else
{
std::cout << "Read: " << line << std::endl;
}
}
catch (interprocess_exception const& e)
{
std::cerr << "Could not lock " << shared_filename << ": " << e.what() << std::endl;
}
std::cout << "Sleeping..." << std::endl;
sleep(2);
}
}