How to convert binary files context into int/long value in C++ - c++

I am storing a value in a binary file with fstream. The value is unsigned short type.
unsigned shord value=1750; //2 byte variable
file.write((char*)&value,sizeof(value));
My problem is that I want to read this binary file in another function, but it gives me some weird symbols (Obviously because it is binary).
Is there any way to get those two bytes and convert them to my old value (1750) ?
Here's what I have tried:
cout <<(unsigned short)(unsigned char)(s2[8]);//s2 variable where the whole body is stored
cout <<(unsigned short)(char*)(s2[8]);
I have tried other things, too, but they were just chickenscratch and aren't worth including here.

Here's how you might do it (note uses C++11 features):
#include <iostream>
#include <fstream>
#include <vector>
using namespace std;
void write_shorts(string filename, vector<unsigned short> shorts)
{
ofstream f;
f.open(filename, ofstream::trunc|ofstream::binary);
for(auto s: shorts)
f.write(reinterpret_cast<char*>(&s), sizeof(s));
}
auto read_shorts(string filename)
{
ifstream f(filename);
vector<unsigned short> res;
short x;
while(f.read(reinterpret_cast<char*>(&x), sizeof(x)))
res.push_back(x);
return res;
}
int main()
{
// Write 3 shorts to a file
write_shorts("myshorts", {4711, 1, 0xffff});
// Read them back into a vector
auto v = read_shorts("myshorts");
cout << "Read " << v.size() << "shorts: " << endl;
for(auto x: v)
cout << x << endl;
}

Related

How to cram map array in buffer to send?

I am trying to move the map array but for this I need to put everything in buffer, here is my code.
#include <iostream>
#include <map>
#include <string>
std::map<std::int, std::string> maphack;
maphack.emplace(1, "CS");
maphack.emplace(2, "PBG");
maphack.emplace(3, "Maicraft");
maphack.insert_or_assign(3, "GTA5");
for (auto &a : maphack) {
std::cout << a.first << " : " << a.second << '\n';
}
How to put everything above in the buffer?
char buffer[64];
send(sock, buffer, AmountToSend, 0);
The simple answer is to use std::ostringstream and std::istringstream which allows to use std streams machinery with a string storage. Which can be converted to char* with c_str() method if needed later, full example is below.
The better answer can be to use more heavy lifting library for serialization, the choice of the format will depend on your program. Do you need binary or text format? Do you want other programs (in other programming languages) be able to read it? Do you need to support different endian system? Do you need to handle error when deserializing malformed input? Boost, Qt, google protobuf, and other ones are out there.
#include <iostream>
#include <map>
#include <sstream>
#include <string>
void write_map(const std::map<int, std::string> &m, std::ostream &out) {
for (auto &a : m) {
out << a.first << " " << a.second << '\n';
}
}
std::map<int, std::string> read_map(std::istream &in) {
std::map<int, std::string> m;
int i;
std::string s;
while (in >> i >> s) {
m.emplace(i, s);
}
return m;
}
int main() {
std::map<int, std::string> m;
m.emplace(1, "CS");
m.emplace(2, "");
m.emplace(3, "Maicraft");
m.insert_or_assign(3, "GTA5");
std::ostringstream out;
write_map(m, out);
std::string data = out.str();
std::cout << "Data:\n" << data << std::endl;
// send data over socket
// ...
std::istringstream in(data);
auto m1 = read_map(in);
std::cout << "Read:\n";
write_map(m1, std::cout);
std::cout << std::endl;
}

How to read hex values into integer with fstream (C++)

I am trying to read a little-endian hex string from a binary file, and put that value into an integer to work with it. When I try to read, instead of getting a number I get ascii symbols. I've tried casts and atoi and nothing seems to work. What is the best way to use fstream to read a hex string into an integer from a file?
This is essentially my program:
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main(int argc, char* argv[]) {
fstream input;
fstream output;
char cbuffer[4];
char revbuffer[8];
input.open(argv[1], fstream::binary | fstream::in);
output.open("output.txt", ios::out | ios::app);
input.seekg(16, input.beg);
input.read(cbuffer, 4);
cout << sizeof(revbuffer) << endl;
cout << cbuffer[0] << cbuffer[1] << cbuffer[2] << cbuffer[3] << endl;
}
If it's an integer value stored in binary format, I guess it's either a int32_t or a uint32_t. Since you mention that the value is stored in little-endian byte order, I guess you want to make sure that the host running your program converts it (if it needs to). C++20 has std::endian. If that's not available to you, there are usually macros for detecting endianness at compiletime that you can use instead of the std::endian tests I've used. I've assumed that the value is a uint32_t below.
#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include <type_traits> // std::endian
// little endian unsigned 32 bit integer to host byte order
inline uint32_t Le32toh(uint32_t le) {
#if __cplusplus <= 201703L
// run-time check
static constexpr uint16_t endian = 1;
if(*reinterpret_cast<const uint8_t*>(&endian)==1) return le;
#else
// compile-time check
static_assert(std::endian::native == std::endian::little || std::endian::native == std::endian::big);
if constexpr (std::endian::native == std::endian::little) return le;
#endif
const uint8_t* c=reinterpret_cast<const uint8_t*>(&le);
return // little-to-big endian conversion
(static_cast<uint32_t>(c[0])<<24) |
(static_cast<uint32_t>(c[1])<<16) |
(static_cast<uint32_t>(c[2])<<8) |
(static_cast<uint32_t>(c[3]));
return le;
}
int main(int argc, char* argv[]) {
std::vector<std::string> args(argv+1, argv+argc);
std::fstream output("output.txt", std::ios::out | std::ios::app);
uint32_t cbuffer;
for(const auto& file : args) {
std::fstream input(file, std::fstream::binary | std::fstream::in);
input.seekg(16, input.beg);
// read directly into the varibles memory
input.read(reinterpret_cast<char*>(&cbuffer), 4);
// output the value unconverted
std::cout << std::hex << cbuffer << "\n";
// convert if needed
cbuffer = Le32toh(cbuffer);
// output the value converted
std::cout << std::hex << cbuffer << "\n";
}
}

Reading or writing binary file incorrectly

The output of the code show gibberish values for all the variables of the Student struct. When the display function is ran.
I've include the relevant code in each of the add and display function for the binary file.
For the second function, does the seekg pointer automatically move to read the the next record each time the for loop runs?
//Student struct
struct Student
{
char name [30];
float labTest;
float assignments;
float exam;
};
//Writing function
afile.open(fileName,ios::out|ios::binary);
Student S;
strcpy(S.name,"test");
S.labTest = rand()%100+1;
S.assignments = rand()%100+1;
S.exam = rand()%100+1;
afile.write(reinterpret_cast<char*>(&S),sizeof(S));
afile.close();
//Reading function
afile.open(fileName,ios::in|ios::binary);
afile.seekg(0,ios::end);
int nobyte = afile.tellg();
int recno = nobyte / sizeof(Student);
Student S;
//Loop and read every record
for(int i = 0;i<recno;i++)
{
afile.read(reinterpret_cast<char*>(&S),sizeof(S));
cout << "Name of Student: " << S.name << endl
<< "Lab mark: " << S.labTest << endl
<< "Assignment mark: " << S.assignments << endl
<< "Exam mark: " << S.exam << endl << endl;
}
afile.close();
There are a lot of problems with your code:
Calling your write function will permanently overwrite the last written data set. You have to add: ios::append, so that new data will be written behind the last data you wrote before.
After you move with afile.seekg(0,ios::end); to get with tellg the file size, you have to go back to the start of the file before reading with afile.seekg(0,ios::beg)
It looks that you use a char array to store a string. This is not c++ style! And it is dangerous how you use it. If you use strcpy, you can copy a string which is longer than the space you reserved for it. So you should prefer std::string for that. But you can't simply write a struct which constains std::string as binary! To get checked copy you can use strncpy, but that is still not c++ ;)
For the second function, does the seekg pointer automatically move to read the the next record each time the for loop runs?
Yes, the file position moves which each successful read and write.
A general remark writing binary data by simply dumping memory content:
That is not a good idea, because you can only read that data back, if you use the same machine type and the same compiler options. That means: A machine with different endianness will read data totally corrupted. Also a different integer type ( 32 bit vs 64 bit ) will break that code!
So you should invest some time how to serialize data in a portable way. There are a lot of libraries around which can be used to read/write also complex data types like std::string or container types.
A hint using SO:
Please provide code which everybody can simply cut and paste and compiled. I did not know what your Student struct is. So I take a lot of assumptions! Is your struct really using char[]? We don't know!
#include <iostream>
#include <fstream>
#include <cstring>
const char* fileName="x.bin";
struct Student
{
char name[100]; // not c++ style!
int labTest;
int assignments;
int exam;
};
// Writing function
void Write()
{
std::ofstream afile;
afile.open(fileName,std::ios::out|std::ios::binary|std::ios::app);
Student S;
strcpy(S.name,"test"); // should not be done this way!
S.labTest = rand()%100+1;
S.assignments = rand()%100+1;
S.exam = rand()%100+1;
afile.write(reinterpret_cast<char*>(&S),sizeof(S));
afile.close();
}
void Read()
{
//Reading function
std::ifstream afile;
afile.open(fileName,std::ios::in|std::ios::binary);
afile.seekg(0,std::ios::end);
int nobyte = afile.tellg();
int recno = nobyte / sizeof(Student);
afile.seekg(0, std::ios::beg);
Student S;
//Loop and read every record
for(int i = 0;i<recno;i++)
{
afile.read(reinterpret_cast<char*>(&S),sizeof(S));
std::cout << "Name of Student: " << S.name << std::endl
<< "Lab mark: " << S.labTest << std::endl
<< "Assignment mark: " << S.assignments << std::endl
<< "Exam mark: " << S.exam << std::endl << std::endl;
}
afile.close();
}
int main()
{
for ( int ii= 0; ii<10; ii++) Write();
Read();
}
EDIT. Apparently, I was a bit too late in responding. Klaus has compiled a better, more comprehensive response dwelling into other problems regarding C-style char [], std::string and the endianness of the platform.
You should append to the file opened for every record. In your code you don't have this, at all. Please write the code in a way we can copy and paste, and test. As a working example, you should write some code that can be compiled and run as below:
#include <algorithm>
#include <cstring>
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
// Student struct
struct Student {
char name[30];
float labTest;
float assignments;
float exam;
};
// Serializer
void serialize_student(const Student &s, const std::string &filename) {
// Append to the file, do not overwrite it
std::ofstream outfile(filename, std::ios::binary | std::ios::app);
if (outfile)
outfile.write(reinterpret_cast<const char *>(&s), sizeof(Student));
}
// Deserializer
std::vector<Student> deserialize_students(const std::string &filename) {
std::ifstream infile(filename, std::ios::binary);
std::vector<Student> students;
Student s;
while (infile.read(reinterpret_cast<char *>(&s), sizeof(Student)))
students.push_back(std::move(s));
return std::move(students);
}
int main(int argc, char *argv[]) {
// Generate records
std::vector<Student> mystudents;
std::generate_n(std::back_inserter(mystudents), 10, []() {
Student s;
std::strcpy(s.name, "test");
s.labTest = rand() % 100 + 1;
s.assignments = rand() % 100 + 1;
s.exam = rand() % 100 + 1;
return s;
});
// Print and write the records
for (const auto &student : mystudents) {
std::cout << student.name << ": [" << student.labTest << ','
<< student.assignments << ',' << student.exam << "].\n";
serialize_student(student, "students.bin");
}
// Read and print the records
auto records = deserialize_students("students.bin");
std::cout << "===\n";
for (const auto &student : records)
std::cout << student.name << ": [" << student.labTest << ','
<< student.assignments << ',' << student.exam << "].\n";
return 0;
}

C++ cereal de-serialization trouble with large size vector

I hope to serialize large size vector with cereal, C++ serialization library.
But, if trying to do that, the exception "Failed to read " + std::to_string(size) + " bytes from input stream! Read " + std::to_string(readSize)" is thrown.
Does anyone know a good solution for this?
I'm using VisualStudio 2017.
The source code is shown below.
#include <iostream>
#include <fstream>
#include "include\cereal\cereal.hpp"
#include "include\cereal\archives\binary.hpp"
#include "include\cereal\types\vector.hpp"
#include "include\cereal\types\string.hpp"
void show(std::vector<int> v) {
for (auto i : v)std::cout << i << ",";
std::cout << std::endl;
}
int main(void) {
const std::string file_name = "out.cereal";
{
std::vector<int> src;
// const int STOP = 10; //OK
const int STOP = 1000; // NG
for (int i = 0; i < STOP; i++)src.push_back(i);
std::cout << "src:" << std::endl;
show(src);
std::ofstream ofs(file_name, std::ios::binary);
cereal::BinaryOutputArchive archive(ofs);
archive(src);
}
{
std::vector<int> dst;
std::fstream fs(file_name);
cereal::BinaryInputArchive iarchive(fs);
iarchive(dst);
std::cout << "dst:" << std::endl;
show(dst);
}
#ifdef _MSC_VER
system("pause");
#endif
return 0;
}
You code works fine for me in Linux, so I think it is to do with the difference between text and binary handling on Windows. Check that you pass std::ios::binary when you are constructing the input stream. Also construct it as std::ifstream rather than just std::fstream.
I think this might have to do with Windows expecting (or adding) a Unicode byte-order mark, which is confusing the serializer.

c++ string to boost::multiprecision::cpp_int

How would I convert a string to "boost::multiprecision::cpp_int"?
Additionally, I have a .txt file with 100 numbers each of 50 digits and I use ifstream to read them line by line into a string array. How can I convert each string from the array into a cpp_int, then add all the 100 numbers and get the sum?
To convert a single string, use the cpp_intconstructor: cpp_int tmp("123");.
For the text file case, read each number in a loop as a std::string via std::getline, then emplace back in a std::vector<cpp_int>. Then use the latter to compute your sum. Example:
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <boost/multiprecision/cpp_int.hpp>
using namespace boost::multiprecision;
int main()
{
std::vector<cpp_int> v;
std::fstream fin("in.txt");
std::string num;
while(std::getline(fin, num))
{
v.emplace_back(num);
}
cpp_int sum = 0;
for(auto&& elem: v)
{
std::cout << elem << std::endl; // just to make sure we read correctly
sum += elem;
}
std::cout << "Sum: " << sum << std::endl;
}
PS: you may do it without a std::vector, via a temporary cpp_int that you construct inside the loop and assign it to sum:
std::string num;
cpp_int sum = 0;
while(std::getline(fin, num))
{
cpp_int tmp(num);
sum += tmp;
}
std::cout << "Sum: " << sum << std::endl;