Is there any ways to find a specific primitive within a binary file(such as fread in MATLAB or BinaryReadLists in Mathematica)? Specifically, I want to scan my file until it reaches, say a int8_t precision number, then store it in a variable, then scan for another primitive(unsigned char, double, etc..)?
I am rewriting code from MATLAB that does this, so the format of the file is known.
I want to read n bytes of only the specified type (32-bit int, char, ..) in a file. Ex: Read only the first 12 bytes of my file if they return to be 8-bit integers
Maybe the solution to your problem is in understanding the difference between these two doc pages:
http://www.mathworks.com/help/matlab/ref/fread.html
http://www.cplusplus.com/reference/cstdio/fread/
Both versions of fread let you pull in an array of items from a binary file. I'm assuming from your question that you know the size and shape of the array you need.
#include <stdio.h>
int main() {
const size_t NumElements = 128; // hopefully you know
int8_t myElements[NumElements];
FILE *fp = fopen("mydata.bin", "rb");
assert(fp != NULL);
size_t countRead = fread(myElements, sizeof(int8_t), NumElements, fp);
assert(countRead = NumElements);
// do something with myElements
}
Your question makes no sense to me, but here's a bunch of random information on how to read a binary file:
struct myobject { //so you have your data
char weight;
double value;
};
//for primitives in a binary format you simply read it in
std::istream& operator>>(std::istream& in, myobject& data) {
return in >> data.weight >> data.value;
//we don't really care about failures here
}
//if you don't know the length, that's harder
std::istream& operator>>(std::istream& in, std::vector<myobject>& data) {
int size;
in >> size; //read the length
data.clear();
for(int i=0; i<size; ++i) { //then read that many myobject instances
myobject obj;
if (in >> obj)
data.push_back(obj);
else //if the stream fails, stop
break;
}
return in;
}
int main() {
std::ifstream myfile("input.txt", std::ios_base::binary); //open a file
std::vector<myobject> array;
if (myfile >> array) //read the data!
//well that was easy
else
std::cerr << "error reading from file";
return 0;
};
Also, you can use the .seek(position) member of ifstream to skip directly to a specific point in the file, if you happen to know where to find the data you're looking for.
Oh, you just want to read the first 12 bytes of the file as 8 bit integers, and then the next 12 bytes as int32_t?
int main() {
std::ifstream myfile("input.txt", std::ios_base::binary); //open a file
std::vector<int8_t> data1(12); //array of 12 int8_t
for(int i=0; i<12; ++i) //for each int
myfile >> data1[i]; //read it in
if (!myfile) return 1; //make sure the read succeeded
std::vector<int32_t> data2(3); //array of 3 int32_t
for(int i=0; i<3; ++i) //for each int
myfile >> data2[i]; //read it in
if (!myfile) return 1; //make sure the read succeeded
//processing
}
Related
Given a file with some empty lines, some lines containing only integers, how would I make an array containing all of the integers? I have found methods for strings, but I need a list of integers. I want to do this using getline, but getline gives a string for "line"
A nonfunctioning example which returns the number of integers in the file and modifies a given array:
int getLinesFromFile(string fileName, int arr[], int arrLen) {
ifstream userFile;
userFile.open(fileName);
if (userFile.is_open()) {
int line;
int arrCount = 0;
while (getline(userFile, line)) {
if (tline.length() != 0 && arrCount < arrLen) {
arr[arrCount] = line;
arrCount++;
}
}
return arrCount;
}
else {
return -1;
}
userFile.close();
}
You can just use the >>-operator to read values. It will ignore any whitespace, including empty lines, between values. Here is a modified version of your function that uses it:
int getLinesFromFile(std::string fileName, int arr[], int arrLen) {
std::ifstream userFile(fileName);
int count = 0;
while(count < arrLen) {
int value;
userFile >> value;
if (!userFile.good())
return -1;
arr[count++] = value;
}
return count;
}
Note that you don't need to open and close the file manually, RAII will take care of that for you. Also, if the file could not be opened successfully, or if any other error occured while reading the file, userFile.good() will return false, so you can use that to detect and return an error. It's unclear if your function is supposed to read exactly arrLen values or if less is also valid. But at least you should take care not to write past the end of the provided array.
I'm trying to use ifstream/ofstream to read/write but for some reason, the data gets corrupted along the way. Heres the read/write methods and the test:
void FileWrite(const char* FilePath, std::vector<char> &data) {
std::ofstream os (FilePath);
int len = data.size();
os.write(reinterpret_cast<char*>(&len), 4);
os.write(&(data[0]), len);
os.close();
}
std::vector<char> FileRead(const char* FilePath) {
std::ifstream is(FilePath);
int len;
is.read(reinterpret_cast<char*>(&len), 4);
std::vector<char> ret(len);
is.read(&(ret[0]), len);
is.close();
return ret;
}
void test() {
std::vector<char> sample(1024 * 1024);
for (int i = 0; i < 1024 * 1024; i++) {
sample[i] = rand() % 256;
}
FileWrite("C:\\test\\sample", sample);
auto sample2 = FileRead("C:\\test\\sample");
int err = 0;
for (int i = 0; i < sample.size(); i++) {
if (sample[i] != sample2[i])
err++;
}
std::cout << err << "\n";
int a;
std::cin >> a;
}
It writes the length correctly, reads it correctly and starts reading the data correctly but at some point(depending on input, usually at around the 1000'th byte) it goes wrong and everything to follow is wrong. Why is that?
for starter, you should open the file stream for binary read and write :
std::ofstream os (FilePath,std::ios::binary);
(edit: assuming char really means "signed char")
Do notice that regular char can hold up to CHAR_MAX/2 value, which is 127.
If the random number is bigger - the result will wrap around, resulting negative value. the stream will try to write this character as a text character, which is invalid value to write. binary format should at least fix this problem.
Also, you shouldn't close the stream yourself here, the destructor does it for you.
Two more simple points:
1) &(data[0]) should be just &data[0], the () are redundant
2) try keep the same convention. you write upper-camel-case for FilePath variable, but lower-camel-case for all the other variables.
I have a method which writes a binary file from an int array. (it could be wrong too)
void bcdEncoder::writeBinaryFile(unsigned int packedBcdArray[], int size)
{
fstream binaryIo;
binaryIo.open("PridePrejudice.bin", ios::out| ios::binary | ios::trunc);
binaryIo.seekp(0);
binaryIo.write((char*)packedBcdArray, size * sizeof(packedBcdArray[0]));
binaryIo.seekp(0);
binaryIo.close();
}
I need to now read that binary file back. And preferably have it read it back into another array of unsigned ints without any information loss.
I have something like the following code, but I have no idea on how reading binary files really works, and no idea how to read it into an array of ints.
void bcdEncoder::readBinaryFile(string fileName)
{
// myArray = my dnynamic int array
fstream binaryIo;
binaryIo.open(fileName, ios::in | ios::binary | ios::trunc);
binaryIo.seekp(0);
binaryIo.seekg(0);
binaryIo.read((int*)myArray, size * sizeof(myFile));
binaryIo.close();
}
Question:
How to complete the implementation of the function that reads binary files?
If you're using C++, use the nice std library.
vector<unsigned int> bcdEncoder::readBinaryFile(string fileName)
{
vector<unsigned int> ret; //std::list may be preferable for large files
ifstream in{ fileName };
unsigned int current;
while (in.good()) {
in >> current;
ret.emplace_back(current);
}
return ret;
}
Writing is just as simple (for this we'll accept an int[] but an std library would be preferable):
void bcdEncoder::writeBinaryFile(string fileName, unsigned int arr[], size_t len)
{
ofstream f { fileName };
for (size_t i = 0; i < len; i++)
f << arr[i];
}
Here's the same thing but with an std::vector
void bcdEncoder::writeBinaryFile(string fileName, vector<unsigned int> arr)
{
ofstream f { fileName };
for (auto&& i : arr)
f << i;
}
To simplify read operation consider storing size (i.e the number of elements in the array) before the data:
void bcdEncoder::writeBinaryFile(unsigned int packedBcdArray[], int size)
{
fstream binaryIo;
binaryIo.open("PridePrejudice.bin", ios::out| ios::binary | ios::trunc);
binaryIo.seekp(0);
binaryIo.write(&size, sizeof(size));
binaryIo.write((char*)packedBcdArray, size * sizeof(packedBcdArray[0]));
binaryIo.close();
}
The read would look something like:
void bcdEncoder::readBinaryFile(string fileName)
{
std::vector<unsigned int> myData;
int size;
fstream binaryIo;
binaryIo.open(fileName, ios::in | ios::binary | ios::trunc);
binaryIo.read(&size, sizeof(size)); // read the number of elements
myData.resize(size); // allocate memory for an array
binaryIo.read(myData.data(), size * sizeof(myData.value_type));
binaryIo.close();
// todo: do something with myData
}
Modern alternative using std::array
Here's a code snippet that uses more modern C++ to read a binary file into an std::array.
const int arraySize = 9216; // Hard-coded
std::array<uint8_t, arraySize> fileArray;
std::ifstream binaryFile("<my-binary-file>", std::ios::in | std::ios::binary);
if (binaryFile.is_open()) {
binaryFile.read(reinterpret_cast<char*>(fileArray.data()), arraySize);
}
Because you're using an std::array you'll need to know the exact size of the file during compile-time. If you don't know the size of the file ahead of time (or rather, you'll need to know that the file has at least X bytes available), use a std::vector and look at this example here: https://stackoverflow.com/a/36661779/1576548
Thanks for the tips guys, looks like I worked it out!! A major part of my problem was that half the arguments and syntax I added to the methods were not required, and actually messed things up. Here are my working methods.
void bcdEncoder::writeBinaryFile(unsigned int packedBcdArray[], int size, string fileName)
{
ofstream binaryIo;
binaryIo.open(fileName.substr(0, fileName.length() - 4) + ".bin", ios::binary);
if (binaryIo.is_open()) {
binaryIo.write((char*)packedBcdArray, size * sizeof(packedBcdArray[0]));
binaryIo.close();
// Send binary file to reader
readBinaryFile(fileName.substr(0, fileName.length() - 4) + ".bin", size);
}
else
cout << "Error writing bin file..." << endl;
}
And the read:
void bcdEncoder::readBinaryFile(string fileName, int size)
{
AllocateArray packedData(size);
unsigned int *packedArray = packedData.createIntArray();
ifstream binaryIo;
binaryIo.open(fileName, ios::binary);
if (binaryIo.is_open()) {
binaryIo.read((char*)packedArray, size * sizeof(packedArray[0]));
binaryIo.close();
decodeBCD(packedArray, size * 5, fileName);
}
else
cout << "Error reading bin file..." << endl;
}
With the AllocateArray being my class that creates dynamic arrays without vectors somewhat safely with destructors included.
I'm trying to read an array object (Array is a class I've made using read and write functions to read and write from binary files. So far the write functions works but it won't read from the file properly for some reason. This is the write function :
void writeToBinFile(const char* path) const
{
ofstream ofs(path, ios_base::out | ios_base::app | ios_base::binary);
if (ofs.is_open())
{
ostringstream oss;
for (unsigned int i = 0; i < m_size; i++)
{
oss << ' ';
oss << m_data[i];
}
ofs.write(oss.str().c_str(), oss.str().size());
}
}
This is the read function :
void readFromBinFile(const char* path)
{
ifstream ifs(path, ios_base::in | ios_base::binary || ios_base::ate);
if (ifs.is_open())
{
stringstream ss;
int charCount = 0, spaceCount = 0;
ifs.unget();
while (spaceCount != m_size)
{
charCount++;
if (ifs.peek() == ' ')
{
spaceCount++;
}
ifs.unget();
}
ifs.get();
char* ch = new char[sizeof(char) * charCount];
ifs.read(ch, sizeof(char) * charCount);
ss << ch;
delete[] ch;
for (unsigned int i = 0; i < m_size; i++)
{
ss >> m_data[i];
m_elementCount++;
}
}
}
those are the class fields :
T* m_data;
unsigned int m_size;
unsigned int m_elementCount;
I'm using the following code to write and then read (1 execution for reading another for writing):
Array<int> arr3(5);
//arr3[0] = 38;
//arr3[1] = 22;
//arr3[2] = 55;
//arr3[3] = 7;
//arr3[4] = 94;
//arr3.writeToBinFile("binfile.bin");
arr3.readFromBinFile("binfile.bin");
for (unsigned int i = 0; i < arr3.elementCount(); i++)
{
cout << "arr3[" << i << "] = " << arr3[i] << endl;
}
The problem is now at the readFromBinFile function, it get stuck in an infinite loop and peek() returns -1 for some reason and I can't figure why.
Also note I'm writing to the binary file using spaces to make a barrier between each element so I would know to differentiate between objects in the array and also a space at the start of the writing to make a barrier between previous stored binary data in the file to the array binary data.
The major problem, in my mind, is that you write fixed-size binary data in variable-size textual form. It could be so much simpler if you just stick to pure binary form.
Instead of writing to a string stream and then writing that output to the actual file, just write the binary data directly to the file:
ofs.write(reinterpret_cast<char*>(m_data), sizeof(m_data[0]) * m_size);
Then do something similar when reading the data.
For this to work, you of course need to save the number of entries in the array/vector first before writing the actual data.
So the actual write function could be as simple as
void writeToBinFile(const char* path) const
{
ofstream ofs(path, ios_base::out | ios_base::binary);
if (ofs)
{
ofs.write(reinterpret_cast<const char*>(&m_size), sizeof(m_size));
ofs.write(reinterpret_cast<const char*>(&m_data[0]), sizeof(m_data[0]) * m_size);
}
}
And the read function
void readFromBinFile(const char* path)
{
ifstream ifs(path, ios_base::in | ios_base::binary);
if (ifs)
{
// Read the size
ifs.read(reinterpret_cast<char*>(&m_size), sizeof(m_size));
// Read all the data
ifs.read(reinterpret_cast<char*>(&m_data[0]), sizeof(m_data[0]) * m_size);
}
}
Depending on how you define m_data you might need to allocate memory for it before reading the actual data.
Oh, and if you want to append data at the end of the array (but why would you, in the current code you show, you rewrite the whole array anyway) you write the size at the beginning, seek to the end, and then write the new data.
When i read TestData.txt file it gives me wrong output. What am i doing wrong. I am using int array so i can do MergeSort after saving data into array.
TestData.txt
-------------------
31791 564974 477059 269094 972335
739154 206345 634644 227684 398536
910177 507975 589785 67117 395140
598829 372499 364165 450187 996527
700285 263407 918021 661467 457544
656297 846316 221731 240676 68287
913 141702 845802 477617 109824
{
int myArray[1000];
int i;
//reading givin data
const char* filename= "TestData.txt";
ifstream file(filename);
if(file.is_open())
{
for(i = 0; i <=999; ++i)
{
file >> myArray[i];//storing data to array
}
}
Need to check if you ifstream is end of file, in that case you get garbage value from out of the file bound.
With One modification, the code would be OK.
Change:
for(i = 0; i <=999; ++i)
to:
for(i = 0; i <=999 && !file.eof(); ++i)
You are reading 1000 enties from your file which contains clearly less than 1000 integers.
The first values of your array must be correct, but after you reach the end of your file the operator>> will not ready anything.
For example here is one way to write it:
const char* filename= "TestData.txt";
std::vector<int> myArray;
std::ifstream file(filename);
if(file.is_open())
{
int v;
while(file >> v) {
myArray.push_back(v);
}
}
int if I'm not wrong can keep data from -32768 to 32767.
So if u have bigger values than that (which you have, from your source file), you won't have the results you are expecting.
btw, it would be nice to know also what output you are getting.