Let's just say I have an .ini file that looks like:
[material#3]
Diffuse=sometexture.jpg
Normal=someNormalMap.jpg
Specular=NULL
And I want to add a specular map value to the 'Specular' label, if I had this string buffer in a byte array I would do something like:
void assignSpecularMap(const char* name)
{
// Seek to position after 'Specular='
int oldValueStrLen = strlen("NULL");
int newValueStrLen = strlen(name);
int shiftAmount = newValueStrLen - oldValueStrLen;
// Shift the entire buffer from after 'NULL' by the shiftAmount
// Write the new value after 'Specular='
}
The shifting of the buffer would be done with a memcpy. This is my naive understanding of how to write new values into a text file. If each entry or section isn't padded to a uniform byte size, which it never is for human readable and editable files, then any change in the entries for these files had to be accompanied by a shifting of the entire buffer, correct?
I'm wondering how to do this with a file stream in C++. It's not the case that the entire stream has to be loaded into a char buffer and do it the way I showed?
Related
I have the following function (so far):
void read_binary_file(std::istream is,
ByteArray arr)
{
int length = is.tellg();
char *buffer = new char[length];
is.read(buffer, length);
// What to do next?
// The goal is to place istream buffer in my `ByteArray` class `values`class,
// ByteArray - an array of `float`, each item should be 4 bytes from the buffer
}
My goal is to place each 4 bytes from the buffer inside my ByteArray->values class. Each item should contain 4 bytes from the buffer.
ByteArray definition:
class ByteArray
{
....
float *values;
}
Limitations: I don't want to use stl/ vector classes.
I couldn't find an example with my current limitations.
Any idea how I can do that?
If I understand correctly, you want to create a ByteArray object and copy bytes from buffer to ByteArray::values[] as floats. Assuming that the file is opened in binary mode & contain floats dumped in correct format+endianness, and total data in file is multiple of sizeof(float):
class ByteArray
{
private:
float* values;
public:
void set(char* buffer, int len)
{
values = new float[len/4];
for(int itr =0; itr < len/4; itr++)
{
values[itr] = *(float*)(buffer+itr*4);
}
}
};
...
arr.set(buffer, length);
Note that i) smarter codes are possible but I kept it as simple as possible for your understanding. ii) Ulrich is right, you should pass istream by reference (as well as ByteArray for most practical purposes):
void read_binary_file(std::istream& is,
ByteArray& arr)
...
If you want to use istream to send bytes byte by byte you can say
arr.values=(float*)buffer;
or
arr.values=new float[length/4];
memcpy(arr.values,buffer,length);
delete[] buffer;
It works until you want to send a float which contains a eof byte by accident. 2 is a float like that, so it isn't uncommon. Then you can't do anything as istream stops at that byte. So I recommend not to send floats byte by byte in stringteams. Send them an other way eg in hexa. (hat way you don't loose precision).
What generated the file you want to read?
I have 640*480 numbers. I need to write them into a file. I will need to read them later. What is the best solution? Numbers are between 0 - 255.
For me the best solution is to write them binary(8 bits). I wrote the numbers into txt file and now it looks like 1011111010111110 ..... So there are no questions where the number starts and ends.
How am I supposed to read them from the file?
Using c++
It's not good idea to write bit values like 1 and 0 to text file. The file size will bigger in 8 times. 1 byte = 8 bits. You have to store bytes, 0-255 - is byte. So your file will have size 640*480 bytes instead of 640*480*8. Every symbol in text file has size of 1 byte minimum. If you want to get bits, use binary operators of programming language that you use. To read bytes much easier. Use binary file for saving your data.
Presumably you have some sort of data structure representing your image, which somewhere inside holds the actual data:
class pixmap
{
public:
// stuff...
private:
std::unique_ptr<std::uint8_t[]> data;
};
So you can add a new constructor which takes a filename and reads bytes from that file:
pixmap(const std::string& filename)
{
constexpr int SIZE = 640 * 480;
// Open an input file stream and set it to throw exceptions:
std::ifstream file;
file.exceptions(std::ios_base::badbit | std::ios_base::failbit);
file.open(filename.c_str());
// Create a unique ptr to hold the data: this will be cleaned up
// automatically if file reading throws
std::unique_ptr<std::uint8_t[]> temp(new std::uint8_t[SIZE]);
// Read SIZE bytes from the file
file.read(reinterpret_cast<char*>(temp.get()), SIZE);
// If we get to here, the read worked, so we move the temp data we've just read
// into where we'd like it
data = std::move(temp); // or std::swap(data, temp) if you prefer
}
I realise I've assumed some implementation details here (you might not be using a std::unique_ptr to store the underlying image data, though you probably should be) but hopefully this is enough to get you started.
You can print the number between 0-255 as the char value in the file.
See the below code. in this example I am printing integer 70 as char.
So this result in print as 'F' on the console.
Similarly you can read it as char and then convert this char to integer.
#include <stdio.h>
int main()
{
int i = 70;
char dig = (char)i;
printf("%c", dig);
return 0;
}
This way you can restrict the file size.
How can I write a series of numbers (in binary) preceded by its length, without knowing the length beforehand?
Can I write the number series to a temporary buffer (e.g. buf.write()), determine the buffer length, and copy the buffer to an ofstream afterwards (e.g. ofs << buf)?
Since you are writing in binary mode to the file, reserve a location for the quantity.
For example, let's have an 32-bit value for the quantity. We could write that at the beginning of the file:
uint32_t number_size = 0U;
// Open output file for read & write.
// Reserve space for the quantity by writing dummy value.
output_file.write((char *)&number_size, sizeof(number_size));
// Perform calculations & writing to file, incrementing "number_size"
// Write the number size to the top of the file:
output_file.seekp(0, SEEK_BEG);
output_file.write((char *)&number_size, sizeof(number_size));
You can place quantity variable anywhere, just remember its file position. When finished, seek to that position and write it in.
I am taking input from a file in binary mode using C++; I read the data into unsigned ints, process them, and write them to another file. The problem is that sometimes, at the end of the file, there might be a little bit of data left that isn't large enough to fit into an int; in this case, I want to pad the end of the file with 0s and record how much padding was needed, until the data is large enough to fill an unsigned int.
Here is how I am reading from the file:
std::ifstream fin;
fin.open('filename.whatever', std::ios::in | std::ios::binary);
if(fin) {
unsigned int m;
while(fin >> m) {
//processing the data and writing to another file here
}
//TODO: read the remaining data and pad it here prior to processing
} else {
//output to error stream and exit with failure condition
}
The TODO in the code is where I'm having trouble. After the file input finishes and the loop exits, I need to read in the remaining data at the end of the file that was too small to fill an unsigned int. I need to then pad the end of that data with 0's in binary, recording enough about how much padding was done to be able to un-pad the data in the future.
How is this done, and is this already done automatically by C++?
NOTE: I cannot read the data into anything but an unsigned int, as I am processing the data as if it were an unsigned integer for encryption purposes.
EDIT: It was suggested that I simply read what remains into an array of chars. Am I correct in assuming that this will read in ALL remaining data from the file? It is important to note that I want this to work on any file that C++ can open for input and/or output in binary mode. Thanks for pointing out that I failed to include the detail of opening the file in binary mode.
EDIT: The files my code operates on are not created by anything I have written; they could be audio, video, or text. My goal is to make my code format-agnostic, so I can make no assumptions about the amount of data within a file.
EDIT: ok, so based on constructive comments, this is something of the approach I am seeing, documented in comments where the operations would take place:
std::ifstream fin;
fin.open('filename.whatever', std::ios::in | std::ios::binary);
if(fin) {
unsigned int m;
while(fin >> m) {
//processing the data and writing to another file here
}
//1: declare Char array
//2: fill it with what remains in the file
//3: fill the rest of it until it's the same size as an unsigned int
} else {
//output to error stream and exit with failure condition
}
The question, at this point, is this: is this truly format-agnostic? In other words, are bytes used to measure file size as discrete units, or can a file be, say, 11.25 bytes in size? I should know this, I know, but I've got to ask it anyway.
Are bytes used to measure file size as discrete units, or can a file be, say, 11.25 bytes in size?
No data type can be less than a byte, and your file is represented as an array of char meaning each character is one byte. Thus it is impossible to not get a whole number measure in bytes.
Here is step one, two, and three as per your post:
while (fin >> m)
{
// ...
}
std::ostringstream buffer;
buffer << fin.rdbuf();
std::string contents = buffer.str();
// fill with 0s
std::fill(contents.begin(), contents.end(), '0');
I'm writing a resource file which I want to insert a bunch of data from various common files such as .JPG, .BMP (for example) and I want it to be in binary.
I'm going to code something to retrieve these data later on organized by index, and this is what I got so far:
float randomValue = 23.14f;
ofstream fileWriter;
fileWriter.open("myFile.dat", ios::binary);
fileWriter.write((char*)&randomValue, sizeof(randomValue));
fileWriter.close();
//With this my .dat file, when opened in notepad has "B!¹A" in it
float retrieveValue = 0.0f;
ifstream fileReader;
fileReader.open("myFile.dat", ios::binary);
fileReader.read((char*)&retrieveValue, sizeof(retrieveValue));
fileReader.close();
cout << retrieveValue << endl; //This gives me exactly the 23.14 I wanted, perfect!
While this works nicely, I'd like to understand what exactly is happening there.
I'm converting the address of randomValue to char*, and writing the values in this address to the file?
I'm curious also because I need to do this for an array, and I can't do this:
int* myArray = new int[10];
//fill myArray values with random stuff
fileWriter.open("myFile.dat", ios::binary);
fileWriter.write((char*)&myArray, sizeof(myArray));
fileWriter.close();
From what I understand, this would just write the first address' value in the file, not all the array. So, for testing, I'm trying to simply convert a variable to a char* which I would write to a file, and convert back to the variable to see if I'm retrieving the values correctly, so I'm with this:
int* intArray = new int[10];
for(int i = 0; i < 10; i++)
{
cout << &intArray[i]; //the address of each number in my array
cout << intArray[i]; //it's value
cout << reinterpret_cast<char*>(&intArray[i]); //the char* value of each one
}
But for some reason I don't know, my computer "beeps" when I run this code. During the array, I'm also saving these to a char* and trying to convert back to int, but I'm not getting the results expected, I'm getting some really long values.
Something like:
float randomValue = 23.14f;
char* charValue = reinterpret_cast<char*>(&randomValue);
//charValue contains "B!¹A" plus a bunch of other (un-initiallized values?) characters, so I'm guessing the value is correct
//Now I'm here
I want to convert charValue back to randomValue, how can I do it?
edit: There's valuable information in the answers below, but they don't solve my (original) problem. I was testing these type of conversions because I'm doing a code that I will pick a bunch of resource files such as BMP, JPG, MP3, and save them in a single .DAT file organized by some criteria I still haven't fully figured out.
Later, I am going to use this resource file to read from and load these contents into a program (game) I'm coding.
The criteria I am still thinking but I was wondering if it's possible to do something like this:
//In my ResourceFile.DAT
[4 bytes = objectID][3 bytes = objectType (WAV, MP3, JPG, BMP, etc)][4 bytes = objectLength][objectLength bytes = actual objectData]
//repeating this until end of file
And then in the code that reads the resource file, I want to do something like this (untested):
ifstream fileReader;
fileReader.open("myFile.DAT", ios::binary);
//file check stuff
while(!fileReader.eof())
{
//Here I'll load
int objectID = 0;
fileReader((char*)&objectID, 4); //read 4 bytes to fill objectID
char objectType[3];
fileReader(&objectType, 3); //read the type so I know which parser use
int objectLength = 0;
fileReader((char*)&objectLength, 4); //get the length of the object data
char* objectData = new char[objectLength];
fileReader(objectData, objectLength); //fill objectData with the data
//Here I'll use a parser to fill classes depending on the type etc, and move on to the next obj
}
Currently my code is working with the original files (BMP, WAV, etc) and filling them into classes, and I want to know how I can save the data from these files into a binary data file.
For example, my class that manages BMP data has this:
class FileBMP
{
public:
int imageWidth;
int imageHeight;
int* imageData;
}
When I load it, I call:
void FileBMP::Load(int iwidth, int iheight)
{
int imageTotalSize = iwidth * iheight * 4;
imageData = new int[imageTotalSize]; //This will give me 4 times the amount of pixels in the image
int cPixel = 0;
while(cPixel < imageTotalSize)
{
imageData[cPixel] = 0; //R value
imageData[cPixel + 1] = 0; //G value
imageData[cPixel + 2] = 0; //B value
imageData[cPixel + 3] = 0; //A value
cPixel += 4;
}
}
So I have this single dimension array containing values in the format of [RGBA] per pixel, which I am using later on for drawing on screen.
I want to be able to save just this array in the binary data format that I am planning that I stated above, and then read it and fill this array.
I think it's asking too much for a code like this, so I'd like to understand what I need to know to save these values into a binary file and then read back to fill it.
Sorry for the long post!
edit2: I solved my problem by making the first edit... thanks for the valuable info, I also got to know what I wanted to!
By using the & operator, you're getting a pointer to the contents of the variable (think of it as just a memory address).
float a = 123.45f;
float* p = &a; // now p points to a, i.e. has the memory address to a's contents.
char* c = (char*)&a; // c points to the same memory location, but the code says to treat the contents as char instead of float.
When you gave the (char*)&randomValue for write(), you simply told "take this memory address having char data and write sizeof(randomValue) chars from there". You're not writing the address value itself, but the contents from that location of memory ("raw binary data").
cout << reinterpret_cast<char*>(&intArray[i]); //the char* value of each one
Here you're expected to give char* type data, terminated with a null char (zero). However, you're providing the raw bytes of the float value instead. Your program might crash here, as cout will input chars until it finds the terminator char -- which it might not find anytime soon.
float randomValue = 23.14f;
char* charValue = reinterpret_cast<char*>(&randomValue);
float back = *(float*)charValue;
Edit: to save binary data, you simply need to provide the data and write() it. Do not use << operator overloads with ofstream/cout. For example:
int values[3] = { 5, 6, 7 };
struct AnyData
{
float a;
int b;
} data;
cout.write((char*)&values, sizeof(int) * 3); // the other two values follow the first one, you can write them all at once.
cout.write((char*)&data, sizeof(data)); // you can also save structs that do not have pointers.
In case you're going to write structs, have a look at #pragma pack compiler directive. Compilers will align (use padding) variable to certain size (int), which means that the following struct actually might require 8 bytes:
#pragma pack (push, 1)
struct CouldBeLongerThanYouThink
{
char a;
char b;
};
#pragma pack (pop)
Also, do not write pointer values itself (if there are pointer members in a struct), because the memory addresses will not point to any meaningful data once read back from a file. Always write the data itself, not pointer values.
What's happening is that you're copying the internal
representation of your data to a file, and then copying it back
into memory, This works as long as the program doing the
writing was compiled with the same version of the compiler,
using the same options. Otherwise, it might or it might not
work, depending on any number of things beyond your control.
It's not clear to me what you're trying to do, but formats like
.jpg and .bmp normally specify the format they want the
different types to have, and you have to respect that format.
It is unclear what you really want to do, so I cannot recommend a way of solving your real problem. But I would not be surprised if running the program actually caused beeps or any other strange behavior in your program.
int* intArray = new int[10];
for(int i = 0; i < 10; i++)
{
cout << reinterpret_cast<char*>(&intArray[i]);
}
The memory returned by new above is uninitialized, but you are trying to print it as if it was a null terminated string. That uninitialized memory could have the bell character (that causes beeps when printed to the terminal) or any other values, including that it might potentially not have a null termination and the insertion operator into the stream will overrun the buffer until it either finds a null or your program crashes accessing invalid memory.
There are other incorrect assumptions in your code, like for example given int *p = new int[10]; the expression sizeof(p) will be the size of a pointer in your architecture, not 10 times the size of an integer.