I am writing a program that will read and write a file format that dictates the content of the file, byte by byte. The nature of this program is that the first two bytes details how many bytes are left in that part of the file, followed by another two bytes that indicates what the part of the file actually represents. This pattern is repeated for the length of the file. This means I have to write the exact numbers buffered by preceding zeros such that each component is the exact size it needs to be. I have written up a dummy file that illustrates my points:
#include <fstream>
#include <stdint.h>
int main() {
std::ofstream outputFile;
outputFile.open("test.txt",
std::ios::out | std::ios::ate | std::ios::binary);
const int16_t HEADER = 0x0002;
int16_t recordSize = 2*sizeof(int16_t);
int16_t version = 0x0258;
outputFile << recordSize << HEADER << version;
outputFile.close();
}
which writes a file named "test.txt" who's hex contents are:
34 32 36 30 30
and for those of us that can read straight hex this translates to:
42600
As you can see the preceding zeros are removed and my record is not what I was hoping it to be. Is there a way to use ofstream to buffer my numbers with zeros as I naively tried to do by using int16_t for all of the writes that I wanted to be exactly two bytes long? Is there another, possibly more stylistically correct way of doing this?
operator<< is for text formatting. You probably want to use .write() instead.
e.g.
outputFile.write(reinterpret_cast<char*>(&recordSize), sizeof(int16_t));
outputFile.write(reinterpret_cast<char*>(&HEADER), sizeof(int16_t));
// ...
Related
I have 640*480 numbers. I need to write them into a file. I will need to read them later. What is the best solution? Numbers are between 0 - 255.
For me the best solution is to write them binary(8 bits). I wrote the numbers into txt file and now it looks like 1011111010111110 ..... So there are no questions where the number starts and ends.
How am I supposed to read them from the file?
Using c++
It's not good idea to write bit values like 1 and 0 to text file. The file size will bigger in 8 times. 1 byte = 8 bits. You have to store bytes, 0-255 - is byte. So your file will have size 640*480 bytes instead of 640*480*8. Every symbol in text file has size of 1 byte minimum. If you want to get bits, use binary operators of programming language that you use. To read bytes much easier. Use binary file for saving your data.
Presumably you have some sort of data structure representing your image, which somewhere inside holds the actual data:
class pixmap
{
public:
// stuff...
private:
std::unique_ptr<std::uint8_t[]> data;
};
So you can add a new constructor which takes a filename and reads bytes from that file:
pixmap(const std::string& filename)
{
constexpr int SIZE = 640 * 480;
// Open an input file stream and set it to throw exceptions:
std::ifstream file;
file.exceptions(std::ios_base::badbit | std::ios_base::failbit);
file.open(filename.c_str());
// Create a unique ptr to hold the data: this will be cleaned up
// automatically if file reading throws
std::unique_ptr<std::uint8_t[]> temp(new std::uint8_t[SIZE]);
// Read SIZE bytes from the file
file.read(reinterpret_cast<char*>(temp.get()), SIZE);
// If we get to here, the read worked, so we move the temp data we've just read
// into where we'd like it
data = std::move(temp); // or std::swap(data, temp) if you prefer
}
I realise I've assumed some implementation details here (you might not be using a std::unique_ptr to store the underlying image data, though you probably should be) but hopefully this is enough to get you started.
You can print the number between 0-255 as the char value in the file.
See the below code. in this example I am printing integer 70 as char.
So this result in print as 'F' on the console.
Similarly you can read it as char and then convert this char to integer.
#include <stdio.h>
int main()
{
int i = 70;
char dig = (char)i;
printf("%c", dig);
return 0;
}
This way you can restrict the file size.
I am taking input from a file in binary mode using C++; I read the data into unsigned ints, process them, and write them to another file. The problem is that sometimes, at the end of the file, there might be a little bit of data left that isn't large enough to fit into an int; in this case, I want to pad the end of the file with 0s and record how much padding was needed, until the data is large enough to fill an unsigned int.
Here is how I am reading from the file:
std::ifstream fin;
fin.open('filename.whatever', std::ios::in | std::ios::binary);
if(fin) {
unsigned int m;
while(fin >> m) {
//processing the data and writing to another file here
}
//TODO: read the remaining data and pad it here prior to processing
} else {
//output to error stream and exit with failure condition
}
The TODO in the code is where I'm having trouble. After the file input finishes and the loop exits, I need to read in the remaining data at the end of the file that was too small to fill an unsigned int. I need to then pad the end of that data with 0's in binary, recording enough about how much padding was done to be able to un-pad the data in the future.
How is this done, and is this already done automatically by C++?
NOTE: I cannot read the data into anything but an unsigned int, as I am processing the data as if it were an unsigned integer for encryption purposes.
EDIT: It was suggested that I simply read what remains into an array of chars. Am I correct in assuming that this will read in ALL remaining data from the file? It is important to note that I want this to work on any file that C++ can open for input and/or output in binary mode. Thanks for pointing out that I failed to include the detail of opening the file in binary mode.
EDIT: The files my code operates on are not created by anything I have written; they could be audio, video, or text. My goal is to make my code format-agnostic, so I can make no assumptions about the amount of data within a file.
EDIT: ok, so based on constructive comments, this is something of the approach I am seeing, documented in comments where the operations would take place:
std::ifstream fin;
fin.open('filename.whatever', std::ios::in | std::ios::binary);
if(fin) {
unsigned int m;
while(fin >> m) {
//processing the data and writing to another file here
}
//1: declare Char array
//2: fill it with what remains in the file
//3: fill the rest of it until it's the same size as an unsigned int
} else {
//output to error stream and exit with failure condition
}
The question, at this point, is this: is this truly format-agnostic? In other words, are bytes used to measure file size as discrete units, or can a file be, say, 11.25 bytes in size? I should know this, I know, but I've got to ask it anyway.
Are bytes used to measure file size as discrete units, or can a file be, say, 11.25 bytes in size?
No data type can be less than a byte, and your file is represented as an array of char meaning each character is one byte. Thus it is impossible to not get a whole number measure in bytes.
Here is step one, two, and three as per your post:
while (fin >> m)
{
// ...
}
std::ostringstream buffer;
buffer << fin.rdbuf();
std::string contents = buffer.str();
// fill with 0s
std::fill(contents.begin(), contents.end(), '0');
I am currently attempting to store a string containing binary code.
When I attempt to write this string to a text file it simply stores each 0 and 1 character in a string format, rather than storing it into 8 bit chunks as I require. This causes the file to be larger than intended, considering it uses 8 bits to store each 0 and 1.
Should I write the string to a .bin file instead of a .txt file?
If so how would I go about doing this, and if possible an example with some working code.
My thanks for any advice in advance.
string encoded = "01010101";
ofstream myfile;
myfile.open ("encoded");
myfile << encoded;
myfile.close();
Clarification:
I have a string made up of 1's and 0's(resulting from a Huffman Tree), I wish to break this string up into 8 bit chunks, I wish to write each character represented by said chink to a compressed file.
std::bitset can convert a string to an integer.
std::bitset<8> bits("01010101");
cout << bits.to_ullong();
I'm only guessing since you don't show any code, but it seems you have a string containing the characters '1' and '0'. If you write that to a file of course it will be as a text. You need to convert it to integers first.
See e.g. std::stoi or std::strtol for functions to convert strings of arbitrary base to integers.
Should I write the string to a .bin file instead of a .txt file?
If you wish so... But that wouldn't make a difference either. ofstream doesn't care about filenames. Just convert the string to a byte (uint8_t) and write that byte to the file:
string s = "10101010";
uint8_t byte = strtoul(s.c_str(), NULL, 2);
myfile << byte;
I am unclear about how reading long integers work. If I say
long int a[1]={666666}
ofstream o("ex",ios::binary);
o.write((char*)a,sizeof(a));
to store values to a file and want to read them back as it is
long int stor[1];
ifstream i("ex",ios::binary);
i.read((char*)stor,sizeof(stor));
how will I be able to display the same number as stored using the information stored in multiple bytes of character array?
o.write does not write character, it writes bytes (if flagged with ios::binary). The char-pointer is used because a char has length 1 Byte.
o.write((char*)a,sizeof(a));
(char*) a is the adress of what o.write should write. Then it writes sizeof(a) bytes to a file. There are no characters stored, just bytes.
If you open the file in a Hex-Editor you would see something like this if a is int i = 10:
0A 00 00 00 (4 Byte, on x64).
Reading is analogue.
Here is a working example:
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main (int argc, char* argv[]){
const char* FILENAM = "a.txt";
int toStore = 10;
ofstream o(FILENAM,ios::binary);
o.write((char*)&toStore,sizeof(toStore));
o.close();
int toRestore=0;
ifstream i(FILENAM,ios::binary);
i.read((char*)&toRestore,sizeof(toRestore));
cout << toRestore << endl;
return 0;
}
Sorry I took so long to see your question.
I think the difference between binary is the binary will read and write the file as is. But the non-binary (i.e. text) mode will fix up the end-of-line '\n' with carriage-return '\r'. The fix-up will change back and forth between '\n' and '\r', or "\n\r" or "\r\n" or leave it as '\n'. What it does depends on whether the target operating system is Mac, Windows, Unix, etc.
I think if you are reading and writing an integer, it will read and write your integer fine and it will look correct. But if some byte(s) of the integer look like '\r' and '\n', then the integer will not read back correctly from the file.
Binary assures that reading back an int will always be correct. But you want text mode to format a file to be read in a text editor such as Windows's Notepad.
I have a dat(binary) file but i wish to convert this file into Ascii (txt) file using c++ but i am very new in c++ programming.so I juct opend my 2 files:myBinaryfile and myTxtFile but I don't know how to read data from that dat file and then how to write those data into new txt file.so i want to write a c+ codes that takes in an input containing binary dat file, and converts it to Ascii txt in an output file. if this possible please help to write this codes. thanks
Sorry for asking same question again but still I didn’t solve my problem, I will explain it more clearly as follows: I have a txt file called “A.txt”, so I want to convert this into binary file (B.dat) and vice verse process. Two questions:
1. how to convert “A.txt” into “B.dat” in c++
2. how to convert “B.dat” into “C.txt” in c++ (need convert result of the 1st output again into new ascii file)
my text file is like (no header):
1st line: 1234.123 543.213 67543.210 1234.67 12.000
2nd line: 4234.423 843.200 60543.232 5634.60 72.012
it have more than 1000 lines in similar style (5 columns per one line).
Since I don’t have experiences in c++, I am struggle here, so need your helps. Many Thanks
All files are just a stream of bytes. You can open files in binary mode, or text mode. The later simply means that it may have extra newline handling.
If you want your text file to contain only safe human readable characters you could do something like base64 encode your binary data before saving it in the text file.
Very easy:
Create target or destination file
(a.k.a. open).
Open source file in binary mode,
which prevents OS from translating
the content.
Read an octet (byte) from source
file; unsigned char is a good
variable type for this.
Write the octet to the destination
using your favorite conversion, hex,
decimal, etc.
Repeat at 3 until the read fails.
Close all files.
Research these keywords: ifstream, ofstream, hex modifier, dec modifier, istream::read, ostream::write.
There are utilities and applications that already perform this operation. On the *nix and Cygwin side try od, *octal dump` and pipe the contents to a file.
There is the debug utility on MS-DOS system.
A popular format is:
AAAAAA bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb cccccccccccccccc
where:
AAAAAA -- Offset from beginning of file in hexadecimal or decimal.
bb -- Hex value of byte using ASCII text.
c -- Character representation of byte, '.' if the value is not printable.
Please edit your post to provide more details, including an example layout for the target file.
Edit:
A complex example (not tested):
#include <iostream>
#include <fstream>
#include <cstdio>
#include <cstdlib>
using namespace std;
const unsigned int READ_BUFFER_SIZE = 1024 * 1024;
const unsigned int WRITE_BUFFER_SIZE = 2 * READ_BUFFER_SIZE;
unsigned char read_buffer[READ_BUFFER_SIZE];
unsigned char write_buffer[WRITE_BUFFER_SIZE];
int main(void)
{
int program_status = EXIT_FAILURE;
static const char hex_chars[] = "0123456789ABCDEF";
do
{
ifstream srce_file("binary.dat", ios::binary);
if (!srce_file)
{
cerr << "Error opening input file." << endl;
break;
}
ofstream dest_file("binary.txt");
if (!dest_file)
{
cerr << "Error creating output file." << endl;
}
// While no read errors from reading a block of source data:
while (srce_file.read(&read_buffer[0], READ_BUFFER_SIZE))
{
// Get the number of bytes actually read.
const unsigned int bytes_read = srce_file.gcount();
// Define the index and byte variables outside
// of the loop to maybe save some execution time.
unsigned int i = 0;
unsigned char byte = 0;
// For each byte that was read:
for (i = 0; i < bytes_read; ++i)
{
// Get source, binary value.
byte = read_buffer[i];
// Convert the Most Significant nibble to an
// ASCII character using a lookup table.
// Write the character into the output buffer.
write_buffer[i * 2 + 0] = hex_chars[(byte >> 8)];
// Convert the Least Significant nibble to an
// ASCII character and put into output buffer.
write_buffer[i * 2 + 1] = hex_chars[byte & 0x0f];
}
// Write the output buffer to the output, text, file.
dest_file.write(&write_buffer[0], 2 * bytes_read);
// Flush the contents of the stream buffer as a precaution.
dest_file.flush();
}
dest_file.flush();
dest_file.close();
srce_file.close();
program_status = EXIT_SUCCESS;
} while (false);
return program_status;
}
The above program reads 1MB chunks from the binary file, converts to ASCII hex into an output buffer, then writes the chunk to the text file.
I think you are misunderstanding that the difference between a binary file and a test file is in the interpretation of the contents.