I'm trying to convert a matrix of (doubles) into an unsigned char so I can then write to a .pmg file... But it isn't working.
void writePNG(vector<double>& matrix)
{
vector<unsigned char> image;
ofstream myfile;
myfile.open("newFile.txt", ios::out); // writing to .txt file for now for testing.
if(!myfile.is_open())
{
cout << "Cannot open file";
}
for(int i=0; (i < 512*512); i++)
{
image[i] = (unsigned char) matrix[i];
}
myfile.close();
}
It won't convert the data. Any ideas?? Thanks :)
bug: You are creating a vector of size 0, and then writing to its non-existent elements.
bug: You never write the data to a file
style: You close the file needlessly. It will be closed when the fstream object goes out of scope
style: You copy the data in a loop. Using vector::vector displays your intent more clearly.
potential bug: You create an output vector of 512x512, regardless of the size of the input vector.
SSCCE Your testcase is incomplete.
Try this:
#include <vector>
#include <iostream>
#include <fstream>
void writePNG(const std::vector<double>& matrix)
{
std::ofstream myfile("newFile.txt", std::ios::out);
if(!myfile.is_open())
{
std::cout << "Cannot open file";
}
std::vector<unsigned char> image (matrix.begin(), matrix.end());
myfile.write(reinterpret_cast<const char*>(&image[0]), image.size());
}
int main () {
writePNG({
72, 101, 108.1, 108.2,
111, 0x2c, 0x20, 0x77,
0x6f, 0x72, 0x6c,
100.3, 10.4});
}
You are creating image using the default constructor of vector, which initializes the vector as empty (containing no elements). The subscript notation (image[i]) does not create an element, only assigns to an already exising one.
You have (at least) two ways to fix it:
declare image using the ctor of vector that allocates the necessary size: vector<unsigned char> image(512*512) -- this will populate the vector with 512*512 elements of default value (0 for unsigned char)
add the elements one-by-one using the push_back method: image.push_back((unsigned char) matrix[i]);
You also will have to write the contents of image to myfile eventually.
Note: it is a good habit to use static_cast<unsigned char>(...) instead of the C-style (unsigned char) ... as the former can find errors that the latter will not flag; this is not an issue in this particular case, though
You image vector has zero size - you would have to at least do a push_back to add an element. Also, the size of a double is not the same size of a char so you are going to lose information.
Related
I want to store multiple arrays which all entries consist of either 0 or 1.
This file would be quite large if i do it the way i do it.
I made a minimalist version of what i currently do.
#include <iostream>
#include <fstream>
using namespace std;
int main(){
ofstream File;
File.open("test.csv");
int array[4]={1,0,0,1};
for(int i = 0; i < 4; ++i){
File << array[i] << endl;
}
File.close();
return 0;
}
So basically is there a way of storing this in a binary file or something, since my data is 0 or 1 in the first place anyways?
If yes, how to do this? Can i also still have line-breaks and maybe even commas in that file? If either of the latter does not work, that's also fine. Just more importantly, how to store this as a binary file which has only 0 and 1 so my file is smaller.
Thank you very much!
So basically is there a way of storing this in a binary file or something, since my data is 0 or 1 in the first place anyways? If yes, how to do this? Can i also still have line-breaks and maybe even commas in that file? If either of the latter does not work, that's also fine. Just more importantly, how to store this as a binary file which has only 0 and 1 so my file is smaller.
The obvious solution is to take 64 characters, say A-Z, a-z, 0-9, and + and /, and have each character code for six entries in your table. There is, in fact, a standard for this called Base64. In Base64, A encodes 0,0,0,0,0,0 while / encodes 1,1,1,1,1,1. Each combination of six zeroes or ones has a corresponding character.
This still leaves commas, spaces, and newlines free for your use as separators.
If you want to store the data as compactly as possible, I'd recommend storing it as binary data, where each bit in the binary file represents one boolean value. This will allow you to store 8 boolean values for each byte of disk space you use up.
If you want to store arrays whose lengths are not multiples of 8, it gets a little bit more complicated since you can't store a partial byte, but you can solve that problem by storing an extra byte of meta-data at the end of the file that specifies how many bits of the final data-byte are valid and how many are just padding.
Something like this:
#include <iostream>
#include <fstream>
#include <cstdint>
#include <vector>
using namespace std;
// Given an array of ints that are either 1 or 0, returns a packed-array
// of uint8_t's containing those bits as compactly as possible.
vector<uint8_t> packBits(const int * array, size_t arraySize)
{
const size_t vectorSize = ((arraySize+7)/8)+1; // round up, then +1 for the metadata byte
vector<uint8_t> packedBits;
packedBits.resize(vectorSize, 0);
// Store 8 boolean-bits into each byte of (packedBits)
for (size_t i=0; i<arraySize; i++)
{
if (array[i] != 0) packedBits[i/8] |= (1<<(i%8));
}
// The last byte in the array is special; it holds the number of
// valid bits that we stored to the byte just before it.
// That way if the number of bits we saved isn't an even multiple of 8,
// we can use this value later on to calculate exactly how many bits we should restore
packedBits[vectorSize-1] = arraySize%8;
return packedBits;
}
// Given a packed-bits vector (i.e. as previously returned by packBits()),
// returns the vector-of-integers that was passed to the packBits() call.
vector<int> unpackBits(const vector<uint8_t> & packedBits)
{
vector<int> ret;
if (packedBits.size() < 2) return ret;
const size_t validBitsInLastByte = packedBits[packedBits.size()-1]%8;
const size_t numValidBits = 8*(packedBits.size()-((validBitsInLastByte>0)?2:1)) + validBitsInLastByte;
ret.resize(numValidBits);
for (size_t i=0; i<numValidBits; i++)
{
ret[i] = (packedBits[i/8] & (1<<(i%8))) ? 1 : 0;
}
return ret;
}
// Returns the size of the specified file in bytes, or -1 on failure
static ssize_t getFileSize(ifstream & inFile)
{
if (inFile.is_open() == false) return -1;
const streampos origPos = inFile.tellg(); // record current seek-position
inFile.seekg(0, ios::end); // seek to the end of the file
const ssize_t fileSize = inFile.tellg(); // record current seek-position
inFile.seekg(origPos); // so we won't change the file's read-position as a side effect
return fileSize;
}
int main(){
// Example of packing an array-of-ints into packed-bits form and saving it
// to a binary file
{
const int array[]={0,0,1,1,1,1,1,0,1,0};
// Pack the int-array into packed-bits format
const vector<uint8_t> packedBits = packBits(array, sizeof(array)/sizeof(array[0]));
// Write the packed-bits to a binary file
ofstream outFile;
outFile.open("test.bin", ios::binary);
outFile.write(reinterpret_cast<const char *>(&packedBits[0]), packedBits.size());
outFile.close();
}
// Now we'll read the binary file back in, unpack the bits to a vector<int>,
// and print out the contents of the vector.
{
// open the file for reading
ifstream inFile;
inFile.open("test.bin", ios::binary);
const ssize_t fileSizeBytes = getFileSize(inFile);
if (fileSizeBytes < 0)
{
cerr << "Couldn't read test.bin, aborting" << endl;
return 10;
}
// Read in the packed-binary data
vector<uint8_t> packedBits;
packedBits.resize(fileSizeBytes);
inFile.read(reinterpret_cast<char *>(&packedBits[0]), fileSizeBytes);
// Expand the packed-binary data back out to one-int-per-boolean
vector<int> unpackedInts = unpackBits(packedBits);
// Print out the int-array's contents
cout << "Loaded-from-disk unpackedInts vector is " << unpackedInts.size() << " items long:" << endl;
for (size_t i=0; i<unpackedInts.size(); i++) cout << unpackedInts[i] << " ";
cout << endl;
}
return 0;
}
(You could probably make the file even more compact than that by running zip or gzip on the file after you write it out :) )
You can indeed write and read binary data. However having line breaks and commas would be difficult. Imagine you save your data as boolean data, so only ones and zeros. Then having a comma would mean you need an special character, but you have only ones and zeros!. The next best thing would be to make an object of two booleans, one meaning the usual data you need (c++ would then read the data in pairs of bits), and the other meaning whether you have a comma or not, but I doubt this is what you need. If you want to do something like a csv, then it would be easy to just fix the size of each column (int would be 4 bytes, a string of no more than 32 char for example), and then just read and write accordingly. Suppose you have your binary
To initially save your array of the an object say pets, then you would use
FILE *apFile;
apFile = fopen(FILENAME,"w+");
fwrite(ARRAY_OF_PETS, sizeof(Pet),SIZE_OF_ARRAY, apFile);
fclose(apFile);
To access your idx pet, you would use
Pet m;
ifstream input_file (FILENAME, ios::in|ios::binary|ios::ate);
input_file.seekg (sizeof(Pet) * idx, ios::beg);
input_file.read((char*) &m,sizeof(Pet));
input_file.close();
You can also add data add the end, change data in the middle and so on.
i want to store an JPEG image into a normal unsigned char array, i'd used ifstream to store it; however, when i checked if the array i'd stored is correct or not ( by rewrite it again to an JPEG image), the image that i rewrote by using the stored array couldn't show correctly, so i think the problem must come from the technique that i use to store the image into an array is not correct. I want an array which can be stored perfectly so that i can use it to rewrite back into a JPEG image again.I'd really appreciate if anyone can help me solve this problem!
int size = 921600;
unsigned char output[size];
int i = 0;
ifstream DataFile;
DataFile.open("abc.jpeg");
while(!DataFile.eof()){
DataFile >> output[i];
i++;
}
/* i try to rewrite the above array into a new image here */
FILE * image2;
image2 = fopen("def.jpeg", "w");
fwrite(output,1,921600, image2);
fclose(image2);
There are multiple problems in the shown code.
while(!DataFile.eof()){
This is always a bug. See the linked question for a detailed explanation.
DataFile >> output[i];
The formatted extraction operator, >>, by definition, skips over all white space characters and ignores them. Your jpg file surely has bytes 0x09, 0x20, and a few others, somewhere in it, and this automatically skips over and does not read them.
In order to do this correctly, you need to use read() and gcount() to read your binary file. Using gcount() correctly should also result in your code detecting the end-of-file condition properly.
Make sure to add error check when opening files. Find the file size and read in to the buffer according to the filesize.
You might also look in to using std::vector<unsigned char> for character storage.
int main()
{
std::ifstream DataFile("abc.jpeg", std::ios::binary);
if(!DataFile.good())
return 0;
DataFile.seekg(0, std::ios::end);
size_t filesize = (int)DataFile.tellg();
DataFile.seekg(0);
unsigned char output[filesize];
//or std::vector
//or unsigned char *output = new unsigned char[filesize];
if(DataFile.read((char*)output, filesize))
{
std::ofstream fout("def.jpeg", std::ios::binary);
if(!fout.good())
return 0;
fout.write((char*)output, filesize);
}
return 0;
}
The pointer to the audio buffer of the XAUDIO_BUFFER structure in XAudio2 is defined as BYTE *pAudioData. When I was using 16-bit Integer PCM, this is what my program looked like:
void buildWaveBuffer(std::vector<unsigned char> &vec)
{
std::string lineString;
int lineInt;
unsigned char lowByte, highByte;
std::ifstream myfile("sineInt16");
if (myfile.is_open())
{
while(myfile.good())
{
std::getline(myfile,lineString,',');
lineInt = atoi(lineString.c_str());
highByte = (lineInt >> 8) & 0x00FF;
lowByte = lineInt & 0x00FF;
vec.push_back(lowByte);
vec.push_back(highByte);
}
myfile.close();
}
}
"sineInt16" being a .csv file. Since the vector is organized sequentially in the memory, I would simply do pAudioData = &vec[0] and it would work. What if I want to change the format of my .csv to float? How do I give a pointer to the first byte in the vector? Should I use another container like a simple array of chars?
How do I give a pointer to the first byte in the vector?
The exact same way, but I'm not sure it will do what you expect. Read the comments to your question.
I am trying to read data from binary file, and having issues. I have reduced it down to the most simple case here, and it still won't work. I am new to c++ so I may be doing something silly but, if anyone could advise I would be very grateful.
Code:
int main(int argc,char *argv[]) {
ifstream myfile;
vector<bool> encoded2;
cout << encoded2 << "\n"<< "\n" ;
myfile.open(argv[2], ios::in | ios::binary |ios::ate );
myfile.seekg(0,ios::beg);
myfile.read((char*)&encoded2, 1 );
myfile.close();
cout << encoded2 << "\n"<< "\n" ;
}
Output
00000000
000000000000000000000000000011110000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Compression_Program(58221) malloc: * error for object 0x10012d: Non-aligned pointer being freed
* set a breakpoint in malloc_error_break to debug
Thanks in advance.
Do not cast a vector<bool>* to a char*. It is does not do anything predictable.
You are reading on encoded2: myfile.read((char*)&encoded2, 1 );. this is wrong. you can to read a bool and then put it in encoded2
bool x;
myfile.read( &x, 1 );
encoded2[0] = x;
Two mistakes here:
you assume the address of a vector is the address of the first element
you rely on vector<bool>
Casting a vector into a char * is not really a good thing, because a vector is an object and stores some state along with its elements.
Here you are probably overwriting the state of the vector, thus the destructor of fails.
Maybe you would like to cast the elements of the vector (which are guaranteed to be stored contiguously in memory). But another trap is that vector<bool> may be implementation-optimized.
Therefore you should do a encoded2.reserve(8) and use myfile.read(reinterpret_cast<char *>(&encoded2[0])).
But probably you want to do something else and we need to know what the purpose is here.
You're overwriting a std::vector, which you shouldn't do. A std::vector is actually a pointer to a data array and an integer (probably a size_t) holding its size; if you overwrite these with practically random bits, data corruption will occur.
Since you're only reading a single byte, this will suffice:
char c;
myfile.read(&c, 1);
The C++ language does not provide an efficient I/O method for reading bits as bits. You have to read bits in groups. Also, you have to worry about Endianess when reading int the bits.
I suggest the old fashioned method of allocating a buffer, reading into the buffer then operating on the buffer.
Allocating a buffer
const unsigned int BUFFER_SIZE = 1024 * 1024; // Let the compiler calculate it.
//...
unsigned char * const buffer = new unsigned char [BUFFER_SIZE]; // The pointer is constant.
Reading in the data
unsigned int bytes_read = 0;
ifstream data_file("myfile.bin", ios::binary); // Open file for input without translations.
data_file.read(buffer, BUFFER_SIZE); // Read data into the buffer.
bytes_read = data_file.gcount(); // Get actual count of bytes read.
Reminders:
delete the buffer when you are
finished with it.
Close the file when you are finished
with it.
myfile.read((char*) &encoded2[0], sizeof(int)* COUNT);
or you can use push_back();
int tmp;
for(int i = 0; i < COUNT; i++) {
myfile.read((char*) &tmp, 4);
encoded2.push_back(tmp);
}
I'm trying to write a small program which will search a binary file for a few bytes and replace these with another bunch of bytes. But everytime I try running this small app I got message about istream_iterator is not dereferenceable.
Maybe someone have a suggestion how to do this in another way (iterators are a little bit a new subject for me).
#include <fstream>
#include <iterator>
#include <algorithm>
using namespace std;
int main() {
typedef istream_iterator<char> input_iter_t;
const off_t SIZE = 4;
char before[SIZE] = { 0x12, 0x34, 0x56, 0x78 };
char after[SIZE] = { 0x78, 0x12, 0x34, 0x65 };
fstream filestream("numbers.exe", ios::binary | ios::in | ios::out);
if (search(input_iter_t(filestream), input_iter_t(), before, before + SIZE) != input_iter_t()) {
filestream.seekp(-SIZE, ios::cur);
filestream.write(after, SIZE);
}
return 0;
}
This is my second attempt to do this but also something is wrong. With small files looks like works OK but with bigger (around 2MB) it works very slowly and never find pattern what I'm looking for.
#include <iostream>
#include <cstdlib>
#include <string>
#include <fstream>
#include <iterator>
#include <vector>
#include <algorithm>
#include <windows.h>
using namespace std;
int main() {
const off_t Size = 4;
unsigned char before[Size] = { 0x12, 0x34, 0x56, 0x78 };
unsigned char after[Size] = { 0x90, 0xAB, 0xCD, 0xEF };
vector<char> bytes;
{
ifstream iFilestream( "numbers.exe", ios::in|ios::binary );
istream_iterator<char> begin(iFilestream), end;
bytes.assign( begin, end ) ;
}
vector<char>::iterator found = search( bytes.begin(), bytes.end(), before, before + Size );
if( found != bytes.end() )
{
copy( after, after + Size, found );
{
ofstream oFilestream( "number-modified.exe" );
copy( bytes.begin(), bytes.end(), ostream_iterator<unsigned char>(oFilestream) );
}
}
return 0;
}
Cheers,
Thomas
Read a larger part of the file to memory, replace it in memory and then dump the bunch to the disk. Reading one byte at a time is very slow.
I also suggest you read about mmap (or MapViewOfFile in win32).
search won't work on an istream_iterator because of the nature of the iterator. It's an input iterator, which means it simply moves forward - this is because it reads from the stream, and once it's read from the stream, it can't go back. search requires a forward iterator, which is an input iterator where you can stop, make a copy, and move one forward while keeping the old one. An example of a forward iterator is a singly-linked list. You can't go backwards, but you can remember where you are and restart from there.
The speed issue is because vector is truly terrible at handling unknown data. Every time it runs out of room, it copies the whole buffer over to new memory. Replace it with a deque, which can handle data arriving one by one. You will also likely get improved performance trying to read from the stream in blocks at a time, as character-by-character access is a pretty bad way to load an entire file into memory.
Assuming the file isn't too large, just read the file into memory, then modify the memory buffer as you see fit, then write it back out to a file.
E.g. (untested):
FILE *f_in = fopen("inputfile","rb");
fseek(f_in,0,SEEK_END);
long size = ftell(f_in);
rewind(f_in);
char* p_buffer = (char*) malloc (size);
fread (p_buffer,size,1,f_in);
fclose(f_in);
unsigned char *p= (unsigned char*)p_buffer;
// process.
FILE *f_out = fopen("outoutfile","wb");
fwrite(p_buffer,size,1,f_out);
fclose(f_out);
free(p_buffer);