For some reason (that I've yet to figure out), the following code doesn't quite work as expected. I'm trying to write an array of shorts to a file and while the file is generated successfully, the values are just not right. The example code is
short* sarray = new short[2000];
for(int i=0;i<2000;i++)
sarray[i]=i*2;
FILE* oFile;
oFile = fopen("E:\\audio_rec\\test_1.raw", "w");
int shortsRead =fwrite(sarray, sizeof(short), 2000, oFile);
fclose(oFile);
The file content (as displayed in a hex editor). It's clear that only the first 5 values (up to 8) are as expected.
Any ideas?
Thanks
In Windows you need to tell fopen that the data is binary or it messes up "\n" to "\r\n".
fopen("E:\\audio_rec\\test_1.raw", "wb");
Related
I'm reading a binary uint32_t data from a file that indicates the size of the next binary block, after that I read that block but the reading pointer is "moving" wrong.
FILE* file = fopen("file.zip", "r");
long pointerA = ftell(file);
uint32_t streamSize = 0;
fread(reinterpret_cast<char*>(&streamSize), sizeof streamSize,1,file);
long pointerB = ftell(file);
char* zipData = new char[streamSize];
fread(zipData, sizeof(char),streamSize,file);
long pointerC = ftell(file);
fseek( file, pointerA + 4 + streamSize, SEEK_SET );
long pointerD = ftell(file);
qDebug()<<"streamSize"<<streamSize<<"Positions"<<pointerA<<pointerB<<pointerC<<pointerD;
PointerA is the original position, PointerB the position after reading that uint32_t, PointerC is the pointer after reading all that binary data and
PointerD is just a check about what I suppose that should be the right behaviour.
Now let's see the debug:
streamSize 2653 Positions 151 156 4627 2808
Why the stream read position has moved too 4627 instead 2808?
Thank you in advance for any tip!
Both users #alan-birtles and #remy-lebeau were right, I opened it as text instead binary that was the issue.
Unfortunatly I cannot mark this as solved.
PS. For begginers this means open file with "rb" instead "r".
You need to open your file in binary mode. When a file is opened in text mode some characters are changed as you read them. For example on Windows when reading '\n' "\r\n" is returned. To open in binary mode add 'b' to your open mode, e.g:
FILE * file = fopen("file.txt", "rb");
Note you need to do the same when writing binary files otherwise the same transformations take place.
std::fstream also needs std::ios_base::binary to be passed to the constructor/open to avoid the same issue.
I am actually writing a c++ program that reads any kind of file and saves it as a bmp file, but first I need to read the file, and thats were the issue is
char fileName[] = "test.jpg";
FILE * inFileForGettingSize;//This is for getting the file size
fopen_s(&inFileForGettingSize, fileName, "r");
fseek(inFileForGettingSize, 0L, SEEK_END);
int fileSize = ftell(inFileForGettingSize);
fclose(inFileForGettingSize);
ifstream inFile;//This is for reading the file
inFile.open(fileName);
if (inFile.fail()) {
cerr << "Error Opening File" << endl;
}
char * data = new char[fileSize];
inFile.read(data, fileSize);
ofstream outFile;//Writing the file back again
outFile.open("out.jpg");
outFile.write(data, fileSize);
outFile.close();
cin.get();
But when I read the file, lets say its a plainttext file it allways outputs some wierd charactes at the end, for example:
assdassaasd
sdaasddsa
sdadsa
passes to:
assdassaasd
sdaasddsa
sdadsaÍÍÍ
So when I do this with a jpg, exe, etc. It corrupts it.
I am not trying to COPY a file, I know there are other ways for that, Im just trying to read a complete file byte per byte. Thanks.
EDIT:
I found out that those 'Í' are equal to the number of end lines the file has, but this doesn't help me much
This is caused by newline handling.
You open the files in text mode (because you use "r" instead of "rb" for fopen and because you don't pass ios::binary to your fstream open calls), and on Windows, text mode translates "\r\n" pairs to "\n" on reading and back to "\r\n" when writing. The result is that the in-memory size is going to be shorter than the on-disk size, so when you try to write using the on-disk size, you go past the end of your array and write whatever random stuff happens to reside in memory.
You need to open files in binary mode when working with binary data:
fopen_s(&inFileForGettingSize, fileName, "rb");
inFile.open(fileName, ios::binary);
outFile.open("out.jpg", ios::binary);
For future reference, your copy routine could be improved. Mixing FILE* I/O with iostream I/O feels awkward, and opening and closing the file twice is extra work, and (most importantly), if your routine is ever run on a large enough file, it will exhaust memory trying to load the entire file into RAM. Copying a block at a time would be better:
const int BUFFER_SIZE = 65536;
char buffer[BUFFER_SIZE];
while (source.good()) {
source.read(buffer, BUFFER_SIZE);
dest.write(buffer, source.gcount());
}
It's a binary file, so you need to read and write the file as binary; otherwise it's treated as text, and assumed to have newlines that need translation.
In your call to fopen(), you need add the "b" designator:
fopen_s(&inFileForGettingSize, fileName, "rb");
And in your fstream::open calls, you need to add std::fstream::binary:
inFile.open(fileName, std::fstream::binary);
// ...
outFile.open("out.jpg", std::fstream::binary);
Does anyone know how to read in a file with raw encoding? So stumped.... I am trying to read in floats or doubles (I think). I have been stuck on this for a few weeks. Thank you!
File that I am trying to read from:
http://www.sci.utah.edu/~gk/DTI-data/gk2/gk2-rcc-mask.raw
Description of raw encoding:
hello://teem.sourceforge.net/nrrd/format.html#encoding (change hello to http to go to page)
- "raw" - The data appears on disk exactly the same as in memory, in terms of byte values and byte ordering. Produced by write() and fwrite(), suitable for read() or fread().
Info of file:
http://www.sci.utah.edu/~gk/DTI-data/gk2/gk2-rcc-mask.nhdr - I think the only things that matter here are the big endian (still trying to understand what that means from google) and raw encoding.
My current approach, uncertain if it's correct:
//Function ripped off from example of c++ ifstream::read reference page
void scantensor(string filename){
ifstream tdata(filename, ifstream::binary); // not sure if I should put ifstream::binary here
// other things I tried
// ifstream tdata(filename) ifstream tdata(filename, ios::in)
if(tdata){
tdata.seekg(0, tdata.end);
int length = tdata.tellg();
tdata.seekg(0, tdata.beg);
char* buffer = new char[length];
tdata.read(buffer, length);
tdata.close();
double* d;
d = (double*) buffer;
} else cerr << "failed" << endl;
}
/* P.S. I attempted to print the first 100 elements of the array.
Then I print 100 other elements at some arbitrary array indices (i.e. 9,900 - 10,000). I actually kept increasing the number of 0's until I ran out of bound at 100,000,000 (I don't think that's how it works lol but I was just playing around to see what happens)
Here's the part that makes me suspicious: so the ifstream different has different constructors like the ones I tried above.
the first 100 values are always the same.
if I use ifstream::binary, then I get some values for the 100 arbitrary printing
if I use the other two options, then I get -6.27744e+066 for all 100 of them
So for now I am going to assume that ifstream::binary is the correct one. The thing is, I am not sure if the file I provided is how binary files actually look like. I am also unsure if these are the actual numbers that I am supposed to read in or just casting gone wrong. I do realize that my casting from char* to double* can be unsafe, and I got that from one of the threads.
*/
I really appreciate it!
Edit 1: Right now the data being read in using the above method is apparently "incorrect" since in paraview the values are:
Dxx,Dxy,Dxz,Dyy,Dyz,Dzz
[0, 1], [-15.4006, 13.2248], [-5.32436, 5.39517], [-5.32915, 5.96026], [-17.87, 19.0954], [-6.02961, 5.24771], [-13.9861, 14.0524]
It's a 3 x 3 symmetric matrix, so 7 distinct values, 7 ranges of values.
The floats that I am currently parsing from the file right now are very large (i.e. -4.68855e-229, -1.32351e+120).
Perhaps somebody knows how to extract the floats from Paraview?
Since you want to work with doubles, I recommend to read the data from file as buffer of doubles:
const long machineMemory = 0x40000000; // 1 GB
FILE* file = fopen("c:\\data.bin", "rb");
if (file)
{
int size = machineMemory / sizeof(double);
if (size > 0)
{
double* data = new double[size];
int read(0);
while (read = fread(data, sizeof(double), size, file))
{
// Process data here (read = number of doubles)
}
delete [] data;
}
fclose(file);
}
I'm trying to write-to-disk an array containing 11.26 million uint16_t values. The total memory size should be ~22 MB. However, the size of my file is 52MB. I'm using fprintf to write the array to disk. I thought maybe the values were being promoted. I tried to be explicit but it seems to make no difference. The size of my file is stubbornly unchanged.
What am I doing wrong? Code follows.
#define __STDC_FORMAT_MACROS
...
uint32_t dbsize = 11262336;
uint16_t* db_ = new uint16_t[dbsize_];
...
char fname[256] = "foo";
FILE* f = fopen(fname, "wb");
if(f == NULL)
{
return;
}
fprintf(f, "%i\t", dbsize_);
for(uint32_t i = 0; i < dbsize_; i++)
{
fprintf(f, "%" SCNu16 "", db_[i]);
}
fclose(f);
You're writing ASCII to your file, not binary.
Try writing your array like this instead of using fprintf in a loop.
fwrite(db_, sizeof(db_[0]), dbsize, f);
fprintf always formats numbers and other types to text, whether you've opened the file in binary mode or not. Binary mode just keeps the runtime from doing things like converting \n to \r\n.
fprintf will convert you number to a series of ASCII characters and write them to a file. Depending on its value, a 32-bit int will be from 1 to 10 characters long when expressed as a string. You need to use fwrite to write raw binary values to a file.
The source of confusion is likely to be that the "b" in FILE* f = fopen(fname, "wb"); does not do what you think it does.
Most significantly, it doesn't change any of the print or scan statements to use binary values instead of ASCII values. Like others have said - use fwrite instead.
I've tried everything I can think of and can't get anything to work. I have a binary file I've written in VB.Net which basically consists of an integer, (in binary of course) that tells me the array size for the following data, then the floats as binary data. The file writes just fine from VB.Net, and I can read it back in through Visual C++ just fine using the following code:
ifstream output("c:\\out.ipv", ios::in | ios::binary);
UInt32 len;
UInt32 *ptr2 = (UInt32*)&len;
output.read((char*)ptr2, 4);
This returns the correct value of 456780, bytes are: 76, 248, 6, 0. When I run the exact same code on my iPad, I get 1043089572. If I use the alternate method below:
NSData *data = [[NSData alloc] initWithContentsOfFile:filePath];
UInt32 num;
const NSRange numV = {0, 4};
[data getBytes:&num range:numV];
This code returns a different value, 124724, and I'm not sure how to read what the exact bytes are that are getting pulled from the file. That's something else I was trying to figure out but couldn't get working. Any idea why the same method that works in Visual C++ won't work on the iPad? I'm really at a loss on this one.
This sounds like an endian issue. You can use any of the functions in <libkern/OSByteOrder.h> to read data in a specified endianness. In your case, you may want to do something like
NSInputStream *istream = [NSInputStream inputStreamWithFileAtPath:filePath];
UInt32 num = 0;
if (istream) {
uint8_t buffer[4];
if ([istream read:buffer maxLength:4] == 4) {
num = OSReadLittleInt32(buffer, 0);
} else {
// there weren't 4 bytes in the file
}
} else {
// the file could not be opened
}
OK, something really strange is going on with my data. I just looked at the raw byte values in both Visual c++ and objective-c, and they don't agree at all. I'm only reading the first four bytes of the file and looking at their values. I'm assuming at this point that I'm not reading them in correctly, but I don't know what I'm missing here. The Visual C++ code I'm using to look at the byte values is below:
ifstream input("c:\\out.ipv", ios::in | ios::binary);
Byte tmp[4];
input.read((char*)&tmp[0], 4);
The values in the tmp array are:
76
248
6
0
If I do the same thing in objective-c:
ifstream input([filePath UTF8String], ios::in | ios::binary);
Byte tmp[4];
input.read((char*)&tmp[0], 4);
I get:
164
72
44
62
What gives? I would have at least expected to get the same byte values. The file containing the four bytes I am having trouble with is here: newout1.ipv
EDIT:
I realized where the 164,72,44,62 byte values are coming from: those are the intial values the Byte array has before I put anything in it. For some reason the line:
input.read((char*)&tmp[0], 4);
isn't doing anything. Any ideas why it's not reading from the file like it should?
FINAL EDIT:
OK, I probably shouldn't post the answer to this since it makes me look really dumb, but I don't want anyone reading these posts to get confused. So the arrays and objects were always returning the same values no matter what, which also happened to be whatever values they had when they were allocated. I had one too many .'s in my filename, so it was trying to read in out..ipv rather than out.ipv. Once I fixed the filename, everything worked exactly how I expected it to. Sorry for the confusion, and thanks for everyones help.