a follow up to my previous question (Reading an entire file in binary mode using C++)
After reading a jpg file in binary mode, the result of the read operation is always 4 bytes. The code is:
FILE *fd = fopen("c:\\Temp\\img.jpg", "rb");
if(fd == NULL) {
cerr << "Error opening file\n";
return;
}
fseek(fd, 0, SEEK_END);
long fileSize = ftell(fd);
int *stream = (int *)malloc(fileSize);
fseek(fd, 0, SEEK_SET);
int bytes_read = fread(stream, fileSize, 1, fd);
printf("%x\n", *stream);
fclose(fd);
The second last printf statement is always printing the first 4 bytes and not the entire file contents. How can I print the entire content of the jpg file?
Thanks.
You want it in C++? This opens a file, reads the entire contents into an array and prints the output to the screen:
#include <fstream>
#include <vector>
#include <iostream>
#include <algorithm>
using namespace std;
void hexdump(void *ptr, int buflen)
{
unsigned char *buf = (unsigned char*)ptr;
int i, j;
for (i=0; i<buflen; i+=16) {
printf("%06x: ", i);
for (j=0; j<16; j++) {
if (i+j < buflen)
printf("%02x ", buf[i+j]);
else
printf(" ");
}
printf(" ");
for (j=0; j<16; j++) {
if (i+j < buflen)
printf("%c", isprint(buf[i+j]) ? buf[i+j] : '.');
}
printf("\n");
}
}
int main()
{
ifstream in;
in.open("C:\\ISO\\ITCHOUT.txt", ios::in | ios::binary);
if(in.is_open())
{
// get the starting position
streampos start = in.tellg();
// go to the end
in.seekg(0, std::ios::end);
// get the ending position
streampos end = in.tellg();
// go back to the start
in.seekg(0, std::ios::beg);
// create a vector to hold the data that
// is resized to the total size of the file
std::vector<char> contents;
contents.resize(static_cast<size_t>(end - start));
// read it in
in.read(&contents[0], contents.size());
// print it out (for clarity)
hexdump(contents.data(), contents.size());
}
}
stream is a pointer to an int (the first element of the array you allocated1). *stream dereferences that pointer and gives you the first int.
A pointer is not an array. A pointer is not a buffer. Therefore, it carries no information about the size of the array it points to. There is no way you can print the entire array by providing only a pointer to the first element.
Whatever method you use to print that out, you'll need to provide the size information along with the pointer.
C++ happens to have a pointer + size package in its standard library: std::vector. I would recommend using that. Alternatively, you can just loop through the array yourself (which means using the size information) and print all its elements.
1Make sure the size of the file is a multiple of sizeof(int)!
Something like the following should do it. bytes_read() gives you the number of blocks read, in your case the block size is the file size so only one block can be read.
You should use a for loop to print the whole file. You're only printing one pointer address.
char *stream = (char *)malloc(fileSize);
fseek(fd, 0, SEEK_SET);
int bytes_read = fread(stream, fileSize, 1, fd);
for(int i=0; i<fileSize; i++){
printf("%d ", stream[i]);
}
I print the chars as numbers as binary data is not readable in the console. I don't know how you wanted the data to be formatted.
This is just meant as reference to your sample. You should really consider using Chad's sample. This is a far worse solution (as mixing C/C++ far too much) just for sake of completeness.
Related
I have a program where I need to operate on different types of files.
I want the input and output files of the following program to be the same.
#include<iostream>
#include<string>
#include<fstream>
#include<sstream>
typedef unsigned char u8;
using namespace std;
char* readFileBytes(string name)
{
ifstream fl(name);
fl.seekg( 0, ios::end );
size_t len = fl.tellg();
char *ret = new char[len];
fl.seekg(0, ios::beg);
fl.read(ret, len);
fl.close();
return ret;
}
int main(int argc, char *argv[]){
string name = "file.pdf";
u8* file = (u8*) readFileBytes(name);
// cout<<str<<endl;
int len = 0;
while(file[len] != '\0')
len++;
cout<<"FILESIZE : "<<len<<endl;
string filename = "file2.pdf";
ofstream outfile(filename,ios::out | ios::binary);
outfile.write((char*) file,len);
outfile.close();
exit(0);
}
The difference between the output and input files is checked using diff
diff file.pdf file2.pdf
What should I do to make file2.pdf the same as file.pdf?
I have tried using xxd to change the binary into hexadecimal but the disadvantage is that the overall size doubles. So therefore I want to operate in binary only.
size_t len = fl.tellg();
char *ret = new char[len];
In this manner the shown code determines the number of characters in the file. This is fine. The only problem with it is that after this number of characters is read, this very important information is completely forgotten and thrown away. This function returns only this ret pointer, and the actual number of characters in it is now an unsolvable mystery.
But then, main() attempts to solve this mystery as follows:
int len = 0;
while(file[len] != '\0')
len++;
This attempts to reverse-engineer the number of characters by looking for the first 0 byte in the buffer.
Which has absolutely nothing to do with anything. The first character in the file may be a 0 byte, so this will calculate that the file is empty, and not ten gigabytes in size.
Or the file can contain just a string "Hello world", which this for loop will happily blow past, then start rooting around in some random memory after this buffer, resulting in undefined behavior.
That's the fatal logical flaw in the shown code: the actual size of the file is thrown away, and instead reverse-engineered in a flawed way.
You will need to rework the code so that the number of characters in the file, the original len, is also returned to main(), and it uses that, instead of attempting to guess what it originally was.
P.S. delete-ing the ret buffer, after you're done with it, would also be a good idea too. An even better idea is to avoid using new, using vector instead, which will happily give you its size() any time you ask for it, and you won't have to worry about deleting the allocated memory.
In order to correctly process binary data, the size must be stored and cannot be computed from a sentinel null byte, because null bytes can be legimate bytes in a binary file. So you should return the read lenght in addition to the buffer, or even better copy each buffer to the new file until you have exhausted the input file:
int main(int argc, char *argv[]){
constexpr size_t sz = 10240; // size of buffer
char buffer[sz];
string name = "file.pdf";
string filename = "file2.pdf";
ifstream fl(name);
ofstream outfile(filename,ios::out | ios::binary);
int len = 0, buflen;
for (;;) {
buflen = fl.read(buf, len);
if (buflen == 0) break; // reached EOF
len += buflen;
if (buflen != outfile.write(buf, buflen)) {
// display an error message
return 1;
}
}
fl.close();
outfile.close()
cout<<"FILESIZE : "<<len<<endl;
exit(0);
}
void demodlg::printData(short* data)
{
FILE* pF;
char buf[50];
snprintf(buf, sizeof(buf), "%s\\%s\\%s%d.binary", "test", "data", "data", frameNum++);
pF = fopen(buf, "wb");
int lines = frameDescr->m_numLines;
int samples = frameDescr->m_pLineTypeDescr[0].m_numSamples;
int l, s;
fprintf(pF, "\t");
for (l = 0; l < lines; l++)
{
fprintf(pF, "%d\t", l);
}
fprintf(pF, "\n");
for (s = 0; s < samples; s++)
{
fprintf(pF, "%d)\t", s);
for (l = 0; l < lines; l++)
{
fprintf(pF, "%d\t", *(data + l * samples + s));
}
fprintf(pF, "\n");
}
fclose(pF);
}
I have the code snippet above which just takes in some data and then writes it out to a binary file. This function gets called about 20-30 times per second, so I'm trying to optimize it as much as possible. Each file that it writes to is about 1 MB in size. Ideally, I'd be able to write 20-30 MB per second. As of now, it's not at that rate.
Does anyone have any ideas on how I can optimize this further?
I originally was writing to a txt file before changing to a binary file, but the different isn't too noticeable, surprisingly.
Also, frameDescr gets updated for every frame so I believe I do need to get access to the lines and samples variables from inside, unfortunately.
I found this post to refer to (Writing a binary file in C++ very fast) but I'm not sure how I can apply it to mine.
Here is a short example of how I would write an array of data to a binary file and how I would read it back.
I do not understand the concept or purpose of lines in your code so I did not attempt to replicate it. If you do have additional data you need to write to allow it to be reconstructed when read I have placed comments to note where you could insert that code.
Keep in mind that the data when written as binary must be read the same way, so if you were writing the text in a particular format to consume it from another program then a binary file will not work for you unless you modify that other program or create an additional step to read the binary data and write the text format before consumption.
Assuming there is a speed advantage to writing the data as binary then adding an additional step to convert the binary data to text format is beneficial because you can do it offline when you're not trying to maintain a particular frame rate.
Normally since you tagged this c++ I would prefer manipulating the data in a vector and perhaps using c++ streams to write and read the data, but I tried to keep this as similar to your code as possible.
#include <cstdio>
#include <stdint.h>
const size_t kNumEntries = 128 * 1024;
void writeData(const char *filename, int16_t *data, size_t numEntries)
{
FILE *f = fopen(filename, "wb");
if (!f)
{
fprintf(stderr, "Error opening file: '%s'\n", filename);
return;
}
//If you have additional data that must be in the file write it here
//either as individual items that are mirrored in the reader,
//or using the pattern showm below for variable sized data.
//Write the number of entries we have to write to the file so the reader
//will know how much memory to allocate how many to read.
fwrite(&numEntries, sizeof(numEntries), 1, f);
//Write the actual data
fwrite(data, sizeof(*data), numEntries, f);
fclose(f);
}
int16_t* readData(const char *filename)
{
FILE *f = fopen(filename, "rb");
if (!f)
{
fprintf(stderr, "Error opening file: '%s'\n", filename);
return 0;
}
//If you have additional data to read, do it here.
//This code whould mirror the writing function.
//Read the number of entries in the file.
size_t numEntries;
fread(&numEntries, sizeof(numEntries), 1, f);
//Allocate memory for the entreis and read them into it.
int16_t *data = new int16_t[sizeof(int16_t) * numEntries];
fread(data, sizeof(*data), numEntries, f);
fclose(f);
return data;
}
int main()
{
int16_t *dataToWrite = new int16_t[sizeof(int16_t) * kNumEntries];
int16_t *dataRead = new int16_t[sizeof(int16_t) * kNumEntries];
for (int i = 0; i < kNumEntries; ++i)
{
dataToWrite[i] = i;
dataRead[i] = 0;
}
writeData("test.bin", dataToWrite, kNumEntries);
dataRead = readData("test.bin");
for (int i = 0; i < kNumEntries; ++i)
{
if (dataToWrite[i] != dataRead[i])
{
fprintf(stderr,
"Data mismatch at entry %d, : dataToWrite = %d, dataRead = %d\n",
i, dataToWrite[i], dataRead[i]);
}
}
delete[] dataRead;
return 0;
}
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Could anyone tell me a simple way, how to read the last X bytes of a specific file?
If I'm right, I should use ifstream, but I'm not sure how to use it. Currently I'm learning C++ ( at least I'm trying to learn :) ).
Input file streams have the seekg() method that repositions the current position to an absolute position or a relative position. One overload takes a positon type that represents an absolute value. The other takes an offset type and direction mask that determines the relative position to move to. Negating the offset allows you to move backward. Specifying the end constant moves the indicator relative to the end.
file.seekg(-x, std::ios_base::end);
This is a C solution, but works and handles errors. The trick is to use a negative index in fseek to "seek from EOF" (ie: seek from the "right").
#include <stdio.h>
#define BUF_SIZE (4096)
int main(void) {
int i;
const char* fileName = "test.raw";
char buf[BUF_SIZE] = { 0 };
int bytesRead = 0;
FILE* fp; /* handle for the input file */
size_t fileSize; /* size of the input file */
int lastXBytes = 100; /* number of bytes at the end-of-file to read */
/* open file as a binary file in read-only mode */
if ((fp = fopen("./test.txt", "rb")) == NULL) {
printf("Could not open input file; Aborting\n");
return 1;
}
/* find out the size of the file; reset pointer to beginning of file */
fseek(fp, 0L, SEEK_END);
fileSize = ftell(fp);
fseek(fp, 0L, SEEK_SET);
/* make sure the file is big enough to read lastXBytes of data */
if (fileSize < lastXBytes) {
printf("File too small; Aborting\n");
fclose(fp);
return 1;
} else {
/* read lastXBytes of file */
fseek(fp, -lastXBytes, SEEK_END);
bytesRead = fread(buf, sizeof(char), lastXBytes, fp);
printf("Read %d bytes from %s, expected %d\n", bytesRead, fileName, lastXBytes);
if (bytesRead > 0) {
for (i=0; i<bytesRead; i++) {
printf("%c", buf[i]);
}
}
}
fclose(fp);
return 0;
}
You need to use he seekg function and pass a negative offset from the end of the stream.
std::ifstream is("file.txt");
if (is)
{
is.seekg(-x, is.end); // x is the number of bytes to read before the end
}
#include <iostream>
#include <fstream>
using namespace std;
int main(int argc, char* argv)
{
ifstream ifs("F:\\test.data", ifstream::binary);
if(ifs.fail())
{
cout << "Error:fail to open file" << endl;
return -1;
}
//read the last 10 bits of file
const int X = 10;
char* buf = new char[X];
ifs.seekg(-X, SEEK_END);
ifs.read(buf, X);
ifs.close();
delete buf;
return 0;
}
Use seekg() for relative positionning from end of file, then use read():
ifstream ifs("test.txt");
int x=10;
char buffer[11]={};
ifs.seekg(-x, ios_base::end);
if (!ifs.read(buffer, x))
cerr << "There's a problem !\n";
else cout <<buffer<<endl;
Note that read() just takes the x bytes from the file and puts them in the buffer, without adding a '\0' at the end. So if you expect a C string, you have to make sure that your buffer ends with a 0.
I'm trying to read a binary file and store it in a buffer. The problem is, that in the binary file are multiple null-terminated characters, but they are not at the end, instead they are before other binary text, so if I store the text after the '\0' it just deletes it in the buffer.
Example:
char * a = "this is a\0 test";
cout << a;
This will just output: this is a
here's my real code:
this function reads one character
bool CStream::Read (int * _OutChar)
{
if (!bInitialized)
return false;
int iReturn = 0;
*_OutChar = fgetc (pFile);
if (*_OutChar == EOF)
return false;
return true;
}
And this is how I use it:
char * SendData = new char[4096 + 1];
for (i = 0; i < 4096; i++)
{
if (Stream.Read (&iChar))
SendData[i] = iChar;
else
break;
}
I just want to mention that there is a standard way to read from a binary file into a buffer.
Using <cstdio>:
char buffer[BUFFERSIZE];
FILE * filp = fopen("filename.bin", "rb");
int bytes_read = fread(buffer, sizeof(char), BUFFERSIZE, filp);
Using <fstream>:
std::ifstream fin("filename.bin", ios::in | ios::binary );
fin.read(buffer, BUFFERSIZE);
What you do with the buffer afterwards is all up to you of course.
Edit: Full example using <cstdio>
#include <cstdio>
const int BUFFERSIZE = 4096;
int main() {
const char * fname = "filename.bin";
FILE* filp = fopen(fname, "rb" );
if (!filp) { printf("Error: could not open file %s\n", fname); return -1; }
char * buffer = new char[BUFFERSIZE];
while ( (int bytes = fread(buffer, sizeof(char), BUFFERSIZE, filp)) > 0 ) {
// Do something with the bytes, first elements of buffer.
// For example, reversing the data and forget about it afterwards!
for (char *beg = buffer, *end=buffer + bytes; beg < end; beg++, end-- ) {
swap(*beg, *end);
}
}
// Done and close.
fclose(filp);
return 0;
}
static std::vector<unsigned char> read_binary_file (const std::string filename)
{
// binary mode is only for switching off newline translation
std::ifstream file(filename, std::ios::binary);
file.unsetf(std::ios::skipws);
std::streampos file_size;
file.seekg(0, std::ios::end);
file_size = file.tellg();
file.seekg(0, std::ios::beg);
std::vector<unsigned char> vec;
vec.reserve(file_size);
vec.insert(vec.begin(),
std::istream_iterator<unsigned char>(file),
std::istream_iterator<unsigned char>());
return (vec);
}
and then
auto vec = read_binary_file(filename);
auto src = (char*) new char[vec.size()];
std::copy(vec.begin(), vec.end(), src);
The problem is definitievely the writing of your buffer, because you read a byte at a time.
If you know the length of the data in your buffer, you could force cout to go on:
char *bf = "Hello\0 world";
cout << bf << endl;
cout << string(bf, 12) << endl;
This should give the following output:
Hello
Hello world
However this is a workaround, as cout is foreseent to output printable data. Be aware that the output of non printable chars such as '\0' is system dependent.
Alternative solutions:
But if you manipulate binary data, you should define ad-hoc data structures and printing. Here some hints, with a quick draft for the general principles:
struct Mybuff { // special strtucture to manage buffers of binary data
static const int maxsz = 512;
int size;
char buffer[maxsz];
void set(char *src, int sz) // binary copy of data of a given length
{ size = sz; memcpy(buffer, src, max(sz, maxsz)); }
} ;
Then you could overload the output operator function:
ostream& operator<< (ostream& os, Mybuff &b)
{
for (int i = 0; i < b.size; i++)
os.put(isprint(b.buffer[i]) ? b.buffer[i]:'*'); // non printables replaced with *
return os;
}
ANd you could use it like this:
char *bf = "Hello\0 world";
Mybuff my;
my.set(bf, 13); // physical copy of memory
cout << my << endl; // special output
I believe your problem is not in reading the data, but rather in how you try to print it.
char * a = "this is a\0 test";
cout << a;
This example you show us prints a C-string. Since C-string is a sequence of chars ended by '\0', the printing function stops at the first null char.
This is because you need to know where the string ends either by using special terminating character (like '\0' here) or knowing its length.
So, to print whole data, you must know the length of it and use a loop similar to the one you use for reading it.
Are you on Windows? If so you need to execute _setmode(_fileno(stdout), _O_BINARY);
Include <fcntl.h> and <io.h>
Is there a function for FILE (fopen?) that allows me to just read one int from a binary file?
So far I'm trying this, but I'm getting some kind of error I can't see cause the program just crashes without telling me.
void opentest()
{
FILE *fp = fopen("dqmapt.mp", "r");
int i = 0;
int j = 0;
int k = 0;
int * buffer;
if (fp)
{
buffer = (int *) (sizeof(int));
i = (int) fread(buffer,1, (sizeof(int)), fp);
fscanf(fp, "%d", &j);
fclose(fp);
}
printf("%d\n", i);
printf("%d\n", j);
}
Now that you have changed your question, let me ask one. What is the format of the file you are trying to read?
For a binary file there are some changes required how you open the file:
/* C way */
FILE *fp = fopen("text.bin", "rb"); /* note the b; this is a compound mode */
/* C++ way */
std::ifstream ifs("test.txt", ios::in | ios::binary);
Reading in the contents is easy. But remember, your file has 2 integers at the begining -- width, height which determine how many more to read i.e. another width * height number of integers. So, your best bet is to read the first two integers first. You will need to use two buffers -- one for the width and height and then depending on their value another one to read the rest of the file. So, lets read in the first two integers:
char buf[ 2 * sizeof(int) ]; /* will store width and height */
Read in the two integers:
/* C way */
fread(buf, sizeof(int), 2, fp); /* the syntax changes, FILE pointer is last */
/* C++ way*/
ifs.read(buf, sizeof buf);
Now, the tricky part. You have to convert the stuff to double. This again depends on your system endianness -- whether a simple assignment works or whether a byte swapping is necessary. As another poster has pointed out WriteInt() writes integers in big-endian format. Figure out what system you are on. And then you can proceed further.
FILE is a C datastructure. It is included in C++ for C compatibility. You can do this:
/* The C way */
#include <stdio.h>
#include <stdlib.h>
int main(void) {
FILE *fp = fopen("test.txt", "r");
int i = 0;
if (fp) {
fscanf(fp, "%d", &i);
fclose(fp);
}
printf("%d\n", i);
}
You can use the std::ifstream thing to open a file for reading. You have to read in the contents using some other incantation to read the file contents and extract the desired information out of it yourself.
/* The C++ way */
#include <fstream>
#include <iostream>
int main() {
std::ifstream ifs("test.txt");
int i = 0;
if (ifs.good()) {
ifs >> i;
}
std::cout << i << std::endl;
}
Note you can use the C style functions in C++ as well, though this is the least recommended way:
/* The C way in C++ */
#include <cstdio>
#include <cstdlib>
int main() {
using namespace std;
FILE *fp = fopen("test.txt", "r");
int i = 0;
if (fp) {
fscanf(fp, "%d", &i);
fclose(fp);
}
printf("%d\n", i);
}
[Note: Both examples assume you have a text file to read from]
Do you want to read a textual representation of an int? Then you can use fscanf, it's sort of the opposite of printf
int n;
if( fscanf(filePointer, "%d", &n) == 1 )
// do stuff with n
If you want to read some binary data and treat it as an int, well that's going to depend how it was written in the first place.
I am not a Java programmer, so this is just based on what I've read in the [docs](http://java.sun.com/j2se/1.4.2/docs/api/java/io/DataOutputStream.html#writeInt(int)).
That said, it says
Writes an int to the underlying output stream as four bytes, high byte first. If no exception is thrown, the counter written is incremented by 4.
So it's a big endian four byte integer. I don't know if it's two's complement or not, but that's probably a safe assumption (and can probably be found somewhere in the java docs/spec). Big endian is the same as network byte order, so you can use ntohl to convert it the endianness of your C++ platform. Beyond that, you just need to read the four bytes, which can be done with fread.
Int represented as text or binary?
For text, use fscanf; for binary, use fread.