C++: reading large pgm file with ifstream - c++

I want to load a 16-bit binary PGM image with a size of 4096x4096 pixels using std::ifstream. The problem is that I can only load smaller files, eg. 512x512. If I try to load the "large" one the data I get is always 0 for every pixel.
Example Code:
int size = width*height;
unsigned short* data = new unsigned short[size];
// Read the terrain data
for(int i = 0; i < size; i++)
{
file >> data[i];
}
If I set size manually to a lower value, this seems to work. Any idea?
Thx Tim

operator >> should not be used for binary extraction operations. Instead, by using read the file will simply input the bytes:
file.read(reinterpret_cast<char*>(data), sizeof data);

Related

OpenCV vs byte array

I am working on a simple C++ image processing application and deciding whether to use OpenCV for loading the image and accessing individual pixels.
My current approach is to simply load the image using fopen, reading the 54 byte header and load the rest of the bytes in a char* array.
To access a specific pixel I use
long q = (long*)(bmpData + x*3 + (bmpSize.height - y - 1) * bmpSize.stride);
To perform a simple color check, for ex. "is blue?"
if (((long*)q | 0xFF000000) == 0xFFFF0000) //for some reason RGB is reversed to BGR
//do something here
Is OpenCV any faster considering all the function calls, parsing, etc.?
Bitmap file header is actually 54 bytes and you can't skip it. You have to read it to find the width, height, bitcount... calculate padding if necessary... and other information.
Depending on how the file is opened, OpenCV will read the header and reads the pixels directly in to a buffer. The only change is that the rows are flipped so the image is right side up.
cv::Mat mat = cv::imread("filename.bmp", CV_LOAD_IMAGE_COLOR);
uint8_t* data = (uint8_t*)mat.data;
The header checks and the small changes made by OpenCV will not significantly affect performance. The bottle neck is mainly in reading the file from the disk. The change in performance will be difficult to measure, unless you are doing a very specific task, for example you want only 3 bytes in a very large file, and you don't want to read the entire file.
OpenCV is overkill for this task, so you may choose other libraries for example CImg as suggested in comments. If you use smaller libraries they load faster, it might be noticeable when your program starts.
The following code is a test run on Windows.
For a large 16MB bitmap file, the result is almost identical for opencv versus plain c++.
For a small 200kb bitmap file, the result is 0.00013 seconds to read in plain C++, and 0.00040 seconds for opencv. Note the plain c++ is not doing much beside reading the bytes.
class stopwatch
{
std::chrono::time_point<std::chrono::system_clock> time_start, time_end;
public:
stopwatch() { reset();}
void reset(){ time_start = std::chrono::system_clock::now(); }
void print(const char* title)
{
time_end = std::chrono::system_clock::now();
std::chrono::duration<double> diff = time_end - time_start;
if(title) std::cout << title;
std::cout << diff.count() << "\n";
}
};
int main()
{
const char* filename = "filename.bmp";
//I use `fake` to prevent the compiler from over-optimization
//and skipping the whole loop. But it may not be necessary here
int fake = 0;
//open the file 100 times
int count = 100;
stopwatch sw;
for(int i = 0; i < count; i++)
{
//plain c++
std::ifstream fin(filename, std::ios::binary);
fin.seekg(0, std::ios::end);
int filesize = (int)fin.tellg();
fin.seekg(0, std::ios::beg);
std::vector<uint8_t> pixels(filesize - 54);
BITMAPFILEHEADER hd;
BITMAPINFOHEADER bi;
fin.read((char*)&hd, sizeof(hd));
fin.read((char*)&bi, sizeof(bi));
fin.read((char*)pixels.data(), pixels.size());
fake += pixels[i];
}
sw.print("time fstream: ");
sw.reset();
for(int i = 0; i < count; i++)
{
//opencv:
cv::Mat mat = cv::imread(filename, CV_LOAD_IMAGE_COLOR);
uint8_t* pixels = (uint8_t*)mat.data;
fake += pixels[i];
}
sw.print("time opencv: ");
printf("show some fake calculation: %d\n", fake);
return 0;
}

Read bmp file header size

I am trying to find file size, file header size width, and height of a bmp file. I have studied the format of bmp file and the arrangement of bytes in file.
When I try this code it shows wrong width and height for different files.
I have tried this for three images so far. This one image results the right measurement.
This one did not:
I don't understand where I went wrong, but the bit depth showed the right value for all three images.
Here is my code:
#include<iostream>
#include<fstream>
#include<math.h>
using namespace std;
int main() {
ifstream inputfile("bmp.bmp",ios::binary);
char c; int imageheader[1024];
double filesize=0; int width=0; int height=0;int bitCount = 0;
for(int i=0; i<1024; i++) {
inputfile.get(c); imageheader[i]=int(c);
}
filesize=filesize+(imageheader[2])*pow(2,0)+(imageheader[3])*pow(2,8)+(imageheader[4])*pow(2,16)+(imageheader[5])*pow(2,24);
cout<<endl<<endl<<"File Size: "<<(filesize/1024)<<" Kilo Bytes"<<endl;
width=width+(imageheader[18])*pow(2,0)+(imageheader[19])*pow(2,8)+(imageheader[20])*pow(2,16)+(imageheader[21])*pow(2,24);
cout<<endl<<"Width: "<<endl<<(width)<<endl;
height=height+(imageheader[22])*pow(2,0)+(imageheader[23])*pow(2,8)+(imageheader[24])*pow(2,16)+(imageheader[25])*pow(2,24);
cout<<endl<<"Height: "<<endl<<(height)<<endl;
bitCount=bitCount+(imageheader[28])*pow(2,0)+(imageheader[29])*pow(2,8);
cout<<endl<<"Bit Depth: "<<endl<<(bitCount)<<endl;
}
Let's start by reading the BMP header in as a series of bytes, not integers. To make this code truly portable, we'll use <stdint> types.
#include <fstream>
#include <stdint.h>
int main()
{
ifstream inputfile("D:/test.bmp", ios::binary);
uint8_t headerbytes[54] = {};
inputfile.read((char*)headerbytes, sizeof(headerbytes));
Now that we've got the header in memory as an array of bytes, we can simply cast the memory address of each header field back into a integer. Referencing the wikipedia page for bmp and the layout diagram.
uint32_t filesize = *(uint32_t*)(headerbytes+2);
uint32_t dibheadersize = *(uint32_t*)(headerbytes + 14);
uint32_t width = *(uint32_t*)(headerbytes + 18);
uint32_t height = *(uint32_t*)(headerbytes + 22);
uint16_t planes = *(uint16_t*)(headerbytes + 26);
uint16_t bitcount = *(uint16_t*)(headerbytes + 28);
Now an astute reader of the code will recognize that the individual fieds of a a BMP headers are stored in little endian format. And that the code above relies on you to have an x86 processor or any other architecture in which the byte layout is Little Endian. On a big endian machine, you'll have to apply a workaround to convert from LE to BE for each of the variables above.
The bug is reading into signed char. This should fix it:
for(int i = 0; i < 1024; i++)
{
//inputfile.get(c); imageheader[i] = int(c);
// This version of get returns int, where -1 means EOF. Should be checking for errors...
imageheader[i] = inputfile.get();
}
Others have commented on improvements to the code so I won't bother.

Write 2D array to PGM grayscale image

I'm creating various 2D arrays of sizes from 100x100 to 2000x2000 elements. The values within the arrays can be clamped down to 0 - 255 gray scale and then need to be written to a PGM image in order to visually represent the data.
For example, I'm declaring the arrays globally as:
element case1[100][100];
element is a structure of double pixelValue and a Boolean value (that won't be used when actually writing to the file but is necessary in the program).
In writing to the PGM image, I am having errors considering the FILE *fp in this area of the code when writing after the header:
int *p
for (int x = 0; x < dimension; x++)
{
for (int y = 0; y < dimension; y++)
{ //also doesn't work as: fp << (unsigned char)case1[x][y].pix;
int pix = case1[x][y].pixelValue;
*p = pix;
fp << (unsigned char)*p;
}
}
fclose(fp);
I'm unsure of how to work with the pointer in order to get the pixelValue from each location within the 2D array. I need to be able to iterate through each pixelValue to get the visual representation of the data.
Thank you for your help!
Used fputc() instead so that I could directly insert values instead of using pointers.

C++ - Trying to read a .PPM image, unexpected output

I'm developing a uni project for reading the image data of P6-type, 255-depth .ppm images. The problem I encounter is that when I try to print the average values of each color (R,G,B) for an Image, the output I get is wrong (the proffessor has given us an output file which says what float values to expect for each given image).
Now, I'm at a loss here. Through many checks, I have concluded that the function reads the whole data from the image, without leaving out pixels or whatever, converts them correctly from 0-255 to 0.f - 1.f values (by dividing with 255.0), adds every red, every green and every blue value to three seperate counters and then divides them by the Width*Height of the given image to get the desired average brightness of each colour. I will provide part of the function that does this process for calculating the average red for a 960*642 Image (sorry for the hardcoded stuff, it's just there for debugging purposes).
The output I get for this is 0.58... when it should be 0.539068. seekg() is called with 14 as an argument because position 14 is the last space after the header and before the data. Could you provide any insight to why this isn't working as expected? One thing I found through the checks is the sum I get after adding all the red float values, is not a float but an int. Possible loss of data? I'm grasping at straws here.
Here is the code:
std::ifstream infile;
infile.open("Image02.ppm", std::ios::in | std::ios::binary);
const unsigned char* buffer;
float * data_ptr;
infile.seekg(0, std::ios::end);
int length = infile.tellg(); //calculating length of data
buffer = new unsigned char[length];
ptr = new unsigned char[length];
data_ptr = new float[length];
infile.seekg(14, std::ios::beg); //restoring pointer to the start of data stream
infile.read((char*)buffer, length); //reading the image
for (int i = 0; i < length; i++){ //casting the char data to floats to get the 0-255 values
data_ptr[i] = ((float)buffer[i]);
data_ptr[i] = data_ptr[i] / 255.f; // converting to 0.0 - 1.0
}
int j = 0;
float a = 0.f;
while (j < length){ //calculating sum of red pixel values
a = a + data_ptr[j];
j = j + 3;
}
std::cout << a / (960*642); //calculating average
FYI, PPM image files that are P6 have their image data stored from left to right, with the first line being line 0 and the last line of the image being the last. They are structured like this R G B R G B R G B so on, where the first RGB correspond to the first pixel and so forth.
Thanks in advance!
You need pixels only for average calculation.
But in your source code, additional 14 garbage values are being used.

Save a Matrix in Matlab to a bin file and read it in c++

I saved an integer 10x10 matrix I in Matlab into a binary file using fwrite:
fid = fopen('True.bin' , 'w');
fwrite(fid , I , 'int');
fclose(fid)
Now when I try to open it in C++ using fstream library, it returns an error. The C++ code is as follows:
int IMG_SIZE = 10;
char * buffer;
long size = IMG_SIZE * IMG_SIZE;
ifstream file ("True.bin", ios::in|ios::binary|ios::ate);
buffer = new char [size];
file.read (buffer, size);
file.close();
Any idea how to load the file into an array, or Eigen matrix of type MatrixXi ?
Thanks
When you write values using the 'int' value for the precision argument to fwrite, it writes them as 4-byte integers, so your 10x10 matrix will take up 10x10x4 = 400 bytes. But you are only allocating a buffer that is 10x10 = 100 bytes long.