Related
Question:
Is there a good way to write a 3D float vector of size (9000,9000,4) to an output file in C++?
My C++ program generates a 9000x9000 image matrix with 4 color values (R, G, B, A) for each pixel. I need to save this data as an output file to be read into a numpy.array() (or similar) using python at a later time. Each color value is saved as a float (can be larger than 1.0) which will be normalized in the python portion of the code.
Currently, I am writing the (9000,9000,4) sized vector into a CSV file with 81 million lines and 4 columns. This is slow for reading and writing and it creates large files (~650MB).
NOTE: I run the program multiple times (up to 20) for each trial, so read/write times and file sizes add up.
Current C++ Code:
This is the snippet that initializes and writes the 3D vector.
// initializes the vector with data from 'makematrix' class instance
vector<vector<vector<float>>> colorMat = makematrix->getMatrix();
outfile.open("../output/11_14MidRed9k8.csv",std::ios::out);
if (outfile.is_open()) {
outfile << "r,g,b,a\n"; // writes column labels
for (unsigned int l=0; l<colorMat.size(); l++) { // 0 to 8999
for (unsigned int m=0; m<colorMat[0].size(); m++) { // 0 to 8999
outfile << colorMat[l][m][0] << ',' << colorMat[l][m][1] << ','
<< colorMat[l][m][2] << ',' << colorMat[l][m][3] << '\n';
}
}
}
outfile.close();
Summary:
I am willing to change the output file type, the data structures I used, or anything else that would make this more efficient. Any and all suggestions are welcome!
Use the old C file functions and binary format
auto startT = chrono::high_resolution_clock::now();
ofstream outfile;
FILE* f = fopen("example.bin", "wb");
if (f) {
const int imgWidth = 9000;
const int imgHeight = 9000;
fwrite(&imgWidth, sizeof(imgWidth), 1, f);
fwrite(&imgHeight, sizeof(imgHeight), 1, f);
for (unsigned int i=0; i<colorMat.size(); ++i)
{
fwrite(&colorMat[i], sizeof(struct Pixel), 1, f);
}
}
auto endT = chrono::high_resolution_clock::now();
cout << "Time taken : " << chrono::duration_cast<chrono::seconds>(endT-startT).count() << endl;
fclose(f);
The format is the following :
[ImageWidth][ImageHeight][RGBA][RGBA[RGBA]... for all ImageWidth * ImageHeight pixels.
Your sample ran in 119s in my machine. This code ran in 2s.
But please note that the file will be huge anyway : you are writing the equivalent of two 8K files without any kind of compression.
Besides that, some tips on your code :
Don't use a vector of floats to represent your pixels. They won't have more components than RGBA. Instead create a simple struct with four floats.
You don't need to look through width and height separately. Internally all lines are put sequentially one after the other. It is easier to create a one dimension array of width * height size.
I am writing a longer program and I found myself needing to read a .bmp file into an array in a specific way so that the rest of the program can use it without extensive rewrites. I failed to find older answers that would resolve my problem, and I am pretty much at the beginner stages.
The image I am trying to read is used to create a text font, so I want to read it character by character into an array, where the pixels belonging to one character are added in order to a 2d bool (true if pixel is not black) array [character_id] [pixel_n]. The dimensions of characters are predetermined and known, and the file is cropped so that they all appear in a single row with no unaccounted margins.
This is the specific file I am trying to read, though here it might not show up as .bmp
As an example, shown here, I want to read the pixels in the order of the yellow line, then jump to another character. For clarity each character is 5px wide and 11px high, with 1px of margin on both sides horizontally.
Based on what I was able to find, I have written a function to do it, but I fail to make it work as intended, as far as I can tell even the pixel values are not being read correctly:
void readBMP(char* filename)
{
int i;
FILE* f = fopen(filename, "rb");
unsigned char info[54];
// read the 54-byte header
fread(info, sizeof(unsigned char), 54, f);
// extract image height and width from header
int width = *(int*)&info[18];
int height = *(int*)&info[22];
// number of pixels in total
int size = 3 * width * height;
unsigned char* data = new unsigned char[size];
// number of characters to read
int counter1 = size / ((font_width + 2) * font_height) / 3 ;
// read the rest of the data at once
fread(data, sizeof(unsigned char), size, f);
fclose(f);
//loop that goes from character to character
for(int i = 0; i < counter1; i++)
{
int tmp = 0;
//loop that reads one character into font_ref array
for(int j = 0; j < font_height; j++)
{
//loop for each row of a character
for(int k = 0; k < font_width; k++)
{
int w = static_cast<int>(data[3*(j*(font_width+2)*(counter1) + i*(font_width + 2) + 1 + k + j*font_width + j)-1]);
if( w != 0 )
font_ref [i][(tmp)] = 1;
else
font_ref [i][(tmp)] = 0;
tmp++;
}
}
}
}
(bool font_ref [150][font_width*font_height]; is the array where the font is being loaded and stored)
this code reads something, but the result is a seemingly random mess and I am unable to resolve that. Here is an example of lowercase alphabet printed using another function in the program, where white pixels represent true bools. I am aware that some libraries exist to work with graphical files, however in this program I wanted to possibly avoid that to learn more lower-level things, and the goal is rather limited and specific.
Thank you in advance for any help with the issue.
The main errors are in the offset computation for a pixel in the bitmap data:
int w = static_cast<int>(data[3*(j*(font_width+2)*(counter1) + i*(font_width + 2) + 1 + k + j*font_width + j)-1]);
j*(font_width+2)*(counter1) - This doesn't take into account that
although you say the file is cropped, there is extra black space to the right of the last character cell, so the true width must be used;
(as drescherjm and user3386109 mentioned) padding bytes are appended to the rows so that their length is a multiple of four bytes.
+ j*font_width + j)-1 - This part makes no sense - perhaps you tried to compensate the above errors.
This would be correct:
int w = data[j*(3*width+3&~3)+3*(i*(font_width+2)+1+k)];
So i found this link regarding my question, but it is for c#
Create a PNG from an array of bytes
I have a variable int array of numbers.
i will call it "pix[ ]"
for now it can be any size from 3 to 256, later possibly bigger.
What i want to do now, is to convert it into a pixel image.
I am still a noobin c++ so pleas excuse me.
I tried to download some libaries that make working with libpng easier, but they do not seem to be working (ubuntu, code::blocks)
So i have questions in the following:
1) how do you create a new bitmap (which libaries, which command)?
2) how do i fill it with information from "pix[ ]" ?
3) how do i save it?
if it is a repost of a question i am happy about a link also ;)
Here is what i worked out so far, thanks for your help.
int main(){
FILE *imageFile;
int x,y,pixel,height=2,width=3;
imageFile=fopen("image.pgm","wb");
if(imageFile==NULL){
perror("ERROR: Cannot open output file");
exit(EXIT_FAILURE);
}
fprintf(imageFile,"P3\n"); // P3 filetype
fprintf(imageFile,"%d %d\n",width,height); // dimensions
fprintf(imageFile,"255\n"); // Max pixel
int pix[100] {200,200,200, 100,100,100, 0,0,0, 255,0,0, 0,255,0, 0,0,255};
fwrite(pix,1,18,imageFile);
fclose(imageFile);
}
i have not fully understood what it does. i can open the output image, but it is not a correct representation of the Array.
If i change things around, for example making a 2 dimensional array, then the image viewer tells me "expected an integer" and doesn't show me an image.
So far so good.
As i have the array before the image i created a function aufrunden to round up to the next int number because i want to create a square image.
int aufrunden (double h)
{
int i =h;
if (h-i == 0)
{
return i;
}
else
{
i = h+1;
return i;
}
}
This function is used in the creation of the image.
If the image is bigger than the information the array provides like this (a is the length of th array)
double h;
h= sqrt(a/3.0);
int i = aufrunden(h);
FILE *imageFile;
int height=i,width=i;
It might happen now, that the array is a=24 long. aufrunden makes the image 3x3 so it has 27 values...meaning it is missing the values for 1 pixel.
Or worse it is only a=23 long. also creating a 3x3 image.
What will fwrite(pix,1,18,imageFile); write in those pixels for information? It would be best if the remaing values are just 0.
*edit never mind, i will just add 0 to the end of the array until it is filling up the whole square...sorry
Consider using a Netpbm format (pbm, pgm, or ppm).
These images are extremely simple text files that you can write without any special libraries. Then use some third-party software such as ImageMagick, GraphicsMagick, or pnmtopng to convert your image to PNG format. Here is a wiki article describing the Netpbm format.
Here's a simple PPM image:
P3 2 3 255
0 0 0 255 255 255
255 0 0 0 255 255
100 100 100 200 200 200
The first line contains "P3" (the "magic number identifying it as a text-PPM), 2 (width), 3 (height), 255 (maximum intensity).
The second line contains the two RGB pixels for the top row.
The third and fourth lines each contain the two RGB pixels for rows 2 and 3.
Use a larger number for maximum intensity (e.g. 1024) if you need a larger range of intensities, up to 65535.
Edited by Mark Setchell beyond this point - so I am the guilty party!
The image looks like this (when the six pixels are enlarged):
The ImageMagick command to convert, and enlarge, is like this:
convert image.ppm -scale 400x result.png
If ImageMagick is a bit heavyweight, or difficult to install you can more simply use the NetPBM tools (from here) like this (it's a single precompiled binary)
pnmtopng image.ppm > result.png
If, as it seems, you have got Magick++ and are happy to use that, you can write your code in C/C++ like this:
////////////////////////////////////////////////////////////////////////////////
// sample.cpp
// Mark Setchell
//
// ImageMagick Magick++ sample code
//
// Compile with:
// g++ sample.cpp -o sample $(Magick++-config --cppflags --cxxflags --ldflags --libs)
////////////////////////////////////////////////////////////////////////////////
#include <Magick++.h>
#include <iostream>
using namespace std;
using namespace Magick;
int main(int argc,char **argv)
{
unsigned char pix[]={200,200,200, 100,100,100, 0,0,0, 255,0,0, 0,255,0, 0,0,255};
// Initialise ImageMagick library
InitializeMagick(*argv);
// Create Image object and read in from pixel data above
Image image;
image.read(2,3,"RGB",CharPixel,pix);
// Write the image to a file - change extension if you want a GIF or JPEG
image.write("result.png");
}
You are not far off - well done for trying! As far as I can see, you only had a couple of mistakes:
You had P3 where you would actually need P6 if writing in binary.
You were using int type for your data, whereas you need to be using unsigned char for 8-bit data.
You had the width and height interchanged.
You were using the PGM extension which is for Portable Grey Maps, whereas your data is colour, so you need to use the PPM extension which is for Portable Pix Map.
So, the working code looks like this:
#include <stdio.h>
#include <stdlib.h>
int main(){
FILE *imageFile;
int x,y,pixel,height=3,width=2;
imageFile=fopen("image.ppm","wb");
if(imageFile==NULL){
perror("ERROR: Cannot open output file");
exit(EXIT_FAILURE);
}
fprintf(imageFile,"P6\n"); // P6 filetype
fprintf(imageFile,"%d %d\n",width,height); // dimensions
fprintf(imageFile,"255\n"); // Max pixel
unsigned char pix[]={200,200,200, 100,100,100, 0,0,0, 255,0,0, 0,255,0, 0,0,255};
fwrite(pix,1,18,imageFile);
fclose(imageFile);
}
If you then run that, you can convert the resulting image to a nice big PNG with
convert image.ppm -scale 400x result.png
If you subsequently need 16-bit data, you would change the 255 to 65535, and store in an unsigned short array rather than unsigned char and when you come to the fwrite(), you would need to write double the number of bytes.
The code below will take an integer array of pixel colors as input and write it to a .bmp bitmap file or, in reverse, read a .bmp bitmap file and store its image contents as an int array. It only requires the <fstream> library. The input parameter path can be for example C:/path/to/your/image.bmp and data is formatted as data[x+y*width]=(red<<16)|(green<<8)|blue;, whereby red, green and blue are integers in the range 0-255 and the pixel position is (x,y).
#include <string>
#include <fstream>
using namespace std;
typedef unsigned int uint;
int* read_bmp(const string path, uint& width, uint& height) {
ifstream file(path, ios::in|ios::binary);
if(file.fail()) println("\rError: File \""+filename+"\" does not exist!");
uint w=0, h=0;
char header[54];
file.read(header, 54);
for(uint i=0; i<4; i++) {
w |= (header[18+i]&255)<<(8*i);
h |= (header[22+i]&255)<<(8*i);
}
const int pad=(4-(3*w)%4)%4, imgsize=(3*w+pad)*h;
char* img = new char[imgsize];
file.read(img, imgsize);
file.close();
int* data = new int[w*h];
for(uint y=0; y<h; y++) {
for(uint x=0; x<w; x++) {
const int i = 3*x+y*(3*w+pad);
data[x+(h-1-y)*w] = (img[i]&255)|(img[i+1]&255)<<8|(img[i+2]&255)<<16;
}
}
delete[] img;
width = w;
height = h;
return data;
}
void write_bmp(const string path, const uint width, const uint height, const int* const data) {
const int pad=(4-(3*width)%4)%4, filesize=54+(3*width+pad)*height; // horizontal line must be a multiple of 4 bytes long, header is 54 bytes
char header[54] = { 'B','M', 0,0,0,0, 0,0,0,0, 54,0,0,0, 40,0,0,0, 0,0,0,0, 0,0,0,0, 1,0,24,0 };
for(uint i=0; i<4; i++) {
header[ 2+i] = (char)((filesize>>(8*i))&255);
header[18+i] = (char)((width >>(8*i))&255);
header[22+i] = (char)((height >>(8*i))&255);
}
char* img = new char[filesize];
for(uint i=0; i<54; i++) img[i] = header[i];
for(uint y=0; y<height; y++) {
for(uint x=0; x<width; x++) {
const int color = data[x+(height-1-y)*width];
const int i = 54+3*x+y*(3*width+pad);
img[i ] = (char)( color &255);
img[i+1] = (char)((color>> 8)&255);
img[i+2] = (char)((color>>16)&255);
}
for(uint p=0; p<pad; p++) img[54+(3*width+p)+y*(3*width+pad)] = 0;
}
ofstream file(path, ios::out|ios::binary);
file.write(img, filesize);
file.close();
delete[] img;
}
The code snippet was inspired by https://stackoverflow.com/a/47785639/9178992
For .png images, use lodepng.cpp and lodepng.h:
#include <string>
#include <vector>
#include <fstream>
#include "lodepng.h"
using namespace std;
typedef unsigned int uint;
int* read_png(const string path, uint& width, uint& height) {
vector<uchar> img;
lodepng::decode(img, width, height, path, LCT_RGB);
int* data = new int[width*height];
for(uint i=0; i<width*height; i++) {
data[i] = img[3*i]<<16|img[3*i+1]<<8|img[3*i+2];
}
return data;
}
void write_png(const string path, const uint width, const uint height, const int* const data) {
uchar* img = new uchar[3*width*height];
for(uint i=0; i<width*height; i++) {
const int color = data[i];
img[3*i ] = (color>>16)&255;
img[3*i+1] = (color>> 8)&255;
img[3*i+2] = color &255;
}
lodepng::encode(path, img, width, height, LCT_RGB);
delete[] img;
}
I'm facing a problem while trying to convert a colored image to grey scale image using C++. I think there's a problem in the use of function GetRGB().
Here is the source code and related header file.
void CCorner::RGBToGrayScale(CImage* pIn, CImage* pOut)
{
//
// INPUT:
// CImage* pIn: The input image with 24bit depth
//
// OUTPUT:
// CImage* pOut: The output image. It has ALREADY been initialized
// with the same dimension as the input image (pIN) and
// formatted to 8bit depth (256 gray levels).
//
int height = pIn->GetHeight();
int width = pIn->GetWidth();
int clrOriginal=pIn->GetColorType();
if (height && width){
for(int i=0;i<height;i++){
for(int j=0;j<width;j++){
byte *r,*g,*b;
r=NULL; g=NULL; b=NULL;
bool colors=pIn->GetRGB(i,j,r,g,b);
double newindex = 0.299**r+0.587**g+0.114**b;
pOut->SetIndex(i,j,newindex);
}
}
}
}
While GetRGB() is defined as
virtual BOOL GetRGB(int x, int y, byte* r, byte* g, byte* b)
{ return implementation->GetRGB(x, y, r, g, b); }
Thanks for helping!
GetRGB is expecting you to provide pointers to actual bytes where it will output the results for r, g and b. This is a way of returning multiple results from a function in C++, by using output parameters.
However, you're giving it pointers to NULL here:
byte *r,*g,*b;
r=NULL; g=NULL; b=NULL;
So there's nowhere to write the results. When you dereference them later (via *r, *g, *b) you'll get undefined behaviour because they're null.
What you should be doing, is this:
byte r, g, b;
bool colors = pIn->GetRGB(i, j, &r, &g, &b);
And naturally, no need to deference them later because they're not pointers:
double newindex = 0.299*r+0.587*g+0.114*b;
You are passing null pointers to a function that expects to write to those memory addresses. Give it valid memory addresses where it can store the result for you.
byte r = 0, g = 0, b = 0;
bool colors = pIn->GetRGB(i, j, &r, &g, &b);
Supposing I am given an image of 2048x2048 and i want to know the total number of colors present in the image, what is the fastest possible algorithm? I came up with two algorithm but they are slow.
Algorithm 1:
Compare the current pixel an the next pixel and if they are different
Check a temporary variable, which contains all the detected colors, to see if the color is present or not
If not present add it to the array(List) and increment noOfColors.
This Algorithm works but is slow. For a 1600x1200 pixels image it takes around 3 sec.
Algorithm 2:
The obvious method of checking the each pixel with all other pixels and recording the no of occurences of the color and incrementing the count. This is very very slow, almost like a hung app. So is there any better approach? I need all the pixel info.
You could use std::set (or std::unordered_set), and simply do a single loop though the pixels, adding the colors to the set. Then the number of colors is the size of the set.
Well, this is suited for parallelization. Split the image in several parts and execute the algorithm for each part in a separate task. To avoid syncing each should have its own storage for the unique colors. When all tasks are done, you aggregate the results.
DRAM is dirt cheap. Use brute force. Fill a tab, count.
On a core2duo # 3.0GHz :
0.35secs for 4096x4096 32 bits rgb
0.20secs after some trivial parallelization (I do know nothing of omp)
However, if you are to use 64bit rgb (one channel = 16 bits) it is another question (not enough memory).
You shall probably need a good hash table function.
Using random pixels, same size takes 10 secs.
Remark: at 0.15 secs, the std::bitset<> solution is faster (it gets slower trivially parallelized !).
Solution, c++11
#include <vector>
#include <random>
#include <iostream>
#include <boost/chrono.hpp>
#define _16M 256*256*256
typedef union {
struct { unsigned char r,g,b,n ; } r_g_b_n ;
unsigned char rgb[4] ;
unsigned i_rgb;
} RGB ;
RGB make_RGB(unsigned char r, unsigned char g , unsigned char b) {
RGB res;
res.r_g_b_n.r = r;
res.r_g_b_n.g = g;
res.r_g_b_n.b = b;
res.r_g_b_n.n = 0;
return res;
}
static_assert(sizeof(RGB)==4,"bad RGB size not 4");
static_assert(sizeof(unsigned)==4,"bad i_RGB size not 4");
struct Image
{
Image (unsigned M, unsigned N) : M_(M) , N_(N) , v_(M*N) {}
const RGB* tab() const {return & v_[0] ; }
RGB* tab() {return & v_[0] ; }
unsigned M_ , N_;
std::vector<RGB> v_;
};
void FillRandom(Image & im) {
std::uniform_int_distribution<unsigned> rnd(0,_16M-1);
std::mt19937 rng;
const int N = im.M_ * im.N_;
RGB* tab = im.tab();
for (int i=0; i<N; i++) {
unsigned r = rnd(rng) ;
*tab++ = make_RGB( (r & 0xFF) , (r>>8 & 0xFF), (r>>16 & 0xFF) ) ;
}
}
size_t Count(const Image & im) {
const int N = im.M_ * im.N_;
std::vector<char> count(_16M,0);
const RGB* tab = im.tab();
#pragma omp parallel
{
#pragma omp for
for (int i=0; i<N; i++) {
count[ tab->i_rgb ] = 1 ;
tab++;
}
}
size_t nColors = 0 ;
#pragma omp parallel
{
#pragma omp for
for (int i = 0 ; i<_16M; i++) nColors += count[i];
}
return nColors;
}
int main() {
Image im(4096,4096);
FillRandom(im);
typedef boost::chrono::high_resolution_clock hrc;
auto start = hrc::now();
std::cout << " # colors " << Count(im) << std::endl ;
boost::chrono::duration<double> sec = hrc::now() - start;
std::cout << " took " << sec.count() << " seconds\n";
return 0;
}
The only feasible algorithm here is building a sort of a histogram of the image colors. The only difference in your case is that instead of calculating the population of each color you need just to know if it's zero or not.
Depending on which color space you work, you may use either an std::set to tag existing colors (as Joachim Pileborg suggested), or just use something like std::bitset, which is obviously faster. This depends on how much distinct colors exist in your color-space.
Also, like Marius Bancila noted, this procedure is a perfect match for parallelization. Calculated the histogram-like data for image parts, and then merge it. Naturally the image division should be based on its memory partition, not the geometric properties. In simple words - split the image vertically (by batches of scan lines), not horizontally.
And, if possible, you should either use some low-level library/code to run through pixels, or try to write your own. At least you must obtain a pointer to scan line and run on its pixels in a batch, rather than doing something like GetPixel for each pixel.
The point, here, is that the ideal representation of an image as 2D array of colors is not the one that happens the way the image is stored on memory (color components can be arranged in "planes", there could be "padding" etc. So getting the pixels using a GetPixel-like function may take time.
The question, then, may even be somehow meaningless if the image is not the result of a "vectorial draw": think to a photograph: between two nearby "greens" you find all the shade of green, so the colors -in this case- are no more no less the ones supported by the encoding of the image itself (2^24, or 256, or 16 or ...), so, unless you are interested on the color distribution (how differently used they are), just counting them makes very few sense.
A workaround can be:
Create an in-memory bitmap having pixel in a "single plane format"
Blit your image into that bitmap using BitBlt or similar (this let the OS to make pixel
conversion from the GPU,if any)
Get the bitmap-bits (this lets you
access the stored values)
Play your "counting algorithm" (whatever
it is) onto those values.
Note that step 1 and 2 can be avoided if you already know that the image is already in planar format.
If you have a multicore system, step 4 can also be assigned to different threads, each working part of the image.
You can use bitset which allows you to set individual bits and has a count function.
You have a bit for each colour, there are 256 values for each of RGB, so that's 256*256*256 bits (16,777,216 colours). The bitset will use a byte for every 8 bits so it will use 2MB.
Use the pixel colour as an index into the bitset:
bitset<256*256*256> colours;
for(int pixel: pixels) {
colours[pixel] = true;
}
colours.count();
This has linear complexity.
Late comer to this answer, but could not help it since this algorithm is brutally fast, developed about 2 or more decades ago, when it really mattered.
3-D Lookup Table Color Matching
http://www.ddj.com/cpp/184403257
Basically, it creates a 3d color loop up table and the search is very fast, I've done some modifications to suit my purpose for image binarization, so I reduced the color space from ff ff ff to f f f, and it's even 10 times faster. As it is right out of the box, I haven't found anything even close, including hash tables.
char * creatematcharray(struct rgb_color *palette, int palettesize)
{
int rval=16, gval=16, bval=16, len, r, g, b;
char *taken, *match, *same;
int i, set, sqstep, tp, maxtp, *entryr, *entryg, *entryb;
char *table;
len=rval*gval*bval;
// Prepare table buffers:
size_t size_of_table = len*sizeof(char);
table=(char *)malloc(size_of_table);
if (table==nullptr) return nullptr;
// Select colors to use for fill:
set=0;
size_t size_of_taken = (palettesize * sizeof(int) * 3) +
(palettesize*sizeof(char)) + (len * sizeof(char));
taken=(char *)malloc(size_of_taken);
same=taken + (len * sizeof(char));
entryr=(int*)(same + (palettesize * sizeof(char)));
entryg=entryr + palettesize;
entryb=entryg + palettesize;
if (taken==nullptr)
{
free((void *)table);
return nullptr;
}
std::memset((void *)taken, 0, len * sizeof(char));
// std::cout << "sizes: " << size_of_table << " " << size_of_taken << std::endl;
match=table;
for (i=0; i<palettesize; i++)
{
same[i]=0;
// Compute 3d-table coordinates of palette rgb color:
r=palette[i].r&0x0f, g=palette[i].g&0x0f, b=palette[i].b&0x0f;
// Put color in position:
if (taken[b*rval*gval+g*rval+r]==0) set++;
else same[match[b*rval*gval+g*rval+r]]=1;
match[b*rval*gval+g*rval+r]=i;
taken[b*rval*gval+g*rval+r]=1;
entryr[i]=r; entryg[i]=g; entryb[i]=b;
}
// ### Fill match_array by steps: ###
for (set=len-set, sqstep=1; set>0; sqstep++)
{
for (i=0; i<palettesize && set>0; i++)
if (same[i]==0)
{
// Fill all six sides of incremented cube (by pairs, 3 loops):
for (b=entryb[i]-sqstep; b<=entryb[i]+sqstep; b+=sqstep*2)
if (b>=0 && b<bval)
for (r=entryr[i]-sqstep; r<=entryr[i]+sqstep; r++)
if (r>=0 && r<rval)
{ // Draw one 3d line:
tp=b*rval*gval+(entryg[i]-sqstep)*rval+r;
maxtp=b*rval*gval+(entryg[i]+sqstep)*rval+r;
if (tp<b*rval*gval+0*rval+r)
tp=b*rval*gval+0*rval+r;
if (maxtp>b*rval*gval+(gval-1)*rval+r)
maxtp=b*rval*gval+(gval-1)*rval+r;
for (; tp<=maxtp; tp+=rval)
if (!taken[tp])
taken[tp]=1, match[tp]=i, set--;
}
for (g=entryg[i]-sqstep; g<=entryg[i]+sqstep; g+=sqstep*2)
if (g>=0 && g<gval)
for (b=entryb[i]-sqstep; b<=entryb[i]+sqstep; b++)
if (b>=0 && b<bval)
{ // Draw one 3d line:
tp=b*rval*gval+g*rval+(entryr[i]-sqstep);
maxtp=b*rval*gval+g*rval+(entryr[i]+sqstep);
if (tp<b*rval*gval+g*rval+0)
tp=b*rval*gval+g*rval+0;
if (maxtp>b*rval*gval+g*rval+(rval-1))
maxtp=b*rval*gval+g*rval+(rval-1);
for (; tp<=maxtp; tp++)
if (!taken[tp])
taken[tp]=1, match[tp]=i, set--;
}
for (r=entryr[i]-sqstep; r<=entryr[i]+sqstep; r+=sqstep*2)
if (r>=0 && r<rval)
for (g=entryg[i]-sqstep; g<=entryg[i]+sqstep; g++)
if (g>=0 && g<gval)
{ // Draw one 3d line:
tp=(entryb[i]-sqstep)*rval*gval+g*rval+r;
maxtp=(entryb[i]+sqstep)*rval*gval+g*rval+r;
if (tp<0*rval*gval+g*rval+r)
tp=0*rval*gval+g*rval+r;
if (maxtp>(bval-1)*rval*gval+g*rval+r)
maxtp=(bval-1)*rval*gval+g*rval+r;
for (; tp<=maxtp; tp+=rval*gval)
if (!taken[tp])
taken[tp]=1, match[tp]=i, set--;
}
}
}
free((void *)taken);`enter code here`
return table;
}
The answer: unordered_map
I use unordered_map, based on my testing.
You should test because your compiler / library may exhibit different performance Comment out #define USEHASH to use map instead.
On my machine, the vanilla unordered_map (a hash implementation) is about twice as fast as map. Inasmuch as different compilers, libraries can vary enormously, you must test to see which is better. In production, I build a fake image on first start of the app, run both algorithms on it and time them, save an indication of which one is faster, and then preferentially use that for all subsequent starts on that the machine. It's nit-picky, but hey, the user's time is valuable to them.
For a DSLR image with 12,106,244 pixels (about 12 megapixels, not a typo) and 11,857,131 distinct colors (also not a typo), map takes about 14 seconds, while unordered map takes about 7 seconds:
Test Code:
#define USEHASH 1
#ifdef USEHASH
#include <unordered_map>
#endif
size = im->xw * im->yw;
#ifdef USEHASH
// unordered_map is about twice as fast as map on my mac with qt5
// --------------------------------------------------------------
#include <unordered_map>
std::unordered_map<qint64, unsigned char> colors;
colors.reserve(size); // pre-allocate the hash space
#else
std::map<qint64, unsigned char> colors;
#endif
...use of either is in a loop where I build a 48-bit value of 0RGB in a 64-bit variable corresponding to the 16-bit RGB values of the image pixels, like so:
for (i=0; i<size; i++)
{
pel = BUILDPEL(i); // macro just shovels 0RGB into 64 bit pel from im
// You'd do the same for your image structure
// in whatever way is fastest for you
colors[pel] = 1;
}
cc = colors.size();
// time here: 14 secs for map, 7 secs for unordered_map with
// 12,106,244 pixels containing 11,857,131 colors on 12/24 core,
// 3 GHz, 64GB machine.