Ranking pixels by colour with OpenCV - c++

I begin a project about the detection.
My idea is to rank every pixels of an image (Mat).
Then, I will be able to exit which colour is dominant.
The difficulty is a colour is not unic. For exemple, Green is rgb(0, 255, 0) but is almost rgb(10, 240, 20) too.
The goal of my ranking is to exit pixels which are almost same colour. Then, with a pourcentage, I think I can locate my object.
So, my question: Is it a way to ranking pixels by colour ?
Thx a lot in advance for your answers.

There isn't a straight method of ranking as you say of pixels in colours.
However, you can find an approximation to the most dominant one.
There are several way in which you can do it:
You can calculate the histogram for each colour channel - split it into the R,G,B and compute the histogram. Then you can see where the peaks of the resulting graphs are - e.g.
If you k-means cluster the pixels at the image - in other words, represent each pixel as a 3D point with coordinated (R, G, B). Then you can segment the pixels into k most occurring colours.
If you resize the image to a 1x1 pixel image, you'll find the average of all pixel values. If there is a dominant colour, where the majority of the pixels are in close proximity, it will give a good approximation.
There however, are all approximations. Your best choice would be to use k-means and to find the cluster that either has the most elements, or is the most dense.
In case you are looking for way to locate an object with a specific colour, you can use a maximum likelihood estimation. Something like this, which was used to classify different objects, such as grass, cars, building and pavement from satellite images. You can use it with a single colour and get a heat-map of where the object is in terms of likelihood (the percentage of probability) of that pixel belonging to your object.

In an ordinary image, there's always a number of colors involved. To best average the pixels carrying almost the same colors is done by color quantization which is reducing number of colors in an image using techniques like K-mean clustering. This is best explained here with Python code:
https://www.pyimagesearch.com/2014/07/07/color-quantization-opencv-using-k-means-clustering/
After successful quantization, you can just try the following code to rank the colors based on their frequencies in the image.
top_n_colors = []
n = 3
colors_count = {}
(channel_b, channel_g, channel_r) = cv2.split(_processed_image)
# Flattens the 2D single channel array so as to make it easier to iterate over it
channel_b = channel_b.flatten()
channel_g = channel_g.flatten()
channel_r = channel_r.flatten()
for i in range(len(channel_b)):
RGB = str(channel_r[i]) + " " + str(channel_g[i]) + " " + str(channel_b[i])
if RGB in colors_count:
colors_count[RGB] += 1
else:
colors_count[RGB] = 1
# taking the top n colors from the dictionary objects
_top_colors = sorted(colors_count.items(), key=lambda x: x[1], reverse=True)[0:n]
for _color in _top_colors:
_rgb = tuple([int(value) for value in _color[0].split()])
top_n_colors.append(_rgb)
print(top_n_colors)

Related

Can I balance an extremely bright picture in python? This picture is a result of thousands of pictures stitched together to form a panorama

My aim is to stitch 1-2 thousand images together. I find the key points in all the images, then I find the matches between them. Next, I find the homography between the two images. I also take into account the current homography and all the previous homographies. Finally, I warp the images based on combined homography. (My code is written in python 2.7)
The issue I am facing is that when I overlay the warped images, they become extremely bright. The reason is that most of the area between two consecutive images is common/overalapping. So, when I overlay them, the intensities of the common areas increase by a factor of 2 and as more and more images are overalid the moew bright the values become and eventually I get a matrix where all the pixels have the value of 255.
Can I do something to adjust the brightness after every image I overlay?
I am combining/overlaying the images via open cv function named cv.addWeighted()
dst = cv.addWeighted( src1, alpha, src2, beta, gamma)
here, I am taking alpha and beta = 1
dst = cv.addWeighted( image1, 1, image2, 1, 0)
I also tried decreasing the value of alpha and beta but here a problem comes that, when around 100 images have been overlaid, the first ones start to vanish probably because the intensity of those images became zero after being multiplied by 0.5 at every iteration. The function looked as follows. Here, I also set the gamma as 5:
dst = cv.addWeighted( image1, 0.5, image2, 0.5, 5)
Can someone please help how can I solve the problem of images getting extremely bright (when aplha = beta = 1) or images vanishing after a certain point (when alpha and beta are both around 0.5).
This is the code where I am overlaying the images:
for i in range(0, len(allWarpedImages)):
for j in range(1, len(allWarpedImages[i])):
allWarpedImages[i][0] = cv2.addWeighted(allWarpedImages[i][0], 1, allWarpedImages[i][j], 1, 0)
images.append(allWarpedImages[i][0])
cv2.imwrite('/root/Desktop/thesis' + 'final.png', images[0])
When you stitch two images, the pixel values of overlapping part do not just add up. Ideally, two matching pixels should have the same value (a spot in the first image should also has the same value in the second image), so you simply keep one value.
In reality, two matching pixels may have slightly different pixel value, you may simply average them out. Better still, you adjust their exposure level to match each other before stitching.
For many images to be stitched together, you will need to adjust all of their exposure level to match. To equalize their exposure level is a rather big topic, please read about "histogram equalization" if you are not familiar with it yet.
Also, it is very possible that there is high contrast across that many images, so you may need to make your stitched image an HDR (high dynamic range) image, to prevent pixel value overflow/underflow.

Pixel-indexing in OpenCV's distance transform

I want to use the function distanceTransform() to find the minimum distance of non-zero pixels to zeros pixels, but also the position of that closest zero pixel. I call the second version of the function with the labelType flag set to DIST_LABEL_PIXEL. Everything works fine and I get the distances to and indices of the closest zero pixels.
Now I want to convert the indices back to pixel locations and I thought the indexing would be like idx=(row*cols+col) or something like this but I had to find out that OpenCV is just counting the zero pixels and using this count as the index. So if I get 123 as the index of the closest pixel this means that the 123th zero pixel is the closest.
How is OpenCV counting them? Probably in a row-wise manner?
Is there an efficient way of mapping the indices back to the locations? Obviously I could recount them and keep track of the counts and positions, if I know how OpenCV counts them, but this seems stupid and not very efficient.
Is there a good reason to use the indexing they used? I mean, are there any advantages over using an absolute indexing?
Thanks in advance.
EDIT:
If you want to see what I mean, you can run this:
Mat mask = Mat::ones(100, 100, CV_8U);
mask.at<uchar>(50, 50) = 0;
Mat dist, labels;
distanceTransform(mask, dist, labels, CV_DIST_L2, CV_DIST_MASK_PRECISE, DIST_LABEL_PIXEL);
cout << labels.at<int>(0,0) << endl;
You will see that all the labels are 1 because there is only one zero pixel, but how am I supposed to find the location (50,50) with that information?
The zero pixels also get labelled - they will have the same label as the non-zero pixels to which they are closest.
So you will have a 2D array of labels, the same size as your source image. If you examine all of the zero pixels in the source image, you can then find the associated label from the 2D array returned. This can then allow you to find which non-zero pixels are associated with each zero pixel by matching the labels.
If you see what I mean.
In python you can use numpy to associate the labels and the coordinates:
import cv2
import numpy as np
# create an image with two 0-lines
a = np.ones((100,100), dtype=np.uint8)
a[50,:] = 0
a[:,70] = 0
dt,lbl = cv2.distanceTransformWithLabels(a, cv2.DIST_L2, 3, labelType=cv2.DIST_LABEL_PIXEL)
# coordinates of 0-value pixels
xy = np.where(a==0)
# print label id and coordinate
for i in range(len(np.unique(lbl))):
print(i,xy[0][i], xy[1][i])

Smooth color transition algorithm

I am looking for a general algorithm to smoothly transition between two colors.
For example, this image is taken from Wikipedia and shows a transition from orange to blue.
When I try to do the same using my code (C++), first idea that came to mind is using the HSV color space, but the annoying in-between colors show-up.
What is the good way to achieve this ? Seems to be related to diminution of contrast or maybe use a different color space ?
I have done tons of these in the past. The smoothing can be performed many different ways, but the way they are probably doing here is a simple linear approach. This is to say that for each R, G, and B component, they simply figure out the "y = m*x + b" equation that connects the two points, and use that to figure out the components in between.
m[RED] = (ColorRight[RED] - ColorLeft[RED]) / PixelsWidthAttemptingToFillIn
m[GREEN] = (ColorRight[GREEN] - ColorLeft[GREEN]) / PixelsWidthAttemptingToFillIn
m[BLUE] = (ColorRight[BLUE] - ColorLeft[BLUE]) / PixelsWidthAttemptingToFillIn
b[RED] = ColorLeft[RED]
b[GREEN] = ColorLeft[GREEN]
b[BLUE] = ColorLeft[BLUE]
Any new color in between is now:
NewCol[pixelXFromLeft][RED] = m[RED] * pixelXFromLeft + ColorLeft[RED]
NewCol[pixelXFromLeft][GREEN] = m[GREEN] * pixelXFromLeft + ColorLeft[GREEN]
NewCol[pixelXFromLeft][BLUE] = m[BLUE] * pixelXFromLeft + ColorLeft[BLUE]
There are many mathematical ways to create a transition, what we really want to do is understand what transition you really want to see. If you want to see the exact transition from the above image, it is worth looking at the color values of that image. I wrote a program way back in time to look at such images and output there values graphically. Here is the output of my program for the above pseudocolor scale.
Based upon looking at the graph, it IS more complex than a linear as I stated above. The blue component looks mostly linear, the red could be emulated to linear, the green however looks to have a more rounded shape. We could perform mathematical analysis of the green to better understand its mathematical function, and use that instead. You may find that a linear interpolation with an increasing slope between 0 and ~70 pixels with a linear decreasing slope after pixel 70 is good enough.
If you look at the bottom of the screen, this program gives some statistical measures of each color component, such as min, max, and average, as well as how many pixels wide the image read was.
A simple linear interpolation of the R,G,B values will do it.
trumpetlicks has shown that the image you used is not a pure linear interpolation. But I think an interpolation gives you the effect you're looking for. Below I show an image with a linear interpolation on top and your original image on the bottom.
And here's the (Python) code that produced it:
for y in range(height/2):
for x in range(width):
p = x / float(width - 1)
r = int((1.0-p) * r1 + p * r2 + 0.5)
g = int((1.0-p) * g1 + p * g2 + 0.5)
b = int((1.0-p) * b1 + p * b2 + 0.5)
pix[x,y] = (r,g,b)
The HSV color space is not a very good color space to use for smooth transitions. This is because the h value, hue, is just used to arbitrarily define different colors around the 'color wheel'. That means if you go between two colors far apart on the wheel, you'll have to dip through a bunch of other colors. Not smooth at all.
It would make a lot more sense to use RGB (or CMYK). These 'component' color spaces are better defined to make smooth transitions because they represent how much of each 'component' a color needs.
A linear transition (see #trumpetlicks answer) for each component value, R, G and B should look 'pretty good'. Anything more than 'pretty good' is going to require an actual human to tweak the values because there are differences and asymmetries to how our eyes perceive color values in different color groups that aren't represented in either RBG or CMYK (or any standard).
The wikipedia image is using the algorithm that Photoshop uses. Unfortunately, that algorithm is not publicly available.
I've been researching into this to build an algorithm that takes a grayscale image as input and colorises it artificially according to a color palette:
■■■■ Grayscale input ■■■■ Output ■■■■■■■■■■■■■■■
Just like many of the other solutions, the algorithm uses linear interpolation to make the transition between colours. With your example, smooth_color_transition() should be invoked with the following arguments:
QImage input("gradient.jpg");
QVector<QColor> colors;
colors.push_back(QColor(242, 177, 103)); // orange
colors.push_back(QColor(124, 162, 248)); // blue-ish
QImage output = smooth_color_transition(input, colors);
output.save("output.jpg");
A comparison of the original image VS output from the algorithm can be seen below:
(output)
(original)
The visual artefacts that can be observed in the output are already present in the input (grayscale). The input image got these artefacts when it was resized to 189x51.
Here's another example that was created with a more complex color palette:
■■■■ Grayscale input ■■■■ Output ■■■■■■■■■■■■■■■
Seems to me like it would be easier to create the gradient using RGB values. You should first calculate the change in color for each value based on the width of the gradient. The following pseudocode would need to be done for R, G, and B values.
redDifference = (redValue2 - redValue1) / widthOfGradient
You can then render each pixel with these values like so:
for (int i = 0; i < widthOfGradient; i++) {
int r = round(redValue1 + i * redDifference)
// ...repeat for green and blue
drawLine(i, r, g, b)
}
I know you specified that you're using C++, but I created a JSFiddle demonstrating this working with your first gradient as an example: http://jsfiddle.net/eumf7/

Scanning and Detecting Object Color in Image

I'm developing a software that detects boxers punching motion. At the moment i used color based segmentation using inRange function and set it to detect blue Minimum value and Blue Maximum value. The problem is that the range is quite wide and my cam at times picks out noise and segments objects of no interest. To improve the software i though of scanning image of a boxing glove and establishing exact Blue color Value before further processing.
It would make sens to me to store that value in a Vector and call it in inRange fiction
// My current function which takes the Minimum and Maximum values of Blue Color
Mat range_out;
inRange(blur_out, Scalar(100, 100, 100), Scalar(120, 255, 255), range_out);
So i would image the vector to go somewhere here.
Scan this above image compute the Blue value
Store this value in an array
recall the array in a inRange function
Could someone suggest a solution to this problem or direct me to a source of information where I can look for answers ?
since you are detecting the boxer gloves in motion so first use motion to separate it from other elements in the scene...use frame differentiation or optical flow to separate the glove and other moving areas from non moving areas...now in those moving area try for some colour detection...
Separe luminosity and cromaticity - your fixed range will not work very well in different light conditions. Your range is wide probably because you are trying to see "blue" in dark and on light at the same time. Convert your image to HSV (or La*b*) and discard V (or L), keeping H and S (or a* and b*).
Learn a color distribution instead a simple range - take some samples and compute a 2D
color histogram on H and S (a* or b*) for pixels on the glove. This histogram will be a model for the color distribution of your object. Then, use c2.calcBackProjection to detect the pixels of interest in your scene.
Clean the result using morphological close operation
Important: on step 2, play a little with different quantization values (ie, different numbers of bins).

Color detection algorithm - How should I do this?

I'm a bit stuck on designing a color detection system - I can't quite figure out a way to do it easily.
-
Basically, I have a library of images, that I want to sort by color. So if the user specifies 'sort by blue', then the most blue images will appear at the top of the results, with the least blue appearing at the bottom.
The problem is that the images aren't all one color, so it is doing two things at the same time:
1 - finding the bluest part of the image
2 - ranking this blue color (based on color hue and amount of this color).
I've tried about 3 or 4 different approaches, with varying results - none work well though, and 2 of these were quite mathematical algorithms (which all work much better on paper than in practice haha).
-
What different ways could I go about the whole process? I'm probably missing some really obvious ways it could work - any help or ideas would be much appreciated :)
-
EDIT: Thanks for all the responses - here's what I've tried so far:
getting the average rgb value for the whole image and comparing it to blue. Comparing was done using normalised rgb 3 space vectors and finding distances between them. This works the least well, an image with no blue could easily appear above an image with partial very strong blue.
finding the dominant color and comparing it to blue (again using 3 space vector distances). This didn't work as there might have been a large blue section of the image that wasn't the most (or in the top couple) of dominant color sections.
finding pixels that are close to blue, averaging all of these and comparing the answer to actual blue.
finding all the pixels that are close to blue, incrementing a count and finding a percentage based on count/total pixels.
Two thoughts come to mind:
Cheap version: convert images to HSV color space, and for each pixel compute cos(H - target_hue) or a reasonable approximation (for blue, target_hue would be 240 degrees), multiply by saturation, and average that quantity over all of the pixels in the image. High values are best. Note that colors that are closer to yellow than to blue have "negative blueness", and that black, white, and pure gray have equally "zero blueness". Note that you really want HSV, not HSL, in this situation, because the "S" in HSL doesn't map well to perceptual saturation. For example, the color #f8f8ff (RGB 248, 248, 255) has a saturation of 100% in HSL (i.e. a pure blue), but it looks nearly white. The same color in HSV has an "S" coordinate of only 3%, which is reasonable.
Less cheap version: convert images to CIELAB color space, discard L, and compute the distance in a*b* space between each pixel and the target color, then average or RMS over each pixel. Low values are best.
I think to measure "blueness" you'll need to take all three components into account, not just the blue. Just for example, [255,255,255] is pure white, not blue -- but [0, 0, 30] is pure blue, even though its blue component is much lower in value.
Alternatively, you could convert to something like HSL or HSV, in which case the "blueness" should be a bit simpler to measure (hue and saturation only).
I'd google for an algorythm for creating 256 colour palettes from 24bit images (see http://en.wikipedia.org/wiki/Color_quantization for more info) then see which colours in this palette dominate if the image was mapped to it. ie, running a tally for each 256 palette entry of how many pixels get mapped into it.
notes,
you of course don't need the whole 256, it's just saying 256 to help explain my thinking.
also by directly studying the algorythim for this palette generation might directly give you an answer.
Do you really need to find the bluest part of the image? Why not just rank the "blueness" of an image as the average blue-component value for all pixels?
Another possibility would be to find the density of pixels that pass a threshold, or minimum blue value necessary to qualify as a blue pixel.
If you have one pixel, I'd say its blueness in terms of RGB is the the value of B / (R + G + B), so 1 is totally blue and 0 is not blue at all and white is 1/3 blue. (Watch out for black, which is a special case.) And the blueness of an image is the average blueness of its pixels. And if that's too costly, just take the average of a fixed number of randomly-chosen pixels.
I would say to take the average of the RGB value itself over the whole picture. I would say that the pseudo below should give you the "average blue" of the picture.
SUMr
SUMg
SUMb
for pixel <- image
SUMr += pixel.r
SUMg += pixel.g
SUMb += pixel.b
SUMr / pixelcount
SUMg / pixelcount
SUMb / pixelcount
If this doesn't work out; then I would think that you would need to rank a "blue" pixel as being higher/lower weighted based on the G/B values. Then add up your weighted value(s) and compare those.
weight
for pixel <- image
tweight = b
b -= r
b -= g
b = 0 if b < 0
weight += tweight
compare weights of all images.