OpenCV: Calculating new red pixel value - c++

Question

I'm currently aiming to adjust the red pixels in an image (more specifically, an eye region to remove red eyes caused by flash), and this works well, but the issue I'm getting is sometimes green patches appear on the skin.
This is a good result (before and after):
I realize why this is happening, but when I go to adjust the threshold to a higher a value (meaning the red intensity must be stronger), less red pixels are picked up and changed, i.e.:
The lower the threshold, the more green shows up on the skin.
I was wondering if there was an alternate method to what I'm currently doing to change the red pixels?
int lcount = 0;
for(int y=0;y<lcroppedEye.rows;y++)
{
for(int x=0;x<lcroppedEye.cols;x++)
{
double b = lcroppedEye.at<cv::Vec3b>(y, x)[0];
double g = lcroppedEye.at<cv::Vec3b>(y, x)[1];
double r = lcroppedEye.at<cv::Vec3b>(y, x)[2];
double redIntensity = r / ((g + b) / 2);
//currently causes issues with non-red-eye images
if (redIntensity >= 1.8)
{
double newRedValue = (g + b) / 2;
cv::Vec3b pixelColor(newRedValue,g,b);
lroi.at<cv::Vec3b>(cv::Point(x,y)) = pixelColor;
lcount++;
}
}
}
EDIT: I can possibly add in a check to ensure the new RGB values are low enough, and so R, G, B values are similar/close values so black/grey pixels are written out only... or have a range of RGB values (greenish) which aren't allowed... would that work?

Answer 1

Adjusting color in RGB space has caveats like this greenish areas you faced. Convert the R,G,B values to a better color space, like HSV or LUV.
I suggest you go for HSV to detect and change the red-eye colors. R/(G+B) is not a good way for calculating red intensity. This means you are calling (R=10,G=1,B=0) a very red color, but it is deadly black. Take a look at the comparison below:
So, you'd better check if Saturation and Value are high values which is the case for a red-eye color. If you encounter other high intensity colors, you may check the Hue is in the range of something like [0-20] and [340-359]. But without this, you are still safe against the white itself, as it has a very low saturation and you won't select white areas anyway.
That was for selecting, for changing the color, it is again better to not use RGB, as changing in that space is not linear as we perceive colors. Looking at the image above, you can see that lowering both the saturation and value would be a good start. But you may experiment with it and see what looks better. Maybe you'll be fine with a dark gray always, that would mean set Saturation to zero, and lower the Value a bit. You may think a dark brown would be better, go for a low saturation and value but set Hue to something about 30 degrees.
References that may help you:
Converting color values in OpenCV
An online tool to experiment with RGB and HSV colors

Answer 2

It may be better to change
double redIntensity = r / ((g + b) / 2);
to
double redIntensity = r / ((g+b+1) / 2);
because g+b can be equal to 0, and you'll get NAN.
Also take alook at cv::floodfill method.

Answer 3

May be it is better to ignore color information at red zones at all, as soon as color information in extra red area is too much distorted by extra red values. So new values could be:
newRedValue = (g+b)/2; newGreenValue = newRedValue; newBlueValue = newRedValue;
Even if you will detect wrong red area its desaturating will give better result than greenish area.
You can also use morphological closing operations (using circle structuring element) to avoid gaps in your red area mask. So you will need perform 3 steps: 1. find red areas and create mask for this 2. do red area mask morphological closing operations 3. desaturate image using this mask
And yes, don't use "r /((g+b)/2)" as it can lead to division by zero error.

Answer 4

Prepare a mask the same size as your lcroppedEye image, which is initially all black (I'll call this image maskImage here onwards).
For every pixel in lcroppedEye(row, col) that pass your (redIntensity >= 1.8) condition, set the maskImage(row, col) pixel to white.
When you are done with all the pixels in lcroppedEye, maskImage will have all redeye-like pixels in white.
If you perform a connected component analysis on this maskImage, you should be able to filter out other regions by considering circle or disk-like-features etc.
Now you can use this maskImage as a mask to apply the color transformation to the ROI of the original image
(You may have to do some preprocessing on maskImage before moving on to connected component analysis. Also you can replace the code segment in the question with split, divide and threshold functions unless there's special reason to iterate through pixels)

Answer 5

The problem seems to be that you replace regardless of the presence of any red eye, so you must somehow test if there is any high red values (more red than your skin).
My guess it that the areas where there is reflection there will also be specific blue and green values, either high or low that should be check so that you for example need high red values combined with low blue and/or low green values.
// first pass, getting the highest red value
int highRed = 0;
cv::Point redPos = cv::Point(0,0);
int lcount = 0;
for(int y=0;y<lcroppedEye.rows;y++)
{
for(int x=0;x<lcroppedEye.cols;x++)
{
double r = lcroppedEye.at<cv::Vec3b>(y, x)[2];
if (redIntensity > highRed)
{
highRed = redIntensity ;
redPos = cv::Point(x,y);
}
}
}
// decide if its red enough, need to find a good minRed value.
if (highRed < minRed)
return;
Original code here with the following changes.
// avoid division by zero, code from #AndreySmorodov
double redIntensity = r / ((g+b+1) / 2);
// add check for actual red colour.
if (redIntensity >= 1.8 && r > highRed*0.75)
// potential add check for low absolute r/b values.
{
double newRedValue = (g + b) / 2;
cv::Vec3b pixelColor(newRedValue,g,b);
lroi.at<cv::Vec3b>(cv::Point(x,y)) = pixelColor;
lcount++;
}
}