Difference between two photos in tollerance variable - c++

Question

I have two photos:
and
I am getting differences between these photos. But these differences include changes of light, shaking of the camera, etc. I want to see only the man in the difference photo. I wrote a threshold value and I succeeded in it. But this threshold does not correct other photos. I can't show wrong examples because of my reputation in stackoverflow. You can run my code on other photos and you can see the disorders. My code is given below. How else can I do this threshold?
#include <Windows.h>
#include <opencv\highgui.h>
#include <iostream>
#include <opencv2\opencv.hpp>
using namespace cv;
using namespace std;
int main() {
Mat siyah;
Mat resim = imread("C:/Users/toshiba/Desktop/z.jpg", CV_LOAD_IMAGE_GRAYSCALE);
Mat resim2 = imread("C:/Users/toshiba/Desktop/t.jpg", CV_LOAD_IMAGE_GRAYSCALE);
if (resim.empty() || resim2.empty())
{
cout << "Dosya Açılamadı " << "\n";
return 0;
}
for (int i = 0; i < resim.rows; i++)
{
for (int j = 0; j <resim.cols; j++)
{
if (resim.data[resim.channels()*(resim.cols*(i)+
(j))] - resim2.data[resim2.channels()*(resim2.cols*(i)+
(j))]>30) {
resim.data[resim.channels()*(resim.cols*(i)+
(j))] = 255;
}
else
resim.data[resim.channels()*(resim.cols*(i)+
(j))] = 0;
//inRange(resim, 150, 255, siyah);
}
}
//inRange(resim, 150, 255, siyah);
namedWindow("Resim", CV_WINDOW_NORMAL);
imshow("Resim", resim);
waitKey();
system("PAUSE");
waitKey();
return 0;
}

Answer 1

If your background is always the same and the pictures which include an object occur rarely enough then you can update your reference image very often such that the changes in lighting from your reference image to the image to be analyzed are always small. You could then measure/compute a threshold that most of the time will work when computing the image difference. I am not sure why your camera is moving - is it not fixed ?

Answer 2

I made the following code with Otsu thresholding and GrabCut algorithm. It doesn't use any pre-set threshold values, but I am still not sure how well it will perform for other images (maybe if you provide several more pictures to test with). The code is in Python, but it mostly consists of calling OpenCV functions and filling matrices, so should be easy to convert to C++ or whatever. The result for your image:
Using just Otsu alone on the difference gives the following mask:
The legs are fine but the rest is messed up. But there seem to be no false positives, so I took the mask as definite foreground, everything else as probable background and fed it to GrabCut.
import cv2
import numpy as np
#read the empty background image and the image with the guy in it,
#convert them into float32, so we don't get integers overflow
img_empty = cv2.imread("000_empty.png", 0).astype(np.float32)
img_guy = cv2.imread("001_guy.jpg", 0).astype(np.float32)
#absolute difference -> back to uint8 for thresholding etc.
diff = np.abs(img_empty - img_guy).astype(np.uint8)
#otsu thresholding
ret2, th = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
#read our image again for GrabCut
img = cv2.imread("001_guy.jpg")
#fill GrabCut mask
mask = np.zeros(th.shape, np.uint8)
mask[th == 255] = cv2.GC_FGD #the value is GC_FGD (foreground) when our thresholded value is 255
mask[th == 0] = cv2.GC_PR_BGD #GC_PR_BGD (probable background) otherwise
#some internal stuff for GrabCut...
bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)
#run GrabCut
cv2.grabCut(img, mask, (0, 0, 1365, 767), bgdModel, fgdModel, 5, cv2.GC_INIT_WITH_MASK)
#convert the `mask` we got from GrabCut into a binary mask,
#then apply it to the original image
mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')
img = img*mask2[:,:,np.newaxis]
#save the results
cv2.imwrite("003_grabcut.jpg", img)