My question is the final paragraph.
I am trying to use one of OpenCV's background subtractors as a means of detecting human hands. The code that tries to do this is as follows:
cv::Ptr<cv::BackgroundSubtractor> pMOG2 = cv::createBackgroundSubtractorMOG2();
cv::Mat fgMaskMOG2;
pMOG2->apply(input, fgMaskMOG2, -1);
cv::namedWindow("FG Mask MOG 2");
cv::imshow("FG Mask MOG 2", fgMaskMOG2);
When I initially ran the program on my own test video I was greeted with this (ignore the name of the right most window):
As you can see a mask is not detected for my moving hand at all, given that the background in my video is completely stationary (there were maybe one or two white pixels at a time showing up in the mask). So I tried using a different video, one that many examples seemed to use which was moving traffic.
You can see it picked up on a moving car -very- slightly. I have tried (for both these videos) setting the "learning threshold" for the apply method to many values between 0 and 1 and there was not much variation at all from the results you can see above.
Have I missed anything with regards to setting up the background subtraction or are the videos particularly hard examples to deal with? Where can I adjust the settings of the background subtraction to favour my setup (if anywhere)? I will repeat the fact that in both videos the camera is stationary.
My answer is in python but convert and try it. Approve if it works.
if (cap.isOpened() == False):
print("Error opening video stream or file")
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
min_thresh=800
max_thresh=10000
fgbg = cv2.createBackgroundSubtractorMOG2()
connectivity = 4
# Read until video is completed
while (cap.isOpened()):
# Capture frame-by-frame
ret, frame = cap.read()
if ret == True:
print("Frame detected")
frame1 = frame.copy()
fgmask = fgbg.apply(frame1)
fgmask = cv2.morphologyEx(fgmask, cv2.MORPH_OPEN, kernel)
output = cv2.connectedComponentsWithStats(
fgmask, connectivity, cv2.CV_32S)
for i in range(output[0]):
if output[2][i][4] >= min_thresh and output[2][i][4] <= max_thresh:
cv2.rectangle(frame, (output[2][i][0], output[2][i][1]), (
output[2][i][0] + output[2][i][2], output[2][i][1] + output[2][i][3]), (0, 255, 0), 2)
cv2.imshow('detection', frame)
cv2.imshow('detection', fgmask)
Update cv2.createBackgroundSubtractorMOG2 by changing history, varThreshold, and detectShadows=True. You can also change kernel sizel, remove noise etc.
Try using MOG subtractor instead of MOG2 background subtractor.. It might help you.
Because most times MOG subtractor would be handy. But the worst thing is MOG subtractor has been moved to bgsegm package. It's a contrib package. It is available in OpenCv git hub page itself.
https://github.com/Itseez/opencv_contrib
Related
Above is the output I've got from my code, however there is a significant amount of shadows in the image, is there any ways that I can do to remove shadows? And also add object tracking that create box for moving car? Thank you so much
//create Background Subtractor objects
Ptr < BackgroundSubtractor > pBackSub;
if (parser.get <String>("algo") == "MOG2")
pBackSub = createBackgroundSubtractorMOG2();
VideoCapture capture(parser.get <String>("input")); //input video
Mat frame, fgMask;
while (true) {
capture >> frame;
if (frame.empty()) //break if frame empty
break;
//update the background model
pBackSub - > apply(frame, fgMask);
//erode the frame with 3x3 kernel
Mat frame_eroded_with_3x3_kernel;
erode(fgMask, frame_eroded_with_3x3_kernel, getStructuringElement(MORPH_RECT, Size(3, 3)));
//dilate the frame with 2x2 kernel
Mat frame_dilate_with_2x2_kernel;
dilate(frame_eroded_with_3x3_kernel, frame_dilate_with_2x2_kernel, getStructuringElement(MORPH_RECT, Size(2, 2)));
//show the current frame and the fg mask
imshow("Frame", frame);
imshow("FG Mask", fgMask);
imshow("After eroded with 3x3 kernel", frame_eroded_with_3x3_kernel);
imshow("After dilate with 2x2 kernel", frame_dilate_with_2x2_kernel);
//get the input from the keyboard
int keyboard = waitKey(30);
if (keyboard == 'q' || keyboard == 27)
break;
}
return 0;
}
It is possible that your output is correct. The first do not use moving camera video. Scene needs to be stable as well with good light conditions. You can try different parameters of MOG2 setting. History influence how previous frames influence the current. varThreshold can significantly help you. detectShadows=false is better, You can try false and true to see the difference. You can remove detected shadow, but the methods have the limitations.
cv::createBackgroundSubtractorMOG2 (int history=500, double varThreshold=16, bool detectShadows=true)
You can enhance the output by using additional filtering and morphological operations for example in case of noise are useful. Search for the information about the following two functions and try them to apply.
cv::dilate
cv::erode
The point is simple. Do not expect the miracle. This is not suitable for many task in computer vision.
The detection and other task are not based on background subtraction in most of the application. In the following image the background subtraction failing due to the car lights changing conditions and shadows.
The detection is based on features that represent car instead of detect what is not a background. This is better way for most of the application. Haar, LBP detection or deeplearning. You can find many tutorials for detection on my page funvision
I think the erosion function in opencv should be able to solve the issue. This function uses a rectangular structuring element of size 3x3 to remove the white dots . I think the size of the element can be given as parameters.
Use fgMask as erosion input.
hey guys i am using opencv 2.4 with python 2.7 on ubuntu14.04
I want to select multiple Region of Interest in an image is it possible to do so.
I want to do motion detection in only the area i have selected to do so any of the following theory can solve my problem but don't know how to implement any of them : -
Mask the area in image which is not ROI
After creating multiple ROI image how to add them such that all those ROI can be on the original location and remaining area be masked
Yes it is possible to do so. Main Idea behind the solution would be creating a mask and setting it to 0 wherever you do not want the motion tracker to track.
If you are using numpythen you can create the mask and set the regions you do not want the detector to use, to zero. (Similar to cv::Rect(start.col, start.row, numberof.cols, numberof.rows) = 0 in c++)
In python using numpy you can create a mask, somewhat like this:
import numpy as np
ret, frame = cap.read()
if frame.ndim == 3
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
elif frame.ndim == 4
gray = cv2.cvtColor(frame, cv2.COLOR_BGRA2GRAY)
else:
gray = frame
# create mask
mask = np.ones_like(gray)
mask[start_row:end_row, start_col:end_col] = 0
mask[another_starting_row:another_ending_row, another_start_col:another_end_col] = 0
# and so on you can create your own mask
# use for loops to create specific masks
It is a bit crude solution but will do the job. check numpy documentation (PDF) for more info.
I am working on object tracking using background subtraction in opencv. I have taken a sample soccer video and my goal is to track the players and filter out the bigger field markings. Due to non-static camera, the big lines are also detected as moving as in this image:
I made use of the Hough Transform to detect lines and after setting appropriate thresholds, was able to filter the half-way line and the image appeared as this:
Now I am concerned about filtering these 2 arcs.
Question 1. What are the ways I can possibly do this? How can I make use of the difference in "properties" the arc(long and thin) and a player(a compact blob) have?
Moreover, the Hough transform function sometimes reports many false positives (Detecting a tall thin player as a straight line or even connecting 2 players to show a longer line).
Question 2. In what way to specify the maximum thickness of the "to be detected" line and to maintain strict standards to detect lines "only"?
Thanks.
I had an old script lying around for a similar function. Unfortunately, it's Python and doesn't use the Hough transform function. Still, you may find it useful.
get_blobs is the important function while __main__ is example usage.
import cv2
def get_blobs(thresh, maxblobs, maxmu03, iterations=1):
"""
Return a 2-tuple list of the locations of large white blobs.
`thresh` is a black and white threshold image.
No more than `maxblobs` will be returned.
Moments with a mu03 larger than `maxmu03` are ignored.
Before sampling for blobs, the image will be eroded `iterations` times.
"""
# Kernel specifies an erosion on direct pixel neighbours.
kernel = cv2.getStructuringElement(cv2.MORPH_CROSS, (3, 3))
# Remove noise and thin lines by eroding/dilating blobs.
thresh = cv2.erode(thresh, kernel, iterations=iterations)
thresh = cv2.dilate(thresh, kernel, iterations=iterations-1)
# Calculate the centers of the contours.
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]
moments = map(cv2.moments, contours)
# Filter out the moments that are too tall.
moments = filter(lambda k: abs(k['mu03']) <= maxmu03, moments)
# Select the largest moments.
moments = sorted(moments, key=lambda k: k['m00'], reverse=True)[:maxblobs]
# Return the centers of the moments.
return [(m['m10'] / m['m00'], m['m01'] / m['m00']) for m in moments if m['m00'] != 0]
if __name__ == '__main__':
# Load an image and mark the 14 largest blobs.
image = cv2.imread('input.png')
bwImage = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
trackers = get_blobs(bwImage, 14, 50000, 3)
for tracker in trackers:
cv2.circle(image, tuple(int(x) for x in tracker), 3, (0, 0, 255), -1)
cv2.imwrite('output.png', image)
Starting from your first image:
The algorithm uses erosion to separate the blobs from the lines.
Moments are then used to filter out the tall and small blobs. Moments are also used to locate the center of each blob.
get_blobs returns a 2-tuple list of the locations of the players. You can see them painted on the last image.
As it stands, the script is really messy. Feel free to use it directly, but I posted it mainly to give you some ideas.
I am writing C++ code with OpenCV where I'm trying to detect a chessboard on an image (loaded from a .jpg file) to warp the perspective of the image. When the chessboard is found by findChessboardCorners(), the rest of my code is working perfectly. But sometimes the function does not detect the pattern, and this behavior seems to be random.
For example, there is one image that works on it's original resolution 2560x1920, but not if I scale it down with GIMP first to 800x600. However, another image seems to do the opposite: doesn't work in original resolution, but does work scaled down.
Here's the bit of my code that does the detection:
Mat grayimg = imread(argv[1], CV_LOAD_IMAGE_GRAYSCALE);
if (img.data == NULL) {
printf("Unable to read image");
return 0;
}
bool patternfound = findChessboardCorners(grayimg, patternsize, corners,
CALIB_CB_ADAPTIVE_THRESH + CALIB_CB_FAST_CHECK);
if (!patternfound) {
printf("Chessboard not found");
return 0;
}
Is there some kind of bug in opencv causing this behavior? Does anyone has any tips on how to pre-process your image, so the function will work more consistently?
I already tried playing around with the parameters CALIB_CB_ADAPTIVE_THRESH, CALIB_CB_NORMALIZE_IMAGE, CALIB_CB_FILTER_QUADS and CALIB_CB_FAST_CHECK. I'm also having the same results when I pass in a color image.
Thanks in advance
EDIT: I'm using OpenCV version 2.4.1
I had a very hard time getting findChessboardCorners to work until I added a white boarder around the chessboard.
I found that as hint somewhere in the more recent documenation.
Before adding the border, it would sometimes be impossible to recognize the keyboard, but with the white border it works every time.
Welcome to the joys of real-world computer vision :-)
You don't post any images, and findChessboardCorners is a bit too high-level to debug. I suggest to display (in octave, or matlab, or with more OpenCV code) the location of the detected corners on top of the image, to see if enough are detected. If none, try to run cvCornerHarris by itself on the image.
Sometimes the cause of the problem is the excessive graininess of the image: try to blur is just a little and see if it helps.
Actually, try to remove the CALIB_CB_FAST_CHECK option, and give it a try.
CALIB_CB_ADAPTIVE_THRESH + CALIB_CB_FAST_CHECK is not the same as CALIB_CB_ADAPTIVE_THRESH | CALIB_CB_FAST_CHECK, you should use | (binary or)
I want to do:
masked = image - mask
But I want to "displace" mask. That is, move it vertically and horizontally (as long as the intersection between it and image is not empty, this would be valid).
I have some hand-coded assembly (which uses MMX instructions) which does this, embedded in a C++ program, but it's unstable when doing vertical displacement, so I thought of using OpenCV instead. Would it be possible to do this calling only one OpenCV function?
Performance is critical; using OpenCV, time should be at least in the same order of magnitude as the assembly code.
EDIT: Here's an example
image (medium frame, see the contrast in the guy's skull):
mask (first frame, no contrast):
image - mask, without displacement. Notice how the contrast path is enhanced, but since the patient moved a little, we can see some skull contours which are visual noise for diagnostic purposes.
image - mask, mask displaced about 5 pixels down. To try and compensate for the noise introduced by the patient's movement, we "displace" the mask slightly so as to remove the contours and see the contrast path better (brightness and contrast were adjusted, that's why it looks a bit darker).
EDIT 2: About the algorithm, I managed to fix its issues. It doesn't crash anymore, but the downside is that it now processes all image pixels (it should only process those which need to be subtracted). Anyway, how to fix the old code is not my question; my question is, how do I do this processing using OpenCV? I'll post some profiling results later.
I know this is in Python, so not what you are after, but translating it to C++ should be very straight forward. It crops both images to matching sizes (required for nearly all operations), determined by the displacement between the images, and their relative sizes. This method should be quick, as cv.GetSubRect doesn't copy anything, so its just down to the cv.AbsDiff function (if you have an actual difference mask, you could use cv.Sub which should make it even quicker). Also this code will handle displacement in any direction and mask and image can be any size (mask can be larger than image). There must be an overlap for a specified displacement. The difference between images can be viewed alone, or the difference 'in-place'.
A nice diagram to illustrate whats going on. The first two squares are example image and mask. The next three squares show a horizontal displacement of the 'mask' of -30, 0, and 30 pixels, and the last one has a displacement of 20, 20.
import cv
image = cv.LoadImageM("image.png")
mask = cv.LoadImageM("mask.png")
image = cv.LoadImageM("image2.png")
mask = cv.LoadImageM("small_mask.png")
image_width, image_height = cv.GetSize(image)
mask_width, mask_height = cv.GetSize(mask)
#displacements here:
horiz_disp = 20
vert_disp = 20
image_horiz = mask_horiz = image_vert = mask_vert = 0
if vert_disp < 0:
mask_vert = abs(vert_disp)
sub_height = min(mask_height + vert_disp, image_height)
else:
sub_height = min(mask_height, image_height - vert_disp)
image_vert = vert_disp
if horiz_disp < 0:
mask_horiz = abs(horiz_disp)
sub_width = min(mask_width + horiz_disp, image_width)
else:
sub_width = min(mask_width, image_width - horiz_disp)
image_horiz = horiz_disp
#cv.GetSubRect returns a rectangular part of an image, without copying any data. - fast.
mask_sub = cv.GetSubRect(mask, (mask_horiz, mask_vert, sub_width, sub_height))
image_sub = cv.GetSubRect(image, (image_horiz, image_vert, sub_width, sub_height))
#Subtracts the mask overlap region from the image overlap region, puts it in image_sub
cv.AbsDiff(image_sub, mask_sub, image_sub)
# Shows diff only:
cv.ShowImage('image_sub', image_sub)
# Shows image with diff section
cv.ShowImage('image', image)
cv.WaitKey(0)