Current situation: I would like to detect rectangles (or squares) inside an image, where the contours of these rectangles are not solid consistent. Like a chessboard, where the outer contours have wholes.
Possible Solution: I am trying to implement an active contour algorithm, which should help me to detect the outside contour of the object. I know some points outside of the object, which could be used to shrink and fit the points as long as the object fits in it.
Search: I have found the cvSnakeImage Function of an older openCV version, which is not maintained and should not be used any more. I have found an active contour C++ implementation, which also uses an older openCV and the boost library. I have tried but was not able to build the code. HiDiYANG/ActiveContour
Post using cvSnake Implementation
Matlab porting to Opencv 3.0
Further articles in this topic: SNAKES: Active Contour Model
Question: Is there a current implementation of the active contour algorithm available in OpenCV? Is there a best implementation available, where I should invest time to understand the implementation?
Example Image:
I have the first image with the the points on the grey border and would like to get the red rectangle (second image).
For the image you have uplaoded, simple union over bounding boxes of contours should give you the result you desired. 'bb_union' is a function you need to write for yourself.
import cv2
img = cv2.imread('path to your image') # BGR image
im = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
im = 255 - im # your contours are black, so invert the image
_, contours, hierarchy = cv2.findContours(img, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
bb = None
for cnt in contours:
rect = cv2. boundingRect(cnt)
if (bb is None):
bb = rect
continue
bb = bb_union(rect, bb)
cv2.rectangle(img, bb, (0,0,255), 2)
Related
This is a picture of a corridor after masking.
I'm trying to follow the "free path" in an indoor environment using opencv, the way I'm trying to locate the free area or whitespace in the image is by traversing the whole array and checking the pixel values, but this seems too slow. I also tried using findContours and edge detection methods from opencv but the largest contour area is pointing at the far left corner of the white area. Any other way I can do this ?
import cv2
im = cv2.imread('YourImagePAth\\image.png')
gray=cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
contours, hierarchy =
cv2.findContours(gray,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)[-2:]
idx =0
for cnt in contours:
idx += 1
x,y,w,h = cv2.boundingRect(cnt)
roi=im[y:y+h,x:x+w]
cv2.rectangle(im,(x,y),(x+w,y+h),(0,0,255),2)
cv2.drawContours(im,contours,-1,(0,255,0),3)
cv2.imshow('img',im)
cv2.waitKey(0)
now if you want the white pixels area use : area = cv.contourArea(cnt)
and if you want the red rectangle area use : w*h (bounding box dimensions).
This is what you should get if you draw the bounding box using the code above :
I am using Python 2.7 and I used following Python and Matlab function for removing noises and fill holes in this image
.
1. Code to remove noise and fill holes using Python and Opencv
img = cv2.imread("binar.png",0)
kernel = np.ones((5,5),np.uint8)
open = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel)
close = cv2.morphologyEx(open, cv2.MORPH_CLOSE, kernel)
Code used in python and scipy using ndimage.binary_closing:
im = cv2.imread("binar.png", cv2.IMREAD_GRAYSCALE)
open_img = ndimage.binary_opening(im)
close_img = ndimage.binary_closing(open_img)
clg = close_img.astype(np.int)
Code used in Matlab: I used imfill and bwareaopen.
The results I got is shown below:
First image from using nd.image.binary_closing. My problem is it doesn't fill all white blobs fully. We can see inbetween minor black portion are still present.
Second image from using cv2.morphologyEx. Same problem in this also, as it also has some minor white portion in between white blobs. Here I faced one more problem. It converts some white pixels into black which should not be otherwise. I mentioned those areas with red color in image 2. Red highlighted portions is connected with larger one blobs but even then they get converted into black pixels.
Third image I got from MATLAB processing in which imfill work perfectly without converting essential white pixels into black.
So, my question is, Is there any method for Python 2.7 with which I can remove noises below certain area and fill the white blobs accurately as in Matlab? One more thing is, I want to find out the centroids and areas of those final processed blobs in last for further used. I can find out these using cv2.connectedComponentsWithStats but I want to find area and centroids after removing noises and filling blobs.
Thanks.
(I think this is not duplicate because I want to do it in Python not in Matlab. )
From Matlab's imfill() documentation:
BW2= imfill(BW,locations) performs a flood-fill operation on background pixels of the input binary image BW, starting from the points specified in locations. (...)
BW2= imfill(BW,'holes') fills holes in the input binary image BW. In this syntax, a hole is a set of background pixels that cannot be reached by filling in the background from the edge of the image.
I2= imfill(I) fills holes in the grayscale image I. In this syntax, a hole is defined as an area of dark pixels surrounded by lighter pixels.
The duplicate that I flagged shows ways to accomplish the third variant usually. However for many images, the second variant will still work fine and is extremely easy to accomplish. From the first variant you see that it mentions a flood-fill operation, which can be implemented in OpenCV with cv2.floodFill(). The second variant gives a really easy method---just flood fill from the edges, and the pixels left over are the black holes which can't be reached from outside. Then if you invert this image, you'll get white pixels for the holes, which you can add to your mask to fill in the holes.
import cv2
import numpy as np
# read image, ensure binary
img = cv2.imread('image.png', 0)
img[img!=0] = 255
# flood fill background to find inner holes
holes = img.copy()
cv2.floodFill(holes, None, (0, 0), 255)
# invert holes mask, bitwise or with img fill in holes
holes = cv2.bitwise_not(holes)
filled_holes = cv2.bitwise_or(img, holes)
cv2.imshow('', filled_holes)
cv2.waitKey()
Note that in this case, I just set the starting pixel for the background at (0,0). However it's possible that there could be, e.g., a white line going down the center which would cut off this operation to stop filling (i.e. stop finding the background) for the other half of the image. The more robust method would be to go through all of the edge pixels on the image, and flood fill every time you come across a black pixel. You can accomplish this more easily with the mask parameter in cv2.floodFill(), which allows you to continue to update the mask each time.
To find the centroids of each blob, you could use contour detection and cv2.moments() to find the centroids of each contour, or you could also do cv2.connectedComponentsWithStats() like you mentioned.
I am working on an image to find contours, I am successfully able to identify almost all contours in the below image. But I am not able to find certain contours like the two grey boxes on the left side. I have tried all the contour methods like Tree, List, External and also tried with thresholding.
Please suggest If I am missing something or can do to improve it
img = cv2.imread(input_image, 0)
kernel = np.ones((5, 5),np.uint8)
morphological_img = cv2.morphologyEx(img, cv2.MORPH_GRADIENT, kernel)
canny_img = cv2.Canny(morphological_img, 200, 300)
input_imag, contours, hierarchy =
cv2.findContours(canny_img,cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) # get contours
Your question does not show or link to the image that it refers to. But here are the steps to improve this:
Unless your image is already binary, the standard method is to apply thresholding first, and then use morphological operations.
If you are interested in only the contours, you can first do canny edge detection and then use morphological operations to enhance the contour that you want to detect.
If canny_img does not show the edge corresponding to the desired contour (try cv2.imshow), findContours want be of much use, if any.
If you can edit the answer to include the image, we will be able to provide more specific answers.
What I need
I'm currently working on an augmented reality kinda game. The controller that the game uses (I'm talking about the physical input device here) is a mono colored, rectangluar pice of paper. I have to detect the position, rotation and size of that rectangle in the capture stream of the camera. The detection should be invariant on scale and invariant on rotation along the X and Y axes.
The scale invariance is needed in case that the user moves the paper away or towards the camera. I don't need to know the distance of the rectangle so scale invariance translates to size invariance.
The rotation invariance is needed in case the user tilts the rectangle along its local X and / or Y axis. Such a rotation changes the shape of the paper from rectangle to trapezoid. In this case, the object oriented bounding box can be used to measure the size of the paper.
What I've done
At the beginning there is a calibration step. A window shows the camera feed and the user has to click on the rectangle. On click, the color of the pixel the mouse is pointing at is taken as reference color. The frames are converted into HSV color space to improve color distinguishing. I have 6 sliders that adjust the upper and lower thresholds for each channel. These thresholds are used to binarize the image (using opencv's inRange function).
After that I'm eroding and dilating the binary image to remove noise and unite nerby chunks (using opencv's erode and dilate functions).
The next step is finding contours (using opencv's findContours function) in the binary image. These contours are used to detect the smallest oriented rectangles (using opencv's minAreaRect function). As final result I'm using the rectangle with the largest area.
A short conclusion of the procedure:
Grab a frame
Convert that frame to HSV
Binarize it (using the color that the user selected and the thresholds from the sliders)
Apply morph ops (erode and dilate)
Find contours
Get the smallest oriented bouding box of each contour
Take the largest of those bounding boxes as result
As you may noticed, I don't make an advantage of the knowledge about the actual shape of the paper, simply because I don't know how to use this information properly.
I've also thought about using the tracking algorithms of opencv. But there were three reasons that prevented me from using them:
Scale invariance: as far as I read about some of the algorithms, some don't support different scales of the object.
Movement prediction: some algorithms use movement prediction for better performance, but the object I'm tracking moves completely random and therefore unpredictable.
Simplicity: I'm just looking for a mono colored rectangle in an image, nothing fancy like car or person tracking.
Here is a - relatively - good catch (binary image after erode and dilate)
and here is a bad one
The Question
How can I improve the detection in general and especially to be more resistant against lighting changes?
Update
Here are some raw images for testing.
Can't you just use thicker material?
Yes I can and I already do (unfortunately I can't access these pieces at the moment). However, the problem still remains. Even if I use material like cartboard. It isn't bent as easy as paper, but one can still bend it.
How do you get the size, rotation and position of the rectangle?
The minAreaRect function of opencv returns a RotatedRect object. This object contains all the data I need.
Note
Because the rectangle is mono colored, there is no possibility to distinguish between top and bottom or left and right. This means that the rotation is always in range [0, 180] which is perfectly fine for my purposes. The ratio of the two sides of the rect is always w:h > 2:1. If the rectangle would be a square, the range of roation would change to [0, 90], but this can be considered irrelevant here.
As suggested in the comments I will try histogram equalization to reduce brightness issues and take a look at ORB, SURF and SIFT.
I will update on progress.
The H channel in the HSV space is the Hue, and it is not sensitive to the light changing. Red range in about [150,180].
Based on the mentioned information, I do the following works.
Change into the HSV space, split the H channel, threshold and normalize it.
Apply morph ops (open)
Find contours, filter by some properties( width, height, area, ratio and so on).
PS. I cannot fetch the image you upload on the dropbox because of the NETWORK. So, I just use crop the right side of your second image as the input.
imgname = "src.png"
img = cv2.imread(imgname)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
## Split the H channel in HSV, and get the red range
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsv)
h[h<150]=0
h[h>180]=0
## normalize, do the open-morp-op
normed = cv2.normalize(h, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_8UC1)
kernel = cv2.getStructuringElement(shape=cv2.MORPH_ELLIPSE, ksize=(3,3))
opened = cv2.morphologyEx(normed, cv2.MORPH_OPEN, kernel)
res = np.hstack((h, normed, opened))
cv2.imwrite("tmp1.png", res)
Now, we get the result as this (h, normed, opened):
Then find contours and filter them.
contours = cv2.findContours(opened, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
print(len(contours))[-2]
bboxes = []
rboxes = []
cnts = []
dst = img.copy()
for cnt in contours:
## Get the stright bounding rect
bbox = cv2.boundingRect(cnt)
x,y,w,h = bbox
if w<30 or h < 30 or w*h < 2000 or w > 500:
continue
## Draw rect
cv2.rectangle(dst, (x,y), (x+w,y+h), (255,0,0), 1, 16)
## Get the rotated rect
rbox = cv2.minAreaRect(cnt)
(cx,cy), (w,h), rot_angle = rbox
print("rot_angle:", rot_angle)
## backup
bboxes.append(bbox)
rboxes.append(rbox)
cnts.append(cnt)
The result is like this:
rot_angle: -2.4540319442749023
rot_angle: -1.8476102352142334
Because the blue rectangle tag in the source image, the card is splited into two sides. But a clean image will have no problem.
I know it's been a while since I asked the question. I recently continued on the topic and solved my problem (although not through rectangle detection).
Changes
Using wood to strengthen my controllers (the "rectangles") like below.
Placed 2 ArUco markers on each controller.
How it works
Convert the frame to grayscale,
downsample it (to increase performance during detection),
equalize the histogram using cv::equalizeHist,
find markers using cv::aruco::detectMarkers,
correlate markers (if multiple controllers),
analyze markers (position and rotation),
compute result and apply some error correction.
It turned out that the marker detection is very robust to lighting changes and different viewing angles which allows me to skip any calibration steps.
I placed 2 markers on each controller to increase the detection robustness even more. Both markers has to be detected only one time (to measure how they correlate). After that, it's sufficient to find only one marker per controller as the other can be extrapolated from the previously computed correlation.
Here is a detection result in a bright environment:
in a darker environment:
and when hiding one of the markers (the blue point indicates the extrapolated marker postition):
Failures
The initial shape detection that I implemented didn't perform well. It was very fragile to lighting changes. Furthermore, it required an initial calibration step.
After the shape detection approach I tried SIFT and ORB in combination with brute force and knn matcher to extract and locate features in the frames. It turned out that mono colored objects don't provide much keypoints (what a surprise). The performance of SIFT was terrible anyway (ca. 10 fps # 540p).
I drew some lines and other shapes on the controller which resulted in more keypoints beeing available. However, this didn't yield in huge improvements.
I have a little problem. I need to create the voronoi diagram of a BW image by using openCV and C++. I should have something like the output of the Matlab function voronoin.
The goal is to create a mask for each region of the diagram.
This is an example I made in Matlab:
matlab voronoi diagram
So, for each region I should create a mask or to have a different color.
I tried the openCV function distanceTransform in order to get the voronoi labels.
Mat bwCoresGoodInv = 255 - bwCoresGood;
distanceTransform(bwCoresGoodInv, distTr,voronoiLabels, CV_DIST_L2, CV_DIST_MASK_PRECISE, DIST_LABEL_PIXEL);
namedWindow( "voronoiDistLab", CV_WINDOW_AUTOSIZE );
voronoiLabels = voronoiLabels*5;
imshow( "voronoiDistLab", voronoiLabels );
the results is the following image:
voronoi labels openCV
as you can see in each region there are differents colors(in particular there is something in correspondence to the cell), is there a way to have just a color?
thank you in advance
If you are asking how to get different colors than the gray scale values provided by displaying the labels, one approach (probably not the most efficient) is to run cv::findContours on an edge-detected image of the label image, and then iterate through each contour found and draw it onto a new image, it can be filled or outlined. It's not super exact and can leave gaps, some dilation on the edge image may be required.
It would be very nice if distanceTransform returned a data-structure that mapped the range of intensity values in the label image to every pixel that has that value, maybe with a vector of binary images where the nth image in the vector is a binary mask with an isolated nth label region- but I think as it is now this would have to be done by the user.