I am working on the problem of segmentation in videos. The user segments the first frame correctly (using grabcut here) and I generate a mask from here. The black pixels all correspond to the background and white pixels correspond to foreground. In all subsequent frame I want to shift these white pixels according to some rule. That is I want to shift all the white pixels by some amount. Is there anyway ( a function probably?) that can help me do this shifting?
As in, a brute-force way will be to visit every pixel and if it white, make the pixel to the (right/left) of it to move by whatever amount I want to shift the mask. I was wondering if there is a smarter way to do this?
So you binarized the image by a threshold, resulting in back- and foreground pixels (like canny)?
You could apply a contour on the foreground pixels. Each contour is stored as a vector of points, therefore you can apply/move a contour on the next frame.
For finding contours in an Image use findContours.
Related
I am new to opencv, coding in c++. I have a task given to me to decode a 2D circle barcode using an encoded array. I am up to the point where I am able to centralize the figure and get the line using Hough transforms.
Need help with how to read the colour in the images, note that each of the two adjacent blocks correspond to a letter.
Any pointers will be highly appreciated. Thanks.
First, you need to load the image. I suspect this isn't a problem because you are already using Hough transforms on it, but:
Mat img = imread(filename)
Once the image is loaded, you can grab any of the pixels using:
Scalar intensity = img.at<uchar>(y, x);
However, what you need to do is threshold the image. As I mentioned in the comments, the image colors are either 0 or 255 for each RGB channel. This is on purpose for encoding the data in case there are image artifacts. If the channel is above a certain color value, then you will consider that it's 'on' and if below, it's 'off'.
Threshold the image using adaptiveThreshold. I would threshold down to binary 1 or 0. This will produce RGB triplets that are one of eight (2^3) possible combinations, from (0,0,0) to (1,1,1).
Then you need to walk the pixels. This is where it gets interesting.
You say each adjacent 2 pixels form a single letter. That's 2^6 or 64 different letters. The next question is: are the letters arranged in scan lines, left-to-right, top to bottom? If yes, then it will be important to orientate the image using the crosshair in the center.
If the image is encoded radially (using polar coordinates) then things get a little trickier. You need to use cvLinearPolar to remap the image.
Otherwise you need to walk the whole image, stepping the size of the RGB blocks and discard any pixels whose distance from the center is greater than the radius of the circle. After reading all of the pixels into an array, group them by pairs.
At some point, I would say that using OpenCV to do this is heading towards machine learning. There has to be some point where you can cut in and use Neural Networks to decode the image for you. Once you have the circle (cutoff radius) and the image centered, you can convert to polar coordinates and discard everything outside the circle by cutting off everything greater than the radius of the circle. Remember, polar coordinates are (r,theta), so you should be able to cutoff the right part of the polar image.
Then you could train a Neural Network to take the polar image as input and spit out the paragraph.
You would have to provide lots of training data, and the trained model would still be reliant on your ability to pre-process the image. This will include any affine transforms in case the image is tilted or rotated. At that point you would say to yourself that you've done all the heavy lifting and the last little bit really isn't that hard.
However, once you get a process working for a clean image, you can start adding to steps to introduce ML to work on dirty images. HoughCircles can be used to detect the part of an image to run detection on. Next, you need to decide if the image inside the circle is a barcode or not.
A good barcode system will have parity bits or some other form of error correction, but you can use machine learning to cleanup output.
My 2 cents anyways.
I wrote code to make a 2D transformation: scaling.
( value = variable from slider 1-10 )
int x=punktx.back();
int y=punkty.back();
DrawPixel(x*value/6.0,y*value/6.0,bits,255,255,255);
And I received that output:
As you can see I received a little breaks in that square. Is it okay or I have wrong code?
It's not how you scale things in Qt. Use QImage::scaled() or QPixmap::scaled() method instead.
Regarding your problem, the breaks are result of the fact that you use the same number of pixels for drawing the large square as for the small square - you would have to fill the gaps between the pixels, but scaling that way doesn't make sense anyway as mentioned above.
the problem is that if you iterate over an input image that's e.g. 10x10 pixels, and output the same number of pixels then you're only drawing 100 pixels no matter how much you "scale" it by. If you're scaling it to fill 20x20 pixels in size but you only draw your original 100 pixels, of course it will have holes in it.
If you want to implement a simple scaling function as a learning exercise, some approaches are:
instead of drawing 1 pixel per original pixel, draw a rectangle of the scaled size, in that pixel's color. This has the advantage that it's only a tiny change to your existing code, so it might be worth trying as a first attempt.
loop over the output pixels, then interpolate where that would be on the input image (reverse the scaling) then draw one pixel of the right color. This avoids some of the overheads with drawing lots of rectangles (which could paint the same output pixels more than once).
as above, but write the data into a bitmap stored in memory, then draw it all at once with a bitmap drawing command.
Also if you really want to get better results you can work out whether an output pixel crosses over several input pixels, then average out the colors, and so on. This will give smoother looking results but could blur things for solid color images.
I've an image like this one ![enter image description here][1]. The non-black part is expanded at each iteration. So, after a certain point, I need to enlarge the final image so the non-black one can fit in. For now, what I'm doing is to find the contour of the non-black image,find the bounding box of the contours and check the width/height of the box. At a first time it works, but after some iterations my program finds a bounding box of size 1 (it seems that it doesn't find any contour). What the problem could be?
Ps: the program is a mosaic from a video file, I followed the opencv tutorial for find homography and other stuff.
EDIT
Sorry but I had to remove images
Just a suggestion:
It's easier to simply iterate through each element in the matrix and record the coordinates of the uppermost, bottommost, leftmost and rightmost non-zero elements. These will be the four corners of your up-right bounding rectangle. Of course it is not necessarily the rectangle of the minimum area enclosing the non-zero pixels (not a rotated rectangle), but further can be used as a ROI.
I want to compute the number of black pixel in arbitrary shapes in a picture. There might be several objects, like in the picture at the bottom.
I suspect that the problem is solveable with dynamic programming, i.e. traverse the pixels row-wise and add the black pixels. I just don't know how to correctly unite the size of two parts.
I'm pretty sure there are algorithms that solve my problem, but i obviously use the wrong search terms.
Can you please provide me with a good (fast) algorithm to do so, Bonus points if the algorithm is written in c++ and compatible to Mat from the OpenCV library. ;)
Result for this (zoomed) picture would be something like: 15 for Object at top left, 60 for big blob,...
I think i found a solution (better ones are obviously welcome!):
Integrated the size computation into a Connected Component Algorithm.
In the Connected Component algorithm, we generate a new Image in which there are labels (numbers) instead of the black pixels. All pixel of one area have the same label.
New to CC-Algo is a table in which the total amount of pixel for each label is stored. That way i know for every connected component the correct size.
Process the image from left to right, top to bottom:
1.) If the next pixel to process is white:
do nothing
2.) If the next pixel to process is black:
i.) If only one of its neighbors (top or left) is black, copy its label and +1 in the size table for that label.
ii.) If both are black and have the same label, copy it and +1 in the size table for that label.
iii.) If they have different labels Copy the label from the left. Update the equivalence table and +1 in the size table for the left label.
iv.) Otherwise, assign a new label and update the size table with that label and value 1.
• Re-label with the smallest of equivalent labels and update size table accordingly
The problem can be solved using flood fill in following way : -
Keep 2-D boolean array to track if pixel is already visited initially set to false
scan the image pixel by pixel.
if pixel is unvisited and black then apply flood fill on it,
During floodfill count the number of call also mark visited pixel made as they are the no of pixels connected.
Terminate floodfill when white pixels are encountered.
Count is the size of the region containing the pixel.
Flood Fill
If I well understoud, in an image like your sample you want your alogirthm to return 6 values on for each black shapes. And, each value the number of black pixels.
The algorithm I would use for this is the following :
Invert Pixels colors of your image (so now, you are looking for white pixels)
Find contours in your image. Don't forget to find only EXTERNAL Countours.
For each contours found :
Draw each contour in a small cv::Mat with a pixel value of 1. then compute the moment of order 0 of this image. The moment of order 0 will be the number of pixel in the shape.
i have several contours that consist of several black regions in my image. Directly adjacent to these black regions are some brighter regions that do not belong to my contours. I want to add these brighter regions to my black region and therefor extend my contour in OpenCv.
Is there a convenient way to extend a contour? I thought about looking at intensity change from my gradient-image created with cv::Sobel and extend until the gradient changes again, meaning the intensity of pixel is going back to the neither black nor bright regions of the image.
Thanks!
Here are example images. The first picture shows the raw Image, the second the extracted Contour using Canny & findContours, the last one the Sobel-Gradient intensity Image of the same area.
I want to include the bright boundaries in the first image to the Contour.
Update: Now i've used some morphological operations on the Sobelgradients and added a contour around them (see Image below). Next step could be to find the adjacent pair of purple & red contours, but it seems very much like a waste of procession time to actually have to search for directly adjacent contours. Any better ideas?
Update 2: My solution for now is to search for morphed gradient (red) contours in a bounding box around my (purple) contours and pick the one with correct orientation & size. This works for gradient contours where the morphological operation closes the "rise" and "fall" gradient areas like in Figure 3. But it is still a bad solution for cases in which the lighted area is wider then in the image above. Any idea is still very much appreciated, thanks!
What you're trying to do is find two different features and merge them. It's not terribly difficult but you have to use multiple copies of the image to make it happen.
Make one copy, and threshold it for the dark portion
Make another copy and threshold it for the light portion
Merge both thresholded images into a new image
Apply a morphological operation like opening or closing (depending on how you threshold) This will connect nearby components
Find contours in the resultant image
Use those contours on your original image. This will work since all the images are the same size and all based off of the original.