Problem
Is there a build-in function for interpolating single pixels?
Given a normal image as Mat and a Point, e.g. an anomaly of the sensor or an outlier, is their some function to repair this Point?
Furthermore, if I have more than one Point connected (let's say a blob with area smaller 10x10) is there a possibility to fix them too?
Trys but not really solutions
It seems that interpolation is implemented in the geometric transformations including resizing images and to extrapolate pixels outside of the image with borderInterpolate, but I haven't found a possibility for single pixels or small clusters of pixels.
A solution with medianBlur like suggested here does not seem appropriate as it changes the whole image.
Alternative
If there isn't a build-in function, my idea would be to look at all 8-connected surrounding pixels which are not part of the blob and calculate the mean or weighted mean. If doing this iteratively, all missing or erroneous pixel should be filled. But this method would be dependent of the applied order to correct each pixel. Are there other suggestions?
Update
Here is an image to illustrate the problem. Left the original image with a contour marking the pixels to fix. Right side shows the fixed pixels. I hope to find some sophisticated algorithms to fix the pixel.
The build-in function inpaint of OpenCV does the desired interpolation of chosen pixels. Simply create a mask with all pixels to be repaired.
See the documentation here: OpenCV 3.2. Description: inpaint and Function: inpaint
Related
I'm trying to remove foreground from two images, here's a sample pair of images:
As you can see, the Budweiser bottle is removed from the scene before the second shot is taken.
These photos were captured from a pinhole camera (iPhone), and, the tricky part is I'm hand-holding the camera, so it cannot be guaranteed that the images are perfectly aligned pixel by pixel, so a simple minus-threshold method will not work.
Then, I've decided to perform image registration using findHomography and warpPerspective from OpenCV, here's the result image:
This image is warped with the matrix I've got from findHomography, it kind of improved the alignment quality, but still not that aligned so I can use a simple way to remove the foreground.
So, finally, I decided to implement a "fuzzy-minus" algorithm: for every pixel in image1, I'll look through a 7x7 neighbour in image2 (a 7 by 7 kernel?), using the minimal difference in grayscale as the result of minus, and threshold the result into binary image, here's what I've got:
And the result is still not good. Notice the white wholes in the bottle, this is produced due to similar grayscale value of foreground and background. So I'm not sure what to do now.
I can think of two ways to solve the problem, the first is to get a better aligned pair of images, and simply minus the pairs; the second is to use a more robust way to extract the foreground.
Can anyone give me some advice on how to deal with this kind of problem? I believe there should be some state-of-art algorithms or processing pipelines, but after googling around, I get nothing.
I'm using OpenCV with C++, it would be fantastic if you can tell me how to do it with these tools in hand.
Big big thanks in advance!
The problem is not in your algorithm. You are having problem because the two scenes were not taken from exactly the same angle, as shown in the animation below. This slight difference highlight the edges in the subtraction.
You need a static camera in order to apply this approach.
I suggest using mathematical morphology on the mask that you got to get rid of the artifacts.
Try applying both opening and closing to get rid of the black and the white small regions.
Mathematical Morphology
Mathematical Morphology in opencv
The difference between the two picture is pretty huge, so you will need to use a large structure element, but I don't think you will be able to get rid of the shadow.
For the two large strips in the background, you may try to use a horizontally shaped structure element as well.
Edit
Is it possible to produce a grayscale image instead of a binary image? if yes, you may try to experiment with the hat method for the shadow, but I am not sure about this point.
This is what I got using two different structure elements for closing THEN opening
Mat mask = imread("mask.jpg",CV_LOAD_IMAGE_GRAYSCALE);
morphologyEx(mask,mask,MORPH_CLOSE,getStructuringElement(CV_SHAPE_ELLIPSE,Size(50,10)));
morphologyEx(mask,mask,MORPH_OPEN,getStructuringElement(CV_SHAPE_ELLIPSE,Size(10,50)));
imshow("open",mask);
imwrite("maskopenclose.jpg",mask);
I would suggest optical flow for alignment and OpenCV's background subtraction algorithm:
http://docs.opencv.org/trunk/doc/tutorials/video/background_subtraction/background_subtraction.html
I suggest that instead of using findHomography try using some of openCV's stereo correspondence functions: http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html
there is a sample code here: https://github.com/Itseez/opencv/blob/master/samples/cpp/stereo_calib.cpp
I am writing an application in C++ using OpenCV to apply a Gaussian filter to individual pixels in an image. For example, I loop through each pixel in the image and if they match a particular RGB value, I want to apply the Gaussian algorithm to only those pixels so that blurring only occurs around those parts of an image.
However, I'm having trouble finding a way to do this. The GaussianBlur() function offered by the OpenCV library only allows me to blur an entire image and not simply apply the algorithm and kernel to one pixel at a time. Does anyone have any ideas on how I could achieve this (e.g. is there another method I don't know about)? I'm hoping I don't have to write the entire algorithm out myself to apply it to just a single pixel.
A friend of mine came up with a good solution:
clone the original matrix
apply GaussianBlur() to the clone
compare each pixel to the RGB value
if match, replace the original's pixel with the clone's pixel
Can't believe how simple that was.
You can code the gaussian blur yourself if you need to apply it only on a few pixels. It is much easier than you think and only takes a few lines. It is a simple stencil operator using a gaussian function for its kernel. All you need is the coordinates of the pixel and its neighbors.
The rest is straight forward. Here is an example of Gaussian matrix, which you can easily code, or generate using a Gaussian function:
In short, it is just a weighed average of the neighboring values.
I'm working on a dilation problem in c++ with opencv. I've captured videoframes of a car park and in order to obtain the best blobs I came up with this.
Erosion (5x5 kernel rectangular), 3 iterations
Dilation GRADIENT (think of it like a color gradient along the y-axis)
So what did I do to get this working? First I needed to know 2 points (x,y) and 2 good dilate kernelsizes at those points. With this information one can inter and extrapolate those values over the whole image. So I calculated ROI's (size and dilation kernelsize) from those parameters. So each ROI has its own predefined kernelsize used for dilation. Note that there isn't any space between two consecutive ROI's (opencv rectangles). Everything is working fine, but there are two side effects:
Buldges on the sides of the blobs. The black line is de border of the ROI!
buldges picture
Blobs which are 'cut off' from the main blob. These aren't actually cut off but the ROI under the one of the blob above dilates (gets pixel information from the above ROI, I think) into blobs who are seperated. It should be one massive blob. blob who shoudn't be there picture
I've tried everything on changing the ROI sizes and left some space between them but the disadvantage is that the blob between 2 seperated ROI's is not dilated.
So my questions are:
What causes those side effects exactly?
What do I have to do to make them go away?
EDIT
So I found my solution: when you call the opencv dilate function, one needs to be sure if the same cv::Mat can be used as destination image. If not you'll be using parts of the original and new image. So all I had to do was including a destination cv::Mat.
This doesn't answer your first question (What causes those side effects for sure), but to make them go away, you can do some variant of the following, assuming the ROI parameters are discrete and not continuous (as seems to be the case).
You can compute the dilation for the entire image using every possible kernel size. Then, after all of those binary images are computed, you can combine them together taking the correct samples from the correct image to get the desired output image. This absolutely will waste a good deal of time, but it should work with no artifacts.
Once you've confirmed that the results you've gotten above (which are pretty much guaranteed to be of as-good-as-possible quality) you can start trying to optimize. One thing I'd try is expanding each of the ROI sizes for computing the dilation by the size of the kernel size. This might get around artifacts that can arise from strange boundary conditions.
This leads to my guess as to what causes the artifacts in the first place: Whenever you take a finite image and run a convolution (or morphological operator) you need to choose what you'll do with the edge pixels. Normally, accessing the pixel at (-4, -1) is meaningless, but to perform the operator you'll have to if your kernel overlaps with it. If OpenCV is doing this edge padding for your subregions, it very easily could give you the artifacts you're seeing.
Hope this helps!
I am working on video frames using OpenCV. My question might be low leveled, but I want to clarify it first.
There are plenty of pixel based methods available in openCV, but can I change them into frame based ones?
To me, it is similar, since the whole frame is also stored in one matrix, and I will read that matrix from the beginning to end to handle it. So for instance for finding average value, the only thing I should change is find the total average of whole pixels for one frame.
But for one pixel, see several frames and decide that pixel's average based on them. But when it comes to build models like GMM, I cannot differentiate it.
Could someone help explain it clearly?
Can I use or change openCV's GMM for global usage?
I think this is a good definition for the problem, though you are working with pixels.
Pixel-based methods: The information of the pixel(x,y) in the resulting processed image is the result of applying transformations to the pixel(x,y) of the original image.
Region-based methods: The pixels in the original image are grouped forming a contiguous regions and transformations are applied to the whole region. Example: the resulting pixel(x,y) is the mean of a patch around the original pixel (x,y).
Finding Circle Edges :
Here are the two sample images that i have posted.
Need to find the edges of the circle:
Does it possible to develop one generic circle algorithm,that could find all possible circles in all scenarios ?? Like below
1. Circle may in different color ( White , Black , Gray , Red)
2. Background color may be different
3. Different in its size
http://postimage.org/image/tddhvs8c5/
http://postimage.org/image/8kdxqiiyb/
Please suggest some idea to write a algorithm that should work out on above circle
Sounds like a job for the Hough circle transform:
I have not used it myself so far, but it is included in OpenCV. Among other parameters, you can give it a minimum and maximum radius.
Here are links to documentation and a tutorial.
I'd imagine your second example picture will be very hard to detect though
You could apply an edge detection transformation to both images.
Here is what I did in Paint.NET using the outline effect:
You could test edge detect too but that requires more contrast in the images.
Another thing to take into consideration is what it exactly is that you want to detect; in the first image, do you want to detect the white ring or the disc inside. In the second image; do you want to detect the all the circles (there are many tiny ones) or just the big one(s). These requirement will influence what transformation to use and how to initialize these.
After transforming the images into versions that 'highlight' the circles you'll need an algorithm to find them.
Again, there are more options than just one. Here is a paper describing an algoritm
Searching the web for image processing circle recognition gives lots of results.
I think you will have to use a couple of different feature calculations that can be used for segmentation. I the first picture the circle is recognizeable by intensity alone so that one is easy. In the second picture it is mostly the texture that differentiates the circle edge, in that case a feature image based based on some kind of texture filter will be needed, calculating the local variance for instance will result in a scalar image that can segment out the circle. If there are other features that defines the circle in other scenarios (different colors for background foreground etc) you might need other explicit filters that give a scalar difference for those cases.
When you have scalar images where the circles stand out you can use the circular Hough transform to find the circle. Either run it for different circle sizes or modify it to detect a range of sizes.
If you know that there will be only one circle and you know the kind of noise that will be present (vertical/horizontal lines etc) an alternative approach is to design a more specific algorithm e.g. filter out the noise and find center of gravity etc.
Answer to comment:
The idea is to separate the algorithm into independent stages. I do not know how the specific algorithm you have works but presumably it could take a binary or grayscale image where high values means pixel part of circle and low values pixel not part of circle, the present algorithm also needs to give some kind of confidence value on the circle it finds. This present algorithm would then represent some stage(s) at the end of the complete algorithm. You will then have to add the first stage which is to generate feature images for all kind of input you want to handle. For the two examples it should suffice with one intensity image (simply grayscale) and one image where each pixel represents the local variance. In the color case do a color transform an use the hue value perhaps? For every input feed all feature images to the later stage, use the confidence value to select the most likely candidate. If you have other unknowns that your algorithm need as input parameters (circle size etc) just iterate over the possible values and make sure your later stages returns confidence values.