I am building a disparity map with OpenCV's SGBM implementation. In particular, I'm using the OpenCV version 2.4.9. Currently, I am working with the following parameters:
int numberOfDisparities = 64;
sgbm.preFilterCap = 25;
sgbm.SADWindowSize = 14;
sgbm.P1 = 8*sgbm.SADWindowSize*sgbm.SADWindowSize;
sgbm.P2 = 32*sgbm.SADWindowSize*sgbm.SADWindowSize;
sgbm.minDisparity = 0;
sgbm.numberOfDisparities = numberOfDisparities;
sgbm.uniquenessRatio = 10;
sgbm.speckleWindowSize = 100;
sgbm.speckleRange = 2;
sgbm.disp12MaxDiff = 1;
sgbm.fullDP = false;
However, the result is not as good as I need:
As you can see, the ground appears striped, which causes problems with the obstacle detection algorithm I am using, which is an implementation of the paper Fast and reliable obstacle detection and segmentation for cross-country navigation. The problem is that, as these strips have a rather constant disparity, which means that teorically they are almost perpendicular to the ground, the algorithm clasifies them as obstacles.
I played a little bit with the smoothing parameters P1 and specially P2, testing if increasing P2 improved it, but it doesn't seem to solve the problem, so I just left as recommended by the OpenCV sample.
The images are from the KITTI dataset and I downloaded them already rectified, so I discarded a bad calibration or camera alignment.
I wonder if it can be due to an untextured ground, although I hope not, since if that's the case it means I would have to change the obstacle detection algorithm.
Any ideas?
Related
I am triangulating 3D points using OpenCV triangulation function for monocular sequence that sometimes works fine but I have noticed when two camera poses are close to each other then the triangulated points are far away. I can understand the issue that is since the camera poses are close then the ray intersection from two cameras is being take place far away from the camera. That is why it creates the 3D points far away. I have also noticed that the distance requirement between two cameras for correct triangulation varies in different cases.Currently I am trying to find parallax between two pose and if that is above a certain threshold(I have chosen 27) then proceed to triangulate but I does not look correct for all the cases.
My code for calculating parallax as following-
float checkAvgParallex(SE3& prevPose, SE3& currPose, std::vector<Point2f>& prevPoints, std::vector<Point2f>& currPoints, Mat& K) {
Eigen::Matrix3d relRot = Eigen::Matrix3d::Identity();
Eigen::Matrix3d prevRot = prevPose.rotationMatrix();
Eigen::Matrix3d currRot = currPose.rotationMatrix();
relRot = prevRot * currRot;
float avg_parallax = 0.;
int nbparallax = 0;
std::set<float> set_parallax;
bearingVectors_t prevBVs;
bearingVectors_t currBVs;
points2bearings(prevPoints, K, prevBVs);
points2bearings(currPoints, K, currBVs);
for (int i = 0; i < prevPoints.size(); i++) {
Point2f unpx = projectCamToImage(relRot * currBVs[i], K);
float parallax = cv::norm(unpx - prevPoints[i]);
avg_parallax += parallax;
nbparallax++;
set_parallax.insert(parallax);
}
if (nbparallax == 0)
return 0.0;
avg_parallax /= nbparallax;
auto it = set_parallax.begin();
std::advance(it, set_parallax.size() / 2);
avg_parallax = *it;
return avg_parallax;
}
And sometime when parallax between camera does not exceed 27 so, triangulation won't work, due to this my further pose calculation in SLAM system stops due to lack of 3D points.
So can anyone suggest me alternative strategy using which I can estimate correct 3D points and my SLAM system wont suffer due to lack of 3D points, please?
I am trying to implement an auto grid detection system for an electrocardiogram, ecg, paper see the figure below.The idea behind is to add the pixel values(only considered the red channel) by going through pixel by pixel of the ecg image as shown in the code below.
QImage image("C:/Users/.../Desktop/ECGProject/electrocardiogram.jpg");
std::vector<int> pixelValues;
for (int y = 0; y < img.height(); y++)
{
int rowSumR = 0, rowSumG = 0, rowSumB = 0;
for (int x = 0; x < img.width(); x++)
{
QRgb rgb = img.pixel(x, y);
rowSumR += qRed(rgb);
}
rowSumR /= img.width();
const int &value = rowSumR/4;
pixelValues.push_back(value)
}
The vector pixelValues contains summed values which has repeated pattern in a y direction. The goal is to detect those repeated pattern (for instance the line drawn in black color on in the ecg image is the interest or what I am looking to identify in a y direction). I also draw the summed pixel value in y direction using matlab(see the figure below) and the red circles are the pattern I am interested in. Any suggestion/algorithm to find these repeated pattern would be appreciated.
[![Ecg paper][1]][1] [![enter image description here][2]][2]
If you need to identify the number of bold red grid lines and "cut off" the similar patterns associated with each "period" in it I would suggest using of pitch tracking algorithms used in speech processing. One such approach, which computes the so-called pitch track is described in this work:
https://www.diva-portal.org/smash/get/diva2:14647/FULLTEXT01.pdf
If you need help implementing that algorithm I can do it for you if you provide me the data.
I wrote a following program for you in matlab:
load data.txt
y = data(:,2);
yr = resample(y,10,1);
xhat = cceps(yr);
figure(1)
subplot(2,1,1)
plot(0:length(xhat)-1,xhat)
subplot(2,1,2)
plot(0:length(yr)-1,yr)
maxima = zeros(10000,1);
cnt = 1;
for i = 2:length(xhat)-1
if xhat(i-1) < xhat(i) && xhat(i+1) < xhat(i)
maxima(cnt) = i-1;
cnt = cnt + 1;
end
end
maxima(cnt:end) = [];
disp(maxima(1:10)/10)
The cepstra are a signal processing tool, which allow detection of periodicity. It actually deconvolve signals. Say, in our case, we have an impuls train and some pattern convolved. Cepstral analysis 'decouples' the impuls train and the pattern. The impuls train period results in a maximum at given time spot in the cepstrum. If you run this program you can state from the output that the fine grained periodicity has mean period of 3.5 pixels and the greedy periodicity (you marked the corresponding impulses red) has mean period of 23.4 pixels (note the interpolation). Based on this observation you can try by the correlation analysis to refine the local placement of impulses with a technique known from speech processing as pitch-analysis (which is based on the correlation analysis). This last step might be necessary since there are apparent irregularities in peaks placement. Let me know if you have further doubts.
I am building an Android app to create panoramas. The user captures a set of images and those images
are sent to my native stitch function that was based on https://github.com/opencv/opencv/blob/master/samples/cpp/stitching_detailed.cpp.
Since the images are in order, I would like to match each image only to the next image in the vector.
I found an Intel article that was doing just that with following code:
vector<MatchesInfo> pairwise_matches;
BestOf2NearestMatcher matcher(try_gpu, match_conf);
Mat matchMask(features.size(),features.size(),CV_8U,Scalar(0));
for (int i = 0; i < num_images -1; ++i)
{
matchMask.at<char>(i,i+1) =1;
}
matcher(features, pairwise_matches,matchMask);
matcher.collectGarbage();
Problem is, this wont compile. Im guessing its because im using OpenCV 3.1.
Then I found somewhere that this code would do the same:
int range_width = 2;
BestOf2NearestRangeMatcher matcher(range_width, try_cuda, match_conf);
matcher(features, pairwise_matches);
matcher.collectGarbage();
And for most of my samples this works fine. However sometimes, especially when im stitching
a large set of images (around 15), some objects appear on top of eachother and in places they shouldnt.
I've also noticed that the "beginning" (left side) of the end result is not the first image in the vector either
which is strange.
I am using "orb" as features_type and "ray" as ba_cost_func. Seems like I cant use SURF on OpenCV 3.1.
The rest of my initial parameters look like this:
bool try_cuda = false;
double compose_megapix = -1; //keeps resolution for final panorama
float match_conf = 0.3f; //0.3 default for orb
string ba_refine_mask = "xxxxx";
bool do_wave_correct = true;
WaveCorrectKind wave_correct = detail::WAVE_CORRECT_HORIZ;
int blend_type = Blender::MULTI_BAND;
float blend_strength = 5;
double work_megapix = 0.6;
double seam_megapix = 0.08;
float conf_thresh = 0.5f;
int expos_comp_type = ExposureCompensator::GAIN_BLOCKS;
string seam_find_type = "dp_colorgrad";
string warp_type = "spherical";
So could anyone enlighten me as to why this is not working and how I should match my features? Any help or direction would be much appreciated!
TL;DR : I want to stitch images in the order they were taken, but above codes are not working for me, how can I do that?
So I found out that the issue here is not with the order the images are stitched but rather the rotation that is estimated for the camera parameters in the Homography Based Estimator and the Bundle Ray Adjuster.
Those rotation angles are estimated considering a self rotating camera and my use case envolves an user rotating the camera (which means that will be some translation too.
Because of that (i guess) horizontal angles (around Y axis) are highly overestimated which means that the algorithm considers the set of images cover >= 360 degrees which results in some overlapped areas that shouldnt be overlapped.
Still havent found a solution for that problem though.
matcher() takes UMat as mask instead of Mat object, so try the following code:
vector<MatchesInfo> pairwise_matches;
BestOf2NearestMatcher matcher(try_gpu, match_conf);
Mat matchMask(features.size(),features.size(),CV_8U,Scalar(0));
for (int i = 0; i < num_images -1; ++i)
{
matchMask.at<char>(i,i+1) =1;
}
UMat umask = matchMask.getUMat(ACCESS_READ);
matcher(features, pairwise_matches, umask);
matcher.collectGarbage();
I have a dataset of 500 cv::Point.
For each point, I need to determine if this point is contained in a ROI modelized by a concave polygon.
This polygon can be quite large (most of the time, it can be contained in a bounding box of 100x400, but it can be larger)
For that number of points and that size of polygon, what is the most efficient way to determine if a point is in a polygon?
using the pointPolygonTest openCV function?
building a mask with drawContours and finding if the point is white or black in the mask?
other solution? (I really want to be accurate, so convex polygons and bounding boxes are excluded).
In general, to be both accurate and efficient, I'd go with a two-step process.
First, a bounding box on the polygon. It's a quick and simple matter to see which points are not inside the box. With that, you can discard several points right off the bat.
Secondly, pointPolygonTest. It's a relatively costly operation, but the first step guarantees that you will only perform it for those points that need better accuracy.
This way, you mantain accuracy but speed up the process. The only exception is when most points will fall inside the bounding box. In that case, the first step will almost always fail and thus won't optimise the algorithm, will actually make it slightly slower.
Quite some time ago I had exactly the same problem and used the masking approach (second point of your statement). I was testing this way datasets containing millions of points and found this solution very effective.
This is faster than pointPolygonTest with and without a bounding box!
Scalar color(0,255,0);
drawContours(image, contours, k, color, CV_FILLED, 1); //k is the index of the contour in the array of arrays 'contours'
for(int y = 0; y < image.rows, y++){
const uchar *ptr = image.ptr(y);
for(int x = 0; x < image.cols, x++){
const uchar * pixel = ptr;
if((int) pixel[1] = 255){
//point is inside contour
}
ptr += 3;
}
}
It uses the color to check if the point is inside the contour.
For faster matrix access than Mat::at() we're using pointer access.
In my case this was up to 20 times faster than the pointPolygonTest.
I want to perform Template matching with mask. In general Template matching can be made faster by converting the image from Spacial domain into Frequency domain. But is there any any method i can apply if i want to perform the same with mask? I'm using opencv c++. Is there any matching function already there in opencv for this task?
My current Approach:
Bitwise Xor Image A & Image B with Mask.
Count the Non-Zero Pixels.
Fill the Resultant matrix with this count.
Search for maxi-ma.
Few parameters I'm guessing now are:
Skip the Tile position if the matches are less than 25%.
Skip the tile position if the matches are less than 25%.
Skip the Tile position if the previous Tile has matches are less than 50%.
My question: is there any algorithm to do this matching already? Is there any mathematical operation which can speed up this process?
With binary images, you can use directly HU-Moments and Mahalanobis distance to find if image A is similar to image B. If the distance tends to 0, then the images are the same.
Of course you can use also Features detectors so see what matches, but for pictures like these, HU Moments or Features detectors will give approximately same results, but HU Moments are more efficient.
Using findContours, you can extract the black regions inside the white star and fill them, in order to have image A = image B.
Other approach: using findContours on your mask and apply the result to Image A (extracting the Region of Interest), you can extract what's inside the star and count how many black pixels you have (the mismatching ones).
I have same requirement and I have tried the almost same way. As in the image, I want to match the castle. The castle has a different shield image and variable length clan name and also grass background(This image comes from game Clash of Clans). The normal opencv matchTemplate does not work. So I write my own.
I follow the ways of matchTemplate to create a result image, but with different algorithm.
The core idea is to count the matched pixel under the mask. The code is following, it is simple.
This works fine, but the time cost is high. As you can see, it costs 457ms.
Now I am working on the optimization.
The source and template images are both CV_8U3C, mask image is CV_8U. Match one channel is OK. It is more faster, but it still costs high.
Mat tmp(matTempl.cols, matTempl.rows, matTempl.type());
int matchCount = 0;
float maxVal = 0;
double areaInvert = 1.0 / countNonZero(matMask);
for (int j = 0; j < resultRows; j++)
{
float* data = imgResult.ptr<float>(j);
for (int i = 0; i < resultCols; i++)
{
Mat matROI(matSource, Rect(i, j, matTempl.cols, matTempl.rows));
tmp.setTo(Scalar(0));
bitwise_xor(matROI, matTempl, tmp);
bitwise_and(tmp, matMask, tmp);
data[i] = 1.0f - float(countNonZero(tmp) * areaInvert);
if (data[i] > matchingDegree)
{
SRect rc;
rc.left = i;
rc.top = j;
rc.right = i + imgTemplate.cols;
rc.bottom = j + imgTemplate.rows;
rcOuts.push_back(rc);
if ( data[i] > maxVal)
{
maxVal = data[i];
maxIndex = rcOuts.size() - 1;
}
if (++matchCount == maxMatchs)
{
Log_Warn("Too many matches, stopped at: " << matchCount);
return true;
}
}
}
}
It says I have not enough reputations to post image....
http://i.stack.imgur.com/mJrqU.png
New added:
I success optimize the algorithm by using key points. Calculate all the points is cost, but it is faster to calculate only server key points. See the picture, the costs decrease greatly, now it is about 7ms.
I still can not post image, please visit: http://i.stack.imgur.com/ePcD9.png
Please give me reputations, so I can post images. :)
There is a technical formulation for template matching with mask in OpenCV Documentation, which works well. It can be used by calling cv::matchTemplate and its source code is also available under the Intel License.