I'm trying to find spheres from a point cloud with pcl::sacSegmentation using RANSAC. The cloud is scanned with an accurate terrestial scanner from one station. The cloud density is about 1cm. The best results so far are shown in the image below. As you can see the cloud contains 2 spheres (r=7,25cm) and a steel beam where the balls are attached.. I am able to find three sphere candidates whose inlier points are extracted from cloud in the image (You can see two circle shapes on the beam near the spheres).
Input point cloud. Inlier points extracted
So, it seems that I am close. Still the found sphere centers are too much (~10cm) away from the truth. Any suggestion how could I improve this? I have been tweaking the model parameters quite some time. Here are the parameters for the aforementioned results:
seg.setOptimizeCoefficients(true);
seg.setModelType(pcl::SACMODEL_SPHERE);
seg.setMethodType(pcl::SAC_RANSAC);
seg.setMaxIterations(500000);
seg.setDistanceThreshold(0.0020);
seg.setProbability(0.99900);
seg.setRadiusLimits(0.06, 0.08);
seg.setInputCloud(cloud);
I also tried to improve the results by including point normals in the model with no better results. Yet there are couple parameters more to adjust so there might be some combinations I had not tried.
I happily give you more information if needed.
Thaks
naikh0u
After some investigation I have come in to conclusion that I can't find spheres with SACSegmentation from a cloud that contains lot of other points that don't belong in any sphere shape. Like in my case the beam is too much for the algorithm.
Thus, I have to choose the points that show some potential being a part of a sphere shape. Also I think, I need to separate the points belonging in different spheres. I tested and saw that my code works pretty well if the input cloud has only sphere points for single sphere with some "natural" noise.
Some have solved this problem by first extracting all points belonging to planes and then searched for spheres. Others have used colors of the target (in case of camera) to extract only needed points.
Deleting plane points should work for my example cloud, but my application may have more complex shapes too so it may be too simple..
..Finally, I got satisfied results by clustering the cloud with pcl::EuclideanClusterExtraction and feeding the clusters for sphere search one by one.
Related
I have a set of 2D points of a known density I want to mesh by taking the holes in account. Basically, given the following input:
I want something link this:
I tried PCL ConcaveHull, but it doens't handle the holes and splitted mesh very well.
I looked at CGAL Alpha shapes, which seems to go in the right direction (creating a polygon from a point cloud), but I don't know how to get triangles after that.
I though of passing the resulting polygons to a constrained triangulation algorithm and mark domains, but I didn't find how to get a list of polygons.
The resulting triangulated polygon is about a two step process at the least. First you need to triangulate your 2D points (using something like a Delaunay2D algorithm). There you can set the maximum length for the triangles and get the the desired shape. Then you can decimate the point cloud and re-triangulate. Another option is to use the convex hull to get the outside polygon, then extract the inside polygon through a TriangulationCDT algorithm, the apply some PolygonBooleanOperations, obtain the desired polygon, and finaly re-triangulate.
I suggest you look into the Geometric Tools library and specifically the Geometric Samples. I think everything you need is in there, and is much less library and path heavy than CGAL (the algorithms are not free for this type of work unless is a school project) or the PCL (I really like the library for segmentation, but their triangulation breaks often and is slow).
If this solves your problem, please mark it as your answer. Thank you!
I am currently using the Point Cloud Library (PCL) in order to do some work with point clouds. Now I need to compute a mesh for some point cloud and thought that the best thing to do is to use Meshlab. So far so good, my problem is that my point cloud has labels, i.e. it is of the following form:
pcl::PointCloud<pcl::PointXYZRGBL> cloud;
Important: I cannot omit the labels, I have to know after the mesh is computed, which point of the mesh has which label. Later, after some manipulation etc. I save this cloud via
pcl::io::savePLYFileBinary(writePath, *cloud);
which works fine IF the cloud is of type
pcl::PointCloud<pcl::PointXYZRGB> cloud;
but does not work for the first case. Does anyone have some idea what I could do to be able to still get a PLY file which contains labels and can be loaded into Meshlab?
Thanks all!
As MeshLab is not able to open your labeled points cloud, I'd suggest to:
Export your point cloud to a format readable by MeshLab (for example, the pcl::PointCloud<pcl::PointXYZRGB> you mentioned).
Reconstruct the triangle mesh using an interpolating method such as a ball pivoting. The interpolating method is necessary in order to preserve the original points as the vertices of the mesh. When you've finished, save the mesh.
Load the mesh at match the vertices with your original point cloud so you can recover the labels and any other associated attribute. In the quick test I've made even the vertices order matches the points' one.
Update
You mentioned in a comment that you were using the Screened Poisson Reconstruction. This method uses input points as positional constraints to improve the precision of the method, but it is still an approximating method, so output vertices are not guaranteed to match input points (and probably won't do).
You can either switch to an interpolating method (if noise and outliers allow you), or to find the closest point for each vertex (using a 1-NN, as you are doing now) to label vertices.
Above is valid for all discrete values. You should also adjust other values, such as color, to better match the reconstruction (vertices not matching points). To do so you can interpolate the corresponding value from the k-NN.
I realize there are many cans of worms related to what I'm asking, but I have to start somewhere. Basically, what I'm asking is:
Given two photos of a scene, taken with unknown cameras, to what extent can I determine the (relative) warping between the photos?
Below are two images of the 1904 World's Fair. They were taken at different levels on the wireless telegraph tower, so the cameras are more or less vertically in line. My goal is to create a model of the area (in Blender, if it matters) from these and other photos. I'm not looking for a fully automated solution, e.g., I have no problem with manually picking points and features.
Over the past month, I've taught myself what I can about projective transformations and epipolar geometry. For some pairs of photos, I can do pretty well by finding the fundamental matrix F from point correspondences. But the two below are causing me problems. I suspect that there's some sort of warping - maybe just an aspect ratio change, maybe more than that.
My process is as follows:
I find correspondences between the two photos (the red jagged lines seen below).
I run the point pairs through Matlab (actually Octave) to find the epipoles. Currently, I'm using Peter Kovesi's
Peter's Functions for Computer Vision.
In Blender, I set up two cameras with the images overlaid. I orient the first camera based on the vanishing points. I also determine the focal lengths from the vanishing points. I orient the second camera relative to the first using the epipoles and one of the point pairs (below, the point at the top of the bandstand).
For each point pair, I project a ray from each camera through its sample point, and mark the closest covergence of the pair (in light yellow below). I realize that this leaves out information from the fundamental matrix - see below.
As you can see, the points don't converge very well. The ones from the left spread out the further you go horizontally from the bandstand point. I'm guessing that this shows differences in the camera intrinsics. Unfortunately, I can't find a way to find the intrinsics from an F derived from point correspondences.
In the end, I don't think I care about the individual intrinsics per se. What I really need is a way to apply the intrinsics to "correct" the images so that I can use them as overlays to manually refine the model.
Is this possible? Do I need other information? Obviously, I have little hope of finding anything about the camera intrinsics. There is some obvious structural info though, such as which features are orthogonal. I saw a hint somewhere that the vanishing points can be used to further refine or upgrade the transformations, but I couldn't find anything specific.
Update 1
I may have found a solution, but I'd like someone with some knowledge of the subject to weigh in before I post it as an answer. It turns out that Peter's Functions for Computer Vision has a function for doing a RANSAC estimate of the homography from the sample points. Using m2 = H*m1, I should be able to plot the mapping of m1 -> m2 over top of the actual m2 points on the second image.
The only problem is, I'm not sure I believe what I'm seeing. Even on an image pair that lines up pretty well using the epipoles from F, the mapping from the homography looks pretty bad.
I'll try to capture an understandable image, but is there anything wrong with my reasoning?
A couple answers and suggestions (in no particular order):
A homography will only correctly map between point correspondences when either (a) the camera undergoes a pure rotation (no translation) or (b) the corresponding points are all co-planar.
The fundamental matrix only relates uncalibrated cameras. The process of recovering a camera's calibration parameters (intrinsics) from unknown scenes, known as "auto-calibration" is a rather difficult problem. You'd need these parameters (focal length, principal point) to correctly reconstruct the scene.
If you have (many) more images of this scene, you could try using a system such as Visual SFM: http://ccwu.me/vsfm/ It will attempt to automatically solve the Structure From Motion problem, including point matching, auto-calibration and sparse 3D reconstruction.
I am currently reading into the topic of stereo vision, using the book of Hartley&Zimmerman alongside some papers, as I am trying to develop an algorithm capable of creating elevation maps from two images.
I am trying to come up with the basic steps for such an algorithm. This is what I think I have to do:
If I have two images I somehow have to find the fundamental matrix, F, in order to find the actual elevation values at all points from triangulation later on. If the cameras are calibrated this is straightforward if not it is slightly more complex (plenty of methods for this can be found in H&Z).
It is necessary to know F in order to obtain the epipolar lines. These are lines that are used in order to find image point x in the first image back in the second image.
Now comes the part were it gets a bit confusing for me:
Now I would start taking a image point x_i in the first picture and try to find the corresponding point x_i’ in the second picture, using some matching algorithm. Using triangulation it is now possible to compute the real world point X and from that it’s elevation. This process will be repeated for every pixel in the right image.
In the perfect world (no noise etc) triangulation will be done based on
x1=P1X
x2=P2X
In the real world it is necessary to find a best fit instead.
Doing this for all pixels will lead to the complete elevation map as desired, some pixels will however be impossible to match and therefore can't be triangulated.
What confuses me most is that I have the feeling that Hartley&Zimmerman skip the entire discussion on how to obtain your point correspondences (matching?) and that the papers I read in addition to the book talk a lot about disparity maps which aren’t mentioned in H&Z at all. However I think I understood correctly that the disparity is simply the difference x1_i- x2_i?
Is this approach correct, and if not where did I make mistakes?
Your approach is in general correct.
You can think of a stereo camera system as two points in space where their relative orientation is known. This are the optical centers. In front of each optical center, you have a coordinate system. These are the image planes. When you have found two corresponding pixels, you can then calculate a line for each pixel, wich goes throug the pixel and the respectively optical center. Where the two lines intersect, there is the object point in 3D. Because of the not perfect world, they will probably not intersect and one may use the point where the lines are closest to each other.
There exist several algorithms to detect which points correspond.
When using disparities, the two image planes need to be aligned such that the images are parallel and each row in image 1 corresponds to the same row in image 2. Then correspondences only need to be searched on a per row basis. Then it is also enough to know about the differences on x-axis of the single corresponding points. This is then the disparity.
I have various point clouds defining RT-STRUCTs called ROI from DICOM files. DICOM files are formed by tomographic scanners. Each ROI is formed by point cloud and it represents some 3D object.
The goal is to get 2D curve which is formed by plane, cutting ROI's cloud point. The problem is that I can't just use points which were intersected by plane. What I probably need is to intersect 3D concave hull with some plane and get resulting intersection contour.
Is there any libraries which have already implemented these operations? I've found PCL library and probably it should be able to solve my problem, but I can't figure out how to achieve it with PCL. In addition I can use Matlab as well - we use it through its runtime from C++.
Has anyone stumbled with this problem already?
P.S. As I've mentioned above, I need to use a solution from my C++ code - so it should be some library or matlab solution which I'll use through Matlab Runtime.
P.P.S. Accuracy in such kind of calculations is really important - it will be used in a medical software intended for work with brain tumors, so you can imagine consequences of an error (:
You first need to form a surface from the point set.
If it's possible to pick a 2d direction for the points (ie they form a convexhull in one view) you can use a simple 2D Delaunay triangluation in those 2 coordinates.
otherwise you need a full 3D surfacing function (marching cubes or Poisson)
Then once you have the triangles it's simple to calculate the contour line that a plane cuts them.
See links in Mesh generation from points with x, y and z coordinates
Perhaps you could just discard the points that are far from the plane and project the remaining ones onto the plane. You'll still need to reconstruct the curve in the plane but there are several good methods for that. See for instance http://www.cse.ohio-state.edu/~tamaldey/curverecon.htm and http://valis.cs.uiuc.edu/~sariel/research/CG/applets/Crust/Crust.html.