I've applied three different methods of getting sets of points as follows.
Every method produces a vector of Points. Each method is in a different color, red, blue, and green.
Here is the combined image, overlaying all 3 of the sets of points
As you can see in the combined image there are spots in which all three sets "agree" on (i.e are generally in the exact same spot). I would like to find these particular spots and combine them into a single coordinate. I'm not sure where to start with approaching this problem. I've looked into K-means clustering, but to me it seems the problem is that K-means will cluster all the points and take the average with surrounding points, shifting the cluster center from the original position. I could loop through all the points in all the vectors that store the points, but as these images get larger with more points, it becomes very costly and inefficient.
Does anybody have any tips on how to approach this problem? I've been using OpenCV with C++.
Notionally, what you want to do is consider the complete tripartite graph on the three sets of points with edges weighted by distance. Then select edges in order of weight until a triangle appears; call those points a corresponding set, choose (say) their centroid to represent them, and remove them from the graph. Stop when the edge length exceeds some tolerance.
The mathematical justification for this approach is that it is independent of point ordering (except in the unlikely case of problematic ties in distances between points).
The practical implementation of this algorithm (for a significant number of points) involves a search data structure that can quickly find nearby points (not just the nearest): bins of the threshold size, a quad trie, or a k-d tree would work. Probably you would create one for each point set and use the other sets’ points as query points.
Related
I hope you are doing well. I am stuck at one part of a visual effect program in C++, and wanted to ask for help.
I have an array of colors at random positions on an image. There can be any number of these "subpixels" that fall over top of any given pixel. The subpixels that overlap a pixel can be at any position within the pixel, since they're distributed randomly throughout the image. All I have access to is their position on the image and their color, which represents what the color should be at that precise subpixel point on the image.
I need to determine what color to make each pixel of the image. In other words, I need to interpolate what the color should be at the centre of each pixel.
Here is a diagram with an example of this on a 5x5 image:
I need to go from this:
To this:
If it aids your understanding, you can think of the first image as a series of random points whose color values were calculated using bilinear interpolation on the second image.
I am writing this in C++, and ideally it will be as fast as possible, but I welcome contributions in any language or just explained with symbols or words. It should be as accurate as possible, but I also welcome solutions that are slightly inaccurate in favour of performance or simplicity.
Please let me know if you need clarification on the problem.
Thank you.
I ended up finding quite a decent solution which, while it doesn't find the absolutely 100% technically correct color for each pixel, was more than good enough and acceptably fast, especially when I added multithreading.
I first create a vector for each pixel/cell that contains pointers to subpixels (points with known colors). When I create a subpixel, I add a pointer to it to the vector representing the pixel/cell that it overlaps and to each of the vectors representing pixels/cells directly adjacent to the pixel/cell that that it overlaps.
Then, I split each pixel/cell into n sub-cells (I found 8 works well). This is not as expensive as you might imagine, because I only have to calculate & compare the distance for those subpixels that are in that pixel/cell's subpixel pointer vector. For each sub-cell, I calculate which subpixel is the closest to its centre. That subpixel's color then contributes 1/nth of the color for that pixel/cell.
I found it was important to add the subpixel pointers to adjacent cell/pixel vectors, so that each sub-cell can take into account subpixels from adjacent pixels/cells. This even makes it produce a reasonable color when there are pixels/cells that have no subpixels overlapping them (as long as the neighboring pixels/cells do).
Thanks for all the comments so far; any ideas about how to speed this up would be appreciated as well.
I performed edge detection on images (with Python 2-7 and OpenCV 3.2) and have results like the following picture, i.e. one-pixel-wide edges not necessarily closed (can have "loose ends"), and with possible holes :
Now I would like to get the "derivative" of these edges, meaning the "slope" at each point, as in the following image :
For the moment, the only way I managed to do it is very locally. For each point of the edge (in red in next "zoomed" picture), I create a circle around it (in pink), mask the circle with the edge to get the red point's neighbors, then compute the slope of these two neighbors.
However, it can be quite messy if edges have holes (which they often do) or are close to other edges (which they often are) and masking all the points is pretty computationally intensive, so I wonder if there is a better way.
My first idea was spline interpolation, but you need to give as input an ordered list of points, which you can't have for a given edge unless you use a pixel neighbor tracking algorithm which can also get quite messy in case of not-that-good edges.
I also thought of findContours but it needs closed edges or else it yields the contour of a one-pixel-wide edge, i.e. two lines on both side of the edges, started at an arbitrary location on the edge, in short it's a mess.
Is there a cleaner and more efficient way than my actual method to achieve what I want ? Does OpenCV have any resources or is its job done after edge detection (I think the latter is more probable !) ?
P.S. : "I don't think there is a better way" is an answer I'm ready to accept !
So, if I understood everything correctly, what you need is an ordered list of your points free from holes, because after that it seems you know how to proceed to obtain your result. So, you should concentrate in getting an ordered gapless list.
FindContours does output an ordered list, but probably not in the order you need. It groups connected pixels with a TOP-DOWN / LEFT-RIGHT priority. So, it swipes each row sequentially, when it hits a white pixel, it finds the first contour. So, in your image, the first contour it finds is actually the one on the right, since it has the closest to 0 Y value.
In the case of this particular image, if you rotate it 90 degrees you'll realize that it will actually order your contours and points in the way you need. But will this always be the case? Only you can tell. If there is a pre-process method to apply to your images that will guarantee that findContours will order your pixels in the correct way, the rest will be easy. If not, I suggest you create your own pixel-connectivity algorithm that will work as you need it to, since all your problems depends on getting an ordered list.
Once you have the ordered list, just interpolate the missing pixels.
If you have an ordered set of pixels, "closing the gaps" is easy, since you just need to find the gaps and interpolate between them as an approximation that probably wont hurt your algorithm.
I have an algorithmic problem on a Cartesian plane.. I need to efficiently search for geometric shapes that intersect with a given point. There are several shapes(rectangle, circle, triangle and polygon) but those are not important, because the determining the actual point inclusion is not a problem here, I will implement those on my own. The problem lies in determining which shapes need to be verified for the inclusion with the given point. Iterating through all of my shapes on plane and running the point inclusion method on each one of them is inefficient as the number of instances of shapes will be quite large. My first idea was to divide the plane for segments(the plane is finite, but too large for any kind of 3D array) and when adding a shape to the database, i would determine which segments it would intersect with and save them within object of the shape. Then when the point for inclusion verification is given, I would only need to determine the segment in which the point is located and then verify the inclusion only with objects which intersect with that segment.
Is that the way to go? I don't know if the method I described is optimal or if i am not missing something. Any help would be appreciated..
Thanks in advance
P.S.: I will be writing this in C++. That is not really relevant as it is more of an algorithmic problem but I wanted to put that out if someone was curious...
The gridding approach can be used here.
See the plane as a raster image where you draw all your shapes using a scan conversion algorithm, making sure that all pixels even partially covered are filled. For every image pixel, keep a list of the shapes that filled it.
A query is then straightforward: find the pixel where the query point falls in time O(1) and check every shape in the list, in time O(K), where K is the list length, approximately equal to the number of intersecting shapes.
If your image is made of N² pixels and you have M objects having an average area A pixels, you will need to store N²+M.A list elements (a shape identifier + a link to the next). You will choose the pixel size to achieve a good compromise between accuracy and storage cost. In any case, you must limit yourself to N²<Q.M, where Q is the total number of queries, otherwise the cost of just initializing the image could exceed the total query time.
In case your scene is very sparse (more voids than shapes), you can use a compressed representation of the image, using a quadtree.
I have a set of point cloud, and I would like to test if there is a corner in a 3D room. So I would like to discuss my approach and if there is a better approach or not in terms of speed, because I want to test it on mobile phones.
I will try to use hough tranform to detect lines, then I will try to see if there are three lines that are intersecting and they make a two plane that are intersecting too.
If the point cloud data comes from a depth sensor, then you have a relatively dense sampling of your walls. One thing I found that works well with depth sensors (e.g. Kinect or DepthSense) is a robust version of the RANSAC procedure that #MartinBeckett suggested.
Instead of picking 3 points at random, pick one point at random, and get the neighboring points in the cloud. There are two ways to do that:
The proper way: use a 3D nearest neighbor query data structure, like a KD-tree, to get all the points within some small distance from your query point.
The sloppy but faster way: use the pixel grid neighborhood of your randomly selected pixel. This may include points that are far from it in 3D, because they are on a different plane/object, but that's OK, since this pixel will not get much support from the data.
The next step is to generate a plane equation from that group of 3D points. You can use PCA on their 3D coordinates to get the two most significant eigenvectors, which define the plane surface (the last eigenvector should be the normal).
From there, the RANSAC algorithm proceeds as usual: check how many other points in the data are close to that plane, and find the plane(s) with maximal support. I found it better to find the largest support plane, remove the supporting 3D points, and run the algorithm again to find other 'smaller' planes. This way you may be able to get all the walls in your room.
EDIT:
To clarify the above: the support of a hypothesized plane is the set of all 3D points whose distance from that plane is at most some threshold (e.g. 10 cm, should depend on the depth sensor's measurement error model).
After each run of the RANSAC algorithm, the plane that had the largest support is chosen. All the points supporting that plane may be used to refine the plane equation (this is more robust than just using the neighboring points) by performing PCA/linear regression on the support set.
In order to proceed and find other planes, the support of the previous iteration should be removed from the 3D point set, so that remaining points lie on other planes. This may be repeated as long as there are enough points and best plane fit error is not too large.
In your case (looking for a corner), you need at least 3 perpendicular planes. If you find two planes with large support which are roughly parallel, then they may be the floor and some counter, or two parallel walls. Either the room has no visible corner, or you need to keep looking for a perpendicular plane with smaller support.
Normal approach would be ransac
Pick 3 points at random.
Make a plane.
Check if each other point lies on the plane.
If enough are on the plane - recalculate a best plane from all these points and remove them from the set
If not try another 3 points
Stop when you have enough planes, or too few points left.
Another approach if you know that the planes are near vertical or near horizontal.
pick a small vertical range
Get all the points in this range
Try and fit 2d lines
Repeat for other Z ranges
If you get a parallel set of lines in each Z slice then they are probably have a plane - recalculate the best fit plane for the points.
I would first like to point out
Even though this is an old post, I would like to present a complementary approach, similar to Hough Voting, to find all corner locations, composed of plane intersections, jointly:
Uniformly sample the space. Ensure that there is at least a distance $d$ between the points (e.g. you can even do this is CloudCompare with a 'space' subsampling)
Compute the point cloud normals at these points.
Randomly pick 3 points from this downsampled cloud.
Each oriented point (point+plane) defines a hypothetical plane. Therefore, each 3 point picked define 3 planes. Those planes, if not parallel and not intersecting at a line, always intersect at a single point.
Create a voting space to describe the corner: The intersection of the 3 planes (the point) might a valid parameterization. So our parameter space has 3 free parameters.
For each 3 points cast a vote in the accumulator space to the corner point.
Go to (2) and repeat until all sampled points are exhausted, or enough iterations are done. This way we'll be casting votes for all possible corner locations.
Take the local maxima of the accumulator space. Depending on the votes, we'll be selecting the corners from intersection of the largest planes (as they'll receive more votes) to the intersection of small planes. The largest 4 are probably the corners of the room. If not, one could also consider the other local maxima.
Note that the voting space is a quantized 3D space and the corner location will be a rough estimate of the actual one. If desired, one could store the planes intersection at that very location and refine them (with iterative optimization similar to ICP or etc) to get a very fine corner location.
This approach will be quite fast and probably very accurate, given that you could refine the location. I believe it's the best algorithm presented so far. Of course this assumes that we could compute the normals of the point clouds (we can always do that at sample locations with the help of the eigenvectors of the covariance matrix).
Please also look here, where I have put out a list of plane-fitting related questions at stackoverflow:
3D Plane fitting algorithms
I'm currently using VTK to load several non-convex 3D objects (polyhedron as vtkPolyData) and I want to compute the minimum distances between pairs of these objects. For this I am using vtkSmartPointer<vtkDistancePolyDataFilter> with the two vtkPolyData as input.
My first question is: Am I assuming right, that this method computes the distance between each vertex-vertex-pair of both input objects (see http://www.vtk.org/Wiki/VTK/Examples/Cxx/PolyData/DistancePolyDataFilter)? I have read at several pages that one cannot compute the minimum distance between two 3D objects this way; is this right? If so, why wouldn't it work this way?
If it is possible to compute the minimum distance of the two objects with this example, then I have two further questions:
How can I determine the point at an input 3D object, where the distance is minimum to the second input object? In other words, how do I get the value from the distancePolyDataFilter-output with the minimum distance?
EDIT: I used another example to measure the distance between each vertex-vertex pair of the two vtkPolyData, so that I can now access the point of the first vtkPolyData, at which the distance to the second vtkPolyData is minimum:
double vtkImplicitPolyDataDistance::EvaluateFunction(double x[3])
But I don't know how to get the point of the second vtkPolyData (the corresponding point for the minimum distance).
Secondly, is there any common way to reduce the set of points in the two vtkPolyData, so that I don't have to compute/compare each vertex-vertex-pair? For each pair of 3D objects, for which I want to compute the minimum distance, I can roughly determine their relative position to wach other. For example I know that object two lies above object one in x-direction. But since the objects are non-convex I cannot say that the maximum x-value of object one is smaller than the minimum x-value of object two. I also know that my 3D objects do not intersect, so there is always a positive minimum distance between them (but again, as the objects are non-convex I cannot work with bounding boxes (or at least I cannot think of any way how to use them)).
I have read at several pages that one cannot compute the minimum distance between two 3D objects this way [between each vertex-vertex-pair of both input objects]; is this right? If so, why wouldn't it work this way?
I was wondering this same question and here is an example illustrating how two meshes can be closer than any of their vertex pairs are.
The red points are their closest points which are not vertices.
Sorry, I cannot speak to the VTK example.