I would like to apply Data-Oriented Design (based on e.g. this article) to my simple physics engine. And I'm focused on optimizing the collision testing as it is the most expensive part of it.
I've organized the bounding spheres that may collide with player into single vector:
struct Sphere{ //I don't split sphere into parts,
//as I usually access both position and radius in my calculations
Point3D position;
float radius;
};
std::vector<BoudingSphere> spheres;
and I test collisions with them inside single function/method. Everything looks clear to me to that point.
The problem is, I also have some more general structures like:
struct Polygon{ //it may e.g. represents the area or be used for more precise tests
std::vector<Point2D> points;
};
I guess it won't be a good practise to just create std::vector<Polygon> the same way, as nested vector (points) will take a lot of place in memory (reserving it).
On the other hand, I cannot assume that there are always 2,3,4 or 10 points (it differs a lot, with the maximum of about 20, but it's usually much less).
And I do not want to switch from Polygon general structure to e.g. series of triangles (as it is faster then separated triangles in many calculations).
What should I do then? I want to go with the spirit of Data-Oriented Design and use the memory/cache efficiently with my Polygon.
Do I have to get rid of the inner vector (points)? How if so?
There can't be a definitive answer to this question as it would require knowledge about the way you access the data in terms of when is it initialized, can it be changed after the initialization stage and many others. However, if your data is relatively stable and you are accessing your polygons in a consistent manner just iterating over all polygons or polygons belonging to one particular object, one approach may be to put the points of your polygons into a separate vector and just have the polygons store the beginning and the end indices to that array.
This way, there are a couple of things that needs to be accessed during traversal. First, the indices stored in the polygons. Second, the points themselves. Both of these accesses are likely to be cache-friendly if the polygons are also laid out in a vector. Similar approach can be applied to polygon sets - just store the polygons in a vector and have a (begin, end) pair in your game objects.
Related
I am working on a graph implementation for a C++ class I am taking. This is what I came up with so far:
struct Edge {
int weight;
Vertex *endpoints[2]; // always will have 2 endpoints, since i'm making this undirected
};
struct Vertex {
int data; // or id
list<Edge*> edges;
};
class Graph {
public:
// constructor, destructor, methods, etc.
private:
list<Vertex> vertices;
};
It's a bit rough at the moment, but I guess I'm wondering... Am I missing something basic? It seems a bit too easy at the moment, and usually that means I'm designing it wrong.
My thought is that a graph is just a list of vertices, which has a list edges, which will have a list of edges, which has two vertex end points.
Other than some functions I'll put into the graph (like: shortest distance, size, add a vertex, etc), am I missing something in the basic implementation of these structs/classes?
Sometimes you need to design stuff like this and it is not immediately apparent what the most useful implementation and data representation is (for example, is it better storing a collection of points, or a collection of edges, or both?), you'll run into this all the time.
You might find, for example, that your first constructor isn't something you'd actually want. It might be easier to have the Graph class create the Vertices rather than passing them in.
Rather than working within the class itself and playing a guessing game, take a step back and work on the client code first. For example, you'll want to create a Graph object, add some points, connect the points with edges somehow, etc.
The ordering of the calls you make from the client will come naturally, as will the parameters of the functions themselves. With this understanding of what the client will look like, you can start to implement the functions themselves, and it will be more apparent what the actual implementation should be
Comments about your implementation:
A graph is a collection of objects in which some pairs of objects are related. Therefore, your current implementation is one potential way of doing it; you model the objects and the relationship between them.
The advantages of your current implementation are primarily constant lookup time along an edge and generalizability. Lookup time: if you want to access the nth neighbor of node k, that can be done in constant time. Generalizability: this represents almost any graph someone could think of, especially if you replace the data type of weight and data with an object (or a Template).
The disadvantages of your current implementation are that it will probably be slower than ideal. Looking across an edge will be cheap, but still take two hops instead of one (node->edge->node). Furthermore, using a list of edges is going to take you O(d) time to look up a specific edge, where d is the degree of the graph. (Your reliance on pointers also require that the graph fits in the memory of one computer; you'd have trouble with Facebook's graphs or the US road network. I doubt that parallel computing is a concern of yours at this point.)
Concerns when implementing a graph:
However, your question asks whether this is the best way. That's a difficult question, as several specific qualities of a graph come in to play.
Edge Information: If the way in which vertices are related doesn't matter (i.e., there is no weight or value to an edge), there is little point in using edge objects; this will only slow you down. Instead, each vertex can just keep a list of pointers to its neighbors, or a list of the IDs of its neighbors.
Construction: As you noticed in the comments, your current implementation requires that you have a vertex available before adding an edge. That is true in general. But you may want to create vertices on the fly as you add edges; this can make the construction look cleaner, but will take more time if the vertices have non-constant lookup time. If you know all vertices before construction the graph, it may be beneficial to explicitly create them first, then the edges.
Density: If the graph is sparse (i.e., the number of edges per vertex is approximately constant), then an adjacency list is again a good method. However, if it is dense, you can often get increased performance if you use an adjacency matrix. Every vertex holds a list of all other vertices in order, and so accessing any edge is a constant time operation.
Algorithm: What problems do you plan on solving on the graph? Several famous graph algorithms have different running times based on how the graph is represented.
Addendum:
Have a look at this question for many more comments that may help you out:
Graph implementation C++
I'm currently coding a physical simulation on a lattice, I'm interested in describing loops in this lattice, they are closed curved composed by the edges of the lattice cells. I'm storing the information on this lattice cells (by information I mean a Boolean variable saying if the edge is valuable or no for composing a loop) in a 3 dimensional Boolean array.
I'm now thinking about a good structure to handle this loops. they are basically a list of edges, so I would need something like an array of 3d integer vectors, each edge being defined by 3 coordinates in my current parameterization. I'm already thinking about building a class around this "list" object as I'll need methods computing the loop diameter and probably more in the future.
But, I'm definitely not so aware of the choice of structure I have to do that, my physics background hasn't taught me enough in C++. And for so, I'd like to hear your suggestion for shaping this piece of code. I would really enjoy discovering some new ways of coding this kid of things.
You want two separate things. One is keeping track of all edges and allowing fast lookup of edge objects by an (int,int,int) index (you probably don't want int there but something like size_t or so). This is entirely independent from your second goal crating ordered subsets of these.
General Collection (1)
Since your edge database is going to be sparse (i.e. only a few of the possible indices will actually identify as a particular edge), my prior suggestion of using a 3d matrix is unsuitable. Instead, you probably want to lookup edges with a hash map.
How easy this is, depends on the expected size of the individual integers. That is, can you manage to have no more than 21 bit per integer (for instance if your integers are short int values, which have only 16 bit), then you can concatenate them to one 64 bit value, which already has an std::hash implementation. Otherwise, you will have to implement your own hash specialisation for, e.g., std::hash<std::array<uint32_t,3>> (which is also quite easy, and highly stackable).
Once you can hash your key, you can throw it into an std::unordered_map and be done with it. That thing is fast.
Loop detection (2)
Then you want to have short-lived data structures for identifying loops, so you want a data structure that extends on one end but never on the other. That means you're probably fine with an std::vector or possibly with an std::deque if you have very large instances (but try the vector first!).
I'd suggest simply keeping the index to an edge in the local vector. You can always lookup the edge object in your unordered_map. Then the question is how to represent the index. If Int represents your integer type (e.g. int, size_t, short, ...) it's probably the most consistent to use an std::array<Int,3> --- if the types of the integers differ, you'll want an std::tuple<...>.
I'm working on a game where exactly one object may exist at location (x, y) where x and y are ints. For example, an object may exist at (0, 0) or it may not, but it is not possible for multiple objects to exist there at once.
I am trying to decide which STL container to use for the problem at hand and the best way to solve this problem.
Basically, I start with an object and its (x, y) location. The goal is to determine the tallest, largest possible rectangle based on that object's surrounding objects. The rectangle must be created by using all objects above and below the current object. That is, it must be the tallest that it can possibly be based on the starting object position.
For example, say the following represents my object grid and I am starting with the green object at location (3, 4):
Then, the rectangle I am looking for would be represented by the pink squares below:
So, assuming I start with the object at (3, 4) like the example shows, I will need to check if objects also exist at (2, 4), (4, 4), (3, 3), and (3, 5). If an object exists at any of those locations, I need to repeat the process for the object to find the largest possible rectangle.
These objects are rather rare and the game world is massive. It doesn't seem practical to just new a 2D array for the entire game world since most of the elements would be empty. However, I need to be to index into any position to check if an object is there at any time.
Instead, I thought about using a std::map like so:
std::map< std::pair<int, int>, ObjectData> m_objects;
Then, as I am checking the surrounding objects, I could use map::find() in my loop, checking if the surrounding objects exist:
if(m_objects.find(std::pair<3, 4>) != m_objects.end())
{
//An object exists at (3, 4).
//Add it to the list of surrounding objects.
}
I could potentially be making a lot of calls to map::find() if I decide to do this, but the map would take up much less memory than newing a 2D array of the entire world.
Does anyone have any advice on a simple algorithm I could use to find what I am looking for? Should I continue using a std::map or is there a better container for a problem like this?
How much data do you need to store at each grid location? If you are simply looking for a flag that indicates neighbors you have at least two "low tech" solutions
a) If your grid is sparse, how about each square keeps a neighbor list? So each square knows which neighboring squares are occupied. You'll have some work to do to maintain the lists when a square is occupied or vacated. But neighbor lists mean you don't need a grid map at all
b) If the grid map locations are truly just points, use 1 bit per grid location. The results map will be 8x8=64 times smaller that one that uses bytes for each grid point. Bit operations are lightening fast. A 10,000x10,000 map will take 100,000,000 bits or 12.5MB (approx)
An improvement would be to use a hashmap, if possible. This would allow you to at least do your potential extensive searches with an expected time complexity of O(1).
There's a thread here ( Mapping two integers to one, in a unique and deterministic way) that goes into some detail about how to hash two integers together.
If your compiler supports C++11, you could use std::unordered_map. If not, boost has basically the same thing: http://www.boost.org/doc/libs/1_38_0/doc/html/boost/unordered_map.html
You may want to consider a spatial data structure. If the data is 'sparse', as you say, then doing a quadtree neighbourhood search might save you a lot of processing power. I would personally use an R-tree, but that's most likely because I have an R-tree library that I've written and can easily import.
For example, suppose you have a 1000x1000 grid with 10,000 elements. Assuming for the moment, a uniformly-random distribution, we would (based on the density) expect no more than, say . . . a chain of three to five objects touching in either dimension (at this density, a chain of three vertically-oriented objects will happen with probability 0.01% of the time). Suppose the object under consideration is located at (x,y). A window search, starting at (x-5,y-5) and going to (x+5,y+5) would give you a list of at most 121 elements to perform a linear search through. If your rect-picking algorithm notices that it would be possible to form a taller rectangle (i.e. if a rect under consideration touches the edges of this 11x11 bounding box), just repeat the window search for another 5x5 region in one direction of the original. Repeat as necessary.
This, of course, only works well when you have extremely sparse data. It might be worth adapting an R-tree such that the leaves are an assoc. data structure (i.e. Int -> Int -> Object), but at that point it's probably best to just find a solution that works on denser data.
I'm likely over-thinking this; there is likely a much simpler solution around somewhere.
Some references on R-trees:
The original paper, for the original algorithms.
The Wikipedia page, which has some decent overview on the topic.
The R-tree portal, for datasets and algorithms relating to R-trees.
I'll edit this with a link to my own R-tree implementation (public domain) if I ever get around to cleaning it up a little.
This sounds suspiciously like a homework problem (because it's got that weird condition "The rectangle must be created by using all objects above and below the current object" that makes the solution trivial). But I'll give it a shot anyway. I'm going to use the word "pixel" instead of "object", for convenience.
If your application really deserves heavyweight solutions, you might try storing the pixels in a quadtree (whose leaves contain plain old 2D arrays of just a few thousand pixels each). Or you might group contiguous pixels together into "shapes" (e.g. your example would consist of only one "shape", even though it contains 24 individual pixels). Given an initial unstructured list of pixel coordinates, it's easy to find these shapes; google "union-find". The specific benefit of storing contiguous shapes is that when you're looking for largest rectangles, you only need to consider those pixels that are in the same shape as the initial pixel.
A specific disadvantage of storing contiguous shapes is that if your pixel-objects are moving around (e.g. if they represent monsters in a roguelike game), I'm not sure that the union-find data structure supports incremental updates. You might have to run union-find on every "frame", which would be pretty bad.
Anyway... let's just say you're using a std::unordered_map<std::pair<int,int>, ObjectData*>, because that sounds pretty reasonable to me. (You should almost certainly store pointers in your map, not actual objects, because copying around all those objects is going to be a lot slower than copying pointers.)
typedef std::pair<int, int> Pt;
typedef std::pair<Pt, Pt> Rectangle;
std::unordered_map<Pt, ObjectData *> myObjects;
/* This helper function checks a whole vertical stripe of pixels. */
static bool all_pixels_exist(int x, int min_y, int max_y)
{
assert(min_y <= max_y);
for (int y = min_y; y <= max_y; ++y) {
if (myObjects.find(Pt(x, y)) == myObjects.end())
return false;
}
return true;
}
Rectangle find_tallest_rectangle(int x, int y)
{
assert(myObjects.find(Pt(x,y)) != myObjects.end());
int top = y;
int bottom = y;
while (myObjects.find(Pt(x, top-1) != myObjects.end()) --top;
while (myObjects.find(Pt(x, bottom+1) != myObjects.end()) ++bottom;
// We've now identified the first vertical stripe of pixels.
// The next step is to "paint-roller" that stripe to the left as far as possible...
int left = x;
while (all_pixels_exist(left-1, top, bottom)) --left;
// ...and to the right.
int right = x;
while (all_pixels_exist(right+1, top, bottom)) ++right;
return Rectangle(Pt(top, left), Pt(bottom, right));
}
I have a 3D point clouds with million of points. I want to store these points in 3D voxel space. The number of voxles along coordinate axis are more than 3000(x), 4000(y), 1500(z), for a total of 3000*4000*1500 voxels. I need to store in a voxel; maximum number of points, min height, max height and centorid. However, 90% of voxels are empty. So it takes lot of memory to store this. Actually, I want to search 26 neighbor voxels of each voxels in later. So What is the best way to store this data in voxel space and get access to this efficiently?
Creating a multidimensional array, is not the best solution, in term of performance...please any hint?
Classical data structures for this kind of data are kd-Trees and octrees..
Also, you should definitely take a look at the spatial searching and sorting data structures implemented in CGAL.
If it's really "just" millions of points, far more than 90% of the voxels will empty. I'd try a hashed multimap (std::unordered_multimap in C++11) from voxel coordinates to points. That gives you O(1) lookup, like an array. It comes with quite a lot overhead though, but it's probably the best compromise.
The only thing you need for this to work is an equality comparison in the voxel class, and a template specialisation std::hash<voxel>. You won't get "maximum number of points" implemented in any automatical way, but is that really useful anyway?
One approach would be to back your actual data with data from a collection.
To illustrate:
struct t_voxel {
size_t nPoints, minHeight, maxHeight, centorid;
};
struct t_voxel_id {
uint16_t index;
};
// one dimension
class t_voxel_collection {
// the actual voxel data needed for the indices represented by the collection of voxelId
std::vector<t_voxel> d_voxel;
// here, empty voxel is designated by t_voxel.index = 0
// this collection is your primary array representation
// these elements just refer to a unique or shared index in this->d_voxel
std::vector<t_voxel_id> d_voxelId;
public:
// >> the interface to access and set, which abstracts the backing collection.
// and prohibits the client from accessing the actual data.
t_voxel get(const size_t& idx) const {
return this->d_voxel[this->d_voxelId[idx].index];
}
// ...
};
You can achieve a big drop in memory consumption this way (assuming you see the direction this is going).
That's not a complete answer, but could help in this scenario.
There are several ways to further optimize and share the voxel data in this collection, depending on your use.
You're going to become unstuck whatever you do, even if you find a perfect memory layout for your sparse grid - that's still too much memory required. The real issue is being able to efficiently store it on disk and intelligently cache the regions of interest.
Thankfully Field3D was developed for just that.
I have been looking for a quadtree/quadtree node implementation on the net for ages. There is some basic stuff but nothing that I would be able to really use it a game.
My purpose is to store objects in a game for processing things such as collision detection.
I am not 100% certain that a quadtree is the best data structure to use, but from what I have read it is. I have already coded a Red-Black tree, but I don't really know if the performance would be good enough for my game (which will be an adventure 3rd person game like Ankh).
How would I write a basic but complete quadtree class (or octree) in C++?
How would you use the quad tree for collisions?
Quadtrees are used when you only need to store things that are effectively on a plane. Like units in a classic RTS where they are all on the ground or just a little bit above it. Essentially each node has links to 4 children that divide the node's space up into evenly distributed quarters.
Octrees do the same but in all three dimensions rather than just two, and thus they have 8 child nodes and partition the space up into eights. They should be used when the game entities are distributed more evenly among all three dimensions.
If you are looking for a binary tree - like a red-black tree - then you want to use a data structure called a binary space partitioning tree (BSP tree) or a version of it called the KD Tree. These partition space into halves using a plane, in the KD tree the planes are orthogonal (on the XZ, XY, ZY axes) so sometimes it works better in a 3D scene. BSP trees divide the scene up using planes in any orientation, but they can be quite useful, and they were used as far back as Doom.
Now because you've partitioned the game space you now don't have to test every game entity against every other game entity to see if they collide, which is an O(n^2) algorithm at best. Instead you query the data structure to return the game entities within a sub-region of the game space, and only perform collision detection for those nodes against each other.
This means that collision detection for all game entities should be n O(nlogn) operation (at worst).
A couple of extra things to watch out for:
Make sure you test game entities from adjacent nodes, not just the ones in the current node, since they could still collide.
Rebalance the data structure after the entities have moved since you may have empty nodes in the data structure now, or ones that contain too many entities for good performance (also the degenerate case of all entities being in the same node).
A red-black tree is not a spatial index; it can only sort on a single ordinal key. A quadtree is (for two dimensions) a spatial index that allows fast lookup and elimination of points. An Octree does the same thing for three dimensions.
The reason to use a quadtree is because you can then split on x- and y-coordinates, an octree on x, y and z, making collision detection trivial.
Quadtree: if an element is not in the topleft, it wont collide with one in topright, bottomleft or bottomright.
It is a very basic class, so I don't understand what you are missing in implementations you found.
I would not write such a class, I'd just borrow it from a project with a suitable license.
I warmly suggest you to use a rendering engine, Ogre3D for instance. As far as I know it supports Octrees for scene management. But you can extend the Octree-based class as you wish. I used to code the stuff I needed by myself, but for complex projects, it's just not the right way.
Trees in general are problematic for this in that any item inserted can lie on a boundary, and all the methods of dealing with that situation are fairly unsatisfactory.
You'll most likely want to sort your objects into moveable and static, and check anything that moved on a given frame against the static objects.
BSP Trees are the accepted solution for static geometry (boundary cases handled by splitting the object into two pieces), for dynamic try something like Sort and Sweep (also known as Sweep and Prune).
Right now STANN is the best open source implementation.
http://sites.google.com/a/compgeom.com/stann/