I need to represent a tree with multiple branches per node. What structure should I use? It's for computing chess game states. It explodes exponentially so memory will be a concern. I'm using C++11 but am open to other standards. Also, pruning should be O(1).
EDIT1
To expand, I am going to be holding a Chess AI competition. The main PvP game is complete already, and I am programming the AI API next. Contestants will write their own AI, and then we will have them compete in a tournament. The winner's AI will be used in Player vs Computer games. I am just thinking about the best structure to store my game states and AI thoughts.
I was reading up on Deep Blue, and it thinks from 5 to ~25 moves ahead. I can imagine most computers capable of handling 5 moves deep with BFS, but anything deeper and I believe I will have to use DFS.
AI's will be timed, and competing AI's will only be played locally, so as not to introduce advantages in CPU power.
I am reading up on Monte Carlo and Alpha Beta searches now.
My initial thoughts on a data structure are as follows :
union CHESS_MOVE {
unsigned short m;
ChessPosT pos[2];
///...
};
class ChessMoveNode {
CHESS_MOVE move;
std::set<ChessMoveNode> nextmoves;
};
class ChessMoveTree {
std::set<ChessMoveNode> next;
};
The board can be calculated at any time by concatenating the path from the root to the leaf. Although recalculating the board could get very expensive over time. Ideas? Should I store the board? The board is stored as an array of 64 char indices holding a piece number. So it's 16 bytes, compared to 2, but the memory use would save a lot of re-calculation of the board state.
For my own personal AI, I will be implementing a board scoring function that will rank the game states, and then all non maximal game states will be discarded, as well as pruning game states that are invalidated by choosing a move.
One simple approach to do this that works well for Monte-Carlo Tree Search (MCTS) is to use a vector of some custom class. Inside the class you have whatever state information you need in addition to child information -- the number of the children and their index in the vector. This avoids storing a separate pointer for each child, which can introduce significant overhead.
So, the root is at index 0. Inside that index there would be two integers indicating that the children start at index 1 and that there are k children. (From index 1 to k.) At index 1 the children would start at index k+1 with l total children, and so on throughout the tree.
This works really well based on the assumptions that (1) the number of children is fixed, (2) that they are all added at once, and (3) that states are not removed from the tree.
If you are trying to prune states from the tree, this doesn't work as well, because you will leave gaps in tree if you remove them. Using explicit pointers for storing each child is expensive, so something else is done in practice.
First, with alpha-beta search you typically search the tree with a DFS and don't store branches. But, you use a hash table to store states and check for duplicates. The branches of the tree can be implicitly computed from the state, so you can reconstruct the tree without storing everything explicitly.
Note, however, that hash tables (called transposition tables in the context of game tree search) are not typically used deep in the tree because there are many states and the cost of storing grows while the benefit of removing duplicates shrinks.
To summarize, based on the assumption that you are doing something alpha-beta like and you have a good reason to store the tree explicitly, I suggest store the states in a hash table and leave the edges to be implicitly computed from a move-generation function. (Which would apply moves and take the hash of the resulting state to find them in the hash table. If they aren't there they have been pruned.)
Related
I'm exploring a tree of moves for a match-3 type game (with horizontal piece swapping), using a DFS to a depth of 4 moves ahead (because working towards bigger clear combos is much higher scoring). I'm evaluating each possible move of each board to look for the best overall next move to maximise score.
I'm representing the game board as a vector<char> with 100 chars per board, however my current DFS implementation doesn't check if a given board state has already been evluated (there's multiple move sequences that could lead to a given board state).
Given that each board has 90 possible moves, a DFS search to depth 4 evaluates 65,610,000 board states (in 100ms), and therefore I'd assume it's impractical to store each board state for the sake of DFS pruning to avoid re-evaluating a given board state. However, I recognise I could significantly prune the search tree if I avoid re-evaluation of previously evaluated states.
Is there an efficient and or memory conserving method to avoid re-evaluating previously evaluated board states in this DFS?
I am designing a Graph in c++ using a hash table for its elements. The hashtable is using open addressing and the Graph has no more than 50.000 edges. I also designed a PRIM algorithm to find the minimum spanning tree of the graph. My PRIM algorithm creates storage for the following data:
A table named Q to put there all the nodes in the beginning. In every loop, a node is visited and in the end of the loop, it's deleted from Q.
A table named Key, one for each node. The key is changed when necessary (at least one time per loop).
A table named Parent, one for each node. In each loop, a new element is inserted in this table.
A table named A. The program stores here the final edges of the minimum spanning tree. It's the table that is returned.
What would be the most efficient data structure to use for creating these tables, assuming the graph has 50.000 edges?
Can I use arrays?
I fear that the elements for every array will be way too many. I don't even consider using linked lists, of course, because the accessing of each element will take to much time. Could I use hash tables?
But again, the elements are way to many. My algorithm works well for Graphs consisting of a few nodes (10 or 20) but I am sceptical about the situation where the Graphs consist of 40.000 nodes. Any suggestion is much appreciated.
(Since comments were getting a bit long): The only part of the problem that seems to get ugly for very large size, is that every node not yet selected has a cost and you need to find the one with lowest cost at each step, but executing each step reduces the cost of a few effectively random nodes.
A priority queue is perfect when you want to keep track of lowest cost. It is efficient for removing the lowest cost node (which you do at each step). It is efficient for adding a few newly reachable nodes, as you might on any step. But in the basic design, it does not handle reducing the cost of a few nodes that were already reachable at high cost.
So (having frequent need for a more functional priority queue), I typically create a heap of pointers to objects and in each object have an index of its heap position. The heap methods all do a callback into the object to inform it whenever its index changes. The heap also has some external calls into methods that might normally be internal only, such as the one that is perfect for efficiently fixing the heap when an existing element has its cost reduced.
I just reviewed the documentation for the std one
http://en.cppreference.com/w/cpp/container/priority_queue
to see if the features I always want to add were there in some form I hadn't noticed before (or had been added in some recent C++ version). So far as I can tell, NO. Most real world uses of priority queue (certainly all of mine) need minor extra features that I have no clue how to tack onto the standard version. So I have needed to rewrite it from scratch including the extra features. But that isn't actually hard.
The method I use has been reinvented by many people (I was doing this in C in the 70's, and wasn't first). A quick google search found one of many places my approach is described in more detail than I have described it.
http://users.encs.concordia.ca/~chvatal/notes/pq.html#heap
Suppose you have an input file:
<total vertices>
<x-coordinate 1st location><y-coordinate 1st location>
<x-coordinate 2nd location><y-coordinate 2nd location>
<x-coordinate 3rd location><y-coordinate 3rd location>
...
How can Prim's algorithm be used to find the MST for these locations? I understand this problem is typically solved using an adjacency matrix. Any references would be great if applicable.
If you already know prim, it is easy. Create adjacency matrix adj[i][j] = distance between location i and location j
I'm just going to describe some implementations of Prim's and hopefully that gets you somewhere.
First off, your question doesn't specify how edges are input to the program. You have a total number of vertices and the locations of those vertices. How do you know which ones are connected?
Assuming you have the edges (and the weights of those edges. Like #doomster said above, it may be the planar distance between the points since they are coordinates), we can start thinking about our implementation. Wikipedia describes three different data structures that result in three different run times: http://en.wikipedia.org/wiki/Prim's_algorithm#Time_complexity
The simplest is the adjacency matrix. As you might guess from the name, the matrix describes nodes that are "adjacent". To be precise, there are |v| rows and columns (where |v| is the number of vertices). The value at adjacencyMatrix[i][j] varies depending on the usage. In our case it's the weight of the edge (i.e. the distance) between node i and j (this means that you need to index the vertices in some way. For instance, you might add the vertices to a list and use their position in the list).
Now using this adjacency matrix our algorithm is as follows:
Create a dictionary which contains all of the vertices and is keyed by "distance". Initially the distance of all of the nodes is infinity.
Create another dictionary to keep track of "parents". We use this to generate the MST. It's more natural to keep track of edges, but it's actually easier to implement by keeping track of "parents". Note that if you root a tree (i.e. designate some node as the root), then every node (other than the root) has precisely one parent. So by producing this dictionary of parents we'll have our MST!
Create a new list with a randomly chosen node v from the original list.
Remove v from the distance dictionary and add it to the parent dictionary with a null as its parent (i.e. it's the "root").
Go through the row in the adjacency matrix for that node. For any node w that is connected (for non-connected nodes you have to set their adjacency matrix value to some special value. 0, -1, int max, etc.) update its "distance" in the dictionary to adjacencyMatrix[v][w]. The idea is that it's not "infinitely far away" anymore... we know we can get there from v.
While the dictionary is not empty (i.e. while there are nodes we still need to connect to)
Look over the dictionary and find the vertex with the smallest distance x
Add it to our new list of vertices
For each of its neighbors, update their distance to min(adjacencyMatrix[x][neighbor], distance[neighbor]) and also update their parent to x. Basically, if there is a faster way to get to neighbor then the distance dictionary should be updated to reflect that; and if we then add neighbor to the new list we know which edge we actually added (because the parent dictionary says that its parent was x).
We're done. Output the MST however you want (everything you need is contained in the parents dictionary)
I admit there is a bit of a leap from the wikipedia page to the actual implementation as outlined above. I think the best way to approach this gap is to just brute force the code. By that I mean, if the pseudocode says "find the min [blah] such that [foo] is true" then write whatever code you need to perform that, and stick it in a separate method. It'll definitely be inefficient, but it'll be a valid implementation. The issue with graph algorithms is that there are 30 ways to implement them and they are all very different in performance; the wikipedia page can only describe the algorithm conceptually. The good thing is that once you implement it some way, you can find optimizations quickly ("oh, if I keep track of this state in this separate data structure, I can make this lookup way faster!"). By the way, the runtime of this is O(|V|^2). I'm too lazy to detail that analysis, but loosely it's because:
All initialization is O(|V|) at worse
We do the loop O(|V|) times and take O(|V|) time to look over the dictionary to find the minimum node. So basically the total time to find the minimum node multiple times is O(|V|^2).
The time it takes to update the distance dictionary is O(|E|) because we only process each edge once. Since |E| is O(|V|^2) this is also O(|V|^2)
Keeping track of the parents is O(|V|)
Outputting the tree is O(|V| + |E|) = O(|E|) at worst
Adding all of these (none of them should be multiplied except within (2)) we get O(|V|^2)
The implementation with a heap is O(|E|log(|V|) and it's very very similar to the above. The only difference is that updating the distance is O(log|V|) instead of O(1) (because it's a heap), BUT finding/removing the min element is O(log|V|) instead of O(|V|) (because it's a heap). The time complexity is quite similar in analysis and you end up with something like O(|V|log|V| + |E|log|V|) = O(|E|log|V|) as desired.
Actually... I'm a bit confused why the adjacency matrix implementation cares about it being an adjacency matrix. It could just as well be implemented using an adjacency list. I think the key part is how you store the distances. I could be way off in my implementation outlined above, but I am pretty sure it implements Prim's algorithm is satisfies the time complexity constraints outlined by wikipedia.
I am trying to implement a Kd tree to perform the nearest neighbor and approximate nearest neighbor search in C++. So far I came across 2 versions of the most basic Kd tree.
The one, where data is stored in nodes and in leaves, such as here
The one, where data is stored only in leaves, such as here
They seem to be fundamentally the same, having the same asymptotic properties.
My question is: are there some reasons why choose one over another?
I figured two reasons so far:
The tree which stores data in nodes too is shallower by 1 level.
The tree which stores data only in leaves has easier to
implement delete data function
Are there some other reasons I should consider before deciding which one to make?
You can just mark nodes as deleted, and postpone any structural changes to the next tree rebuild. k-d-trees degrade over time, so you'll need to do frequent tree rebuilds. k-d-trees are great for low-dimensional data sets that do not change, or where you can easily afford to rebuild an (approximately) optimal tree.
As for implementing the tree, I recommend using a minimalistic structure. I usually do not use nodes. I use an array of data object references. The axis is defined by the current search depth, no need to store it anywhere. Left and right neighbors are given by the binary search tree of the array. (Otherwise, just add an array of byte, half the size of your dataset, for storing the axes you used). Loading the tree is done by a specialized QuickSort. In theory it's O(n^2) worst-case, but with a good heuristic such as median-of-5 you can get O(n log n) quite reliably and with minimal constant overhead.
While it doesn't hold as much for C/C++, in many other languages you will pay quite a price for managing a lot of objects. A type*[] is the cheapest data structure you'll find, and in particular it does not require a lot of management effort. To mark an element as deleted, you can null it, and search both sides when you encounter a null. For insertions, I'd first collect them in a buffer. And when the modification counter reaches a threshold, rebuild.
And that's the whole point of it: if your tree is really cheap to rebuild (as cheap as resorting an almost pre-sorted array!) then it does not harm to frequently rebuild the tree.
Linear scanning over a short "insertion list" is very CPU cache friendly. Skipping nulls is very cheap, too.
If you want a more dynamic structure, I recommend looking at R*-trees. They are actually desinged to balance on inserts and deletions, and organize the data in a disk-oriented block structure. But even for R-trees, there have been reports that keeping an insertion buffer etc. to postpone structural changes improves performance. And bulk loading in many situations helps a lot, too!
Efficient way of threading an octree such that the pointers contained by each octcell in an oct make it easy in the traversal through the tree at the same level.
We have to make use of fully threaded trees here so that i can use openmp to parallelize the code at the same level.
I have some experience with oct-trees and coded several myself. The basic problem is that the tree has (at least) two directions of traversal: horizontal (between daughter cells) and vertical (between mother and daughter cells), which cannot be mapped to linear memory. Thus, traversing the tree (for example for neighbour search) will inevitably result in cache misses.
For a most efficient implementations, you should have all (up to 8) daughter cells of a non-final cell to be in one contiguous block of memory, avoiding both cache misses when traversing over them and the need to link them up with pointers. Each cell then only need one pointer/index for their first daughter cell and, possibly (depending on the needs of your applications), a pointer to their mother cell.
Similarly, any particles/positions sorted by the tree should be ordered such that all contained within a cell are contiguous in memory, at all tree levels. Then each cell only has to store the first and number of particles, allowing access to all them at at every level of the tree (not just final cells).
In practice, such an ordering can be achieved by first building a fully linked tree and then mapping it to the form described above. The overhead of this mapping is minor but the gain in a applications substantial.
Finally, when re-building the tree with only slightly changed particle positions, it makes for a significant speed up (depending on your algorithm) to feed the particles in the previous tree order to the tree building algorithm.