I am looking for the most effective way to generate all routes between nodes inside a vector. Imagine the following (pseudo-)code:
vector<vector<Node>> step(vector<Node> list, Node nextStep)
{
// 1. Erase current Node to avoid an infinite loop
for(vector<Node>::iterator it = list.begin(); it != list.end(); ++it)
{
if((*it) == nextStep)
{
list.erase(e);
break;
}
}
// 2. Dig deeper into the list if more elements are available
vector<vector<Node>> returnVector;
if(list.size() > 0)
{
for(vector<Node>::iterator it = list.begin(); it != list.end(); ++it)
{
if(/* (*it) meets certain conditions*/)
{
vector<vector<Node>> tmp = step(list, (*it)); // Here the function calls itself
for(vector<vector<Node>>::iterator n = tmp.begin(); n != tmp.end(); ++n)
{
// 3. Insert "nextStep" at the beginning of the new path
(*n).insert((*n).begin(), nextStep);
// 4. Add new path to returnVector
returnVector.push_back((*n));
}
if(/* "nextStep" meets certain conditions */)
{
// 5. Add "nextStep" as Node aswell
returnVector.push_back(std::vector<Node>{nextStep});
}
}
}
}
return returnVector;
}
/*****
.
.
.
.
*/
void run()
{
vector<Node> v(100, Node());
for(auto n : v)
{
step(v, n);
}
}
This function calculates all routes in a vector recursively. I have already tested it and it works quite well - as long as the count of elements inside the vector does not get too big.
I am very new to this area of programming (recursion and functions where performance actually really matters). As I have already written above I am looking for a more effective way to do the same task. The size of the vector can get quite big and the perfomance suffers drastically, blowing up CPU and RAM usage which is probably because of the recursion.
I know that there can be very many combinations, but maybe there are already better algorithms for this task.
Related
I'm working on a D* Lite path planner in C++. The program maintains a priority queue of cells (U), each cell have two cost values, and a key can be calculated for a cell which determine it's order on the priority queue.
using Cost = float;
using HeapKey = pair<Cost, Cost>;
using KeyCompare = std::greater<std::pair<HeapKey, unsigned int>>;
vector<pair<HeapKey, unsigned int>> U;
When a cell is added it is done so by using:
U.push_back({ k, id });
push_heap(U.begin(), U.end(), KeyCompare());
As part of the path planning algorithm cells sometimes need to be removed, and here lies the current problem as far as I can see. I recently had help on this site to speed my program up quite a bit by using push_heap instead of make_heap, but now it seems that the part of the program that removes cells is the slowest part. Cells are removed from the priority queue by:
void DstarPlanner::updateVertex(unsigned int id) {
...
...
auto it = find_if(U.begin(), U.end(), [=](auto p) { return p.second == id; });
U.erase(it);
...
...
}
From my tests this seems to take roughly 80% of the time my program use for path planning. It was my hope coming here that a more time-saving method existed.
Thank you.
EDIT - Extra information.
void DstarPlanner::insertHeap(unsigned int id, HeapKey k) {
U.push_back({ k, id });
push_heap(U.begin(), U.end(), KeyCompare());
in_U[id]++;
}
void DstarPlanner::updateVertex(unsigned int id) {
Cell* u = graph.getCell(id);
if (u->id != id_goal) {
Cost mincost = infinity;
for (auto s : u->neighbors) {
mincost = min(mincost, graph.getEdgeCost(u->id, s->id) + s->g);
}
u->rhs = mincost;
}
if (in_U[id]) {
auto it = find_if(U.begin(), U.end(), [=](auto p) { return p.second == id; });
U.erase(it);
in_U[id]--;
}
if (u->g != u->rhs) {
insertHeap(id, u->calculateKey());
}
}
vector<int> DstarPlanner::ComputeShortestPath() {
vector<int> bestPath;
vector<int> emptyPath;
Cell* n = graph.getCell(id_start);
while (U.front().first < n->calculateKey() || n->rhs != n->g) {
auto uid = U.front().second;
Cell* u = graph.getCell(uid);
auto kold = U.front().first;
pop_heap(U.begin(), U.end(), KeyCompare());
U.pop_back();
in_U[u->id]--;
if (kold < u->calculateKey()) {
insertHeap(u->id, u->calculateKey());
} else if (u->g > u->rhs) {
u->g = u->rhs;
for (auto s : u->neighbors) {
if (!occupied(s->id)) {
updateVertex(s->id);
}
}
} else {
u->g = infinity;
for (auto s : u->neighbors) {
if (!occupied(s->id)) {
updateVertex(s->id);
}
}
updateVertex(u->id);
}
}
bestPath=constructPath();
return bestPath;
}
find_if does a linear search. It maybe faster to use:
std::map/std::set -> Standard binary search tree implementations
std::unordered_map/std::unordered_set -> Standard hash table implementations
These may use a lot of memory if your elements (key-value pairs) are small integers. To avoid that you can use 3rd party alternatives like boost::unordered_flat_map.
How do you re-heapify after U.erase(it)? Do you ever delete multiple nodes at once?
If deletions need to be atomic between searches, then you can
swap it with end() - 1,
erase end() - 1, and
re-heapify.
Erasing end() - 1 is O(1) while erasing it is linear in std::distance(it, end).
void DstarPlanner::updateVertex(unsigned int id) {
...
// take the id by reference since this is synchronous
auto it = find_if(U.begin(), U.end(), [&](const auto& p) { return p.second == id; });
*it = std::move(*(U.end() - 1));
U.erase((U.end() - 1));
std::make_heap(U.begin(), U.end()); // expensive!!! 3*distance(begin, end)
...
}
If you can delete multiple nodes between searches, then you can use a combination of erase + remove_if to only perform one mass re-heapify. This is important be heapify is expensive.
it = remove_if(begin, end, [](){ lambda }
erase(it, end)
re-heapify
void DstarPlanner::updateVertex(const std::vector<unsigned int>& sorted_ids) {
...
auto it = remove_if(U.begin(), U.end(), [&](const auto& p) { return std::binary_search(ids.begin(), ids.end(), p.second); });
U.erase(it, U.end());
std::make_heap(U.begin(), U.end()); // expensive!!! 3*distance(begin, end)
...
}
Doing better
You can possibly improve on this by replacing std::make_heap (which makes no assumptions about the heapiness of [begin(), end()) with a custom method that re-heapifies a former heap around "poison points" -- it only needs to initially inspect the elements around the elements that were swapped. This sounds like a pain to write and I'd only do it if the resulting program was still too slow.
Have you thought of...
Just not even removing elements from the heap? The fact you're using a heap tells me that the algorithm designers suggested a heap. If they suggested a heap, then they likely didn't envision random removals. This is speculation on my part. I'm otherwise not familiar with D* lite.
I have following problem definition and searching on an efficient way (a dirty way already found):
I have a set of correspondences whith integer IDs, e.g.:
(0,9)
(1,5)
(9,2)
(2,3)
what i want is a set of arrays which all have connected correspondecnes included, in my example that would be
(0,9,2,3)
(1,5)
My dataset is really big so i need it very efficient, best in C++ and tbb.
What i currently did and what works (but is in fact slow and single threadded):
struct point
{
std::set<size_t> others;
};
std::map<size_t, point> globalList;
//globalList is filled with input data set, for my example:
globalList[0].others.insert(0);
globalList[0].others.insert(9);
globalList[1].others.insert(1);
globalList[1].others.insert(5);
globalList[9].others.insert(9);
globalList[9].others.insert(2);
globalList[2].others.insert(2);
globalList[2].others.insert(3);
bool changed;
do
{
changed = false;
for (auto it1 = globalList.begin(); it1 != globalList.end(); ++it1 )
{
for (auto it2 = it1 ; it2 != globalList.end(); ++it2 )
{
if (it2 == it1 )
continue;
auto findIt = it2->second.others.find(it1->first);
bool merge = false;
if( findIt != it2->second.others.end())
{
merge = true;
}
else
{
for( auto otherIt = it1->second.others.begin(); otherIt != it1->second.others.end(); ++otherIt )
{
findIt = it2->second.others.find(*otherIt );
if (findIt != it2->second.others.end())
{
merge = true;
break;
}
}
}
if(merge )
{
it1->second.others.insert(it2->second.others.begin(), it2->second.others.end());
auto it2remove = it2;
--it2;
globalList.erase(it2remove );
changed= true;
}
}
}
} while (changed);
}`
any suggestions, tips (links to algorithms, e.g. in boost) or implementations would be great....
What you want to do is basically find connected components in a graph. In your case you are starting with a set of edges (each pair is an edge).
There is for example the boost graph library, which has an implementation.
It looks like finding the longest path in trees. What do you do with loops ? I would try with a tree or graph storage of your items.
You are looking for Union Find or Disjoint Set Data Structure
An efficient implementation along with a great tutorial can be found here.
I am quite green regarding vectors, and this is my first time actually using them for collision checking. This is for my project, and I am stumped on how to implement the collision. The current Collision check and response codes I have seem to be ... bad design.
This is my code:
for(auto it = ArrayofEntities.begin(); it != ArrayofEntities.end(); it++)
{
CEntity * go = (*it);
for(auto i = ArrayofEntities.begin(); i != ArrayofEntities.end();)
{
//Collision for entities. Collision Event returns the iterator after an element is erased.
CEntity * other = (*i);
if (go != other)
{
if (!theCollision.CheckCollision(go, other, false, false, false, false)) //Checks if it has collided go with other
{
i = go->CollisionEvent(*other, ArrayofEntities); //Run collision code, setting i to the iterator which is returned.
//break;
}
else
{
i++;
}
}
else
{
i++;
}
}
}
CEntity is the base class for all the entities.
My CheckCollision just returns a true or false on the collision, and my collision event runs the collision and returns an iterator (because I might have to destroy things in the vector).
My collision event is below
vector<CEntity*>::iterator bullet::CollisionEvent(CEntity &other, vector<CEntity*> & theArray)
{
case ZOMBIE:
{
other.hp -= power * 0.01;//Effect
int Counter, index, bulletindex;
auto it = theArray.begin();
//Find the bullet and the other in the array.
for (it = theArray.begin(), Counter = 0; it != theArray.end();it++, Counter++)
{
CEntity *go = NULL;
go = (*it);
if (go == &other)
{
index = Counter;
}
if(go->ID == BULLET && go->GetX() == GetX() && go->GetY() == GetY())
{
bulletindex = Counter;
}
}
this->~bullet();//Delete the bullet
theArray.erase(theArray.begin() + bulletindex);
if(other.hp <= 0)
{
other.~CEntity();
it = theArray.erase(theArray.begin() + index); //delete from array.
return it;
}
it = theArray.begin() + index;
return it;
}
}
I have basically done this like how I would do an array. Just check it against itself. The error it gives is "Vector Iterator not Incrementable", on the first for loop after the collision event has been run.
So my question: 1) What am I doing wrong?
2) Is my thinking to do this like checking arrays wrong?
This is my school project, so I have full control of the codes.
I would prefer to have a quick fix over a complete rewriting of all the collision codes, but if it really comes down to it, I will rewrite my codes.
If you look at the implementation of std::remove_if, you'll see that they've solved the issue of iterator invalidation in another way. instead of erasing elements, they move them to the end of the array.
This may be the easiest solution for you as well. Keep an iterator which points after the last "live" entirty. It starts out at .end but as bullets hit things, you swap the entities to the back of your range and decrement that last-live iterator.
Then, when you're done looping over your array, you clean up with a single call to .erase.
And yes, you should use either std::unique_ptr<CEntity> or std::shared_ptr<CEntity> in the collection. In that way, .erase won't just erase the pointer but also the object pointed to.
How can i delete all objects which are works finished
I using the following code but get list iterator not incrementable
How can I remove it without deleting it
list<A*> myList;
for(list<A*>::iterator it = myList.begin(); it !=myList.end(); ++it ){
(*it )->DoSomething();
if((*it )->WorksFnished()){
//myList.erase(it ); <- It's works but I get exception after the loop
//myList.remove(*it ); <- It's works but I get exception after the loop
}
}
erase returns an iterator
list<A*> myList;
list<A*>::iterator it = myList.begin();
while( it != myList.end() ) {
(*it)->DoSomething();
if( (*it)->WorksFnished() ) {
it = myList.erase(it);
} else {
++it;
}
}
You can make use of the fact that erase returns a new iterator, as described in other answers here. For performance-critical code, that might be the best solution. But personally, I would favor splitting the loop into separate processing and removal steps for readability and clarity:
// Assumes C++ 11 compatible compiler
list<A*> myList;
// Processing
for(const auto* each : myList){
each->DoSomething();
}
// Deletion
myList.remove_if([](A* each) {
return each->WorksFnished();
});
If you don't want to use remove_if, some alternatives are:
Copy all objects you want to keep into a new list, then std::swap it with your current list
Use a temporary list toBeRemoved, and add all objects that should be removed to that. When you're finished iterating over the actual list, iterate toBeRemoved and call myList.erase for each element
Some workaround..
increment the number of objects from the list that has WorkFnished.
then after the loop. if the accumulator match the list size, clear it.
size_t nFinished = 0;
list<A*> myList;
for(list<A*>::iterator it = myList.begin(); it !=myList.end(); ++it ){
(*it )->DoSomething();
if((*it )->WorksFnished()){
nFinished++;
}
}
if (nFinished == myList.size())
{
myList.clear();
}
If you use erase you have to assign it back to the iterator. In this case, we have to take care of the incrementing ourselves depending if the current element was erased or not.
list<A*> myList;
for (auto it = myList.begin(); it != myList.end(); )
{
(*it)->DoSomething();
if( (*it)->WorksFnished() ) {
it = myList.erase(it); // Sets it to the next element
} else {
++it; // Increments it since no erasing
}
}
std::list::erase
Return: An iterator pointing to the new location of the element that followed the last element erased by the function call. This is the container end if the operation erased the last element in the sequence.
I am writing a solver for the N-Puzzle (see http://en.wikipedia.org/wiki/Fifteen_puzzle)
Right now I am using a unordered_map to store hash values of the puzzle board,
and manhattan distance as the heuristic for the algorithm, which is a plain DFS.
so I have
auto pred = [](Node * lhs, Node * rhs){ return lhs->manhattanCost_ < rhs->manhattanCost_; };
std::multiset<Node *, decltype(pred)> frontier(pred);
std::vector<Node *> explored; // holds nodes we have already explored
std::tr1::unordered_set<unsigned> frontierHashTable;
std::tr1::unordered_set<unsigned> exploredHashTable;
This works great for n = 2 and 3.
However, its really hit and miss for n=4 and above. (stl unable to allocate memory for a new node)
I also suspect that I am getting hash collisions in the unordered_set
unsigned makeHash(const Node & pNode)
{
unsigned int b = 378551;
unsigned int a = 63689;
unsigned int hash = 0;
for(std::size_t i = 0; i < pNode.data_.size(); i++)
{
hash = hash * a + pNode.data_[i];
a = a * b;
}
return hash;
}
16! = 2 × 10^13 (possible arrangements)
2^32 = 4 x 10^9 (possible hash values in a 32 bit hash)
My question is how can I optimize my code to solve for n=4 and n=5?
I know from here
http://kociemba.org/fifteen/fifteensolver.html
http://www.ic-net.or.jp/home/takaken/e/15pz/index.html
that n=4 is possible in less than a second on average.
edit:
The algorithm itself is here:
bool NPuzzle::aStarSearch()
{
auto pred = [](Node * lhs, Node * rhs){ return lhs->manhattanCost_ < rhs->manhattanCost_; };
std::multiset<Node *, decltype(pred)> frontier(pred);
std::vector<Node *> explored; // holds nodes we have already explored
std::tr1::unordered_set<unsigned> frontierHashTable;
std::tr1::unordered_set<unsigned> exploredHashTable;
// if we are in the solved position in the first place, return true
if(initial_ == target_)
{
current_ = initial_;
return true;
}
frontier.insert(new Node(initial_)); // we are going to delete everything from the frontier later..
for(;;)
{
if(frontier.empty())
{
std::cout << "depth first search " << "cant solve!" << std::endl;
return false;
}
// remove a node from the frontier, and place it into the explored set
Node * pLeaf = *frontier.begin();
frontier.erase(frontier.begin());
explored.push_back(pLeaf);
// do the same for the hash table
unsigned hashValue = makeHash(*pLeaf);
frontierHashTable.erase(hashValue);
exploredHashTable.insert(hashValue);
std::vector<Node *> children = pLeaf->genChildren();
for( auto it = children.begin(); it != children.end(); ++it)
{
unsigned childHash = makeHash(**it);
if(inFrontierOrExplored(frontierHashTable, exploredHashTable, childHash))
{
delete *it;
}
else
{
if(**it == target_)
{
explored.push_back(*it);
current_ = **it;
// delete everything else in children
for( auto it2 = ++it; it2 != children.end(); ++it2)
delete * it2;
// delete everything in the frontier
for( auto it = frontier.begin(); it != frontier.end(); ++it)
delete *it;
// delete everything in explored
explored_.swap(explored);
for( auto it = explored.begin(); it != explored.end(); ++it)
delete *it;
return true;
}
else
{
frontier.insert(*it);
frontierHashTable.insert(childHash);
}
}
}
}
}
Since this is homework I will suggest some strategies you might try.
First, try using valgrind or a similar tool to check for memory leaks. You may have some memory leaks if you don't delete everything you new.
Second, calculate a bound on the number of nodes that should be explored. Keep track of the number of nodes you do explore. If you pass the bound, you might not be detecting cycles properly.
Third, try the algorithm with depth first search instead of A*. Its memory requirements should be linear in the depth of the tree and it should just be a matter of changing the sort ordering (pred). If DFS works, your A* search may be exploring too many nodes or your memory structures might be too inefficient. If DFS doesn't work, again it might be a problem with cycles.
Fourth, try more compact memory structures. For example, std::multiset does what you want but std::priority_queue with a std::deque may take up less memory. There are other changes you could try and see if they improve things.
First i would recommend cantor expansion, which you can use as the hashing method. It's 1-to-1, i.e. the 16! possible arrangements would be hashed into 0 ~ 16! - 1.
And then i would implement map by my self, as you may know, std is not efficient enough for computation. map is actually a Binary Search Tree, i would recommend Size Balanced Tree, or you can use AVL tree.
And just for record, directly use bool hash[] & big prime may also receive good result.
Then the most important thing - the A* function, like what's in the first of your link, you may try variety of A* function and find the best one.
You are only using the heuristic function to order the multiset. You should use the min(g(n) + f(n)) i.e. the min(path length + heuristic) to order your frontier.
Here the problem is, you are picking the one with the least heuristic, which may not be the correct "next child" to pick.
I believe this is what is causing your calculation to explode.