I have a std::map declared thusly in a legacy MFC application:
typedef std::map<long, CNutrientInfo> NUTRIENT_INFO_MAP;
typedef NUTRIENT_INFO_MAP::const_iterator NUTRIENT_INFO_ITER;
typedef NUTRIENT_INFO_MAP::value_type NUTRIENT_INFO_PAIR;
static NUTRIENT_INFO_MAP m_NutrientInfoMap;
m_NutrientInfoMap is populated when the app loads by looping through a table and creating an instance of CNutrientInfo and then inserting it into the std:map like:
m_NutrientMapInfo.insert(NUTRIENT_INFO_PAIR(nutrient.GetId(), nutrient));
The std::map now contains a list of the nutrients that have been defined by the database. At some point a user can add a new nutrient to this list and it checks to see if what the user is adding already exists in the list. It does that check like:
NUTRIENT_INFO_ITER iter = m_NutrientInfoMap.begin();
while (iter != m_NutrientInfoMap.end())
{
m = (*iter).second;
if (_stricmp(m.GetFullName().c_str(), name.c_str()) == 0)
{
return m;
}
iter++;
}
Or at least it's supposed to. When the function actually gets called it never advances past the initial line of the while loop. Placing a breakpoint there simply shows that the line in question is called over and over and never advances beyond it, which hangs the app. If you step into the actual comparison it compares correctly and then returns to the while loop line. Stepping in again to advance into the body of the loop simply returns to the while loop line. This same logic is used elsewhere in the app with no trouble so I'm stumped as to what's going on in this case. I've re-written the logic above using a for-loop and it works just fine, so it's not like I can't work around it, but C++ isn't my strongest language and as this is a legacy app that I'm trying to help support, I'd really like to learn and understand WHY this is doing what's it's doing for future reference. Plus, since the logic works elsewhere and not here perhaps there's an underlying cause that is what actually needs to be addressed.
Any suggestions or thoughts on this would be greatly appreciated.
Thanks in advance.
Is your example actually pasted from the source? Maybe it looks more like:
while (iter != m_NutrientInfoMap.end()); // <== note the semi-colon
{
m = (*iter).second;
if (_stricmp(m.GetFullName().c_str(), name.c_str()) == 0)
{
return m;
}
iter++;
}
There is nothing in the code posted above which can cause that behavior. Are you sure that you are incrementing some other iterator inside the loop or may be there two iterators with the same name (one inside the loop) with different scopes and you are incrementing the wrong iterator ? If this is not the case, then only other alternative I could see is to note down the value of m_NutrientInfoMap.end() and check why ++iter is not evaluating into that value.
Related
When I use a std::map, it seems that accessing elements takes a different amount of time based on the method used.
First Method: Direct Access
cnt += umap[t];
Second Method:
if (umap.find(t) != umap.end()){
cnt += umap[t];
}
The second method seems to be quite a bit faster than the first method, and I don't understand why. Can someone explain the differences between these two methods?
Each of the code snippets is doing a different thing.
The first snippet takes more time because it inserts a key t to umap, if it does not exist (and initializes it with zero), before adding it to cnt.
In second snippet no key is inserted. Because of the if condition, umap[t] is (called then) added to cnt only when umap has the key t.
The second snippet can be more optimized by temporarily storing the iterator returned by find. In second snippet, operator[] internally calls find method again, which in turn increases time complexity.
Hence, an attempt like this will prove much faster (picked up from user253751's comment):
if(auto it = umap.find(t); it != umap.end())
cnt += it->second;
I am new to C++, I would appreciate if anyone and help me to validate the below function or help me improve it.
void RecursiveKnn(std::set<PointId> &refSet, const PointId &id, const KD3Index &kid, int const lvl)
{
if (refSet.find(id) == refSet.end())
refSet.insert(id);
if (lvl < m_knn)
{
auto ids = kid.neighbors(id, 7);
for (auto x : ids)
{
if (x == id)
continue;
RecursiveKnn(refSet, x, kid, lvl+1);
}
}
}
I have written a recursive function to run and generate a set of hierarchical objects. basically, start with one object, get next/nearby objects and so on for the next level. Along with it, I want to avoid duplicates as well. The levels are limited to 3 - 4 and do not expect to go any further.
This function is called millions of time and is taking forever to run. I would really appreciate if anyone can suggest any improvement. On top of my head I am sure std::set is not the correct data structure to use but, I don't know what to use.
EDIT: The reason, I find the function to have the performance problem is that. At first I have a single function with 3 nested for loop. which worked within reasonable time. When I changed it to a recursive function, The process did not complete for more than an hour.
Without more information my guess would be that multiple PointIds can be neighbors with the same PointIds, even though there is a parent/child relation that holds.
In other words, this code is performing a depth-first search on a directed acyclic graph but is not checking for revisits the way a typical depth-first search does. This means you will be exploring the same nodes many times which is extremely inefficient.
Change
if (refSet.find(id) == refSet.end())
refSet.insert(id);
to
if (refSet.find(id) != refSet.end())
return;
refSet.insert(id);
Also you should use an std::unordered_set instead of an std::set but that is a lesser concern if the above is true.
This is wasteful.
if (refSet.find(id) == refSet.end())
refSet.insert(id);
There's no need to check if something is a member of a set before inserting it, just insert it anyway.
refSet.insert(id);
That said this is not an order of magnitude improvement.
This might also help
const auto& ids = kid.neighbors(id, 7);
Depends on what neighbors returns but it looks like you're copying some collection or other.
First off, good morning/day/evening and thank you to whoever is taking their time to read this.
The Setup:
In my code I have two different classes: ColObj and QuadNode (that is, the 'collision object' and a node in a quad tree used to check for proximity of objects. I know there are probably libraries out there for this, but I need to write my own system, so they would be of no use here). Things work like this: When a ColObj object is created, it is added into an appropriate QuadNode (the node has a std::list of pointers to ColObj), so that the node can inform it when it collides with something; the ColObj object also receives a pointer to the node that's holding it and a list iterator to the list iterator containing its address so when it's out of the node's bounds or gets destroyed it can 'leave' it, and clean up the node, that is, remove and reference to itself from the node. I made it like this because in a lot of cases it's going to be a frequent operation and I want it to be in constant time.
The Code:
This is the method used to 'attach' a ColObj to a QuadNode. I suspect the problem is not in here.
void QuadNode::obj_add(ColObj *obj) {
std::cout<<"QuadNode at depth ("<<depth<<") received new ColObj.\n";
objects.push_back(obj);
obj->holder = this;
obj->my_iter = std::prev( objects.end() );
if ((int)objects.size() > MAX_OBJECTS && depth < MAX_DEPTH) split();
}
This is the QuadNode method that a ColObj uses to clean up the node. Here is where the problem occures for some reason.
void QuadNode::obj_list_erase(std::list<ColObj*>::iterator iter) {
std::list<ColObj*>::iterator iter2 = objects.begin();
objects.erase(iter);
}
The first line in this method is simply to provide additional information for debugging and will be removed afterwards.
The Error:
The strangest part is that, for the most part, the code works fine. Then at one point, randomly, it throws an assertion failure, saying that a "list iterator is not incrementable". That's the first strange thing, I'm not trying to increment it anywhere in my code (though I know that std::list::erase returns the following iterator, but I never attempt this operation on an invalid or "past-the-last" iterator).
Anyway, Visual Studio offers to fire up the debugger and put a break point in the code, so naturally I agree. So here's the weirdest part:
Local and auto variables, debugger screenshot
(I can't embed an image since I'm new here, so it is what it is).
So, unless I'm gravely mistaken here, it's telling me that the passed iterator is equal to be beginning iterator of the list, that its element is still present in the list and corresponds to the first (or rather zero-th) element of the list. And yet, the erase() method fails.
For what it's worth, I've noticed that every single time the program breaks, the passed iterator points to the zero-th element of the list, though I can confirm that the method usually works even when there's only one element in the list.
Additional info and conclusion:
I'm not manually incrementing the iterator anywhere else in the code (which is pretty small and simple anyway).
The IDE I'm using is Visual Studio Community 2015, but I don't know the compiler version. (Microsoft and their naming schemes...)
I tried finding another thread about this on SO but every one I checked was about wrongly placed i++ in list iterations, so sorry if this is a duplicate thread.
I'm completely confused by this problem, because usually between the excellent debugger, std::cout and browsing SO I somehow fix the issue, but this time around nothing useful is coming up, so any advice or suggestion would be very welcome.
Edit:
One thing I have tried "just 'cause" editing the QuadNode::obj_list_erase method so that it compares the passed iterator with the first iterator of its list (objects.begin()) and if they're equal use objects.pop() to remove it, else erase it normally. It didn't work, saying that the iterators weren't compatible, whatever that means...
After finding out that I cannot even compare the passed iterator with any other iterator from the list that was supposed to be holding it (I was getting Assertion failure: iterators not compatible), I searched SO for more info on what it means, and... Andrew Kashpur was right. I did manage to invalidate an iterator by removing the pointed element from the list and putting it back immediately, but without updating the iterator.
Moral of the story: An iterator can seem to point to a "correct" memory location, it may even point to the same address as some valid iterator does, but that does not make it valid or compatible.
I want to insert a pair< string, vector<float> > into a map, first it works, but after several loops, it cannot insert any more and throw me a segmentation fault. Can anybody give a possible reason?
Btw: I first read a file and generate the map (about 200,000 elements) and I read another file and update the old map. the error occurs while the updating step.
Can anybody help me with the info I gave above? Thanks a lot
The code is pretty long.....I just erase the previous key and then insert a new one, it seems not complicated.....but drives me crazy....could you guess what happened here?
Thanks A lot for all your answers! And I found it is really a good place for solving problems. Thanks again, I'll try to simplify my codes and add it here today or tomorrow.
Update: I used the code from MSN and it works, thanks a lot that you solved my problem without seeing my code......also many thanks to other kind-hearted people here! However, i can only choose one as the answer.
Are you inserting using the iterator you called erase() on? Or using that iterator in any way? After erase(p) is called, p is invalidated.
Without more information, it's not easy to say, but what are you inserting? Could it simply be that you run out of memory? Although, I do think normal C++ would throw an exception in that case, are you using any custom allocators, malloc, or arrays on the stack which are overrun perhaps?
Perhaps a snippet of code describing what you do could be helpful in determining the cause of your problem.
could you guess what happened here?
You're abusing memory in some way.
There are a lot of ways to do that, in C++!
Rather than guess, and without reading your code, I suggest run the kind of platform-specific debugger which will detect this problem, for example valgrind.
Your alternative is to make the problem smaller: reproduce the problem in only a few lines of code, which you can then post for people to look at.
The type in question is pair<string, vector<float> >. You will be copying that pair on every insert. If either the string or the vector are big then you could be running out of memory.
Edit: to fix running out of memory, you can change how you insert key-value pairs to:
pair<map::iterator, bool> insert_result= map.insert(make_pair(name, vector<float>());
if (insert.second) { insert_result.first->second.swap(vector_read_in); }
That will ensure that you do not copy memory, only move it.
It easily happens if you either modify the keys of the elements already in the data structure or if you have a bad compare function, which misleads the search algorithm.
If you can detect which concrete insert operation causes the seg.fault, then try debugging/logging with what values the compare function is called.
Alternatively, you should print the contents of the map before the erroneous insert, the keys will probably not be in order.
please post some code, you cant expect us to debug your problem on guess work alone.
...but ill give it a stab anyway :) Also what compiler, and system are you doing this on?
If you are reading the data in a loop you may run out of stack space which would cause a seg fault.
Ill edit my answer if you post some code.
Remember that if you are looping through the map, finding stuff to delete, save off the key for later deletion. If you delete while iterating, you run the risk of invalidating the iteration loop.
std::map<string, vector<float> >::iterator iter = my_map.begin();
while (iter != my_map.end()) {
if (somethingBadAboutItem(iter)) {
my_map.erase(iter); // this can mess up my_map iteration above
// and cause bad data access later
}
++iter;
}
instead, try
std::map<string, vector<float> >::iterator iter = my_map.begin();
std::vector<string> keys_to_delete;
while (iter != my_map.end()) {
if (somethingBadAboutItem(iter)) {
keys_to_delete.push_back(iter->first);
}
++iter;
}
for (std::size_t i = 0; i < keys_to_delete.size(); ++i) {
iter = my_map.find(keys_to_delete[i]);
my_map.erase(iter);
}
I am interested if other people have found something more elegant than this, but this is my preferred technique.
Bill: Have you tried something like this:
for(std::map<string, vector<float> >::iterator iter = my_map.begin(); iter != my_map.end();) {
if(somethingBadAboutItem(iter)) {
my_map.erase(iter++);
} else {
++iter;
}
}
The key is to postincrement the iterator when deleting, so it's still valid when incremented but you erase (and hence invalidate) a copy pointing to the previous item.
I'm not sure this technique necessarily works for all STL containers; I can't remember all the invalidation rules offhand (seems unlikely to work for vector for example, but you're normally better off using remove_if for those for non-trivial sizes) but it should be fine for map.
I have a C++ STL set with a custom ordering defined.
The idea was that when items get added to the set, they're naturally ordered as I want them.
However, what I've just realised is that the ordering predicate can change as time goes by.
Presumably, the items in the set will then no longer be in order.
So two questions really:
Is it harmful that the items would then be out of order? Am I right in saying that the worst that can happen is that new entries may get put into the wrong place (which actually I can live with). Or, could this cause crashes, lost entries etc?
Is there a way to "refresh" the ordering of the set? You can't seem to use std::sort() on a set. The best I can come up with is dumping out the contents to a temp container and re-add them.
Any ideas?
Thanks,
John
set uses the ordering to lookup items. If you would insert N items according to ordering1 and insert an item according to ordering2, the set cannot find out if the item is already in.
It will violate the class invariant that every item is in there only once.
So it does harm.
The only safe way to do this with the STL is to create a new set with the changed predicate. For example you could do something like this when you needed to sort the set with a new predicate:
std::set<int> newset( oldset.begin(), oldset.end(), NewPred() );
This is actually implementation dependent.
The STL implementation can and usually will assumes the predicate used for sorting is stable (otherwise, "sorted" would not be defined). It is at least possible to construct a valid STL implementation that formats your hard drive when you change the behavior of the predicate instance.
So, yes, you need to re-insert the items into a new set.
Alternatively, you could construct your own container, e.g. a vector + sort + lower_bound for binary search. Then you could re-sort when the predicates behavior changes.
I agree with the other answers, that this is going to break in some strange and hard to debug ways. If you go the refresh route, you only need to do the copy once. Create a tmp set with the new sorting strategy, add each element from the original set to the tmp set, then do
orig.swap(tmp);
This will swap the internals of the sets.
If this were me, I would wrap this up in a new class that handles all of the details, so that you can change implementations as needed. Depending on your access patterns and the number of times the sort order changes, the previously mentioned vector, sort, lowerbound solution may be preferable.
If you can live with an unordered set, then why are you adding them into a set in the first place?
The only case I can think of is where you just want to make sure the list is unique when you add them. If that's the case then you could use a temporary set to protect additions:
if (ts.insert (value).second) {
// insertion took place
realContainer.push_back (value);
}
An alternative, is that depending on how frequently you'll be modifying the entries in the set, you can probably test to see if the entry will be in a different location (by using the set compare functionality) and where the position will move then remove the old entry and re-add the new one.
As everyone else has pointed out - having the set unordered really smells bad - and I would also guess that its possible got undefined behaviour according to the std.
While this doesn't give you exactly what you want, boost::multi_index gives you similar functionality. Due to the way templates work, you will never be able to "change" the ordering predicate for a container, it is set in stone at compile time, unless you are using a sorted vector or something similar, to where you are the one maintaining the invariant, and you can sort it however you want at any given time.
Multi_index however gives you a way to order a set of elements based on multiple ordering predicates at the same time. You can then select views of the container that behave like an std::set ordered by the predicate that you care about at the time.
This can cause lost entries, when searching for an element in a set the ordering operator is used this means that if an element was placed to the left of the root and now the ordering operator says it's to the right then that element will not longer be found.
Here's a simple test for you:
struct comparer : public std::binary_function<int, int, bool>
{
static enum CompareType {CT_LESS, CT_GREATER} CompareMode;
bool operator()(int lhs, int rhs) const
{
if(CompareMode == CT_LESS)
{
return lhs < rhs;
}
else
{
return lhs > rhs;
}
}
};
comparer::CompareType comparer::CompareMode = comparer::CT_LESS;
typedef std::set<int, comparer> is_compare_t;
void check(const is_compare_t &is, int v)
{
is_compare_t::const_iterator it = is.find(v);
if(it != is.end())
{
std::cout << "HAS " << v << std::endl;
}
else
{
std::cout << "ERROR NO " << v << std::endl;
}
}
int main()
{
is_compare_t is;
is.insert(20);
is.insert(5);
check(is, 5);
comparer::CompareMode = comparer::CT_GREATER;
check(is, 5);
is.insert(27);
check(is, 27);
comparer::CompareMode = comparer::CT_LESS;
check(is, 5);
check(is, 27);
return 0;
}
So, basically if you intend to be able to find the elements you once inserted you should not change the predicate used for insertions and find.
Just a follow up:
While running this code the Visual Studio C debug libraries started throwing exceptions complaining that the "<" operator was invalid.
So, it does seem that changing the sort ordering is a bad thing. Thanks everyone!
1) Harmful - no. Result in crashes - no. The worst is indeed a non-sorted set.
2) "Refreshing" would be the same as re-adding anyway!