I have a C++11 list of complex elements that are defined by a structure node_info. A node_info element, in particular, contains a field time and is inserted into the list in an ordered fashion according to its time field value. That is, the list contains various node_info elements that are time ordered. I want to remove from this list all the nodes that verify some specific condition specified by coincidence_detect, which I am currently implementing as a predicate for a remove_if operation.
Since my list can be very large (order of 100k -- 10M elements), and for the way I am building my list this coincidence_detect condition is only verified by few (thousands) elements closer to the "lower" end of the list -- that is the one that contains elements whose time value is less than some t_xv, I thought that to improve speed of my code I don't need to run remove_if through the whole list, but just restrict it to all those elements in the list whose time < t_xv.
remove_if() though does not seem however to allow the user to control up to which point I can iterate through the list.
My current code.
The list elements:
struct node_info {
char *type = "x";
int ID = -1;
double time = 0.0;
bool spk = true;
};
The predicate/condition for remove_if:
// Remove all events occurring at t_event
class coincident_events {
double t_event; // Event time
bool spk; // Spike condition
public:
coincident_events(double time,bool spk_) : t_event(time), spk(spk_){}
bool operator()(node_info node_event){
return ((node_event.time==t_event)&&(node_event.spk==spk)&&(strcmp(node_event.type,"x")!=0));
}
};
The actual removing from the list:
void remove_from_list(double t_event, bool spk_){
// Remove all events occurring at t_event
coincident_events coincidence(t_event,spk_);
event_heap.remove_if(coincidence);
}
Pseudo main:
int main(){
// My list
std::list<node_info> event_heap;
...
// Populate list with elements with random time values, yet ordered in ascending order
...
remove_from_list(0.5, true);
return 1;
}
It seems that remove_if may not be ideal in this context. Should I consider instead instantiating an iterator and run an explicit for cycle as suggested for example in this post?
It seems that remove_if may not be ideal in this context. Should I consider instead instantiating an iterator and run an explicit for loop?
Yes and yes. Don't fight to use code that is preventing you from reaching your goals. Keep it simple. Loops are nothing to be ashamed of in C++.
First thing, comparing double exactly is not a good idea as you are subject to floating point errors.
You could always search the point up to where you want to do a search using lower_bound (I assume you list is properly sorted).
The you could use free function algorithm std::remove_if followed by std::erase to remove items between the iterator returned by remove_if and the one returned by lower_bound.
However, doing that you would do multiple passes in the data and you would move nodes so it would affect performance.
See also: https://en.cppreference.com/w/cpp/algorithm/remove
So in the end, it is probably preferable to do you own loop on the whole container and for each each check if it need to be removed. If not, then check if you should break out of the loop.
for (auto it = event_heap.begin(); it != event_heap.end(); )
{
if (coincidence(*it))
{
auto itErase = it;
++it;
event_heap.erase(itErase)
}
else if (it->time < t_xv)
{
++it;
}
else
{
break;
}
}
As you can see, code can easily become quite long for something that should be simple. Thus, if you need to do that kind of algorithm often, consider writing you own generic algorithm.
Also, in practice you might not need to do a complete search for the end using the first solution if you process you data in increasing time order.
Finally, you might consider using an std::set instead. It could lead to simpler and more optimized code.
Thanks. I used your comments and came up with this solution, which seemingly increases speed by a factor of 5-to-10.
void remove_from_list(double t_event,bool spk_){
coincident_events coincidence(t_event,spk_);
for(auto it=event_heap.begin();it!=event_heap.end();){
if(t_event>=it->time){
if(coincidence(*it)) {
it = event_heap.erase(it);
}
else
++it;
}
else
break;
}
}
The idea to make erase return it (as already ++it) was suggested by this other post. Note that in this implementation I am actually erasing all list elements up to t_event value (meaning, I pass whatever I want for t_xv).
Related
I'm writing a piece of C++ that checks to see whether particular elements of a vector return true, and uses remove_if() to remove them if not. After this, I use vector.size() to check to see if there are any elements remaining in the vector, and then return the function if not.
At the moment, I do vector.erase() after remove_if(), as it doesn't actually reduce the size of the vector. However, this code needs to run fast, and recursively changing the size of a vector in memory is probably not ideal. However, returning if the vector is zero (instead of running the rest of the function) probably also saves time.
Is there a nice way to check how many elements remain in the vector without erasing?
here's the code:
auto remove = remove_if(sight.begin(), sight.end(), [](const Glance *a) {
return a->occupied;
});
sight.erase(remove, sight.end());
if (sight.size() == 0) {
// There's nowhere to move
return;
}
EDIT:
thanks for the help + the guidance. From the answers it's clear that the wording of the question isn't quite correct: erase() doesn't change the size of the vector in memory, but the capacity. I had mis-remembered the explanation from this post, which nicely articulates why erase() is slower than remove() for multiple removals (as you have to copy the location of the elements in the vector multiple times).
I used Instruments to benchmark the code I had originally against Johannes' suggestion, and the difference was marginal, though Johannes' was consistently slightly faster (~9.8% weight vs ~8.3% weight for the same code otherwise). The linked article should explain why. ✨
You can use std::distance(sight.begin(), remove); to get the number of remaining elements:
auto remove = remove_if(sight.begin(), sight.end(), [](const Glance *a) {
return a->occupied;
});
size_t remaining = std::distance(sight.begin(), remove);
if (remaining == 0) {
// There's nowhere to move
return;
}
But if you are only interested in 0 you can do:
auto remove = remove_if(sight.begin(), sight.end(), [](const Glance *a) {
return a->occupied;
});
if (remove == sight.begin()) {
// There's nowhere to move
return;
}
Instead of erasing elements that satisfy a criteria and then checking the number of remaining elements just to find out how many elements did not satisfy the criteria, you could simply iterate the container and count those elements. The standard library has an algorithm for that: std::count_if.
My apologies for the lengthy explanation.
I am working on a C++ application that loads two files into two 2D string vectors, rearranges those vectors, builds another 2D string vector, and outputs it all in a report. The first element of the two vectors is a code that identifies the owner of the item and the item in the vector. I pass the owner's identification to the program on start and loop through the two vectors in a nested while loop to find those that have matching first elements. When I do, I build a third vector with components of the first two, and I then need to capture any that don't match.
I was using the syntax "vector.erase(vector.begin() + i)" to remove elements from the two original arrays when they matched. When the loop completed, I had my new third vector, and I was left with two vectors that only had elements, which didn't match and that is what I needed. This was working fine as I tried the various owners in the files (the program accepts one owner at a time). Then I tried one that generated an out of range error.
I could not figure out how to do the erase inside of the loop without throwing the error (it didn't seem that swap and pop or erase-remove were feasible solutions). I solved my problem for the program with two extra nested while loops after building my third vector in this one.
I'd like to know how to make the erase method work here (as it seems a simpler solution) or at least how to check for my out of range error (and avoid it). There were a lot of "rows" for this particular owner; so debugging was tedious. Before giving up and going on to the nested while solution, I determined that the second erase was throwing the error. How can I make this work, or are my nested whiles after the fact, the best I can do? Here is the code:
i = 0;
while (i < AIvector.size())
{
CHECK:
j = 0;
while (j < TRvector.size())
{
if (AIvector[i][0] == TRvector[j][0])
{
linevector.clear();
// Add the necessary data from both vectors to Combo_outputvector
for (x = 0; x < AIvector[i].size(); x++)
{
linevector.push_back(AIvector[i][x]); // add AI info
}
for (x = 3; x < TRvector[j].size(); x++) // Don't need the the first three elements; so start with x=3.
{
linevector.push_back(TRvector[j][x]); // add TR info
}
Combo_outputvector.push_back(linevector); // build the combo vector
// then erase these two current rows/elements from their respective vectors, this revises the AI and TR vectors
AIvector.erase(AIvector.begin() + i);
TRvector.erase(TRvector.begin() + j);
goto CHECK; // jump from here because the erase will have changed the two increments
}
j++;
}
i++;
}
As already discussed, your goto jumps to the wrong position. Simply moving it out of the first while loop should solve your problems. But can we do better?
Erasing from a vector can be done cleanly with std::remove and std::erase for cheap-to-move objects, which vector and string both are. After some thought, however, I believe this isn't the best solution for you because you need a function that does more than just check if a certain row exists in both containers and that is not easily expressed with the erase-remove idiom.
Retaining the current structure, then, we can use iterators for the loop condition. We have a lot to gain from this, because std::vector::erase returns an iterator to the next valid element after the erased one. Not to mention that it takes an iterator anyway. Conditionally erasing elements in a vector becomes as simple as
auto it = vec.begin()
while (it != vec.end()) {
if (...)
it = vec.erase(it);
else
++it;
}
Because we assign erase's return value to it we don't have to worry about iterator invalidation. If we erase the last element, it returns vec.end() so that doesn't need special handling.
Your second loop can be removed altogether. The C++ standard defines functions for searching inside STL containers. std::find_if searches for a value in a container that satisfies a condition and returns an iterator to it, or end() if it doesn't exist. You haven't declared your types anywhere so I'm just going to assume the rows are std::vector<std::string>>.
using row_t = std::vector<std::string>;
auto AI_it = AIVector.begin();
while (AI_it != AIVector.end()) {
// Find a row in TRVector with the same first element as *AI_it
auto TR_it = std::find_if (TRVector.begin(), TRVector.end(), [&AI_it](const row_t& row) {
return row[0] == (*AI_it)[0];
});
// If a matching row was found
if (TR_it != TRVector.end()) {
// Copy the line from AIVector
auto linevector = *AI_it;
// Do NOT do this if you don't guarantee size > 3
assert(TR_it->size() >= 3);
std::copy(TR_it->begin() + 3, TR_it->end(),
std::back_inserter(linevector));
Combo_outputvector.emplace_back(std::move(linevector));
AI_it = AIVector.erase(AI_it);
TRVector.erase(TR_it);
}
else
++AI_it;
}
As you can see, switching to iterators completely sidesteps your initial problem of figuring out how not to access invalid indices. If you don't understand the syntax of the arguments for find_if search for the term lambda. It is beyond the scope if this answer to explain what they are.
A few notable changes:
linevector is now encapsulated properly. There is no reason for it to be declared outside this scope and reused.
linevector simply copies the desired row from AIVector rather than push_back every element in it, as long as Combo_outputvector (and therefore linevector) contains the same type than AIVector and TRVector.
std::copy is used instead of a for loop. Apart from being slightly shorter, it is also more generic, meaning you could change your container type to anything that supports random access iterators and inserting at the back, and the copy would still work.
linevector is moved into Combo_outputvector. This can be a huge performance optimization if your vectors are large!
It is possible that you used an non-encapsulated linevector because you wanted to keep a copy of the last inserted row outside of the loop. That would prohibit moving it, however. For this reason it is faster and more descriptive to do it as I showed above and then simply do the following after the loop.
auto linevector = Combo_outputvector.back();
I want to know if anyone has a quick way for adding an element to a std::list<T*> if the element is not already in it.
It's a generic function and I can not use loops so something like this
template <class T>
bool Class<T>::addElement(const T* element)
{
for (list<T*>::iterator it = list_.begin(); it != list_.end(); it++)
{
if (element == *it)
return false;
}
list_.push_back(element);
return true;
}
Is not ok because of the loop. Does anyone have ideas?
Why is what you have "not ok"? Looks perfectly fine and readable to me (modulo missing typename).
If you really don't want to use a loop, you can accomplish the same by using the algorithm to does precisely that loop: std::find:
template <class T>
bool Class<T>::addElement(const T* element)
{
if (std::find(list_.begin(), list_.end(), element) != list_.end()) {
return false;
}
list_.push_back(element);
return true;
}
If you can add other members to your class, you could add an index such as a std::unordered_set. That container stores a list of unique values, and can be searched for specific values in O(1) complexity, which implies that no full-loop search is done by the implementation for checking if the value already exists. It will stay fast even if you have a lot of values already stored.
With std::list, using library functions such as std::find will avoid explicitely writing a loop, but the implementation will perform the loop and this will be slow when a lot of values are already stored (O(n) complexity)
You can use intrusive list instead of the std::list. In this case each element in the list keeps its node data, so you can just query that data to find out if the element is already in the list. The disadvantage is that all elements in this list must be able to provide such data, and you can't put in such lists, for example, integer or boolean elements.
If you still need the std::list and/or the elements can be of any type, then the only way of fast queryng whether the element already exists in the list is to use an index. The indexes can be stored in separate std::unordered_set for fast lookups. You can use for indexes either the list's values "as is" or calculate the indexes using any custom function.
I have been struggling to put a vector object into a project im doing
I have read what little i could find about doing this and decided to give it a go.
std::vector<BrickFalling> fell;
BrickFalling *f1;
I created the vector. This next piece works fine until i get to the erase
section.
if(brickFall == true){
f1 = new BrickFalling;
f1->getBrickXY(brickfallx,brickfally);
fell.push_back(*f1);
brickFall = false;
}
// Now setup an iterator loop through the vector
vector<BrickFalling>::iterator it;
for( it = fell.begin(); it != fell.end(); ++it ) {
// For each BrickFalling, print out their info
it->printBrickFallingInfo(brick,window,deadBrick);
//This is the part im doing wrong /////
if(deadBrick == true)// if dead brick erase
{
BrickFalling[it].erase;//not sure what im supposed to be doing here
deadBrick = false;
}
}
You can totally avoid the issue by using std::remove_if along with vector::erase.
auto it =
std::remove_if(fell.begin(), fell.end(), [&](BrickFalling& b)
{ bool deadBrick = false;
b.printBrickFallingInfo(brick,window,deadBrick);
return deadBrick; });
fell.erase(it, fell.end());
This avoids the hand-writing of the loop.
In general, you should strive to write erasure loops for sequence containers in this fashion. The reason is that it is very easy to get into the "invalid iterator" scenario when writing the loop yourself, i.e. not remembering to reseat your looping iterator each time an erase is done.
The only issue with your code which I do not know about is the printBrickFallingInfo function. If it throws an exception, you may introduce a bug during the erasure process. In that case, you may want to protect the call with a try/catch block to ensure you don't leave the function block too early.
Edit:
As the comment stated, your print... function could be doing too much work just to determine if a brick is falling. If you really are attempting to print stuff and do even more things that may cause some sort of side-effect, another approach similar in nature would be to use std::stable_partition.
With std::stable_partition you can "put on hold" the erasure and just move the elements to be erased at one position in the container (either at the beginning or at the end) all without invalidating those items. That's the main difference -- with std::stable_partition, all you would be doing is move the items to be processed, but the items after movement are still valid. Not so with std::remove and std::remove_if -- moved items are just invalid and any attempt to use those items as if they are still valid is undefined behavior.
auto it =
std::stable_partition(fell.begin(), fell.end(), [&](BrickFalling& b)
{ bool deadBrick = false;
b.printBrickFallingInfo(brick,window,deadBrick);
return deadBrick; });
// if you need to do something with the moved items besides
// erasing them, you can do so. The moved items start from
// fell.begin() up to the iterator it.
//...
//...
// Now we erase the items since we're done with them
fell.erase(fell.begin(), it);
The difference here is that the items we will eventually erase will lie to the left of the partitioning iterator it, so our erase() call will remove the items starting from the beginning. In addition to that, the items are still perfectly valid entries, so you can work with them in any way you wish before you finally erase them.
The other answer detailing the use of remove_if should be used whenever possible. If, however, your situations does not allow you to write your code using remove_if, which can happen in more complicated situations, you can use the following:
You can use vector::erase with an iterator to remove the element at that spot. The iterator used is then invalidated. erase returns a new iterator that points to the next element, so you can use that iterator to continue.
What you end up with is a loop like:
for( it = fell.begin(); it != fell.end(); /* iterator updated in loop */ )
{
if (shouldDelete)
it = fell.erase(it);
else
++it;
}
The following C++ code fills a vector with a number of objects and then removes some of these objects, but it looks like it deletes the wrong ones:
vector<Photon> photons;
photons = source->emitPhotons(); // fills vector with 300 Photon objects
for (int i=0; i<photons.size(); i++) {
bool useless = false;
// process photon, set useless to true for some
// remove useless photons
if (useless) {
photons.erase(photons.begin()+i);
}
}
Am I doing this correctly? I'm thinking the line photons.erase(photons.begin()+i); might be the problem?
Definietly the wrong way of doing it, you never adjust i down as you delete..
Work with iterators, and this problem goes away!
e.g.
for(auto it = photons.begin(); it != photons.end();)
{
if (useless)
it = photons.erase(it);
else
++it;
}
There are other ways using algorithms (such as remove_if and erase etc.), but above is clearest...
the elegant way would be:
std::vector<Photon> photons = source->emitPhotons();
photons.erase(
std::remove_if(photons.begin(), photons.end(), isUseless),
photons.end());
and:
bool isUseless(const Photon& photon) { /* whatever */ }
The proper version will look like:
for (vector<Photon>::iterator i=photons.begin(); i!=photons.end(); /*note, how the advance of i is made below*/) {
bool useless = false;
// process photon, set useless to true for some
// remove useless photons
if (useless) {
i = photons.erase(i);
} else {
++i;
}
}
You should work with stl::list in this case. Quoting the STL docs:
Lists have the important property that insertion and splicing do not invalidate iterators to list elements, and that even removal invalidates only the iterators that point to the elements that are removed.
So this would go along the lines of:
std::list<Photon> photons;
photons = source->emitPhotons();
std::list<Photon>::iterator i;
for(i=photons.begin();i!=photons.end();++i)
{
bool useless=false;
if(useless)
photons.erase(i);
}
Erasing elements in the middle of a vector is very inefficient ... the rest of the elements need to be "shifted" back one slot in order to fill-in the "empty" slot in the vector created by the call to erase. If you need to erase elements in the middle of a list-type data-structure without incurring such a penalty, and you don't need O(1) random access time (i.e., you're just trying to store your elements in a list that you'll copy or use somewhere else later, and you always iterate through the list rather than randomly accessing it), you should look into std::list which uses an underlying linked-list for its implementation, giving it O(1) complexity for modifications to the list like insert/delete.