I have the following code:
static std::map<int,int> myFunction(std::list<int>& symbols){
std::map<int,int> currCounts;
std::map<int,int> payHits;
for (std::list<int>::iterator l_itr = symbols.begin(); l_itr != symbols.end(); ++l_itr){
myFunction_helper(*l_itr, l_itr, symbols, currCounts, payHits, 0);
}
return payHits;
}
static inline void myFunction_helper(int next, std::list<int>::iterator& pos, std::list<int> remainingSymbols, std::map<int,int> currCounts, std::map<int,int>& payHits, int i){
currCounts[next] = currCounts.count(next) > 0 ? currCounts[next] + 1 : 1;
remainingSymbols.erase(pos);
if (i < numTiles && remainingSymbols.size() > 0){
if (currCounts[next] == hitsNeeded[next]){
int pay = symbolPays[next];
payHits[pay] = payHits.count(pay) > 0 ? payHits[next] + 1 : 1;
}
else{
for (std::list<int>::iterator l_itr = remainingSymbols.begin(); l_itr != remainingSymbols.end(); ++l_itr){
myFunction_helper(*l_itr, l_itr, remainingSymbols, currCounts, payHits, i+1);
}
}
}
else{
payHits[0] = payHits.count(0) > 0 ? payHits[0] + 1 : 1;
}
}
It is supposed to take a set of values and given some requirements (numTiles (int), hitsNeeded (a map of symbols and number of times they need to be chosen to win)). My code builds on visual studios (most recent version), but when I try executing it I get the error "list erase iterator out of range" the first time the myFunction_helper is called. How do I avoid this? I purposefully passed the remainingSymbols by value so that I can modify it without affecting other recursive stack frame members. How do I fix this and whyis this raising an exception?
Solution
Remove the iterator from the arguments. Then as you iterate you use the following snippet of code:
int next = *l_itr;
l_itr = symbols.erase(l_itr);
myFunction_helper(next, remainingSymbols, currCounts, payHits, i+1);
symbols.push_front(next);
And similarly for the outer function. Pushing the element to the front doesn't disrupt the iteration over the list and allows for what I want (pushing to the front is super cheap on lists too).
Agree with the Comments below. This is a crap answer because we don't know enough about the business case to suggest a good solution. I'm leaving an edited version it here because I just reverted the vandalized question and it does explain why the attempt failed.
Why This is raising an exception
std::list<int> remainingSymbols is pass by value, so pos is no longer relevant. It refers to the source list, not the copy of the source list in remainingSymbols. Using an iterator for one list in another, even a copy, is fatal.
solution
The common solution is to solution is to pass remainingSymbols by reference: std::list<int> & remainingSymbols, but since this will break backtracking, you can't do that.
Instead you will have to pass a different identifier for the position, perhaps the index. Unfortunately interating and re-iterating a list is an expensive task that almost always outweighs the quick insert and delete benefits of a list.
You cannot use iterator from one container with another one, you could use offset instead, but that would be very ineffective with std::list. Additionally usingstd::list with int is not a good idea in general - your data is small and most probably you use at least twice more memory for maintaining list items than data itself plus cache misses. You better use std::vector<int> and pass offset, not iterator. Additionaly with vector<> you can use move erase idiom but even deleting int in middle of vector is relatively cheap, most probably less expensive than cost of jumping of std::list nodes.
Related
I want to implement an algorithm that basically moves every value(besides the last one) one place to the left, as in the first element becomes the second element, and so on.
I have already implemented it like this:
for(int i = 0; i < vct.size() - 1; i++){
vct[i] = vct[i + 1];
}
which works, but I was just wondering if there is a faster, optionally shorter way to achieve the same result?
EDIT: I have made a mistake where I said that I wanted it to move to the right, where in reality I wanted it to go left, so sorry for the confusion and thanks to everyone for pointing that out. I also checked if the vector isn't empty beforehand, just didn't include it in the snippet.
As a comment (or more than one?) has pointed out, the obvious choice here would be to just use a std::deque.
Another possibility would be to use a circular buffer. In this case, you'll typically have an index (or pointer) to the first and last items in the collection. Removing an item from the beginning consists of incrementing that index/pointer (and wrapping it around to the beginning of you've reached the end of the buffer). That's quite fast, and constant time, regardless of collection size. There is a downside that every time you add an item, remove an item, or look at an item, you need to do a tiny bit of extra math to do it. But it's usually just one addition, so overhead is pretty minimal. Circular buffers work well, but they have a fair number of corner cases, so getting them just right is often kind of a pain. Worse, many obvious implementations waste one the slot for one data item (though that often doesn't matter a lot).
A slightly simpler possibility would reduce overhead by some constant factor. To do this, use a little wrapper that keeps track of the first item in the collection, along with your vector of items. When you want to remove the first item, just increment the variable that keeps track of the first item. When you reach some preset limit, you erase the first N elements all at once. This reduces the time spent shifting items by a factor of N.
template <class T>
class pseudo_queue {
std::vector<T> data;
std:size_t b;
// adjust as you see fit:
static const int max_slop = 20;
void shift() {
data.erase(data.begin(), data.begin() + b);
}
public:
void push_back(T &&t) { data.push_back(std::move(t); }
void pop_back() { data.pop_back(); }
T &back() { return data.back(); }
T &front() { return data[b]; }
void pop_front() {
if (++b > max_slop) shift();
}
std::vector<T>::iterator begin() { return data.begin() + b; }
std::vector<T>::iterator end() { return data.end(); }
T &operator[](std::size_t index) { return data[index + b]; }
};
If you want to get really tricky, you can change this a bit, so you compute max_slop as a percentage of the size of data in the collection. In this case, you can change the computational complexity involved, rather than just leaving it linear but with a larger constant factor than you currently have. But I have no idea how much (if at all) you care about that--it's only likely to matter much if you deal with a wide range of sizes.
assuming you really meant moving data to the right, and that your code has a bug,
you have std::move_backwards from <algorithms> and its sibling std::move to do that, but accessing data backwards may be inefficient.
std::move_backward(vct.begin(),vct.end()-1,vct.end());
if you actually meant move data to the left, you can use std::move
std::move(vct.begin()+1,vct.end(),vct.begin())
you can also use std::copy and std::copy_backward instead if your object is trivially copiable, the syntax is exactly the same and it will be faster for trivially copyable objects (stack objects).
you can just do a normal loop to the right, assuming vct.size() is bigger than 1.
int temp1 = vct[0]; // assume vct of int
int temp2; // assume vct of int
for(int i = 0;i<vct.size() - 1;i++){
temp2 = vct[i+1];
vct[i+1] = temp1;
temp1 = temp2;
}
and your version is what's to do if you are moving to the left.
also as noted in the comments, you should check that your list is not empty if you are doing the loop version.
I have a C++11 list of complex elements that are defined by a structure node_info. A node_info element, in particular, contains a field time and is inserted into the list in an ordered fashion according to its time field value. That is, the list contains various node_info elements that are time ordered. I want to remove from this list all the nodes that verify some specific condition specified by coincidence_detect, which I am currently implementing as a predicate for a remove_if operation.
Since my list can be very large (order of 100k -- 10M elements), and for the way I am building my list this coincidence_detect condition is only verified by few (thousands) elements closer to the "lower" end of the list -- that is the one that contains elements whose time value is less than some t_xv, I thought that to improve speed of my code I don't need to run remove_if through the whole list, but just restrict it to all those elements in the list whose time < t_xv.
remove_if() though does not seem however to allow the user to control up to which point I can iterate through the list.
My current code.
The list elements:
struct node_info {
char *type = "x";
int ID = -1;
double time = 0.0;
bool spk = true;
};
The predicate/condition for remove_if:
// Remove all events occurring at t_event
class coincident_events {
double t_event; // Event time
bool spk; // Spike condition
public:
coincident_events(double time,bool spk_) : t_event(time), spk(spk_){}
bool operator()(node_info node_event){
return ((node_event.time==t_event)&&(node_event.spk==spk)&&(strcmp(node_event.type,"x")!=0));
}
};
The actual removing from the list:
void remove_from_list(double t_event, bool spk_){
// Remove all events occurring at t_event
coincident_events coincidence(t_event,spk_);
event_heap.remove_if(coincidence);
}
Pseudo main:
int main(){
// My list
std::list<node_info> event_heap;
...
// Populate list with elements with random time values, yet ordered in ascending order
...
remove_from_list(0.5, true);
return 1;
}
It seems that remove_if may not be ideal in this context. Should I consider instead instantiating an iterator and run an explicit for cycle as suggested for example in this post?
It seems that remove_if may not be ideal in this context. Should I consider instead instantiating an iterator and run an explicit for loop?
Yes and yes. Don't fight to use code that is preventing you from reaching your goals. Keep it simple. Loops are nothing to be ashamed of in C++.
First thing, comparing double exactly is not a good idea as you are subject to floating point errors.
You could always search the point up to where you want to do a search using lower_bound (I assume you list is properly sorted).
The you could use free function algorithm std::remove_if followed by std::erase to remove items between the iterator returned by remove_if and the one returned by lower_bound.
However, doing that you would do multiple passes in the data and you would move nodes so it would affect performance.
See also: https://en.cppreference.com/w/cpp/algorithm/remove
So in the end, it is probably preferable to do you own loop on the whole container and for each each check if it need to be removed. If not, then check if you should break out of the loop.
for (auto it = event_heap.begin(); it != event_heap.end(); )
{
if (coincidence(*it))
{
auto itErase = it;
++it;
event_heap.erase(itErase)
}
else if (it->time < t_xv)
{
++it;
}
else
{
break;
}
}
As you can see, code can easily become quite long for something that should be simple. Thus, if you need to do that kind of algorithm often, consider writing you own generic algorithm.
Also, in practice you might not need to do a complete search for the end using the first solution if you process you data in increasing time order.
Finally, you might consider using an std::set instead. It could lead to simpler and more optimized code.
Thanks. I used your comments and came up with this solution, which seemingly increases speed by a factor of 5-to-10.
void remove_from_list(double t_event,bool spk_){
coincident_events coincidence(t_event,spk_);
for(auto it=event_heap.begin();it!=event_heap.end();){
if(t_event>=it->time){
if(coincidence(*it)) {
it = event_heap.erase(it);
}
else
++it;
}
else
break;
}
}
The idea to make erase return it (as already ++it) was suggested by this other post. Note that in this implementation I am actually erasing all list elements up to t_event value (meaning, I pass whatever I want for t_xv).
My apologies for the lengthy explanation.
I am working on a C++ application that loads two files into two 2D string vectors, rearranges those vectors, builds another 2D string vector, and outputs it all in a report. The first element of the two vectors is a code that identifies the owner of the item and the item in the vector. I pass the owner's identification to the program on start and loop through the two vectors in a nested while loop to find those that have matching first elements. When I do, I build a third vector with components of the first two, and I then need to capture any that don't match.
I was using the syntax "vector.erase(vector.begin() + i)" to remove elements from the two original arrays when they matched. When the loop completed, I had my new third vector, and I was left with two vectors that only had elements, which didn't match and that is what I needed. This was working fine as I tried the various owners in the files (the program accepts one owner at a time). Then I tried one that generated an out of range error.
I could not figure out how to do the erase inside of the loop without throwing the error (it didn't seem that swap and pop or erase-remove were feasible solutions). I solved my problem for the program with two extra nested while loops after building my third vector in this one.
I'd like to know how to make the erase method work here (as it seems a simpler solution) or at least how to check for my out of range error (and avoid it). There were a lot of "rows" for this particular owner; so debugging was tedious. Before giving up and going on to the nested while solution, I determined that the second erase was throwing the error. How can I make this work, or are my nested whiles after the fact, the best I can do? Here is the code:
i = 0;
while (i < AIvector.size())
{
CHECK:
j = 0;
while (j < TRvector.size())
{
if (AIvector[i][0] == TRvector[j][0])
{
linevector.clear();
// Add the necessary data from both vectors to Combo_outputvector
for (x = 0; x < AIvector[i].size(); x++)
{
linevector.push_back(AIvector[i][x]); // add AI info
}
for (x = 3; x < TRvector[j].size(); x++) // Don't need the the first three elements; so start with x=3.
{
linevector.push_back(TRvector[j][x]); // add TR info
}
Combo_outputvector.push_back(linevector); // build the combo vector
// then erase these two current rows/elements from their respective vectors, this revises the AI and TR vectors
AIvector.erase(AIvector.begin() + i);
TRvector.erase(TRvector.begin() + j);
goto CHECK; // jump from here because the erase will have changed the two increments
}
j++;
}
i++;
}
As already discussed, your goto jumps to the wrong position. Simply moving it out of the first while loop should solve your problems. But can we do better?
Erasing from a vector can be done cleanly with std::remove and std::erase for cheap-to-move objects, which vector and string both are. After some thought, however, I believe this isn't the best solution for you because you need a function that does more than just check if a certain row exists in both containers and that is not easily expressed with the erase-remove idiom.
Retaining the current structure, then, we can use iterators for the loop condition. We have a lot to gain from this, because std::vector::erase returns an iterator to the next valid element after the erased one. Not to mention that it takes an iterator anyway. Conditionally erasing elements in a vector becomes as simple as
auto it = vec.begin()
while (it != vec.end()) {
if (...)
it = vec.erase(it);
else
++it;
}
Because we assign erase's return value to it we don't have to worry about iterator invalidation. If we erase the last element, it returns vec.end() so that doesn't need special handling.
Your second loop can be removed altogether. The C++ standard defines functions for searching inside STL containers. std::find_if searches for a value in a container that satisfies a condition and returns an iterator to it, or end() if it doesn't exist. You haven't declared your types anywhere so I'm just going to assume the rows are std::vector<std::string>>.
using row_t = std::vector<std::string>;
auto AI_it = AIVector.begin();
while (AI_it != AIVector.end()) {
// Find a row in TRVector with the same first element as *AI_it
auto TR_it = std::find_if (TRVector.begin(), TRVector.end(), [&AI_it](const row_t& row) {
return row[0] == (*AI_it)[0];
});
// If a matching row was found
if (TR_it != TRVector.end()) {
// Copy the line from AIVector
auto linevector = *AI_it;
// Do NOT do this if you don't guarantee size > 3
assert(TR_it->size() >= 3);
std::copy(TR_it->begin() + 3, TR_it->end(),
std::back_inserter(linevector));
Combo_outputvector.emplace_back(std::move(linevector));
AI_it = AIVector.erase(AI_it);
TRVector.erase(TR_it);
}
else
++AI_it;
}
As you can see, switching to iterators completely sidesteps your initial problem of figuring out how not to access invalid indices. If you don't understand the syntax of the arguments for find_if search for the term lambda. It is beyond the scope if this answer to explain what they are.
A few notable changes:
linevector is now encapsulated properly. There is no reason for it to be declared outside this scope and reused.
linevector simply copies the desired row from AIVector rather than push_back every element in it, as long as Combo_outputvector (and therefore linevector) contains the same type than AIVector and TRVector.
std::copy is used instead of a for loop. Apart from being slightly shorter, it is also more generic, meaning you could change your container type to anything that supports random access iterators and inserting at the back, and the copy would still work.
linevector is moved into Combo_outputvector. This can be a huge performance optimization if your vectors are large!
It is possible that you used an non-encapsulated linevector because you wanted to keep a copy of the last inserted row outside of the loop. That would prohibit moving it, however. For this reason it is faster and more descriptive to do it as I showed above and then simply do the following after the loop.
auto linevector = Combo_outputvector.back();
I've implemented a merge function for vectors, which basically combines to sorted vectors in a one sorted vector. (yes, it is for a merge sort algorithm). I was trying to make my code faster and avoid overheads, so I decided not to use the push_back method on the vector, but try to use the array syntax instead which has lesser over head. However, something is going terribly wrong, and the output is messed up when i do this. Here's the code:
while(size1<left.size() && size2 < right.size()) //left and right are the input vectors
{
//it1 and it2 are iterators on the two sorted input vectors
if(*it1 <= *it2)
{
final.push_back(*it1); //final is the final vector to output
//final[count] = *it1; // this does not work for some reason
it1++;
size1++;
//cout<<"count ="<<count<<" size1 ="<<size1<<endl;
}
else
{
final.push_back(*it2);
//final[count] = left[size2];
it2++;
size2++;
}
count++;
//cout<<"count ="<<count<<" size1 ="<<size1<<"size2 = "<<size2<<endl;
}
It seems to me that the two methods should be functionally equivalent.
PS I have already reserved space for the final vector so that shouldnt be a problem.
You can't add new objects to vector using operator[]. .reserve() doesn't add them neither. You have to either use .resize() or .push_back().
Also, you are not avoiding overheads at all; call cost of operator[] isn't really much better that push_back() one, so until you profile your code thorougly, just use push_back. You can still use reserve to make sure unneccessary allocations won't be made.
In most of the cases, "optimizations" like this don't really help. If you want to make your code faster, profile it first and look for the hot paths.
There is a huge difference between
vector[i] = item;
and
vector.push_back(item);
Differences:
The first one modifies the element at index i and i must be valid index. That is,
0 <= i < vector.size() must be true
If i is an invalid index, the first one invokes undefined behavior, which means ANYTHING can happen. You could, however, use at() which throws exception if i is invalid:
vector.at(i) = item; //throws exception if i is invalid
The second one adds an element to the vector at the end, which means the size of the vector increases by one.
Since, sematically both of them do different thing, choose the one which you need.
I'm using a priority queue as a scheduler with one extra requirement. I need to be able to cancel scheduled items. This equates to removing an item from the middle of the priority queue.
I can't use std::priority_queue as access to any element other than top is protected.
I'm trying to use the algorithm's heap functions. But I'm still missing the piece I need. When I remove an element I from the middle of the heap I want it to rebuild itself efficiently. C++ provides these heap functions:
std::make_heap O(3n)
std::push_heap O(lg(n))
std::pop_heap O(2 lg(n))
I want a new function like std::repair_heap with a big-O < 3n. I'd provide it with location of the hole where the canceled item used to reside and it would properly adjust the heap.
It seems to be a huge oversight to not to provide a std::repair_heap function. Am I missing something obvious?
Is there library that provides an stl-compliant std::repair_heap?
Is there a better data structure for modeling a scheduler?
NOTE:
I'm not using an std::map for a few reasons.
A heap has constant memory overhead.
A heap has awesome cache locality.
I guess you know which element in the heap container (index n) you want to delete.
Set the value v[n] = BIG; the value BIG is really bigger than any other values in the heap.
Call std::push_heap( v.begin(), v.begin()+n+1 );
Call std::pop_heap( v.begin(), v.end() );
Call v.pop_back();
Done
Operation is O(ln(n))
RE: request for proof
First, a qualifier:
This method assumes something about the algorithm used by std push_heap.
Specifically, it assumes that std push_heap( v.begin(), v.begin()+n+1 )
will only alter the range [0, n]
for those elements which are ascendants of n, i.e., indices in the following set:
A(n)={n,(n-1)/2,((n-1)/2-1)/2....0}.
Here is a typical spec for std push_heap:
http://www.cplusplus.com/reference/algorithm/push_heap/
"Given a heap range [first,last-1), this function extends the range considered a heap to [first,last) by placing the value in (last-1) into its corresponding location in it."
Does it guarantee to use the "normal heap algorithm" that you read about in textbooks?
You tell me.
Anyway, here is the code which you can run and see, empirically, that it works.
I am using VC 2005.
#include <algorithm>
#include <vector>
#include <iostream>
bool is_heap_valid(const std::vector<int> &vin)
{
std::vector<int> v = vin;
std::make_heap(v.begin(), v.end());
return std::equal(vin.begin(), vin.end(), v.begin());
}
int _tmain(int argc, _TCHAR* argv[])
{
srand(0);
std::vector<int> v;
for (int i=0; i<100; i++)
{
v.push_back( rand() % 0x7fff );
}
std::make_heap(v.begin(), v.end());
bool bfail = false;
while( v.size() >= 2)
{
int n = v.size()/2;
v[n] = 0x7fffffff;
std::push_heap(v.begin(), v.begin()+n+1);
std::pop_heap(v.begin(), v.end());
v.resize(v.size()-1);
if (!is_heap_valid(v))
{
std::cout << "heap is not valid" << std::endl;
bfail = true;
break;
}
}
if (!bfail)
std::cout << "success" << std::endl;
return 0;
}
But I have another problem, which is how to know the index "n" which needs to be deleted. I cannot see how to keep track of that (know the place in the heap) while using std push_heap and std pop_heap. I think you need to write your own heap code and write the index in the heap to an object every time the object is moved in the heap. Sigh.
Unfortunately, the standard is missing this (fairly important) function. With g++, you can use the non-standard function std::__adjust_heap to do this, but there's no easy portable way of doing it -- and __adjust_heap is slightly different in different versions of g++, so you can't even do it portably over g++ versions.
How does your repair_heap() work? Here's my guess:
If your heap is defined by some iterator range, say (heapBegin, heapEnd). The element you want to remove is the root of some subtree of the heap, which is defined by some subrange (subHeapBegin, subHeapEnd). Use std::pop_heap(subHeapBegin, subHeapEnd), then if subHeapEnd != heapEnd, swap the values at *(subHeapEnd-1) and *(heapEnd-1), which should put your deleted item at the end of the heap container. Now you have to percolate the element at *(subHeapEnd-1) up in your subheap. If I haven't missed something, which is possible, then all that remains is to chop the end element off of the heap container.
Before going to the trouble of trying to code that correctly (I've skipped some details like calculating subHeapBegin and subHeapEnd), I'd run some tests to determine if make_heap() really slows you down. Big-O is useful, but it's not the same thing as actual execution time.
It seems to me that removing from the middle of a heap might mean the entire heap has to be rebuilt: The reason there's no repair_heap is because it would have to do the same (big-oh) work as make_heap.
Are you able to do something like put std::pair<bool, Item> in the heap and just invalidate items instead of removing them? Then when they finally get to the top just ignore the item and move along.
You can try ‘std::multiset’ which is implemented as the heap structure and support ‘std::erase’ operation, so you could ‘std::find’ the element then erase it.
Here's a bit of delphi code i used to remove items from a heap. I don't know this C++ of which you speak and don't have a repair function, but hey..
first the pop, so you get an idea of how the thing works:
function THeap.Pop: HeapItem;
begin
if fNextIndex > 1 then begin
Dec(fNextIndex);
Result:= fBuckets[1]; //no zero element
fBuckets[1] := fBuckets[fNextIndex];
fBuckets[fNextIndex] := nil;
FixHeapDown; //this has a param defaulting to
end
else
Result:= nil;
end;
now to contrast, the deletion:
procedure THeap.Delete(Item: HeapItem);
var
i:integer;
begin
for i:=1 to pred(fNextIndex) do
if Item=fBuckets[i] then begin
dec(fNextIndex);
fBuckets[i] := fBuckets[fNextIndex];
fBuckets[fNextIndex] := nil;
FixHeapDown(i);
break;
end;
end;
its of course a no-no to even think about
doing what we're doing here, but hey, costs
do change sometimes and jobs do get canceled.
enjoy.
i hope this helps.