Given a std::priority_queue to which elements are being added faster than they are being removed by the usual process of repeatedly popping the best element, so that the program is going to run out of memory unless something is done,
Is there any way to throw away the worst half of the elements, while leaving the best half to be processed one at a time as normal?
There isn't a direct way. But a binary heap doesn't really support that operation anyways.
But it's not hard to indirectly do so:
Create a temporary empty priority queue
Swap the main and temporary queues
Enter a loop that pops from the temporary and pushes to the main
Stop when you're happy with the number of copied elements
Destroy the temporary queue.
Clearly not, since the interface to a std::priority_queue is so extremely limited. You could implement your own priority queue that will let you do this using make_heap, push_heap and pop_heap (this is how std::priority_queue is implemented) and implementing your own function to remove the worst half of the elements.
The std::priority_queue is a 2-heap and as such only partially ordered. The data-structure is not useful to locate the best half of the elements differently than extracting them.
Related
I am confused about time complexity of push pop of priority queue and insertion deletion of max_heap.Do they take same time when the input is very long like 10^5 ??which one will be faster priority queue or max_heap??I am trying to solve a problem where input is very long.But when I use heap I get runtime error.
priority_queue is implemented in terms of a heap, it's just providing the interface that guarantees the heap invariant is maintained (unlike using a plain vector + make_heap, where it's initially a heap, but the developer can break the invariant by misusing the vector directly afterwards).
If you're having troubles with the result of make_heap that you don't have with priority_queue, it's probably because your code for manipulating the heap is incorrect; they should both work equally well (priority_queue just simplifies things for you a bit). Without a reproducer, I can't say what you did wrong.
I have an algorithm which essentially takes elements of type T, pushes them to a priority_queue, and then takes the top element of the queue, modifies it, updates the heap, takes the (possibly new) top element, and so on.
I'd like to try the Boost heap library to replace std::priority_queue to avoid having to update by a pop followed by a push of the same element. However, I cannot seem to find a way to access a handle to the top of the queue, to update it.
Specifically there seems to be only two ways to get a handle to an element:
s_handle_from_iterator(it). However, the iterator returned by begin() is not necessarily the top of the stack, and this method does not take ordered_iterator, so it seems I'm out of luck?
push(elem) returns the handle. However, using that seems to mean that I have to store the handle somewhere, probably in the element itself, which seems like a waste of space (and time, when handling the elements).
Is there a better way?
Note that for a slightly different use case (iterating the whole heap in order) this answer seems to suggest that there really isn't a better way...
I have a priority_queue, and I want to modify some of it's contents (the priority value), will the queue be resorted then?
It depends if it resorts on push/pop (more probable, becouse you just need to "insert", not resort whole), or when accessing top or pop.
I really want to change some elements in the queue. Something like that:
priority_queue<int> q;
int a=2,b=3,c=5;
int *ca=&a, *cb=&b, cc=&c;
q.push(a);
q.push(b);
q.push(c); //q is now {2,3,5}
*ca=4;
//what happens to q?
// 1) {3,4,5}
// 2) {4,2,5}
// 3) crash
priority_queue copies the values you push into it. Your assignment at the end there will have zero effect on the order of the priority queue, nor the values stored inside of it.
Unfortunately, the std::priority_queue class doesn't support the increase/decrease_key operations that you're looking for. Of course it's possible to find the element within the heap you want to update, and then call make_heap to restore the binary heap invariants, but this can't be done as efficiently as it should be with the std:: container/algorithms. Scanning the heap to find the item is O(N) and then make_heap is O(N) on top of that - it should be possible to do increase/decrease_key in O(log(N)) for binary heaps that properly support updates.
Boost provides a set of priority queue implementations, which are potentially more efficient than the std::priority_queue (pairing heaps, Fibonacci heaps, etc) and also offer mutability, so you can efficiently perform dynamic updates. So all round, using the boost containers is potentially a much better option.
Okay, after searching a bit I found out how to "resort" queue, so after each priority value change you need to call:
std::make_heap(const_cast<Type**>(&queue.top()),
const_cast<Type**>(&queue.top()) + queue.size(),
ComparerClass());
And queue must be then
std::priority_queue<Type*,vector<Type*>,ComparerClass> queue;
Hope this helps.
I stumbled on this issue while considering the use of priority queues for an A* algorithm.
Basically, C++ priority queues are a very limited toy.
Dynamically changing the priority of a queued element requires to perform a complete reconstruction of the underlying heap manually, which is not even guaranteed to work on a given STL implementation and is grossly inefficient.
Besides, reconstructing the heap requires butt-ugly code, which would have to be hidden in yet another obfuscated class/template.
As for so many other things in C++, you'll have to reinvent the wheel, or find whatever fashionable library that reinvented it for you.
I'm trying to find a fast way to do input but I've learned that using STL for such purposes might be slow.
I have a callback that fires whenever I get Keyboard input.
It creates an object with (int _key, int _state, int _life)
Everytime I receive this callback, I push_back the object to my std::vector;
Every frame I check the top of this vector and remove the "dead" input.
The vector can be polled for whatever input is valid at that moment which means it will be searched frequently.
Optimizations:
-All the memory should be contiguous so although Link Lists are better for dynamic allocation, should I stick with STL's vector? I'm always adding to the top and removing from the bottom so what data struct should I use?
-I was thinking of having a buffer(second vector) that continuously receives new input from the callback, then each frame copy the data from that vector to the top of my active input vector. Since the active vector will be polled, would this increase performance since it won't be wasting time getting added to during the loop?
Basically I'm trying to squeeze as much performance from this vector as possible and I could use some help.
What you are describing, adding data in one end, removing in another, is the archetypical description of a queue. This is implemented in the standard library with the std::queue class.
This queue class is a so-called container adapter, meaning it uses another container for the actual storage. By default it uses std::deque, but that container doesn't keep its data in a contiguous memory area. However you can declare a std::queue with almost any other standard container, like std::vector (which is the only container guaranteed to store data in a contiguous memory area):
std::queue<int, std::vector> my_queue_with_vector;
my_queue_with_vector.push(1);
my_queue_with_vector.push(2);
my_queue_with_vector.push(3);
while (!my_queue_with_vector.empty())
{
std::cout << my_queue_with_vector.top() << '\n';
my_queue_with_vector.pop(); // Remove top element in the queue
}
std::deque makes the best container. It guarantees O(1) pop_front and push_back() and has random access and a good degree (although not completely) of cache behaviour.
But if you absolutely must have complete contiguity, then you'll need to look into a custom circular buffer container (there's one in Boost IIRC), as pop_front() on a vector is rather expensive.
Edit: As another poster has pointed out, keyboard input is so infrequent even for a very fast typist that I find it difficult to believe that this is a bottleneck in your system.
Sounds like you want a std::deque. Adjacents elements are nearly always allocated continuously in memory and it has constant time insertion and removal at begin and end.
Short answer: It doesn't matter. std::vector and std::list can easily handle millions of insert operations per second, but most typists don't type faster than 10 characters per second on a keyboard.
Long answer: push_back and erase are usually very cheap on a vector if the vector is small (< 100) and the copy/swap operations of the objects stored on the vector are cheap. The allocations used to insert into or remove from an std:list are usually more expensive. If it becomes an issue, measure the cost.
An std::deque also has allocations and depending on the implementation is likely more expensive than the vector in your case, if my assumption that your vector rarely if ever contains more than 10 items - all of which are cheap to copy - is correct.
I'm trying to design a task scheduler to a game engine. A task could be an animation, a trigger controller, etc.
My problem is what container to choose. The idea is: when you insert a new task, the container must reorder and put the task in the proper place. Once executed, task could change and be scheduled again or deleted. This is mainly push and pop.
But, if possible, it would be nice if I could have random access to an element, but not vital. No matter if the container supports one or more elements with the same key.
I think that priority queue fits my needs but I saw that is based on vector implementation, and I think that this container must be somehow optimized to push and pop.
Opinions?
(source: adrinael.net)
(original source: Liam Devine)
A priority queue seems to be the best option for you.
As you can see, the pop functions has a constant complexity and the push function is logarithmic in time.
std::vector is pretty good for this task, especially if the "steady-state" size of the container remains reasonably constant (you have a number of tasks on the queue doesn't differ widely).
If you need an updatable queue (and std::priority_queue is not), I would suggest you use the d_ary_heap_indirect (which can be found in the Boost.Graph "detail" folder). This is a priority queue used a lot for Dijkstra and A* algorithms that require an updatable priority queue. Random-access is necessary, anyways. Also, using an indirect makes the insertion and deletion from the queue quite efficient. Finally, you can choose your container (as a template argument), but it has to be random-access (so, you can try either vector or deque). Pop is constant-time, push and/or update is log-time, and the proper choice of container will make the container insertion constant-amortized (and the d_ary_heap_indirect amortizes a second time as well, so I wouldn't worry about that).
The vector is optimized for push and pop at one end. :-)
To prioritize you will have to sort the tasks. A vector isn't that bad, if the number of objects is reasonably small, even if it means copying objects during the sort.
Other containers, like linked lists, instead suffer from the need to allocate a new node for each object.
You can specify the container type you want with std::priority_queue.
However: you're storing pointers (I presume, since it sounds like what you're is
polymorphic and has identity), so copying is cheap. You're managing it
as a heap (that's what std::priority_queue does), so insertions are done
using push_back and a number of swaps (lg(n) max). I can't see any
reason to even consider another structure than std::vector.
std::priority_queue does hide all of the direct access operators (e.g.
operator[]). It does this because if you modify an entry, you're
likely to invalidate the heap (which is a class invariant of the class).
If you do want to provide direct read access, however, the underlying
container is only protected, not private, so you can derive from it
and add the operators you want. I'd very much limit it to const
operators, however.
Depends on how often you're going to be adding tasks and pulling tasks off (and presumably executing them) and how many there are.
If you're going to have tons of little tasks, then prefer priority queue because the cost of node allocation will probably not hurt you as much as the asymptotic growth of n log n for the sort.
If you're going to have a small number of tasks that constantly keep changing priority, then sorting a vector might be reasonable, but you want to use an sorting algorithm that works well when the list is almost sorted.
Scheduling is an art though and you're going to have to profile it once you build it. There's probably too little information at this point so say. I'd lean towards a priority queue, but keep other options in mind if performance isn't adequate.