Is there any inbuilt function for deleting a given element (other than top element) in priority queue class of C++ STL? If not how to delete it in O(log n)?Should i implement the heap data structure from scratch for this 'delete' functionality?
Is there any inbuilt function for deleting a given element (other than top element) in priority queue class of C++ STL?
No.
If not how to delete it in O(log n)?
By using another container. std::set is the simplest compromise. A custom heap implementation may be more optimal.
There is no inbuilt function for deleting a given element(other than top element) in priority queue.
I would recommend you to use std::set which performs the operations in O(logN) by implementing binary tree. But in case you need more better time complexity use std::unordered_set which performs operations in O(1) time and uses hashing.
So my advice will be that use std::set or std::unordered_set & don't restrict yourself to priority queue only.
As suggested by this solution, you can do something like this:
template<typename T>
class custom_priority_queue : public std::priority_queue<T, std::vector<T>>
{
public:
template< typename UnaryPredicate >
T pop_match_or_top(UnaryPredicate p) {
auto it = std::find_if(this->c.begin(), this->c.end(), p);
if (it != this->c.end()) {
T value = std::move(*it);
this->c.erase(it);
std::make_heap(this->c.begin(), this->c.end(), this->comp);
return value;
}
else {
T value = this->top();
this->pop();
return value;
}
}
};
This is specially useful when you need to take elements that are close to the top but are not exactly the top.
Related
When configured for mutability, boost::heap::d_ary_heap uses a std::list in addition to the vector that holds the values of the heap nodes. I realize that the handles which are being provided for making the mutable_heap_interface work are in fact iterators of this list, but I'm wondering why such an expensive solution was chosen, and if there's a leaner way to achieve mutability with boost::heap::d_ary_heap.
Mutability requires a way to find the index of a node in the heap vector, given the node itself. Some kind of backward pointer needs to be maintained. Can't this be achieved by storing this backwards pointer in the node, and maintain it by the move/copy constructors/assignment-operators of the value type?
Is there a good reason why it needs to be as expensive as a doubly-linked list?
This is kind of an answer to my own question that only speculates why the boost design is as it is, and presents a partial solution to what I would have liked to get with the boost data structure. I'm still interested in receiving further insight into the rationale behind the boost implementation, and of course also feedback on the solution I present below.
Let me first explain the piece of code below, before going on to discuss its merits and problems, and then comment on the boost.heap implementation, why it presumably is like it is, and why I don't like it.
The code below is based on the venerable std::priority_queue. It splits the node managed by the priority queue into a handle and a body. The handle goes into the heap at the core of the priority_queue, and therefore moves around in the underlying vector as entries are added or removed. The handle only contains the priority value and a pointer to the body, in order to make it cheap to move it around. The body is a potentially large object that remains stationary in memory. It holds a backpointer to the handle, because the handle must be invalidated when the body's priority changes, or the body disappears.
Since the handle moves around in the heap, the backpointer in the body must be updated each time the handle changes location. This is done in the move constructor and the move assignment operator of the handle. If a handle gets invalidated, both the pointer in it and the backpointer pointing at it are nulled.
#include <queue>
//! Priority queue that works with handles to managed objects.
template<typename Prio, typename Object> struct PriorityQueue {
struct Entry;
//! Each heap entry is a handle, consisting of a pointer to the managed object and a priority value.
struct Entry {
Object *obj_;
Prio val_;
Entry(Entry const &) =delete;
Entry &operator=(Entry const &) =delete;
~Entry() {
if(obj_)
obj_->setLink(nullptr);
}
Entry(Object &obj, Prio val)
: obj_{&obj}
, val_{val}
{
if(obj_)
obj_->setLink(this);
}
Entry(Entry &&v)
: obj_{v.obj_}
, val_{v.val_}
{
if(obj_)
obj_->setLink(this);
v.obj_ = nullptr;
}
Entry &operator=(Entry &&v) {
if(&v != this) {
val_ = v.val_;
if(obj_)
obj_->setLink(nullptr);
obj_ = v.obj_;
if(obj_)
obj_->setLink(this);
v.obj_ = nullptr;
}
return *this;
}
friend bool operator<(Entry const &a, Entry const &b) {
return a.val_ < b.val_;
}
};
Prio add(Object &obj, Prio val) {
while(!heap_.empty() && !heap_.top().obj_)
heap_.pop();
heap_.emplace(obj, val);
return heap_.top().val_;
}
Prio remove(Object &obj) {
// We can't remove the entry straight away, so we null the pointer
// and leave the entry in the heap, where it will eventually bubble
// up to the root position, from where it can be removed.
if(obj.getLink()) {
obj.getLink()->obj_ = nullptr;
obj.setLink(nullptr);
}
while(!heap_.empty() && !heap_.top().obj_)
heap_.pop();
return heap_.empty() ? INT64_MAX : heap_.top().val_;
}
Prio update(Object &obj, Prio val) {
remove(obj);
return add(obj, val);
}
std::priority_queue<Entry> heap_;
};
//! Example of a managed object.
struct MyObject {
MyObject(MyObject const &) =delete;
MyObject &operator=(MyObject const &) =delete;
PriorityQueue<int, MyObject>::Entry *getLink() const {
return link_;
}
void setLink(PriorityQueue<int, MyObject>::Entry *link) {
link_ = link;
}
PriorityQueue<int, MyObject>::Entry *link_;
};
Unfortunately, std::priority_queue doesn't support mutability, i.e. you can't remove entries except the root entry, so the fallback is to leave handles in the heap, but invalidate them by breaking the relationship with the body. They will eventually bubble up towards the root, where they can be removed. Obviously, that means that they inflate the size of the heap needlessly, consuming some additional memory and CPU time, which may or may not be significant. If std::priority_queue would expose the internal heap maintenance functions, it would be possible to delete or update entries directly.
It would be possible to reduce the handle size even more by holding the priority in the body rather than the handle, but then the body would need to be consulted for each priority comparison, which would destroy locality of reference. The chosen approach avoids this by holding everything in the handle that is relevant for heap maintenance. The updating of the backpointer in the body by the move constructor and move assignment operator is a write-only operation, which needn't hinder performance, since there typically are write buffers in modern processors that can swallow the associated latency.
For optimizing cache performance, one would wish to use a d-ary heap instead of a binary heap, so that all children of a node (i.e. their handles), which are adjacent in the vector, occupy one cache line. Alas, that's not supported by std::priority_queue, either.
The latter would be supported by boost.heap, but in order to also support mutability, they introduce an additional std::list for the management of the backpointers, which I suspect is rooted in the age of the library. It dates back to before C++11, when move support wasn't yet available in the language. Presumably, only minimal maintenance has been done to it since. I'd welcome them bringing the library up to date and use the opportunity to provide leaner implementations.
So, the bottom line is that I have at least a suspicion that answers my original question, and a design that addresses some of my goals, leaving me with a workable but not yet optimal solution based on the standard library.
Thanks go to the commenters, and remember if you have additional insight to add, you're most welcome.
I try to find optimal data structure for next simple task: class which keeps N last added item values in built-in container. If object obtain N+1 item it should be added at the end of the container and first item should be removed from it. It like a simple queue, but class should have a method GetAverage, and other methods which must have access to every item. Unfortunately, std::queue doesn't have methods begin and end for this purpose.
It's a part of simple class interface:
class StatItem final
{
static int ITEMS_LIMIT;
public:
StatItem() = default;
~StatItem() = default;
void Reset();
void Insert(int val);
int GetAverage() const;
private:
std::queue<int> _items;
};
And part of desired implementation:
void StatItem::Reset()
{
std::queue<int> empty;
std::swap(_items, empty);
}
void StatItem::Insert(int val)
{
_items.push(val);
if (_items.size() == ITEMS_LIMIT)
{
_items.pop();
}
}
int StatItem::GetAverage() const
{
const size_t itemCount{ _items.size() };
if (itemCount == 0) {
return 0;
}
const int sum = std::accumulate(_items.begin(), _items.end(), 0); // Error. std::queue doesn't have this methods
return sum / itemCount;
}
Any ideas?
I'm not sure about std::deque. Does it work effective and should I use it for this task or something different?
P.S.: ITEMS_LIMIT in my case about 100-500 items
The data structure you're looking for is a circular buffer. There is an implementation in the Boost library, however in this situation since it doesn't seem you need to remove items you can easily implement one using a std::vector or std::array.
You will need to keep track of the number of elements in the vector so far so that you can average correctly until you reach the element limit, and also the current insertion index which should just wrap when you reach that limit.
Using an array or vector will allow you to benefit from having a fixed element limit, as the elements will be stored in a single block of memory (good for fast memory access), and with both data structures you can make space for all elements you need on construction.
If you choose to use a std::vector, make sure to use the 'fill' constructor (http://www.cplusplus.com/reference/vector/vector/vector/), which will allow you to create the right number of elements from the beginning and avoid any extra allocations.
I need to implement a queue containing unique entries(no duplicates) in C or C++. I am thinking of maintaining a reference of elements already available in queue but that seems very inefficient.
Kindly let me know your suggestions to tackle this.
How about an auxiliary data structure to track uniqueness:
std::queue<Foo> q;
std::set<std::reference_wrapper<Foo>> s;
// to add:
void add(Foo const & x)
{
if (s.find(x) == s.end())
{
q.push_back(x);
s.insert(std::ref(q.back())); // or "s.emplace(q.back());"
}
}
Or, alternatively, reverse the roles of the queue and the set:
std::set<Foo> s;
std::queue<std::reference_wrapper<Foo>> q;
void add(Foo const & x)
{
auto p = s.insert(x); // std::pair<std::set<Foo>::iterator, bool>
if (s.second)
{
q.push_back(std::ref(*s.first)); // or "q.emplace_back(*s.first);"
}
}
queuing:
use std::set to maintain your set of unique elements
add any element that you were able to add to the std::set to the std::queue
dequeueing:
remove element from std::queue and std::set
std::queue is a container adaptor and uses relatively few members of the underlying Container. You can easily implement a custom container that contains both: an unordered_map of reference_wrapper<T> and a deque<T>. It needs at least members front and push_back. Check inside that hash_map when push_back of your container is called and reject accordingly (possibly throw). To give the complete example:
#include <iostream>
#include <set>
#include <deque>
#include <queue>
#include <unordered_set>
#include <functional>
namespace std {
// partial specialization for reference_wrapper
// is this really necessary?
template<typename T>
class hash<std::reference_wrapper<T>> {
public:
std::size_t operator()(std::reference_wrapper<T> x) const
{ return std::hash<T>()(x.get()); }
};
}
template <typename T>
class my_container {
// important: this really needs to be a deque and only front
// insertion/deletion is allowed to not get dangling references
typedef std::deque<T> storage;
typedef std::reference_wrapper<const T> c_ref_w;
typedef std::reference_wrapper<T> ref_w;
public:
typedef typename storage::value_type value_type;
typedef typename storage::reference reference;
typedef typename storage::const_reference const_reference;
typedef typename storage::size_type size_type;
// no move semantics
void push_back(const T& t) {
auto it = lookup_.find(std::cref(t));
if(it != end(lookup_)) {
// is already inserted report error
return;
}
store_.push_back(t);
// this is important to not have dangling references
lookup_.insert(store_.back());
}
// trivial functions
bool empty() const { return store_.empty(); }
const T& front() const { return store_.front(); }
T& front() { return store_.front(); }
void pop_front() { lookup_.erase(store_.front()); store_.pop_front(); }
private:
// look-up mechanism
std::unordered_set<c_ref_w> lookup_;
// underlying storage
storage store_;
};
int main()
{
// reference wrapper for int ends up being silly
// but good for larger objects
std::queue<int, my_container<int>> q;
q.push(2);
q.push(3);
q.push(2);
q.push(4);
while(!q.empty()) {
std::cout << q.front() << std::endl;
q.pop();
}
return 0;
}
EDIT: You will want to make my_container a proper model of container (maybe also allocators), but this is another full question. Thanks to Christian Rau for pointing out bugs.
There is one very important point you've not mentioned in your question, and that is whether your queue of items is sorted or have some kind of ordering (called a Priority queue), or unsorted (called a plain FIFO). The solution you choose will depend only on the answer to this question.
If your queue is unsorted, then maintaining an extra data structure in addition to your queue will be more efficient. Using a second structure which is ordered in some way to maintain the contents of your queue will allow you check if an item already exists in your queue or not much quicker that scanning the queue itself. Adding to the end of an unsorted queue takes constant time and can be done very efficiently.
If your queue must be sorted, then placing the item into the queue requires you to know the item's position in the queue, which requires the queue to be scanned anyway. Once you know an item's position, you know if the item is a duplicate because if it's a duplicate then an item will already exist at that position in the queue. In this case, all work can be performed optimally on the queue itself and maintaining any secondary data structure is unnecessary.
The choice of data structures is up to you. However, for (1) the secondary data structure should not be any kind of list or array, otherwise it will be no more efficient to scan your secondary index as to scan the original queue itself.
I'm finding that std::sort is very slow with sorting only 1000 items.
In class template template <typename T> class TableModel : public QAbstractTableModel I have the following function to sort a table.
template<typename T>
void TableModel<T>::sort(int column, Qt::SortOrder order = Qt::AscendingOrder) {
if(order == Qt::AscendingOrder) {
qSort(m_list.begin(), m_list.end(), less<T>(column));
} else {
qSort(m_list.begin(), m_list.end(), greater<T>(column));
}
reset();
}
I notice if I only have the randomly shuffle my table is shuffles then displays instantly. So this leads me to think that its sort that is slow. Can anyone help me speed up the sorting of a QTable?
Here is the less struct.
template<typename T>
struct less {
int index;
less(int index) : index(index) {}
bool operator()(const T& first, const T& second) {
return T::less(first, second, index);
}
};
T::less is a function and all it does it the less than comparison based on the index given.
Slow is defined as a 5 seconds for only 1000 items when I need to handle about 100,000 items later on.
I suspect that m_list is storing the items by value and that swapping them is expensive. You could try to either implement a faster swap or store them in the container by smart pointer.
Of course a profiler could help you pinpoint the problem much more precisely.
Since m_list is a QList it does not have the same interface or performance characteristics as a normal list. For example, apparently a QList stores an array of T* internally. This representation could be sorted without any copying if the sort algorithm is aware of this implementation detail. By contrast std::sort is probably deep copying the values around, or maybe moving them, which is going to be more work than sorting pointers in the QList array.
It's probably best to use Qt containers with Qt algorithms, since Qt algorithms are more likely to be specialized for Qt containers. Or you could avoid using Qt containers and just stick with the standard library.
Anyway, try using Qt's qSort algorithm:
template<typename T>
void TableModel<T>::sort(int column, Qt::SortOrder order = Qt::AscendingOrder) {
if(order == Qt::AscendingOrder) {
qSort(m_list.begin(), m_list.end(), less<T>(column));
} else {
qSort(m_list.begin(), m_list.end(), greater<T>(column));
}
reset();
}
Original answer
std::sort can't take advantage of the fact that nodes in the list can be moved around without copying the element. Assuming you're using std::list or something similar, use the sort member function.
template<typename T>
void TableModel<T>::sort(int column, Qt::SortOrder order = Qt::AscendingOrder) {
std::random_shuffle(m_list.begin(), m_list.end());
if(order == Qt::AscendingOrder) {
m_list.sort(less<T>(column));
} else {
m_list.sort(greater<T>(column));
}
reset();
}
If you can't do that then you may be able to optimize all those copies by making sure that your elements are move-enabled if you're using C++11.
I'm implementing a A* search algorithm but I keep running into problems with the priority queue. I have implemented a custom comparator for the priority queue according to this article
This is the relevant code:
class CNode;
struct CompareNode : public binary_function<CNode*, CNode*, bool> {
bool operator()(const CNode* lhs, const CNode* rhs) const {
return lhs->m_costFromStart+lhs->m_heuristic > rhs->m_costFromStart+rhs->m_heuristic;
}
};
bool AStarSearch(CNode* start, CNode* end) {
priority_queue<CNode*, vector<CNode*>, CompareNode> open;
...
}
Call stack:
std::_Debug_heap ...
std::pop_heap ...
std::priority_queue<CNode *,std::vector<CNode *,std::allocator<CNode *> >,CompareNode>::pop()
AStarSearch(CNode * start=0x0f9a23b8, CNode * end=0x0f9a24e8)
Greater then was used as I needed a min heap for this algorithm.
The implementation seems to work fine and the problem goes away when it is run in release mode but the priority queue occasionally throws "Invalid heap" assertion failures in debug mode when the priority queue is pop()ed.
I'm not familiar with binary_function in stl but the problem seems to lie with the comparator. Removing the comparator or changing the sign to less then removes the error but that would give me a max heap. Is there something I'm missing?
Thanks for the help. Turns out I did not rebuild the heap after changing the cost of nodes in the priority queue. Calling
make_heap(const_cast<CNode**>(&open.top()), const_cast<CNode**>(&open.top()) + open.size(),
CompareNode());
after every modification to the pq solved the problem.