Related
Suppose I have a list<int> and maintaining it in ordered state. Can I isert new values into it with logarithmic complexity with code like this
#include <iostream>
#include <random>
#include <list>
#include <algorithm>
using namespace std;
ostream& operator<<(ostream& out, const list<int> data) {
for(auto it=data.begin(); it!=data.end(); ++it) {
if(it!=data.begin()) {
out << ", ";
}
out << (*it);
}
return out;
}
int main() {
const int max = 100;
mt19937 gen;
uniform_int_distribution<int> dist(0, max);
list<int> data;
for(int i=0; i<max; ++i) {
int val = dist(gen);
auto it = lower_bound(data.begin(), data.end(), val);
data.insert(it, val);
}
cout << data << endl;
}
I would say not, because it is impossible to position iterator in list in O(1) but documentation says strange:
The number of comparisons performed is logarithmic in the distance
between first and last (At most log2(last - first) + O(1)
comparisons). However, for non-LegacyRandomAccessIterators, the number
of iterator increments is linear. Notably, std::set and std::multiset
iterators are not random access, and so their member functions
std::set::lower_bound (resp. std::multiset::lower_bound) should be
preferred.
i.e. it doesn't recomment to use this function for set which is alterady search tree internally. Which containers this function is inteded to use then? How to insert and maintain sorted then?
Will std::lower_bound be logarithmic for list<>?
No. Quote from documentation:
for non-LegacyRandomAccessIterators, the number of iterator increments is linear.
Which containers this function is inteded to use then?
std::lower_bound is intended for any container that is - or can be - ordered, and doesn't have faster lower bound algorithm that relies on its internal structure - which excludes std::set and std::multiset as mentioned in the documentation.
Is it possible to get an iterator over a vector that filters some element with a predicate, i.e. showing a view of the vector?
I think remove_if does something similar but I have not found whether I can use it as I want to or not.
Something like:
auto it = filter(vec.begin(), vec.end(), predicate);
// I can reuse the iterator like:
for (auto i = it; i != vec.end(); i++)
// ...
Edit: (A bit more context to get the best answer) I am doing a lot of queries in an sqlite database of log data in order to print a report.
The performances are not good at the moment because of the number of request needed. I believe querying once the database and storing the result in a vector of smart pointers (unique_ptr if possible), then querying the vector with pure C++ may be faster.
Using copy_if is a good way to do the queries, but I don't need to copy everything and it might cost too much at the end (not sure about that), I should have mentioned than the data are immutable in my case.
As #Jarod42 mentioned in the comments one solution would be using ranges:
#include <algorithm>
#include <iostream>
#include <vector>
#include <range/v3/view/filter.hpp>
#include <range/v3/view/transform.hpp>
int main()
{
std::vector<int> numbers = { 1, 2, 3 ,4, 5 };
auto predicate = [](int& n){ return n % 2 == 0; };
auto evenNumbers = numbers | ranges::view::filter(predicate);
auto result = numbers | ranges::view::filter(predicate)
| ranges::view::transform([](int n) { return n * 2; });
for (int n : evenNumbers)
{
std::cout << n << ' ';
}
std::cout << '\n';
for (int n : result)
{
std::cout << n << ' ';
}
}
evenNumbers is a range view adapter which sticks to the numbers range and changes the way it iterates.
result is a ranges of numbers that have been filtered on the predicate and then have been applied a funciton.
see the compile at compiler-explorer
credit: fluentcpp
Your question
Can we get an iterator that filters a vector from a predicate in C++?
in the sense you are asked it, can only be answered with: No. At the moment not (C++17). As per your requirement the iterator then would have to store the predicate and checking that for each modification of the position or for all dereferencing stuff. I.e before any dereferencing, the predicate would need to be checked. Because other code could modifiy your std::vector. The the iterator would need to check the predicate all the time. Also standard functionality like begin, end, distance would be rather complicated.
So you could create your own iterator by deriving from an existing iterator. Store the predicate and overload most of the functions to take care of the predicate. Very, very complicated, much work and maybe not, what you want to have. This would be the only way to get exact your requested functionality.
For work arounds, there are are many other possible solutions. Peolple will show you here.
But if I read your statement
"showing a view of the vector"
then life becomes easier. You can easily create a view of a vector by copying it conditionally with std::copy_if, as oblivion has written. That is in my opinion the best answer. It is none destructive. But it is a snapshot and not the original data. So, it is read only. And, it does not take into account changes to the original std::vector after the snapshot has been taken.
The second option, a combination of std::remove_if and std::erase, will destroy the original data. Or better said, it will invalidate the filtered out data. You could also std::copy_if the unwanted data to a backup area, std::remove_if them, and at the end add them again to the vector.
All these methods are critical, if the original data will be modified.
Maybe for you the standard std::copy_if is best to create a view. You would then return an iterator of copy and work with that.
#include <iostream>
#include <vector>
#include <algorithm>
int main()
{
std::vector<int> testVector{ 1,2,3,4,5,6,7 }; // Test data
std::vector<int> testVectorView{}; // The view
// Create predicate
auto predForEvenNumbers = [](const int& i) -> bool { return (i % 2 == 0); };
// And filter. Take a snapshot
std::copy_if(testVector.begin(), testVector.end(), std::back_inserter(testVectorView), predForEvenNumbers);
// Show example result
std::vector<int>::iterator iter = testVectorView.begin();
std::cout << *iter << '\n';
return 0;
}
Please note. For big std::vectors, it will become a very expensive solution . . .
I have a data structure that consists of three int values that represent a coordinate, and a double that represents a value at that coordinate. I would like to store them together, and sort them on value. Values are not unique. Right now, I have them in a struct and I sort them using a lambda, as is shown in the code below. As this is a piece of performance-critical code, I am looking for an implementation that gives the fastest sorting. The list will contain 10^6 to 10^7 elements.
What is the most elegant way to solve this? I am not trying to use std::sort, but I am mostly asking whether to store the data in a struct is the best solution, or are there better alternatives?
#include <vector>
#include <algorithm>
#include <iostream>
struct Data
{
int i;
int j;
int k;
double d;
};
int main()
{
std::vector<Data> v;
v.push_back({1,2,3,0.6});
v.push_back({1,2,3,0.2});
v.push_back({1,2,3,0.5});
v.push_back({1,2,3,0.1});
v.push_back({1,2,3,0.4});
std::sort(v.begin(), v.end(), [](const Data& a, const Data& b)
{ return a.d < b.d; });
for (auto d : v)
std::cout << d.i << ", " << d.j << ", "
<< d.k << ", " << d.d << std::endl;
return 0;
}
The fastest way to sort them is to not have to sort them.
At the expense of some slightly slower insertion, you could store your entire container sorted, and insert only in the correct place. A std::set could help you here, or you could roll your own.
edit: A std::multiset would provide the same advantages if you need to allow values that compare equal.
Duplicate Question, Fastest way to search and sort vectors is a far better answer than I could give.
Summary,
You need a better sample set, 5 entries isn't going to tell you anything. You're not going to be able to beat std::sort. In particular to you, the floating point compare will be the painful bit.
I currently have a std::map<std::string,int> that stores an integer value to a unique string identifier, and I do look up with the string. It does mostly what I want, except that it does not keep track of the insertion order. So when I iterate the map to print out the values, they are sorted according to the string; but I want them to be sorted according to the order of (first) insertion.
I thought about using a vector<pair<string,int>> instead, but I need to look up the string and increment the integer values about 10,000,000 times, so I don't know whether a std::vector will be significantly slower.
Is there a way to use std::map or is there another std container that better suits my need?
I'm on GCC 3.4, and I have probably no more than 50 pairs of values in my std::map.
If you have only 50 values in std::map you could copy them to std::vector before printing out and sort via std::sort using appropriate functor.
Or you could use boost::multi_index. It allows to use several indexes.
In your case it could look like the following:
struct value_t {
string s;
int i;
};
struct string_tag {};
typedef multi_index_container<
value_t,
indexed_by<
random_access<>, // this index represents insertion order
hashed_unique< tag<string_tag>, member<value_t, string, &value_t::s> >
>
> values_t;
You might combine a std::vector with a std::tr1::unordered_map (a hash table). Here's a link to Boost's documentation for unordered_map. You can use the vector to keep track of the insertion order and the hash table to do the frequent lookups. If you're doing hundreds of thousands of lookups, the difference between O(log n) lookup for std::map and O(1) for a hash table might be significant.
std::vector<std::string> insertOrder;
std::tr1::unordered_map<std::string, long> myTable;
// Initialize the hash table and record insert order.
myTable["foo"] = 0;
insertOrder.push_back("foo");
myTable["bar"] = 0;
insertOrder.push_back("bar");
myTable["baz"] = 0;
insertOrder.push_back("baz");
/* Increment things in myTable 100000 times */
// Print the final results.
for (int i = 0; i < insertOrder.size(); ++i)
{
const std::string &s = insertOrder[i];
std::cout << s << ' ' << myTable[s] << '\n';
}
Tessil has a very nice implementaion of ordered map (and set) which is MIT license. You can find it here: ordered-map
Map example
#include <iostream>
#include <string>
#include <cstdlib>
#include "ordered_map.h"
int main() {
tsl::ordered_map<char, int> map = {{'d', 1}, {'a', 2}, {'g', 3}};
map.insert({'b', 4});
map['h'] = 5;
map['e'] = 6;
map.erase('a');
// {d, 1} {g, 3} {b, 4} {h, 5} {e, 6}
for(const auto& key_value : map) {
std::cout << "{" << key_value.first << ", " << key_value.second << "}" << std::endl;
}
map.unordered_erase('b');
// Break order: {d, 1} {g, 3} {e, 6} {h, 5}
for(const auto& key_value : map) {
std::cout << "{" << key_value.first << ", " << key_value.second << "}" << std::endl;
}
}
Keep a parallel list<string> insertionOrder.
When it is time to print, iterate on the list and do lookups into the map.
each element in insertionOrder // walks in insertionOrder..
print map[ element ].second // but lookup is in map
If you need both lookup strategies, you will end up with two containers. You may use a vector with your actual values (ints), and put a map< string, vector< T >::difference_type> next to it, returning the index into the vector.
To complete all that, you may encapsulate both in one class.
But I believe boost has a container with multiple indices.
What you want (without resorting to Boost) is what I call an "ordered hash", which is essentially a mashup of a hash and a linked list with string or integer keys (or both at the same time). An ordered hash maintains the order of the elements during iteration with the absolute performance of a hash.
I've been putting together a relatively new C++ snippet library that fills in what I view as holes in the C++ language for C++ library developers. Go here:
https://github.com/cubiclesoft/cross-platform-cpp
Grab:
templates/detachable_ordered_hash.cpp
templates/detachable_ordered_hash.h
templates/detachable_ordered_hash_util.h
If user-controlled data will be placed into the hash, you might also want:
security/security_csprng.cpp
security/security_csprng.h
Invoke it:
#include "templates/detachable_ordered_hash.h"
...
// The 47 is the nearest prime to a power of two
// that is close to your data size.
//
// If your brain hurts, just use the lookup table
// in 'detachable_ordered_hash.cpp'.
//
// If you don't care about some minimal memory thrashing,
// just use a value of 3. It'll auto-resize itself.
int y;
CubicleSoft::OrderedHash<int> TempHash(47);
// If you need a secure hash (many hashes are vulnerable
// to DoS attacks), pass in two randomly selected 64-bit
// integer keys. Construct with CSPRNG.
// CubicleSoft::OrderedHash<int> TempHash(47, Key1, Key2);
CubicleSoft::OrderedHashNode<int> *Node;
...
// Push() for string keys takes a pointer to the string,
// its length, and the value to store. The new node is
// pushed onto the end of the linked list and wherever it
// goes in the hash.
y = 80;
TempHash.Push("key1", 5, y++);
TempHash.Push("key22", 6, y++);
TempHash.Push("key3", 5, y++);
// Adding an integer key into the same hash just for kicks.
TempHash.Push(12345, y++);
...
// Finding a node and modifying its value.
Node = TempHash.Find("key1", 5);
Node->Value = y++;
...
Node = TempHash.FirstList();
while (Node != NULL)
{
if (Node->GetStrKey()) printf("%s => %d\n", Node->GetStrKey(), Node->Value);
else printf("%d => %d\n", (int)Node->GetIntKey(), Node->Value);
Node = Node->NextList();
}
I ran into this SO thread during my research phase to see if anything like OrderedHash already existed without requiring me to drop in a massive library. I was disappointed. So I wrote my own. And now I've shared it.
Here is solution that requires only standard template library without using boost's multiindex:
You could use std::map<std::string,int>; and vector <data>; where in map you store the index of the location of data in vector and vector stores data in insertion order. Here access to data has O(log n) complexity. displaying data in insertion order has O(n) complexity. insertion of data has O(log n) complexity.
For Example:
#include<iostream>
#include<map>
#include<vector>
struct data{
int value;
std::string s;
}
typedef std::map<std::string,int> MapIndex;//this map stores the index of data stored
//in VectorData mapped to a string
typedef std::vector<data> VectorData;//stores the data in insertion order
void display_data_according_insertion_order(VectorData vectorData){
for(std::vector<data>::iterator it=vectorData.begin();it!=vectorData.end();it++){
std::cout<<it->value<<it->s<<std::endl;
}
}
int lookup_string(std::string s,MapIndex mapIndex){
std::MapIndex::iterator pt=mapIndex.find(s)
if (pt!=mapIndex.end())return it->second;
else return -1;//it signifies that key does not exist in map
}
int insert_value(data d,mapIndex,vectorData){
if(mapIndex.find(d.s)==mapIndex.end()){
mapIndex.insert(std::make_pair(d.s,vectorData.size()));//as the data is to be
//inserted at back
//therefore index is
//size of vector before
//insertion
vectorData.push_back(d);
return 1;
}
else return 0;//it signifies that insertion of data is failed due to the presence
//string in the map and map stores unique keys
}
You cannot do that with a map, but you could use two separate structures - the map and the vector and keep them synchronized - that is when you delete from the map, find and delete the element from the vector. Or you could create a map<string, pair<int,int>> - and in your pair store the size() of the map upon insertion to record position, along with the value of the int, and then when you print, use the position member to sort.
One thing you need to consider is the small number of data elements you are using. It is possible that it will be faster to use just the vector. There is some overhead in the map that can cause it to be more expensive to do lookups in small data sets than the simpler vector. So, if you know that you will always be using around the same number of elements, do some benchmarking and see if the performance of the map and vector is what you really think it is. You may find the lookup in a vector with only 50 elements is near the same as the map.
Another way to implement this is with a map instead of a vector. I will show you this approach and discuss the differences:
Just create a class that has two maps behind the scenes.
#include <map>
#include <string>
using namespace std;
class SpecialMap {
// usual stuff...
private:
int counter_;
map<int, string> insertion_order_;
map<string, int> data_;
};
You can then expose an iterator to iterator over data_ in the proper order. The way you do that is iterate through insertion_order_, and for each element you get from that iteration, do a lookup in the data_ with the value from insertion_order_
You can use the more efficient hash_map for insertion_order since you don't care about directly iterating through insertion_order_.
To do inserts, you can have a method like this:
void SpecialMap::Insert(const string& key, int value) {
// This may be an over simplification... You ought to check
// if you are overwriting a value in data_ so that you can update
// insertion_order_ accordingly
insertion_order_[counter_++] = key;
data_[key] = value;
}
There are a lot of ways you can make the design better and worry about performance, but this is a good skeleton to get you started on implementing this functionality on your own. You can make it templated, and you might actually store pairs as values in data_ so that you can easily reference the entry in insertion_order_. But I leave these design issues as an exercise :-).
Update: I suppose I should say something about efficiency of using map vs. vector for insertion_order_
lookups directly into data, in both cases are O(1)
inserts in the vector approach are O(1), inserts in the map approach are O(logn)
deletes in the vector approach are O(n) because you have to scan for the item to remove. With the map approach they are O(logn).
Maybe if you are not going to use deletes as much, you should use the vector approach. The map approach would be better if you were supporting a different ordering (like priority) instead of insertion order.
This is somewhat related to Faisals answer. You can just create a wrapper class around a map and vector and easily keep them synchronized. Proper encapsulation will let you control the access method and hence which container to use... the vector or the map. This avoids using Boost or anything like that.
// Should be like this man!
// This maintains the complexity of insertion is O(logN) and deletion is also O(logN).
class SpecialMap {
private:
int counter_;
map<int, string> insertion_order_;
map<string, int> insertion_order_reverse_look_up; // <- for fast delete
map<string, Data> data_;
};
There is no need to use a separate std::vector or any other container for keeping track of the insertion order. You can do what you want as shown below.
If you want to keep the insertion order then you can use the following program(version 1):
Version 1: For counting unique strings using std::map<std::string,int> in insertion order
#include <iostream>
#include <map>
#include <sstream>
int findExactMatchIndex(const std::string &totalString, const std::string &toBeSearched)
{
std::istringstream ss(totalString);
std::string word;
std::size_t index = 0;
while(ss >> word)
{
if(word == toBeSearched)
{
return index;
}
++index;
}
return -1;//return -1 when the string to be searched is not inside the inputString
}
int main() {
std::string inputString = "this is a string containing my name again and again and again ", word;
//this map maps the std::string to their respective count
std::map<std::string, int> wordCount;
std::istringstream ss(inputString);
while(ss >> word)
{
//std::cout<<"word:"<<word<<std::endl;
wordCount[word]++;
}
std::cout<<"Total unique words are: "<<wordCount.size()<<std::endl;
std::size_t i = 0;
std::istringstream gothroughStream(inputString);
//just go through the inputString(stream) instead of map
while( gothroughStream >> word)
{
int index = findExactMatchIndex(inputString, word);
if(index != -1 && (index == i)){
std::cout << word <<"-" << wordCount.at(word)<<std::endl;
}
++i;
}
return 0;
}
The output of the above program is as follows:
Total unique words are: 9
this-1
is-1
a-1
string-1
containing-1
my-1
name-1
again-3
and-2
Note that in the above program, if you have a comma or any other delimiter then it is counted as a separate word. So for example lets say you have the string this is, my name is then the string is, has count of 1 and the string is has count of 1. That is is, and is are different. This is because the computer doesn't know our definition of a word.
Note
The above program is a modification of my answer to How do i make the char in an array output in order in this nested for loop? which is given as version 2 below:
Version 2: For counting unique characters using std::map<char, int> in insertion order
#include <iostream>
#include <map>
int main() {
std::string inputString;
std::cout<<"Enter a string: ";
std::getline(std::cin,inputString);
//this map maps the char to their respective count
std::map<char, int> charCount;
for(char &c: inputString)
{
charCount[c]++;
}
std::size_t i = 0;
//just go through the inputString instead of map
for(char &c: inputString)
{
std::size_t index = inputString.find(c);
if(index != inputString.npos && (index == i)){
std::cout << c <<"-" << charCount.at(c)<<std::endl;
}
++i;
}
return 0;
}
In both cases/versions there is no need to use a separate std::vector or any other container to keep track of the insertion order.
Use boost::multi_index with map and list indices.
A map of pair (str,int) and static int that increments on insert calls indexes pairs of data. Put in a struct that can return the static int val with an index () member perhaps?
Say I have a std::vector. Say the vectors contain numbers. Let's take this std::vector
1,3,5,4,3,4,5,1,6,3
std::sort<std::less<int>> will sort this into
1,1,3,3,3,4,4,5,5,6,
How would I ammend sort so that at the same time it is sorting, it also computes the quantity of numbers at the same level. So say in addition to sorting, it would also compile the following dictionary [level is also int]
std::map<level, int>
<1, 2>
<2, 3>
<3, 2>
<4, 2>
<5, 1>
<6, 1>
so there are 2 1's, 3 3's, 2 4's, and so on.
The reason I [think] I need this is because I don't want to sort the vector, THEN once again, compute the number of duplicates at each level. It seems faster to do it both in one pass?
Thank you all! bjskishore123 is the closest thing to what I was asking, but all the responses educated me. Thanks again.
As stated by #bjskishore123, you can use a map to guarantee the correct order of your set. As a bonus, you will have an optimized strucutre to search (the map, of course).
Inserting/searching in a map takes O(log(n)) time, while traversing the vector is O(n). So, the alghorithm is O(n*log(n)). Wich is the same complexity as any sort algorithm that needs to compare elements: merge sort or quick sort, for example.
Here is a sample code for you:
int tmp[] = {5,5,5,5,5,5,2,2,2,2,7,7,7,7,1,1,1,1,6,6,6,2,2,2,8,8,8,5,5};
std::vector<int> values(tmp, tmp + sizeof(tmp) / sizeof(tmp[0]));
std::map<int, int> map_values;
for_each(values.begin(), values.end(), [&](int value)
{
map_values[value]++;
});
for(std::map<int, int>::iterator it = map_values.begin(); it != map_values.end(); it++)
{
std::cout << it->first << ": " << it->second << "times";
}
Output:
1: 4times
2: 7times
5: 8times
6: 3times
7: 4times
8: 3times
I don't think you can do this in one pass. Let's say you provide your own custom comparator for sorting which somehow tries to count the duplicates.
However the only thing you can capture in the sorter is the value(maybe reference but doesn't matter) of the current two elements being compared. You have no other information because std::sort doesn't pass any thing else to the sorter.
Now the way std::sort works it will keep swapping elements until they reach the proper location in the sorted vector. That means a single member can be sent to the sorter multiple times making it impossible to count exactly. You can count how many times a certain element and all others value equal to it have been moved but not exactly how many of them are in there.
Instead of using a vector,
While storing number one by one, Use std::multiset container
It stores internally in sorted order.
While storing each number, use a map to keep track of the number of occurrences of each number.
map<int, int> m;
Each time a number is added do
m[num]++;
So, no need of another pass to calculate the number of occurrences, although you need to iterate in map to get each occurrence count.
=============================================================================
THE FOLLOWING IS AN ALTERNATE SOLUTION WHICH IS NOT RECOMMENDED .
GIVING IT AS YOU ASKED A WAY WHICH USES STD::SORT.
Below code makes use of comparison function to count the occurrences.
#include <iostream>
#include <map>
#include <vector>
#include <algorithm>
using namespace std;
struct Elem
{
int index;
int num;
};
std::map<int, int> countMap; //Count map
std::map<int, bool> visitedMap;
bool compare(Elem a, Elem b)
{
if(visitedMap[a.index] == false)
{
visitedMap[a.index] = true;
countMap[a.num]++;
}
if(visitedMap[b.index] == false)
{
visitedMap[b.index] = true;
countMap[b.num]++;
}
return a.num < b.num;
}
int main()
{
vector<Elem> v;
Elem e[5] = {{0, 10}, {1, 20}, {2, 30}, {3, 10}, {4, 20} };
for(size_t i = 0; i < 5; i++)
v.push_back(e[i]);
std::sort(v.begin(), v.end(), compare);
for(map<int, int>::iterator it = countMap.begin(); it != countMap.end(); it++)
cout<<"Element : "<<it->first<<" occurred "<<it->second<<" times"<<endl;
}
Output:
Element : 10 occurred 2 times
Element : 20 occurred 2 times
Element : 30 occurred 1 times
If you have lots of duplicates, the fastest way to accomplish this task is probably to first count duplicates using a hash map, which is O(n), and then to sort the map, which is O(m log m) where m is the number of unique values.
Something like this (in c++11):
#include <algorithm>
#include <unordered_map>
#include <utility>
#include <vector>
std::vector<std::pair<int, int>> uniqsort(const std::vector<int>& v) {
std::unordered_map<int, int> count;
for (auto& val : v) ++count[val];
std::vector<std::pair<int, int>> result(count.begin(), count.end());
std::sort(result.begin(), result.end());
return result;
}
There are lots of variations on the theme, depending on what you need, precisely. For example, perhaps you don't even need the result to be sorted; maybe it's enough to just have the count map. Or maybe you would prefer the result to be a sorted map from int to int, in which case you could just build a regular std::map, instead. (That would be O(n log m).) Or maybe you know something about the values which make them faster to sort (like the fact that they are small integers in a known range.) And so on.