C++ custom iterator over container based on items property - c++

Question

I have a container of Employees with fields std::string name and bool is_manager. I want to iterate over container for regular and manager employees. The number of items in a container can be very large so I do not want to do a linear scan with checking is_manager property. Also the number of managers is very small i.e. 10 out of 100000, so doing a full scan over the container is ineffective. So I want to pre-cache the memory addresses of Employees with and without is_manager==true and have RegularEmployeeIterator and ManagerEmployeeIterator, I think this can be pre-cached and organized as a list/vector of pointers?
And I want to able to sort the container of Employees by name field and retain the ability to iterate over regulars and managers.
How to implement that in C++? Specifically, I have no idea how iterators are implemented in C++, how to define several of them for a single collection, based on property value, does my idea with caching the addresses works, etc.

Answer 1

organized as a list/vector of pointers?
Whatever the problem, a list is not the data structure you need.
1. Unrealistic answer for a beginner, but you claim it's the problem you're solving
ok actually I have tens of millions entries
I mean, this is very clearly a learning exercise, and you insisting that your data is 10s of millions of entries large is... mediocrily helpful, because that's the point where if access times are important, you stop storing the composite object in one container:
std::vector<Employee> employees; //10⁷ employees
but would group the data according to the properties you're going to work on at the same time:
std::vector<bool> bossiness; //10⁷ bits – std::vector<bool> has an optimization!
std::vector<std::string> names; //10⁷ std::strings
and as a matter of fact, if you know your data doesn't change, you wouldn't even do that, because the names vector is a dereferencing nightmare that wastes a lot of memory on redundant information, if you could as well just go
std::vector<bool> bossiness; //10⁷ bits – std::vector<bool> has an optimization!
std::string all_names; // a **very** long string containing all names, one after the other
std::vector<size_t> name_begins; // 10⁷ name beginnings; through all_names.substr(name_begins[i], name_begins[i+1]) you can access the i.th name
Now, to speed up looking for bosses, you just start by making a run-time encoded list of 64bit-regions in your bossiness vector where at least one bit is set. You could do elegant k-d trees if your problem becomes multidimensional, but at the sparsity you have, runtime encoding on machine word sizes will probably still beat the hell out of that.
But that's an optimization level you need when writing a database system or a 3D game with millions of vertices. You're learning C++. You're not writing these kinds of things, so:
2. Realistic answer that you didn't want when offered in the question
i.e. 10 out of 100000
so, let's really go with a problem size of 10⁵. I.e., a small problem.
You need to do your Employee vector, and add a bosses vector:
std::vector<Employee> employees;
std::vector<size_t> boss_indices;
Then you need to do your linear search once:
// if you know a safe and not too outlandish upper bound for the number of managers, reserve that memory once to avoid resizing the vector while filling it, as that's very expensive:
boss_indices.reserve(size_t(employees.size() * fraction_of_managers));
for(size_t idx = 0; idx < employees.size(); ++idx) {
if(employees[idx].is_manager) {
boss_indices.push_back(idx)
}
}
congratulations, an easy to use vector indices. Indices into std::vector are just as good as pointers to elements (it's a simple pointer deref, both ways, and the additional offset is usually merged into the deref operation on any modern CPU I know), but survive the target vector being moved.
And I want to able to sort the container of Employees by name field and retain the ability to iterate over regulars and managers.
have a class
#include <algorithm>
struct SortableEmployee {
const Employee* empl;
bool operator <(const SortableEmployee& other) const {
return std::lexicographical_compare(
empl->name.cbegin(); empl->name.cend(),
other.empl->name.cbegin(), other.empl->name.cend());
}
SortableEmployee(Employee* underlying) : empl(underlying){
}
};
and put it in a std::set to get a sorted version that you can iterate through:
std::set<SortableEmployee> namebook;
for(const auto& individual : employees) {
namebook.emplace(&individual);
}
You can then iterate through it linearly as well
for(const auto& sorted_empl : namebook) {
std::cout << std::format("{}: is {}a manager\n",
sorted_empl.empl->name,
sorted_empl.empl->is_manager ? "" : "not ");
}