Inserting and deleting elements from vector *at the same time* - c++

Goal
Consider a sorted std::vector x. We want to erase from this vector all elements at positions indicated by vector positionsToErase. We also want to insert the values of vector valuesToInsert at positions positionsToInsert.
These deletions and insertions must happen at the same time, in the sense that if we erase first, then it will invalidates the positions at which we want to insert values (and vice-versa). I think that will be made clear with the below example
Example
Example of function definition
template<typename T>
void insertEraseAtPositions(
std::vector<T>& x, // vector to modify. Is sorted and must remain sorted
std::vector<T>& valuesToInsert, // is not sorted
std::vector<size_t>& positionsToInsert, // is not sorted. This could be figured out inside the function but I happen to already know the positions at which values must be inserted
std::vector<size_t>& positionsToErase // is not sorted
);
Note that non are constant and modifications can be made in-place.
Example of arguments
std::vector<int> x = {0, 10, 20, 21, 30, 50, 60, 70, 81, 90}; // vector to modify
std::vector<int> valuesToInsert = {40, 80, 100}; // Values to insert are '40', '80' and '100'
std::vector<size_t> positionsToErase = {3, 8}; // Erase elements '21' and '81'
std::vector<size_t> positionsToInsert = {5, 8, 10}; // Insert where are currently located the elements '50', '81' and past the current last element.
Expected output
x = {0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100};
Important notes
Performance is very important and it is hence not possible to insert and erase one-by-one (even if we progressively modify positions accordingly) as it would involve way too many copies (or move).
Typically, x is of size 1,000 to 100,000. positionsToInsert (and valuesToInsert) are of size 1-20 and positionsToErase is of size 1-5. x typically has a capacity that allows inserting the values without reallocating. I hence expect (but might be wrong) that modifications in-place would be faster.
I can also supply iterators instead of indices (std::vector<std::vector<T>::iterator> instead of std::vector<size_t>) for positionsToErase and positionsToInsert if you prefer.
Current work
I wrote a code to insert at positions but I failed to include the possibility to erase too. Here is the code in case it helps.
// Return indices representing the order of elements
template <typename T>
std::vector<uint32_t> sort_indices(const std::vector<T> &v) {
// initialize original index locations
std::vector<uint32_t> idx(v.size());
std::iota(idx.begin(), idx.end(), 0);
// sort indexes based on comparing values in v
std::sort(idx.begin(), idx.end(),
[&v](uint32_t i1, uint32_t i2) {return v[i1] < v[i2];});
return idx;
}
template <typename T>
void reorder(std::vector<T>& v, std::vector<uint32_t>& order)
{
auto v2 = v;
for (uint32_t i = 0 ; i < v.size() ; i++)
{
v[i] = v2[order[i]];
}
}
//// Insert multiple elements at specified positions into vector
template<typename T>
void insertAtPositions(std::vector<T>& x, std::vector<T>& values, std::vector<size_t>& positions)
{
// assert values and positions are the same size
assert(values.size() == positions.size());
// Special case - values is empty
if (values.size() == 0) return;
// Special case - single value to insert
if (values.size() == 1)
{
x.insert(positions.front(), values.front());
return;
}
// sort the values and the positions where those values should be inserted
auto indices = sort_indices(positions);
reorder(positions, indices);
reorder(values, indices);
// Special case - x is empty
if (x.size() == 0)
{
x.swap(values);
return;
}
// Allocate memory to x
x.resize(x.size() + values.size());
// Move things to make room for insertions and insert
int pos_index = positions.size()-1;
for (size_t i = x.size()-1 ; pos_index >= 0 ; --i)
{
if (i == positions[pos_index] + pos_index)
{
// A new value should go at index i
x[i] = std::move(values[pos_index]);
--pos_index;
} else
{
// The value from index 'i-pos_index-1' must go to index 'i'
x[i] = std::move(x[i-pos_index-1]);
}
}
}

Modifying it in place is a no-go.
Consider that you have to insert something at every position. You would need to copy every single item into a temp place then copy them back.
You might argue that you could do it from the end, backwards. But if we have some deletions we would also need to store some of the elements there, potentially getting back to copying every element into some temp storage and back.
I think the fastest way would be to allocate a new array, and build it up, using the original as temp storage. This way you are guaranteed that each element is copied exactly once.
Now, depending on the types used (like ints, or pointers) this could be a lot faster than anything else you might cook up. If copies are expensive consider using moves, or pointers.
If you are worried about performance, you should benchmark you code and tune it. It's hard to argue precisely about performance without data.

Related

How to calculate the sum of individual elements from two 2D arrays?

I want to write a function addArrays which will, as parameters, take two 2D arrays of type int and of dimensions 3x4, and it's job is to add individual elements of each index from the given arrays and display it in the console.
In main(), I created two 2D arrays arrA and arrB of appropriate sizes with intitialized members and check the functionality of the created function.
My code so far:
#include <iostream>
using namespace std;
void addArrays(int x[3][4], int y[3][4]);
int main()
{
int arrA[3][4] = { {7, 8, 13, 22}, {56, 4, 78, 3}, {22, 13, 46, 5} };
int arrB[3][4] = { {32, 47, 56, 14}, {33, 100, 19, 64}, {4, 18, 157, 84} };
}
void addArrays(int x[3][4], int y[3][4])
{
}
Honestly, I know how to work with 1D arrays, but not displaying the sum of all individual elements. I know I have to use a for loop, but again, I'm confused as to how to pass in a 2D array and use it.
You mention you know how to work with 1D arrays, it's the same for 2D arrays, only with one more dimension.
In a 1D array you use arrA[0] to access the first element of the array, in a 2D array you use arrA[0][0] to access the first element in the first line of the array, arrA[0][1] to access the second element in the first line of the array. To access the first element in the second line you would use arrA[1][0] and so on, you get the idea.
So to loop through all the elements in the array you can use nested loops like so:
void addArrays(int x[3][4], int y[3][4])
{
for( int i = 0; i < 3; i++){ // make sure to use the correct dimensions
for(int j = 0; j < 4; j++){ // 3 lines and 4 columns
// x[i][j] accesses elements in array x
// y[i][j] accesses elements in array y
}
}
}
I think you'll manage to do the math yourself. After that you just need to send data to the standard output, i.e. to print data in the console. For that, as you may know, you use std::cout.
Side notes:
In the function void addArrays(int x[3][4], int y[3][4]){...} you may omit the first dimension of the array i.e. int x[][4] or int (*x)[4] instead of int x[3][4], since the argument will decay to a pointer to array.
Since it seems that you are not to change the values of the passed arrays, using const is recommend. You would have void addArrays(const int (*x)[4], const int (*y)[4]);
As you are using C++, you can take advantage of the possibility of using references, something like void addArrays(const int (&x)[3][4], const int (&y)[3][4]){/*same code*/}, the benefit being you must pass a correct object to the function otherwise the program will fail to compile whereas in the previous versions if you pass, for example, NULL, i.e. addArrays(arrA, NULL); the program will compile fine but will result in undefined behavior when you run it. References are safer and you should use them when possible.
It's more or less consensual among more experienced C++ programmers that the usage of using namespace std; is not a good practice, you can read more about it, and find alternatives following the link.
I will start this for you and try to give you an idea of the general structure, but since you have not shown your attempt at the problem I won't fill things in for you.
The basic idea here when looping through 2D arrays (of size MxN) is that you can really just think about it in terms of looping through M arrays of length N.
void loopThroughArray(int arr[M][N])
{
// Loop over M arrays
for (int m = 0; m < M; ++m) {
// For each m'th array, loop over its N contents
for (int n = 0; n < N; ++n) {
// Doing work
arr[m][n] = 1234;
}
}
}

C++ Making sure 2D vector is compact in memory

I'm writing a C++ program to perform calculations on a huge graph and therefore has to be as fast as possible. I have a 100MB textfile of unweighted edges and am reading them into a 2D vector of integers (first index = nodeID, then a sorted list of nodeIDs of nodes which have edges to that node). Also, during the program, the edges are looked up exactly in the order in which they're stored in the list. So my expectation was that, apart from a few bigger gaps, it'd always be nicely preloaded to the cache. However, according to my profiler, iterating through the edges of a player is an issue. Therefore I suspect, that the 2D vector isn't placed in memory compactly.
How can I ensure that my 2D vector is as compact as possible and the subvectors in the order in which they should be?
(I thought for example about making a "2D array" from the 2D vector, first an array of pointers, then the lists.)
BTW: In case it wasn't clear: The nodes can have different numbers of edges, so a normal 2D array is no option. There are a couple ones with lots of edges, but most have very few.
EDIT:
I've solved the problem and my program is now more than twice as fast:
There was a first solution and then a slight improvement:
I put the lists of neighbour ids into a 1D integer array and had another array to know where a certain id's neighbour lists start
I got a noticeable speedup by replacing the pointer array (a pointer needs 64 bit) with a 32 bit integer array containing indices instead
What data structure are you using for the 2d vector? If you use std::vector then the memory will be contiguous.
Next, if pointers are stored then only the address will take advantage of the vectors spacial locality. Are you accessing the object pointed to when iterating the edges and if so this could be a bottleneck. To get around this perhaps you can setup your objects so they are also in contiguous memory and take advantage of spacial locality.
Finally the way in which you access the members of a vector affects the caching. Make sure you are accessing in an order advantageous to the container used (eg change column index first when iterating).
Here's some helpful links:
Cache Blocking Techniques
SO on cache friendly code
I have written a few of these type structures by having a 2D view onto a 1D vector and there are lots of different ways to do it. I have never made one that allows the internal arrays to vary in length before so this may contain bugs but should illustrate the general approach:
#include <cassert>
#include <iostream>
#include <vector>
template<typename T>
class array_of_arrays
{
public:
array_of_arrays() {}
template<typename Iter>
void push_back(Iter beg, Iter end)
{
m_idx.push_back(m_vec.size());
m_vec.insert(std::end(m_vec), beg, end);
}
T* operator[](std::size_t row) { assert(row < rows()); return &m_vec[m_idx[row]]; }
T const* operator[](std::size_t row) const { assert(row < rows()); return &m_vec[m_idx[row]]; }
std::size_t rows() const { return m_idx.size(); }
std::size_t cols(std::size_t row) const
{
assert(row <= m_idx.size());
auto b = m_idx[row];
auto e = row + 1 >= m_idx.size() ? m_vec.size() : m_idx[row + 1];
return std::size_t(e - b);
}
private:
std::vector<T> m_vec;
std::vector<std::size_t> m_idx;
};
int main()
{
array_of_arrays<int> aoa;
auto data = {2, 4, 3, 5, 7, 2, 8, 1, 3, 6, 1};
aoa.push_back(std::begin(data), std::begin(data) + 3);
aoa.push_back(std::begin(data) + 3, std::begin(data) + 8);
for(auto row = 0UL; row < aoa.rows(); ++row)
{
for(auto col = 0UL; col < aoa.cols(row); ++col)
{
std::cout << aoa[row][col] << ' ';
}
std::cout << '\n';
}
}
Output:
2 4 3
5 7 2 8 1

Copying from one dimensional vector vector<int> starts to first element of two dimensional vector pair vector<pair<int,int>>matrix

I have multiple 3 one dimensional vectors (vector<int> starts, vector<int> ends, vector<int> points). Each having specific number of elements.
I want to create a two dimensional vector vector<pair<int,int>>matrix in such a sequence :
from beginning of matrix to size of start first element of matrix is elements of vector<int> starts and second element is "-1"
Append now the elements of vector<int> ends to matrix such that first element of matrix is elements of vector<int> ends and second element is "-2"
Append now the elements of vector<int> points to matrix such that first element of matrix is elements of vector<int> points and second element is Index of points.
Visual Representation :-
Input:
starts: {1, 2, 3}
ends: {4, 5, 6}
points: (7, 8, 9}
Output:
matrix: { {1, -1}, {2, -1}, {3, -1}, {4, -2}, {5, -2}, {6, -2}, {7, 0}, {8, 1}, {9, 2} }
Currently I am using a push_back with for-loop function which works perfectly fine but when the input size is big code is very slow.
Code I am using is as follows:
vector<pair<int,int>> fast_count_segments(
vector<int> starts,
vector<int> ends,
vector<int> points)
{
int i = 0;
vector<pair<int,int>>matrix;
for(i; i<starts.size(); i++) {
matrix.push_back(make_pair(starts[i],-1));
}
for(i; i<starts.size()+ends.size(); i++) {
matrix.push_back(make_pair(ends[i-starts.size()],-2));
}
for(i; i<starts.size()+ends.size()+points.size(); i++) {
matrix.push_back(make_pair(
points[i-starts.size()-ends.size()],
i-(starts.size()+ends.size())
));
}
return matrix;
}
Can you please help on how to fill the 2D vector quickly with these requirements without iterating through each element. I am using C++11. Thanks in Advance !!
Preliminary concern: As #datenwolf and others note - Your resulting data structure is not a 2D matrix (unless you mean a boolean matrix in sparse representation). Are you sure that's what you want to be populating?
Regardless, here are a few ideas to possibly improve speed:
Don't take the input vectors by value! That's useless copying... take their .data(), or their .cbegin() iterator, or take a span<int> parameter.
Use the reserve() method on the target vector to avoid multiple re-allocations.
Use .emplace_back() instead of .push_back() to construct the points in place, rather than constructing-then-moving every point. Although, to be honest, the compiler will probably optimize those constructions away, anyway.
Put the .size() values of the input vectors in local variables. This will only help if, for some reason, the compiler suspects that size will not be constant throughout the execution of the function.
Make sure you're passing optimization switches to the compiler (e.g. -O2 or -O3 to GCC and clang). This might seem obvious to you but sometimes it's so obvious you forget to check it's actually been done.
Some aesthetic comments:
No need to use the same counter for all vectors. for(int i = 0; i < whatever; i++) can be used multiple times.
No need for raw for loops, you can use for(const auto& my_element : my_vector) for the first two loops. The third loop is trickier, since you want the index. You can use std::difference() working with iterators, or go with Python-style enumeration described here.
You might consider using std::transform() with a back_emplacer output iterators instead of all three loops. No-loop code! That would mean using std::difference() in the transformer lambda instead of the third loop.
This incorporates the suggestions from #einpoklum's answer, but also cleans up the code.
std::vector<std::pair<int,int>> fast_count_segments(
std::vector<int> const & starts,
std::vector<int> const & ends,
std::vector<int> const & points)
{
std::vector<std::pair<int,int>> matrix(starts.size() + ends.size() + points.size());
auto out = std::transform(starts.cbegin(), starts.cend(),
matrix.begin(),
[](int i) { return std::pair<int,int>{i, -1}; });
out = std::transform(ends.cbegin(), ends.cend(),
out,
[](int i) { return std::pair<int,int>{i, -2}; });
int c = 0;
std::transform(points.cbegin(), points.cend(),
out,
[&c](int i) { return std::pair<int,int>{i, c++}; });
return matrix;
}
You could even write all the transforms as a single expression. Whether this is easier to read is highly subjective, so I'm not recommending it per se. (Try reading it like you would nested function calls.)
std::vector<std::pair<int,int>> fast_count_segments(
std::vector<int> const & starts,
std::vector<int> const & ends,
std::vector<int> const & points)
{
std::vector<std::pair<int,int>> matrix(starts.size() + ends.size() + points.size());
int c = 0;
std::transform(points.cbegin(), points.cend(),
std::transform(ends.cbegin(), ends.cend(),
std::transform(starts.cbegin(), starts.cend(),
matrix.begin(),
[](int i) { return std::pair<int,int>{i, -1}; }),
[](int i) { return std::pair<int,int>{i, -2}; }),
[&c](int i) { return std::pair<int,int>{i, c++}; });
return matrix;
}

match elements between two std::vectors

I am writing a module that estimates optical flow. At each time step it consumes an std::vector where each element of the vector is a current pixel location and a previous pixel location. The vector is not ordered. New pixels that were previously not seen will be present and flow locations that were not found will be gone. Is there a correct way to match elements in the new vector to the set of optical flow locations being estimated?
The vectors are on the order of 2000 elements.
These are the approaches I am considering:
naively iterate through the new vector for each estimated optical flow location
naively iterating through the new vector but removing each matched location so the search gets faster as it goes on
run std::sort on my list and the new list at every time step. Then iterate through the new vector starting at the last matched index +1
I'm suspecting that there is an accepted way to go about this but I don't have any comp sci training.
I'm in c++ 11 if that is relevant.
// each element in the new vector is an int. I need to check if
// there are matches between the new vec and old vec
void Matcher::matchOpticalFlowNaive(std::vector<int> new_vec)
{
for(int i = 0; i < this->old_vec.size(); i++)
for(int j =0; j < new_vec.size(); j++)
if(this->old_vec[i] == new_vec[j]){
do_stuff(this->old_vec[i], new_vec[j])
j = new_vec.size();
}
}
Not sure to understand what do you need but, supposing that your Matcher is constructed with a vector of integer, that there ins't important the order and that you need check this vector with other vectors (method matchOpticalFlowNaive()) to do something when there is a match, I suppose you can write something as follows
struct Matcher
{
std::set<int> oldSet;
Matcher (std::vector<int> const & oldVect)
: oldSet{oldVect.cbegin(), oldVect.cend()}
{ }
void matchOpticalFlowNaive (std::vector<int> const & newVec)
{
for ( auto const & vi : newVec )
{
if ( oldSet.cend() != oldSet.find(vi) )
/* do something */ ;
}
}
};
where the Matcher object is constructed with a vector that is used to initialize a std::set (or a std::multi_set, or a unordered set/multiset?) to make simple the work in matchOpticalFlowNaive()

Swapping two values within a 2D array

I am currently working on a 15 puzzle programming assignment. My question here is about how I would go about swapping the empty tile with an adjacent tile.
So, for example, let's go with the initial setup board.
I have:
int originalBoard[4][4] = {
{1 , 2, 3, 4},
{5 , 6, 7, 8},
{9 ,10,11,12},
{13,14,15, 0}};
So here, the locations of 12, 15, and 0 (the empty tile) in the array are [3][4], [4][3], and [4][4] respectively. What would be a method of swapping 0 out with either 12 or 15?
What I had in mind for this was creating a loop that would keep track of the empty tile every time I made a move.
I believe an optimal method would be to have two functions. 1 that would update the location of the empty tile, and 1 to make the move.
So, right off the top of my head I would have:
void locateEmptyTile(int& blankRow, int& blankColumn, int originalBoard[4][4])
{
for (int row = 0; row < 4; row++)
{
for (int col = 0; col < 4; col++)
{
if (originalBoard[row][col] == 0)
{
blankRow = row;
blankColumn = col;
}
}
}
}
void move(int& blankRow, int& blankColumn, int originalBoard[4][4])
{
}
And in my main function I would have the variables: int blankRow and int blankColumn
Now, how would I take that data from locateEmptyTile and apply it into the move function in the relevant practical manner? The process does not currently connect within my head.
I appreciate any little bits of help.
If you're just asking for swap function you can use std::swap:
#include <algorithm> // until c++11
#include <utility> // since c++11
...
int m[3][3];
...
//somewhere in the code
std::swap(m[i][j], m[j][i]); // this swaps contents of two matrix cells
...
Or you can just write where you want to swap contents of two variables (in example int a and int b):
int temp = a;
a = b;
b = temp;
As you can see swapping is the same as with normal arrays, c++ does not know if you are swapping two matrix cells or two array elements, it just knows that you are swapping two memory blocks with certain type.
A basic swap concept (pre-C++11) is hold a temporary variable. Simply...
template<typename T, typename U>
void swap(T& lhs, U& rhs) {
T t = lhs;
lhs = rhs;
rhs = t;
}
So, you don't need to reference blankRow and blankCol, you just need to reference the values on the grid. Lets say that you want to swap what you know is blank positioned at (2, 1) with (2, 2)...
swap(originalBoard[2][1], originalBoard[2][2]);
... will swap the values within originalBoard.
If you are using C++11 or later, just use std::swap() to swap positions. That's exactly what it does.
If you would like originalBoard to be immutable an result in a totally different board, just copy it first before applying the switch.