My question is as follows:
I have a function to manipulate content of an array of MAX elements. This function will simply look like the following:
//GLobal Variables
uint8_t my_array[ MAX ];
#define EMPTY 0xFF
...
void initArray( void )
{
for( uint8_t i=0; i<MAX; i++ )
{
my_array[ i ] = EMPTY;
}
}
void manipulateArray( uint8_t value )
{
for( uint8_t i=0; i<MAX; i++ )
{
if( EMPTY == my_array[ i ] )
{
my_array[ i ] = value;
break;
}
}
}
...
int main( void )
{
...
initArray();
...
while( false == exit_flag )
{
manipulateArray( value );
//get new value from user
//update exit_flag based on new value
}
...
return 0;
}
But then I thought that if I end up doing a lot of insertion/deletion, then i would be using for loops like crazy which is bound to affect the speed of the program or big O(N). So I thought what if I use another global
variable to keep track of where next empty sport in the array is for insertion instead of looping through it every time:
//GLobal Variables
uint8_t my_array[ MAX ];
uint8_t idx = 0;
...
void manipulateArray( uint8_t value )
{
my_array[ idx++ ] = value;
}
Is my assumption here correct? Also is it true that it would be better to use another data structure in this particular case that is more suitable to the nature of operations (a lot of insertion & a bit-less deletion): vectors, linked lists...
Thanks in advance,
Interpreting you generally, I take you to be asking about the problem of "inserting" values be overwriting the EMPTY value, and of "deleting" values by replacing them with EMPTY. In that context, you propose to maintain a global variable that tracks the next "empty" position, so as to avoid having to search the array for that position.
Indeed, if you know the location of the next position for insertion, then you can perform the insertion in O(1) steps, whereas if you need to perform a linear search, the best possible bound is O(n). Maintaining metadata such as you propose is a perfectly good strategy if you will always be inserting at or deleting from the end of (the non-empty portion of) the array, for then you can maintain the auxiliary variable in O(1) steps, too.
But suppose you need to support deletions from arbitrary positions, without moving the other array elements, and you also want to be able to re-fill those positions with your insertion function. In that case you have to solve a problem of maintaining information about where multiple empty positions are. A single scalar variable is not enough, and relying on the array itself for that requires you to search the array for empty positions, which is back to where you started.
The alternative is to use a more complex data structure -- an array or a linked list, for example -- to track the openings in the main array. In this way you could achieve O(1) complexity for any number of insertions and deletions at any positions in any sequence, at the cost of using O(n) memory to maintain the metadata about open array positions. This is a classic space vs. speed tradeoff: implementing a faster algorithm requires using more memory, but you can conserve memory by using a slower algorithm.
Related
Given a vector of integers, I want to wrote a fast (not obvious O(n^2)) algorithm to remove all odd elements from it.
My idea is: iterate through vector till first odd element, then copy everything before it to the end of vector (call push_back method) and so on until we have looked through all original elements (except copied ones), then remove all of them, so that only the vector's tail survive.
I wrote the following code to implement it:
void RemoveOdd(std::vector<int> *data) {
size_t i = 0, j, start, end;
uint l = (*data).size();
start = 0;
for (i = 0; i < l; ++i)
{
if ((*data)[i] % 2 != 0)
{
end = i;
for (j = start, j < end, ++j)
{
(*data).push_back((*data)[j]);
}
start = i + 1;
}
}
(*data).erase((*data).begin(), i);
}
but it gives me lots of errors, which I can't fix. I'm very new to the programming, so expect that all of them are elementary and stupid.
Please help me with error corrections or another algorithm implementation. Any suggestions and explanations will be very appreciative. It is also better not to use algorithm library.
You can use the remove-erase idiom.
data.erase(std::remove_if(data.begin(), data.end(),
[](int item) { return item % 2 != 0; }), data.end());
You don't really need to push_back anything (or erase elements at the front, which requires repositioning all that follows) to remove elements according to a predicate... Try to understand the "classic" inplace removal algorithm (which ultimately is how std::remove_if is generally implemented):
void RemoveOdd(std::vector<int> & data) {
int rp = 0, wp = 0, sz = data.size();
for(; rp<sz; ++rp) {
if(data[rp] % 2 == 0) {
// if the element is a keeper, write it in the "write pointer" position
data[wp] = data[rp];
// increment so that next good element won't overwrite this
wp++;
}
}
// shrink to include only the good elements
data.resize(wp);
}
rp is the "read" pointer - it's the index to the current element; wp is the "write" pointer - it always points to the location where we'll write the next "good" element, which is also the "current length" of the "new" vector. Every time we have a good element we copy it in the write position and increment the write pointer. Given that wp <= rp always (as rp is incremented once at each iteration, and wp at most once per iteration), you are always overwriting either an element with itself (so no harm is done), or an element that has already been examined and either has been moved to its correct final position, or had to be discarded anyway.
This version is done with specific types (vector<int>), a specific predicate, with indexes and with "regular" (non-move) assignment, but can be easily generalized to any container with forward iterators (as its done in std::remove_if) and erase.
Even if the generic standard library algorithm works well in most cases, this is still an important algorithm to keep in mind, there are often cases where the generic library version isn't sufficient and knowing the underlying idea is useful to implement your own version.
Given pure algorithm implementation, you don't need to push back elements. In worst case scenario, you will do more than n^2 copy. (All odd data)
Keep two pointers: one for iterating (i), and one for placing. Iterate on all vector (i++), and if *data[I] is even, write it to *data[placed] and increment placed. At the end, reduce length to placed, all elements after are unecessary
remove_if does this for you ;)
void DeleteOdd(std::vector<int> & m_vec) {
int i= 0;
for(i= 0; i< m_vec.size(); ++i) {
if(m_vec[i] & 0x01)
{
m_vec.erase(m_vec.begin()+i);
i--;
}
}
m_vec.resize(i);
}
I have 2 large 2d arrays which is 100s*100s. which has one big loop to do the operation for several times. Inside it there is 3 loops; first loop store in arr1 the sum of each cell in arr2 multiplied by number, 2nd loop stream the 2 arrays to a file and the third loop store in arr2 the sum of the two arrays divided by number.
The code explains better:
for(int i=1;i<x+1;i++) {//initialize
for(int j=1;j<y+1;j++) {
arr1[i][j]=i*j*5.5;
arr2[i][j]=0.;
}
}
for (int i=0;i<x+2;i++) {//padding
vi[i][0]=5;
vi[i][y+1]=-5;
}
for (int j=0;j<y+2;j++) {//padding
vi[0][j]=10.;
vi[x+1][j]=-10.;
}
for(int t=0;t<times;++t) {
for(int i=1;i<x+1;++i) {
for(int j=1;j<y+1;j++) {
arr2[i][j]=(arr1[i+1][j]+arr1[i-1][j]+arr1[i][j-1]+arr1[i][j+1])*1.5;
}
}
arr2[1][1]=arr2[1][y]=arr2[x][1]=arr2[x][y]=0.;
for(int i=1;i<x+1;++i) {
for(int j=1;j<y+1;j++) {
arr1[i][j]=(arr1[i][j]+arr2[i][j])*0.5;
if(arr2[i][j]+arr1[i][j]>5.)
cout<<"\n"<<t<<" "<<i-1<<" "<<j-1<<" "<<arr1[i][j]<<" "<<arr2[i][j];
}
}
}
the whole code works in more then 14s. How should I optimize the code to work in a fastest time possible.
You could use a 3rd array to temporary store the array values of arr2 for the next run.
After the first loop is done, you overwrite arr2 with the temporary array - like this you don't need the second loop. You will save half of the time.
for (n=0;n<x;n++)
{
for (i=0;i<maxi;i++)
{
for (j=0;j<maxj;j++)
{
arr1[i][j]=(arr2[i+1][j]+arr2[i-1][j]+arr2[i][j+1]+arr2[i][j-1])*1.5;
arr_tmp[i][j] = (arr1[i][j]+arr2[i][j])*0.5;
}
}
arr2 = arr_tmp;
}
Note: The OP's code has changed dramatically in response to comments about padding and such. There wasn't really anything wrong with the original code -- which is what I have based this answer on.
Assuming that your 2D arrays are indexed row-major (the first index is the row, and the second index is the column), your memory accesses are already in the correct order for best cache utilization (you are accessing nearby elements as you progress). Your latest code calls this assumption into question since you seem have renamed 'maxi' to be 'x' which would suggest that you are indexing a column-major 2D array (which is very non-standard for C/C++).
It wasn't specified how you were declaring your 2D arrays, and that could make a difference, but I got a big improvement by converting your implementation to use raw pointers. I also eliminated the second loop (from your original post) by combining the operations and alternating the direction for each iteration. I changed the weighting coefficients so that they added up to 1.0 so that I could test this more easily (by generating an image output).
typedef std::vector< std::vector<double> > Array2D;
void run( int x, Array2D & arr2 )
{
Array2D temp = arr2; // easy way to create temporary array of the correct size
int maxi=arr2.size(), maxj=arr2[0].size();
for (int n=0;n<x;n++)
{
Array2D const & src = (n&1)?temp:arr2; // alternate direction
Array2D & dst = (n&1)?arr2:temp;
for (int i=1;i<maxi-1;i++)
{
double const * sp0=&src[i-1][1], * sp1=&src[i][1], * sp2=&src[i+1][1];
double * dp=&dst[i][1];
for (int j=1;j<maxj-1;j++)
{
dp[0]=(sp0[0]+sp1[-1]+4*sp1[0]+sp1[+1]+sp2[0])*0.125;
dp++, sp0++, sp1++, sp2++;
}
}
}
if ( (x&1) ) arr2=temp; // copy the result back if the iteration count was odd
} /**/
Other things you could look into (somewhat platform-dependent):
restrict keyword for pointers (not standard C++)
prefetch requests -- a compiler/processor specific way of reducing memory access latency
make sure you have enabled optimizations when you compile
depending on the size of the array, you might find it advantageous to columnize your algorithm to make better use of available cache
Take advantage of available compute resources (very platform-dependent):
Create a SIMD-based implementation
Take advantage of your multi-core CPU -- OpenMP
Take advantage of your GPU -- OpenCL
I created a program that finds the median of a list of numbers. The list of numbers is dynamic in that numbers can be removed and inserted (duplicate numbers can be entered) and during this time, the new median is re-evaluated and printed out.
I created this program using a multimap because
1) the benefit of it being already being sorted,
2) easy insertion, deletion, searching (since multimap implements binary search)
3) duplicate entries are allowed.
The constraints for the number of entries + deletions (represented as N) are: 0 < N <= 100,000.
The program I wrote works and prints out the correct median, but it isn't fast enough. I know that the unsorted_multimap is faster than multimap, but then the problem with unsorted_multimap is that I would have to sort it. I have to sort it because to find the median you need to have a sorted list. So my question is, would it be practical to use an unsorted_multimap and then quick sort the entries, or would that just be ridiculous? Would it be faster to just use a vector, quicksort the vector, and use a binary search? Or maybe I am forgetting some fabulous solution out there that I haven't even thought of.
Though I'm not new to C++, I will admit, that my skills with time-complexity are somewhat medicore.
The more I look at my own question, the more I'm beginning to think that just using a vector with quicksort and binary search would be better since the data structures basically already implement vectors.
the more I look at my own question, the more I'm beginning to think that just using vector with quicksort and binary search would be better since the data structures basically already implement vectors.
If you have only few updates - use unsorted std::vector + std::nth_element algorithm which is O(N). You don't need full sorting which is O(N*ln(N)).
live demo of nth_element:
#include <algorithm>
#include <iterator>
#include <iostream>
#include <ostream>
#include <vector>
using namespace std;
template<typename RandomAccessIterator>
RandomAccessIterator median(RandomAccessIterator first,RandomAccessIterator last)
{
RandomAccessIterator m = first + distance(first,last)/2; // handle even middle if needed
nth_element(first,m,last);
return m;
}
int main()
{
vector<int> values = {5,1,2,4,3};
cout << *median(begin(values),end(values)) << endl;
}
Output is:
3
If you have many updates and only removing from middle - use two heaps as comocomocomocomo suggests. If you would use fibonacci_heap - then you would also get O(N) removing from arbitary position (if don't have handle to it).
If you have many updates and need O(ln(N)) removing from arbitary places - then use two multisets as ipc suggests.
If your purpose is to keep track of the median on the fly, as elements are inserted/removed, you should use a min-heap and a max-heap. Each one would contain one half of the elements... There was a related question a couple of days ago: How to implement a Median-heap
Though, if you need to search for specific values in order to remove elements, you still need some kind of map.
You said that it is slow. Are you iterating from the beginning of the map to the (N/2)'th element every time you need the median? You don't need to. You can keep track of the median by maintaining an iterator pointing to it at all times and a counter of the number of elements less than that one. Every time you insert/remove, compare the new/old element with the median and update both iterator and counter.
Another way of seeing it is as two multimaps containing half the elements each. One holds the elements less than the median (or equal) and the other holds those greater. The heaps do this more efficiently, but they don't support searches.
If you only need the median a few times you can use the "select" algorithm. It is described in Sedgewick's book. It takes O(n) time on average. It is similar to quick sort but it does not sort completely. It just partitions the array with random pivots until, eventually, it gets to "select" on one side the smaller m elements (m=(n+1)/2). Then you search for the greatest of those m elements, and this is the median.
Here is how you could implement that in O(log N) per update:
template <typename T>
class median_set {
public:
std::multiset<T> below, above;
// O(log N)
void rebalance()
{
int diff = above.size() - below.size();
if (diff > 0) {
below.insert(*above.begin());
above.erase(above.begin());
} else if (diff < -1) {
above.insert(*below.rbegin());
below.erase(below.find(*below.rbegin()));
}
}
public:
// O(1)
bool empty() const { return below.empty() && above.empty(); }
// O(1)
T const& median() const
{
assert(!empty());
return *below.rbegin();
}
// O(log N)
void insert(T const& value)
{
if (!empty() && value > median())
above.insert(value);
else
below.insert(value);
rebalance();
}
// O(log N)
void erase(T const& value)
{
if (value > median())
above.erase(above.find(value));
else
below.erase(below.find(value));
rebalance();
}
};
(Work in action with tests)
The idea is the following:
Keep track of the values above and below the median in two sets
If a new value is added, add it to the corresponding set. Always ensure that the set below has exactly 0 or 1 more then the other
If a value is removed, remove it from the set and make sure that the condition still holds.
You can't use priority_queues because they won't let you remove one item.
Can any one help me what is Space and Time complexity of my following C# program with details.
//Passing Integer array to Find Extreme from that Integer Array
public int extreme(int[] A)
{
int N = A.Length;
if (N == 0)
{
return -1;
}
else
{
int average = CalculateAverage(A);
return FindExtremes(A, average);
}
}
// Calaculate Average of integerArray
private int CalculateAverage(int[] integerArray)
{
int sum = 0;
foreach (int value in integerArray)
{
sum += value;
}
return Convert.ToInt32(sum / integerArray.Length);
}
//Find Extreme from that Integer Array
private int FindExtremes(int[] integerArray, int average) {
int Index = -1; int ExtremeElement = integerArray[0];
for (int i = 0; i < integerArray.Length; i++)
{
int absolute = Math.Abs(integerArray[i] - average);
if (absolute > ExtremeElement)
{
ExtremeElement = integerArray[i];
Index = i;
}
}
return Index;
}
You are almost certainly better off using a vector. Possibly maintaining an auxiliary vector of indexes to be removed between median calculations so you can delete them in batches. New additions can also be put into an auxiliary vector, sorted, then merged in.
I finally determined that this function is responsible for the majority of my bottleneck issues. I think its because of the massively excessive random access that happens when most of the synapses are already active. Basically, as the title says, I need to somehow optimize the algorithm so that I'm not randomly checking a ton of active elements before landing on one of the few that are left.
Also, I included the whole function in case of other flaws that can be spotted.
void NetClass::Explore(vector <synapse> & synapses, int & n_syns) //add new synapses
{
int size = synapses.size();
assert(n_syns <= size );
//Increase the age of each active synapse by 1
Age_Increment(synapses);
//make sure there is at least one inactive vector left
if(n_syns == size)
return;
//stochastically decide whether a new connection is added
if((rand_r(seedp) %1000) < ( x / (1 +(n_syns * ( y / 100)))))
{
n_syns++; //a new synapse has been created
//main inefficiency here
while(1)
{
int syn = rand_r(seedp) % (size);
if (!synapses[syn].active)
{
synapses[syn].active = true;
synapses[syn].weight = .04 + (float (rand_r(seedp) % 17) / 100);
break;
}
}
}
}
void NetClass::Age_Increment(vector <synapse> & synapses)
{
for(int q=0, int size = synapses.size(); q < size; q++)
if(synapses[q].active)
synapses[q].age++;
}
Pass a random number, k, in the range [0, size-n_syns) to Age_Increment. Have Age_Increment return the kth empty slot.
Since you're already traversing the whole list in Age_Increment, update that function to return the list of the indexes of inactive synapses.
You can then pick a random item from that list directly.
This is similar to the problem of finding free blocks in memory management, so I would take a look at algorithms used in that domain, specifically free lists, which is a list of free positions. (These are usually implemented as linked lists to be able to pop elements off an end efficiently. Random access in a linked list would still be O(n) - with a smaller n, but still not the best choice for your use case.)
This is not a homework.
I'm using a small "priority queue" (implemented as array at the moment) for storing last N items with smallest value. This is a bit slow - O(N) item insertion time. Current implementation keeps track of largest item in array and discards any items that wouldn't fit into array, but I still would like to reduce number of operations further.
looking for a priority queue algorithm that matches following requirements:
queue can be implemented as array, which has fixed size and _cannot_ grow. Dynamic memory allocation during any queue operation is strictly forbidden.
Anything that doesn't fit into array is discarded, but queue keeps all smallest elements ever encountered.
O(log(N)) insertion time (i.e. adding element into queue should take up to O(log(N))).
(optional) O(1) access for *largest* item in queue (queue stores *smallest* items, so the largest item will be discarded first and I'll need them to reduce number of operations)
Easy to implement/understand. Ideally - something similar to binary search - once you understand it, you remember it forever.
Elements need not to be sorted in any way. I just need to keep N smallest value ever encountered. When I'll need them, I'll access all of them at once. So technically it doesn't have to be a queue, I just need N last smallest values to be stored.
I initially thought about using binary heaps (they can be easily implemented via arrays), but apparently they don't behave well when array can't grow anymore. Linked lists and arrays will require extra time for moving things around. stl priority queue grows and uses dynamic allocation (I may be wrong about it, though).
So, any other ideas?
--EDIT--
I'm not interested in STL implementation. STL implementation (suggested by a few people) works a bit slower than currently used linear array due to high number of function calls.
I'm interested in priority queue algorithms, not implemnetations.
Array based heaps seem ideal for your purpose. I am not sure why you rejected them.
You use a max-heap.
Say you have an N element heap (implemented as an array) which contains the N smallest elements seen so far.
When an element comes in you check against the max (O(1) time), and reject if it is greater.
If the value coming in is lower, you modify the root to be the new value and sift-down this changed value - worst case O(log N) time.
The sift-down process is simple: Starting at root, at each step you exchange this value with it's larger child until the max-heap property is restored.
So, you will not have to do any deletes which you probably will have to, if you use std::priority_queue. Depending on the implementation of std::priority_queue, this could cause memory allocation/deallocation.
So you can have the code as follows:
Allocated Array of size N.
Fill it up with the first N elements you see.
heapify (you should find this in standard text books, it uses sift-down). This is O(N).
Now any new element you get, you either reject it in O(1) time or insert by sifting-down in worst case O(logN) time.
On an average, though, you probably will not have to sift-down the new value all the way down and might get better than O(logn) average insert time (though I haven't tried proving it).
You only allocate size N array once and any insertion is done by exchanging elements of the array, so there is no dynamic memory allocation after that.
Check out the wiki page which has pseudo code for heapify and sift-down: http://en.wikipedia.org/wiki/Heapsort
Use std::priority_queue with the largest item at the head. For each new item, discard it if it is >= the head item, otherwise pop the head item and insert the new item.
Side note: Standard containers will only grow if you make them grow. As long as you remove one item before inserting a new item (after it reaches its maximum size, of course), this won't happen.
Most priority queues I work are based on linked lists. If you have a pre-determined number of priority levels, you can easily create a priority queue with O(1) insertion by having an array of linked lists--one linked list per priority level. Items of the same priority will of course degenerate into either a FIFO, but that can be considered acceptable.
Adding and removal then becomes something like (your API may vary) ...
listItemAdd (&list[priLevel], &item); /* Add to tail */
pItem = listItemRemove (&list[priLevel]); /* Remove from head */
Getting the first item in the queue then becomes a problem of finding the non-empty linked-list with the highest priority. That may be O(N), but there are several tricks you can use to speed it up.
In your priority queue structure, keep a pointer or index or something to the linked list with the current highest priority. This would need to be updated each time an item is added or removed from the priority queue.
Use a bitmap to indicate which linked lists are not empty. Combined with a find most significant bit, or find least significant bit algorithm you can usually test up to 32 lists at once. Again, this would need to be updated on each add / remove.
Hope this helps.
If amount of priorities is small and fixed than you can use ring-buffer for each priority. That will lead to waste of the space if objects is big, but if their size is comparable with pointer/index than variants with storing additional pointers in objects may increase size of array in the same way.
Or you can use simple single-linked list inside array and store 2*M+1 pointers/indexes, one will point to first free node and other pairs will point to head and tail of each priority. In that cases you'll have to compare in avg. O(M) before taking out next node with O(1). And insertion will take O(1).
If you construct an STL priority queue at the maximum size (perhaps from a vector initialized with placeholders), and then check the size before inserting (removing an item if necessary beforehand) you'll never have dynamic allocation during insert operations. The STL implementation is quite efficient.
Matters Computational see page 158. The implementation itself is quite well, and you can even tweak it a little without making it less readable. For example, when you compute the left child like:
int left = i / 2;
You can compute the rightchild like so:
int right = left + 1;
Found a solution ("difference" means "priority" in the code, and maxRememberedResults is 255 (could be any (2^n - 1)):
template <typename T> inline void swap(T& a, T& b){
T c = a;
a = b;
b = c;
}
struct MinDifferenceArray{
enum{maxSize = maxRememberedResults};
int size;
DifferenceData data[maxSize];
void add(const DifferenceData& val){
if (size >= maxSize){
if(data[0].difference <= val.difference)
return;
data[0] = val;
for (int i = 0; (2*i+1) < maxSize; ){
int next = 2*i + 1;
if (data[next].difference < data[next+1].difference)
next++;
if (data[i].difference < data[next].difference)
swap(data[i], data[next]);
else
break;
i = next;
}
}
else{
data[size++] = val;
for (int i = size - 1; i > 0;){
int parent = (i-1)/2;
if (data[parent].difference < data[i].difference){
swap(data[parent], data[i]);
i = parent;
}
else
break;
}
}
}
void clear(){
size = 0;
}
MinDifferenceArray()
:size(0){
}
};
build max-based queue (root is largest)
until it is full, fill up normally
when it is full, for every new element
Check if new element is smaller than root.
if it is larger or equal than root, reject.
otherwise, replace root with new element and perform normal heap "sift-down".
And we get O(log(N)) insert as a worst case scenario.
It is the same solution as the one provided by user with nickname "Moron".
Thanks to everyone for replies.
P.S. Apparently programming without sleeping enough wasn't a good idea.
It's better to implement your own class using std::array and heap algorithms.
`template<class T, int fixed_size = 5>
class fixed_size_arr_pqueue_v2
{
std::array<T, fixed_size> _data;
int _size = 0;
int parent(int i)
{
return (i - 1)/2;
}
void heapify(int i, bool downward = false)
{
int l = 2*i + 1;
int r = 2*i + 2;
int largest = 0;
if (l < size() && _data[l] > _data[i])
largest = l;
else
largest = i;
if (r < size() && _data[r] > _data[largest])
largest = r;
if (largest != i)
{
std::swap(_data[largest], _data[i]);
if (!downward)
heapify(parent(i));
else
heapify(largest, true);
}
}
public:
void push(T &d)
{
if (_size == fixed_size)
{
//min elements in a max heap lies at leaves only.
auto minItr = std::min_element(begin(_data) + _size/2, end(_data));
auto minPos {minItr - _data.begin()};
auto min { *minItr};
if (d > min)
{
_data.at(minPos) = d;
if (_data[parent(minPos)] > d)
{
//this is unlikely to happen in our case? as this position is a leaf?
heapify(minPos, true);
}
else
heapify(parent(minPos));
}
return ;
}
_data.at(_size++) = d;
std::push_heap(_data.begin(), _data.begin() + _size);
}
T pop()
{
T d = _data.front();
std::pop_heap(_data.begin(), _data.begin() + _size);
_size--;
return d;
}
T top()
{
return _data.front();
}
int size() const
{
return _size;
}
};`