I have a function that saves data on a 2D array. The problem is, the array is to be sent on the internet 50 times a second, so I want the size of the array to be as little as possible, with no waste. So, let's say that my program manages to update 3 times in this 50th of a second, i want the second dimension of the array to be exatly 3. If it updates 10 times, it'll be 10, if just 1, it'll be 1. I am trying to achieve this by incresing the second dimension size by 1 before every update.
char* a[254];
/* every tick */
for (int i=0; i<254; i++)
{
a[i]=new char[counter]; //counter is incremented by one every update
}
would this work, or it would just move the bounds of the array and mess the already existing data up?
EDIT: I'll try to explain a little bit better. If I have a [2][2] array, the third position is [2][1]. Let's say all the array is 0 except for [2][1] wich is 1. Now, if i increment the second dimension with new, will 1 be moved to the new [2][1] position or it would still remain on the third, wich is now [1][3]?
Yes, every "new" call will allocate new memory w/o copying the already existing data. Moreover memory leak will met
Related
This is a Windows C program. In the below code I'm converting a delimited char (pbliblist) into an array. The array is passed to a function from a third party.
My question is how can this be done without knowing how many entries there are (libcount). I was thinking I might be able to 'new' each array entry within the token loop.
LPCTSTR * LibList = new LPCTSTR[libcount];
token = strtok_s(pbliblist, seps, &next_token);
while (token != NULL) {
LibList[cnt] = token;
token = strtok_s(NULL, seps, &next_token);
cnt++;
}
There's different ways you can do it.
But, right off the bat, out of the box, you can't just "add" to an array in C, and, honestly, you don't want to.
There's a couple things you can try.
One, is to assume the size. If you know, for example, that there won't be more than 100 fields in the string, then you can just use 100 for libcount. This "wastes" memory, but it's really efficient, because it's allocated only once. Just make sure you check for the boundary condition and signal an error if you go over 100, because "it should never happen".
Now, if you do the 100 thing, at the end, now that you do, indeed, know how many items you read, you can then create a NEW array, exactly as big as you want, copy the elements from the original array, free it, and return the new one.
Similarly, continuing down this path, there's no need to new the original 100 array. Simply allocate it on the stack, parse the elements in to it, and at the end copy them to the newly allocated result array. The ones on the stack "vanish" when the stack unwinds (super cheap).
If you don't want to have any limit, you can still expand on the above concept.
Start with an original array (but this time, you'll want to allocate it). Then start filling it up. When it fills, and you have more coming, create a new, bigger array, copy the old one to the new one, and release the old one.
In the end, you'll have everything in a single array. You don't want to do this for each element, however. You don't want to new 4 bytes, then new 8 bytes, then new 12 bytes, copying all the time. If you do that you fragment your heap something awful, and you copy way, way to much.
Instead you do it in chunks. 10 entries, 20 entries, 40 entries.
At the end, you can, like above, copy the array a last time to a correctly size block of memory (or not, if this array is going to be returned and destroyed 5ms later, kind of "who cares" about the excess at the end).
Finally, of course, since you're talking C++, there's something in the template libraries that does all this for you (not that I know what it is, I imagine you could just push to extend a vector). But, see, I can barely spell C++, so...don't defer to me for expertise and details on that.
I am using OpenCV 3.2 on C++ and I cannot find a way to do the following task:
Suppose I have 10 pointers to double arrays, say row_0,...,row_9, where each array contains 20 elements. I want to create a cv::Mat object having 10 rows and 20 columns such that its 0th row starts at address row_0, 1st row starts at address row_1 and so on. In other words I already have each row stored contiguously in the memory (however the entire block of 10 rows may not be contiguous) and I want to 'gather' them into a Mat object. How can I accomplish this?
Of course I can declare a 10*20 array, copy the rows successively into it and then call the cv::Mat(int rows,int cols,int type,void *data) constructor, but this requires unnecessary copying of the data. The matrix I need is actually much bigger than 10x20. Moreover I am needing to do this many times in my application, so copying would make my program slow.
I know that I can remove something in the middle of an array such as
char* cArray = (char*) malloc(sizeof(char) * sizeTracker);
using the memmove function. In this way something will be removed from the array without having to use a temp array, switching to vectors, etc. The question here is that can I add a new index in the middle of the array (is there a function for it)? Or let's say that using realloc I add a new index at the end, then how can I move the values down efficiently?
Alternative Answer
I have been thinking about this and the comments where #DietmarKühl started talking about inserting blocks like a deque does. The problem with this is that a deque is a linked list of blocks so then you can't start with an array. if you start with an array and then want to insert something in the middle you have to do something else and I think I have an idea - it isn't fleshed out very much so it may not work but I will share it anyway. Please leave comments telling me what you think of the idea.
If you had an array of items and then want to add an item into the middle all you really want to do is add a block and update the mapping. The mapping is the thing that makes it all work - but it slows down access because you need to check the mapping before every access of the array.
The mapping would be a binary tree. It would start empty but the nodes would contain a value: if the index you want is < the value you traverse the left pointer and if it is >= you traverse the right pointer.
So, an example:
Before the insert:
root -> (array[100000], offset: 0)
After the insert at 5000:
root -> {value: 5000,
left: (array[100000], offset: 0),
right: {value: 5001,
left: (newarray[10], offset: -5000),
right: (array[100000], offset: 1),
}
}
I have used blocks of 10 here - newarray is 10 in size. If you just randomly insert indexes all over the place the block size should be 1 but if you insert groups of consecutive indexes having a blovk size larger than 1 would be good. It really depends on your usage pattern...
When you check index 7000 you check the root node: 7000 is >= 5000 so you follow the right pointer: 7000 is >= 5001 so you follow the right pointer: it points to the original array with an offset of 1 so you access array[index+offset].
When you check index 700 you check the root node: 700 is < 5000 so you follow the left pointer: it points to the original array with an offset of 0 so you access array[index+offset].
When you check index 5000 you check the root node: 5000 is >= 5000 so you follow the right pointer: 5000 is < 5001 so you follow the left pointer: it points to the new array with an offset of -5000 so you access newarray[index+offset].
Of course optimizations to this would be really important to make this useful - you would have to balance the tree after each insert because otherwise the right side would be much much longer than the left side.
The downside to this is that accesses to the array are now O(log inserts) instead of O(1) so if there are lots of inserts you will want to realloc every so often to compact the data structure back to an array but you could save that for an opportune time.
Like I said it isn't very fleshed out so it may not work in practice but I hope it is worth sharing anyway.
Original Answer
If you have a C style array and want to insert an index in the middle you would need to either have an array larger than you need (plus a variable like sizeTracker to keep track of the size).
Then if there was room left you could just memmove the last half of the array out one to create a spot in the middle.
If there wasn't any room left you could malloc another whole array that includes extra space and then memmove the first half and memmove the second half separately leaving a gap.
If you want to make the malloc amortized constant time you need to double the size of the array each time you reallocate it. The memmove becomes one machine instruction on x86 but even then it will still be O(n) because of moving every value.
But performance isn't any worse then your deleting trick - if you can delete everywhere throughout the array the cost is O(n) for that as well because you memmove half the values in average when you delete.
There is no custom C function which allows to increase an array using the C memory function and inserting an object into the middle. Essentially you'd build the functionality using malloc(), free(), memmove() (when enough space is available and elements are just moved back within the memory), or memcpy() (if you need to allocate new memory and you want to avoid first copying and then moving the tail).
In C++ where object locations tend to matter you'd obviously use std::copy(), std::reverse_copy() and/or std::move() (both forms thereof) as there may be relevant structors for the respect objects. Most likely you'd also obtain memory different, e.g., using operator new() and/or an allocator if you really travel in terms of raw memory.
The fun implementation of the actual insertion (assuming there is enough space for another element) is using std::rotate() to construct the last element and then shuffle elements:
void insert(T* array, std::size_t size, T const& value) {
// precodition: array points to at least size+1 elements
new(array + size) T(value);
std::rotate(array, array + size, array + size + 1);
}
Of course, this doesn't avoid potentially unnecessarily shuffling elements when the array needs to be relocated. In that case it more effective to allocate new memory and move the initial objects to the start, add the newly inserted element, move the trailing objects to the location right past the new object.
If you are using manually allocated memory you have to reallocate and you should hope this operation does not move the memory block to a new location. Then the best is to use the rotate algorithm.
By the way, prefer stl containers such as vectors to manually allocated memory for this kind of tasks. If you are using vectors you should have reserved memory.
You have marked this post as C++.
can I add a new index in the middle of the array (is there a function
for it)
No. From cppreference.com, std::array:
std::array is a container that encapsulates fixed size arrays.
I interpret this to mean you can change the elements, but not the indexes.
(sigh) But I suspect C style arrays are still allowed.
And I notice Dietmar's answer also says no.
My code takes an int input and sets that as the array's size, and I have some test prints that print out the index of the array starting from 0 to 4..
std::cout<<array[0]<<std::endl;
std::cout<<array[1]<<std::endl;
std::cout<<array[2]<<std::endl;
std::cout<<array[3]<<std::endl;
std::cout<<array[4]<<std::endl;
However, I noticed that if the input is smaller than 5, say 2 for instance, then the first two cout print out correctly, but then the rest print out 0 or random numbers like 17 and 135137. Is this an out of bounds thing that happens when you index beyond the array size or is this a problem in my code? (I know I have to change the print statements)
The arrays are dynamically allocated by the way, which I think shouldn't matter.
Is this an out of bounds thing that happens when you index beyond the array size or is this a problem in my code?
Both.
Assuming array itself has a size of at least 5 elements, the initial contents of it before you set the values to anything are undefined; essentially random (they're just whatever happened to be hanging out in that particular block of memory that your array now occupies). If array itself has a size of less than 5, the values are still undefined but accessing them also runs the risk of crashing the program. In either case, the fact that you are printing values beyond the end of the initialized, valid data in your array is a problem with your code.
If you allocate an array of n elements, accessing the (n+1)th element is undefined behaviour (UB). (Note after comments: The (n+1)th element is the element with index n. So if array has only size 3, accessing array[3] already causes UB).
So, yes it is an "out of bounds thing" and it is a problem of your code (because it's you who accesses the array beyond its size.
Why not loop to print out simply the existing element instead of hardcoding the indices?
I need to access a two-dimensions matrix with a C++ code. If the matrix is mat[n][m], I have to access (in a for-loop) these positions:
mat[x][y], mat[x-1][y-m-1], mat[x-1][y], mat[x][y-1]
At the next iteration I have to do:
x=x+1
And then, again:
mat[x][y], mat[x-1][y-m-1], mat[x-1][y], mat[x][y-1]
What could be the best way to have these positions nearest in memory to speedup my code?
If you are iterating horizontally, arrange your matrix as mat[y][x], especially if it is an array of arrays (the layout of the matrix isn't clear in your answer).
Since you didn't provided sufficient information, it's hard to say which way is better.
You could try to unroll your loop for continuous memory access.
For example, read from mat[x][y] 4 times, then mat[x-1][y-m-1] 4 times, then mat[x-1][y] 4 times, then mat[x][y-1] 4 times. After that, you process the loaded 4 sets of data in one iteration.
I bet the bottleneck is not the memory access itself. It should be the calculation of memory address. This approach of memory access can be written in SIMD load so you could reduce 3/4 time cost of memory address calculating.
If you have to process your task sequentially, you could try not to use multidimensional subscribes. For example:
for( x=0; x<n; x++ )
doSomething( mat[x][y] );
could be done with:
for( x=y; x<n*m; x+=m )
doSomething( mat[0][x] );
In second way you avoided one lea instruction.
If I get this right, you loop through your entire array, although you only mention x = x + 1 as an update (nothing for y). I would then see the array as one-dimensional, with a single counter i going from 0 to the total array length. Then the four values to access in each loop would be
mat[i], mat[i-S-m-1], mat[i-S], mat[i-1]
where S is the stride (rows or columns depending on your representation). This requires less address computations, regardless of memory layout. It also takes less index checks/updates because there's only one counter i. Plus, S+m+1 is constant, so you could define it as such.