In the reference manual of GSL it is written that
The pointer block stores the location of the memory block in which the vector elements are located (if any). If the vector owns this block then the owner field is set to one and the block will be deallocated when the vector is freed.
But what does it mean 'If the vector owns the block'?
The following is the structure of gsl_vector
typedef struct
{
size_t size;
size_t stride;
double * data;
gsl_block * block;
int owner;
} gsl_vector;
According to the documentation:
For consistency all memory is allocated through a gsl_block structure.
Next:
Vectors and matrices are made by slicing an underlying block.
Basically, you can use an existing block of memory to get a new vector using, for example, (for some reason undocumented) functions alloc_from_block or alloc_from_vector. In such cases owner is set to 0 and when you free a vector that initial block stays allocated:
void
FUNCTION (gsl_vector, free) (TYPE (gsl_vector) * v)
{
RETURN_IF_NULL (v);
if (v->owner)
{
FUNCTION(gsl_block, free) (v->block) ;
}
free (v);
}
Related
I'm building a buffer for network connections where you can explicitly allocate memory or you can supply it on your own via some sequential container(eg.:std::vector,std::array)these memory chunks are stored in a list what we use later for read/write operations. (the chunks are needed for handle multiple read/write requests)
I have a question about the last part, I want to make a pointer to the container's data and then tell the container to not care about it's data anymore.
So something like move semantics.
std::vector<int> v = {9,8,7,6,5,4,3,2,1,0};
std::vector<int> _v(std::move(v));
Where _v has all the values of v and v left in a safe state.
The problem is if I just make a pointer for v.data() after the lifetime of the container ends, the data pointed by the pointer releases with the container.
For example:
// I would use span to make sure It's a sequential container
// but for simplicity i use a raw pointer
// gsl::span<int> s;
int *p;
{
std::vector<int> v = {9,8,7,6,5,4,3,2,1,0};
// s = gsl::make_span(v);
p = v.data();
}
for(int i = 0; i < 10; ++i)
std::cout << p[i] << " ";
std::cout << std::endl;
Now p contains some memory trash and i would need the memory previously owned by the vector.
I also tried v.data() = nullptr but v.data() is rvalue so it's not possible to assign it. Do you have any suggestions, or is this possible?
edit.:
To make it more clear what i'm trying to achieve:
class readbuf_type
{
struct item_type // representation of a chunk
{
uint8_t * const data;
size_t size;
inline item_type(size_t psize)
: size(psize)
, data(new uint8_t[psize])
{}
template <std::ptrdiff_t tExtent = gsl::dynamic_extent>
inline item_type(gsl::span<uint8_t,tExtent> s)
: size(s.size())
, data(s.data())
{}
inline ~item_type()
{ delete[] data; }
};
std::list<item_type> queue; // contains the memory
public:
inline size_t read(uint8_t *buffer, size_t size); // read from queue
inline size_t write(const uint8_t *buffer, size_t size); // write to queue
inline void *get_chunk(size_t size)
{
queue.emplace_back(size);
return queue.back().data;
}
template <std::ptrdiff_t tExtent = gsl::dynamic_extent>
inline void put_chunk(gsl::span<uint8_t,tExtent> arr)
{
queue.emplace_back(arr);
}
} readbuf;
I have the get_chunkfunction what basically just allocates memory with the size, and I have put_chunk what I'm struggling with, the reason i need this because before you can write to this queue you need to allocate memory and then copy all the elements from the buffer(vector,array) you're trying to write from to the queue.
Something like:
std::vector<int> v = {9,8,7,6,5,4,3,2,1,0};
// instead of this
readbuf.get_chunk(v.size);
readbuf.write(v.data(), v.size());
// we want this
readbuf.put_chunk({v});
Since we're developing for distributed systems memory is crucial and that's why we want to avoid the unnecessary allocation, copying.
ps.This is my first post, so sorry if i wasn't precise in the first place..
No, it is not possible to "steal" the buffer of the standard vector in the manner that you suggest - or any other standard container for that matter.
You've already shown one solution: Move the buffer into another vector, instead of merely taking the address (or another non-owning reference) of the buffer. Moving from the vector transfers the ownership of the internal buffer.
It would be possible to implement such custom vector class, whose buffer could be stolen, but there is a reason why vector doesn't make it possible. It can be quite difficult to prove the correctness of your program if you release resources willy-nilly. Have you considered how to prevent the data from leaking? The solution above is much simpler and easier to verify for correctness.
Another approach is to re-structure your program in such way that no references to the data of your container outlive the container itself (or any invalidating operation).
Unfortunately the memory area of the vector cannot be detached from the std::vector object. The memory area can be deleted even if you insert some data to the std::vector object. Therefore use of this memory area later is not safe, unless you are sure that this particular std::vector object exists and is not modified.
The solution to this problem is to allocate a new memory area and copy the content of the vector to this newly allocated memory area. The newly allocated memory area can be safely accessed without worrying about the state of the std::vector object.
std::vector<int> v = {1, 2, 3, 4};
int* p = new int[v.size()];
memcpy(p, v.data(), sizeof(int) * v.size());
Don't forget to delete the memory area after you are finished using this memory area.
delete [] p;
Your mistake is in thinking that the pointer "contains" memory. It doesn't contain anything, trash or ints or otherwise. It is a pointer. It points to stuff. You have deleted that stuff and not transferred it anywhere else, so it can't work any more.
In general, you will need a container to put this information in, be it another vector, or even your own hand-made array. Just having a pointer to data does not mean you have data.
Furthermore, since it is impossible to ask a vector to relinquish its buffer to a non-vector thing, a vector is really your only chance in this particular case. It's not quite clear why that's not a good enough solution for you. :)
Not sure what you try to achieve but I would use moving semantic like this:
#include <iostream>
#include <memory>
#include <vector>
int main() {
std::unique_ptr<std::vector<int>> p;
{
std::vector<int> v = {9,8,7,6,5,4,3,2,1,0};
p = std::move(make_unique<std::vector<int>>(v));
}
for(int i = 0; i < 10; ++i)
std::cout << (*p)[i] << " ";
std::cout << std::endl;
return 0;
}
I need to have a 2D array of double.
Its width is around 900. Its height as well (the same width value).
Dealing with two loops (one for the width and one for the height), I really need to get access to all the pixels of the 900X900 image that I will process.
The size of the array is too big (error when specifying the number of raw and column).
I thought about establishing that with a dynamic array to optimize the time of calculation and to free the memory everytime I deal with one pixel on the two loops.
But I really cannot find the syntax I would like to have to declare a 2D dynamic array (malloc, setting array element values and freeing the memory).
Wrap it in a class:
class Matrix2D {
typedef std::vector<double> Column;
std::vector<Column> columns;
public:
Matrix2D(unsigned int width, unsigned int height) :
columns(width, Column(height)) {
}
double& at(unsigned int i, unsigned int j) {
return columns[i][j];
}
};
Matrix2D matrix(900, 900);
matrix.at(45, 65) = 1234.5678;
I need to have a 2D array of double
Since you are using C++ you should use STL classes that will take care of ugly memory management for you. So you are actually looking for std::vector< std::vector<double> >, or for the sake of the readability of your code:
#include <vector>
typedef std::vector<double> DVector; // row represented by vector of doubles
typedef std::vector<DVector> MyVector; // 2D array as a vector of these rows
And then avoid using dynamic allocation wherever it's possible to do so. Take advantage of RAII idiom:
{
MyVectorarr; // vector object with automatic storage duration
} // <-- vector is automatically destructed when execution goes out of scope
Questions that might help you:
Multi-dimensional vector
Initialization of a vector of vectors?
vector of vector
I associate malloc with pure C, not C++ (as the prior answer points yout, you should use std::vector). However, if you really want to:
// allocate the memory in a block
double* block = (double *) malloc(sizeof(double) * xSize * ySize);
// allocate memory for the accessor array
double* accessor = (double*) malloc(sizeof(double*) * xSize);
// assign memory addresses
double* curPtr = block;
for (int i = 0; i < xSize; ++i) {
accessor[i] = curPtr;
curPtr += ySize;
}
// you can now access the array via accessor[x][y]
// now need to free malloced memory:
free(accessor);
free(block);
If you do it this way, I highly suggest tying it to the RAII pattern, otherwise you'll eventually get a memory leak. Using the STL's containers is a better approach.
First I want to say that, I have a vector which has thousand of vectors inside. Each of these inside vectors has thousand of numbers inside. I want to keep memory management safe and memory usage at minimum as much as possible.
I want to ask that if I have a code similiar to below
int size = 10;
vector<vector<double>>* something = new vector<vector<double>>(size);
vector<double>* insideOfSomething;
for(int i = 0; i < size; i++){
insideOfSomething = &(something->at(i));
//...
//do something with insideOfSomething
//...
}
I know that 'something' will be created in heap. What I don't understand is where the vectors are placed, 'insideOfSomething' points? If they are created in stack, then this means that I have a vector pointer, which points a vector in heap, that has vectors inside which are created in stack? (I'm very confused right now.)
If I have a code similiar to the one below;
vector<vector<double>*>* something = new vector<vector<double>*>(size);
vector<double>* insideOfSomething;
for(int i = 0; i < size; i++){
something->at(i) = new vector<double>();
insideOfSomething = something->at(i);
//...
//do something with inside insideOfSomething
//...
}
right know all of my vectors are stored in heap, right?
Which one is more usefull according to the memory management?
You should avoid allocating vectors on the heap and just declare them on the stack since the vector will manage its objects on the heap for you. Anywhere you want to avoid creating a copy you can just use a reference or const reference (which ever is necessary).
vector<vector<double> > something(size);
for(int i = 0; i < size; i++)
{
vector<double> &insideOfSomething = something.at(i);
//use insideOfSomething
}
Let's take a random, simplistic implementation of vector, as I think this will help you.
template <class T, class Alloc>
class vector
{
private:
T* buffer;
std::size_t vector_size;
std::size_t vector_capacity
Alloc alloc;
public:
...
};
In this case, if we write:
vector<int> v;
v.push_back(123);
... the pointer, buffer, the integrals: vector_size and vector_capacity, and the allocator object, alloc, will all be created on the stack (along with allocating any additional memory necessary for structure padding and alignment).
However, vector itself will allocate memory on the heap to which this buffer pointer will store its base address. That will always be on the heap and will contain the actual contents of the vector as we think of them.
This is still more efficient than this:
vector<int>* v = new vector<int>;
v->push_back(123);
...
delete v;
... as this would involve a heap allocation/deallocation for the vector itself (including its data members) in addition to the memory vector itself allocates for its internal contents (the buffer). It also introduces an additional level of indirection.
Now if we have a vector of Somethings (vector of vector or anything else):
vector<Something> v;
Those Something instances are always going to be allocated within a contiguous heap buffer since they would reside in the dynamically allocated memory blocks that vector creates and destroys internally.
In vector<> all data stored in heap
And i think you should simply use
vector< vector<double> > something;
I want to keep memory management safe and memory usage at minimum as much as possible.
Then
vector<vector<double>>* something = new vector<vector<double>>(size);
is already not good. As said in the other answers, vector already has its data on the heap, no need to mess around with new to achieve this. In fact, the objects' location is like
S t a c k H e a p
(vector<double>) sthng[0]
(vector<vector<double>>) sthng (vector<double>) sthng[1]
...
- - - - - -
(double) sthng[0][0]
(double) sthng[0][1]
...
- - - - - -
(double) sthng[1][0]
(double) sthng[1][1]
...
(of course, there is no particular ordering of the blocks on the heap)
Joe and hired777's answers explain that a vector will be allocated on the heap no matter what. I'll try to give some insight on the reason for this.
A vector is a resizeable container. Generally it doubles in size when it reaches capacity which means it needs to be able to allocate more memory than it had already allocated. Hence even when you declare vector inside a function and hence on the stack, internally it's holding a pointer to it's data on the heap and on going out of the function's scope, it's destructor will delete this data from the heap.
I wrote a program, which computes the flow shop scheduling problem.
I need help with optimizing the slowest parts of my program:
Firstly there is array 2D array allocation:
this->_perm = new Chromosome*[f];
//... for (...)
this->_perm[i] = new Chromosome[fM1];
It works just fine, but a problem occurs, when I try to delete array:
delete [] _perm[i];
It takes extremely long to execute line above. Chromosome is array of about 300k elements - allocating it takes less than a second but deleting takes far more than a minute.
I would appreciate any suggestions of improving delete part.
On a general note, you should never manually manage memory in C++. This will lead to leaks, double-deletions and all kinds of nasty inconveniences. Use proper resource-handling classes for this. For example, std::vector is what you should use for managing a dynamically allocated array.
To get back to your problem at hand, you first need to know what delete [] _perm[i] does: It calls the destructor for every Chromosome object in that array and then frees the memory. Now you do this in a loop, which means this will call all Chromosome destructors and perform f deallocations. As was already mentioned in a comment to your question, it is very likely that the Chromosome destructor is the actual culprit. Try to investigate that.
You can, however, change your memory handling to improve the speed of allocation and deallocation. As Nawaz has shown, you could allocate one big chunk of memory and use that. I'd use a std::vector for a buffer:
void f(std::size_t row, std::size_t col)
{
int sizeMemory = sizeof(Chromosome) * row * col;
std::vector<unsigned char> buffer(sizeMemory); //allocation of memory at once!
vector<Chromosome*> chromosomes(row);
// use algorithm as shown by Nawaz
std::size_t j = 0 ;
for(std::size_t i = 0 ; i < row ; i++ )
{
//...
}
make_baby(chromosomes); //use chromosomes
in_place_destruct(chromosomes.begin(), chromosomes.end());
// automatic freeing of memory holding pointers in chromosomes
// automatic freeing of buffer memory
}
template< typename InpIt >
void in_place_destruct(InpIt begin, InpIt end)
{
typedef std::iterator_traits<InpIt>::value_type value_type; // to call dtor
while(begin != end)
(begin++)->~value_type(); // call dtor
}
However, despite handling all memory through std::vector this still is not fully exception-safe, as it needs to call the Chromosome destructors explicitly. (If make_baby() throws an exception, the function f() will be aborted early. While the destructors of the vectors will delete their content, one only contains pointers, and the other treats its content as raw memory. No guard is watching over the actual objects created in that raw memory.)
The best solution I can see is to use a one-dimensional arrays wrapped in a class that allows two-dimensional access to the elements in that array. (Memory is one-dimensional, after all, on current hardware, so the system is already doing this.) Here's a sketch of that:
class chromosome_matrix {
public:
chromosome_matrix(std::size_t row, std::size_t col)
: row_(row), col_(col), data_(row*col)
{
// data_ contains row*col constructed Chromosome objects
}
// note needed, compiler generated dtor will do the right thing
//~chromosome_matrix()
// these rely on pointer arithmetic to access a column
Chromosome* operator[](std::size_t row) {return &data_[row*col_];}
const Chromosome* operator[](std::size_t row) const {return &data_[row*col_];}
private:
std::size_t row_;
std::size_t col_;
std::vector<chromosomes> data_
};
void f(std::size_t row, std::size_t col)
{
chromosome_matrix cm(row, col);
Chromosome* column = ch[0]; // get a whole column
Chromosome& chromosome1 = column[0]; // get one object
Chromosome& chromosome2 = cm[1][2]; // access object directly
// make baby
}
check your destructors.
If you were allocating a built-in type (eg an int) then allocating 300,000 of them would be more expensive than the corresponding delete. But that's a relative term, 300k allocated in a single block is pretty fast.
As you're allocating 300k Chromosomes, the allocator has to allocate 300k * sizeof the Chromosome object, and as you say its fast - I can't see it doing much beside just that (ie the constructor calls are optimised into nothingness)
However, when you come to delete, it not only frees up all that memory, but it also calls the destructor for each object, and if its slow, I would guess that the destructor for each object takes a small, but noticeable, time when you have 300k of them.
I would suggest you to use placement new. The allocation and deallocation can be done just in one statement each!
int sizeMemory = sizeof(Chromosome) * row * col;
char* buffer = new char[sizeMemory]; //allocation of memory at once!
vector<Chromosome*> chromosomes;
chromosomes.reserve(row);
int j = 0 ;
for(int i = 0 ; i < row ; i++ )
{
//only construction of object. No allocation!
Chromosome *pChromosome = new (&buffer[j]) Chromosome[col];
chromosomes.push_back(pChromosome);
j = j+ sizeof(Chromosome) * col;
}
for(int i = 0 ; i < row ; i++ )
{
for(int j = 0 ; j < col ; j++ )
{
//only destruction of object. No deallocation!
chromosomes[i][j].~Chromosome();
}
}
delete [] buffer; //actual deallocation of memory at once!
std::vector can help.
Special memory allocators too.
Is there a way to posix_memalign a std::vector without creating a local instance of the vector first?
The problem I'm encountering is that I need to tell posix_memalign how much space to allocate and I don't know how to say
sizeof(std::vector<type>(n))
without actually creating a new vector.
Thanks
Well, there are two sizes here. The vector itself is typically no more than a pointer or two to some allocated memory, and unsigned integers keeping track of size and capacity. There is also the allocated memory itself, which is what I think you want.
What you want to do is make a custom allocator that the vector will use. When it comes time, it will use your allocator and you can have your own special functionality. I won't go over the full details of an allocator, but the specifics:
template <typename T>
struct aligned_allocator
{
// ...
pointer allocate(size_type pCount, const_pointer = 0)
{
pointer mem = 0;
if (posix_memalign(&mem, YourAlignment, sizeof(T) * pCount) != 0)
{
throw std::bad_alloc(); // or something
}
return mem;
}
void deallocate(pointer pPtr, size_type)
{
free(pPtr);
}
// ...
};
And then you'd use it like:
typedef std::vector<T, aligned_allocator<T> > aligned_T_vector;
aligned_T_vector vec;
vec.push_back( /* ... */ ); // array is aligned
But to reiterate the first point, the size of a vector is the same regardless of how many elements it's holding, as it only points to a buffer. Only the size of that buffer changes.