I need a class (in C++11) which stores a couple of fields(including a buffer). I started with malloc() in a constructor and free() in a destructor (I didn't touch C/C++ for quite some time so that was all I remembered).
Next thing I remembered (because of crashes) that I need to implement a copy constructor and an assignment operator. Now, I have a full screen of code just for a class with 3 fields (one of which is the buffer).
A question.
What should I use? (I am dazzled by amount of choices - std::vector, std::array, std::shared_ptr, boost::scoped_ptr and may be something else).
Functionality for this buffer which I am looking for are:
as little as possible memory management
getting rid of these copy constructors and assignment operators
ability to use it as void* (I have to pass it to functions which accept "void*")
ability to access read/write it randomly (I may need to get a random range out of it and write a random range to it)
allocate it on the heap (buffer can be reasonably large)
preferably usage of some standard facility
You should just use std::vector<unsigned char> as your buffer. This already has all the necessary constructors, operators, and destructor, so there's nothing special you need to do except use it.
I'm not a C++ expert, so I'd welcome any criticisms to this solution, but a smart pointer to an array might work.
#include <iostream>
#include <memory>
void do_something(void *buffer)
{
char* char_buffer = (char*)buffer;
std::cout << char_buffer[0] << std::endl;
}
int main()
{
size_t size = 10;
// reference counted pointer to auto-delete the buffer
std::shared_ptr<char> buffer(new char[size], std::default_delete<char[]>());
// use the underlying pointer
// http://stackoverflow.com/questions/27819809/why-is-there-no-operator-for-stdshared-ptr
buffer.get()[0] = 'a';
do_something(buffer.get());
// buffer deallocated at the end of scope
}
as little as possible memory management
The smart pointer takes care of that for you
getting rid of these copy constructors and assignment operators
I think the smart pointer handles them
ability to use it as void* (I have to pass it to functions which accept "void*")
Use .get() and a cast.
ability to access read/write it randomly (I may need to get a random range out of it and write a random range to it)
It's still a pointer at the end of this, so I can't see why that wouldn't work (I haven't tried it).
allocate it on the heap (buffer can be reasonably large)
It's on the heap
preferably usage of some standard facility
<memory>
Alternatively, checkout <vector>'s .data() function, which returns a pointer to the vector's underlying array.
I think you need some kind of memory buffer. Check this simple c++ buffer class.
Not using c++11 is intentional because of my company development environment.
https://github.com/jeremyko/CumBuffer
Related
In my code I use buffers currently allocated this way:
char* buf1 = (char*)malloc(size);
However at some points in the code I want to reassign the pointer to some place else in memory. The problem is that there are other places in the code that still need to be able to access the pointer buf1.
What's the best way to do this in C++? Right now I am considering writing a struct with a single char* in it, then allocating an object of this struct type and passing it to the places where I need to, and referring to wrapped pointer to get the current value of buf1.
However it seems that this is similar to what unique_ptr does. If I use unique_ptr how can I wrap a char* with it? I had some trouble with testing this and I'm not sure it's supported.
To clarify: these buffers are bytes of varying sizes.
In general, this question cannot be answered. There are simply way too many things you could be wanting to be doing with an array of char. Without knowing what it actually is that you want to do, its impossible to say what may be good abstractions to use…
If you want to do stuff with strings, just use std::string. If you want a dynamically-sized buffer that can grow and shrink, use std::vector.
If you just need a byte buffer the size of which is determined at runtime or which you'd just generally want to live in dynamic storage, I'd go with std::unique_ptr. While std::unique_ptr<T> is just for single objects, the partial specialization std::unique_ptr<T[]> can be used for dealing with dynamically allocated arrays. For example:
auto buffer = std::unique_ptr<char[]> { new char[size] };
Typically, the recommended way to create an object via new and get an std::unique_ptr to it would be to use std::make_unique. And if you want your buffer initialized to some particular value, you should indeed use std::make_unique<char[]>(value). However, std::make_unique<T[]>() will value-initialize the elements of the array it creates. In the case of a char array, that effectively means that your array will be zero-initialized. In my experience, compilers are, unfortunately, unable to optimize away the zero-initialization, even if the entire buffer would be overwritten first thing right after being created. So if you want an uninitialized buffer for the sake of avoiding the overhead of initialization, you can't use std::make_unique. Ideally, you'd just define your own function to create a default-initialized array via new and get an std::unique_ptr to it, for example:
template <typename T>
inline std::enable_if_t<std::is_array_v<T> && (std::extent_v<T> == 0), std::unique_ptr<T>> make_unique_default(std::size_t size)
{
return std::unique_ptr<T> { new std::remove_extent_t<T>[size] };
}
and then
auto buffer = make_unique_default<char[]>(new char[size]);
It seems that C++20 will include this functionality in the form of std::make_unique_default_init. So that would be the preferred method then.
Note that, if you're dealing with plain std::unique_ptr, you will still have to pass around the size of the buffer separately. You may want to bundle up an std::unique_ptr and an std::size_t if you're planning to pass around the buffer
template <typename T>
struct buffer_t
{
std::unique_ptr<T[]> data;
std::size_t size;
};
Note that something like above struct represents ownership of the buffer. So you'd want to use this, e.g., when returning a new buffer from a factory function, e.g.,
buffer_t makeMeABuffer();
or handing off ownership of the buffer to someone else, e.g.,
DataSink(buffer_t&& buffer)
You would not want to use it just to point some function to the buffer data and size do some processing without transferring ownership. For that, you'd just pass a pointer and size, or, e.g., use a span (starting, again, with C++20; also available as part of GSL)…
First of all, my motivation is to do efficient memory management on top of a C like computational kernel. And I tried to use the std::unique_ptr and std::vector, my code looks like below
// my data container
typedef std::unique_ptr<double> my_type;
std::vector<my_type> my_storage;
// when I need some memory for computation kernel
my_storage.push_back(my_type());
my_storage.back.reset(new double[some_length]);
// get pointer to do computational stuff
double *p_data=my_storage.back.get();
Notice here in practice p_data may be stored in some other container(e.g. map) to indexing each allocated array according to the domain problem, nevertheless, my main questions are
Here is std::vector a good choice? what about other container like std::list/set?
Is there fundamental problem with my allocation method?
Suppose after I use p_data for some operations, now I want to release the memory chunk pointed by the raw pointer p_data, what is the best practice here?
First of all, if you are allocating an array you need to use the specialization std::unique_ptr<T[]> or you won't get a delete [] on memory release but a simple delete.
std::vector is a good choice unless you have any explicit reason to use something different. For example, if you are going to move many elements inside the container then a std::list could perform better (less memmove operations to shift things around).
Regarding how to manage memory, it depends mainly on the pattern of utilization. If my_storage is mainly responsible for everything (which in your specification it is, since unique_ptr expresses ownership), it means that it will be the only one who can release memory. Which could be done simply by calling my_storage[i].reset().
Mind that storing raw pointers of managed objects inside other collections leads to dangling pointers if memory is released, for example:
using my_type = std::unique_ptr<double[]>;
using my_storage = std::vector<my_type>;
my_storage data;
data.push_back(my_type(new double[100]));
std::vector<double*> rawData;
rawData.push_back(data[0].get());
data.clear(); // delete [] is called on array and memory is released
*rawData[0] = 1.2; // accessing a dangling pointer -> bad
This could be a problem or not, if data is released by last then there are no problems, otherwise you could store const references to std::unique_ptr so that at least you'd be able to check if memory is still valid, e.g.:
using my_type = std::unique_ptr<double[]>;
using my_managed_type = std::reference_wrapper<const my_type>;
std::vector<my_managed_type> rawData;
Using std::unique_ptr with any STL container , including std::vector, is fine in general. But you are not using std::unique_ptr the correct way (you are not using the array specialized version of it), and you don't need to resort to using back.reset() at all. Try this instead:
// my data container
typedef std::unique_ptr<double[]> my_type;
// or: using my_type = std::unique_ptr<double[]>;
std::vector<my_type> my_storage;
my_type ptr(new double[some_length]);
my_storage.push_back(std::move(ptr));
// or: my_storage.push_back(my_type(new double[some_length]));
// or: my_storage.emplace_back(new double[some_length]);
I need to have a variable that is declared in a struct to be initialised with a value later. It's basically because its initialisation depends on another member of the structure, which only has a value after some functions have been done.
This sounds a bit weird, so I'll show my code:
struct frame
{
Mat thresholded;
vector<vector<Point> > contrs;
vector<Moments> momts;
};
frame obj_detect(frame img)
{
// Get contours from image
findContours(img.thresholded, img.contrs, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
// Initialise moments vector big enough for all contours' moments
img.moments(img.contrs.size());
...
return img;
}
int main()
{
frame frame1;
frame1 = obj_detect(frame1);
...
}
Currently this code throws this error:
error: no match for call to ‘(std::vector<cv::Moments>) (std::vector<std::vector<cv::Point_<int> > >::size_type)’
How should I initialise the moments vector array so that it has enough space for all the contours?
There's nothing wrong or weird about what you're trying to do. This is an example of an initializer function (or what would be called a constructor for a class).
What isn't entirely clear is whether struct1 is meant to be the struct you are initializing or if it is an input that you are using to return a new struct (since your function does define its return type as my_struct as well). In either case, it is generally recommended to pass your structs by reference rather than by value or as return values.
You could try something like this:
void my_function(const my_struct& input_struct, my_struct& output_struct)
{
...
output_struct.size = ...;
output_struct.my_array = new char[output_struct.size];
...
}
Of course, if you really are using C++, you should question why you are using a struct to represent what seems to be a string?
Once you've allocated memory this way, it is important that you free the memory as well in order to avoid memory leaks. A single object can be deallocated using delete, but an array should be deallocated using delete [], for example:
delete [] some_struct.my_array;
Additionally, it is considered good practice to set pointers to null after deallocating them to avoid referencing stale memory segments. This can be done like this:
some_struct.my_array = nullptr;
Finally, this all gets a bit tedious to manage, particularly if the lifetime and ownership of the object is more complex. To deal with this, the standard library has unique_ptr and shared_ptr objects that will automatically deallocate an object when it is no longer being used.
I don't think it makes sense to go into detail about the differences and uses of each since there are innumerable resources here and elsewhere on the topic.
Seems to me this is very simple question
my_struct my_function(my_struct struct1)
{
...
struct1.my_array = malloc(struct1.size); // Initialise the array
...
}
You have to use malloc()
char* struct1.my_array = new char[struct1.size];
Edited:
Actually you can use malloc or new. In this case - doesn't matter. If you prefer malloc you have to free memory with free(), otherwise you have to use new/delete. Usually new/delete is good to be used while creating objects, when you are dealing with structs and primitive type stick to malloc.
New/Delete operator are usually invokes constructors/destructors and they are a little bit slower than malloc/free. So why you have to pay (even a little) performance cost for nothing?
I have a class Bullet that takes several arguments for its construction. However, I am using a dynamic memory array to store them. I am using C++ so i want to conform to it's standard by using the new operator to allocate the memory. The problem is that the new operator is asking for the constructor arguments when I'm allocating the array, which I don't have at that time. I can accomplish this using malloc to get the right size then fill in form there, but that's not what i want to use :) any ideas?
pBulletArray = (Bullet*) malloc(iBulletArraySize * sizeof(Bullet)); // Works
pBulletArray = new Bullet[iBulletArraySize]; // Requires constructor arguments
Thanks.
You can't.
And if you truly want to conform to C++ standards, you should use std::vector.
FYI, it would probably be even more expensive than what you're trying to achieve. If you could do this, new would call a constructor. But since you'll modify the object later on anyway, the initial construction is useless.
1) std::vector
A std::vector really is the proper C++ way to do this.
std::vector<Bullet> bullets;
bullets.reserve(10); // allocate memory for bullets without constructing any
bullets.push_back(Bullet(10.2,"Bang")); // put a Bullet in the vector.
bullets.emplace_back(10.2,"Bang"); // (C++11 only) construct a Bullet in the vector without copying.
2) new [] operator
It is also possible to do this with new, but you really shouldn't. Manually managing resources with new/delete is an advanced task, similar to template meta-programming in that it's best left to library builders, who'll use these features to build efficient, high level libraries for you. In fact to do this correctly you'll basically be implementing the internals of std::vector.
When you use the new operator to allocate an array, every element in the array is default initialized. Your code could work if you added a default constructor to Bullet:
class Bullet {
public:
Bullet() {} // default constructor
Bullet(double,std::string const &) {}
};
std::unique_ptr<Bullet[]> b = new Bullet[10]; // default construct 10 bullets
Then, when you have the real data for a Bullet you can assign it to one of the elements of the array:
b[3] = Bullet(20.3,"Bang");
Note the use of unique_ptr to ensure that proper clean-up occurs, and that it's exception safe. Doing these things manually is difficult and error prone.
3) operator new
The new operator initializes its objects in addition to allocating space for them. If you want to simply allocate space, you can use operator new.
std::unique_ptr<Bullet,void(*)(Bullet*)> bullets(
static_cast<Bullet*>(::operator new(10 * sizeof(Bullet))),
[](Bullet *b){::operator delete(b);});
(Note that the unique_ptr ensures that the storage will be deallocated but no more. Specifically, if we construct any objects in this storage we have to manually destruct them and do so in an exception safe way.)
bullets now points to storage sufficient for an array of Bullets. You can construct an array in this storage:
new (bullets.get()) Bullet[10];
However the array construction again uses default initialization for each element, which we're trying to avoid.
AFAIK C++ doesn't specify any well defined method of constructing an array without constructing the elements. I imagine this is largely because doing so would be a no-op for most (all?) C++ implementations. So while the following is technically undefined, in practice it's pretty well defined.
bool constructed[10] = {}; // a place to mark which elements are constructed
// construct some elements of the array
for(int i=0;i<10;i+=2) {
try {
// pretend bullets points to the first element of a valid array. Otherwise 'bullets.get()+i' is undefined
new (bullets.get()+i) Bullet(10.2,"Bang");
constructed = true;
} catch(...) {}
}
That will construct elements of the array without using the default constructor. You don't have to construct every element, just the ones you want to use. However when destroying the elements you have to remember to destroy only the elements that were constructed.
// destruct the elements of the array that we constructed before
for(int i=0;i<10;++i) {
if(constructed[i]) {
bullets[i].~Bullet();
}
}
// unique_ptr destructor will take care of deallocating the storage
The above is a pretty simple case. Making non-trivial uses of this method exception safe without wrapping it all up in a class is more difficult. Wrapping it up in a class basically amounts to implementing std::vector.
4) std::vector
So just use std::vector.
It's possible to do what you want -- search for "operator new" if you really want to know how. But it's almost certainly a bad idea. Instead, use std::vector, which will take care of all the annoying details for you. You can use std::vector::reserve to allocate all the memory you'll use ahead of time.
Bullet** pBulletArray = new Bullet*[iBulletArraySize];
Then populate pBulletArray:
for(int i = 0; i < iBulletArraySize; i++)
{
pBulletArray[i] = new Bullet(arg0, arg1);
}
Just don't forget to free the memory using delete afterwards.
The way C++ new normally works is allocating the memory for the class instance and then calling the constructor for that instance. You basically have already allocated the memory for your instances.
You can call only the constructor for the first instance like this:
new((void*)pBulletArray) Bullet(int foo);
Calling the constructor of the second one would look like this (and so on)
new((void*)pBulletArray+1) Bullet(int bar);
if the Bullet constructor takes an int.
If what you're really after here is just fast allocation/deallocation, then you should look into "memory pools." I'd recommend using boost's implementation, rather than trying to roll your own. In particular, you would probably want to use an "object_pool".
I have a lot of data in a list, say several kbytes in each element, I would like to extract each by each to do some numeric processing. These data are originally stored as float[]. Since the processing involves a lot of indexing and global calculation, I think valarray might be easy to program. But if I use valarray, I may have to copy from the array to the valarray first, and then copy back to the array. Is there any way to avoid this? Any way such that to let me work directly on the arrays? Or do you have better ways to solve similar problems?
The valarray type does not provide any way to use an existing array for its data store; it always makes a copy for itself. Instead of storing your data in an ordinary array, store the values directly in the valarray from the start. Call v.resize to set the size, and either assign values into it with the [] operator, or use &v[0] to get a pointer to the first value and use it as you would an iterator or buffer pointer — elements of a valarray are stored contiguously in memory.
Warning: ugly hack.
On my system (MS Visual Studio) the valarray class is defined like this:
template<class _Ty>
class valarray
{
...
private:
_Ty *_Myptr; // current storage reserved for array
size_t _Mysize; // current length of sequence
size_t _Myres; // length of array
};
So i can build my own class that has the same layout (with a good level of confidence):
struct my_valarray_hack
{
void *_Myptr;
size_t num_of_elements;
size_t size_of_buffer;
};
Then create an empty valarray and overwrite its internal variables so it points to your data.
void do_stuff(float my_data[], size_t size)
{
valarray<float> my_valarray;
my_valarray_hack hack = {my_data, size, size};
my_valarray_hack cleanup;
assert(sizeof(my_valarray) == sizeof(hack));
// Save the contents of the object that we are fiddling with
memcpy(&cleanup, &my_valarray, sizeof(cleanup));
// Overwrite the object so it points to our array
memcpy(&my_valarray, &hack, sizeof(hack));
// Do calculations
...
// Do cleanup (otherwise, it will crash)
memcpy(&my_valarray, &cleanup, sizeof(cleanup));
// Destructor is silently invoked here
}
This is not a recommended way of doing things; you should consider it only if you have no other way to implement what you want (maybe not even then). Possible reasons why it could fail:
Layout of valarray may be different in another mode of compilation (examples of modes: debug/release; different platforms; different versions of Standard Library)
If your calculations resize the valarray in any manner, it will try to reallocate your buffer and crash
If the implementation of valarray assumes its buffer has e.g. 16-byte alignment, it may crash, do wrong calculations or just work slowly (depending on your platform)
(I am sure there are some more reasons for it not to work)
Anyway, it's described as "undefined behavior" by the Standard, so strictly speaking anything may happen if you use this solution.