I am currently working on a threadsafe circular buffer for storing pointers. As basis I used the code from this thread: Thread safe implementation of circular buffer. My code is shown below.
Because I want to store pointers in the circular buffer, I need to make sure that allocated memory is deleted, in case the buffer is full and the first element is overwritten, as mentioned in the boost documentation:
When a circular_buffer becomes full, further insertion will overwrite the stored pointers - resulting in a memory leak. 1
So I tried to delete the first element in the add method, in case the buffer is full and the type T of the template is actually a pointer. This leads to a C2541-error, because I try to delete an object, which is not seen as a pointer.
Is my basic approach correct? How can I solve the above issue?
#pragma once
#include <boost/thread/condition.hpp>
#include <boost/thread/mutex.hpp>
#include <boost/thread/thread.hpp>
#include <boost/circular_buffer.hpp>
#include <type_traits>
#include "Settings.hpp"
template <typename T>
class thread_safe_circular_buffer : private boost::noncopyable
{
public:
typedef boost::mutex::scoped_lock lock;
thread_safe_circular_buffer(bool *stop) : stop(stop) {}
thread_safe_circular_buffer(int n, bool *stop) : stop(stop) {cb.set_capacity(n);}
void add (T imdata) {
monitor.lock();
std::cout << "Buffer - Add Enter, Size: " << cb.size() << "\n";
if(cb.full())
{
std::cout << "Buffer - Add Full.\n";
T temp = cb.front();
if(std::is_pointer<T>::value)
delete[] temp;
}
std::cout << "Buffer - Push.\n";
cb.push_back(imdata);
monitor.unlock();
std::cout << "Buffer - Add Exit.\n";
}
T get() {
std::cout << "Buffer - Get Enter, Size: " << cb.size() << "\n";
monitor.lock();
while (cb.empty())
{
std::cout << "Buffer - Get Empty, Size: " << cb.size() << "\n";
monitor.unlock();
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
if(*stop)
return NULL;
monitor.lock();
}
T imdata = cb.front();
// Remove first element of buffer
std::cout << "Buffer - Pop.\n";
cb.pop_front();
monitor.unlock();
std::cout << "Buffer - Get Exit.\n";
return imdata;
}
void clear() {
lock lk(monitor);
cb.clear();
}
int size() {
lock lk(monitor);
return cb.size();
}
void set_capacity(int capacity) {
lock lk(monitor);
cb.set_capacity(capacity);
}
bool *stop;
private:
boost::condition buffer_not_empty;
boost::mutex monitor;
boost::circular_buffer<T> cb;
};
The error tells you the problem: you can't delete things that aren't pointers. When T isn't a pointer type, delete[] temp; doesn't make sense. It's also a bad idea if your buffer is storing things that weren't allocated with new[], or when your circular buffer doesn't conceptually 'own' the pointers.
I think you maybe misunderstand the whole problem. The warning from the boost documentation only applies to situations where you can't afford to "lose" any of the data stored in the buffer. One example where this is a problem — the one they highlight specifically — is if you were storing raw pointers in the buffer that are your only references to some dynamically allocated memory.
There are, I think, only three reasonable designs:
Don't use a circular buffer when you can't afford to lose data (e.g. this can mean modifying your data so you can afford to lose it; e.g. circular_buffer<unique_ptr<T[]>> for storing dynamically allocated arrays) . And consequently, make your class not worry about what to do with lost data.
Make your class take a 'deleter'; i.e. a function object that specifies what to do with a data element that is about to be overwritten. (and you probably want to have a default parameter of "do nothing")
Change the functionality of the buffer to do something other than overwriting when full (e.g. block)
Do as boost does: let the user of your code handle this. If objects are stored, its destructor should handle possible memory management, if pointers are stored, you have no way of knowing what it actually points to: arrays, objects, memory that needs to be freed, memory that is managed elsewhere, not dynamic memory.
Related
I'm running a single-threaded system in FreeRTOS with limited resources.
I already preallocate buffers for the RapidJSON allocators as so:
char valueBuffer[2048];
char parseBuffer[1024];
rapidjson::MemoryPoolAllocator<FreeRTOSRapidJSONAllocator> valueAllocator (valueBuffer, sizeof(valueBuffer))
rapidjson::MemoryPoolAllocator<FreeRTOSRapidJSONAllocator> parseAllocator (parseBuffer, sizeof(parseBuffer));
The issue I have is that every time one of the allocators is used, its size keeps on increasing (and allocating new memory if necessary) unless they are cleared. The problem with calling Clear() on the allocators is that Malloc is called again when the allocators are next resized, which I want to avoid.
Is there a way to simply reuse the existing preallocated memory, such as by setting the allocators' size back to zero, for example?
I resolved this by creating a custom allocator. Essentially a copy of rapidjson::MemoryPoolAllocator with the addition of the following method:
void Reset()
{
chunkHead_->size = 0;
chunkHead_->next = 0;
}
Which should be called every time you're done with the last string that was parsed.
I needed to do the same, and thought I'd note what from reading the allocator code seems to be an alternative:
While I again create a char buffer[2048], I do not create and keep an allocator alongside it.
Rather, I create and delete an allocator anew when needed, while re-using the memory block. From reading the code of the allocator I see no Malloc so that should all be on the stack, no?
Edit - code example:
class MyClass
{
public:
myClass() :
_json_log_string_buffer(&_json_string_buffer_allocator,
_json_buffer_size) {}
(...)
private:
static constexpr int _json_buffer_size {4096};
char _json_logger_buffer[_json_buffer_size];
rapidjson::CrtAllocator _json_string_buffer_allocator;
rapidjson::StringBuffer _json_log_string_buffer;
};
MyClass::log_to_json()
{
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>
json_logger_allocator {_json_logger_buffer,
sizeof(_json_logger_buffer)};
rapidjson::Document json_doc(&json_logger_allocator);
auto& allocator = json_doc.GetAllocator();
json_doc.SetObject();
// using it:
json_doc.AddMember("id", id(), allocator);
json_doc.AddMember("KEY", "VALUE",
allocator);
(...)
// Here, monitoring the size of allocator, it never goes
// beyond the 4096 on repeat invocations.
// If I had it as a member, only creating once, it would grow.
std::cout << "JSON allocator size: " << allocator.Size() <<
", capacity: " << allocator.Capacity() << std::endl;
// Bonus point: I'm also using a rapidjson::StringBuffer.
// From a first read it doesn't seem to re-allocate.
// It doesn't use a MemoryPoolAllocator so I cannot do the
// same for it as I did for rapidjson::Document.
_json_log_string_buffer.Clear();
rapidjson::Writer<rapidjson::StringBuffer>
writer(_json_log_string_buffer);
json_doc.Accept(writer);
auto as_string = _json_log_string_buffer.GetString();
// Using string with logger...
}
Deleting the double pointer is will cause the harmful effect like crash the program and programmer should try to avoid this as its not allowed.
But sometime if anybody doing this then i how do we take care of this.
As delete in C++ is noexcept operator and it'll not throw any exceptions. And its written type is also void. so how do we catch this kind of exceptions.
Below is the code snippet
class myException: public std::runtime_error
{
public:
myException(std::string const& msg):
std::runtime_error(msg)
{
cout<<"inside class \n";
}
};
void main()
{
int* set = new int[100];
cout <<"memory allcated \n";
//use set[]
delete [] set;
cout <<"After delete first \n";
try{
delete [] set;
throw myException("Error while deleting data \n");
}
catch(std::exception &e)
{
cout<<"exception \n";
}
catch(...)
{
cout<<"generic catch \n";
}
cout <<"After delete second \n";
In this case i tried to catch the exception but no success.
Pleas provide your input how we'll take care of these type of scenario.
thanks in advance!!!
Given that the behaviour on a subsequent delete[] is undefined, there's nothing you can do, aside from writing
set = nullptr;
immediately after the first delete[]. This exploits the fact that a deletion of a nullptr is a no-op.
But really, that just encourages programmers to be sloppy.
Segmentation fault or bad memory access or bus errors cannot be caught by exception. Programmers need to manage their own memory correctly as you do not have garbage collection in C/C++.
But you are using C++, no ? Why not make use of RAII ?
Here is what you should strive to do:
Memory ownership - Explicitly via making use of std::unique_ptr or std::shared_ptr and family.
No explicit raw calls to new or delete. Make use of make_unique or make_shared or allocate_shared.
Make use of containers like std::vector or std::array instead of creating dynamic arrays or allocating array on stack resp.
Run your code via valgrind (Memcheck) to make sure there are no memory related issues in your code.
If you are using shared_ptr, you can use a weak_ptr to get access to the underlying pointer without incrementing the reference count. In this case, if the underlying pointer is already deleted, bad_weak_ptr exception gets thrown. This is the only scenario I know of when an exception will be thrown for you to catch when accessing a deleted pointer.
A code must undergo multiple level of testing iterations maybe with different sets of tools before committing.
There is a very important concept in c++ called RAII (Resource Acquisition Is Initialisation).
This concept encapsulates the idea that no object may exist unless it is fully serviceable and internally consistent, and that deleting the object will release any resources it was holding.
For this reason, when allocating memory we use smart pointers:
#include <memory>
#include <iostream>
#include <algorithm>
#include <iterator>
int main()
{
using namespace std;
// allocate an array into a smart pointer
auto set = std::make_unique<int[]>(100);
cout <<"memory allocated \n";
//use set[]
for (int i = 0 ; i < 100 ; ++i) {
set[i] = i * 2;
}
std::copy(&set[0], &set[100] , std::ostream_iterator<int>(cout, ", "));
cout << std::endl;
// delete the set
set.reset();
cout <<"After delete first \n";
// delete the set again
set.reset();
cout <<"After delete second \n";
// set also deleted here through RAII
}
I'm adding another answer here because previous answers focus very strongly on manually managing that memory, while the correct answer is to avoid having to deal with that in the first place.
void main() {
std::vector<int> set (100);
cout << "memory allocated\n";
//use set
}
This is it. This is enough. This gives you 100 integers to use as you like. They will be freed automatically when control flow leaves the function, whether through an exception, or a return, or by falling off the end of the function. There is no double delete; there isn't even a single delete, which is as it should be.
Also, I'm horrified to see suggestions in other answers for using signals to hide the effects of what is a broken program. If someone is enough of a beginner to not understand this rather basic stuff, PLEASE don't send them down that path.
I'm trying to make this code work, but the object keep getting destroyed...
I've found that it has to do with the object being copied to the vector, but can't find any way to prevent it...
#include <iostream>
#include <string>
#include <vector>
using namespace std;
class Obje
{
private:
static int instances;
int id;
public:
static int getInstances();
void getId();
virtual void myClass();
Obje(int auxId);
~Obje();
};
int Obje::instances = 0;
int Obje::getInstances()
{
return instances;
}
Obje::Obje(int auxId)
{
this->id = auxId;
cout << "Obje Created." << endl;
Obje::instances++;
}
Obje::~Obje()
{
cout << "Obje Destroyed." << endl;
Obje::instances--;
}
void Obje::myClass()
{
cout << "Obje." << endl;
}
void Obje::getId()
{
cout << this->id << endl;
}
int main()
{
vector <Obje> list;
Obje *a = new Obje(59565);
list.push_back(*a);
Obje *b = new Obje(15485);
list.push_back(*b);
for(vector<Obje>::iterator it = list.begin(); it != list.end(); ++it)
{
it->getId();
}
return 0;
}
It Generates this output:
Obje Created.
Obje Created.
Obje Destroyed.
59565
15485
Obje Destroyed.
Obje Destroyed.
What does it mean the T(const T& new); i've saw as fix for this?
First of all, it is a bad practice to allocate an object in heap without using smart pointers and forgetting to delete it. Especially, when you are creating it just to make a copy of it.
list.push_back(*a); creates a copy of *a in vector. To create an item in vector without copying another item, you can do list.emplace_back(/*constructor parameters*/);, which is available from c++11. (see http://en.cppreference.com/w/cpp/container/vector/emplace_back)
So, to make the result behavior match your expectations, you should go
vector <Obje> vec;
vec.emplace_back(59565);
vec.emplace_back(15485);
for(const auto & item : vec)
{
item.getId();
}
By the way, it is also a quite bad practice to call a vector as a list, as a list is a different container type and reading such code may be confusing a bit. I guess, I am starting being annoying, but it is better to call method getId as showId as now it returns nothing.
Regarding the use of heap, new and pointer, see my comment in your question.
Regarding the issue object was destroyed, the vector maintains an internal buffer to store object. When you push_back new object to the vector, if its internal buffer is full, it will (the stuff which will be executed when exception occurs won't be mentioned here.):
allocate new internal buffer which is big enough to store its new data.
move data from old internal buffer to new internal buffer.
destroy old buffer.
Hence, your object will be destroyed and copied to new location in this case, hence copy constructor will make it clearer to you.
P/S: AFAIK, some compilers move its data by memmove or std::move
I have a list that stores objects.
list<MyObject> l;
I also have a method that returns a pointer to one of those objects, if it exists, or nullptr otherwise.
MyObject* get(int x) {
for (std::list<MyObject>::iterator it = l.begin(); it != l.end(); ++it) {
if (it->X == x) {
return &(*it);
}
}
return nullptr;
}
If I get() a pointer to an element, and while I am using it, it gets erased from another thread, the pointer becomes invalid, and weird things happen :)
What I wanted to know is if there is a way of returning some special kind of pointer in get(), so that if I call erase on an element and that element is still being referenced, its memory won't be released until the pointer to it goes out of scope.
I considered using a reference, but I need the possibility of returning nullptr on get, so I can check from the caller if the return was indeed a valid object.
Can someone suggest a better way of doing this, to avoid these memory issues?
As recommended you should use some smart_pointer to manage the shared ownership.
Some recomendations:
Use always as default, std::vector
If could use C++11 use the standard shared_ptr for shared ownership, if not, use boost version.
Use the algorithm header as much as you can (in this case find_if is the right one).
You should also try to use the algorithm for the search of the specific element. Here is some sample code:
#include <algorithm>
#include <iostream>
#include <vector>
#include <memory>
struct MyObject {
int X;
MyObject(int x_value) : X(x_value) {}
};
using element_t = std::shared_ptr<MyObject>;
std::vector<element_t> l{
std::make_shared<MyObject>(3), std::make_shared<MyObject>(4),
std::make_shared<MyObject>(5), std::make_shared<MyObject>(6),
std::make_shared<MyObject>(7), std::make_shared<MyObject>(8)};
element_t get(int x) {
auto it = std::find_if(std::begin(l), std::end(l),
[x](const element_t& elem) { return elem->X == x; });
element_t found;
if (it != std::end(l)) {
found = *it;
}
return found;
}
int main() {
auto f1 = get(6);
if (f1) {
std::cout << "encontrado " << f1->X << std::endl;
} else {
std::cout << "6 no se encontro" << std::endl;
}
auto f2 = get(10);
if (f2) {
std::cout << "encontrado " << f2->X << std::endl;
} else {
std::cout << "10 no se encontro" << std::endl;
}
return 0;
}
Before using smart pointers, you might want to make sure you can spell out the reason why you can't (or don't want to) design a system where your objects have only one owner at a given time.
Smart pointers will avoid invalid data access, but they have all sorts of more or less hidden problems
they cost additional memory, force you to use them and their move semantics everywhere, and might easily become tricky, e.g. if you keep circular references or want an object to return a smart pointer to itself,
std:: containers become basically as useless as when you fill them with any kind of pointers (a vector of pointers is not a vector of objects),
you don't control where the deallocation takes place, so you might have your objects deleted by any task referencing them, possibly a time-critical one,
having no clear idea of who owns what is more often than not a recipe for disaster.
For instance, having one thread decide to delete objects while another grabs some from the same storage without any synchronization is very dangerous indeed. It is a bit as if one thread considered the object invalid while the other would consider it valid.
Does not strike me as the most robust design, but surely you have your reasons.
I think you could start by using unique_ptrs and see if that suits your needs, instead of jumping to shared_ptrs right away.
I'm a little confused about the best practice for how to do this. Say I have a class that for example allocs some memory. I want it to self destruct like an auto but also put it in a vector for some reason unknown.
#include <iostream>
#include <vector>
class Test {
public:
Test();
Test(int a);
virtual ~Test();
int counter;
Test * otherTest;
};
volatile int count = 0;
Test::Test(int a) {
count++;
counter = count;
std::cout << counter << "Got constructed!\n";
otherTest = new Test();
otherTest->counter = 999;
}
Test::Test() {
count++;
counter = count;
std::cout << counter << "Alloced got constructed!\n";
otherTest = NULL;
}
Test::~Test() {
if(otherTest != 0){
std::cout << otherTest->counter << " 1Got destructed" << counter << "\n";
otherTest->counter = 888;
std::cout << otherTest->counter << " 2Got destructed" << counter << "\n";
}
}
int vectorTest(){
Test a(5);
std::vector<Test> vecTest;
vecTest.push_back(a);
return 1;
}
int main(){
std::cout << "HELLO WORLD\n";
vectorTest();
std::cout << "Prog finished\n";
}
In this case my destructor gets called twice all from counter 1, the alloc' object has already been set to 888 (or in a real case freed leading to bad access to a deleted object). What's the correct case for putting a local variable into a vector, is this some kind of design that would never happen sensibly. The following behaves differently and the destructor is called just once (which makes sense given its an alloc).
int vectorTest(){
//Test a(5);
std::vector<Test> vecTest;
vecTest.push_back(*(new Test(5)));
return 1;
}
How can I make the local variable behave the same leading to just one call to the destructor? Would a local simply never be put in a vector? But aren't vectors preferred over arrays, what if there are a load of local objects I want to initialize separately and place into the vector and pass this to another function without using free/heap memory? I think I'm missing something crucial here. Is this a case for some kind of smart pointer that transfers ownership?
A vector maintains its own storage and copies values into it. Since you did not implement a copy constructor, the default one is used, which just copies the value of the pointer. This pointer is thus deleted twice, once by the local variable destructor and once by the vector. Don't forget the rule of three. You either need to implement the copy and assignment operators, or just use a class that already does this, such as shared_ptr.
Note that this line causes a memory leak, since the object you allocated with new is never deleted:
vecTest.push_back(*(new Test(5)));
In addition to what Dark Falcon wrote: To avoid reallocating when inserting into a vector, you typically implement a swap function to swap local element with a default-constructed one in the vector. The swap would just exchange ownership of the pointer and all will be well. The new c++0x also has move-semantics via rvalue-references to help with this problem.
More than likely, you'd be better off having your vector hold pointers to Test objects instead of Test objects themselves. This is especially true for objects (like this test object) that allocate memory on the heap. If you end up using any algorithm (e.g. std::sort) on the vector, the algorithm will be constantly allocating and deallocating memory (which will slow it down substantially).