So, I'm curious about this thing I can't figure out.
I'm creating some new objects and passing them to a function which stores them in a std::stack.
However, when i want to delete them - they do not actually get deleted, and as such, memory usage will proceed to climb "forever" with my test loop.
Why?
bool StateMachine::changeState(BaseState *state) {
if (state == nullptr) {
delete states.top();
states.pop();
if (states.size() == 0) {
return false;
}
} else if (state != states.top()) {
states.push(state);
}
return true;
}
Test loop:
while (true) {
machine.changeState(new MenuState);
machine.changeState(nullptr);
}
Using a std::unique_ptr instead of raw works, and now ram usage is constant, but still - I wanna know.
Cheers!
Your code should be correct given the preconditions you've mentioned, but notice that you can allocate and delete objects without reclaiming operating system allocated memory, especially if you leave allocation holes in memory. So check both if memory starts growing then stops and for memory leaks elsewhere, like inside BaseState.
If you're in doubt about preconditions, add an else clause in your if and print something. I should never happen, but if it does, then there may be some problem calling states.top().
Related
what happens to data created in local scope of thread if thread is terminated, memory leak?
void MyThread()
{
auto* ptr = new int[10];
while (true)
{
// stuff
}
// thread is interrupted before this delete
delete[] ptr;
}
Okay, my perspective.
If the program exits, the threads exit wherever they are. They don't clean up. But in this case you don't care. You might care if it's an open file and you want it flushed.
However, I prefer a way to tell my threads to exit cleanly. This isn't perfect, but instead of while (true) you can do while (iSHouldRun) and set the field to false when it's time for the thread to exit.
You can also set a flag that says, iAmExiting at the end, then myThread.join() once the flag is set. That gives your exit code a chance to clean up nicely.
Coding this from the beginning helps when you write your unit tests.
The other thing -- as someone mentioned in comments -- use RAII. Pretty much if you're using raw pointers, you're doing something you shouldn't do in modern C++.
That's not an absolute. You can write your own RAII classes. For instance:
class MyIntArray {
MyArray(int sizeIn) { ... }
~MyArray() { delete array; }
private:
int * array = nullptr;
int size = 0;
};
You'll need a few more methods to actually get to the data, like an operator[]. Now, this isn't any different than using std::vector, so it's only an example of how to implement RAII for your custom data, for instance.
But your functions should NEVER call new like this. It's old-school. If your method pukes somehow, you have a memory leak. If it pukes on exit(), no one cares. But if it pukes for another reason, it's a problem. RAII is a much, much better solution than the other patterns.
I run into a corner case while using a shared_ptr based "database" that doubles as an LRU.
Since C++17, the shared_ptr::use_count is imprecise, so I have trouble deciding on which elements can be safely removed from the LRU.
I cannot remove still-in-use elements, as that would break contracts in the rest of the code.
As far as I understand it locking a mutex is not enough, since only a mutex unlock will force a memory barrier. I could still read an outdated value even when holding the lock.
I could of course slap a memory barier after I lock a mutex inside the LRU, but I'm a bit worried about the performance impact.
Here is an outline of how the lru works:
template <k,v>
class DB{
shared_ptr<V> emplace(k, args...) {
lock guard();
remove_elements_if_needed();
insert_if_new(k, args...);
refresh_lru(k);
return ptr;
}
};
General solution I'd propose is while under a lock, identify a candidate element to remove. Then attempt to remove it by doing this:
Create a weak_ptr instance to an element you think might be a good
candidate to remove
Delete the item from the list as you normally would
Try to restore the item as a shared_ptr from the weak_ptr
If you promote the weak_ptr back to shared_ptr and it's still null,
then you are done.
If the new shared_ptr is not null, then you know someone still has a
reference to the item. Put the item back into your data structure
exactly as you found it.
Something like this. Since I don't have your code yet, I'm winging it wrt to your implementation. But I'm guessing you have a collection of "nodes". Each node has a member that is the shared_ptr instance that you hand back to callers.
bool remove_unused_element() {
bool removed = false;
for (node& n : lru) {
weak_ptr<X> wp = n.item;
n.item.reset();
shared_ptr<X> sp = wp.lock();
if (sp != nullptr){
n.item = sp ; // restore this node, someone is still using it
}
else {
lru.erase(n);
removed = true;
break;
}
}
return removed;
}
void remove_elements_if_needed() {
bool result = true;
while ((lru.size() > max_lru_size) && result) {
result = remove_unused_element();
}
}
All of the above code assumes you have acquired the mutex as you show in your pseudo code.
I'm going crazy with this piece of code. I have a thread that calls regularly to this method:
void deliverMsgQ() {
if(delMsgQ_mutex.try_lock() == false){
return;
}
while(delMsgQ.empty() == false){
std::vector<unsigned char> v = delMsgQ.front();
delMsgQ.pop();
}
delMsgQ_mutex.unlock();
}
void processInMsgQ() {
if(inMsgQ_mutex.try_lock()){
if(delMsgQ_mutex.try_lock() == false){
inMsgQ_mutex.unlock();
}
}else{
return;
}
while(!inMsgQ.empty()){
std::vector<unsigned char> msg;
inMsgQ.front()->getData(msg);
std::cout << "Adding to del-msg-q: " << msg.size() << std::endl;
delMsgQ.push(msg);
delete inMsgQ.front();
inMsgQ.pop();
}
inMsgQ_mutex.unlock();
delMsgQ_mutex.unlock();
}
I have another thread pushing vector to the queue also periodically. These two threads are the only ones that touch the queue delMsgQ.
My problems comes in the first function posted, for some reason delMsgQ.empty() at some point returns false even though it has no vectors in it, and therefore I end up calling pop twice. This causes the size function to become a huge unrealistic number and then the program goes into segmentation fault. I can fix this if I add an additional check right before calling pop, but I would expect that checking once would be enough since I'm also using mutexes. So the other alternative would be that maybe I'm using mutexes wrong, but as far as I know this is the proper way to use them in this case. So I was hoping that maybe someone smarter could let me know if there is something I'm missing? I hope this code is enough, I can provide with more code if necessary although no other function touch the queue that is failing.
best regards
Your code in processInMsgQ() (spaced out a little better but functionally identical) is problematic:
if (inMsgQ_mutex.try_lock()) {
if (delMsgQ_mutex.try_lock() == false) {
// Point A
inMsgQ_mutex.unlock();
// Point B.
}
} else {
return;
}
// Point C.
In the case where it locks inMsgQ_mutex but fails to lock delMsgQ_mutex (point A), it will free the first and then drop through to point C. This means you will be doing stuff that requires both locks without either lock, and that's unlikely to end well :-)
As a solution, you could put another return at point B but the following code is probably cleaner:
// If either lock fails, return ensuring that neither is locked.
if (! inMsgQ_mutex.try_lock()) {
return;
}
if (! delMsgQ_mutex.try_lock() {
inMsgQ_mutex.unlock();
return;
}
// At this point, you have both locks. Carry on ...
You'll notice I've also changed your some_boolean == false to the more usual ! some_boolean. That's the more accepted way of doing that particular check.
I'm not at ease with the way memory's managed in the following example :
void X::foo(void) {
// test1 :
for (auto i = this->vec.begin(); i != this->vec.end(); ++i) {
if ( ... ) {
return;
}
}
// test2
...
// test3
...
}
Even if there's no memory allocation for iterators, I would like to be sure that there's no problem quitting such abruptly the loop. Is it safer to break the loop, check the result of the test and then quit the function without passing throuh test2 and test3 ?
This is safe as long as you return any possibly allocated resources back to the system. If you are in c++ and utilizing structures who properly implement the ideas of RAII, e.g., shared_ptr etc. Then there will be no problem.
Return essentially "immediately" leaves the function call. However, before it does, the compiler ensures destruction of any temporary objects first.
variables declared inside the for are allocated in the "automatic memory" meaning that they will exist inside the variables scope itself: "i" will be deallocated automatically when it goes out of scope.
so yes, it's safe to return "abruptly" from the loop.
I am opening an audio file to read it and I get an abandoned malloc block from this caller each time.
In a loop I set data like this (which is marked as the memory usage in instruments as 99.7%) data = (short*)malloc(kSegmentSize*sizeof(short));
and free it like this free(data); at the end of each iteration.
Im not really sure what is happening here and would appreciate any help.
EDIT: KSegmentSize varies in the thousands, from minimum 6000 - max 50000 (speculative)
Instruments trace:
Not having the exact code:
Pretty sure you're having this problem b/c something between the malloc and free is throwing (and you're probably catching it already so you don't exit the loop). Depending on if this is happening in C (or objective-C) or C++ code, you have slightly different methods of resolution.
In C++, wrap the malloc/free in the RAII pattern so that when the stack is unwound the free is called.
class MyData {
public:
A(size_t numShorts) : dataPtr(0) { dataPtr = malloc(numShorts * sizeof(short)); }
~A() { free(dataPtr); }
operator short*() { return dataPtr; }
private:
short* dataPtr;
}
MyData data(numShorts);
// do your stuff, you can still use data as you were before due the 'operator short*'
// allow the dtor to be called when you go out of scope
In Objective-C you need to use a finally block:
void* myPtr = 0;
#try { myPtr = malloc(...); }
#catch {}
#finally { free(myPtr); }
Suggest that you start by simplifying, for example comment out (preferably using #if 0) all of the code except the malloc/free. Run the code and ensure no abandoned heap blocks. Then gradually re-introduce the remaining code and re-run until you hit the problem, then debug.
Sorry to answer my own question, but after commenting out code back up the stack trace the actual issue was to do with the file not be disposed.
Calling ExtAudioFileDispose(audioFile); solved this hidden bug. Instruments was not entirely clear and marked mallocs as the leak. To be fair the mallocs where from data that was within the file referenced by the ExtAudioOpenFile method, not disposing the file reference left a leak.