C++ operator delete override not always used - c++

I have some C++ unit tests using google test. Threw together some code to override the new/delete operators to check for leaks in the unit tests. Having an issue though. Some of the google test new/deletes use my overridden methods, but some don't, so I get false errors in the tracking code -- sometimes seeing that memory was leaked even though it was really deleted and sometimes seeing that malloc returns
Here is my minimal new/delete overrides (just prints the addresses for manual inspection):
void * operator new(size_t size)
{
void * addr = malloc(size);
std::cout << " tracking create: " << addr << "(size " << size << ")" << std::endl;
return addr;
}
void * operator new[](size_t size)
{
void * addr = malloc(size);
std::cout << " tracking create: " << addr << "(size " << size << ")" << std::endl;
return addr;
}
void operator delete(void * addr) noexcept
{
std::cout << " tracking delete: " << addr << std::endl;
free(addr);
}
void operator delete[](void * addr) noexcept
{
std::cout << " tracking delete: " << addr << std::endl;
free(addr);
}
And here is the google test line that does NOT go through my overridden delete (gtest-port.h):
void reset(T* p = NULL) {
if (p != ptr_) {
if (IsTrue(sizeof(T) > 0)) { // Makes sure T is a complete type.
delete ptr_;
}
ptr_ = p;
}
}
When I break on the delete ptr_ line in gdb, then step, it steps directly to the ptr_ = p line, so there isn't something else overriding that delete.
I'm building gtest as an archive file and linking it in when I build my unit tests. In case it matters: I'm on windows building with mingw using cygwin.
Here's a minimal example, 2 files min.cpp and minmain.cpp. Here's min.cpp:
#include <iostream>
#include <string>
// Overload the new/delete operators to check for memory errors
void * operator new(size_t size)
{
void * addr = malloc(size);
std::cout << " tracking create: " << addr << "(size " << size << ")" << std::endl;
return addr;
}
void * operator new[](size_t size)
{
void * addr = malloc(size);
std::cout << " tracking create: " << addr << "(size " << size << ")" << std::endl;
return addr;
}
void operator delete(void * addr) noexcept
{
std::cout << " tracking delete: " << addr << std::endl;
free(addr);
}
void operator delete[](void * addr) noexcept
{
std::cout << " tracking delete: " << addr << std::endl;
free(addr);
}
minmain.cpp:
#include "gtest/gtest.h"
TEST(MinTest, MinimalTest)
{
int test = 5;
test++;
test++;
test++;
ASSERT_EQ(test, 8);
}
int main(int argc, char *argv[])
{
char* t = new char();
t[0] = 't'; std::cout << "t is " << t[0] << std::endl;
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
compiled with:
/bin/x86_64-w64-mingw32-g++.exe -std=c++11 -D_USE_MATH_DEFINES -g -Wall -I../third_party/googletest-1.8.0/googletest/include -c min.cpp -o min.o
to create the min.o, then compile main and link all together with:
/bin/x86_64-w64-mingw32-g++.exe -std=c++11 -D_USE_MATH_DEFINES -g -Wall -I../third_party/googletest-1.8.0/googletest/include -o minmain minmain.cpp min.o ../third_party/googletest-1.8.0/googletest/make/gtest_main.a
Using version 1.8.0 of gtest, break at gtest-port.h:1145 to get to the delete ptr_ line, then step.
Here is some sample output from running the example above (first few lines of output):
tracking create: 0x30e4c0(size 392)
tracking create: 0xa477e0(size 392)
tracking create: 0xa477e0(size 392)
tracking create: 0xa477e0(size 392)
tracking create: 0xa477e0(size 392)
tracking create: 0xa47b80(size 28)
tracking delete: 0xa47b80
The fact that I get tracked creates on the same address with no tracked deletes in between is a problem, because there were deletes in between allowing the same address to be allocated again, but those deletes did not go through my overridden delete operator.
Why does that delete ptr_; line in gtest not use my overridden delete function?

Looks like this is a bug in MinGW:
MinGW bug #634
A work-around is to link the static version of libstdc++ instead of letting it link the dynamic library. Not the most ideal solution, but it's good enough for my unit tests and it allows me to override correctly.
I modified by compile/link command to the following to do this:
/bin/x86_64-w64-mingw32-g++.exe -std=c++11 -D_USE_MATH_DEFINES -g -Wall -I../third_party/googletest-1.8.0/googletest/include -o minmain minmain.cpp min.o ../third_party/googletest-1.8.0/googletest/make/gtest_main.a /cygdrive/c/cygwin64/lib/gcc/x86_64-w64-mingw32/6.4.0/libstdc++.a
Mucho thanko to Peter for putting me down the right path to finding this.

Related

Will moving from released pointers leak memory?

I have the following code:
std::unique_ptr<T> first = Get();
…
T* ptr_to_class_member = GetPtr(obj);
*ptr_to_class_member = std::move(*first.release());
Will this behave as expected with no copies, 1 move and without memory leak?
*ptr_to_class_member = std::move(*first.release());
just calls the move assignment operator of T with the object pointed to by first as argument. This may properly transfer some data, but delete is not called or the object so neither T::~T is executed nor does the memory of the object get freed.
In the example of T = std::string this would result in the backing storage of the string object properly being transfered from the rhs to the lhs of the move assignment, but dynamically allocated memory of size sizeof(std::string) would still be leaked.
For some classes the lack of a destructor invocation for the object could result in additional trouble, since move assignment simply needs to leave the rhs in an unspecified, but valid state which could still require freeing of additional resources.
You need to do
*ptr_to_class_member = std::move(*first);
first.reset();
in order to prevent memory leaks.
To show what's going wrong here, the following code implements prints for memory (de)allocation and special member functions:
#include <iostream>
#include <memory>
#include <new>
#include <utility>
struct TestObject
{
TestObject()
{
std::cout << "TestObject::TestObject() : " << this << '\n';
}
TestObject(TestObject&& other)
{
std::cout << "TestObject::TestObject(TestObject&&) : " << this << ", " << &other << '\n';
}
TestObject& operator=(TestObject&& other)
{
std::cout << "TestObject::operator=(TestObject&&) : " << this << ", " << &other << '\n';
return *this;
}
~TestObject()
{
std::cout << "TestObject::~TestObject() : " << this << '\n';
}
void* operator new(size_t size)
{
void* const result = ::operator new(size);
std::cout << "memory allocated for TestObject: " << result << '\n';
return result;
}
void operator delete(void* mem)
{
std::cout << "memory of TestObject deallocated: " << mem << '\n';
::operator delete(mem);
}
};
template<class Free>
void Test(Free free, char const* testName)
{
std::cout << testName << " begin -------------------------------------------\n";
{
auto ptr = std::make_unique<TestObject>();
std::cout << "object creation done\n";
free(ptr);
}
std::cout << testName << " end ---------------------------------------------\n";
}
int main()
{
TestObject lhs;
Test([&lhs](std::unique_ptr<TestObject>& ptr)
{
lhs = std::move(*ptr);
ptr.reset();
}, "good");
Test([&lhs](std::unique_ptr<TestObject>& ptr)
{
lhs = std::move(*ptr.release());
}, "bad");
}
Possible output:
TestObject::TestObject() : 0000009857AFF994
good begin -------------------------------------------
memory allocated for TestObject: 000001C1D5715EF0
TestObject::TestObject() : 000001C1D5715EF0
object creation done
TestObject::operator=(TestObject&&) : 0000009857AFF994, 000001C1D5715EF0
TestObject::~TestObject() : 000001C1D5715EF0
memory of TestObject deallocated: 000001C1D5715EF0
good end ---------------------------------------------
bad begin -------------------------------------------
memory allocated for TestObject: 000001C1D5715EF0
TestObject::TestObject() : 000001C1D5715EF0
object creation done
TestObject::operator=(TestObject&&) : 0000009857AFF994, 000001C1D5715EF0
bad end ---------------------------------------------
TestObject::~TestObject() : 0000009857AFF994
You can clearly see the destructor call and deallocation missing in the second case, which is the one matching the code you're asking about.

Haxe/Neko exceptions when exiting a call to a C++ FFI that iterates over items in an abstract kind

When I return a value of abstract kind containing a vector from one FFI, and iterate over it in another FFI, there's an exception as the second FFI call finishes. I can access the items in the vector one by one without a problem, but iterating over the values seems to cause a problem. It doesn't matter if I use a for each loop, an iterator, or even loop over the indices.
Haxe code:
class VectorExample {
public static function main() {
var vec = createVec();
// dumpForIntIter(vec); // fails on exit
dumpEach(vec); // works
}
private static var createVec = neko.Lib.load("vectorLib", "createVec", 0);
private static var dumpForIntIter = neko.Lib.load("vectorLib", "dumpForIntIter", 1);
private static var dumpEach = neko.Lib.load("vectorLib", "dumpEach", 1);
}
Cpp code:
#include <iostream>
#include <string>
#include <vector>
#include <neko.h>
void free_vec( value handle ) {
std::cout << "freeing vec" << std::endl;
std::vector<std::string>* vec = (std::vector<std::string>*) val_data(handle);
delete vec;
}
DEFINE_KIND(k_vector);
value createVec() {
auto vec = new std::vector<std::string>();
vec->push_back(std::string("one"));
vec->push_back(std::string("two"));
vec->push_back(std::string("three"));
value handle = alloc_abstract(k_vector, vec);
val_gc(handle, free_vec);
return handle;
}
DEFINE_PRIM(createVec,0);
// fails on exit
void dumpForIntIter( value handle ) {
auto vec = (std::vector<std::string>*) val_data(handle);
std::cout << "size: " << vec->size() << std::endl;
for (int ii=0; ii<vec->size(); ii++)
std::cout << " item: " << vec->at(ii) << std::endl;
}
DEFINE_PRIM(dumpForIntIter,1);
// works
void dumpEach( value handle ) {
int ii = 0;
auto vec = (std::vector<std::string>*) val_data(handle);
std::cout << "size: " << vec->size() << std::endl;
std::cout << " item: " << vec->at(0) << std::endl;
ii++;
std::cout << " item: " << vec->at(1) << std::endl;
ii++;
std::cout << " item: " << vec->at(2) << std::endl;
ii++;
}
DEFINE_PRIM(dumpEach,1);
Command to build the Neko module:
g++ -o vectorLib.ndll -shared -fPIC -std=c++11 -I/usr/include/x86_64-linux-gnu \
-L/usr/lib/x86_64-linux-gnu -lneko -ldl vectorLib.cpp
Output when running with dumpForIntIter:
size: 3
item: one
item: two
item: three
freeing vec
Called from ? line 1
Called from VectorExample.hx line 6
Uncaught exception - vectorLib#dumpForIntIter
Note that although freeing vec is the last thing in the log, the problem happens even if free_vec is empty or non existent.
Since dumpForIntIter and dumpForEach seem basically equivalent to me, I suspect there is a problem in createVec.
Docs: Neko FFI
UPDATE:
I disabled the exception trapping in nekovm and found that the problem is a segfault. This is the output from valgrind:
Jump to the invalid address stated on the next line
at 0x2A50DA10C1C9AED6: ???
Address 0x2a50da10c1c9aed6 is not stack'd, malloc'd or (recently) free'd
Can't extend stack to 0x2a50da10c1c99f88 during signal delivery for thread 1:
no stack segment
Process terminating with default action of signal 11 (SIGSEGV)
Access not within mapped region at address 0x2A50DA10C1C99F88
at 0x2A50DA10C1C9AED6: ???
UPDATE 2:
If I change the vector<string> to vector<int> the problem persists. If I change to a char** (using malloc or new)there's no problem. If I change to just a string there's no problem. Seems like alloc_abstract doesn't like vector.

How to track memory usage using EASTL?

The Electronic Arts EASTL library's east::allocator requires the user to implement a special new operator (as shown in this sample). This new operator has a const char* name, which is supposed to be used to log application-specific information about memory allocations performed EASTL containers. The EASTL Best Practices guide also mentions that one should "Name containers to track memory usage."
However, there does not appear to be any corresponding const char* name passed by eastl::allocator::deallocate, so there does not appear to be a built-in way to log memory deallocations.
How is this EASTL debugging functionality intended to be used? I think I must be missing something. An EA Software Engineer also gave a related presentation in 2015, where he showed some internal tools developed by EA for memory debugging, and it seems to suggest that they tracking both allocations and deallocations (for example, to ensure that the memory for "game level 1" is freed when "game level 2" begins).
The allocators by the EASTL are stateful and bound to the instance of the container; meaning they are defined at instance level:
What EASTL does is use a more familiar memory allocation pattern
whereby there is only one allocator class interface and it is used by
all containers. Additionally EASTL containers let you access their
allocators and query them, name them, change them, etc.
EASTL has chosen to make allocators not be copied between containers
during container swap and assign operations. This means that if
container A swaps its contents with container B, both containers
retain their original allocators. Similarly, assigning container A to
container B causes container B to retain its original allocator.
Containers that are equivalent should report so via operator==; EASTL
will do a smart swap if allocators are equal, and a brute-force swap
otherwise.
from https://github.com/questor/eastl/blob/master/doc/EASTL%20Design.html
So I would add a member to the allocator class and track the memory count inside it, like following:
#ifndef EASTL_CUSTOM_ALLOCATOR_H_
#define EASTL_CUSTOM_ALLOCATOR_H_
#include "new_implementation.hpp"
#include <EASTL/list.h>
#include <iostream>
#define DEBUG_MACRO
class EASTL_CustomAllocator {
public:
EASTL_CustomAllocator(const char* pName = EASTL_NAME_VAL(EASTL_ALLOCATOR_DEFAULT_NAME))
: m_pName(pName), m_totalAmountOfBytesAllocated(0) {
#ifdef DEBUG_MACRO
std::cout << m_pName << ": default construct allocator" << std::endl;
#endif
}
EASTL_CustomAllocator(const EASTL_CustomAllocator& x)
: m_pName(x.m_pName),
m_totalAmountOfBytesAllocated(x.m_totalAmountOfBytesAllocated) {
#ifdef DEBUG_MACRO
std::cout << m_pName << ": copy construct allocator" << std::endl;
#endif
}
EASTL_CustomAllocator(const EASTL_CustomAllocator& x, const char* pName)
: m_pName(pName),
m_totalAmountOfBytesAllocated(x.m_totalAmountOfBytesAllocated) {
#ifdef DEBUG_MACRO
std::cout << m_pName << ": copy construct allocator" << std::endl;
#endif
}
EASTL_CustomAllocator& operator=(const EASTL_CustomAllocator& x) {
#ifdef DEBUG_MACRO
std::cout << m_pName << ": copy assignment" << std::endl;
#endif
m_pName = x.m_pName;
m_totalAmountOfBytesAllocated = x.m_totalAmountOfBytesAllocated;
return *this;
}
void* allocate(size_t num_of_bytes, int flags = 0) {
m_totalAmountOfBytesAllocated += num_of_bytes;
void* p = ::new((char*)0, flags, 0, (char*)0, 0) char[num_of_bytes];
#ifdef DEBUG_MACRO
std::cout << m_pName << ": allocate " << num_of_bytes << " bytes" << " at: " << (void*) p << std::endl;
#endif
return p;
}
void* allocate(size_t num_of_bytes, size_t alignment, size_t offset, int flags = 0) {
m_totalAmountOfBytesAllocated += num_of_bytes;
void* p = ::new(alignment, offset, (char*)0, flags, 0, (char*)0, 0) char[num_of_bytes];
#ifdef DEBUG_MACRO
std::cout << m_pName << ": allocate " << num_of_bytes << " bytes" << " at: " << (void*) p << std::endl;
#endif
return p;
}
void deallocate(void* p, size_t num_of_bytes) {
m_totalAmountOfBytesAllocated -= num_of_bytes;
#ifdef DEBUG_MACRO
std::cout << m_pName << ": deallocate " << num_of_bytes << " bytes" << " at: " << (void*) p << std::endl;
#endif
delete[](char*)p;
}
const char* get_name() const {
return m_pName;
}
void set_name(const char* pName) {
m_pName = pName;
}
size_t get_totalAmountOfBytesAllocated() const {
return m_totalAmountOfBytesAllocated;
}
protected:
const char* m_pName; // Debug name, used to track memory.
size_t m_totalAmountOfBytesAllocated; // keeps track of the memory currently allocated
};
bool operator==(const EASTL_CustomAllocator& a, const EASTL_CustomAllocator& b) {
if (&a == &b) {
return true; // allocator a and b are equal if they are the same
}
else {
return false; // otherwhise, return false, because the state m_totalAmountOfBytesAllocated needs to be increased/decreased on splice and swap
}
}
bool operator!=(const EASTL_CustomAllocator& a, const EASTL_CustomAllocator& b) {
return false;
}
#endif /* EASTL_CUSTOM_ALLOCATOR_H_ */
Pass that custom allocator type as a template parameter to a eastl container like following (also you can set a instance with a user-defined name at construction and even later by set_allocator()):
eastl::list<int, EASTL_CustomAllocator>
list(EASTL_CustomAllocator("EASTL Some Name"));
But I'm not sure how the debugging functionality is intended to be used.

using int pointer in multiple couts

I'm quite new to the world of pointers in C/C++ so this may be quite an easy question for you:
The following C++-Code works normally
#include <iostream>
int main()
{
int theInt = 1337;
int & theReference = theInt;
int * thePointer = &theInt;
std::cout << "int: " << theInt << "\n";
std::cout << "referenz: " << theReference << "\n";
std::cout << "pointer: " << *thePointer << "\n";
std::cout << "pointer: " << *thePointer << "\n";
//std::cout << "foo" << "\n";
return 0;
}
but stops working when changing
//std::cout << "foo" << "\n";
to
std::cout << "foo" << "\n";
.
By "stops working" I mean: "is blocked by my norton security as a potential threat" (resulting in a return code of "0x76F531AF" if this is any help). Since norton normally doesn't interfere withe my programming I assume the double use of the int pointer in cout somehow results in a segfault...
Thx for your help!
PS:
I use Code::Blocks on Windows 8.1 with the GCC compiler and GDB debugger from TDM-GCC (version 4.7.1, 32 bit).
EDIT: removed deletion of pointer -> problem remains.
You can only delete objects created on the heap (using new or C-style malloc and such).
// allocate on the heap
int *intOnTheHeap = new int;
// do some stuff with it
*intOnTheHeap = 0;
(*intOnTheHeap)++;
std::cout << *intOnTheHeap << std::endl;
// deallocate
delete intOnTheHeap;
If you take a pointer to a local variable, it will point to an entry on the stack. You don't need to and shouldn't deallocate that memory yourself. The memory is "freed" by changing the stackpointer automatically when your variable runs out of scope (at the end of the function).
void myFunction() {
int localVariable;
int *pointerToLocalVariable = &localVariable;
// forbidden and unnecessary:
//delete pointerToLocalVariable;
// here (end of the block) the memory on the stack
// will be freed automatically
}
Since I got the same error after Norton-Interception in totally different contexts, this seems to be a case of Code::Blocks Norton incompatibility.

How to use overloaded new operator used in QT namespace

I am writing a C++ application which used QT libraries. I want to detect memory leaks in my application and in QT. So, I have overloaded the new and delete operator in my main.cpp using this reference http://lists.trolltech.com/qt-interest/2002-04/msg00933.html, but QT is not using the overloaded operators. Apparently, it appears to be a namespace issue. How to resolve this.
int numAllocUnits = 0;
ofstream myLogFile("/root/memLeak.log");
class MemoryLeak_Manav
{
public:
MemoryLeak_Manav() {
if (!myLogFile.is_open()) {
cout << "Unable to open file";
}
myLogFile << "Memory Leak Detection log File" << endl;
printf("Memory Leak Detection On ... ");
}
public:
~MemoryLeak_Manav() {
myLogFile.close();
if(numAllocUnits)
printf("\nError: Memory leak detected: %d\n\n", numAllocUnits);
else printf("\nNo memory leak detected.\n\n");
}
public:
void *operator new [] (size_t size);
void *operator new (size_t size);
void operator delete [] (void *p);
void operator delete (void *p);
};
void * MemoryLeak_Manav::operator new(size_t size)
{
void *newPtr;
numAllocUnits++;
newPtr = malloc(size);
printf("malloc [%p allocated %d bytes]\n", newPtr, size);
myLogFile << "malloc [" << newPtr << "allocated" << size << "bytes" << endl;
return newPtr;
}
void MemoryLeak_Manav::operator delete(void *p)
{
numAllocUnits--;
free(p);
}
void * MemoryLeak_Manav::operator new [] (size_t size)
{
void *newPtr;
numAllocUnits++;
newPtr = malloc(size);
printf("malloc [%p allocated %d bytes]\n", newPtr, size);
myLogFile << "malloc [" << newPtr << "allocated" << size << "bytes" << endl;
return newPtr;
}
void MemoryLeak_Manav::operator delete [] (void *p)
{
numAllocUnits--;
printf("free %p\n", p);
myLogFile << "free" << p << endl;
free(p);
}
The memLeak.log file is empty and I am not seeing any printf's messages also.
You can't overload new in a library because a library is already compiled. To replace new in Qt, you have to obtain Qt sources, put overload in whatever base file of them and then recompile. It isn't as hard as it sounds, by the way.
You made your operators member of a class. Like that, they will only be used within the namespace of the class.
Just define the operators as global operators (without a class) and it should work
If you can test it on Linux, then have a look at using the Valgrind Memcheck tool.
This provides very detailed information on leaks and bad memory use (double free, corruption, partial free), including full stack traces.