I am trying to get a name of a column from an oracle table using the MetaData class. I get a vector of MetaData objects from the ResultSet and then I loop over them executing the getString() function on each item, the problem is that on the second iteration, when exiting the loop to start a new iteration, it crashes on gives me CrtIsValidHeapPointer Assertion.
/*
* If this ASSERT fails, a bad pointer has been passed in. It may be
* totally bogus, or it may have been allocated from another heap.
* The pointer MUST come from the 'local' heap.
*/
_ASSERTE(_CrtIsValidHeapPointer(pUserData));
The data being pointed to by pUserData is actually valid, so I suspect my heap from the external API DLL is not the same as the CRT heap, the question is how do I resolve this issue?
my code:
std::vector<oracle::occi::MetaData> data = res->getColumnListMetaData();
for (std::vector<oracle::occi::MetaData>::iterator iter = data.begin(); iter != data.end(); iter++)
{
//Crash on second iteration after this statement
std::string s = (iter->getString(oracle::occi::MetaData::ATTR_NAME));
int i = iter->getInt(oracle::occi::MetaData::ATTR_DATA_TYPE);
std::cout << i << std::endl;
}
Does anybody have any suggestions or has anybody had this problem and solved it?
OS = Windows, VS2008, Oracle 11.2
Nothing in that code does any direct heap deallocation, although, of course, std::string does allocate and deallocate heap memory. However, that shouldn't be a problem unless
the heap is corrupted by some other operation or
you pass std::string across executable boundaries, resulting in one executable (e.g. the DLL) allocating memory and another (e.g. the EXE) deallocating it.
You seem to be expecting the latter:
The data being pointed to by pUserData is actually valid, so I suspect my heap from the external API DLL is not the same as the CRT heap, the question is how do I resolve this issue?
That might indeed be the case. If you have control over both executables, you can make them both use the same dynamic RTL ("Multi-threaded Debug DLL" or something like that in VC).
However, in general it's not a good idea to have one executable free the resources of another one. Usually you should pass resources back to the API you obtained them from, so that it can be freed where it was allocated.
Related
I need some clarification on the following code.
In the following code, I am retrieving a private implementation of an interface from a DLL.
After the DLL has been unloaded(CASE #2), the string retrieved from the interface, and the interface itself both become invalid, and access violations occur(expected) when they are accessed.
But what's confusing me, is that when when I reload the DLL, I can use the interface and the string again, without re-retrieving them from the DLL.
Is this supposed to work this way?
How can the memory become invalid, and them suddenly become valid again once the DLL is loaded again?
Is this just luck, and the DLL conveniently loading into the exact same place in memory the second time because of the relative simplicity of my test program?
int main(int argc, int *argv[])
{
Interface *itf;
const char *name;
{
// loads the dll and gets functions
PersistenInterface pi("PersistentInterface.dll");
// returns a private implementation of the interface
itf = pi.CreateInterface();
name = itf->GetName();
// CASE #1
cout << name << endl; // suceeds
} // dll is unloaded in ~PersistenInterface()
// CASE #2
//cout << name << endl; // crashes
//cout << itf->GetName() << endl; // crashes
{
PersistenInterface pi("PersistentInterface.dll");
// CASE #3
cout << name << endl; // suceeds !?
cout << itf->GetName() << endl; // suceeds !?
}
return 0;
}
I hope you are not returning a local object on stack from DLL. It will definitely crash your system because the local object is destroyed after function returns from the DLL. It has nothing to do with the concept of DLL. If you try to reference a local object after its containing function returns, you'll see the same problem.
If the memory is dynamically allocated on heap inside DLL, it's still valid after the DLL is uninitialized. You can still use the memory in your main application, because DLL itself doesn't own anything. Remember, the heap belongs to the process, not DLL.
However, the general rule is that the owner of that memory(who creates that memory, e.g. the DLL in your case) is responsible to release the same memory. Allocating memory in DLL and deallocating it in the other side may cause you unnecessary troubles, if the two sides use different instances of CRT, e.g.
The answer is a little complex. Actually, if you load a DLL, it is not really loaded in your heap memory. It is loaded in global memory and this memory is mapped into your address-space.With this mechanism the OS saves memory, because it can map the same code into different processes. So if you access an address within the DLL space, the access will be redirect into the global memory where the DLL really exist. After unloading, there is no memory anymore at this address. Your access will not be redirect any more, a exception will be thrown. This is different to a malloc and free (or new and delete) where only the memory is marked as unused, but still there with all its data (unless zero out memory is activated). Static DLL data (like constant strings) and code of a DLL acts like described above. After you load a DLL again, the addresses you have, which obviously direct to static data or code, became valid again. At least Windows XP loads a DLL always at the same address-space. But do not rely on this! For security reason, modern OS can decide to map DLLs of your process on different addresses on every time loadLibrary is used or every time you start your process. In the debugger or with Windows XP, DLL's are always mapped at the same address as far as I know, that's the reason of the behavior for your code. It is not a bug or a feature, it is by DLL concept.
I have been using tcmalloc for a few months in a large project, and so far I must say that I am pretty happy about it, most of all for its HeapProfiling features which allowed to track memory leaks and remove them.
In the past couple of weeks though we experienced random crashes in our application, and we could not find the source of the random crash. In a very particular situation, When the application crashed, we found ourselves with a completely corrupted stack for one of the application threads. Several times instead I found that threads were stuck in tcmalloc::PageHeap::AllocLarge(), but since I dont have debug symbols of tcmalloc linked, I could not understand what the issue was.
After nearly one week of investigation, today I tried the most simple of things: removed tcmalloc from linkage to avoid using it, just to see what happened. Well... I finally found out what the problem was, and the offending code looks very much like this:
void AllocatingFunction()
{
Object object_on_stack;
ProcessObject(&object_on_stack);
}
void ProcessObject(Object* object)
{
...
// Do Whatever
...
delete object;
}
Using libc the application still crashed but I finally saw that I was calling delete on an object which was allocated on the stack.
What I still cant figure out is why tcmalloc instead kept the application running regardless of this very risky (if not utterly wrong) object deallocation, and the double deallocation when object_on_stack goes out of scope when AllocatingFunction ends. The fact is that the offending code could be called repeatedly without any hint of the underlying abomination.
I know that memory deallocation is one of those "undefined behaviour" when not used properly, but my surprise is such a different behaviour between "standard" libc and tcmalloc.
Does anyone have some sort of explanation of insight, on why tcmalloc keeps the application running?
Thanks in advance :)
Have a Nice Day
very risky (if not utterly wrong) object deallocation
well, I disagree here, it is utterly wrong, and since you invoke UB, anything can happen.
It very much depends on what the tcmalloc code acutally does on deallocation, and how it uses the (possibly garbage) data around the stack at that location.
I have seen tcmalloc crash on such occasions too, as well as glibc going into an infinite loop. What you see is just coincidence.
Firstly, there was no double free in your case. When object_on_stack goes out of scope there is no free call, just the stack pointer is decreased (or rather increased, as stack grows down...).
Secondly, during delete TcMalloc should be able to recognize that the address from a stack does not belong to the program heap. Here is a part of free(ptr) implementation:
const PageID p = reinterpret_cast<uintptr_t>(ptr) >> kPageShift;
Span* span = NULL;
size_t cl = Static::pageheap()->GetSizeClassIfCached(p);
if (cl == 0) {
span = Static::pageheap()->GetDescriptor(p);
if (!span) {
// span can be NULL because the pointer passed in is invalid
// (not something returned by malloc or friends), or because the
// pointer was allocated with some other allocator besides
// tcmalloc. The latter can happen if tcmalloc is linked in via
// a dynamic library, but is not listed last on the link line.
// In that case, libraries after it on the link line will
// allocate with libc malloc, but free with tcmalloc's free.
(*invalid_free_fn)(ptr); // Decide how to handle the bad free request
return;
}
Call to invalid_free_fn crashes.
I have a c++ code. But it is not releasing memory properly. Tell me where I am wrong, here is my code
1 void MyClass::MyFunction(void)
2 {
3 for (int i=0; i<count; i++)
4 {
5 _bstr_t xml = GetXML(i);
6 // some work
7 SysFreeString(xml);
8 }
9 }
GetXML (line 5) returns me a BSTR. At this memory of program increases. But after SysFreeString (line 7) memory does not release. What I am doing wrong here?
First:
// This makes a copy.
// This is where the leak is. You are leaking the original string.
_bstr_t xml = GetXML();
// You want to use this, to attach the BSTR to the _bstr_t
_bstr_t xml = _bstr_t(GetXML(), false);
Second, don't do this:
SysFreeString(xml);
The _bstr_t class will do that for you.
Third, BSTR will not release the memory to the OS immediately, it caches recently used strings in order to make SysAllocString faster. You shouldn't expect to see memory usage go straight down after SysFreeString.
You can control this behaviour for debugging purposes:
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q139071
Lastly, when viewing memory usage in Task Manager you need to look at the column "Commit Size" not "Working Set". Go to Menu->View->Select Columns to show the column. And also note that this really only helps over a period of time - the memory may not be released to the OS immediately, but if you have no leaks, it shouln't go up forever, over a course of hours.
I suppose you should use :
xml.Attach(GetXML(i));
operator= looks like it is actually assigning new value - which means copying it. That value returned by GetXML stays unfreed.
also there should be no need for SysFreeString(xml);
Task Manager only provides the amount of memory allocated to the process. When C++ releases memory ( C's free) it does not necessarily return the memory to the Operating system so Task Manager will not necessarily show memory going doem until the process ends.
What Task Manager can show is if you keep allocating memory and not releasing it then the memory size of the process will keep increasing, if this happens you probably hava a memory leak.
When programming you need to use memory profilers to see if you are releasing memory. In Windows I used Rational's Purify to give me this information but it costs a lot. MS C runtime can be used to track memory. MSDN provides an overview here, read and follow th links.
As to your code and as per other comments and answers one of the points of using _bstr_t class is to do the memory and other resource management for you so you should not call SysFreeString
The destructor of _bstr_t will call SysFreeString(xml), so you don't need to call SysFreeString(xml) again. An additional memory free will result in crash.
This question already has answers here:
Closed 13 years ago.
Possible Duplicates:
How can I get the size of an array from a pointer in C?
Is there any way to determine the size of a C++ array programmatically? And if not, why?
I get a pointer to a chunk of allocated memory out of a C style function.
Now, it would be really interesting for debugging purposes to know how
big the allocated memory block that this pointer points is.
Is there anything more elegant than provoking an exception by blindly running over its boundaries?
Thanks in advance,
Andreas
EDIT:
I use VC++2005 on Windows, and GCC 4.3 on Linux
EDIT2:
I have _msize under VC++2005
Unfortunately it results in an exception in debug mode....
EDIT3:
Well. I have tried the way I described above with the exception, and it works.
At least while I am debugging and ensuring that immediately after the call
to the library exits I run over the buffer boundaries. Works like a charm.
It just isn't elegant and in no way usable in production code.
It's not standard but if your library has a msize() function that will give you the size.
A common solution is to wrap malloc with your own function that logs each request along with the size and resulting memory range, in the release build you can switch back to the 'real' malloc.
If you don't mind sleazy violence for the sake of debugging, you can #define macros to hook calls to malloc and free and pad the first 4 bytes with the size.
To the tune of
void *malloc_hook(size_t size) {
size += sizeof (size_t);
void *ptr = malloc(size);
*(size_t *) ptr = size;
return ((size_t *) ptr) + 1;
}
void free_hook (void *ptr) {
ptr = (void *) (((size_t *) ptr) - 1);
free(ptr);
}
size_t report_size(ptr) {
return * (((size_t *) ptr) - 1);
}
then
#define malloc(x) malloc_hook(x)
and so on
The C runtime library does not provide such a function. Furthermore, deliberately provoking an exception will not tell you how big the block is either.
Usually the way this problem is solved in C is to maintain a separate variable which keeps track of the size of the allocated block. Of course, this is sometimes inconvenient but there's generally no other way to know.
Your C runtime library may provide some heap debug functions that can query allocated blocks (after all, free() needs to know how big the block is), but any of this sort of thing will be nonportable.
With gcc and the GNU linker, you can easily wrap malloc
#include <stdlib.h>
#include <stdio.h>
void* __real_malloc(size_t sz);
void* __wrap_malloc(size_t sz)
{
void *ptr;
ptr = __real_malloc(sz);
fprintf(stderr, "malloc of size %d yields pointer %p\n", sz, ptr);
/* if you wish to save the pointer and the size to a data structure,
then remember to add wrap code for calloc, realloc and free */
return ptr;
}
int main()
{
char *x;
x = malloc(103);
return 0;
}
and compile with
gcc a.c -o a -Wall -Werror -Wl,--wrap=malloc
(Of course, this will also work with c++ code compiled with g++, and with the new operator (through it's mangled name) if you wish.)
In effect, the statically/dynamically loaded library will also use your __wrap_malloc.
No, and you can't rely on an exception when overrunning its boundaries, unless it's in your implementation's documentation. It's part of the stuff you really don't need to know about to write programs. Dig into your compiler's documentation or source code if you really want to know.
There is no standard C function to do this. Depending on your platform, there may be a non-portable method - what OS and C library are you using?
Note that provoking an exception is unreliable - there may be other allocations immediately after the chunk you have, and so you might not get an exception until long after you exceed the limits of your current chunk.
Memory checkers like Valgrind's memcheck and Google's TCMalloc (the heap checker part) keep track of this sort of thing.
You can use TCMalloc to dump a heap profile that shows where things got allocated, or you can just have it check to make sure your heap is the same at two points in program execution using SameHeap().
Partial solution: on Windows you can use the PageHeap to catch a memory access outside the allocated block.
PageHeap is an alternate memory manager present in the Windows kernel (in the NT varieties but nobody should be using any other version nowadays). It takes every allocation in a process and returns a memory block that has its end aligned with the end of a memory page, then it makes the following page unaccessible (no read, no write access). If the program tries to read or write past the end of the block, you'll get an access violation you can catch with your favorite debugger.
How to get it: Download and install the package Debugging Tools for Windows from Microsoft: http://www.microsoft.com/whdc/devtools/debugging/default.mspx
then launch the GFlags utility, go to the 3rd tab and enter the name of your executable, then Hit the key. Check the PageHeap checkbox, click OK and you're good to go.
The last thing: when you're done with debugging, don't ever forget to launch GFlags again, and disable PageHeap for the application. GFlags enters this setting into the Registry (under HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\), so it is persistent, even across reboots.
Also, be aware that using PageHeap can increase the memory needs of your application tremendously.
The way to do what you want is to BE the allocator. If you filter all requests, and then record them for debugging purposes, then you can find out what you want when the memory is free'd.
Additionally, you can check at the end of the program to see if all allocated blocks were freed, and if not, list them. An ambitious library of this sort could even take FUNCTION and LINE parameters via a macro to let you know exactly where you are leaking memory.
Finally, Microsoft's MSVCRT provides a a debuggable heap that has many useful tools that you can use in your debug version to find memory problems: http://msdn.microsoft.com/en-us/library/bebs9zyz.aspx
On Linux, you can use valgrind to find many errors. http://valgrind.org/
In my application I am creating an object pretty much like this :
connect() {
mVHTGlove = new vhtGlove(params);
}
and once I am about to close application I call this one :
disconnect() {
if (mVHTGlove)
delete mVHTGlove;
}
This call always triggers a breakpoint with the following message :
Windows has triggered a breakpoint in
DesignerDynD.exe.
This may be due to a corruption of the
heap, which indicates a bug in
DesignerDynD.exe or any of the DLLs it
has loaded.
This may also be due to the user
pressing F12 while DesignerDynD.exe
has focus.
The output window may have more
diagnostic information.
I cannot modify the vhtGlove class to fix the corruption of the stack as it is an external library provided only in the form of header files, lib files and dlls.
Is there any way to use this class in a clean way ?
**** EDIT ::: I tried to strip things down to a bare minimum, however I get the same results... here you have the ENTIRE code.
#include "vhandtk/vhtCyberGlove.h"
#include "vhandtk/vhtIOConn.h"
#include "vhandtk/vhtBaseException.h"
using namespace std;
int main(int argc, char* argv[])
{
vhtCyberGlove* testGlove = NULL;
vhtIOConn gloveAddress("cyberglove", "localhost", "12345", "com1", "115200");
try
{
testGlove = new vhtCyberGlove(&gloveAddress,false);
if (testGlove->connect())
cout << "Glove connected successfully" << endl;
else
{
throw vhtBaseException("testGlove()->connect() returned false.");
}
if (testGlove->disconnect())
{
cout << "Glove disconnected successfully" << endl;
}
else
{
throw vhtBaseException("testGlove()->disconnect() returned false.");
}
}
catch (vhtBaseException *e)
{
cout << "Error with gloves: " << e << endl;
system("pause");
exit(0);
}
delete testGlove;
return 0;
}
Still crashes on deletion of the glove.
EDIT #2 :: If I just allocate and delete an instance of vhtCyberGlove it also crashes.
int main(int argc, char* argv[])
{
vhtCyberGlove* testGlove = NULL;
vhtIOConn gloveAddress("cyberglove", "localhost", "12345", "com1", "115200");
testGlove = new vhtCyberGlove(&gloveAddress,false);
delete testGlove; //<<crash!
return 0;
}
Any ideas?
thanks!
JC
One possiblity is that mVHTGlove isn't being initialized to 0. If disconnect was then called without a connect ever being called, then you'd be attempting to deallocate a garbage pointer. Boom.
Another possibility is that you are actually corrupting the stack a bit before that point, but that is where the corruption actually causes the crash. A good way to check that would be to comment out as much code as you can and still get the program to run, then see if you still get the corruption. If you don't, slowly bring back in bits of code until you see it come back.
Some further thoughts (after your edits).
You might check and see if the API doesn't have its own calls for memory management, rather than expecting you to "new" and "delete" objects manually. The reason I say this is that I've seen some DLLs have issues that looked a lot like this when some memory was managed inside the DLL and some outside.
The heap corruption error is reported when the vhtGlove is deleted. However, it may just as well be your own code that causes the corruption. This often happens as a result of overwriting a buffer allocated on the heap, perhaps from a call to malloc. Or you are perhaps deleting the same object twice. You can avoid this by using a smart pointer like std::auto_ptr to store the pointer to the object.
One thing you might try to track down the source of the corruption is to look at the memory location pointed to by mVHTGlove using Visual Sudio's "Memory" window when the heap corruption is detected. See if you see anything in that memory that looks obviously like something that overran a buffer. For example, if you see a string used elsewhere in the program, then go review the code that manipulates that string -- it might be overrunning its buffer.
Given vhtCyberGlove's implementation is on another DLL, I would look for heaps mismatch. In VS, for example, this would happen if the DLL is linked to the Release CRT, while your EXE is linked to the Debug CRT. When this is the case, each module uses a different heap, and as soon as you try to free memory using the wrong heap, you'll crash.
In your case, it is possible that vhtCyberGlove gets some stuff that is allocated on the other DLL, but when you delete the vhtCyberGlove instance the stuff is being deleted directly, namely referring to your heap rather than the DLL's. And when trying to free a pointer that points to another heap, your effectively corrupting yours.
If this is indeed the case, without having more details I can offer two fixes:
Make sure your EXE uses the same heap as the DLL. Will probably lock you in Release mode, so it's not the best way to go
Get the provider of vhtCyberGlove to manage its memory usage properly...
You are passing the address of a local vhtIOConn to the constructor. Is it possible that the object is assuming ownership of this pointer and trying to delete it in the destructor?