I'm using the Windows credentials store like this:
PCREDENTIAL cred = nullptr;
if (CredRead(entryName, 1, 0, &cred) != TRUE || !cred)
return -1;
// ... code which handles cred.UserName and cred.CredentialBlob
CredFree(cred);
As you can see I free the buffer as required. However, I see that the LPBYTE pointer CredentialBlob is still valid an still contains the password in memory. Do I have to SecureZeroMemory it manually and who owns the buffer? I found no other source code which does that...
I haven't found anything, https://msdn.microsoft.com/library/aa919793.aspx contains just the following generic statement:
Clear credential data from memory after use
Do not leave credentials in memory after use. Clear all credential data from temporary storage after use by calling SecureZeroMemory.
You own the buffer. The documentation states:
Any pointers contained within the buffer are pointers to locations within this single allocated block.
In an ideal world, CredFree would zero the entire block before freeing it, and it might be worth submitting a suggestion to Microsoft to this effect, but as things stand, your best bet is probably to do the following before calling CredFree:
SecureZeroMemory (cred->CredentialBlob, cred->CredentialBlobSize);
However, I see that the LPBYTE pointer CredentialBlob is still valid
How do you determine this? Most likely you are committing UB by reading dead memory.
an still contains the password in memory
This is more concerning but the text you quoted tells you what to do about it.
Related
if I allocate some memory using malloc() is there a way to mark it readonly. So memcpy() fails if someone attempt to write to it?
This is connected to a faulty api design where users are miss-using a const pointer returned by a method GetValue() which is part of large memory structure. Since we want to avoid copying of large chunk of memory we return live pointer within a structured memory which is of a specific format. Now problem is that some user find hack to get there stuff working by writing to this memory directly and avoid SetValue() call that does allocation and properly handing in memory binary format that we have developed. Although there hack sometime work but sometime it causes memory access violation due to incorrect interpretation of control flags which has been overridden by user.
Educating user is one task but let say for now we want there code to fail.
I am just wondering if we can simply protect against this case.
For analogy assume someone get a blob column from sqlite statement and then write back to it. Although in case of sqlite it will not make sense but this somewhat happing in our case.
On most hardware architectures you can only change protection attributes on entire memory pages; you can't mark a fragment of a page read-only.
The relevant APIs are:
mprotect() on Unix;
VirtualProtect() on Windows.
You'll need to ensure that the memory page doesn't contain anything that you don't want to make read-only. To do this, you'll either have to overallocate with malloc(), or use a different allocation API, such as mmap(), posix_memalign() or VirtualAlloc().
Depends on the platform. On Linux, you could use mprotect() (http://linux.die.net/man/2/mprotect).
On Windows you might try VirtualProtect() (http://msdn.microsoft.com/en-us/library/windows/desktop/aa366898(v=vs.85).aspx). I've never used it though.
Edit:
This is not a duplicate of NPE's answer. NPE originally had a different answer; it was edited later and mprotect() and VirtualProtect() were added.
a faulty api design where users are miss-using a const pointer returned by a method GetValue() which is part of large memory structure. Since we want to avoid copying of large chunk of memory we return live pointer within a structured memory which is of a specific format
That is not clearly a faulty API design. An API is a contract: you promise your class will behave in a particular way, clients of the class promise to use the API in the proper manner. Dirty tricks like const_cast are improper (and in some, but not all cases, have undefined behaviour).
It would be faulty API design if using const_cast lead to a security issue. In that case you must copy the chunk of memory, or redesign the API. This is the norm in Java, which does not have the equivalent of const (despite const being a reserved word in Java).
Obsfucate the pointer. i.e. return to the client the pointer plus an offset, now they can't use the pointer directly.
whenever the pointer is passed to your code via the official API, subtract the offset and use the pointer as usual.
I have some code I wrote a few years ago. It has been working fine, but after a recent rebuild with some new, unrelated code elsewhere, it is no longer working. This is the code:
//myobject.h
...
inline CMapStringToOb* GetMap(void) {return (m_lpcMap);};
...
The above is accessed from the main app like so:
//otherclass.cpp
...
CMapStringToOb* lpcMap = static_cast<CMyObject*>(m_lpcBaseClass)->GetMap();
...
Like I said, this WAS working for a long time, but it's just decided to start failing as of our most recent build. I have debugged into this, and I am able to see that, in the code where the pointer is set, it is correctly setting the memory address to an actual value. I have even been able to step into the set function, write down the memory address, then move to this function, let it get 0xfdfdfdfd, and then manually get the memory address in the debugger. This causes the code to work. Now, from what I've read, 0xfdfdfdfd means guarding bytes or "no man's land", but I don't really understand what the implications of that are. Supposedly it also means an off by one error, but I don't understand how that could happen, if the code was working before.
I'm assuming from the Hungarian notation that you're using Visual Studio. Since you do know the address that holds the map pointer, start your program in the debugger and set a data breakpoint when that map pointer changes (the memory holding the map pointer, not the map pointed to). Then you'll find out exactly when it's getting overwritten.
0xfdfdfdfd typically implies that you have accessed memory that you weren't supposed to.
There is a good chance the memory was allocated and subsequently freed. So you're using freed memory.
static_cast can modify a pointer and you have an explicit cast to CMyObject and an implicit cast to CMapStringToOb. Check the validity of the pointer directly returned from GetMap().
Scenarios where "magic" happens almost always come back to memory corruption. I suspect that somewhere else in your code you've modified memory incorrectly, and it's resulting in this peculiar behavior. Try testing some different ways of entering this part of the code. Is the behavior consistent?
This could also be caused by an incorrectly built binary. Try cleaning and rebuilding your project.
This MSND article says SecureZeroMemory() is good for cases when sensitive data stored in memory for a while should be for sure overwritten as soon as possible when no longer needed. Answers to this SO question explain why this can make a difference.
Now is there any sence in using SecureZeroMemory() for initializing just every memory block? For example in one project I see code like the following:
ICONINFO ii;
::SecureZeroMemory(&ii, sizeof(ICONINFO));
if (::GetIconInfo(hIcon, &ii))
{
//do stuff, then
//release bitmaps
if(ii.hbmMask)
::DeleteObject(ii.hbmMask);
if(ii.hbmColor)
::DeleteObject(ii.hbmColor);
}
why use SecureZeroMemory() here instead of ZeroMemory(), memset() or value initialization? I mean if the compiler decides initialization is unnecessary and wants to optimize it out - why would I enforce it? Is there any reason to use SecureZeroMemory() here?
SecureZeroMemory is never optimized-away by a compiler. That is important if you need to worry about the contents of your memory to be cleaned, say if it contains very sensitive user info, e.g. banking software, passwords, etc. Obviously if there's no need for you to worry about such things, you can use any other way of cleaning memory buffers or not cleaning at all if it's not necessary.
It makes no sense to use SecureZeroMemory to initialize an icon info structure. It can only overwrite bytes on the stack frame that should have been securely erased elsewhere. That horse already escaped the barn. It doesn't even make sense to initialize it at all, the return value of GetIconInfo() tells you that it got initialized.
SecureZeroMemory() only makes sense after memory was filled with secure data.
Many Win32 API functions have parameters specified to be "out". For example, GetIconInfo() description says about the second parameter that The function fills in the structure's members.
This implies that the function doesn't ever read the original values stored in the "out" parameter - only changes them - and therefore the caller is free to skip initialization.
Yet in one project I see the following:
ICONINFO ii;
::SecureZeroMemory(&ii, sizeof(ICONINFO));
if (::GetIconInfo(hIcon, &ii))
{
//do stuff, then
//release bitmaps
if(ii.hbmMask)
::DeleteObject(ii.hbmMask);
if(ii.hbmColor)
::DeleteObject(ii.hbmColor);
}
Is there any sense in that SecureZeroMemory() call? What could happen without it?
Well, in general I think initialisation is not needed, but good practice if you don't know exactly what the called function does with the values in the output variable.
In this specific case, the ICONINFO structure has two HBITMAP members which are essentially pointers to bitmaps. In the general case I'd say that if you are passing pointers to a function then you have to be certain that:
You pass in pointers that point to nothing and the
function you call creates the thing
pointed to for you and makes sure
your pointer points to it. (and probably leaves you to manage the newly allocated stuff) or
You pass in a pointer that points to
something (i.e. you allocated something for it) and the
function uses what you allocated.
The GetIconInfo() function fits the first case. So for clarity and perhaps even security it looks like a good idea to me to ensure the HBITMAP members of the ICONINFO structure are actually zero, rather than a random value that can lead to all kinds of nastiness further down the road.
So my verdict in this case would also be: not necessary but good practice.
Nothing. I mean if one is pretty sure that whatever is written there before call is discarded then there is no reason for doing that. But we don't know how internally the API will
function unless we developed the API, then it would be a good idea to initialize it.
This implies that the function doesn't ever read the original values stored in the "out"
parameter - only changes them - and therefore the caller is free to skip initialization.
Perhaps it isn't about the function reading the fields. Maybe its for detecting fields unfilled by the function? I don't know if this is necessary in this case, just pointing out that it might not be about reading.
Does anybody know a "technique" to discover memory leaks caused by smart pointers? I am currently working on a large project written in C++ that heavily uses smart pointers with reference counting. Obviously we have some memory leaks caused by smart pointers, that are still referenced somewhere in the code, so that their memory does not get free'd. It's very hard to find the line of code with the "needless" reference, that causes the corresponding object not to be free'd (although it's not of use any longer).
I found some advice in the web, that proposed to collect call stacks of the increment/decrement operations of the reference counter. This gives me a good hint, which piece of code has caused the reference counter to get increased or decreased.
But what I need is some kind of algorithm that groups the corresponding "increase/decrease call stacks" together. After removing these pairs of call stacks, I hopefully have (at least) one "increase call stack" left over, that shows me the piece of code with the "needless" reference, that caused the corresponding object not to be freed. Now it will be no big deal to fix the leak!
But has anybody an idea for an "algorithm" that does the grouping?
Development takes place under Windows XP.
(I hope someone understood, what I tried to explain ...)
EDIt: I am talking about leaks caused by circular references.
Note that one source of leaks with reference-counting smart pointers are pointers with circular dependancies. For example, A have a smart pointer to B, and B have a smart pointer to A. Neither A nor B will be destroyed. You will have to find, and then break the dependancies.
If possible, use boost smart pointers, and use shared_ptr for pointers which are supposed to be owners of the data, and weak_ptr for pointers not supposed to call delete.
The way I do it is simply:
- on every AddRef() record call-stack,
- matching Release() removes it.
This way at the end of the program I'm left with AddRefs() without maching Releases. No need to match pairs,
If you can reproduce the leak in a deterministic way, a simple technique I often used is to number all your smart pointers in their order of construction (use a static counter in the constructor), and report this ID together with the leak. Then run the program again, and trigger a DebugBreak() when the smart pointer with the same ID gets constructed.
You should also consider this great tool : http://www.codeproject.com/KB/applications/visualleakdetector.aspx
To detect reference cycles you need to have a graph of all reference-counted objects. Such a graph is not easy to construct, but it can be done.
Create a global set<CRefCounted*> to register living reference-counted objects. This is easier if you have common AddRef() implementation - just add this pointer to the set when object's reference count goes from 0 to 1. Similarly, in Release() remove object from the set when it's reference count goes from 1 to 0.
Next, provide some way to get the set of referenced objects from each CRefCounted*. It could be a virtual set<CRefCounted*> CRefCounted::get_children() or whatever suits you. Now you have a way to walk the graph.
Finally, implement your favorite algorithm for cycle detection in a directed graph. Start the program, create some cycles and run cycle detector. Enjoy! :)
What I do is wrap the smart pointer with a class that takes FUNCTION and LINE parameters. Increment a count for that function and line every time the constructor is called, and decrement the count every time the destructor is called. then, write a function that dumps the function/line/count information. That tells you where all of your references were created
What I have done to solve this is to override the malloc/new & free/delete operators such that they keep track in a data structure as much as possible about the operation you are performing.
For example, when overriding malloc/new, You can create a record of the caller's address, the amount of bytes requested, the assigned pointer value returned and a sequence ID so all your records can be sequenced (I do not know if you deal with threads but you need to take that into account, too).
When writing the free/delete routines, I also keep track of the caller's address and the pointer info. Then I look backwards into the list and try to match the malloc/new counterpart using the pointer as my key. If I don't find it, raise a red flag.
If you can afford it, you can embed in your data the sequence ID to be absolutely sure who and when allocation call was made. The key here is to uniquely identify each transaction pair as much as we can.
Then you will have a third routine displaying your memory allocations/deallocation history, along with the functions invoking each transaction. (this can be accomplished by parsing the symbolic map out of your linker). You will know how much memory you will have allocated at any time and who did it.
If you don't have enough resources to perform these transactions (my typical case for 8-bit microcontrollers), you can output the same information via a serial or TCP link to another machine with enough resources.
It's not a matter of finding a leak. In case of smart-pointers it'll most probably direct to some generic place like CreateObject(), which is being called thousands of time. It's a matter of determining what place in the code didnt call Release() on ref-counted object.
Since you said that you're using Windows, you may be able to take advantage of Microsoft's user-mode dump heap utility, UMDH, which comes with the Debugging Tools for Windows. UMDH makes snapshots of your application's memory usage, recording the stack used for each allocation, and lets you compare multiple snapshots to see which calls to the allocator "leaked" memory. It also translates the stack traces to symbols for you using dbghelp.dll.
There's also another Microsoft tool called "LeakDiag" that supports more memory allocators than UMDH, but it's a bit more difficult to find and doesn't seem to be actively maintained. The latest version is at least five years old, if I recall correctly.
If I were you I would take the log and write a quick script to do something like the following (mine is in Ruby):
def allocation?(line)
# determine if this line is a log line indicating allocation/deallocation
end
def unique_stack(line)
# return a string that is equal for pairs of allocation/deallocation
end
allocations = []
file = File.new "the-log.log"
file.each_line { |line|
# custom function to determine if line is an alloc/dealloc
if allocation? line
# custom function to get unique stack trace where the return value
# is the same for a alloc and dealloc
allocations[allocations.length] = unique_stack line
end
}
allocations.sort!
# go through and remove pairs of allocations that equal,
# ideally 1 will be remaining....
index = 0
while index < allocations.size - 1
if allocations[index] == allocations[index + 1]
allocations.delete_at index
else
index = index + 1
end
end
allocations.each { |line|
puts line
}
This basically goes through the log and captures each allocation/deallocation and stores a unique value for each pair, then sort it and remove pairs that match, see what's left.
Update: Sorry for all the intermediary edits (I accidentally posted before I was done)
For Windows, check out:
MFC Memory Leak Detection
I am a big fan of Google's Heapchecker -- it will not catch all leaks, but it gets most of them. (Tip: Link it into all your unittests.)
First step could be to know what class is leaking.
Once you know it, you can find who is increasing the reference:
1. put a breakpoint on the constructor of class that is wrapped by shared_ptr.
2. step in with debugger inside shared_ptr when its increasing the reference count: look at variable pn->pi_->use_count_
Take the address of that variable by evaluating expression (something like this: &this->pn->pi_.use_count_), you will get an address
3. In visual studio debugger, go to Debug->New Breakpoint->New Data Breakpoint...
Enter the address of the variable
4. Run the program. Your program will stop every time when some point in the code is increasing and decreasing the reference counter.
Then you need to check if those are matching.