I developed a small program which was working fine until I made a really minor change in some unrelated part of the code. From that point onwards the program throws an unhandled win32 exception and Microsoft Visual Studio Just in time debugger kicks in.
I am using codeblocks and my compiler is the gcc compiler. What is frustrating is that the program works fine if I choose to debug from codeblocks with the gdb. This is what does not make sense to me.
Since I can not debug with gdb to see what's wrong (because it runs fine in debugging mode), I put printfs here and there to find the root of it all. I isolated in one function but it just does not make sense.
bool FileReader::readBitmap(int fileNum)
{
char check;
int dataOffset;
int dataSize;
string fileName;
//used for quick int to string conversion
std::ostringstream stringstream;
stringstream<<fileNum;
string fileNumber = stringstream.str();
fileName = "img"+fileNumber+".bmp";
ifstream stream(fileName.c_str(),ios::in|ios::binary);
stream.read(&check,1);
//checking if it is a bitmap file
if(check != 'B')
return false;
stream.read(&check,1);
if(check != 'M')
return false;
stream.seekg(BMPBPP);
stream.read(&check,1);
//if it is not a monochrome bitmap
if(((int)check) != 1)
return false;//quit
//get the dataoffset
stream.seekg(DATAOFFSET);
stream.read(&check,1);
dataOffset = (int)check;
//get the data size in bytes
stream.seekg(DATASIZEINBYTES);
stream.read(&check,1);
dataSize = (int)check;
//if this is the first image we read
if(firstImageRead)
{
//allocate the image buffer
imgBuffer = (char*) malloc(dataSize);
//and make sure it does not get re-allocated
firstImageRead = false;
}
//get the actual bitmap data
stream.seekg(dataOffset);
stream.read(imgBuffer,dataSize);
stream.close();
return true;
}
-BIG- EDIT: Trying to find what the problem could be I moved the ifstream from the function to being a private member of the class. And the function now does EXACTLY the same only that it uses stream.open() to open the file.
Now it works with no problems. So the problem lies somehow ... in the ifstream being initialized every time inside the function, as opposed to just being used inside the function. Still ... does not make sense and this should not have occured.
I am really intrigued to find what the problem was here?
Honestly does anyone have any idea what this could be attributed to?
A few points to investigate:
Is firstImageRead initialized to true?
The rest of the code doesn't know how big imgBuffer is, so further processing is probably reading beyond the end of the buffer. How does the rest of your code determine how much data to read from imgBuffer?
If dataSize of any image is bigger than for the first one, imgBuffer will be too small.
If the character you read at position DATASIZEINBYTES happens to be negative, you will try to malloc() about 2GB.
Sidenote: Is it correct, that you read only one byte for the image size? Are the images that small?
It would help if you showed the exception and callstack of the failure.
My guess would be there is a sharing violation, you're opening the file again before it was closed. In the cases where you exit early, the file is not being closed.
Are you sure the file exists and that you are reading from the stream ok?
What about that global 'imgBuffer', make sure nothing is calling free on it between several calls to FileReader::readBitmap()
Hello and thanks for you input, but
unfortunately the file does exist and
the only time this buffer is freed is
at the end of the program. It has to
be this function since if I empty it
and just return true; , the program
runs fine ... but has no input (no
file read).
Make sure you are allocating enough space for the buffer. Remember that the malloc() is only called the first time you call this function. If dataSize is say 1000 the first time, and if you on the next file want to read 2000 bytes, dataSize will be 2000, but the buffer is only allocated for 1000 bytes.
I am just answering my own question since I found what the problem was. As most people guessed it was indeed a pointer index going out of bounds by just +1. What made it really hard to spot was that the debugger pointed me to a totally different direction.
That also explains why adding one more private member in the class 'fixed' the problem. It allocated more memory for the fileReader object, and writing out of bounds wrote on the memory occupied by the additional private member and did not cause an unhandled exception.
What do we learn from all this? Be very very careful when setting indices .... since well this is not the first time this happened to me :)
Related
As the title suggests, I'm doing some low-level volume access (as close to C code as possible) to make a disk clone program, I want to set all the free clusters to zero to use a simple compression and keep the size small.
I've been beating my head off a wall forever trying to figure out why I can't get the FSCTL_GET_VOLUME_BITMAP function working properly... so if possible please don't link me to any external reading as I've probably already been there and its either been C#, invalid links, or has no explanation I am looking for.
I want to understand the buffer itself more than i need the actual code.
The simplest way I can ask is what is the proper way to read from an array with a length of [1] in C/C++ like the one used by VOLUME_BITMAP_BUFFER?
The only way I can even assign anything to it is by recreating it with my own huge buffer and I still end up with errors, even after locking the volume in Recovery mode. I do get all needed permissions to access the raw disk just on a side note.
I know I'm likely missing some fundamental basic in C++ that would allow me to read from the memory its stored in, but I just can't figure it out without getting errors.
In case I happen to be looping through the bytes wrong which is causing my error, I added how I was doing it...although that still leaves me with the Buffer question.
I know you can call multiple times, but I have to assume its not 8 bytes at a time.
Something like this (pardon my code..I typed this on my phone so it likely has errors)...I tried adding any relevant cause of failure in case, but the buffer is the real question.
#define BYTE_MASK = 0x80;
#define BITS_PER_BYTE = 8;
void function foo() {
const int BUFFER_SIZE = 268435456;
struct {
LARGE_INTEGER StartingLcn;
LARGE_INTEGER BitmapSize;
BYTE Buffer[BUFF_SIZE];
} volBuff;
// I want to use VOLUME_BITMAP_BUFFER
/* Part of a larger loop checking for errors and more data
BYTE Mask = 1;
BOOL b = DeviceIoControl(vol, FSCTL_GET-VOLUME_BITMAP, &lcnStart, sizeof(STARTING_LCN_INPUT_BUFFER), &volBuff, sizeof(volBuff), &dwRet);
*/
for (x = 0; x < (bmpLen / BITS_PER_BYTE;) {
if ((volBuff.Buffer[x] & Mask) != 0) {
NotFree++;
} else {
FreeSpc++;
}
// I did try not dividing the size
if (Mask == BYTE_MASK) {
Mask = 1;
x++;
} else {
Mask = (Mask << 1);
}
}
return;
}
I've literally put an entire project on hold for I don't even know how long just being stubborn at this point...and I can't find any answer that actually explains the buffer, forget the rest of the process.
If someone wants to be more thorough I won't complain after all my attempts, but the buffer is driving me crazy at this point.
Any help would be greatly appreciated.
I can safely assume the one answer I was given
"...array with a length of [1]..." there is no way in Standard C++ of accessing the additional bytes. You can either: (a) pray that your compiler can do this (via an extension) or (b) write a C module (where this is well defined) that you can call from C++. - Richard Critton"
Was as correct of an answer I can expect after my extensive attempts to make this work any other way, especially since I was only able to make my own array work using standard C and not C++ directly.
I wanted to put a close on this since my computer is out of use for a bit.
If the problem continues after I dig through some examples for defragmenting in C that I FINALLY came across I'll ask a more direct question with full code to support it.
That answer was enough to remove the wall I had hit and get me thinking again. I thank you for that.
This is my first ever project that I've managed to complete so I'm a bit unsure of how to reference an executable vs a project being worked on and debugged in "debug mode" or whether there's multiple ways to do so etc, etc.
To be more specific, however, I encountered a heap corruption issue that only occurred when Visual Studio 2019 had been set to Release Mode, spit out the "exe" version of my program, and then went through its first debugging session in that form. It turns out (I'm probably wrong, but this is the last thing I changed before the issue completely disappeared) that the following code:
std::unique_ptr<std::vector<Stat>> getSelStudStats(HWND listboxcharnames) {
std::unique_ptr<std::vector<Stat>> selStats = std::make_unique<std::vector<Stat>>();
int pos = ListBox_GetCurSel(listboxcharnames);
int len = ListBox_GetTextLen(listboxcharnames, pos);
const wchar_t* buffer = new const wchar_t[++len];
ListBox_GetText(listboxcharnames, pos, buffer);
for (int i = 0; i < getSize(); i++) {
Character character = getCharacterPtr(i);
std::wstring name = character.getName();
if (name.compare(buffer) == 0) {
*selStats = character.getAllStats();
return selStats;
}
}
return selStats;
delete[] buffer;
}
was not assigning the correct size to the buffer variable through len. By adding the prefix increment operator to len, the null terminator character that would come along with the list box's text was now being accounted for; Consequently, the heap corruption error stopped occurring.
While I'm glad to have figured out the issue, I don't know why VS2019 didn't bring this issue up in Debug Mode. In attempting to debug the issue, I've learned that optimizations in Release Mode can change the structure and order of code execution.
Is there something in this block of code that would create the error I had, but only in Release Mode/executable form?
EDITED: I removed the asterisks that were originally surrounding ++len in my attempt to highlight the change that I reference making. Apologies for the confusion it, understandably, caused.
Docs explain the behavior:
When you request a memory block, the debug heap manager allocates from the base heap a slightly larger block of memory than requested and returns a pointer to your portion of that block. For example, suppose your application contains the call: malloc( 10 ). In a Release build, malloc would call the base heap allocation routine requesting an allocation of 10 bytes. In a Debug build, however, malloc would call _malloc_dbg, which would then call the base heap allocation routine requesting an allocation of 10 bytes plus approximately 36 bytes of additional memory.
So in debug you don't overrun your buffer. However, it may cause other bugs later (but unlikely for one byte overrun.)
I am using a lot of string functions like strncpy, strncat, sprintf etc. in my code. I know there are better alternatives to these, but I was handed over an old project, where these functions were used, so I have to stick with them for compatibility and consistency. My supervisor is very fussy about error checking and robustness, and insists that I check for buffer-overflow violations everytime I use these functions. This has created a lot of if-else statements in my code, which do not look pretty. My question is, is it really necessary to check for overflow everytime I call one of these functions? Even if I know that a buffer overflow can't possibly occur e.g. when storing an integer in a string using the sprintf function
sprintf(buf,"%d",someInteger);
I know that the maximum length of an unsigned integer on a 64-bit system can be 20 digits. buf on the other hand is well over 20 characters long. Should I still check for buffer overflow in this case?
I think the way to go is using exceptions. Exceptions are very useful when you must decouple the normal control-flow of a program and error-checking.
What you can do is create a wrapper for every string function in which you perform the error checking and throw an exception if a buffer overflow would occur.
Then, in your client code, you can simply call your wrappers inside a try block, and then check for exceptions and return error codes inside the catch block.
Sample code (not tested):
int sprintf_wrapper( char *buffer, int buffer_size, const char *format, ... )
{
if( /* check for buffer overflow */ )
throw my_buffer_exception;
va_list arg_ptr;
va_start( arg_ptr, format );
int ret = sprintf( buffer, , format, arg_ptr );
va_end(arg_ptr);
return ret;
}
Error foo()
{
//...
try{
sprintf_wrapper(buf1, 100, "%d", i1);
sprintf_wrapper(buf2, 100, "%d", i2);
sprintf_wrapper(buf3, 100, "%d", i3);
}
catch( my_buffer_exception& )
{
return err_code;
}
}
Maybe write a test case that you can invoke to simply test the buffer to reduce code duplication and ugliness.
You could abstract the if/else statements into a method of another class, and then pass in the buffer and length expected.
By nature, these buffers are VERY susceptible to overwrites, so be careful ANYTIME you take input in from a user/outside source. You could also try getting a string length (using strlen), or checking for the /0 end string character yourself, and comparing that to the buffer size. If you loop for the /0 character,and it's not there, you will get into an infinite loop if you don't constrain the max size of your loop by the expected buffer size, so check for this too.
Another option, is to refactor code, such that every time those methods are used, you replace them with a length safe version you write, where it calls a method with those checks already in place (but have to pass the buffer size to it). This may not be possible for some projects, as the complexity may be very hard to unit test.
Let me address your last paragraph first: You write code once, in contrast to how long it will be maintained and used. Guess how long you think your code will be in use, and then multiply that by 10-20 to figure out how long it will actually be in use. At the end of that window it's completely likely that an integer could be much bigger and overflow you buffer, so yes you must do buffer checking.
Given that you have a few options:
Use the "n" series of functions like snprintf to prevent buffer overflows and tell your users that it's undefined what will happen if the buffers overflow.
Consider it fatal and either abort() or throw an uncaught exception when a length violation occurs.
Try to notify the user there's a problem and either abort the operation or attempt to let the user modify input and retry.
The first two approaches are definitely going to be easier to implement and maintain because you don't have to worry about getting the right information back to the user in a reasonable way. In any of the cases you could most likely factored into a function as suggested in other answers.
Finally let me say since you've tagged this question C++ and not C, think long and hard about slowly migrating your code base to C++ (because your code base is C right now) and utilize the C++ facilities which then totally remove the need for these buffer checks, as it will happen automatically for you.
You can use gcc "-D_FORTIFY_SOURCE=1 D_FORTIFY_SOURCE=2" for buffer overflow detection.
https://securityblog.redhat.com/2014/03/26/fortify-and-you/
Apparently this function in SDL_Mixer keeps dying, and I'm not sure why. Does anyone have any ideas? According to visual studio, the crash is caused by Windows triggering a breakpoint somewhere in the realloc() line.
The code in question is from the SVN version of SDL_Mixer specifically, if that makes a difference.
static void add_music_decoder(const char *decoder)
{
void *ptr = realloc(music_decoders, num_decoders * sizeof (const char **));
if (ptr == NULL) {
return; /* oh well, go on without it. */
}
music_decoders = (const char **) ptr;
music_decoders[num_decoders++] = decoder;
}
I'm using Visual Studio 2008, and music_decoders and num_decoders are both correct (music_decoders contains one pointer, to the string "WAVE", and music_decoders. ptr is 0x00000000, and the best I can tell, the crash seems to be in the realloc() function. Does anyone have any idea how I could handle this crash problem? I don't mind having to do a bit of refactoring in order to make this work, if it comes down to that.
For one thing, it's not valid to allocate an array of num_decoders pointers, and then write to index num_decoders in that array. Presumably the first time this function was called, it allocated 0 bytes and wrote a pointer to the result. This could have corrupted the memory allocator's structures, resulting in a crash/breakpoint when realloc is called.
Btw, if you report the bug, note that add_chunk_decoder (in mixer.c) is broken in the same way.
I'd replace
void *ptr = realloc(music_decoders, num_decoders * sizeof (const char **));
with
void *ptr = realloc(music_decoders, (num_decoders + 1) * sizeof(*music_decoders));
Make sure that the SDL_Mixer.DLL file and your program build are using the same C Runtime settings. It's possible that the memory is allocated using one CRT, and realloc'ed using another CRT.
In the project settings, look for C/C++ -> Code Generation. The Runtime Library setting there should be the same for both.
music_decoders[num_decoders++] = decoder;
You are one off here. If num_decoders is the size of the array then the last index is num_decoders - 1. Therefore you should replace the line with:
music_decoders[num_decoders-1] = decoder;
And you may want to increment num_decoders at the beginning of the function, not at the end since you want to reallow for the new size, not for the old one.
One additional thing: you want to multiply the size with sizeof (const char *), not with double-star.
Ah, the joys of C programming. A crash in realloc (or malloc or free) can be triggered by writing past the bounds of a memory block -- and this can happen anywhere else in your program. The approach I've used in the past is some flavor of debugging malloc package. Before jumping in with a third party solution, check the docs to see if Visual Studio provides anything along these lines.
Crashes are not generally triggered by breakpoints. Are you crashing, breaking due to a breakpoint or crashing during the handling of the breakpoint?
The debug output window should have some information as to why a CRT breakpoint is being hit. For example, it might notice during the memory operations that guard bytes around the original block have been modified (due to a buffer overrun that occurred before add_music_decoder was even invoked). The CRT will check these guard pages when memory is freed and possibly when realloced too.
I have this strange call stack and I am stumped to understand why.
It seems to me that asio calls open ssl's read and then gets a negative return value (-37) .
Asio seems to then try to use it inside the memcpy function.
The function that causes this call stack is used hunderds of thousands of times without this error.
It happens only rarely, about once a week.
ulRead = (boost::asio::read(spCon->socket(), boost::asio::buffer(_requestHeader, _requestHeader.size()), boost::asio::transfer_at_least(_requestHeader.size()), error_));
Note that request header's size is exactly 3 bytes always.
Could anyone shed some light on possible reasons?
Note: I'm using boost asio 1.36
Here is the crashing call stack crash happens in memcpy because of the huge "count":
A quick look at evp_lib.c shows that it tries to pull a length from the cipher context, and in your case gets a Very Bad Value(tm). It then uses this value to copy a string (which does the memcpy). My guess is something is trashing your cipher, be it a thread safety problem, or a reading more bytes into a buffer than allowed.
Relevant source:
int EVP_CIPHER_set_asn1_iv(EVP_CIPHER_CTX *c, ASN1_TYPE *type)
{
int i=0,j;
if (type != NULL)
{
j=EVP_CIPHER_CTX_iv_length(c);
OPENSSL_assert(j <= sizeof c->iv);
i=ASN1_TYPE_set_octetstring(type,c->oiv,j);
}
return(i);
}