Too much Memory is Allocated to `char* x = new char[32]` - c++

#include <iostream>
using namespace std;
struct my_chunk
{
int size;
char* data;
};
my_chunk* make_chunk()
{
my_chunk* new_chunk = new my_chunk;
new_chunk->size = 32;
new_chunk->data = new char[32];
new_chunk->data[0] = 'h';
new_chunk->data[1] = 'e';
new_chunk->data[2] = 'l';
new_chunk->data[3] = 'l';
new_chunk->data[4] = 'o';
new_chunk->data[5] = '5';
new_chunk->data[5] = 'h';
new_chunk->data[6] = 'e';
new_chunk->data[7] = 'l';
new_chunk->data[8] = 'l';
new_chunk->data[9] = 'o';
new_chunk->data[10] = 'h';
new_chunk->data[11] = 'e';
new_chunk->data[12] = 'l';
new_chunk->data[13] = 'l';
new_chunk->data[14] = 'o';
new_chunk->data[15] = 'h';
new_chunk->data[16] = 'e';
new_chunk->data[17] = 'l';
new_chunk->data[18] = 'l';
new_chunk->data[19] = 'o';
new_chunk->data[20] = 'h';
new_chunk->data[21] = 'e';
new_chunk->data[22] = 'l';
new_chunk->data[23] = 'l';
new_chunk->data[24] = 'o';
new_chunk->data[25] = 'h';
new_chunk->data[26] = 'e';
new_chunk->data[27] = 'l';
new_chunk->data[28] = 'l';
new_chunk->data[29] = 'h';
new_chunk->data[30] = 'e';
new_chunk->data[31] = 'l';
return new_chunk;
}
void main()
{
my_chunk* same_chunk;
same_chunk = make_chunk();
std::cout << same_chunk->data;
std::cout << std::endl;
system("pause");
}
That is the simple code I compile. No matter the size I size my char* data to it adds some form of padding. It does not appear to be an alignment issue, perhaps I am wrong on that point however.
What I know is that when I size my char* data = new char[size] I can easily access beyond element [size]. The fact I can access beyond and set those elements suggests to me a massive problem has occurred.
For clarification that means in my code above, I could add a line that goes new_chunk->data[38] = 'x' without any error, crash, or anything. I have tested it, it works fine.
This isn't a huge issue, as I am given enough memory to fit my data. The only problem is that I don't understand why this is happening, and would prefer to fix it.
Also this is the output of my Program:
hellohellohellohellohellohellhel²²²²½½½½½½½½¯■¯■
Press any key to continue . . .
Edit:
This has exploded with helpful insights, perhaps I can get one more bit of help related to all this.
Why does Visual Studio 2013 show the char* beyond its length? It shows the "hellohellohellohellohellohellhel²²²²½½½½½½½½¯■¯■" which was suggesting to me that it was allocating too much memory. As a side note, the output is always the same (so far). This is when I debug, looking at the variables it shows exactly what is output.

char* need a trailing '\0' to be printed properly by std::cout. So this line std::cout << same_chunk->data; will iterate in memory till it find a zero...
That can cause a crash, printing garbage, ...
By the way there is no bound checking for pointer access in C++ so whenever you write data[X] the program try to go to adresse of data + X time the size of one data element (here char).
If you want bound access (and you want it) use either a std::string (neat for characters) or a std::vector (neat for anything).

Here's another perspective based on a lower-level view:
When you call new (or malloc), the library (libc?) will request some memory from the OS. This memory is most likely in page form (ie. 4K, 2M, etc bytes large). Depending on the algorithm that the library uses to manage dynamic memory, a couple of things can happen:
Your data[] pointer happens to be right at the back edge of this page, and you get a page fault (program crash, expected behaviour)
More likely, the library allocated you some space in the middle of a page. Since C++ doesn't do bounds checking (as answered by others), it treats this data* pointer as a pointer to a list of bytes in memory. Since the granularity of the space allocated to the heap is rather large, you can be accessing mapped pages (ie. no program crash), that have rubbish values (ie. uninitialised values).
Also, another thing to note is that when you request a block of memory 32 bytes long, nothing dictates you get one that is exactly 32 bytes long. new[] might give you a region 1024 bytes long, or 400000 bytes long. The only guarantee is that it is at least 32 bytes long. Therefore, this is another reason (although not the main reason) that your program doesn't crash.

C++ doesn't really check to make sure an array index is within the initial chunk of memory allocated for the array. When you are accessing the "extra" memory, you're basically just accessing some unrelated memory and casting it as a character. It isn't ALLOCATED as part of the array, just ACCESSIBLE as if it were. And assigning values to those random memory locations is just overwriting memory randomly... Bad idea.

C and C++ do not do bounds checking. So basically you just got lucky that you didn't get a segfault when you accessed a location past the bounds of your allocated memory.
the [38] notation basically says move to the equivalent address of data + 38 * sizeof(char *). So if this space were to be marked as poison, you woulda been out of luck.

Related

Warning C6386 - Buffer overrun while writing to 'LINES_DATA.Lines'

I know this question has been asked before, but I couldn't quite fix my code, even reading other topics.
Does anyone know why is it throwing this warning?
Warning C6386 Buffer overrun while writing to 'LINES_DATA.Lines': the writable size is 'LINES_DATA.NumLines4' bytes, but '8' bytes might be written.*
"
LINES_DATA.NumLines = line_i; //line_i = 100
LINES_DATA.Lines = new int* [LINES_DATA.NumLines];
line_i = 0;
for (rapidxml::xml_node<>* pNode = pRoot->first_node(); pNode; pNode = pNode->next_sibling())
{
LINES_DATA.Lines[line_i] = new int[COLUMNSIZE]; //COLUMNSIZE = 5
for (int pos_i = 0; pos_i < COLUMNSIZE; pos_i++)
{
LINES_DATA.Lines[line_i][pos_i] = pNode->value()[pos_i] - '0';
}
line_i++;
}
I get the warning in this line:
LINES_DATA.Lines[line_i] = new int[COLUMNSIZE];
Thank you so much
If the array (LINES_DATA.Lines) hasline_i elements then LINES_DATA.Lines[line_i] is not valid.
Arrays are zero based so LINES_DATA.Lines has elements 0 to line_i-1
It's just a Code Analysis warning. The compiler isn't smart enough to work out your program's entire runtime behaviour.
Your code does have major risks of buffer over-runs, particularly if the XML contains more than 100 elements. You should be using smart pointers and/or STL containers here.

C++ dynamic alocate, pointers

I get this error :
"crt detected that the application wrote to memory after end of heap
buffer"
this program should write every second sign...
char x = 1;
while ( x != 0) {
char *ptr(0);
ptr = new char;
fgets(ptr, 100001, stdin);
for (char i = 0; i < sizeof(ptr); i++) {
cout << *(ptr + i);
i++;
}
cout << endl;
delete ptr;
ptr = 0;
}
ptr = new char; allocates exactly one byte for the pointer. Than fgets(ptr, 100001, stdin); attempts to read up to 100001 characters into it, obviously well beyond it's allocated boundary.
The immediate solution would be to make sure you allocate the buffer with adequate size using array form of new, i.e. ptr = new char[100001];. This, in turn, will lead to changing the delete into it's array form as well: delete[] ptr
Also, keep in mind that sizeof(ptr) will give you the size of the pointer (4 or 8 bytes on most platforms). You would want to use strlen or similar to find out when the string ends.
All suggestions above should be taken in light of the learning exercise. In real life, you should not do this, and instead use std::string and modern ways of string-oriented reading. (std::getline would be the most readily available).
sizeof returns size in bytes of the object representation of type. https://en.cppreference.com/w/cpp/language/sizeof
One other point major point to note is that ptr = new char will only allocate exactly one byte from the heap. But in your code you are trying to read 100001 bytes and obviously this is beyond the accessible memory range of that pointer.
you can modify the error as ptr = new char[100001]; When you do this, there is another change that you must do in order to avoid memory leaks.
That being, delete[] ptr. Because you have allocated 100001 adjacent bytes, you have to de-allocate them all.

Overwriting data via array vulnerabilities

I am trying to demonstrate a buffer overflow via an array index (when there isn't any bounds checking). What I am trying to do is change my bool authenticated = false to true by passing in a bad value.
I am using GCC 4.8.5
arrayVulnerability(int size)
{
int array[4];
bool authenticated = false;
for (int i = 0; i < size; i++)
{
array[i] = size;
}
}
My understanding is that my memory is set up as follows:
I was hoping that by passing an int larger than 4 I would be able to overwrite that position to true but it's not working. I'm curious if I have my memory misunderstood or if I am missing something?
Edit:
I printed out the locations as suggested and got the following:
bool authenticated = 0x7ffc4741612f
array[0] = 0x7ffc47416130
array[1] = 0x7ffc47416134
array[2] = 0x7ffc47416138
array[3] = 0x7ffc4741613c
array[4] = 0x7ffc47416140
So it looks like bool authenticated is before my array and my memory layout was wrong. I'm still confused about why it is before my array however.
The most likely implementation of automatic storage, the stack, grows downwards as objects are allocated. This means that array is allocated a certain address, and then authenticated is allocated a lower address. You can do some quick experiments to verify if this is the case. Either look at the state of an object defined before array, or print the addresses of the objects.

zlib deflate and dynamic output buffer

Ok, i remade my question a bit, i do not understand how TO properly deflate the content into the DYNAMIC buffer, who needs to be constantly reallocating. I re-wrote my code a bit, and it work ONLY if no reallocating of the buffer happens, so on the small amount of data, reallocating breaks somehow the output stream.
void test_deflate_dynamic(char*str)
{
if(store == NULL) // first call to the function allocate some memory etc
{
gzip_stream.zalloc = Z_NULL;
gzip_stream.zfree = Z_NULL;
gzip_stream.opaque = Z_NULL;
result = deflateInit(&gzip_stream,9);
if(result!=Z_OK) { printf("bad\r\n"); exit(0); }
total_buf_size =deflateBound(&gzip_stream,strlen(str));
printf("d_bound_init=%d\r\n",total_buf_size);
store = realloc(store,total_buf_size); // first allocation
gzip_stream.avail_out = total_buf_size;
gzip_stream.next_out = store;
gzip_stream.avail_in = strlen(str)+1;
gzip_stream.next_in = str;
result = deflate(&gzip_stream,Z_NO_FLUSH);
}
else
{
gzip_stream.avail_in = strlen(str)+1;
gzip_stream.next_in = str;
int t_size;
printf ("avail_out=%d\r\n",gzip_stream.avail_out);
t_size = deflateBound(&gzip_stream,strlen(str));
printf("d_bound=%d\r\n",t_size);
total_buf_size += t_size;
gzip_stream.avail_out = total_buf_size;
store = realloc(store,total_buf_size);
gzip_stream.next_out = store;
result = deflate(&gzip_stream,Z_NO_FLUSH);
}
}
As you can see i'm using the function deflateBound to detect how much more data i need to allocate, so first, is it correct to use deflateBound? Second, is the pointer which modified by realloc and then re-assigned to the z_stream, still points to the beginning of the data? So basically if i'm using multiple reallocations the end data is broken. End: How do i proper detect, how much data do i need to allocate for my output deflate buffer, and is it correct to use the dynamic re-allocating buffer in the z_stream?
realloc ensures the data is copied to the new location when the new size didn't fit into the old memory position. calloc just zeros the allocated memory, and doesn't copy over the old data. Therefore you must pass bigger values to calloc. You probably only need one call to calloc when you pass it that big numbers whereas realloc can accept smaller increments. Make sense?
Cheers

array assignment access violation

I’ve read many Q&A's which seemed similar to this problem but haven’t found any answers yet:
I have to make some assignments to a dynamic byte array in the fillbyte function like this:
int Error;
result = fillbyte (&Error);
if I comment the line shown below, everything works fine. but if that line gets executed, the second time that this function is called, access violation exception will be raised however the first time the code runs properly and everything goes alright. I can’t seem to find the problem with this line of code or another way to fill the array with password bytes.
Bool fillbyte(int *Error)
{
byte BCC;
byte *Packet1 = new byte;
*Packet1 = 0x01;
*(Packet1+1) = 'P';
*(Packet1+2) = '1';
*(Packet1+3) = STX;
*(Packet1+4) = '(';
int add = sizeof(readingprops.password)*2;
for(int i=0;i<add;i++)
{
*(Packet1+(5+i)) = readingprops.password[i]; //this line raises the problem
}
*(Packet1+add+5) = ')';
*(Packet1+add+6) = ETX;
BCC = calc.CalcBCC(Packet1,add+7);
*(Packet1+add+7) = BCC;
SerialPort.Write(Packet1,add+8);
delete Packet1;
return true;
}
Any help would be appreciated
I don't see how it can ever work. You allocate one byte on the heap but treat it as multiple bytes:
byte *Packet1 = new byte;
*Packet1 = 0x01;
*(Packet1+1) = 'P'; // !!!
*(Packet1+2) = '1'; // !!!
*(Packet1+3) = STX; // !!!
*(Packet1+4) = '('; // !!!
Here you allocate just one byte
byte *Packet1 = new byte;
and then use the pointer beyond the allocated memory
*(Packet1+1) = 'P';
*(Packet1+2) = '1';
*(Packet1+3) = STX;
*(Packet1+4) = '(';
This causes undefined behaviour, sometimes it may work. So you want something like
byte Packet1 = new byte[size]
where size is appropriate for your needs (probably add + 8, since this is the amount of bytes you write to in that function). Then delete it with delete[]. You could also use stack allocation, or std::vector<byte> since this is c++.