This question already has answers here:
In what way is ::operator new[]() different than ::operator new()?
(2 answers)
Closed 9 years ago.
What is the difference between the two allocations below:
for (I = 0; I < 10000000; i++)
P = new CMyObject;
And
P = new CMyOjbect[10000000];
Will the first allocation method cause more fragments during allocation and consume more actual memory?
One of them allocates 10000000 elements independently. The objets could in principle be scattered all over the virtual memory space. The other allocates a single array of 10000000 contiguous elements.
In the first case, you have to call delete in each instance separately (which you can't do, so you have a memory leak.) In the second case, you need to call delete [] on P to de-allocate the whole array.
Yes.
The overhead associated with each memory allocation depends on the OS and whether the code is built with or without debugging symbols.
Regardless, there is positive overhead per each allocation. Hence, the overhead of allocating N objects in one call is substantially less than allocating one object each N times, specially when N is 10000000.
Take a look following code:
#include <stdlib.h>
#include <iostream>
struct Object
{
Object() : i(0) {}
int i;
};
int N = 1000000;
void test1()
{
Object* p = new Object[N];
}
void test2()
{
for (int i = 0; i != N; ++i )
Object* p = new Object;
}
int main(int argc, char** argv)
{
int i = atoi(argv[1]);
if ( i == 1 )
{
test1();
}
else
{
test2();
}
std::cout << "Enter a number: ";
std::cin >> i;
return 0;
}
Platform: cygwin32, Compiler: g++ without debugging symbols
Memory used for test1: 4,760K
Memory used for test2: 16,492K
Platform: Windows 7, 64 bit, Compiler: Visual Studio 2008 without debugging symbols
Memory used for test1: 4,936K
Memory used for test2: 16,712K
Platform: Windows 7, 64 bit, Compiler: Visual Studio 2008 with debugging symbols
Memory used for test1: 5,016K
Memory used for test2: 48,132K
There's also the extra book keeping that has to be done to make sure that the allocated memory is deallocated. The point of this exercise was to just demonstrate the overhead costs associated with the two ways of allocating memory.
In the first case you are allocating 10000000 objects but you will have only the last one available as you overwrite the previously the allocated objects. ---> Memory leak
In the second case you allocate an array of 10000000 objects. You can delete those with
delete [] P;
Each allocation consumes a bit of time and uses (I would assume, this may be avoidable) a bit of extra memory. So method 1 is certainly going to be slower, will very probably use more memory and would probably cause more fragmentation.
Related
Below is my question and code:
When code run to line 26, the memory obtained by this process does not return to OS?
But, if I delete line 16, the memory will be released correctly?
I know this is not the regular way to use so many small memory blocks, but I was very curious to know the reason.
I have run this program with MALLOC_MMAP_MAX_=1000000 MALLOC_MMAP_THRESHOLD_=1024, but nothing changed.
int i = 0;
std::cout << "waitting for input, you can check current memory" << std::endl;
std::cin >> i;
char** ptr = new char *[1000000];
std::map<int, char *> tMap;
for (unsigned long i = 0; i < 1000000; i ++)
{
ptr[i] = new char[3000];
tMap.insert(make_pair(i, ptr[i])); //line 16
}
std::cout << "waitting for input, you can check current memory" << std::endl;
std::cin >> i;
for (unsigned long i = 0; i < 1000000; i ++)
{
delete []ptr[i];
}
delete []ptr;
std::cout << "waitting for input, you can check current memory" << std::endl;
std::cin >> i; //line 26
return 0;
here are more materials, And I have checked memory of tMap, less than 100M.
1、allocated memory and stop, check memory res:
holds 2.9G memory
2、deallocate memory and stop, check memory res:
holds 2.9G memory
C++ doesn't have garbage collection, so keeping an extra copy of a pointer doesn't stop the memory from being deallocated.
What happens after delete[] ptr[i] is that the map is full of dangling pointers that can no longer be used.
Another thought: What you might see as a memory leak is the fact that the tMap also allocates dynamic memory to store the inserted data. That memory will be released when the map goes out of scope, just after line 27.
1、when code run to line: 26, the memory obtained by this process does not return to OS ?
There's no guarantee any memory will be released by a C++ program to the Operating System just because it's deleted properly by the program. In many C++ runtimes, dynamically allocated memory that's deleted will still be reserved for future use by the same program, and not released to the OS. GCC/Linux is an example of an compiler/runtime-environment where larger allocations are usually done in shared memory segments that can be released to the Operating System before the program terminates, such that the OS or other programs can use them.
2、But, if I delete line: 16, the memory wile be released correctly ?
Line 16 doesn't make any difference to the later deletion/deallocation of the memory at line 22 (which may return it to the pool of dynamic memory that the application may later re-allocate, or actually release it to the OS as mentioned above). It does involve more dynamic allocations for the std::map elements itself though.
Note that the tMap destructor does not itself delete or release the memory in any way. To have memory automatically released - either by pointer-like variables or containers there-of - use smart pointers such as std::shared_ptr or std::unique_ptr (you can google them for information).
Here is the code.
First I try to malloc and free a big block memory, then I malloc many small blocks memory till it run out of memory, and I free ALL those small blocks.
After that, I try to malloc a big block memory.
#include <stdio.h>
#include <stdlib.h>
int main (int argc, char **argv)
{
static const int K = 1024;
static const int M = 1024 * K;
static const int G = 1024 * M;
static const int BIG_MALLOC_SIZE = 1 * G;
static const int SMALL_MALLOC_SIZE = 3 * K;
static const int SMALL_MALLOC_TIMES = 1 * M;
void **small_malloc = (void **)malloc(SMALL_MALLOC_TIMES * sizeof(void *));
void *big_malloc = malloc(BIG_MALLOC_SIZE);
printf("big malloc first time %s\n", (big_malloc == NULL)? "failed" : "succeeded");
free(big_malloc);
for (int i = 0; i != SMALL_MALLOC_TIMES; ++i)
{
small_malloc[i] = malloc(SMALL_MALLOC_SIZE);
if (small_malloc[i] == NULL)
{
printf("small malloc failed at %d\n", i);
break;
}
}
for (int i = 0; i != SMALL_MALLOC_TIMES && small_malloc[i] != NULL; ++i)
{
free(small_malloc[i]);
}
big_malloc = malloc(BIG_MALLOC_SIZE);
printf("big malloc second time %s\n", (big_malloc == NULL)? "failed" : "succeeded");
free(big_malloc);
return 0;
}
Here is the result:
big malloc first time succeeded
small malloc failed at 684912
big malloc second time failed
It looks like there are memory fragments.
I know memory fragmentation happens when there are many small empty space in memory but there is no big enough empty space for big size malloc.
But I've already free EVERYTHING I malloc, the memory should be empty.
Why I can't malloc big block at the second time?
I use Visual Studio 2010 on Windows 7, I build 32-bits program.
The answer, sadly, is still fragmentation.
Your initial large allocation ends up tracked by one allocation block; however when you start allocating large numbers of 3k blocks of memory your heap gets sliced into chunks.
Even when you free the memory, small pieces of the block remain allocated within the process's address space. You can use a tool like Sysinternals VMMap to see these allocations visually.
It looks like 16M blocks are used by the allocator, and once these blocks are freed up they never get returned to the free pool (i.e. the blocks remain allocated).
As a result you don't have enough contiguous memory to allocate the 1GB block the second time.
Even I know just a little about this, I found the following thread Why does malloc not work sometimes? which covers the similar topic as yours.
It contains the following links:
http://www.eskimo.com/~scs/cclass/int/sx7.html (Pointer Allocation Strategies)
http://www.gidforums.com/t-9340.html (reasons why malloc fails? )
The issue is likely that even if you free every allocation, malloc does not return all the memory to the operating system.
When your program requested the numerous smaller allocations, malloc had to increase the size of the "arena" from which it allocates memory.
There is no guarantee that if you free all the memory, the arena will shrink to the original size. It's possible that the arena is still there, and all the blocks have been put into a free list (perhaps coalesced into larger blocks).
The presence of this lingering arena in your address space may be making it impossible to satisfy the large allocation request.
This question already has answers here:
What is the difference between Static and Dynamic arrays in C++?
(13 answers)
Closed 9 years ago.
I'm a beginer. I'm confused about the difference between them. I have read a few answers and realized one of the difference is that the dynamic array can be deleted while the normal array can't. But are there any other differences? Such as their functions, sizes or whatsoever?
Well I have read such an example, and I don't see any difference if I replace the dynamic array {p= new (nothrow) int[i];} with a normal array {int p [i];}.
#include <iostream>
#include <new>
using namespace std;
int main ()
{
int i,n;
int * p;
cout << "How many numbers would you like to type? ";
cin >> i;
p= new (nothrow) int[i];
if (p == 0)
cout << "Error: memory could not be allocated";
else
{
for (n=0; n<i; n++)
{
cout << "Enter number: ";
cin >> p[n];
}
cout << "You have entered: ";
for (n=0; n<i; n++)
cout << p[n] << ", ";
delete[] p;
}
return 0;
}
But are there any other differences?
Compile and run this, see what happens:
int main(int argc, char** argv){
char buffer[1024*1024*64];
buffer[0] = 0;
return 0;
}
Explanation:
Well, normal array is placed either on stack or within code segment (if it is global variable or static local variable). At least on windows/linux PCs. The stack has limited size (although you can change it using ulimit in linux and compiler settings on windows). So your array is too big for the stack, your program will instantly crash upon entering the function with that array ("segmentation fault" on linux, "stack overflow" or "access violation" on windows (forgoet which one)). Default limit for array size is 1Megabyte on windows (x86), and 8 megabytes on linux.
You cannot determine size of a block allocated with new. int *p = new int[146]; std::cout << sizeof(p) << std::endl. will print sizeof(int*), not size of allocated memory. However, sizeof works on arrays.
Theoretically, using dynamic memory allocation, you can allocate as much memory as you want (operating system may impose limits, though, and on 32bit system maximum allocated block size will be 2..3 GB). You can also control resource usage by freeing the memory, so your program won't be eating system ram/swap file for no reason.
Dynamic arrays are not automatically freed, you have delete them manually.
That's just a brief overview.
Dynamic memory allocation provides finer resource control and removes some limitations of local variables.
Speaking of which, although you CAN use new/delete, if you want to use arrays with variable size, you should use std::vector instead. Manual memory management is prone to errors, so you should make compiler do it for you when possible. As a result, it is advised to at least study STL(Standard Template Library) , smart pointers, RAII, Rule of Three.
Dynamic is just that. You can change the size of the resulting array at runtime rather than compile time. There are compiler extensions that allow you to use static arrays (int array[CONSTANT]) like a dynamic array in that you can specify the size at runtime but those are nonstandard.
Dynamic arrays are also allocated on the heap rather than the stack. This allows you to use all of the available memory if necessary to create the array. The stack has a limited size that depends on the OS.
Allocating memory on the heap allows you to for example do this:
int* createArray(int n) {
int* arr = new int[n];
return arr;
}
int main()
{
int* myArray = createArray(10);
// this array is now valid and can be used
delete[] myArray;
}
I'm sure there are numerous other intricacies between dynamic vs static arrays but those points are the big ones.
I have the following program:
//simple array memory test.
#include <iostream>
using namespace std;
void someFunc(float*, int, int);
int main() {
int convert = 2;
float *arr = new float[17];
for(int i = 0; i < 17; i++) {
arr[i] = 1.0;
}
someFunc(arr, 17, convert);
for(int i = 0; i < 17; i++) {
cout << arr[i] << endl;
}
return 0;
}
void someFunc(float *arr, int num, int flag) {
if(flag) {
delete []arr;
}
}
When I put the following into gdb and insert a break point at float *arr ..., I step through the program and observe the following:
Printing the array arr after it has been initialized gives me 1 17 times.
Inside someFunc too, I print arr before delete to get the same print as above.
Upon going back into main, when I print arr, I get the first digit as 0 followed by 16 1.0s.
My questions:
1. Once the array has been deleted in someFunc, how am I still able to access arr without a segfault in someFunc or main?
2. The code snippet above is a test version of another piece of code that runs in a bigger program. I observe the same behaviour in both places (first digit is 0 but all others are the same. If this is some unexplained memory error, how am I observing the same thing in different areas?
3. Some explanations to fill the gaps in my understanding are most welcome.
A segfault occurs when you access a memory address that isn't mapped into the process. Calling delete [] releases memory back to the memory allocator, but usually not to the OS.
The contents of the memory after calling delete [] are an implementation detail that varies across compilers, libraries, OSes and especially debug-vs-release builds. Debug memory allocators, for instance, will often fill the memory with some tell-tale signature like 0xdeadbeef.
Dereferencing a pointer after it has been deleteed is undefined behavior, which means that anything can happen.
Once the array has been deleted, any access to it is undefined behavior.
There's no guarantee that you'll get a segment violation; in fact,
typically you won't. But there's no guarantee of what you will get; in
larger programs, modifying the contents of the array could easily result
in memory corruption elsewhere.
delete gives the memory back to the OS memory manager, but does not necessarily clears the contents in the memory(it should not, as it causes overhead for nothing). So the values are retained in the memory. And in your case, you are accessing the same memory -- so it will print what is in the memory -- it is not necessarily an undefined behaviour(depends on memory manager)
Re 1: You can't. If you want to access arr later, don't delete it.
C++ doesn't check for array boundaries. Only if you access a memory which you are not allowed to you will get segfault
I'm making an application that is going to be using many dynamically created objects (raytracing). Instead of just using [new] over and over again, I thought I'd just make a simple memory system to speed things up. Its very simple at this point, as I don't need much.
My question is: when I run this test application, using my memory manager uses the correct amount of memory. But when I run the same loop using [new], it uses 2.5 to 3 times more memory. Is there just something I'm not seeing here, or does [new] incur a huge overhead?
I am using VS 2010 on Win7. Also I'm just using the Task Manager to view the process memory usage.
template<typename CLASS_TYPE>
class MemFact
{
public:
int m_obj_size; //size of the incoming object
int m_num_objs; //number of instances
char* m_mem; //memory block
MemFact(int num) : m_num_objs(num)
{
CLASS_TYPE t;
m_obj_size = sizeof(t);
m_mem = new char[m_obj_size * m_num_objs);
}
CLASS_TYPE* getInstance(int ID)
{
if( ID >= m_num_objs) return 0;
return (CLASS_TYPE*)(m_mem + (ID * m_obj_size));
}
void release() { delete m_mem; m_mem = 0; }
};
/*---------------------------------------------------*/
class test_class
{
float a,b,c,d,e,f,g,h,i,j; //10 floats
};
/*---------------------------------------------------*/
int main()
{
int num = 10 000 000; //10 M items
// at this point we are using 400K memory
MemFact<test_class> mem_fact(num);
// now we're using 382MB memory
for(int i = 0; i < num; i++)
test_class* new_test = mem_fact.getInstance(i);
mem_fact.release();
// back down to 400K
for(int i = 0; i < num; i++)
test_class* new_test = new test_class();
// now we are up to 972MB memory
}
There is a minimum size for a memory allocation, depending on the CRT you are using. Often that's 16 bytes. Your object is 12 bytes wide (assuming x86), so you're probably wasting at least 4 bytes per allocation right there. The memory manager also has it's own structures to keep track of what memory is free and what memory is not -- that's not free. Your memory manager is probably much simplier (e.g. frees all those objects in one go) which is inherently going to be more efficient than what new does for the general case.
Also keep in mind that if you're building in debug mode, the debugging allocator will pad both sides of the returned allocation with canaries in an attempt to detect undefined behavior. That'll probably put you over the 16 byte boundary and into the next one -- probably a 32 byte allocation, at least. That'll be disabled when you build in release mode.
Boy, I sure hope that nobody wants to allocate any non-PODs from your memory manager. Or objects of dynamic size. And doesn't mind instantiating it for every type. Or creating as many as they like all at once. Or having their lifetime be longer than the MemFact.
In fact, there is a valid pattern known as an Object Pool, which is similar to yours but doesn't suck. The simple answer is that operator new is required to be ultra flexible- it's objects must live forever until delete is called- and their destructor must be called too, and they must all have completely separate, independent lifetimes. It must be able to allocate variable-size objects, and of any type, at any time. Your MemFact meets none of these requirements. The Object Pool also has less requirements, and is significantly faster than regular new because of it, but it also doesn't completely fail on all the other fronts.
You're trying to compare an almost completely rotten apple with an orange.