If I compile this program like this dmd -profile=gc test.d and run it :-
import std.stdio;
void add(ref int[] data)
{
data ~= 1;
data ~= 2;
}
void main()
{
int[] a;
a.reserve(1000);
a.add();
writeln(a);
}
Then I get a profilegc.log like this :-
bytes allocated, allocations, type, function, file:line
4 1 int[] test.add test.d:5
4 1 int[] test.add test.d:6
I would expect that the reserve call would allocate GC memory and the lines 5 and 6 would not allocate any memory because memory is already reserved in the array.
So either :-
My understanding of .reserve is wrong.
The profiler is giving an incorrect result.
There is a problem with reserve
Is one or more of these true? (I want to be able to do a reserve to guarantee that allocations only happen at times I want them to and the profiler doesn't reassure me that this works...)
Related
This question already has answers here:
Segmentation fault on large array sizes
(7 answers)
Closed 3 years ago.
Program with large global array:
int ar[2000000];
int main()
{
}
Program with large local array:
int main()
{
int ar[2000000];
}
When I declare an array with large size in the main function, the program crashes with "SIGSEGV (Segmentation fault)".
However, when I declare it as global, everything works fine. Why is that?
Declaring the array globally causes the compiler to include the space for the array in the data section of the compiled binary. In this case you have increased the binary size by 8 MB (2000000 * 4 bytes per int). However, this does mean that the memory is available at all times and does not need to be allocated on the stack or heap.
EDIT: #Blue Moon rightly points out that an uninitialized array will most likely be allocated in the bss data segment and may, in fact, take up no additional disk space. An initialized array will be allocated statically.
When you declare an array that large in your program you have probably exceeded the stack size of the program (and ironically caused a stack overflow).
A better way to allocate a large array dynamically is to use a pointer and allocate the memory on the heap like this:
using namespace std;
int main() {
int *ar;
ar = malloc(2000000 * sizeof(int));
if (ar != null) {
// Do something
free(ar);
}
return 0;
}
A good tutorial on the Memory Layout of C Programs can be found here.
int a[10];
The above code will create a array of four int variable sizes & thus the programme will be able to store only 4 integers.
Now consider the following commands
int *a,*b,*c,*d;
a= (int *)malloc(sizeof(int));
b= (int *)malloc(sizeof(int));
c= (int *)malloc(sizeof(int));
d= (int *)malloc(sizeof(int));
The above part of code will create four int type pointer & will allocate them memory of int size.
I learnt that dynamic memory allocation allocates memory at rum time.
I want to know that irrespective of using array or malloc(dynamic memory allocation), the user will be getting only four int sized space to store.If we rule out that it is a pointer variable with int size memory, then what will be the use of dynamic memory allocation.In both cases , the user will get only four int spaces & to get more he will need to access the source code.So why do we use malloc or dynamic memory allocation ?
Consider
int a,*b;
cin >> a;
b= (int *)malloc(a*sizeof(int));
The user types a number a and gets a ints. The number a is not known to either to programmer or the compiler here.
As pointed out in the comments, this is still bad style in C++, use std::vector if possible. Even new is still better than malloc. But i hope the (bad) example helps to clarify the basic idea behind dynamic memory allocation.
You're right that it's all just memory. But there is a difference in usage.
In the general case, you don't necessarily know ahead of time the amount of memory you will need and then time when such memory can be safely released. malloc and its friends are written so that they can keep track of memory used this way.
But in many special cases, you happen to know ahead of time how much memory you will need and when you will stop needing it. For example, you know you need a single integer to act as a loop counter when running a simple loop and you'll be done with it once the loop has finished executing. While malloc and its friends can still work for you here, local variables are simpler, less error prone and will likely be more efficient.
int a[10];
The above line of code will allocate an array of 10 int's of automatic storage duration, if it was within a local scope.
int *a,*b,*c,*d;
The above, however, will allocate 4 pointers to int also of automatic storage duration, likewise if it was within a local scope.
a= (int *)malloc(sizeof(int));
b= (int *)malloc(sizeof(int));
c= (int *)malloc(sizeof(int));
d= (int *)malloc(sizeof(int));
And finally, the above will allocate int variable per each pointer dynamically. So, every pointer of the above will be pointing to a single int variable.
Do note that dynamically allocated memory can be freed and resized at runtime unlike static memory allocation. Memory of automatic storage duration are freed when run out of scope, but cannot be resized.
If you program in C, casting the result of malloc is unnecessary.
I suggest you to read this: Do I cast the result of malloc?
Then what your doing in your code with the 4 pointers is unnecessary; in fact you can just allocate an array of 4 int with one malloc:
int *a;
a = malloc(4 * sizeof(int));
This question already has answers here:
Segmentation fault on large array sizes
(7 answers)
Closed 3 years ago.
Program with large global array:
int ar[2000000];
int main()
{
}
Program with large local array:
int main()
{
int ar[2000000];
}
When I declare an array with large size in the main function, the program crashes with "SIGSEGV (Segmentation fault)".
However, when I declare it as global, everything works fine. Why is that?
Declaring the array globally causes the compiler to include the space for the array in the data section of the compiled binary. In this case you have increased the binary size by 8 MB (2000000 * 4 bytes per int). However, this does mean that the memory is available at all times and does not need to be allocated on the stack or heap.
EDIT: #Blue Moon rightly points out that an uninitialized array will most likely be allocated in the bss data segment and may, in fact, take up no additional disk space. An initialized array will be allocated statically.
When you declare an array that large in your program you have probably exceeded the stack size of the program (and ironically caused a stack overflow).
A better way to allocate a large array dynamically is to use a pointer and allocate the memory on the heap like this:
using namespace std;
int main() {
int *ar;
ar = malloc(2000000 * sizeof(int));
if (ar != null) {
// Do something
free(ar);
}
return 0;
}
A good tutorial on the Memory Layout of C Programs can be found here.
This question already has answers here:
In what way is ::operator new[]() different than ::operator new()?
(2 answers)
Closed 9 years ago.
What is the difference between the two allocations below:
for (I = 0; I < 10000000; i++)
P = new CMyObject;
And
P = new CMyOjbect[10000000];
Will the first allocation method cause more fragments during allocation and consume more actual memory?
One of them allocates 10000000 elements independently. The objets could in principle be scattered all over the virtual memory space. The other allocates a single array of 10000000 contiguous elements.
In the first case, you have to call delete in each instance separately (which you can't do, so you have a memory leak.) In the second case, you need to call delete [] on P to de-allocate the whole array.
Yes.
The overhead associated with each memory allocation depends on the OS and whether the code is built with or without debugging symbols.
Regardless, there is positive overhead per each allocation. Hence, the overhead of allocating N objects in one call is substantially less than allocating one object each N times, specially when N is 10000000.
Take a look following code:
#include <stdlib.h>
#include <iostream>
struct Object
{
Object() : i(0) {}
int i;
};
int N = 1000000;
void test1()
{
Object* p = new Object[N];
}
void test2()
{
for (int i = 0; i != N; ++i )
Object* p = new Object;
}
int main(int argc, char** argv)
{
int i = atoi(argv[1]);
if ( i == 1 )
{
test1();
}
else
{
test2();
}
std::cout << "Enter a number: ";
std::cin >> i;
return 0;
}
Platform: cygwin32, Compiler: g++ without debugging symbols
Memory used for test1: 4,760K
Memory used for test2: 16,492K
Platform: Windows 7, 64 bit, Compiler: Visual Studio 2008 without debugging symbols
Memory used for test1: 4,936K
Memory used for test2: 16,712K
Platform: Windows 7, 64 bit, Compiler: Visual Studio 2008 with debugging symbols
Memory used for test1: 5,016K
Memory used for test2: 48,132K
There's also the extra book keeping that has to be done to make sure that the allocated memory is deallocated. The point of this exercise was to just demonstrate the overhead costs associated with the two ways of allocating memory.
In the first case you are allocating 10000000 objects but you will have only the last one available as you overwrite the previously the allocated objects. ---> Memory leak
In the second case you allocate an array of 10000000 objects. You can delete those with
delete [] P;
Each allocation consumes a bit of time and uses (I would assume, this may be avoidable) a bit of extra memory. So method 1 is certainly going to be slower, will very probably use more memory and would probably cause more fragmentation.
I'm following a book on C++ programming, and I'm following the exercises. One exercise asks me to create a program that produces a memory leak. Will this program produce such a leak?
int main()
{
int * pInt = new int;
*pInt = 20;
pInt = new int;
*pInt =50;
return 0;
}
Considering it is a trivial example, not having a delete paired with your new is a leak. In order to prevent a leak in this case you would need the following:
int * pInt = new int;
*pInt = 20;
delete pInt ;
pInt = new int;
*pInt =50;
delete pInt ;
A decent tool to use to detect memory leaks is Valgrind. I ran the tool on your sample code, like so:
valgrind ./a.out
and this is part of the output it produced:
==14153== HEAP SUMMARY:
==14153== in use at exit: 8 bytes in 2 blocks
==14153== total heap usage: 2 allocs, 0 frees, 8 bytes allocated
==14153==
==14153== LEAK SUMMARY:
==14153== definitely lost: 8 bytes in 2 blocks
Which confirms that indeed the program does leak memory.
Yes. To avoid leaks, every time you call new, you have to have a matching call to delete. You have 2 calls to new and no calls to delete, so you have 2 leaks.
Note that when your program exits, the OS will free up all the memory you've allocated with new. So memory leaks are really only a problem for non-trivial programs.
One exercise asks me to create a program that produces a memory leak.
Will this program produce such a leak?
an utter exercise , and your code is a better answer to exercise !
Pointers and memory leaks. These are truly the items that consume most of the debugging time for developers
Memory leak
Memory leaks can be really annoying. The following list describes some scenarios that result in memory leaks.
Reassignment, I'll use an example to explain reassignment.
char *memoryArea = malloc(10);
char *newArea = malloc(10);
This assigns values to the memory locations shown in Figure 4 below.
http://www.ibm.com/developerworks/aix/library/au-toughgame/fig4.gif
Figure 4. Memory locations
memoryArea and newArea have been allocated 10 bytes each and their respective contents are shown in Figure 4. If somebody executes the statement shown below (pointer reassignment )
memoryArea = newArea;
then it will surely take you into tough times in the later stages of this module development.
In the code statement above, the developer has assigned the memoryArea pointer to the newArea pointer. As a result, the memory location to which memoryArea was pointing to earlier becomes an orphan, as shown in Figure 5 below. It cannot be freed, as there is no reference to this location. This will result in a memory leak of 10 bytes.
http://www.ibm.com/developerworks/aix/library/au-toughgame/fig5.gif
Figure 5. Memory leak
Before assigning the pointers, make sure memory locations are not becoming orphaned.
Freeing the parent block first
Suppose there is a pointer memoryArea pointing to a memory location of 10 bytes. The third byte of this memory location further points to some other dynamically allocated memory location of 10 bytes, as shown in Figure 6.
http://www.ibm.com/developerworks/aix/library/au-toughgame/fig6.gif
Figure 6. Dynamically allocated memory
free(memoryArea)
**If memoryArea is freed by making a call to free, then as a result the newArea pointer also will become invalid. The memory location to which newArea was pointing cannot be freed, as there is no pointer left pointing to that location. In other words, the memory location pointed by newArea becomes an orphan and results in memory leak.
Whenever freeing the structured element, which in turn contains the pointer to dynamically allocated memory location, first traverse to the child memory location (newArea in the example) and start freeing from there, traversing back to the parent node.
The correct implementation here will be:
free( memoryArea->newArea);
free(memoryArea);
Improper handling of return values
At time, some functions return the reference to dynamically allocated memory. It becomes the responsibility of the calling function to keep track of this memory location and handle it properly.**
char *func ( )
{
return malloc(20); // make sure to memset this location to ‘\0’…
}
void callingFunc ( )
{
func ( ); // Problem lies here
}
In the example above, the call to the func() function inside the callingFunc() function is not handling the return address of the memory location. As a result, the 20 byte block allocated by the func() function is lost and results in a memory leak.
Sharp Reference at :
http://www.ibm.com/developerworks/aix/library/au-toughgame/
Update:
your interest let me for an edit
Simple rules to avoid Memory Leaks in C
You are allocating memory for p and q:
p=new int [5];
/* ... */
q=new int;
But you are only freeing p using an invalid operator, since arrays should be deleted using delete[]. You should at some point free both p and q using:
delete[] p;
delete q;
Note that since you are making your pointers point to the other pointer's allocated buffer, you might have to check which delete operator corresponds to which new operation.
You should use delete[] on the buffer allocated with new[] and delete with the buffer allocated with new.
Rule 1: Always write “free” just after “malloc”
int *p = (int*) malloc ( sizeof(int) * n );
free (p);
Rule 2: Never, ever, work with the allocated pointer. Use a copy!
int *p_allocated = (int*) malloc ( sizeof(int) * n );
int *p_copy = p_allocated;
// do your stuff with p_copy, not with p_allocated!
// e.g.:
while (n--) { *p_copy++ = n; }
...
free (p_allocated);
Rule 3: Don’t be parsimonious. Use more memory.
Always start by allocating more memory than you need. After you finish debugging, go back and cut on memory use. If you need an array 1000 integers long, allocate 2000, and only after you make sure everything else is OK – only then go back and cut it down to 1000.
Rule 4: Always carry array length along with you
Wherever your array goes, there should go with it it’s length. A nice trick is to allocate an array sized n+1, and save n into it’s 0 place:
int *p_allocated = (int*) malloc ( sizeof(int) * (n+1) );
int *p_copy = p_allocated+1;
p_copy[-1] = n;
// do your stuff with p_copy, not with p_allocated!
free (p_allocated);
Rule 5: Be consistent. And save comments
The most important thing is to be consistent and to write down what you do. I am always amazed at how many programmers seem to think that comments are a waste of time. They are imperative. Without comments, you probably won’t remember what you did. Imagine returning to your code a year after you wrote it, and spending countless hour trying to recall what that index does. Better to spend a couple of seconds writing it down.
Also, if you are consistent, you will not fail often. Always use the same mechanism for passing arrays and pointers. Don’t change the way you do things lightly. If you decide to use my previous trick, use it everywhere, or you might find yourself referring back to a nonexistent place because you forgot what type of reference you chose.
Ref : http://mousomer.wordpress.com/2010/11/03/simple-rules-to-avoid-memory-leaks-in-c/
Yes, this produces not one, but two memory leaks: both allocated ints are leaked. Moreover, the first one is leaked irrecoverably: once you assign pInt a new int the second time, the first allocated item is gone forever.
Will this program produce suck a leak?
Yes, it will.
Yes and no. When pInt is overwritten with a new int pointer, you lose that memory that was previously allocated, however when the program returns, most modern operating systems will clean up this memory, as well as the memory lost by not deallocating pInt at the end.
So in essence, yes, something like this will result in two memory leaks.
It does, because you allocate space with the statement "new int", but do not use "delete" to free the space.