I am learning c++ and in one lesson was a code which is about exception handling. The code is not mine, it is just the example for "try and catch". So this question is not about the code quality
My question to this code is actually: is the output and calculation of the memory size correct?
When I allocate a block of memory with new double(1000), isn't the size then 8000 bytes ?
The cerr output only counts as 1kB instead of 8kB. Am I wrong?
I got the size of 1 double with sizeof(double) to confirm it is 8 bytes.
#include <iostream>
#include <cstdlib>
#include <new>
using namespace ::std;
int main()
{
int i = 0;
double *q;
try
{
while (1)
{
q = new double[1000];
i++;
}
}
catch (bad_alloc &ex)
{
cerr << "The memory is used up. " << i
<< " Kilobyte were available." << endl;
exit(1);
}
}
To summarize what #Peter said in his comment: Your i variable is counting the number of allocations, not the total amount of memory allocated.
Note, however, that even if you "fix" this, what you'll get is not the amount of "available memory", nor even the amount of available memory rounded down to a multiple of 8000. This is because "available memory" is not a very well-defined concept. The operating system may be willing to let you allocate a zillion bytes; but it might not actually do anything visible to other processes until you start writing into that memory. And even if you do write to it - it could swap unused memory pages to the hard disk / SSD, to make room for the pages you're working on.
If you wanted to check what the maximum amount of memory you can allocate using new, you might consider using a binary-search-like procedure to obtain the size; I won't spell it out in case that's your homework. (And of course, this too will not be accurate since other processes' memory use fluctuates.)
Also consider reading: How to get available memory C++/g++?
Finally, some nitpicks:
You're using inconsistent indentation. That's confusing.
i is not such a good name for a variable. num_allocations would fit better. When you use a more meaningful name you also commit to its semantics, which makes it more difficult to get them mixed up.
Try to avoid "magic numbers" like 1000. Define constants using enum or constexpr. For example: enum { Kilo = 1000 };.
There doesn't seem to be a good reason to use double in such a program - which has nothing to do with floating-point arithmetic.
You are absolutely correct. It should be:
cerr << "The memory is used up. " << sizeof(double) * i
<< " Kilobyte were available." << endl;
Related
I've been lately experimenting with dynamically allocated arrays. I got to conclusion that they have to store their own size in order to free the memory.
So I dug a little in memory with pointers and found that 6*4 bytes directly before and 1*4 bytes directly after array don't change upon recompilation (aren't random garbage).
I represented these as unsigned int types and printed them out in win console:
Here's what I got:
(array's content is between fdfdfdfd uints in representation)
So I figured out that third unsigned int directly before the array's first element is the size of allocated memory in bytes.
However I cannot find any information about rest of them.
Q: Does anyone know what the memory surrounding array's content means and care to share?
The code used in program:
#include <iostream>
void show(unsigned long val[], int n)
{
using namespace std;
cout << "Array length: " << n <<endl;
cout << "hex: ";
for (int i = -6; i < n + 1; i++)
{
cout << hex << (*(val + i)) << "|";
}
cout << endl << "dec: ";
for (int i = -6; i < n + 1; i++)
{
cout << dec << (*(val + i)) << "|";
}
cout << endl;
}
int main()
{
using namespace std;
unsigned long *a = new unsigned long[15]{ 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14 };
unsigned long *b = new unsigned long[15]{ 0 };
unsigned long *c = new unsigned long[17]{ 0 };
show(a, 15);
cout << endl;
show(b, 15);
cout << endl;
show(c, 17);
cout << endl;
cout << endl;
system("PAUSE");
delete[] a;
delete[] b;
delete[] c;
}
It typically means that you carried out your experiments using a debugging configuration of the project and debugging version of the standard library. That version of the library uses some pre-defined bit-patterns to mark the boundaries of each allocated memory block ("no man's land" areas). Later, it checks if these bit-patterns survived intact (e.g. at the moment of delete[]). If they did not, it implies that someone wrote beyond the boundaries of the memory block. Debug version of the library will issue a diagnostic message about the problem.
If you compile your test program in release (optimized) configuration with release (optimized) version of the standard library, these "no man's land" areas will not be created, these bit-patterns will disappear from memory and the associated memory checks will disappear from the code.
Note also the the memory layout you observed is typically specific for arrays of objects with no destructors or with trivial destructors (which is basically the same thing). In your case you were working with plain unsigned long.
Once you start allocating arrays of objects with non-trivial destructors, you will observe that it is not just the size of memory block (in bytes) that's stored by the implementation, but the exact size of the array (in elements) is typically stored there as well.
"I got to conclusion that they have to store their own size in order to free the memory." No they don't.
Array does not free it's memory. You never get an array from new/malloc. You get a pointer to memory under which you can store an array, but if you forget size you have requested you cannot get it back. The standard library often does depend on OS under the hood as well.
And even OS does not have to remember it. There are implementations with very simple memory management which basically returns you current pointer to the free memory, and move the pointer by the requested size. free does nothing and freed memory is forgotten.
Bottom line, memory management is implementation defined, and outside of what you get nothing is guaranteed. Compiler or OS can mess with it, so you need to look documentation specific for the environment.
Bit patterns that you talk about, are often used as safe guards, or used for debugging. E.g: When and why will an OS initialise memory to 0xCD, 0xDD, etc. on malloc/free/new/delete?
Having this simple code in C++:
#include <iostream>
#include <memory>
#include <vector>
using namespace std;
class Empty{};
int main() {
array<unique_ptr<Empty>, 1024> empties;
for(size_t i =0; i < 1024; i++){
empties[i] = make_unique<Empty>();
}
for(auto& element : empties){
cout << "ptr: " << element.get() << endl;
}
return 0;
}
when running in ideone.com or Windows we get following result:
ptr: 0x2b601f0c9ca0
ptr: 0x2b601f0c9cc0
ptr: 0x2b601f0c9ce0
ptr: 0x2b601f0c9d00
ptr: 0x2b601f0c9d20
ptr: 0x2b601f0c9d40
ptr: 0x2b601f0c9d60 ...
For me it's totally strange. what kind of allocation algorithms in OS or standard library might cause, that there happened no allocation at address, that ends with number different than 0?
The reason I did this experiment is, that given the OS uses buddy algorithm, that manages pages and allocation request will cause OS to allocate a piece of contiguous memory, then a couple of next allocations (until running out of allocated memory) should be allocated quite nearby. If that would be the case , then probably cache issue with lists would not be so significant in some cases , but I got results I was not expecting anyways.
Also second number from the right in allocated nodes is totally randomly. What may cause such a behavior?
The minimum size of a class in C++ is one byte, if I recall correctly. As there is a consistent 32 byte spacing between the class, it could be that that happens to be the sizeof the empty class you made. To determine this, try adding
std::cout << "Empty class size: " << sizeof(Empty) << std::endl;
It probably won't be 32 bytes, instead, there will probably be some consistent spacing between each object.
Note:
Does this compile for you. It doesn't for me because empties cant be implicitly initialised.
Please notice that the printed pointers is inaccurate, the OS allow you to see them as subsequent pointers when they could be allocated to an entirely different pages.
This is bit of a strange one...I've written a very short bit of code to allocate some memory for an integer, save a value to it and print out the memory address where it is saved:
#include <iostream>
using namespace std;
int main (void) {
int * b = new int;
*b = 12345;
cout << " -> *b = " << *b << endl;
cout << " -> b = " << b << endl;
return 0;
}
Lets say this returns the following:
-> *b = 123456
-> b = 0x7f9429c04bf0
So far as I know there is no reason why this value is not still chilling in memory as I didn't actively remove it - so, just for fun, I try and run the following:
#include <iostream>
using namespace std;
int main (void) {
int * b = reinterpret_cast <int*> (0x7f9429c04bf0);
cout << " -> *b = " << *b << endl;
cout << " -> b = " << b << endl;
return 0;
}
which throws a segfault - does anyone know why this isn't allowed? I mean...it clearly isn't a good idea and I have no plans to use it in practise but I am curious.
Cheers!
Jack
Each process has its own virtual memory space separate from other processes. When the process terminates, its memory is reclaimed by the operating system.
The reason it's throwing a segfault is because the OS is unhappy with your program trying to access memory that does not belong to it.
The whole idea behind having protected memory is to isolate processes so that they can't mess with each other's memory and cause nastiness to happen. And even if you could access random memory locations, you wouldn't really find anything interesting there. It's basically the same kind of stuff you get when accessing an uninitialized pointer.
Each Process runs in its own address space, the address ur passing to reinterpret_cast should be accessible in the address space of the current process, which it isn't as the second process has a different address space layout. Also, each iteration of even the first program will give u different addresses, which is the whole point of ASLR(Address Space Layout Randomization), that is, to randomize key parts of the process memory on each new instance. Having static addresses, as used to be the case pre-ASLR, would cause havoc, leading to easy exploitation of vulnerable programs.
Read More about ASLR: http://en.wikipedia.org/wiki/Address_space_layout_randomization
Virtual Memory:
http://en.wikipedia.org/wiki/Virtual_memory
Even if you ran both programs at once I would hope you got kicked for that, in stead of getting the right answer. The implication that you think I should be able to access another programs data by guessing addresses is scary, although if you want some history, there used to be a game called "core wars" which involved doing exactly that to try to make each other crash ...
I suppose the real answer is that this has "undefined behaviour" and you should be grateful it didn't just implode.
Most modern C++ platforms work with virtual memory provided by the underlying OS. Virtual memory is not a trivial physical form of storage you seem to believe it is. Virtual memory is just an imaginary conceptual storage that exists only as long as the process runs. It simulates "memory-like" behavior every time you access your process address space.
Your access to 0x7f9429c04bf0 is not access to physical memory address, it is access to the process virtual address space, which will be redirected to some physical location that you cannot predict.
And when your process ends, its virtual memory disappears forever. It was simulated anyway, fake in a sense. When you start another process, it gets its own virtual memory that has no connection to the old one whatsoever. In that new process access to 0x7f9429c04bf0 will lead to some other physical location you cannot predict (or, as in your case, crash, if 0x7f9429c04bf0 is not even valid).
To expect that your value is "still chilling in memory" would be rather naive. In fact, your value has never really been in any "memory" suitable for any kind of "chilling".
I am compiling following code VS2012 - 32BIT. I know that max_size() returns
"maximum potential size the container can reach" which is in my case: 1.073.741.823 (yeay)
So how can i know, how many object my container can really store? (I have 64gb RAM)
unsigned int = 100000000;
vector<int*> data;
std::cout<<"max cap: "<<data.max_size()<<std::endl;
for(unsigned int i = 0; i < operationUnit; i++)
data.push_back(new int());
This will end-up in a bad-alloc. However, as i am targetting x64 this problem doesn't occur, as the max-cap is much higher, but i still cannot figure the exact elements, when i would like to reduce it down to clamp user-input.
thanks!
Well, it is OS dependent ofcourse, but the results would be, similar for every one. For example when run as 32bit executable, consistenly a build with VS2012 will stop at 26,906,977 elements in a vector of int*, not posing a threat to your memory (not even by close).
Now it gets interesting when you build a 64bit version, in which case, throwing a bad_alloc happens when (almost) all your memory is drained. In that case, no C++ not any other language can protect you.
In the screenshot that follows I'm posting an example of this happening: by the time bad_alloc gets thrown, the program is in no position to catch it or do anything with it. The OS steps in and kills every process and memory is deallocated at once (see graph). In the respective 32 version the exception was caught normally and deallocation would take about 10 minutes.
Now this is a very simplistic way of seeing this, I'm sure OS gurus could supply more insights but feel free to try this at home (and burn out some memory - I can't stop thinking that I can smell something burnt after this)
the code in the screenshot
#include <iostream>
#include <vector>
#include <exception>
using namespace std;
int main()
{
vector<int*> maxV;
try
{
while (true) maxV.push_back(new int);
}
catch (bad_alloc &e)
{
cout << "I caught bad alloc at element no " << maxV.size() << "\n";
for (auto i : maxV) delete i;
}
catch (exception &e)
{
cout << "Some other exception happened at element no " << maxV.size() << "\n";
for (auto i : maxV) delete i;
}
return 0;
}
You can't. OS could totally run out of memory. You may find that using deque data structure can become larger before error than vector for huge amounts of data, as the data is not contiguous and so it is less effected by memory fragmentation, which can be significant when you end up allocating more than half your entire memory..
I have this code:
size_t count = new_data.size();
std::cout << "got the count: " << count << std::endl;
double g_x[count];
std::cout << "First array done\n";
double g_y[count];
std::cout << "Allocated array of size" << count << std::endl;
which gives me the output:
got the count: 1506538
Segmentation fault: 11
I honestly don't understand why. It work on another data set, but not on this one.
You're probably just getting a stack overflow here. Try to dynamically allocate the memory, i.e. use the heap.
double* g_x = new double[count];
...
delete[] g_x;
Even better solution would be to use std::vector<>:
#include <vector>
...
std::vector<double> g_x(count); // Creates vector with the specified size.
If you want a dynamically sized array, you either need to create it using new, or use one of the STL containers.
Take a look at some of the answers at Static array vs. dynamic array in C++
Your problem is that in C and C++ arrays are by definition defined at compile time so what you do is wrong and it's very strange that this code even compile (compiler should scream at you)
However if you need a runtime defined array you should either use std::vector or manually allocate memory (not reccomended if you don't have very specific needs)
P.S. you should check your compiler warning level and the verbosity because this code has serious flaws so it shouldn't even compile (it's also bad for you if you have a very low warning level etc because you could pick up some bad coding habits because you are using not knowingly compiler specific extensions to the language and this is going to drive you mad when changing environement)