I have a struct with a dynamic array:
struct test{int* arr;};
After allocating space for the arr array(arr=new int[100]), using sizeof returns 4 bytes, which is the size of the struct without the array elements. Is there another built-in function like sizeof that can return the size while keeping dynamically allocated space into account? Or do I have to do this myself?
+
I need this because I want to make it easier to save/load the contents of the struct to/from a file.
There's no way to get the memory usage due to an object and the other objects it points to, because it's not a well defined concept.
Two objects might point to the same arr block. Are they both responsible for consuming the memory?
What about recursion if you have an array of structures containing pointers? What about a cycle?
Maybe arr points to the stack. Does that count as using memory?
malloc might round up the requested allocation size, or allocate internal bookkeeping structures. Do such effects count?
Some operating systems do provide a facility to retrieve the argument to malloc (or sometimes a rounded-up value, because the underlying system might genuinely have no use for the original argument), but in standard C and C++, POSIX and in general practice, you are responsible for tracking allocation sizes yourself.
Unfortunately I think you are out of luck. The size returned by sizeof is the size of the pointer which cannot lead to the correct size of what it points to (in your case a dynamic array).
I suggest you to use std::vector. It has a size() member function that returns the number of elements in use:
struct test {
std::vector<int> arr;
}
test x;
// ...
x.arr.size();
Related
I am writing a simple stack program in c++. I have dynamically allocated the array.
I was wondering if there's any way to delete(deallocate) the specific block of an array.
int *arr = NULL ;
arr = new int[some_size];
.
.
.
.
//now suppose I want to deallocate a specific block
//suppose
int i = 4;
//can I do something like
delete[i] arr;
I know this is sort of dumb question but still I would like to know.
Thank you.
It is only possible to deallocate the entire allocation - not parts of it. There is no such syntax as delete[i]. Only a pointer returned directly from new[] may be passed to delete[].
However, you can implement your own allocator that can work on a single block from the global allocator, giving smaller allocations that can be deallocated individually. This is an advanced technique: Not for beginners. It is (probably) not something that you need to do to implement any program. But in some cases it can be useful for optimisation if you know what you are doing.
how am I going to implement my own allocator
Allocate some memory M from the global allocator.
Write a function that takes the number of bytes as argument. In the function, use some data structure to keep track of which parts of M have previously been allocated. Pick a range of bytes from M which has not been allocated. Mark it as allocated in the data structure, Return a pointer to the beginning of that range.
Write another function that takes a pointer as argument. Mark the memory that was reserved for that allocation as being free.
Even better, implement a type using these functions that conforms to the Allocator concept in the standard library so that the allocator can be used as a template argument to standard containers.
Arrays a kind of data structure that can store a fixed-size sequential collection of elements of the same type.
Arrays have a very special property of continuous memory allocation which means compiler allocates consecutive blocks of memory to a array.It gives array advantages such as
random access of elements
binary search
less memory consumption(as no pointer pointing to next element is required)
So , in general it is not possible to delete a specific block from a contiguous memory. So for this specific problem Linked List was invented.
But in C++ we have std:vector these are not static array but vectors which occupies more space than a static array.
Assuming that question is modified to vectors we can use pop_back function or erase function.
1.pop_back: Simply deletes the last vector .
int main()
{
std::vector<int> numbers;
print(numbers);
numbers.push_back(5);
numbers.push_back(3);
numbers.push_back(4);
print(numbers);
numbers.pop_back();
print(numbers);
}
2.erase: Erases the specified elements from the container.
Removes the element at pos.
Removes the elements in the range [first, last).
https://www.cplusplus.com/reference/string/string/erase/
I have a struct with a flexible array member that I need to use.
struct Record
{
uint32_t length;
Data contents[];
};
I'm able to initialize this and use it by doing something like this: (it would also work with malloc or any other dynamic allocation)
vector<Data> members;
vector<uint8_t> buffer;
Record myRecord;
buffer.resize(sizeof(Record) + members.size() * sizeof(Data));
myRecord = *(reinterpret_cast<Record*>(buffer.data());
myRecord.length = static_cast<uint32_t>(members.size());
// copy members to myRecord.contents
That works just fine. But now I need to have an interface that operates on batches of Record, and I have been trying to use an std::vector for this. Then problems start appearing, and I'm guessing it's because std::vector arranges all elements contiguously on memory, and since sizeof(Record) won't take into account the size of the contents (each vector element will hold only 4 bytes, instead of 4 bytes + size_of_contents * sizeof(Data)), the vector elements are actually sharing memory and then each element starts overwriting the contents of the previous element. Does that make sense?
If this really is the problem, I was wondering if there's any way to "force" the vector to allocate a specific size for each element (instead of whatever sizeof returns for the element's type). That way I could make sure that each vector element would have enough size. If that's not possible, is there an alternative solution? Maybe a different container that would allow me to do so? Please keep in mind that I do need to use the struct as it's defined (I would love to just replace the whole thing for a vector but unfortunately that's not possible)
Your principle problem is this:
myRecord = *(reinterpret_cast<Record*>(buffer.data());
That's simply overwriting the data in a stack variable. That does not change the address of myRecord to suddenly point to buffer.data(). Which means when you later do myRecord.contents[...] = ..., you're going to be trashing the stack.
What you almost certainly intended was:
Record *myRecord = (reinterpret_cast<Record*>(buffer.data());
Then you would have a pointer to memory managed by buffer, which would have sufficient storage for the myRecord->contents array.
You cannot treat Record like a value type. As far as C++'s object model is concerned, it's not a value type. It cannot be copied or moved like most C++ types. You can only manipulate it through a pointer/reference to the specific allocation you use here.
That being said, using a vector to manage the storage for your Record* like this is really weird. It'd be better to use a unique_ptr, since resizing the allocation would be a really bad idea.
std::unique_ptr<char[]> storage = new char[sizeof(Record) + (members.size() * sizeof(Data))];
This also prevents the system from initializing the memory, since you're going to overwrite it anyway.
I was wondering if there's any way to "force" the vector to allocate a specific size for each element (instead of whatever sizeof returns for the element's type).
No. vector manages a contiguous array of elements of the same type. And in C++, all objects of the same type have the same size.
I don't have a background in C or C++, so static arrays puzzle me a little. What are they for? Why are they allocated on the stack?
I imagine there's a performance benefit. Stack allocation is faster and there's no need for garbage collection. But why does the length need to be known at compile time? Couldn't you create a fixed-size array at runtime and allocate it on the stack?
Dynamic arays or slices in D are represented by a struct that contains a pointer and a length property. Is the same true for static arrays? How are they represented?
If you pass them to a function, they are copied in their entirety (unless you use ref), what is the rationale behind that?
I realize that dynamic arrays and slices are much more imprtant in D than static arrays, that's why the documentation doesn't dwell on them very long, but I'd still like to have a bit more background. I guess the peculiarities of static arrays have to do with how stack allocation works.
static arrays hail from the C language where calls to (the slow) alloc were discouraged to the extreme because of memory leaks (when you forgot to free the allocated array), double freeing (as a result of...), dangling pointers (all dangers of manual memory management that can be avoided with GC)
this meant that constructs such as
int foo(char* inp){
char[80] buff;
strcpy(inp,buff);//don't do this this is a invite for a buffer overflow
//...
return 1;
}
were common instead of the alloc/free calls where you need to be sure everything you allocated was freed exactly once over the course of the program
technically you CAN dynamically allocate on the stack (using assembly if you want to) however this can cause some issues with the code as the length will only be known at runtime and lessen possible optimization that the compiler may apply (unrolling an iteration over it for example)
static arrays are mostly used for buffers because of the fast allocation possible on the stack
ubyte[1024] buff=void;//assigning void avoids the initializer for each element cause we are writing to it first thing anyway
ubyte[] b;
while((b=f.rawRead(buff[])).length>0){
//...
}
they can implicitly convert to a slice of the array (or explicitly with the slice operator []) so you can use them near interchangeably with normal dynamic arrays
Static arrays are value types in D2. Without static arrays, there would be no simple way to have 100 elements in a struct that are actually stored in the struct.
Static arrays carry their size as part of their type. This allows you to e.g. declare: alias ubyte[16] IPv6Address;
Unlike in C, D2 static arrays are value types through and through. This means that they are passed by value to functions, like structs. Static arrays generally behave like structs with N members as far as memory allocation and copying goes.
By the way, you can use alloca to allocate a variable amount of memory on the stack. C also has variable-length arrays.
This is very similar to this question, but the answers don't really answer this, so I thought I'd ask again:
Sometimes I interact with functions that return variable-length structures; for example, FSCTL_GET_RETRIEVAL_POINTERS in Windows returns a variably-sized RETRIEVAL_POINTERS_BUFFER structure.
Using malloc/free is discouraged in C++, and so I was wondering:
What is the "proper" way to allocate variable-length buffers in standard C++ (i.e. no Boost, etc.)?
vector<char> is type-unsafe (and doesn't guarantee anything about alignment, if I understand correctly), new doesn't work with custom-sized allocations, and I can't think of a good substitute. Any ideas?
I would use std::vector<char> buffer(n). There's really no such thing as a variably sized structure in C++, so you have to fake it; throw type safety out the window.
If you like malloc()/free(), you can use
RETRIEVAL_POINTERS_BUFFER* ptr=new char [...appropriate size...];
... do stuff ...
delete[] ptr;
Quotation from the standard regarding alignment (expr.new/10):
For arrays of char and unsigned char, the difference between the
result of the new-expression and the address returned by the
allocation function shall be an integral multiple of the strictest
fundamental alignment requirement (3.11) of any object type whose size
is no greater than the size of the array being created. [ Note:
Because allocation functions are assumed to return pointers to storage
that is appropriately aligned for objects of any type with fundamental
alignment, this constraint on array allocation overhead permits the
common idiom of allocating character arrays into which objects of
other types will later be placed. — end note ]
I don't see any reason why you can't use std::vector<char>:
{
std::vector<char> raii(memory_size);
char* memory = &raii[0];
//Now use `memory` wherever you want
//Maybe, you want to use placement new as:
A *pA = new (memory) A(/*...*/); //assume memory_size >= sizeof(A);
pA->fun();
pA->~A(); //call the destructor, once done!
}//<--- just remember, memory is deallocated here, automatically!
Alright, I understand your alignment problem. It's not that complicated. You can do this:
A *pA = new (&memory[i]) A();
//choose `i` such that `&memory[i]` is multiple of four, or whatever alignment requires
//read the comments..
You may consider using a memory pool and, in the specific case of the RETRIEVAL_POINTERS_BUFFER structure, allocate pool memory amounts in accordance with its definition:
sizeof(DWORD) + sizeof(LARGE_INTEGER)
plus
ExtentCount * sizeof(Extents)
(I am sure you are more familiar with this data structure than I am -- the above is mostly for future readers of your question).
A memory pool boils down to "allocate a bunch of memory, then allocate that memory in small pieces using your own fast allocator".
You can build your own memory pool, but it may be worth looking at Boosts memory pool, which is a pure header (no DLLs!) library. Please note that I have not used the Boost memory pool library, but you did ask about Boost so I thought I'd mention it.
std::vector<char> is just fine. Typically you can call your low-level c-function with a zero-size argument, so you know how much is needed. Then you solve your alignment problem: just allocate more than you need, and offset the start pointer:
Say you want the buffer aligned to 4 bytes, allocate needed size + 4 and add 4 - ((&my_vect[0] - reinterpret_cast<char*>(0)) & 0x3).
Then call your c-function with the requested size and the offsetted pointer.
Ok, lets start from the beginning. Ideal way to return variable-length buffer would be:
MyStruct my_func(int a) { MyStruct s; /* magic here */ return s; }
Unfortunately, this does not work since sizeof(MyStruct) is calculated on compile-time. Anything variable-length just do not fit inside a buffer whose size is calculated on compile-time. The thing to notice that this happens with every variable or type supported by c++, since they all support sizeof. C++ has just one thing that can handle runtime sizes of buffers:
MyStruct *ptr = new MyStruct[count];
So anything that is going to solve this problem is necessarily going to use the array version of new. This includes std::vector and other solutions proposed earlier. Notice that tricks like the placement new to a char array has exactly the same problem with sizeof. Variable-length buffers just needs heap and arrays. There is no way around that restriction, if you want to stay within c++. Further it requires more than one object! This is important. You cannot make variable-length object with c++. It's just impossible.
The nearest one to variable-length object that the c++ provides is "jumping from type to type". Each and every object does not need to be of same type, and you can on runtime manipulate objects of different types. But each part and each complete object still supports sizeof and their sizes are determined on compile-time. Only thing left for programmer is to choose which type you use.
So what's our solution to the problem? How do you create variable-length objects? std::string provides the answer. It needs to have more than one character inside and use the array alternative for heap allocation. But this is all handled by the stdlib and programmer do not need to care. Then you'll have a class that manipulates those std::strings. std::string can do it because it's actually 2 separate memory areas. The sizeof(std::string) does return a memory block whose size can be calculated on compile-time. But the actual variable-length data is in separate memory block allocated by the array version of new.
The array version of new has some restrictions on it's own. sizeof(a[0])==sizeof(a[1]) etc. First allocating an array, and then doing placement new for several objects of different types will go around this limitation.
Is it true that a pointer assigned to the starting address of a dynamically allocated array does not have the information of the size of the array? So we have to use another variable to store its size for later processing the array through the pointer.
But when we free the dynamically allocated array, we don't specify the size, instead we just "free ptr" or "delete [] ptr". How could free or delete know the size of the array? Can we use the same scheme to avoid storing the size of the array in another variable?
Thanks!
Yes, this is true.
delete knows the size of the memory chunk because new adds extra information to the chunk (usually before the area returned to the user), containing its size, along with other information. Note that this is all very much implementation specific and shouldn't be used by your code.
So to answer your last question: No - we can't use it - it's an implementation detail that's highly platform and compiler dependent.
For example, in the sample memory allocator demonstrated in K&R2, this is the "header" placed before each allocated chunk:
typedef long Align; /* for alignment to long boundary */
union header { /* block header */
struct {
union header *ptr; /* next block if on free list */
unsigned size; /* size of this block */
} s;
Align x; /* force alignment of blocks */
};
typedef union header Header;
size is the size of the allocated block (that's then used by free, or delete).
The funny thing is that historically it was delete [20] arr; just as it is arr = new int[20]. However practice proved that the information on size can be painlessly stored by the allocator, and since most people using it then stored it anyway, it was added to the standard.
What is more funny, and little known, is the fact that this "extended delete syntax" is in fact supported by a few C++ compilers (despite being incorrect even in face of the C++98 standard), although none require it.
int* arr = new int[20];
delete [20] arr;
The sad part about this all however is, that there's no standard-conforming way to retrieve that passed size for your own use :-/
It is true that the array does not contain the size of the array, you have to store that information for later. When deleting an array through delete or free it is the pointer to the allocated memory you pass. The memory manager used (either by the system or your own custom from overriding new and delete) knows the memory area that is freed, and keeps track of it. Hope it makes sense.
Yes, it's true. This is part of why you should rarely try to deal with this directly, and use a standard container instead. About the only time it makes sense to deal with it is if you decide to implement a container yourself (in which case you'll normally track the size information in your container's implementation).