Is it possible to incrementally increase the amount of allocated memory on a free store that a pointer points to? For example, I know that this is possible.
char* p = new char; // allocates one char to free store
char* p = new char[10]; // allocates 10 chars to free store
but what if I wanted to do something like increase the amount of memory that a pointer points to. Something like...
char input;
char*p = 0;
while(cin >> input) // store input chars into an array in the free store
char* p = new char(input);
obviously this will just make p point to the new input allocated, but hopefully you understand that the objective is to add a new char allocation to the address that p points to, and store the latest input there. Is this possible? Or am I just stuck with allocating a set number.
The C solution is to use malloc instead of new -- this makes realloc available. The C++ solution is to use std::vector and other nice containers that take care of these low-level problems and let you work at a much higher, much nicer level of abstraction!-)
You can do this using the function realloc(), though that may only work for memory allocated with malloc() rather than "new"
having said that, you probably don't want to allocate more memory a byte at a time. For efficiency's sake you should allocate in blocks substantially larger than a single byte and keep track of how much you've actually used.
realloc
You appear to be using C++. While you can use realloc, C++ makes it possible to avoid explict memory management, which is safer, easier, and likely more efficient than doing it yourself.
In your example, you want to use std::vector as a container class for chars. std::vector will automatically grow as needed.
In fact, in your case you could use a std::istreambuf_iterator and std:push_back to std::copy the input into a std::vector.
Related
I know that there is no way in C++ to obtain the size of a dynamically created array, such as:
int* a;
a = new int[n];
What I would like to know is: Why? Did people just forget this in the specification of C++, or is there a technical reason for this?
Isn't the information stored somewhere? After all, the command
delete[] a;
seems to know how much memory it has to release, so it seems to me that delete[] has some way of knowing the size of a.
It's a follow on from the fundamental rule of "don't pay for what you don't need". In your example delete[] a; doesn't need to know the size of the array, because int doesn't have a destructor. If you had written:
std::string* a;
a = new std::string[n];
...
delete [] a;
Then the delete has to call destructors (and needs to know how many to call) - in which case the new has to save that count. However, given it doesn't need to be saved on all occasions, Bjarne decided not to give access to it.
(In hindsight, I think this was a mistake ...)
Even with int of course, something has to know about the size of the allocated memory, but:
Many allocators round up the size to some convenient multiple (say 64 bytes) for alignment and convenience reasons. The allocator knows that a block is 64 bytes long - but it doesn't know whether that is because n was 1 ... or 16.
The C++ run-time library may not have access to the size of the allocated block. If for example, new and delete are using malloc and free under the hood, then the C++ library has no way to know the size of a block returned by malloc. (Usually of course, new and malloc are both part of the same library - but not always.)
One fundamental reason is that there is no difference between a pointer to the first element of a dynamically allocated array of T and a pointer to any other T.
Consider a fictitious function that returns the number of elements a pointer points to.
Let's call it "size".
Sounds really nice, right?
If it weren't for the fact that all pointers are created equal:
char* p = new char[10];
size_t ps = size(p+1); // What?
char a[10] = {0};
size_t as = size(a); // Hmm...
size_t bs = size(a + 1); // Wut?
char i = 0;
size_t is = size(&i); // OK?
You could argue that the first should be 9, the second 10, the third 9, and the last 1, but to accomplish this you need to add a "size tag" on every single object.
A char will require 128 bits of storage (because of alignment) on a 64-bit machine. This is sixteen times more than what is necessary.
(Above, the ten-character array a would require at least 168 bytes.)
This may be convenient, but it's also unacceptably expensive.
You could of course envision a version that is only well-defined if the argument really is a pointer to the first element of a dynamic allocation by the default operator new, but this isn't nearly as useful as one might think.
You are right that some part of the system will have to know something about the size. But getting that information is probably not covered by the API of memory management system (think malloc/free), and the exact size that you requested may not be known, because it may have been rounded up.
You will often find that memory managers will only allocate space in a certain multiple, 64 bytes for example.
So, you may ask for new int[4], i.e. 16 bytes, but the memory manager will allocate 64 bytes for your request. To free this memory it doesn't need to know how much memory you asked for, only that it has allocated you one block of 64 bytes.
The next question may be, can it not store the requested size? This is an added overhead which not everybody is prepared to pay for. An Arduino Uno for example only has 2k of RAM, and in that context 4 bytes for each allocation suddenly becomes significant.
If you need that functionality then you have std::vector (or equivalent), or you have higher-level languages. C/C++ was designed to enable you to work with as little overhead as you choose to make use of, this being one example.
There is a curious case of overloading the operator delete that I found in the form of:
void operator delete[](void *p, size_t size);
The parameter size seems to default to the size (in bytes) of the block of memory to which void *p points. If this is true, it is reasonable to at least hope that it has a value passed by the invocation of operator new and, therefore, would merely need to be divided by sizeof(type) to deliver the number of elements stored in the array.
As for the "why" part of your question, Martin's rule of "don't pay for what you don't need" seems the most logical.
There's no way to know how you are going to use that array.
The allocation size does not necessarily match the element number so you cannot just use the allocation size (even if it was available).
This is a deep flaw in other languages not in C++.
You achieve the functionality you desire with std::vector yet still retain raw access to arrays. Retaining that raw access is critical for any code that actually has to do some work.
Many times you will perform operations on subsets of the array and when you have extra book-keeping built into the language you have to reallocate the sub-arrays and copy the data out to manipulate them with an API that expects a managed array.
Just consider the trite case of sorting the data elements.
If you have managed arrays then you can't use recursion without copying data to create new sub-arrays to pass recursively.
Another example is an FFT which recursively manipulates the data starting with 2x2 "butterflies" and works its way back to the whole array.
To fix the managed array you now need "something else" to patch over this defect and that "something else" is called 'iterators'. (You now have managed arrays but almost never pass them to any functions because you need iterators +90% of the time.)
The size of an array allocated with new[] is not visibly stored anywhere, so you can't access it. And new[] operator doesn't return an array, just a pointer to the array's first element. If you want to know the size of a dynamic array, you must store it manually or use classes from libraries such as std::vector
we have to take input of a string such as :
1<=|S| <=10^6 (length of string)
and perform some operation , let's leave operation , I want to only know how to take input of such a long string like 10^6 ?
Can we take like this char S[1000001];
or which would be other better way?
Kindly, help
Forget about using a c-style string and use a std::string instead. A std::string can hold std::string::max_size characters. On most implementations today that should be 4294967294 characters.
The benefit with use a std::string is that it automatically grows to accommodate the size you need. Since the memory is dynamically allocated you don't have to worry about running out of stack space as well. You could run out of memory on the heap as you would need about 4 GB of RAM to hold a max size string thought.
Can we take like this char S[1000001];
You can... but if S is an automatic variable, you'll spend most of your stack-space on that one array and you'll probably run out (depending on available stack space and how much the rest of your program needs).
Large arrays like these should be allocated dynamically. Unfortunately, it's not easy to say how big arrays/objects should be allocated dynamically. It depends on a few things such as:
Amount of total stack space which depends on the platform and may be configurable at run- or linktime.
Amount of stack space needed by the rest of your program. This depends on how deep nested function calls do you have and how much memory your functions need.
I use a few kilobytes as a rule of thumb to decide if I'll need dynamic memory. Or a few dozen bytes inside a recursive function that is expected to go deep.
or which would be other better way?
std::string allocates it's buffer dynamically, provides ways to manipulate the string, makes sure that you don't forget the zero terminator and takes care of mamory management (which would be an issue if you did dynamic allocation manually). I highly recommend you use it.
it depends if the string characters are unicode or not
you could use:
char *txt = new char[1000000]
wchar_t *txt = new wchar_t[1000000];
also you can try using std::string or std::wstring like NathanOliver said
Use std::string.
Alternatively, if your data isn't being resized, you could opt to use a simple char array. Due to the size of the char array, I would suggest you allocate it with new and free it with delete. Otherwise, you may exhaust your stack size and cause a stack overflow.
I'm new to C++, trying to learn by myself (I've got Java background).
There's this concept of dynamic memory allocation that I can assign to an array (for example) using new.
In C (and also in C++) I've got malloc and realloc that are doing that. In C++ they've added the new for some reason I can't understand.
I've read a lot about the difference between a normal array that goes to the stack while the dynamic allocated array goes to the heap.
So what I understand is that by using new I'm allocating space in the heap which will not be deleted automatically when finished a function let's say, but will remain where it is until I finally, manually free it.
I couldn't find practical examples of using the dynamic memory allocation over the normal memory.
It's said that I can't allocate memory through runtime when using normal array. Well, probably I didn't understand it right because when I tried to create a normal array (without new) with a capacity given as an input by the user (like arr[input]), it worked fine.
here is what I mean:
int whatever;
cin>>whatever;
int arr2[whatever];
for (int i = 0; i < whatever; i++) {
arr2[i]=whatever;
cout<<arr2[i];
}
I didn't really understand why it's called dynamic when the only way of extending the capacity of an array is to copy it to a new, larger array.
I understood that the Vector class (which I haven't yet learned) is much better to use. But still, I can't just leave that gap of knowledge begin and I must understand why exactly it's called dynamic and why should I use it instead of a normal array.
Why should I bother freeing memory manually when I can't really extend it but only copy it to a new array?
When you know the size of an array at compile time you can declare it like this and it will live on the stack:
int arr[42];
But if you don't know the size at compile time, only at runtime, then you cannot say:
int len = get_len();
int arr[len];
In this case you must allocate the array at runtime. In this case the array will live on the heap.
int len = get_len();
int* arr = new int[len];
When you no longer need that memory you need to do a delete [] arr.
std::vector is a variable size container that allows you to allocate and reallocate memory at runtime without having to worry about explicitly allocating and freeing it.
int len = get_len();
std::vector<int> v(len); // v has len elements
v.resize(len + 10); // add 10 more elements to the vector
For static allocation, you must specify the size as a constant:
MyObj arrObject[5];
For dynamic allocation, that can be varied at run-time:
MyObj *arrObject = new MyObj[n];
The different between new and malloc is that new will call the ctor for all those objects in the array, while malloc just gives you raw memory.
if you wanna use an array and you dont know the exact size at the compile time, thats when dynamic memory allocation steps in. See the example below,
int a[3] = {1,2,3}; //<= valid in terms of syntax;
however,
int size = 3;
int a[size] = {1,2,3} //<= compile error
in order to fix this,
int* ArrayPtr = new int[size];
also, when freeing it, call delete[] ArrayPtr; instead of delete alone, coz we are talking abt freeing a BLOCK of memory at this moment.
In C (and also in C++) I've got malloc and realloc that are doing that. In C++ they've added the "new" for some reason I can't understand.
malloc and realloc take the number of bytes to allocate instead of the type you want to allocate, and also don't call any constructors (again, they only know about the size to allocate). This works fine in C (as it really has more of a size system than a type system), but with C++'s much more involved type system, it falls short. In contrast, new is type safe (it doesn't return a void* as malloc does) and constructs the object allocated for you before returning.
It's said that I can't allocate memory through runtime when using normal array. Well, probably I didn't understand it right because when I tried to create a normal array (without "new") with a capacity given as an input by the user (like arr[input]). it worked fine.
This is a compiler extension (and part of C99), it is NOT standard C++. The standard requires that a 'normal' array have a bound which is known at compile time. However, it seems your compiler decided to support variable length 'normal' arrays anyways.
I didn't really understand why it's called dynamic when the only way of extending the capacity of an array is to copy it to a new, larger array.
Its dynamic in that you don't know the size until run time (and thus can be different across different invocations). Compile time vs. run time is a distinction you don't often run across in other languages (in my experience at least), but it is crucial to understanding C++.
For instance:
int* pArray;
pArray = new array[];
instead of:
int* pArray;
pArray = new array[someNumber];
Since pointers are able to dynamically change the size of an array at run time, and the name of the pointer points to the first element of an array, shouldn't the default size be [1]? Does anyone know what's happening behind the scene?
Since pointers are able to dynamically change the size of an array at run time
This is not true. They can't change the size unless you allocate a new array with the new size.
If you want to have an array-like object that dynamically changes the size you should use the std::vector.
#include<vector>
#include<iostream>
...
std::vector<int> array;
array.push_back(1);
array.push_back(2);
array.push_back(3);
array.push_back(4);
std::cout << array.size() << std::endl; // should be 4
When you create an array with new, you are allocating a specific amount of memory for that array. You need to tell it how many items are to be stored so it can allocate enough memory.
When you "resize" the array, you are creating a new array (one with even more memory) and copying the items over before deleting the old array (or else you have a memory leak).
Quite simply, C++ arrays have no facility to change their size automatically. Therefore, when allocating an array you must specify it size.
Pointers cannot change an array. They can be made to point to different arrays at runtime, though.
However, I suggest you stay away from anything involving new until you have learned more about the language. For arrays changing their size dynamically use std::vector.
Pointers point to dynamically allocated memory. The memory is on the heap rather than the stack. It is dynamic because you can call new and delete on it, adding to it and removing from it at run time (in simple terms). The pointer has nothing to do with that - a pointer can point to anything and in this case, it just happens to point to the beginning of your dynamic memory. The resizing and management of that memory is completely your responsibility (or the responsibility of the container you may use, e.g. std::vector manages dynamic memory and acts as a dynamic array).
They cannot change the size dynamically. You can get the pointer to point to a new allocation of memory from the heap.
Behind the scenes there is memory allocated, a little chunk of silicium somewhere in your machine is now dedicated to the array you just newed.
When you want to "resize" your array, it is only possible to do so in place if the chunk of silicium has some free space around it. Most of the times, it is instead necessary to reserve another, bigger, chunk and copy the data that were in the first... and obviously relinquish the first (otherwise you have a memory leak).
This is done automatically by STL containers (like std::vector or std::deque), but manually when you yourself call new. Therefore, the best solution to avoid leaks is to use the Standard Library instead of trying to emulate it yourself.
int *pArray = new int; can be considered an array of size 1 and it kinda does what you want "by default".
But what if I need an array of 10 elements?
Pointers do not have any magical abilites, they just point to memory, therefore:
pArray[5] = 10; will just yield a run-time error (if you are lucky).
Therefore there is a possibility to allocate an array of needed size by calling new type[size].
I have pointer str:
char* str = new char[10];
I use the memory block str points to to store data.
How can I allocate more bytes for the buffer pointed to by str and not lose old data stored in the buffer?
Use std::string instead. It will do what you need without you worrying about allocation, copy etc. You can still access the raw memory via the c_str() function.
Even std::vector<char> will work well for you.
new[] another buffer, copy the data there (use memcpy() for that), then delete[] the old one, assign the new buffer address to the pointer originally holding the old buffer address.
You cannot using the new construction. For that you need to use the good old malloc, realloc, and free (do not mix malloc/realloc/free and new/delete).
The realloc function is what you are searching for. You had to use malloc/free instead of new/delete to use it
If you are really using C++, the most correct solution would be to use std::vector. I assume that you are not using that information as a standard string, in that case you should use std::string (which is an specialization of std::vector, so no big deal). You are creating at least 10 chars. This gives me the hint that you are probably quite sure that you'll need 10 chars, but maybe you'll nedd more. Maybe you are worried about the performance problems involved in allocating and deallocating memory. In that case, you can create your string and then reserve the estimated capacity that you expect you'll need, so there won't be any reallocation at least until you get to that limit.
int main()
{
std::string s;
s.reserve( 10 );
// do whatever with s
}
As others have already pointed out, the use of std::string or std::Vector will get you the benefit of forgetting about copy, resizing or deleting the reserved memory.
You have to allocate a different, bigger string array, and copy over the data from str to that new string array.
Allocation is a bit like finding a parking place.
You're asking here if it's possible to add a trailer on your car that has been parked for a fews days.
The answer is, in C there exists something called realloc that allows you to do following thing.
If I have already enough place to add my trailer, do so. If not park in another place big enough for your trailer and your car, which is equivalent to copying your data.
In other words you'll get strong and random performance hits.
So what would you do in the real world? If you knew you might need to add some trailers to your car you'd probably park in a bigger place than required. And when exceeding the size required for the place, you'd move your car and your trailers to a place with a nice margin for future trailers.
That's precisely what the STL's string and vector is doing for you. You can even give them a hint of the size of your futures trailer by calling "reserve". Using std::string is probably the best answer to your problem.
You can use realloc: http://www.cplusplus.com/reference/clibrary/cstdlib/realloc/
I would add that this approach is not the favored c++ approach (depending on your needs you could use std::vector<char> for instance).