C++ - The start of block memory - c++

I mean to give pointer to variable in struct\memory and then give it to some function and it'll give me pointer to the start of the struct\memory.
There is a function that does it?
To understand this question:
char* ptr=((char*)malloc(12))+3;
//MemStart(ptr) == ptr-3
MemStart'll be the function that'll give that result in the code.

Unless there is something specific to that structure's representation in memory (such as it always starts with the bytes 0xff, 0xff), there is no way to determine where a particular structure or array starts. Modern architectures are Von Neuman machines, meaning memory has no inherent meaning.
Note that many architectures have alignment issues or optimizations, meaning structures or arrays may need to start on 16-bit, 32-bit or 64-bit word boundaries, but these are architecture specific.
[Edit: Added the following]
The library may introduce guard bytes at the start or end of the allocated block of memory or fill memory with known constants to look for buffer overflows or errant pointers. However, these are usually omitted in release mode and, even if they are present, may also be valid data.
You could inspect the memory allocation table for dynamically allocated memory but the array/struct could be allocated on the stack rather than the heap. Also, what happens if you have a structure within an array? Which value does it return? If you want to restrict it to just dynamically allocated memory where you want the start of whatever memory was allocated only, Basile has a great answer.

Most malloc implementations don't provide what you want (the starting address of a dynamically allocated block, given some internal pointer to it). And the standards don't require anything like that. So you may want to use both pointer to base, and offset (perhaps packing them inside some C++ class fat_pointer with operator* and operator-> giving the illusion of pointers, etc, etc).
You could of course re-implement your own malloc above operating system memory segment allocation (i.e. mmap(2) on Linux or Posix) and de-allocation munmap(2) primitive syscalls.
You could also have your specialized allocator; it might allocate your zones with a large power-of-two alignment (e.g. 256 or 4096 bytes) using posix_memalign, then use bits manipulation on the intptr_t cast of your pointer.
Notice that implementing any serious memory allocator means caring about unportable details of your operating system and architecture (alignment, ...).
You could also use Boehm's conservative garbage collector, that is use GC_malloc instead of malloc (and don't bother GC_free-ing your dynamic data); then you have the GC_base function which does exactly what you want:
/* Return a pointer to the base (lowest address) of an object given */
/* a pointer to a location within the object. */
/* I.e. map an interior pointer to the corresponding bas pointer. */
/* Note that with debugging allocation, this returns a pointer to the */
/* actual base of the object, i.e. the debug information, not to */
/* the base of the user object. */
/* Return 0 if displaced_pointer doesn't point to within a valid */
/* object. */
/* Note that a deallocated object in the garbage collected heap */
/* may be considered valid, even if it has been deallocated with */
/* GC_free. */
GC_API void * GC_base(void * displaced_pointer);

You can use the offsetof() macro from <cstddef>:
#include <cstddef>
#include <iostream>
using namespace std;
typedef struct
{
int a;
int b;
} S;
int main()
{
S s = { 1, 2 };
cout << "address of s:" << &s << endl;
cout << "address of s.a:" << &s.a << endl;
cout << "address of s.b:" << &s.b << endl;
int* pb = &s.b;
S* ps = (S*)((char*)pb - offsetof(S, b));
cout << "ps:" << ps << endl;
return 0;
}
Output (ideone):
address of s:0xbffd266c
address of s.a:0xbffd266c
address of s.b:0xbffd2670
ps:0xbffd266c

Related

C++ std::make_unique usage

This is the first time I am trying to use std::unique_ptr but I am getting an access violation
when using std::make_unique with large size .
what is the difference in this case and is it possible to catch this type of exceptions in c++ ?
void SmartPointerfunction(std::unique_ptr<int>&Mem, int Size)
{
try
{
/*declare smart pointer */
//Mem = std::unique_ptr<int>(new int[Size]); // using new (No crash)
Mem = std::make_unique<int>(Size); // using make_unique (crash when Size = 10000!!)
/*set values*/
for (int k = 0; k < Size; k++)
{
Mem.get()[k] = k;
}
}
catch(std::exception& e)
{
std::cout << "Exception :" << e.what() << std::endl;
}
}
When you invoke std::make_unique<int>(Size), what you actually did is allocate a memory of size sizeof(int) (commonly 4bytes), and initialize it as a int variable with the number of Size. So the size of the memory you allocated is only a single int, Mem.get()[k] will touch the address which out of boundary.
But out of bounds doesn't mean your program crash immediately. As you may know, the memory address we touch in our program is virtual memory. And let's see the layout of virtual memory addresses.
You can see the memory addresses are divided into several segments (stack, heap, bss, etc). When we request a dynamic memory, the returned address will usually located in heap segment (I use usually because sometimes allocator will use mmap thus the address will located at a memory shared area, which is located between stack and heap but not marked on the diagram).
The dynamic memory we obtained are not contiguous, but heap is a contiguous segment. from the OS's point of view, any access to the heap segment is legal. And this is what the allocator exactly doing. Allocator manages the heap, divides the heap into different blocks. These blocks, some of which are marked "used" and some of which are marked "free". When we request a dynamic memory, the allocator looks for a free block that can hold the size we need, (split it to a small new block if this free block is much larger than we need), marks it as used, and returns its address. If such a free block cannot be found, the allocator will call sbrk to increase the heap.
Even if we access address which out of range, as long as it is within the heap, the OS will regard it as a legal operation. Although it might overwrite data in some used blocks, or write data into a free block. But if the address we try to access is out of the heap, for example, an address greater than program break or an address located in the bss. The OS will regard it as a "segment fault" and crash immediately.
So your program crashing is nothing to do with the parameter of std::make_unique<int>. It just so happens that when you specify 1000, the addresses you access are out of the segment.
std::make_unique<int>(Size);
This doesn't do what you are expecting!
It creates single int and initializes it into value Size!
I'm pretty sure your plan was to do:
auto p = std::make_unique<int[]>(Size)
Note extra brackets. Also not that result type is different. It is not std::unique_ptr<int>, but std::unique_ptr<int[]> and for this type operator[] is provided!
Fixed version, but IMO you should use std::vector.

Why does reallocating with realloc, a pre-allocated memory using allocator::allocate conserve the old start memory address?

The main problem was reallocating memory while expanding it and conserving data and the first starting memory address which is used by many other parts of the program (such as a static starting memory)
This doesn't work with realloc because he deallocate the precedent allocated memory and affects another with a new starting memory address:
using namespace std;
int *t = static_cast<int*>(malloc( 2*sizeof(int)));
cout << "address " << t << endl;
t = static_cast<int*>(realloc(t,10*sizeof(int)));
cout << "address " << t << endl;
=========================
// both of the addresses are different
address 0x55c454fc5180
address 0x55c454fc55b0
after testing many solutions (even direct access to the memory by system call), I found this one :
allocator<int> alloc;
int *t = alloc.allocate(2*sizeof(int));
cout << "address " << t << endl;
// reallocating memory using realloc
t = static_cast<int*>(realloc(t, 10*sizeof(int)));
cout << "address " << t << endl;
=========================
// now the addresses are the same
address 0x55c454fc5180
address 0x55c454fc5180
I tried to explain how it's possible but not able to match the both functioning and I want to know why and how it works.
Using realloc on an address allocated with std::allocator, new or anything similar has undefined behavior. It can only be used when the address comes from the malloc/calloc/realloc family of allocation functions. Never mix them.
In general realloc does not guarantee that the address of the allocation remains unchanged. There is no guarantee that realloc will be able to expand the allocation in place (e.g. there might not be enough memory free after the current allocation). It is the defined behavior of realloc to copy the memory block to a new allocation where sufficient space is free in such a situation. This also means that realloc can only be used with trivially-copyable types in C++.
If your program depends on the address of the allocation remaining unchanged, then it can't expand the allocation. You can have one of these, not both.
Also, you are using std::allocator<int> wrong. The argument to .allocate should be the number of elements of the array to allocate, not the number of bytes. Then afterwards you are supposed to call std::allocator_traits<std::allocator<int>>::construct or std::construct_at or a placement-new on each element of the array to construct and start the lifetime of the array elements.
I am not sure why you are trying to use std::allocator here, but it is unlikely that you need it. If you just intend to create an array of int, you should use new int[n], not std::allocator. Or rather don't worry with manual memory management at all and just use std::vector<int>.

Unexpected size when using malloc()

I wanted to test my knowledge, and wrote a small program, which looks like this:
#include <iostream>
#include <cstdlib>
int main()
{
int* mem1 = (int*) malloc(2 * sizeof(int));
int* mem2 = (int*) malloc(5 * sizeof(int));
std::cout << "mem1: " << sizeof(mem1) << std::endl;
std::cout << "mem2: " << sizeof(mem2) << std::endl;
return 0;
}
I get the output:
mem1: 8
mem2: 8
When I change the values, that malloc(2 * sizeof(int)) to malloc(3 * sizeof(int)), the output also doesn't change.
This is the Tutorial I used, so I am not entirely sure, if this is memory safe, with the conversion to int* when calling malloc.
I have also found this question, but I don't find it to be very helpful in my case.
Clang++ is the Compiler I used, but I don't think it makes any difference.
My best guess is, that it allocates the same memory, because it doesn't know where the memory ends.
sizeof is telling you the compile-time size of the pointer (it would report the same value even if you never called malloc at all). There is no standards-compliant way to determine the amount of memory allocated by malloc; you have to store it side-band if you need to preserve that information.
The 8 bytes that you see is the size of the pointer(int*). sizeof() is done at compile time and there's no way for the compiler to know this size at compile time(malloc is dynamic memory allocation).
The expression sizeof(mem1) will always return the same value, no matter the size of the allocated memory block! This is because mem1 is a pointer to the allocated memory, and the size of a pointer does not change (within a given environment, that is).
In your case (presumably, a build on a 64-bit platform) pointers are 8 bytes in length. You could build this for a different platform (say, a 32-bit system) and you may get a different size (for example, 4); but, on that platform, that size will then always be 4.
As mentioned above, there are a few problems in this question:
In C++ you dynamically allocate memory using the new operator. The option of using malloc is for working with c-style APIs, when needed dynamic allocations that will be de-allocated by c-free function. In order to make such an allocation, it is best advised to use std::malloc which you can read about in this link (cpp reference).
The operator sizeof is calculating the size of the static type, meaning that it is giving you back the size of integer pointer, which is constant, depending on your CPU architecture.
Working with raw pointers in C++ is ill-advised, you should always make sure that for each item dynamically created in memory there is an owner, that is the one responsible to delete it when no longer needed. In C++11 and later, this can be achieved by using smart pointers, such as std::unique_ptr. In fact, C++ has created a great Idiom for resource management, called RAII (Resource Allocation Is Initialization), which sets it apart from many other languages.

How would I use the sized operators delete/delete[] and why are they better?

C++14 introduced "sized" versions of operator delete, i.e.
void operator delete( void* ptr, std::size_t sz );
and
void operator delete[]( void* ptr, std::size_t sz );
Reading through N3536, it seems that those operators were introduced to increase performance. I know that the typical allocator used by operator new "stores" the size of the bulk memory somewhere, and that's how typical operator delete "knows" how much memory to return to the free store.
I am not sure however why the "sized" versions of operator delete will help in terms of performance. The only thing that can speed things up is one less read operation regarding the size from the control block. Is this indeed the only advantage?
Second, how can I deal with the array version? AFAIK, the size of the allocated array is not simply sizeof(type)*number_elements, but there may be some additional bytes allocated as the implementation may use those bytes as control bytes. What "size" should I pass to operator delete[] in this case? Can you provide a brief example of usage?
Dealing with your second question first:
If present, the std::size_t size argument must equal the size argument passed to the allocation function that returned ptr.
So, any extra space that might be allocated is the responsibility of the runtime library, not the client code.
The first question is more difficult to answer well. The primary idea is (or at least seems to be) that the size of a block often isn't stored right next to the block itself. In most cases, the size of the block is written, and never written again until the block is deallocated. To avoid that data polluting the cache while the block is in use, it can be kept separately. Then when you go to deallocate the block, the size will frequently have been paged out to disk, so reading it back in is quite slow.
It's also fairly common to avoid explicitly storing the size of every block explicitly at all. An allocator will frequently have separate pools for different sizes of blocks (e.g., powers of 2 from 16 or so up to around a couple kilobytes or so). It'll allocate a (fairly) large block from the OS for each pool, then allocate pieces of that large block to the user. When you pass back an address, it basically searches for that address through the different sizes of pools to find which pool it came from. If you have a lot of pools and a lot of blocks in each pool, that can be relatively slow.
The idea here is to avoid both of those possibilities. In a typical case, your allocations/deallocations are more or less tied to the stack anyway, and when they are the size you're allocating will likely be in a local variable. When you deallocate, you'll typically be at (or at least close to) the same level of the stack as where you did the allocation, so that same local variable will be easily available, and probably won't be paged out to disk (or anything like that) because other variables stored nearby are in use as well. For the non-array form, the call to ::operator new will typically stem from a new expression, and the call to ::operator delete from the matching delete expression. In this case, the code generated to construct/destroy the object "knows" the size it's going to request (and destroy) based solely on the type of object being created/destroyed.
For the size argument to C++14 operator delete you must pass the same size you gave to operator new, which is in bytes. But as you discovered it's more complicated for arrays. For why it's more complicated, see here: Array placement-new requires unspecified overhead in the buffer?
So if you do this:
std::string* arr = new std::string[100]
It may not be valid to do this:
operator delete[](arr, 100 * sizeof(std::string)); # BAD CODE?
Because the original new expression was not equivalent to:
std::string* arr = new (new char[100 * sizeof(std::string)]) std::string[100];
As for why the sized delete API is better, it seems that today it is actually not but the hope is that some standard libraries will improve performance of deallocation because they actually do not store the allocation size next to each allocated block (the classical/textbook model). For more on that, see here: Sized Deallocation Feature In Memory Management in C++1y
And of course the reason not to store the size next to every allocation is that it is a waste of space if you don't truly need it. For programs which make many small dynamic allocations (which are more popular than they ought to be!), this overhead can be significant. For example in the "plain vanilla" std::shared_ptr constructor (rather than make_shared), a reference count is dynamically allocated, so if your allocator stores the size next to it, it might naively require about 25% overhead: one "size" integer for the allocator plus the four-slot control block. Not to mention memory-pressure: if the size is not stored next to the allocated block, you avoid loading a line from memory on deallocation--the only information you need is given in the function call (well, you also need to look at the arena or free-list or whatever, but you needed that in any case, you still get to skip one load).
Some relevant info: Currently VS 17 implementation of sized delete[] seems broken. It always returns the generic pointer size (void*). The g++ 7.3.1 give the size of the full array plus 8 bytes overhead. Haven't tested it on other compilers, but as you see neither of them giving the expected result.
WRT to usefulness of it, as is alluded to in the selected answer, the main usefulness come in play when you have your custom allocators, either to pass to stl containers or just use for local memory management. In these cases it could be very useful to have user size array size be returned to you, so you can free the proper size from your allocators. I can see it is possible to avoid having to use this though. Here's a code you can use for testing the "correctness" of sized delete[] implementation in your compiler:
#include <iostream>
#include <sstream>
#include <string>
std::string true_cxx =
#ifdef __clang__
"clang++";
#elif _MSC_VER
"MVC";
#else
"g++";
#endif
std::string ver_string(int a, int b, int c) {
std::ostringstream ss;
ss << a << '.' << b << '.' << c;
return ss.str();
}
std::string true_cxx_ver =
#ifdef __clang__
ver_string(__clang_major__, __clang_minor__, __clang_patchlevel__);
#elif _MSC_VER
#ifdef _MSC_FULL_VER
#if _MSC_FULL_VER == 170060315
"MSVS 2012; Platform Toolset v110";
#elif _MSC_FULL_VER == 170051025
"MSVS 2012; Platform Toolset v120_CTP_Nov2012";
#elif _MSC_FULL_VER == 180020617
"MSVS 2013; Platform Toolset v120";
#elif _MSC_FULL_VER == 191426431
"MSVS 2017; Platform Toolset v140";
#else
"Not recognized";
#endif
#endif // _MSC_FULL_VER
#else
ver_string(__GNUC__, __GNUC_MINOR__, __GNUC_PATCHLEVEL__);
#endif
// sized class-specific deallocation functions
struct X {
static void operator delete(void* ptr, std::size_t sz)
{
std::cout << "custom delete for size " << sz << '\n';
::operator delete(ptr);
}
static void operator delete[](void* ptr, std::size_t sz)
{
std::cout << "custom delete[] for size " << sz << '\n';
::operator delete(ptr);
}
char16_t c[2];
};
int main() {
X* p1 = new X;
delete p1;
X* p2 = new X[10];
for (int i = 0; i < 10; ++i)
p2[i] = X{ (char16_t)i };
std::cout << "Compiler: "<<true_cxx.c_str()<<": Version:" << true_cxx_ver.c_str() << std::endl;
delete[] p2;
}

Determine the nature of parameter in runtime

I have a function
void fname(char* Ptr)
{
...
}
I want to know inside this function whether this pointer Ptr holds the address of dynamically allocated memory using new char[] or the address of locally allocated memory in the calling function. Is there any way I can determine that? I think <typeinfo> doesn't help here.
One way to do this is to have your own operator new functions and keep track of everything allocated so that you can just ask your allocation library if the address given is one it allocated. The custom allocator then just calls the standard one to actually do the allocation.
Another approach (messy and details highly OS dependent) may be to examine the process layout in virtual memory and hence determine which addresses refer to which areas of memory.
You can combine these ideas by actually managing your own memory pools. So if you get a single large chunk of system memory with known address bounds and use that for all new'd memory, you can just check that an address in is the given range to answer your question.
However: Any of these ideas is a lot of work and not appropriate if this problem is the only purpose in doing so.
Having said all that, if you do want to implement something, you will need to work carefully through all the ways that an address might be generated.
For example (and surely I've missed some):
Stack
Return from new
Inside something returned from new.
Was returned from new but already deleted (hopefully not, but that's why we need diagnostics)
statically allocated
static constant memory
command line arguments/ environment
code addresses.
Now, ignoring all that for a moment, and assuming this is for some debug purpose rather than system design, you might be able to try this kind of thing:
This is ugly, unreliable, not guaranteed by the standard, etc etc, but might work . . .
char* firstStack = 0;
bool isOnStack(const void* p)
{
char* check =(char*)p;
char * here = (char*)&check;
int a = firstStack - check;
int b = check - here;
return (a*b > 0);
}
void g(const char* p)
{
bool onStack = isOnStack(p);
std::cout << p << (onStack ? "" : " not" ) << " on stack " << std::endl;
}
void f()
{
char stuff[1024] = "Hello";
g(stuff);
}
void h()
{
char* nonsense = new char[1024];
strcpy(nonsense, "World");
g(nonsense);
delete [] nonsense;
}
int main()
{
int var = 0;
firstStack = (char*)&var;
f();
h();
}
Output:
Hello on stack
World not on stack
The short answer: no, you can't. You have no way of knowing whether Ptr is a pointer to a single char, the start of a statically allocated array, a pointer to a single dynamically allocated char, or the start of an array thereof.
If you really wanted to, you try an overload like so:
template <std::size_t N>
void fname(char Ptr[N])
{
// ...
}
which would match when passed a statically allocated array, whereas the first version would be picked when dealing with dynamically allocated memory or a pointer to a single char.
(But note that function overloading rules are a bit complicated in the presence of templates -- in particular, a non-template function is preferred if it matches. So you might need to make the original function take a "dummy" template parameter if you go for this approach.)
In vc++ there is an assertion _CrtIsMemoryBlock (http://msdn.microsoft.com/en-us/library/ww5t02fa.aspx#BKMK_CRT_assertions) that can be used to check if a pointer was allocated from the heap. This will only work when a debug heap is being used but this is fine if you are just wanting to add some 'debug only' assertions. This method has worked well for me in the past under Windows.
For Linux however I know of no such equivalent.
Alternatively you could use an inline assembler block to try to determine the if it is a stack address or not. This would be hardware dependent as it would rely heavily not only on the processor type but also on the memory model being used (flat address model vs segmented etc). Its probably best to avoid this type of approach.