Hell'o
I want to create my own dynamic array (vector) class, but don't know how to allocate memory on addres whom I point to. In function add I added line like:
int * object = new (this->beginning + this->lenght) int (paramValue); But visual studio shows me an error message "operator new cannot be called with the given arguments". How to make it works, which arguments should I send to the new operator?
(I am not sure to understand your question, but....)
You might want to use the placement new operator (but to implement a <vector> like thing you don't need that). Then you'll need to #include <new>
But you probably don't need that. Just call plain new from your constructor, and plain delete from your destructor. Something like int*arr = new int[length]; (in constructor) and later delete[] arr; (in destructor).
(it looks that you are misunderstanding something; I recommend spending several days reading a good C++ programming book)
how to allocate memory on address whom I point to
Insufficient information -- what kind of system? custom hardware? OS?
On a desktop, you could use 2 steps. You allocate a block of bytes using something like:
uint8_t* myMemoryBlock = new uint8_t[1000]; // 1000 byte block
Then you might contemplate using placement new at the address "you point to" using 'myMemoryBlock', with a cast.
On a desktop, the dynamic memory system can be used this way...
But if you are planning to create a user defined type any way, just new that type, and let the dynamic memory fall where it may, as opposed to positioning it on myMemoryBlock.
On a desktop, there is (generally) no memory your user-privilege level executable can access with 'new'. All other memory is protected.
mmap on Linux maps devices or files into your executables memory range. I am unfamiliar with such devices, but I have used mmap with files.
update 2017/03/19
Note 1 - user-privilege level tasks are typically blocked from accessing other / special memory.
Note 2 - memory addresses, such as 'myMemoryBlock' above, are virtual, not physical. This includes code addresses, automatic memory addresses, dynamic memory addresses. If your processor has memory management hardware support, your coding has special efforts to access physical addresses, in memory or otherwise.
On a single-board-computer (SBC), (with or without an OS) I would expect that the address you wish to 'allocate' will not be within the 'dynamic' memory set up by the board support package (BSP).
On this kind of embedded system (on a SBC), someone (an architect) has 'mapped' this 'special' memory to an address range not in use for other purposes (i.e. not part of dynamic memory). Here, you simply find out what the address is, and use it by casting the uintXX_t value to a pointer of appropriate type. Something like:
myDataType* p = reinterpret_cast<myDataType*>(premappedAddress);
For more info, you should seek out other sites discussing embedded systems.
Related
How to detect programmatically count of bytes allocated by process on Heap?
This test should work from process itself.
I think mallinfo() is what you want:
#include <malloc.h>
struct mallinfo *info;
info = mallinfo();
printf ("total allocated space: %llu bytes\n", info->uordblks);
printf ("total free space: %llu bytes\n", info->fordblks);
The struct mallinfo structure is technical, and specific to the malloc() implementation. But the information you want is in there. Here is how I report the values:
mallinfo.arena = "Total Size (bytes)"
mallinfo.uordblks = "Busy size (bytes)"
mallinfo.fordblks = "Free size (bytes)"
mallinfo.ordblks = "Free blocks (count)"
mallinfo.keepcost = "Top block size (bytes)"
mallinfo.hblks = "Blocks mapped via mmap() (count)"
mallinfo.hblkhd = "Bytes mapped via mmap() (bytes)"
These two are allegedly not used, but they seem to change on my system, and thus might be valid:
mallinfo.smblks = "Fast bin blocks (count)"
mallinfo.fsmblks = "Fast bin bytes (bytes)"
And the other interesting value is returned by "sbrk (0)"
There are a number of possibilities.
How accurate do you need it to be? You can get some useful data via cat /proc/${PID}/status | grep VmData.
You can #define your own malloc(), realloc(), calloc(), and free() functions, wrapping the real functions behind your own counter. You can do really cool things here with __FILE__, __LINE__, & __func__ to facilitate identifying core leaks in simple tests. But it will only instrument your own code!
(Similarly, you can also redefine the default operator new and operator delete methods, both array and non-array variants, and both throwing std::bad_alloc and std::nothrow_t variants. Again, this will only instrument your own code!)
(Be aware: On most C++ systems, new ultimately invokes malloc(). It doesn't have to. Especially with in-place new! But typically new does make use of malloc(). (Or it operates on a region of memory that has previously been malloc()'ed.) Otherwise you'd get into really funky stuff with multiple heap managers...)
You can use sbrk(0) to see where the data segment is currently set. That's not so great. It's a very coarse measurement, and it doesn't account for holes (unused memory regions) in the heap. (You're much better off with the VmData line from /proc/${PID}/status.) But if you're just looking for a general idea...
You can trap malloc()/free()/etc by writing your own shared library and forcing your process to use it instead of the real versions via LD_PRELOAD. You can use dlopen()/dlsym() to load & invoke the *real* malloc()/free()/etc. This works quite beautifully. The original code is unmodified, not even recompiled. But be aware of re-entrant situations when coding this library, and that your process will initially invoke malloc()/calloc()/realloc() before dlopen()/dlsym() can complete loading the real functions.
You might check out tools like Valgrind, though that's really aimed more at memory leaks.
Then again, perhaps mtrace() is what you want? Or __malloc_hook? Very proprietary (GNU) & nonstandard... But you are tagged "Linux"...
There's no easy, automatic way to do it, if that's what you're asking. You basically have to manually keep track of heap allocations yourself using a counter variable. The problem is that it's difficult to control which parts of your program are allocating memory on the heap, especially if you're using a lot of libraries out of your control. To complicate things further, there's two ways a program might allocate heap memory: new or malloc. (Not to mention direct OS calls like sbrk.)
You can override global operator new, and have each call to new increase a global tally. However, this won't necessarily include times when your program calls malloc, or when your program uses some class-specific new override. You can also override malloc using a macro, but this is not necessarily portable. And you'd also have to override all the variations of malloc, like realloc, calloc, etc. All of this is further complicated by the fact that on some implementations, new itself may call malloc.
So, essentially, it's very difficult to do this properly from within your program. I'd recommend using a memory profiler tool instead.
A speculative solution: redefine new and delete operators.
On each new operator call, a number of bytes to allocate is passed. Allocate a bit more memory and store the amount of bytes allocated within. Add this amount to global variable that holds the heap size.
On delete operator call, check the value you stored before you dispose the memory. Subtract it from that global variable.
If you're on windows, you can use GetProcessHeap(), HeapQueryInfo() to retrieve information about the processes heap. An example of walking the heap from MSDN
Since you've tagged your question 'linux' it might help to look at some of the information provided in the /proc directory. I haven't researched this a lot so I can only give you a starting point.
/proc/<your programs pid> contains files with some information about your process from the viewpoint of the kernel. There is a symlink /proc/self that will always about the process your investigating this from.
The files you might be most interested in are stat, statm and status. The latter is more human-readable, whereas the former two give the same info in a more machine-readable format.
A starting point about how to interpret the content of those files is available in the proc(5) manpage.
Other then keeping track of all your memory allocations I don't believe there is a way to calculate the heap size or usage
Use malloc_info(). Have fun with the XML aspect of it. mallinfo() (as shown in another answer) does a similar thing but is restricted to 32-bit values... a foolish thing to rely on in 2010+.
How do memory leak detectors actually work? What are the underlying concepts in general? Can take C++ as the language to explain this.
There are a couple of different ways that leak detectors work. You can replace the implementation of malloc and free with ones that can track more information during allocation and are not concerned with performance. This is similar to how dmalloc works. In general, any address that is malloc'ed but not free'd is leaked.
The basic implementation is actually pretty simple. You just maintain a lookup table of every allocation and its line number, and remove the entry when it is freed. Then when the program is done you can list all leaked memory. The hard part is determining when and where the allocation should have been freed. This is even harder when there are multiple pointers to the same address.
In practice, you'll probably want more than just the single line number, but rather a stack trace for the lost allocations.
Another approach is how valgrind works which implements an entire virtual machine to keep track of addresses and memory references and associated bookkeeping. The valgrind approach is much more expensive, but also much more effective as it can also tell you about other types of memory errors like out of bounds reads or writes.
Valgrind essentially instruments the underlying instructions and can track when a given memory address has no more references. It can do this by tracking assignments of addresses, and so it can tell you not just that a piece of memory was lost, but exactly when it became lost.
C++ makes things a little harder for both types of leak detectors because it adds the new and delete operators. Technically new can be a completely different source of memory than malloc. However, in practice many real C++ implementations just use malloc to implement new or have an option to use malloc instead of the alternate approach.
Also higher level languages like C++ tend to have alternative higher level ways of allocating memory like std::vector or std::list. A basic leak detector would report the potentially many allocations made by the higher level modes separately. That's much less useful than saying the entire container was lost.
Here's a published technical paper on how our CheckPointer tool works.
Fundamentally it tracks the lifetimes of all values (heap and stack), and their sizes according their types as defined by the language. This allows CheckPointer to find not only leaks, but out-of-array bound accesses, even for arrays in the stack, which valgrind won't do.
In particular, it analyzes the source code to find all pointer uses.
(This is quite the task just by itself).
It keeps track of pointer meta data for each pointer, consisting of
A reference to the object meta data for the heap-allocated object or global or local variable orfunction pointed to by the pointer and
The address range of the (sub)object of the object that the pointer may currently access. This may be smaller than the address range of the
whole object; e.g. if you take the address of a struct member, the instrumented source code will only allow access to that member when using the resulting pointer.
It also tracks the kind and location of each object, i.e. whether
it is a function, a global, thread-local or local variable, heap-allocated memory, or a string literal constant:
The address range of the object that may be safely accessed, and
For each pointer stored in the heap-allocated object or variable, a reference to the pointer metadata for that pointer.
All this tracking is accomplished by transforming the original program source, into a program which does what the original program does, and interleaves various meta-data checking or updating routines. The resulting program is compiled and run. Where a meta-data check fails at runtime, a backtrace is provided with a report of the type of failure (invalid pointer, pointer outside valid bounds, ...)
This is tagged C and C++ and no operating system is mentioned. This answer is for Windows.
C
Windows has the concept of virtual memory. Any memory a process can get is virtual memory. This is done through VirtualAlloc() [MSDN]. You can imagine the leak detector to put a breakpoint on that function and whenever it is called, it gets the callstack and saves it somewhere. Then it can do similar for VirtualFree()[MSDN].
The difference can then be identified and shown along with the callstacks that have been saved.
C++
C++ has a different concept: it takes the large 64kb blocks which it gets from VirtualAlloc() and splits it into smaller pieces, called the Heap. The C++ heap manager comes from Microsoft and offers new methods HeapAlloc() [MSDN] and HeapFree()[MSDN].
Then, you could do the same as before, but actually, that feature is already built-in. Microsoft's GFlags [MSDN] tool can enable the tracking:
In this case it will save up to 50 MB of callstack information for C++ heap manager calls.
Since that settings can also be enabled via the Windows Registry, a memory leak detector can make use of it easily.
General concept
As you can see, the general concept is to keep track of allocations and deallocations, compare them and show the callstacks of the difference.
I am currently working on a code, incremental garbage collection which is just a simulation. What i mean is that in the program, the user wil enter the amount of physical memory to be assigned and also will be entering keywords like x = alloc(10MB) which expects me to allocate the object "x" 10MB of the physical memory. So i will be needing a start pointer as an end pointer for the code.
My doubt: What would be the best way to assign this "physical memory" ? I came across malloc and calloc where many recommended not to use it unless necessary. Also there is the new operator. So i wanted to know if there was any other better way. And this physical memory will remain fixed through the process.
Any help is appreciated.
In C++, instead of allocating raw memory or even using raw pointers, it's generally encouraged to use the collection classes of the C++ standard library, such as std::vector. If you're writing a simulation for garbage collection, I can imagine that you design a class for unified access to GC-managed memory, and then you allocate a vector of them.
If, however, this is not the case, you can (although in C++, you reallly shouldn't) use malloc(), or calloc() if you need zero-initialized memory.
Also is there a way i can make another pointer point to the end address of the block.?
Sure, use pointer arithmetic.
T *ptr = malloc(sizeof(*ptr) * N_ELEMENTS);
T *endPastOne = ptr + N_ELEMENTS;
If I have a special hardware unit with some storage in it is connected to the computer and is memory mapped, so that
its storage is accessible in the address range 0x55500000 – 0x555fffff how an I interface this hardware unit to
my C++ program so that dynamic memory is allocated in this hardware unit, not in my computer’s memory?
I need to implement a class which has the following function in it.
void * allocMemoryInMyHardware(int numberOfBytesToAllocate);
which returns a pointer to the allocated memory chunk, or null if unable to allocate.
You need to write your own allocator. Search internet for a sample code and tweak it. If you have simple requirements, basic allocator can be written from scratch in 2-4 hours. This approach will work if your platform does not have virtual memory management and code can access your range of addresses directly. Otherwise you need to dive into the driver development on your platform.
Typical strategy is to add header to each allocated unit and organize a double linked list for free memory areas. NT heaps work in similar way.
I think you can use the placement new syntax for this purpose. Using this, you can tell where objects shall be constructed:
char memory[10];
int* i = new (memory) int(42);
I am trying to write a variable monitoring class that allows me to pass it a pointer (ideally void*) addressing a memory location which would normally be completely out-of-scope or inaccessible for the class. The class would then periodically display on the screen in text the contents of that memory location - interpreted in a user defined way (eg. (int*) ). I would only ever read from memory using this pointer and it would serve as a dirty hack to enable a kind of watch window during development for the variables that I am temporarily interested in monitoring during run-time - without introducing a lot of code to bring these variables in scope / accessible to the class.
I am using VC++ 2010 and it seems to flat out refuse to let me even write an out of scope memory location address to the pointer.
I guess there's a lot going on under the hood in windows such that this approach may have very limited applicability as memory locations change but I am using native C++ so am hoping that my addresses are persistent enough to be useful. Also, I can see that it would not like me accessing a memory location that my program is not actually using for security reasons...
Any ideas how I can do this? (I realise that using such pointers gives rise to undefined behaviour so would only ever read from them and display the value).
Thanks.
Trying to dereference pointers that point outside any space which you can account for is pretty much meaningless. The address you may be accessing might not even be mapped into the memory space of your process so there is actually nothing even to look at.
When your process starts, You don't actually have 4 GB at your disposal. the memory space size is 4 GB but it is mostly made of holes that are not mapped to your process.
Eventually it all comes down to where you got the pointer you're trying to use. Memory addresses which you usually can account for may come from:
heap allocations - anything inside the ranges allocated by malloc or new and not yet freed or deleted
stack space, global variables - anything you define as variables in your program inside the scopes of your current position in the program. Accessing anything defined in other scopes is meaningless (for instance, returning a pointer to a local variable from a function)
code segments - addresses inside the segments of memory that contain the DLL or EXE of your process that were not unloaded. Usually you can access them only for read-only access. You can find such addresses for instance by looking up the return address of a function.
Accessing a pointer in a memory chunk you just deallocated is exactly a case of such meaninglessness. Once you deallocated your memory, there is a certain chance that it was already returned to the OS and that that address is no longer mapped to your process.
You can further read about this here