I am writing an application in C++Builder 6 Enterprise.
The maximum memory the software allows me to reserve is around 870 MB, no more. The physical memory available on the system is 8 GB and the PC is running Windows 7.
Immediately after a memory allocation statement like malloc(870000000) is executed, Task Manager says the memory used by the whole system is 2.5 GB.
My question is, why can't I allocate up to end of available memory?
C++Builder 6 was released in 2002, and can produce only 32bit apps. The ability to produce 64bit apps was added in C++Builder XE3 in 2012.
A 32bit app cannot access more than 4GB max, no matter what.
Apps written in C++Builder 6 are not Large Address Aware (and it is not safe to manually mark them as such, as the RTL and memory manager are not LAA compatible), so the most memory they can hope to access is 2GB max (the other 2GB is reserved for Windows to use).
When you ask malloc() to allocate ~830 MB (not 870 MB, which would be 912261120 instead of 870000000), you are asking it to allocate 1 contiguous block of memory, which is likely to fail in a non-trivial app.
Even if the app were Large Address Aware, that would up the accessible memory to only 3GB on 32bit Windows (only if the /3GB flag is enabled on Windows startup), and 4GB on 64bit Windows.
So, you will never be able to get a 32bit app to allocate anywhere close to the full 8GB. You need a 64bit app for that.
Related
I have code which is 32 bit and i think compiler too. But when i am compiling my c++ code, its taking more than 2 GB memory. As per my understanding on 32 bit system no process can take more than 2 GB.
Any suggestions how can i achieve this? I found lot of posts on this but those
are not helpful as they are adding swaps. But i already have 8 GB ram. So my problem is not available memory, its size of compiling process which could not be more than 2 GB.
Even i have 8 GB ram I have tried to adding swap and that's also not working.
On Windows 32 Bit, the maximum amount of RAM is 4 GB. By default, this address space is seperated into kernel memory and process memory, both being 2 GB large. Most programs don't need more than 2 GB of memory, but if you do, you can enlarge the process memory by specifying the /3GB switch, leaving less memory for the kernel.
Read here for more information: https://msdn.microsoft.com/en-us/library/windows/hardware/ff556232(v=vs.85).aspx
Edit: Keep in mind that if you want to make use of this additional memory, you also need to compile your program with the /LARGEADDRESSAWARE switch. That will set a flag in the Process Environment Block of your program, making Windows aware that your program might need more than 2 GB of memory.
Since you stated you have 8GB of RAM, I am presuming your OS and CPU are actually 64-bit. So you are asking how to make a 32-bit program access more than 2GB of virtual address space, on a 64-bit OS, i.e. running under WOW64.
In that case, using the /LARGEADDRESSAWARE linker option in Visual Studio will give your app 4GB of virtual address space, under WOW64. You won't see any benefit in 32-bit Windows, unless you force your users to boot their OS with a certain flag.
I believe your app doesn't really need more than 2GB of RAM, but it's impossible to tell without knowing any details.
In any case, the one correct answer is: switch to a 64-bit app, which will get you 8TB of virtual address space. That's 8 terabytes.
I made a 32 bit c++ program which is always run on x64 machines. A client is saying that running 5 instances of this process is using causing all of their 24 GB RAM to be used.
Immediately I would think there was a memory leak but I am unable to reproduce this memory issue.
Doing a bit more research into memory allocations I found Memory Limits for Windows. This tells me that a 32 bit process will not be allowed more than 2 GB of memory by the OS.
Is it at all possible that a 32 bit application on a 64 bit windows will be able to have a memory leak use more than 2 GB?
P.S. Killing the process results in the memory being restored to normal operating levels (about 2 GB).
[EDIT] I have now seen that most of the memory being used is Kernel Memory: Nonpaged. Does this mean that it is some system resource which is being used and not a memory leak?
[UPDATE] The problem is not a driver or memory leak. It seems to be a process handle leak. There is something which is continuously starting new handles to a file. This was found using perfmon to monitor the process. As a rule of thumb if a process has more than 2000 to 3000 handles you should investigate. Especially if that number is increasing every few seconds.
As stated in Memory Limits for Windows, limit for 32-bit process on 64-bit system is 4 GB with IMAGE_FILE_LARGE_ADDRESS_AWARE set, thus your 5 processes could consume 20 GB of memory total. This can be set through LARGEADDRESSAWARE option, which expands virtual address space.
It is obviously possible, as the client is experiencing it.
(maybe you did expect like some ideas how? You don't provide much info or code, so in a very general way I would suggest the memory allocation may be not in the app itself directly. Maybe the app itself will take only ~1-2GiB, but will also stir the OS to do something stupid, like virtual memory map of file of size of 4+GiB, or other devices lock, where the device driver does something stupid, etc...)
You should profile the memory usage on the target system to have idea how much your code does use. Then you can try to search for the rest of it.
In general, using the /LARGEADDRESSAWARE:ON linker switch can allow a 32bit application use more than 2GB. Also using the Address Windowing Extensions can allow using more memory. But if you aren't using any of these techniques in your application then it should have the 2GB range. However since the upper 2GB range is used for system resources maybe you are leaking system resources?
I am trying to allocate memory of 1 GiB using malloc() on Windows and it fails. I know malloc's uncertainty. What is best solution to allocate memory of 1 GiB?
If you are using a 32-bit (x86) application, you are unlikely to be able to allocate a 1 GB continuous chunk of memory (and certainly can't allocate 2GB). As to why this happens, you should see the venerable presentation "Why Your Windows Game Won't Run In 2,147,352,576 Bytes" (Gamefest 2007) attached to this blog post.
You should build your application as an x64 native (x64) application instead.
You could enable /LARGEADDRESSAWARE and stick with a 32-bit application on Windows x64, but it has a number of quirks and may limit what kinds of 3rd party support libraries you can use. A better solution is to use x64 native if possible.
Use the /LARGEADDRESSAWARE flag to tell Windows that you're not doing funny things with addresses. This unlocks an extra 2GB of address space on Win64.
I have a 32-bit application consisting one EXE and multiple DLLs. The EXE has been built with /LARGEADDRESSAWARE flag set. So I expect on a 64-bit OS I should get 4 GB of user address space. But on some 64-bit Win 7 systems I am getting only 2 GB of user address space.
The physical memory is 8 GB if that matters. What could be reason for this behavior?
After browsing through MSDN, I found the following:
On http://msdn.microsoft.com/en-us/library/windows/desktop/aa366770(v=vs.85).aspx (the page for MEMORYSTATUSEX which is used by GlobalMemoryStatusEx (http://msdn.microsoft.com/en-us/library/windows/desktop/aa366589(v=vs.85).aspx) ) the description for ullTotalVirtual is:
this value is approximately 2 GB for most 32-bit processes on an x86 processor and approximately 3 GB for 32-bit processes that are large address aware running on a system with 4-gigabyte tuning enabled.
The 4GB tuning page is: http://msdn.microsoft.com/en-us/library/windows/desktop/bb613473(v=vs.85).aspx and it says something like:
On 64-bit editions of Windows, 32-bit applications marked with the IMAGE_FILE_LARGE_ADDRESS_AWARE flag have 4 GB of address space available.
Itanium editions of Windows Server 2003: Prior to SP1, 32-bit processes have only 2 GB of address space available.
Also, the memory limits page (http://msdn.microsoft.com/en-us/library/aa366778.aspx#memory_limits) can come handy if you want to determine the total memory your system supports.
However the real useful information comes from Mark Russinowich's blog: http://blogs.technet.com/b/markrussinovich/archive/2008/07/21/3092070.aspx
While 4GB is the licensed limit for 32-bit client SKUs, the effective limit is actually lower and dependent on the system's chipset and connected devices. The reason is that the physical address map includes not only RAM, but device memory as well, and x86 and x64 systems map all device memory below the 4GB address boundary to remain compatible with 32-bit operating systems that don't know how to handle addresses larger than 4GB.
So the conclusion is that yes, this might depend on the configuration of the system. Maybe you can complete your question with a table, with the amount of the memory you get on each system and some important system configuration settings, and we might discover a pattern in this case.
The problem is that an Application has a whole has to be large Adress aware - so that pointers are to be treated as unsigned.
If however on "some" system some of your used DLL is not large adressaware this renders your whole program not large address aware.
http://blogs.msdn.com/b/oldnewthing/archive/2010/09/22/10065933.aspx
I have a x64 application. It is a bit heavy application (in terms of thread counts, memory needed etc). I'm running this application on a 4GB RAM 64-bit Windows7 Quad core machine. But what i notice is after my application takes around 2.2-2.3 GB Ram it crashes. Is this required behavious? Can 64bit applications at max take only 2.3-2.4 GB RAM from 4GB RAM?
No, check that you don't have any overflows or other bugs first. Even 32-bit applications can address - use more RAM, so it would be ridiculous if it was limited on windows 64-bit.
The remaining memory is free for use, right?
The amount of RAM is of no concern, a process allocates virtual memory. The amount of virtual memory you can allocate in a 64-bit process is only limited by the operating system's ability to provide pages to map the virtual memory to RAM. Which is normally only limited by the largest allowed paging file, assuming you didn't create your own mapping with CreateFileMapping(). It is a system setting, Control Panel + System + Advanced. There's an upper limit as well imposed by the Windows edition, 16 gigabytes for Windows 7 Home Premium, 192 gigabytes for Professional and up.
RAM is only used when your program actually accesses the virtual memory. Which generates a page fault when the memory isn't mapped yet. A 64-bit process typically slows down to a crawl due to these page faults if you allocate and use memory considerably beyond the amount of RAM, well before you consume all available pages.
Since you are considerably South of the typical maximum paging file size, this is surely just another plain old heap corruption bug.
Debug your program. Crashing isn't supposed to happen just because you allocate a lot of memory. The usual suspects: math overflows, memory corruptions, failure to handle errors from memory-allocating routines.