Memory allocation failed even when there is still enough memory - c++

Question

I am working on Linux (ubuntu 13.04 exactly) and currently I have a question: Why memory allocation will fail even when there is still enough memory?
I wrote a simple test application today and I encountered this issue when running this test app. Below is the code snippet I used to have the test:
#include <stdio.h>
#include <unistd.h>
#include <list>
#include <vector>
#include <strings.h>
using namespace std;
unsigned short calcrc(unsigned char *ptr, int count)
{
unsigned short crc;
unsigned char i;
//high cpu-consumption code
//implements CRC algorithm: Cylic
//Redundancy code
}
void* CreateChild(void* param){
vector<unsigned char*> MemoryVector;
pid_t PID = fork();
if (PID == 0){
const int MEMORY_TO_ALLOC = 1024 * 1024;
unsigned char* buffer = NULL;
while(1){
buffer = NULL;
try{
buffer = new unsigned char [MEMORY_TO_ALLOC]();
calcrc(buffer, MEMORY_TO_ALLOC );
MemoryVector.push_back(buffer);
} catch(...){
printf("an exception was thrown!\n");
continue;
} //try ... catch
} //while
} // if pid == 0
return NULL;
}
int main(){
int children = 4;
while(--children >= 0){
CreateChild(NULL);
};
while(1) sleep(3600);
return 0;
}
During my test, the above code starts throwing exception when there is around 220M RAM available. And from the moment on, it looks like the application is not able to get more memory any more
because the free memory shown by TOP command remains to be above 210M. So why would this happen?
UPDATE
1. Software && Hardware Information
The RAM is 4G and swap is around 9G bytes. Running "uname -a" gives: Linux steve-ThinkPad-T410 3.8.0-30-generic #44-Ubuntu SMP Thu Aug 22 20:54:42 UTC 2013 i686 i686 i686 GNU/Linux
2. Statistic Data during the Test
Right after Test App Starts Throwing Exception
steve#steve-ThinkPad-T410:~$ free
total used free shared buffers cached
Mem: 3989340 3763292 226048 0 2548 79728
-/+ buffers/cache: 3681016 308324
Swap: 9760764 9432896 327868
10 minutes after Test App Starts Throwing Exception
steve#steve-ThinkPad-T410:~$ free
total used free shared buffers cached
Mem: 3989340 3770808 218532 0 3420 80632
-/+ buffers/cache: 3686756 302584
Swap: 9760764 9436168 324596
20 minutes after Test App Starts Throwing Exception
steve#steve-ThinkPad-T410:~$ free
total used free shared buffers cached
Mem: 3989340 3770960 218380 0 4376 104716
-/+ buffers/cache: 3661868 327472
Swap: 9760764 9535700 225064
40 minutes after Test App Starts Throwing Exception
steve#steve-ThinkPad-T410:~$ free
total used free shared buffers cached
Mem: 3989340 3739168 250172 0 2272 139108
-/+ buffers/cache: 3597788 391552
Swap: 9760764 9556292 204472

Answer 1

May be you have no more 1MB sequential memory pages in your address space. you have free space fragmentation.

Answer 2

During my test, the above code starts throwing exception when there is around 220M memory. And from the moment on, it looks like the application is not able to get more memory any more because the free memory shown by TOP command remains to be above 210M. So why would this happen?
The output of top is updated every N seconds (configured), and doesn't really show the current status.
On the other hand, memory allocation is super fast.
What happens is your program eats memory, and at certain point (when top shows 200 Mb free) it starts failing.

Answer 3

You are running on x86-32 so your processes are 32-bit. Even with more memory + swap they will be limited by their address space, run:
grep “HIGHMEM” /boot/config-`uname -r`
grep “VMSPLIT” /boot/config-`uname -r`
to see how your kernel is configured.
Maybe your 4 child processes are limited to 3G and are using 12G + other processes ~700M to reach the numbers you are seeing.
EDIT:
So your kernel is configured to give each user-space process 3G of address-space, of which some will be taken up with the program, libraries and initial runtime memory (which will be shared due to the fork).
Therefore you have 4 children using ~3G each - ~12G + ~780M of other programs. That leaves about 220M free once the children start reporting errors.
You could just run another child process, or you could reinstall with AND64/x86-64 version of Ubuntu, where each process will be able to allocate much more memory.