how can I measure memory consumed by process? process quits really quickly so utilities like top are useless. I tried using massif by valgrind, but it measures only memory allocated via malloc/new + stack, and not static variables for example. --pages-as-heap doesn't help as well because it shows mapped memory as well.
Something that might work for you is using a script that will repeatedly run 'ps' immediately after your program starts. I've written up the following script that should work for your purposes, just replace the variables at the top with your specific details. It currently runs 'netstat' in the background (notice the & symbol) and samples the memory 10 times with 0.1 second intervals between the samples, writing the results of the memory checking to a file as it goes. I've run this on cygwin and it works (minus the -o rss,vsz parameters), I don't have access to a linux machine at the moment but it should be simple to adapt if for some reason it doesn't immediately work.
#! /bin/bash
saveFileName=saveFile.txt
userName=jacob
programName=netstat
numberOfSamples="10"
delayBetweenSamples="0.1"
saveFileName=saveFile
i="0"
$programName &
while [ $i -lt $numberOfSamples ]
do
ps -u $userName -o rss,vsz | grep $programName >> $saveFileName
i=$[$i+1]
sleep $delayBetweenSamples
done
If your program completes so fast that the delay between executing it and running ps in the script is too long you might consider running your program with a delay and using a very high sample frequency to try and catch it. You can do that by using 'sleep' and two ampersands like sleep 2 && netstat . That will wait 2 seconds and then run netstat.
If none of this sounds good to you, perhaps try running your program within a debugger. I believe gdb has some memory tracking options you could look into.
Related
I would like to run the t32 cmm script for accessing memory and read/write test. And tried below code.
&srcadd=0x50000000
&destadd=0x50700000
REPEAT 100.
//WHILE (&srcadd<&destadd)
(
D.S ND:0x10:&srcadd %LE %Long 0xdeadface
&srcadd=&srcadd+0x4
)
This code worked for first time and not working on second run, seen different behavior that some of the location not written... random failures if I run the script on X86 core.
Can you please provide any insights on this ?
Also can you tell me how we can store the data from the memory to compare whether its written properly. I am validating internal memory and read/write test. It would be great if we have any sample t32 script for the same.
I've been trying to profile my C++ application in Linux by following this article on perf record. My understanding is all I need to do is run perf record program [program_options], where program is the program executable and [program options] are the arguments I want to pass to the program. However, when I try to profile my application like this:
perf record ./csvJsonTransducer -enable-AVX-deletion test.csv testout.json
perf returns almost immediately with a report. It takes nearly 30 seconds to run./csvJsonTransducer -enable-AVX-deletion test.csv testout.json without perf, though, and I want perf to monitor my program for the entirety of its execution, not return immediately. Why is perf returning so quickly? How can I make it take the entire run of my program into account?
Your commands seems ok. Try change the paranoid level at /proc/sys/kernel/perf_event_paranoid. Setting this parameter to -1 (as root) should solve permission issues:
echo "-1" > /proc/sys/kernel/perf_event_paranoid
You can also try to set the event that you want to monitor with perf record. The default event is cycles (if supported). Check man perf-list.
Try the command:
perf record -e cycles ./csvJsonTransducer -enable-AVX-deletion test.csv testout.json
to force the monitoring of cycles.
This is a sequel to my previous question. I am using fork to create child process. Inside child, I am giving command to run a process as follows:
if((childpid=fork())==0)
{
system("./runBinary ");
exit(1)
}
My runBinary has the functionality of measuring how much time it takes from start to finish.
What amazes me is that when I run runBinary directly on command-line, it takes ~60 seconds. However, when I run it as a child process, it takes more, like ~75 or more. Is there something which I can do or am currently doing wrong, which is leading to this?
Thanks for the help in advance.
MORE DETAILS: I am running on linux RHEL server, with 24 cores. I am measuring CPU time. At a time, I only fork 8 child (sequentially), each of which is bound to different core, using taskset (not shown in code). The system is not loaded except for my own program.
The system() function is to invoke the shell. You can do anything inside it, including running a script. This gives you a lot of flexibility, but it comes with a price: you're loading a shell, and then runBinary inside it. Although I don't think loading the shell would be responsible to so much time difference (15 seconds is a lot, after all), since it doesn't seem you need that - just to run the app - try using something from the exec() family instead.
Without profiling the application, if the parent process which forks has a large memory space, you might find that there is time spent attempting to fork the process itself, and attempts to duplicate the memory space.
This isn't a problem in Red Hat Enterprise Linux 6, but was in earlier versions of Red Hat Enterprise Linux 5.
I need to run a linux command such as "df" from my linux daemon to know free space,used space, total size of the parition and other info. I have options like calling system,exec,popen etc..
But as this each command spawn a new process , is this not possible to run the commands in the same process from which it is invoked?
And at the same time as I need to run this command from a linux daemon, as my daemon should not hold any terminal. Will it effect my daemon behavior?
Or is their any C or C++ standard API for getting the mounted paritions information
There is no standard API, as this is an OS-specific concept.
However,
You can parse /proc/mounts (or /etc/mtab) with (non-portable) getmntent/getmntent_r helper functions.
Using information about mounted filesystems, you can get its statistics with statfs.
You may find it useful to explore the i3status program source code: http://code.stapelberg.de/git/i3status/tree/src/print_disk_info.c
To answer your other questions:
But as this each command spawn a new process , is this not possible to run the commands in the same process from which it is invoked?
No; entire 'commands' are self-contained programs that must run in their own process.
Depending upon how often you wish to execute your programs, fork();exec() is not so bad. There's no hard limits beyond which it would be better to gather data yourself vs executing a helper program. Once a minute, you're probably fine executing the commands. Once a second, you're probably better off gathering the data yourself. I'm not sure where the dividing line is.
And at the same time as I need to run this command from a linux daemon, as my daemon should not hold any terminal. Will it effect my daemon behavior?
If the command calls setsid(2), then open(2) on a terminal without including O_NOCTTY, that terminal might become the controlling terminal for that process. But that wouldn't influence your program, because your program already disowned the terminal when becoming a daemon, and as the child process is a session leader, it cannot change your process's controlling terminal.
I developed a software (in C++) which needs to be continuously running. That basically means that it has to be restarted each time it stops.
I was thinking about using cron jobs to check every minutes if it is still alive, but there might be a cleaner way or standard way of doing this.
Thanks in advance
Fedora and Ubuntu use upstart, which has a the ability to automatically restart your deamon if it exits.
I believe the easiest way to do this is to have a script that will start your program and if it gets returned to it just restarts it.
#!/bin/sh
while true; do
./Your_program
done
Monit can do what you want and much more.
cron is an option if your app will be smart enough to check for itself running (this is to avoid many copies of it running). This is usually done in a standard way via PID files.
There are two proper ways to do it on *nix:
Use the OS infrastructure (like smf/svc on solaris, upstart on Ubuntu, etc...). This is the proper way as you can stop/restart/enable/disable/reconfigure at any time.
Use "respawn" in /etc/inittab (enabled at boot time).
launchtool is a program I used for this purpose, it will monitor your process and restart it as needed, it can also wait a few seconds before reinvocation. This can be useful in case there are sockets that need to be released before the app can start again. It was very useful for my purposes.
Create the program you wish to have running continually as a child of a "watcher" process that re-starts it when it terminates. You can use wait/waitpid (or SIGCHILD) to tell when the child terminates. I would expect someone's written code to do this (it's pretty much what init(8) does)
However, the program presumably does something. You might want not only to check the application is running, but that it is not hung or something and is providing the service that it is intended to. This might mean running some sort of probe or synthetic transaction to check it's operating correctly.
EDIT: You may be able to get init to do this for you - give it a type of 'respawn' in inittab. From the man page:
respawn
The process will be restarted whenever it terminates (e.g. getty).
How about a script that check about every 10 minutes to see if the application is running and if it isn't it will restart the computer. If the application is running, then it just continues to check.
Here is my script using PrcView is a freeware process viewer utility. And I used notepad.exe as my example application that needs to be running, I am not sure the command for checking every 10 minutes and where it would go in my script.
#echo off
PATH=%PATH%;%PROGRAMFILES%\PV;%PROGRAMFILES%\Notepad
PV.EXE notepad.exe >nul
if ERRORLEVEL 1 goto Process_NotFound
:Process_Found
echo Notepad is running
goto END
:Process_NotFound
echo Notepad is not running
shutdown /r /t 50
goto END
:END
This is not so easy. If you're thinking "I know, I'll write a program to watch for my program, or see if such a program already exists as a standard service!" then what if someone kills that program? Who watches the watcher?