I have two programs, one in C++, the other in assembler. I want to compare how much memory they use when running respectively. How can I do this?
I am doing the testing on Windows, but I also would like to know how to do it on Linux.
On Linux, try valgrind. It's an amazing tool with too many features for mere mortals to totally comprehend. Have a look at valgrind's massif.
Run the program in one shell. Open another shell and run 'top' command. it will list running processes and home much memory they consume. you can, i guess, poll /proc/yourprocessid/stat to see how much memory it is using over time.
On Windows you can use Performance Monitor.
Performance monitor usage
Start Performance Monitor from Start menu/ Administrative Tools/ Performance
If you want to start the logging:
Select performance log and alert>Current Log option in the left side of the browser.
Select New Log Settings.
Give an apropriate name to the log e.g. performance_Server for Server
It will prompt you one menu. In “general” tabs click on the add button and select the process you want monitor. (Change the performance object to process, for “select counters from list” select “private bytes”, for “select instances from list”, select the process you want to monitor.) After that click on Add and close. Now change the interval as per test case requirement. Now go to “log files” tab change the log file type to either csv or tsv format. Now apply and press OK.
If you want to start/stop the logging:
Select the particular log you want to start and stop.
In toolbar above you will see start and stop button.
If you want to check the content of a log file:
Click Options/Data From…
Select the log file to be viewed, click OK
Go to the chart screen (View/Chart)
Click Edit/Add to chart
Add the required items to the chart. (In case the memory leakage is to be checked, then you need to view the PrivateBytes of the processes and the _Total of them)
Read the values from the chart (Min and Max values are displayed at the bottom of the chart)
If you want to monitor network transfer:
Display the chart screen (View/Chart)
Click Edit/Add to log, and select the items Network Interface\Bytes Sent If you set it in dl
Or Network Inerface\ Bytes Received if you set it in the CRS-PC+
Click Done
Monitor memory usage:
In menu Start/Programs/Administrative Tools/ start the program Performance Monitor
Click on the button to open the window that adds processes
Fill the fields as follows:
Object: Process
Counter: Private Bytes
Instance: a process whose Memory occupation need to be displayed
Click on Add button
Repeat the last two steps for every process the memory needs to be displayed
Close the window that adds processes
On the bottom of the Performance Monitor window, there is the list of the processes previously selected.
How to use the logged data
Now open the file Perfmon_.csv or Perfmon_.tsv using WordPad or Excel.
If you have opened the file using Excel, then using the option Save As, save the file in the Microsoft Excel format.
The Windows Task manager can show you the memory usage of each process. I guess you could use Valgrind instead, but I don't see the point in that. On Linux, use Valgrind or ps.
On Windows you can use the GetProcessMemoryInfo Function.
Here is an example on how to use it:
Collecting Memory Usage Information For a Process
Depends on your operating system - you would expect to have tools to tell you the memory consumed when the apps are running.
Trying to infer the answer by inspecting the code would be very difficult, run the apps, use your platform's diagnoistics.
Depending on the size of the programs, this could be nearly unfeasible.
If they are not very large, then you can see how much memory they allocate; for instance, an int would take up 4 bytes, a char would take 1 byte, etc. Assembly is very transparent in how much memory it is using, even on an x86 machine. Cpp is nearly as transparent, as long as you faithfully track object creation and memory destruction/allocation.
If the program is huge, you'll need to use specific tools for tracking/profiling memory use, like GlowCode (http://www.glowcode.com/summary.htm).
On Windows, you can use Microsoft's Performance Monitor to do it. Start, Run, "perfmon". This tool will report on all sorts of statistics about processes, and provide graphs for you. In general, you're going to be interested in reporting on the "Private working set". This will tell you how much memory your process has reserved for its own use.
If you want to just get your heap usage, and you want to do it programatically, you should look into the CRT Debug Heap.
I'm not sure about Linux though, sorry.
On Windows, I have found Address Space Monitor very useful, especially for looking at how fragmented your memory is.
Related
I'm writing an application that encrypts its data. It can then display it unencrypted using the app's UI after a user enters password. My goal is to minimize exposure of plaintext data while in RAM. For that I want to prevent swapping it to disk as much as possible.
I know that I can adjust my process's working set (by calling SetProcessWorkingSetSize API) and then lock those sensitive pages in RAM (by calling VirtualLock.) That, in theory, should minimize the chances of it being written to disk.
The question I have is, can I do the same with the memory that is used by the common controls in my dialog window, namely in Edit boxes, combo boxes, and most importantly RichEdit control?
PS. I'm assuming that they all use data from the heap for my process. Correct?
EDIT: Seeing all the comments below I need to clarify. By saying "lock", I don't mean, "lock it with a padlock and key so that no one can see it." I meant, lock it as with the VirtualLock API.
You can use EM_SETHANDLE to set a handle for an edit control's initial allocation, then respond to EN_ERRSPACE when (if) it runs out of space and needs more.
From there, it's up to you to also use VirtualLock on that block of memory to keep it in RAM as much as possible. If you're going to do this a lot, you probably want to consider superclassing the control(s) to keep from duplicating the code everywhere.
For better or worse, I don't believe there's an equivalent for rich text controls though.
There is actually a native windows API to help minimize the exposure of crucial items like passwords to memory scraping.
See CryptProtectMemory as a starting point.
I have an application (didn't write it) that is producing APPCRASH dumps in C:\Windows\SysWOW64. The application while dumping is crippled, but operating at bare minimum capacity to not lose data. The issue is that these dumps are so large that the system is spending most of it's time writing these and the application is falling far behind in processing and will start losing data soon.
The plan is to either entirely disable it, or mount it to a RAM drive and purge them as soon as they hit the RAM drive.
Now I've looked into using this key:
http://msdn.microsoft.com/en-us/library/windows/desktop/bb787181%28v=vs.85%29.aspx
But all it does is generate a second dump now instead of redirect the original.
The dump is named:
dump-2013_03_31-15_23_55_772.dmp
This is generally the realm of developers on Windows (with stuff like C/C++) so I'd like to hit them up, don't think ServerFault could get me any answers on this.
Additionally: It's not cycling dump files (they'll fill the 20GBs left on the hard drive), so I'm not sure if this is Windows behavior or custom code in the app (if it is... ick!).
To write a DumpFile, an app has to call the function "MiniDumpWriteDump" so this is not a behavior of the system or something you can control, it is application driven. If it dumps on crashes, it uses "SetUnhandledExceptionFilter" to set its own handling routine, before(!) the OS takes over. Unfortunately I didn't found a way to overwrite this handler from an other process, so the only hope left is, that there is a register entry for the app switching the behavior or change the path (as my applications have it for exactly the reason you describe).
I use AVG and it recently detected a virus. It has before ;) but this was the first time I noticed this.
When I went into the folder containing the virus, AVG immediately, automatically, detected the virus without me even clicking on the application. So I though how could it know a virus was there even when I did not even click (single click) on it.
The only possible answer is that it continuously checks the explorer folder location of all windows and scans all the files in the folder. But how does it see what folder is being viewed by me?
Please explain (if possible) with a C program that does what ever AVG did.
Also : I use Windows if that helps.
When you open a folder a bunch of file system operations is executed (you can use tools like FileMon or ProcMon to take a look at this). Your AV software monitors file access.
There are multiple ways to do this monitoring, e.g. Filter Drivers - you can find a great sample at http://www.codeproject.com/Articles/43586/File-System-Filter-Driver-Tutorial
So when you opened the folder, AV software noticed that you opened a directory, consulted its own data, and informed you about the virus.
I say 'consulted its own data', as AV tools usually don't scan files on access - they do it when the files are written to, as it doesn't make sense to scan files which were marked as clean if they haven't changed since the last scan.
Most virus scanners operate on the principle of API hooks/filters. Whenever windows needs to process a command, like opening a folder, clicking a window, executing a file, etc it generates an api call along with some information like the window coordinates clicked, or a string representing a file. Other programs can request a hook into one or more of these functions which basically says 'instead of executing this function, send it to me first, then I might send it back'. This is how many viruses work (preventing you from deleting them, or copying your keystrokes, for example), how many games/apps work (keyboard, joysticks, drag-and-drop), as well as malware detectors and firewalls.
The latter group hooks the commands, checks any incoming ones to see if they're on the level, then either allows them to resume or blocks them. In this example, opening the folder likely triggered a syscall to parse a directory, and the scanner parsed it too (eg 'realtime protection'). To view all of your hookable functions as well as what is using them, google for a free program called 'sanity check' (previously called 'rootkit hook analyzer'). Most of the red entries will be from either windows firewall or avg, so don't worry too much about what you find.
Looking at Coldfusion's Server Monitor, I see many requests with averages from 100KB to 700KB average memory usage. This seems high to me, is it?
If it is high, it would explain why the server seems to love to eat up RAM and never let it go. What would be a good way of finding out why the memory usage is so high. There is nothing that I can think of that would be using this much memory. Are there any good ways of debugging Coldfusion requests?
We've had good luck with SeeFusion and simply using Jconsole on the servers to monitor memory and threads.
One thing we did find is that during the time in which the data is being rendered to be returned to the user the memory usage was highest. Once the form was displayed memory dropped off significantly. Make sure garbage collection is set properly, as well.
Are you actually getting 'out of memory' exceptions? Or 'out of stack space' errors? Check the logs carefully as I had experience with CF restarted spontaneaouly before - check the server.log files.
If the answer is yes, then - as mentioned - you can use a tool like JConsole to monitor memory usage. If you see that memory goes up and up prior to these exceptions then you need to find out where your memory leak is. The best way to do this is to get a dump of the JVM at that time and use a meomry analysis tool. I documented our experience with this.
If there are no exceptions etc. then be aware than JRun does use a lot of memory. Typically between 500MB and 1.5GB on our Win32 servers even if the CF Admin setting for the JVM heap size is 1024MB. JRun and is running on top of Java, running an application server so this will be high enough. What you need to watch out for is an ever-increasing use of memory using JConsole.
As for GC tuning - only mess with these settings if you really have a problem. It's complicated and will result in server instability if done incorrectly or without understanding of how Java GC works. You can get the basics here, and then you can pass the settings (e.g. -XX:MaxPermSize=192m) as JVM argument in CF admin.
Getting to the bottom of memory issues can be challenging and time consuming so best of luck!
Sounds like a memory leak. You want to generate a JVM heap dump - a snapshot of JVM memory. Easiest way to do so is with jvisualvm. Then analyze that snapshot using the Leaks Report in Eclipse Memory Analyzer. Marc Escher has a blog entry on this topic.
Steps to Setup a JVM JMX Port for getting a ColdFusion Heap Dump
Edit jvm.config and add to java.args: -Dcom.sun.management.jmxremote.port=3333 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=true
Copy \ColdFusion9\runtime\jre\lib\management\jmxremote.access.template -> jxmremote.access
Edit jxmremote.access add "controlRole changeIt" to the end of the file replacing "changeIt" with password of your choice
In Windows Explorer > jxmremote.access > Right Click > Properties > Security > Advanced
Owner > Administrators
Permissions > Uncheck "Allow inheritable...." > Dialog Click "Copy"
Add "Administrators" with Full Control
Remove all other Permission entries
Click "OK"
Restart ColdFusion
Generating a Heap Dump
on ColdFusion box install the JDK
run jvisualvm from jdk-install-dir/bin/jvisualvm
open a JMX connection (e.g. localhost:3333)
on the "monitor" tab click the Heap Dump button and save the result by right clicking its node
take this file and open in Eclipse Memory Analyzer and run the Leaks Report
dig through it, use the Eclipse help to understand how to evaluate
I've used this approach to find a number of memory leaks in ColdFusion code.
Lets say I open some application or process. Did some work with that. Now I closed it.
Need to know whether this application caused any memory leak.
i.e used up some heap memory and not cleared it properly.
Can I get this statistics some how? I'm using Visual Studio (for development) under Windows OS.
Even I would be interested in knowing this information for any 3rd party application.
When an application closes all resources are automatically released by Windows.
A quick & dirty tool to get an indication for memory/resource-leaks inside an application is Perfmon.
The actions executed by an application, can cause other processes to use more memory. SQL Server can make its cache size bigger, maybe you have opened Word or Explorer, the Windows Search engine might kick in because you saved some file. The virus scanner can be more active, etc.....
Have a look at CrtSetDbgFlag:
http://msdn.microsoft.com/en-us/library/5at7yxcs(v=VS.100).aspx