We've been running for years with BoundsChecker for Visual C++ 6 (I think it was BoundsChecker 5 or 6, too). We've upgaded to VS2008 (finally!), and now need a follow-up for the outdated BoundsChecker.
How's the landscape?
What tools are out there?
Any new kids in town?
Any new ideas dealing with the problems we used memory profilers for?
Your recent experiences with these tools?
Recommendations?
The main application is C++ with many COM DLL's, we are looking to track native, C++ and COM leaks and objects. Bounds Checker for that size was already a pain in performance, sorting out the slew of data and some of its limitations.
Support for managed applications (primarily C#) is required, though that may be a separate tool.
Related (but IMO incomplete) question: Modern equivalent of BoundsChecker for Visual Studio 2008
[edit]
Regardign the comment, "In modern C++, you just use self-checking types, and bounds are never broken" :
Reference counted smart pointers can have cyclic references. Interfacing COM components is inherently unsafe, as it requires a lot of manual memory management. I've had a UI-less 3rd party service leak GDI handles so it crashed our overnight tests - the vendor blamed it on a "strange" Microsoft API. I have to interface C-based libraries, I have tons of legacy code that assumes allocation trickery in the sense of Numerical Recipes is a good thing and variable names longer than 3 letters are for typists. I have code from engineers for whom a std::vector<double>::iterator looks much more scary than a double ***, good luck developing and testing these without a solid background in signal processing.
So unless you come here, rewrite and encapsulate the core of a million lines of code in fool-proof C++ classes and make sure a few dozen products still work as before, keep your smart-assery to yourself. I wish I wouldn't need a memory checker, but I do. Thank you.
Disclaimer and warning: I work for Micro Focus, owner of the DevPartner Studio and BoundsChecker products.
BoundsChecker 10.5, part of DevPartner Studio 10.5 (though you can buy it by itself), supports Visual Studio 2005, 2008 and 2010 unmanaged code for 32 and 64 bit applications in essentially the same way it supported 32 bit applications on Visual Studio 6.0. While enhancing it to support X64 applications, we found and fixed quite a few very old problems, and made a start at working in spite of the .NET 4.0 code present in some VS 2010 applications. I say "in spite of" because .NET 4.0 turns out to do a lot of very nasty things in the process space, doing some things that Microsoft warns everybody else not to do, and has a certain amount of built-in resistance to tools like BoundsChecker, which are essentially gigantic viruses.
Anyway, since that release (February 4th), we have updated it to work on Windows 7 SP1 (which isn't quite public yet), and as far as BoundsChecker is concerned, we work with Visual Studio 2010 SP1 as well. We also discovered a nasty .NET 4.0 trap, and figured out to prevent it from taking us down. These enhancements and fixes will be available in our next public update, hopefully within the next month or so.
I have a massive application (here at work), and the new bounds checker 10.5 (supports 64 bit apps now) pretty much works with it. The trick Max is not to turn on all the checker features of devpartner bounds checker at once. Turn on just memory leaks, or turn on just some other feature, then run your app. And by all means exclude modules you don't need. There are quite a few things you can use to tune your settings so it goes faster. But yes, it does take a performance hit. But that is the name of the ballgame.
Intel's Parrallel inspector gave us thousands and thousands of false positives. Didn't use it.
Purify only works on 32 bit apps. i.e. small 32 bit native apps. Forget about using it with a managed C++ app.
And just for the record, if you have a large 32 bit app, memory analysis tools in general won't work very much, because of the massive memory overhead. And since you have very limited memory in a 32 bit address space, you quickly run out of room, and the tools fail.
We evaluated Boundschecker, Intel's Inspector and Purify.
They were all more or less crap.
For our main application, BoundsChecker would not start it after many hours; it only worked for a couple of smaller applications; but find a couple of things (I think we're still in contact with them to figure things out)
Intel's Inspector works, but does not instrument the code, it runs on the executable only (maybe works better when used with the whole suite of Intel products).
Purify failed miserably; we were never able to use it.
We're still in limbo about that.
Max.
Boundschecker: I just bought a (&(^ subscription which only entitles me to use the damned product for 99 days, so I'm pretty damned upset about that) but anyhow I was having big memory troubles and thought I ought to run this thing. It seems to catch lots of interesting things, but is so damned slow that, well put it this way: my appliciation is still in the DLL init code; it has been running for at least a couple hours, and so far it hasn't even gotten as far as the app does normally in the first COUPLE SECONDS. Boundschecker used to be 'the shit' back in numega days, but it seems like it is really another technological orphan being peddled by an opportunitstic business entity, like borland compilers.
So I really like it when it works, it has lots of great info. I just need to see if I will be able to actually get any decent results. It is currently using 4+ GB of RAM and hasn't even started up fully yet. Since I use win7/64 with a crippled home edition which will only recognize 12GB, I may run out of memory before anything really interesting happens. And it will be sometime a few days from now...
Related
Lately I have been developing in C++ with QT Creator. All is well and I'm nearly at the point of packaging and distributing my application. But obviously before any release you better make sure you have everything right. So I'm at the testing stage, and something tells me that I have some slight memory issues. Nothing serious, but I like to obsess over these ;-).
So I decided to try some memory leak detection libraries. For starters I looked at this question. I disregarded Purify and Insure++ because of the costs. Someone else suggested to use the MSVC compiler so I could use the CRT memory validation routines. Which I considered but decided against for now because of the implications it would have on using the debugger from within QT Creator.
Then I stumbled across DUMA which I got to compile using the MinGW32 compiler that comes with QT. However, I found that it doesn't really play well with QT because of the incredible amount of segmentation faults. I know they are how DUMA works, but I'm pretty sure I did not screw things up as much as DUMA tried to make me believe.
Another something I tried was the google-performance-tools which I sadly cannot get to compile with the MinGW32 version supplied with QT, even if I add in the required dependencies.
All of this brings me to my question: Is there any working solution out there for QT Creator developers that want to check their programs for memory leaks?
Thanks,
Personally, I find the valgrind / memcheck / callgrind / kcachegrind combo to be too powerful to pass up for memory leaks and performance analysis. Being free (gratis) is especially nice since I cannot personally afford the cost of some of these other professional tools.
I've been working on a cross platform Qt application for a couple years now. As I develop, I bounce back and forth between windows and linux when I want to test features and functionality. However, I always do my performance analysis in linux. The memory leaks and performance concerns that I find are always ones that are injected by my team of developers on top of Qt. So when these problems are fixed, regardless of being identified in Linux, the benefits will carry over into windows.
I suggest you investigate setting up a pure linux build environment that you can test in. Perhaps the easiest solution for you would be to create a virtual machine to do your performance analysis on. My preferred combination is VirtualBox and Ubuntu.
May be you can use these libs for windows
Hooks for Malloc [gnu.org]
Backtrace [gnu.org]
valgrind [valgrind.org]
As I'm more and more dissapointed with VS 2010 I'm trying to find some alternative and I was looking at Embarcadero's new edition of C++ env.
Is there any point of learning new (not popular I think) product when VS practically dominates market?
Thanks.
Although I'm not really a Windows programmer, I have been using Borland/Embarcadero to-and-fro during the past 10 years. Here are my personal opinions of why you should not consider it:
The general quality of Builder has dropped significantly over the years. Borland Builder 6 was the last high-quality product, from there the IDE itself has become more and more buggy. The IDE typically crashes once per 1-2 weeks of usage.
No undo in the RAD design. Yes I know, it is quite unbelievable. Even the earliest versions of utter crap like VB had this. But Builder year 2011 doesn't! If you slip on your keyboard and accidentally alter a component, you shall be punished!
The debugger is next to useless. This might have been fixed in the latest version, but in several versions you can't single step through the program without collapsing struct/class variables in the watch window, which is of course very frustrating.
Documentation is very poor, often non-existant, and may be written in Object Pascal, even though you ordered the C++ IDE. The help files also have a tendency to linger as evil ghost processes in your computer, making it impossible to shut down Windows before the ghost is busted.
Personally I'm considering switching to Visual Studio.
I've been using both the Embarcadero Borland, now RAD Studio 2010, c++ and VS2008 every day for the last 6 months. My programming philosophy has always been to use the right tool for the project, no matter what that particular tool is. So a couple of my observations/opinions are -
Advantages
The WYSIWYG screen designer is good. It acts a lot like the WinForms editor in VS2008, but for c++. In VS2008, the only package for c++ that I've used that is close is Qt. My biggest compliant is documentation, but that applies to most software, so it isn't just their problem.
For many builtin classes, they are built on TObject class. This base class functions a lot like Object in C#. The biggest advantage this gives you as a c++ programmer, if you follow a few rules, is mostly automatic memory management. It's not garbage collection, rather, list of related objects that are deleted together.
Disadvantages-
The RAD Studio 2010 C++ environment exists primarily to support Delphi. That is their real strength anyway. Nowhere does anything say this. It is just an overall feel that I've gotten from using system.
Limited support for 3rd party libraries.
It cannot link with any Microsoft compatible c++ library. This includes both Microsoft and 3rd party libraries. They use a different "name mangling" format from Microsoft. So everything has to wrapped in a c language wrapper.
We use the Boost Libraries a lot in VS2008. But in Rad Studio, it only has limited support for Boost.
I've found the overall speed of the generated code to significantly slower than that produced by VS2008.
Please remember, that these are just one person's opinions.
I would suggest that you download a demo version of the product and try it for yourself.
If you want to be 100% up-to-date, you have to use the development environment provided by the platform's vendor.
If you do not mind waiting few months/years for new things to get ported over (or your market allows for it) then you surely you can venture into the unknown.
It's not that Borland's IDEs (unlike MS' VS family) needed any advanced training to start using them and be already productive. That is the main reason why they remain popular in many niches.
Any suggestions? This SO post talks about Visual Leak Detector, but I'm looking for other tools. Also, please don't recommend this.
I've used Visual Leak Detector quite recently and it works pretty well. It's very simple to build it in (just include vld.h) and doesn't require any instrumentation. I found that I needed to write a couple of scripts though to parse the output into something more useful for my purposes (it is crying out for a decent GUI). Note the new website -- the project is being actively maintained on Codeplex -- the Codeproject link is very old.
Try Deleaker or DevPartner, these both are good enough.
update:
C++ Memory Validator, works fine and fairly priced.
If you can afford some money use the Intel Parallel Inspector (https://www.intel.com/content/www/us/en/developer/tools/oneapi/inspector.html).
It doesn't require any code modifications and provides good reports.
I had to look for good tools to do that in work and that's the best tool (by far) that I found.
As suggested by Noah Roberts, i too felt Memory validator as the best tool for detecting leaks in Visual studio.
Purify also works.
Asked a similar question not long ago but related to 2010 specifically: VS2010 profiler/leak detection
If you're not using 2010 you might consider AQtime. It's not too terrible most the time. There are times thought when it fails to find an object related to the leak and just says "crt memory" or whatever.
Another I recently tried is called "Memory Validator". Not exactly a nice interface but it does seem to work, and work with 2010 too (just doesn't have any other kind of profiling in it, which is something I expect in a commercial tool like that).
Maybe GlowCode as was mentioned in other, similar questions, on SO when I searched before asking.
There is also application verifier. It can track a whole bunch of other issues as well apart from leaks like places where you forget to free win32 objects such as handles etc ...
The MSDN link is: http://msdn.microsoft.com/en-us/library/ms220948(VS.80).aspx
Jacob, the BoundsChecker tool inside DevPartner Studio contains two types of leak detectors for VC++. The uninstrumented version will tabulate leaks at process shutdown. The compile time instrumented detector will break on leaks at the time the leak occurrs showing both the current call stack at the point of leakage, and the allocation stack trace showing where the leaking memory was allocated. The 64-bit version of DPS and BoundsChecker 10.5 ships February 4, 2011 with x64 application support for Visual Studio 2010, 2008, and 2005. We also have a new pricing model so you can license just the BoundsChecker features and ugrade to the full DPS suite only if you find you need the other profilers or static analysis capabilities later on. Be sure to haggle with your sales rep too. Paying list price is like paying sticker price on a new car. Shameless plug: I work on the DevPartner team. DPS 10.5 represents my team's biggest release of tech capability since Compuware closed the NuMega lab. Please evaluate it for your own application and tech stack when 10.5 goes live. Cheers. Matt Schuetze, DevPartner Product Owner (and yes that means PO in normal Scrum team roles.)
VLD cannot be trusted for more serious projects.
Intel has a good tool called Inspector.
After 3 days searching for a good tool to find leaks on x64 binaries with C++ in Visual Studio, this is the only good option I found.
Try Dr.Memory.
They have proper visual studio integration and documentation is excellent.
Easy to use too.
https://sourceforge.net/projects/diagnostic/
http://diagnostic.sourceforge.net/index.html
I had tried so many memory leak detectors that it's difficult to count them all. Some of them crashed, some of them produced invalid results, some of them simply did not help. Eventually I've made my own leak detector, but invested so much effort into it - so made it non-free for time being. Managed / native / 32 & 64-bit architectures supported.
If you ever want to develop this tool further, please feel free to contact me.
This question already has answers here:
What are some good profilers for native C++ on Windows? [closed]
(8 answers)
Closed 9 years ago.
Does windows have any decent sampling (eg. non-instrumenting) profilers available? Preferably something akin to Shark on MacOS, although i am willing to accept that i am going to have to pay for such a profiler on windows.
I've tried the profiler in VS Team Suite and was not overly impressed, and was wondering if there were any other good ones.
[Edit: Erk, i forgot to say this is for C/C++, rather than .NET -- sorry for any confusion]
For Windows, check out the free Xperf that ships with the Windows SDK. It uses sampled profile, has some useful UI, & does not require instrumentation. Quite useful for tracking down performance problems. You can answer questions like:
Who is using the most CPU? Drill down to function name using call stacks.
Who is allocating the most memory?
Outstanding memory allocations (leaks)
Who is doing the most registry queries?
Disk writes? etc.
I know I'm adding my answer months after this question was asked, but I thought I'd point out a decent, open-source profiler: Very Sleepy.
It doesn't have the feature count that some of the other profilers mentioned before do, but it's a pretty respectable sampling profiler that will work very well in most situations.
Intel VTune is good and is non-instrumenting. We evaluated a whole bunch of profilers for Windows, and this was the best for working with driver code (though it does unmanaged user level code as well). A particular strength is that it reads all the Intel processor performance counters, so you can get a good understanding of why your code is running slowly, and it was useful for putting prefetch instructions into our code and sorting out data layout to work well with the cache lines, and the way cache lines get invalidated in multi core systems.
It is commercial, and I have to say it isn't the easiest UI in the world.
AMD's CodeAnalyst is FREE here
We use both VTune and AQTime, and I can vouch for both. Which works best for you depends on your needs. Both have free trial versions - I suggest you give them a go.
The Windows Driver Kit includes a non-instrumenting user/kernel sampling profiler called "kernrate". It seems useful for profiling multi-process applications, applications that spend most of their time in the kernel, and device drivers (of course). It's also available in the KrView (Kernrate Viewer) and Windows Server 2003 Resource Kit Tools packages.
Kernrate works on Windows 2000 and later (unlike Xperf, which requires Vista / Server 2008). It's command-line based and the documentation has a somewhat intimidating list of options. I'm not sure if it can record call stacks or just the program counter. If you use a symbol server, make sure to put an up-to-date dbghelp.dll and symsrv.dll in the same directory as kernrate.exe to prevent it from using the ancient version of dbghelp.dll that is installed in %SystemRoot%\system32.
I have tried Intel's vtune with a rather large project about two years ago. It was an instrumenting profiler then and it took so long to instrument the DLL that I was attempting to profile that I eventually lost patience after an hour.
The one tool that I have had quite good success and which i would highly recommend is that of AQTime. It not only provides excellent performance profiling resources but it also doe really good memory profiling which has been of significant help to me in tracking down memory leaks.
Luke Stackwalker seems promising -- it's not as polished as I'd like, but it is open source and it does do something that seems very close to what #Mike Dunlavey keeps saying we ought to do. (Of course, it then tries to smoosh it all down into the typically-unhelpful call graphs that Mike is so weary of, but it shouldn't be too hard to fix that with the source as our ally.)
It even seems to count time spent waiting in the kernel, as far as I can tell...
I'm not sure what a non-instrumenting profiler is, but I can say for .NET I love RedGate's ANTS Profiler. Version 3 beats the MS version for ease of use and Version 4, which allows arbitrary time slices, makes MS look like a joke.
What good profilers do you know?
What is a good way to measure and tweak the performance of a C++ MFC application?
Is Analysis of algorithms really neccesary? http://en.wikipedia.org/wiki/Algorithm_analysis
I strongly recommend AQTime if you are staying on the Windows platform. It comes with a load of profilers, including static code analysis, and works with most important Windows compilers and systems, including Visual C++, .NET, Delphi, Borland C++, Intel C++ and even gcc. And it integrates into Visual Studio, but can also be used standalone. I love it.
If you're (still) using Visual C++ 6.0, I suggest using the built-in profiler. For more recent versions you could try Compuware DevPartner Performance Analysis Community Edition.
For Windows, check out Xperf which ships free with the Windows SDK. It uses sampled profile, has some useful UI, & does not require instrumentation. Quite useful for tracking down performance problems. You can answer questions like:
Who is using the most CPU? Drill down to function name using call stacks.
Who is allocating the most memory?
Who is doing the most registry queries?
Disk writes? etc.
You will be quite surprised when you find the bottlenecks, as they are probably not where you expected!
It's been a while since I profiled unmanaged code, but when I did I had good results with Intel's vtune. I'm sure somebody else will tell us if that's been overtaken.
Algorithmic analysis has the potential to improve your performance more profoundly than anything you'll find with a profiler, but only for certain classes of application. If you operate over reasonably large sets of data, algorithmic analysis might find ways to be more efficient in CPU/Memory/both, but if your app is mainly form-fill with a relational database for storage, it might not offer you much.
Intel Thread Checker via Vtune performance analyzer- Check this picture for the view i use the most that tells me which function eats up the most of my time.
I can further drill down inside and decompose which functions inside them eats up more time etc. There are different views based on what you are watching (total time = time within fn + children), self time (time spent only in code running inside the function etc).
This tool does a lot more than profiling but i haven't explored them all. I would definitely recommend it. The tool is also available for downloading as a fully functional trial version that can run for 30 days. If you have cost constraints, i would say this window is all that you require to pin point your problem.
Trial download here - https://registrationcenter.intel.com/RegCenter/AutoGen.aspx?ProductID=907&AccountID=&ProgramID=&RequestDt=&rm=EVAL&lang=
ps : I have also played with Rational Rational but for some reason I did not take much to it. I suspect Rational might be more expensive than Intel too.
Tools (like true time from DevPartner) that let you see hit counts for source lines let you quickly find algorithms that have bad 'Big O' complexity. You still have to analyse the algorithm to determine how to reduce the complexity.
I second AQTime, having both AQTime and Compuwares DevPartner, for most cases. The reason being that AQTime will profile any executable that has a valid PDB file, whereas TrueTime requires you to make an instrumented build. This greatly speeds up and simplifies ad hoc profiling. DevPartner is also quite a bit more expensive if this is an issue. Where DevPartner comes into its own is with BoundsChecker, which I still rate as a better tool for catching leaks and overwrites than AQTimes execution profiler. TrueTime can be slighly more accurate than AQTime, but I have never found this to be an issue.
Is profiling worthwhile, IMO yes, if you need performance gains on a local application. I think you also learn a lot about how your program and algorithms really work, and the cost implications of using certain types of object classes for storing and iterating through your data.
Glowcode is a very nice profiler (when it works). It can attach to a running program and requires only symbol files - you don't need to rebuild.
Some versions pf visual studio 2005 (and maybe 2008) actually come with a pretty good performance profiler.
if you have it it should be available under the tools menu
or you can search for a way to open the "performance explorer" window to start a new performance session.
A link to MSDN
FYI, Some versions of Visual Studio only come with a non-optimizing compiler. For one of my my MFC apps if I compile it with MINGW/MSYS ( gcc compiler ) with -o3 then it runs about 5-10x as fast as my release compile with Visual Studio.
For example I have an openstreetmap xml compiler and it takes about 3 minutes ( the gcc compiled version) to process a 2.7GB xml file. My visual studio compile of the same code takes about 18 minutes to run.