I am interested to learn about KVM. I started trace-cmd as follows:
trace-cmd record -p function_graph -g kvm_vcpu_ioctl -o nofuncgraph-irqs qemu-system-x86_64 ***
I analyzed the output file using Kernelshark. It is my first time using it, so I might not know how to use it properly. Anyway I found it hard to find the start and end of function calls.
What I am interested in is something like seeing the functions either as a flamegraph, but I am not interested at timing at all, I am only interested in hierarchy. Or seeing it in a way similar to an IDE where I can collapse those block of functions that I am not interested in.
Is there a way to generate a flamegraph out of the trace-cmd output or a tool to help with visualizing the call graph that is generated?
Related
Situation:
I'm attempting to get coverage reports for a project that uses both C++ and Python. I'm using LCOV/GCOV for C++, and attempting to use Coverage.py for the python stuff. The only issue is, most of the python code that's being used is simply utility functions being called one function at a time. No initialization, no real life-cycle, or exit. So no real way to use the API to start/stop/save, or use the coverage command line to measure.
With this, I thought the easiest way to accomplish this would be using the sitecustomize.py method like outlined here. I have gotten that to work, and it measures all configured python code as expected. Now I'm looking at how to accomplish this with compiled python code (.pyc).
I can get it to work if I keep source(.py) and (.pyc) in the same directory when running, and then reporting. However, I'm looking for a way to RUN the files and generate the measurement data. Then at a later time point to the actual source files, and run the actual reports. Ideally I wouldn't need the source(.py) files at all, but I haven't found a way to accomplish this.
Objective:
In the end I want to be able to compile the python files(.pyc), install them on the target, and run coverage like stated above. It will generate coverage data files, then pull those files to my host machine which houses the source(.py) .. and do the actual coverage reporting.
Is this possible currently?
[Edit] Thanks to Ned's advice, I looked into the [paths] usage, and it worked exactly how I needed it to.
I'm attempting to build a large legacy code base that has troubles building under new toolchain. In order to speed up fixing problems, I run
make -k
to build everything that can be built, so that I can later focus on unbuildable stuff. But even then a single make takes a minute to figure out the next problem to work on (this code base uses a tangled mess of Makefiles which take ages to parse).
Is there any way to list all targets that failed during a single make -k run?
I'd redirect the make -k output to a file and then look for the error patterns in it. I use vim and I'm typically looking for these:
make:\ \*\*\*
\*\*\*\ \[
A (custom) log parser can be written as well as needed.
When debugging, it is also worth to look out for synchronization irregularities, where part of the stderr message may be missing!
I am interested in testing speed of some function calls from the code written in C/C++. I searched and I was directed to use Valgrind platform with Callgrind tool.
I have briefly read the manual, but I am still wondering how I can utilize the functionality of the tool to evaluate the time of my function runtime speed.
I was wondering if I could get some pointers how I can achieve my goal.
Any help would be appreciated.
Compile your program with debug symbols (e.g. GDB symbols works fine, which are activated with the "-ggdb" flag).
If you are executing your program like this:
./program
Then run it with Valgrind+Callgrind with this command:
valgrind --tool=callgrind ./program
Callgrind will then produce a file called callgrind.out.1234 (1234 is the process ID and will probably be different when you run). Open this file with:
cg_annotate callgrind.out.1234
You may want to use grep to extract your function name. In the left column the number of instructions used for that function is displayed. Functions that use a comparatively low amount of instructions will be ignored, though.
If you want to see the output some nice graphics, I would recommend you to install KCachegrind.
I'm using a Physics Toolkit (Geant4) which consists of several thousand C++ header and class files. In order to use the toolkit, you have to write a series of your own class files, that give the toolkit some basic information about what you are trying to model. You then write a main() file which registers these files with the toolkit, 'make' it and then execute the final program. I am using Ubuntu 10.10 as the platform to do this.
I'd like to better understand how the toolkit operates. Specifically, I'd like to find out what functions in what class files, throughout the toolkit, are called and in what order, when the program runs.
One, somewhat brute-force method would be to label each function in each file e.g. insert cout << "File Name, Function Name" << endl as the first statement in each function body and have this all output to a textfile. However, there are some 3000 files I'd need to go through, which would be somewhat... time-consuming.
Is there an easier way of finding out what functions have been called? I've searched through the toolkits manual and, unless I have missed something, I see no way there of doing this via the toolkit. I guess I would need some command at the terminal or an external program?!?
Any help, suggestions or advice would be greatly appreciated!
On ubuntu you'll have a choice of profilers.
I personally love
valgrind --tool=callgrind ./myprogram
kcachegrind
For this because it creates very good callgraphs and statistics (tree map visualizations).
The big FAQ profiler topic is here: How can I profile C++ code running in Linux?
Off the top off my head: gprof (needs instrumentation), oprofile and perf record -g are easy to get started with as well
I have frequently encounter the following debugging scenario:
Tester provide some reproduce steps for a bug. And to find out where the problem is, I try to play with these reproduce steps to get the minimum necessary reproduce steps. Sometimes, luckily I found that when do a minor change to the steps, the problem is gone.
Then the job turns to find the difference in code workflow between these two reproduce steps. This job is tedious and painful especially when you are working on a large code base and it go through a lot code and involve lots of state changes which you are not familiar with.
So I was wondering is there any tools available to compare "code workflow". As I've learned the "wt" command in WinDbg, I thought it might be possible to do it. For example, I can run the "wt" command on some out most functions with 2 different reproduce steps and then compare the difference between outputs. Then it should be easy to found where the code flow starts to diverge.
But the problem with WinDBG is "wt" is quite slow (maybe I should use a log file instead of output to screen) and not very user-friendly (compared with visual studio debugger) ... So I want to ask you guys is there any existing tools available . or is it possible and difficult to develop a "plug-in" for visual studio debugger to support this functionality ?
Thanks
I'd run it under a profiler in "coverage" mode, then use diff on the results to see which parts of the code were executed in one run by not the other.
Sorry, I don't know of a tool which can do what you want, but even if it existed it doesn't sound like the quickest approach to finding out where the lower layer code is failing.
I would recommend to instrument your layer's code with high-level logs so you can know which module fails, stalls, etc. In debug, your logger can write to file, to output debug window, etc.
In general, failing fast and using exceptions are good ways to find out easily where things go bad.
Doing something after the fact is not going to cut it, since your problem is reproducing it.
The issue with bugs is seldom some interal wackiness but usually what the user's actually doing. If you log all the commands that the user enters then they can simply send you the log. You can substitute button clicks, mouse selects, etc. This will have some cost but certainly much less than something that keeps track of every method visited.
I am assuming that if you have a large application that you have good logging or tracing.
I work on a large server product with over 40 processes and over one million lines of code. Most of the time the error in the trace file is enough to identify the location of problem. However sometimes the error I see in the trace file is caused by some earlier code and the reason for this can be hard to spot. Then I use a comparative debugging technique:
Reproduce the first scenario, copy the trace to a new file (if the application is multi threaded ensure you only have the trace for the thread that does the work).
Reproduce the second scenario, copy the trace to a new file.
Remove the timestamps from the log files (I use awk or sed for this).
Compare the log files with winmerge or similar, to see where and how they diverge.
This technique can be a little time consuming, but is much quicker than stepping through thousand of lines in the debugger.
Another useful technique is producing uml sequence diagrams from trace files. For this you need the function entry and exit positions logged consistently. Then write a small script to parse your trace files and use sequence.jar to produce uml diagrams as png files. This is a great way to understand the logic of code you haven't touched in a while. I wrapped a small awk script in a batch file, I just provide trace file and line number to start then it untangles the threads and generates the input text to sequence.jar then runs its to create the uml diagram.