Using pprof with gperftools resulting in curl error - profiling

So I have been doing the following:
$ pprof /bin/ls ls.prof
Using local file /bin/ls.
Gathering CPU profile from http://ls.prof/pprof/profile?seconds=30 for 30 seconds to
/home/user/csteifel/pprof/ls.1414597606.ls.prof
Be patient...
curl: (7) couldn't connect to host
Failed to get profile: curl 'http://ls.prof/pprof/profile?seconds=30' > /home/user/csteifel/pprof/.tmp.ls.1414597606.ls.prof: No such file or directory
I'm not sure what is goign on here because this is one of the examples they show here: http://google-perftools.googlecode.com/svn/trunk/doc/cpuprofile.html
Now I understand ls isn't going to actually have information to it but I also know that it shouldn't be giving me an error about curl in this case it should be something else. What am I doing wrong here?
I have also tried doing this to a sample program that I created (eg: pprof --callgrind /home/user/csteifel/testing2/X86_64_DEBUG/el6/wtf ~/testing2/prof.out > callgrind.out and I get a similar error:
Using local file /home/user/csteifel/testing2/X86_64_DEBUG/el6/wtf.
Use of uninitialized value $host in substitution (s///) at /home/user/csteifel/usr/local/lib/bin/pprof line 3195.
Use of uninitialized value $hostport in concatenation (.) or string at /home/user/csteifel/usr/local/lib/bin/pprof line 3197.
Use of uninitialized value $prefix in concatenation (.) or string at /home/user/csteifel/usr/local/lib/bin/pprof line 3197.
Use of uninitialized value $host in substitution (s///) at /home/user/csteifel/usr/local/lib/bin/pprof line 3195.
Use of uninitialized value $hostport in concatenation (.) or string at /home/user/csteifel/usr/local/lib/bin/pprof line 3197.
Use of uninitialized value $prefix in concatenation (.) or string at /home/user/csteifel/usr/local/lib/bin/pprof line 3197.
Use of uninitialized value $host in sprintf at /home/user/csteifel/usr/local/lib/bin/pprof line 3364.
Gathering CPU profile from http:///pprof/profile?seconds=30 for 30 seconds to
/home/user/csteifel/pprof/wtf.1414597016.
Be patient...
curl: (6) Couldn't resolve host 'http:'
Failed to get profile: curl 'http:///pprof/profile?seconds=30' > /home/user/csteifel/pprof/.tmp.wtf.1414597016.: No such file or directory

Quick answer (and the fix to my problem):
If you choose method 1 of running the profiler, with an environment variable, gcc by default just ignores your link, because you're not using any symbols from that library. You need to include it with the -Wl,--no-as-needed flag like so:
-Wl,--no-as-needed -lprofiler -Wl,--as-needed
You can read more about it here.
A more thorough answer with hints for other potential problem:
pprof looks for a local file called ls.prof, which contains info about the runtime of various components of /bin/ls (this is why you compile the program in question with the -g flag, so that it can see the symbols).
Now, why is the file not there? Because it hasn't been generated! Your /bin/ls hasn't been compiled with the -lprofiler flag. If it had been, and you activated the library by one of the two ways listed in the documentation:
Define the environment variable CPUPROFILE to the filename to dump the profile to. For instance, to profile /usr/local/bin/my_binary_compiled_with_libprofiler_so
% env CPUPROFILE=/tmp/mybin.prof /usr/local/bin/my_binary_compiled_with_libprofiler_so
In your code, bracket the code you want profiled in calls to ProfilerStart() and ProfilerStop(). (These functions are declared in .) ProfilerStart() will take the profile-filename as an argument.
If you did either of those, compiled ls with the libraries, every time you run it you'll see something like
% ls -la ~
% <output>
% PROFILE: interrupts/evictions/bytes = 204/0/256
That means the profile file has been generated, and you can now look at it with
% pprof binary_compiled_with_lprofiler profile_file

Related

LLVM14, why i can't use opt --print option?

I build the LLVM from source in release edtion.Now get the loop or cycle information is what i need.But it just can't work.
$opt -passes=print<cycles> input.ll -o /dev/null
zsh: no such file or directory: cycles
The unix command opt -passes=print<cycles> input.ll means to run the command opt -passes=print while reading from the file named cycles and writing to the file named input.ll.
Have a look at the answers to this question for more about this shell syntax.
You may want to use opt -passes='print<cycles>', that will at least not be interpreted by the shell, but I fear that won't work either. I've no idea what print<cycles> is supposed to achieve.

gdb `list` command only shows one line number instead of contents when attaching to a process

I invoked gdb to attach a current running process with this command sudo gdb binary PID. After that I set breakpoints and typed continue in gdb. Then I sent a request to this process to hit the breakpoint. After that, when I type command list, it only shows one line instead of multiple lines as expected, and it only shows line number instead of contents. And may I ask what does the output of command n mean? On the internet some docs mentioned it means the next line to be executed. But from the output itself, it doesn't make too much sense to me(after two n commands the last l command shows 169 instead of 172 or 174). Can anyone help answer above two questions? Really appreciate it.
(gdb) l
164 in CBFEMultiSectionResponseModule.cc
(gdb) n
172 in CBFEMultiSectionResponseModule.cc
(gdb) l
167 in CBFEMultiSectionResponseModule.cc
(gdb) n
174 in CBFEMultiSectionResponseModule.cc
(gdb) l
169 in CBFEMultiSectionResponseModule.cc
The build command line for this source file is like this:
/usr/bin/g++ -c -fPIC -DMODULEADAPTER_BUILTIN_VERSION=\"2.36.375.10894.aff30c2\" -DLINUX -D_GNU_SOURCE -D_THREAD_SAFE -DUSE_STD_YUTSTRING -I../api -I. -I/home/y/include/ydisc \
-I/home/y/include/avro -I../.. -I../../.. -I../../../external_interfaces -I../../../sg_interfaces -I/home/y/include64 -I/home/y/include
\-fPIC -g -O2 -Wall -Werror -Wno-invalid-offsetof -fno-strict-aliasing -pipe -MD -MP
\-DYAHOO_PLATFORM_MAJOR=6 -DYAHOO_PLATFORM_MINOR=10 CBFEMultiSectionResponseModule.cc -o x86_64-linux-gcc/CBFEMultiSectionResponseModule.o
This is the filesystem type:
-bash-4.1$ df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/vda1 ext4 246G 97G 137G 42% /
tmpfs tmpfs 12G 30M 12G 1% /dev/shm
Below is the output of info source:
(gdb) info source
Current source file is CBFEMultiSectionResponseModule.cc
Compilation directory is /home/myusername/ufe/modules/multisectionresponse/impl
Source language is c++.
Compiled with DWARF 2 debugging format.
Does not include preprocessor macro info.
Below is the output of shell cat:
(gdb) shell cat /home/myusername/ufe/modules/multisectionresponse/impl/CBFEMultiSectionResponseModule.cc
cat: /home/myusername/ufe/modules/multisectionresponse/impl/CBFEMultiSectionResponseModule.cc: No such file or directory
when I type command list, it only shows one line instead of multiple lines as expected, and it only shows line number instead of contents
This is most likely happening because GDB has no access to the source. The sudo is the key here. Your source likely resides on a filesystem that doesn't allow root access, such as NFS.
it doesn't make too much sense to me(after two n commands the last l command shows 169 instead of 172 or 174).
You are debugging optimized code. See e.g. this answer.
Update:
The path to source is correct in the compilation environment. However, the runtime environment is different than the compilation env ..
Well, why didn't you tell us that?
My answer is correct: GDB doesn't list source because source is inaccessible (it's just inaccessible for a different reason from what I guessed).
If you want GDB list command to work in the runtime environment, then you must make the source available (though not necessarily in the same location; use dir command to point GDB to the location where sources are available).
Update 2:
. I used to think GDB has some magic ways to get the source code from the binary.
The binary does not contain sources (that would make significantly larger). Instead, it contains references to the source locations.
In particular, the compiler encodes into the binary for each translation unit (each .cpp file):
Compilation directory
Name of the source file(s) (there could be more than one due to #includes).
A mapping from program counter to file/line that the particular chunk of assembly was generated for.
(There is additional info describing variable locations, types, etc. But these are irrelevant to the list command.)
GDB decodes above into, locates the source file(s), and allows you to set breakpoints by file/line, lists the source when you hit the breakpoint, etc.

Determine place of segmentation error CMake

I am trying to figure out where a segmentation error is made. Thesame issue has been solved below for a different compiler. I am using CMake and am wondering if this compiler also has this function. CMake has no -g build option so I have no idea where to look.
Determine the line of code that causes a segmentation fault?
Here is also some info on what build options there are available for the debugger, maybe this points out what debugger I use since I have no idea.
/usr/bin/make: invalid option -- 'g'
Usage: make [options] [target] ...
Options:
-b, -m Ignored for compatibility.
-B, --always-make Unconditionally make all targets.
-C DIRECTORY, --directory=DIRECTORY
Change to DIRECTORY before doing anything.
-d Print lots of debugging information.
--debug[=FLAGS] Print various types of debugging information.
-e, --environment-overrides
Environment variables override makefiles.
--eval=STRING Evaluate STRING as a makefile statement.
-f FILE, --file=FILE, --makefile=FILE
Read FILE as a makefile.
-h, --help Print this message and exit.
-i, --ignore-errors Ignore errors from recipes.
-I DIRECTORY, --include-dir=DIRECTORY
Search DIRECTORY for included makefiles.
-j [N], --jobs[=N] Allow N jobs at once; infinite jobs with no arg.
-k, --keep-going Keep going when some targets can't be made.
-l [N], --load-average[=N], --max-load[=N]
Don't start multiple jobs unless load is below N.
-L, --check-symlink-times Use the latest mtime between symlinks and target.
-n, --just-print, --dry-run, --recon
Don't actually run any recipe; just print them.
-o FILE, --old-file=FILE, --assume-old=FILE
Consider FILE to be very old and don't remake it.
-O[TYPE], --output-sync[=TYPE]
Synchronize output of parallel jobs by TYPE.
-p, --print-data-base Print make's internal database.
-q, --question Run no recipe; exit status says if up to date.
-r, --no-builtin-rules Disable the built-in implicit rules.
-R, --no-builtin-variables Disable the built-in variable settings.
-s, --silent, --quiet Don't echo recipes.
-S, --no-keep-going, --stop
Turns off -k.
-t, --touch Touch targets instead of remaking them.
--trace Print tracing information.
-v, --version Print the version number of make and exit.
-w, --print-directory Print the current directory.
--no-print-directory Turn off -w, even if it was turned on implicitly.
-W FILE, --what-if=FILE, --new-file=FILE, --assume-new=FILE
Consider FILE to be infinitely new.
--warn-undefined-variables Warn when an undefined variable is referenced.
This program built for x86_64-pc-linux-gnu
Report bugs to <bug-make#gnu.org>

Problems with using gperftools on Mac OS X

I have found several conflicting answers over this topic. This blog post requires libuwind, but that doesn't work on Mac OS X. I included #include <google/profiler.h> in my code, however my compiler (g++) could not find the library. I installed gperftools via homebrew. In addition, I found this stackoverflow question showing this:
Then I ran pprof to generate the output:
[hidden ~]$ pprof --text ./a.out cpu.profile
Using local file ./a.out.
Using local file cpu.profile.
Removing __sigtramp from all stack traces.
Total: 282 samples
107 37.9% 37.9% 107 37.9% 0x000000010d72229e
16 5.7% 43.6% 16 5.7% 0x000000010d721a5f
12 4.3% 47.9% 12 4.3% 0x000000010d721de8
...
Running that command (without any of the prior steps) gets me this:
[hidden]$ pprof --text ./a.out cpu.profile
Using remote profile at ./a.out.
Failed to get the number of symbols from http://cpu.profile/pprof/symbol
Why does it try to access an internet site on my machine and a local file on his/hers?
Attempting to link lib profiler as a dry run with g++ gets me:
[hidden]$ g++ -l libprofiler
ld: library not found for -llibprofiler
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I have looked at the man pages, the help option text, the official online guide, blog posts, and many other sources.
I am so confused right now. Can someone help me use gperftools?
The result of my conversation with #osgx was this script. I tried to clean it up a bit. It likely contains quite a few unnecessary options too.
The blog post https://dudefrommangalore.wordpress.com/2012/02/09/profiling-c-code-using-google-performance-tools/ "Profiling C++ code using Google Performance Tools" 2012 by dudefrommangalore missed the essential step.
You should link your program (which you want to be profiled) with cpu profiler library of gperftools library.
Check official manual: http://goog-perftools.sourceforge.net/doc/cpu_profiler.html, part "Linking in the Library"
add -lprofiler to the link-time step for your executable. (It's also probably possible to add in the profiler at run-time using LD_PRELOAD, but this isn't necessarily recommended.)
Second step is to collect the profile, run the code with profiling enabled. In linux world it was done by setting controlling environment variable CPUPROFILE before running:
CPUPROFILE=name_of_profile ./program_to_be_profiled
Third step is to use pprof (google-pprof in ubuntu world). Check that there is not-empty name_of_profile profile file generated; it there is no such file, pprof will try to do remote profile fetch (you see output of such try).
pprof ./program_to_be_profiled name_of_profile
First you need to run your program with profiling enabled.
This is usually first linking your program with libprofiler and then running it with CPUPROFILE=cpu.profile.
I.e.
$ CPUPROFILE=cpu.profile my_program
I think that later step is what you have been missing.
The program will create this cpu.profile file when it exits. And then you can use pprof (preferably from github.com/google/pprof) on it to visualize/analyze.

How to get a coredump from `abrt`

I'm running Fedora 23, and just recently discovered that abrt handles the coredumps from my crashed application, and places all kind of stuff in /var/spool/abrt/ccpp-date-pid. Is there a command to fetch a coredump from abrt, without manually copying it from the indicated folder? Or could I have abrt feed the coredump to gdb, and also load the binary?
I would prefer not to change /proc/sys/kernel/core_pattern.
I'm thinking somehting along the lines of:
$ cc -g -o foo main.c
$ ./foo
segmentation fault (core dumped)
$ abrt-magic d55ba08dd0535a223d4a7...
(gdb) # time to do post mortem debugging...
Where of course abrt-magic would be replaced with some command
Not quite what you want, but you can use abrt-cli list to list the ids and abrt-cli info -d on a given id to get the backtrace. You need to configure abrtd to save info for non-package dumps with:
sed -i 's/ProcessUnpackaged = no/ProcessUnpackaged = yes/' /etc/abrt/abrt-action-save-package-data.conf
You can also get an "old-fashioned" core dump in the usual current directory of the process, if the ulimit -c value allows it, by setting
MakeCompatCore = yes
in config file /etc/abrt/plugins/CCpp.conf.