I wrote a project in c++ with 10 threads. One thread loads the data into memory(write the buffer) and other 9 threads are simultaneously read the buffer and store data in SQLite database, All threads are handled with the mutex to avoid conflicts.
Now I need to evaluate the performance of this project such as time to success per threads, memory usages etc. How can I go it in c++ environment? I used Valgrind to check these. But I think it not working.
This is the code I run with Valgrind,
valgrind --tool=memcheck --leak-check=yes ./executable
It gives a message like this,
callers=20 --track-fds=yes ./monerosci
==24262== Memcheck, a memory error detector
==24262== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==24262== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==24262== Command: ./monerosci
==24262==
valgrind: m_syswrap/syswrap-linux.c:5361
(vgSysWrap_linux_sys_fcntl_before): Assertion 'Unimplemented
functionality' failed.
valgrind: valgrind
host stacktrace:
==24262== at 0x38083F48: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==24262== by 0x38084064: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==24262== by 0x380841F1: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==24262== by 0x380FB399: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==24262== by 0x380D6234: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==24262== by 0x380D2D2A: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==24262== by 0x380D43DE: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==24262== by 0x380E3946: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
How can I test the performance of the project in C++?
Well it seems there's two separate problems here:
1) memcheck is failing to run due to a bug or some limitation. Apparently one variation of a fcntl call is not supported by your version of valgrind. Maybe you should reduce the code size, remove libraries, until you can pinpoint which call is triggering this problem. Or just run it under a different version of valgrind. However, I think memcheck will not give you the data you want...
2) memcheck is not a tool for profiling. Valgrind is composed of several different tools that can be switched by using the --tool parameter. Here's an overview of them. The one that most likely will give you the info you want is callgrind.
Related
I am trying to profile the performance of RocksDB using Callgrind / KCacheGrind on a mac. I am running the command valgrind --tool=callgrind ./simple_example on one of the example programs that comes with RocksDB in the examples folder. I seem to be getting a memory leak, though, which prevents me from being able to do the performance profiling that I ultimately want to do.
==54628== Callgrind, a call-graph generating cache profiler
==54628== Copyright (C) 2002-2017, and GNU GPL'd, by Josef Weidendorfer et al.
==54628== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==54628== Command: ./simple_example
==54628==
==54628== For interactive control, run 'callgrind_control -h'.
==54628==
==54628== Process terminating with default action of signal 11 (SIGSEGV)
==54628== Access not within mapped region at address 0x18
==54628== at 0x1016D25BA: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib)
==54628== by 0x1016D250C: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib)
==54628== by 0x1016D1BF8: thread_start (in /usr/lib/system/libsystem_pthread.dylib)
==54628== If you believe this happened as a result of a stack
==54628== overflow in your program's main thread (unlikely but
==54628== possible), you can try to increase the size of the
==54628== main thread stack using the --main-stacksize= flag.
==54628== The main thread stack size used in this run was 8388608.
--54628:0:schedule VG_(sema_down): read returned -4
==54628==
==54628== Events : Ir
==54628== Collected : 14961779
==54628==
==54628== I refs: 14,961,779
Segmentation fault: 11
I'm working on a program on a CentOS 7.5.1804 using the g++ 6.3.0 compiler
When I run the program, it returns a error:
-bash-4.2$ ./lost_drv
*** Error in `./lost_drv': free(): invalid pointer: 0x0000000002375cf8 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81499)[0x7f5c65e85499]
./lost_drv(_ZN5boost10filesystem4pathD2Ev+0x18)[0x521564]
/lib64/libc.so.6(__cxa_finalize+0x9a)[0x7f5c65e3df4a]
/lib64/libboost_filesystem.so.1.53.0(+0x7573)[0x7f5c68317573]
======= Memory map: ========
Valgrind reveals it to be an issue with the boost::filesystem::path destructor
-bash-4.2$ valgrind ./lost_drv
==32523== Memcheck, a memory error detector
==32523== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==32523== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==32523== Command: ./lost_drv
==32523==
==32523== Invalid free() / delete / delete[] / realloc()
==32523== at 0x4C2B1CD: operator delete(void*) (vg_replace_malloc.c:576)
==32523== by 0x521563: boost::filesystem::path::~path() (path.hpp:55)
==32523== by 0x73DBF49: __cxa_finalize (in /usr/lib64/libc-2.17.so)
==32523== by 0x5053572: ??? (in /usr/lib64/libboost_filesystem.so.1.53.0)
==32523== by 0x40101C9: _dl_fini (in /usr/lib64/ld-2.17.so)
==32523== by 0x73DBBD8: __run_exit_handlers (in /usr/lib64/libc-2.17.so)
==32523== by 0x73DBC26: exit (in /usr/lib64/libc-2.17.so)
==32523== by 0x73C444B: (below main) (in /usr/lib64/libc-2.17.so)
The error occurs after the final return statement, as everything is getting cleaned up. The program apparently works OK prior to that.
This is an extensive program, but I've simplified it a bit by putting a return statement immediately following the main statement:
int main(int argc, char *argv[])
{
return 0;
...
I am using boost::filesystem to check path integrity ... code like this:
try
{
boost::filesystem::path fn(ifilenm);
if(!boost::filesystem::exists(fn) || !boost::filesystem::is_regular_file(fn))
{
cout << "ERROR: Input file does not exist or isn't a file:\t" << ifilenm << "\n";
return -2;
}
}
catch (const boost::filesystem::filesystem_error& ex)
{
cout << "ERROR: Failed accessing input file. File doesn't exist or isn't a file or isn't accesible:\t" << ifilenm << "\n";
return -3;
}
Which isn't being run in this degenerate case of course. There aren't any global boost variables being initialized before execution either.
I've tried this with the system boost install (1.53) as well as static & dynamic links against boost 1.59 built with the g++ 6.3 compiler. Same behavior.
One more bit ... there's another version of this executable which accesses some additional functionality via linking against some other 3rd party libraries. That build doesn't have this problem.
One other data point - if I rebuild the whole business with the default CentOS 7 compiler (g++ 4.8.5), I don't get this error. But that's not an option overall, because the 3rd party libraries used for the full monty require g++ 6
Any idea what's going on?
Thanks
I have a program that I know has some weird memory issues going on, so I turned to Valgrind. However, I am getting the following mysterious output:
==32006== Memcheck, a memory error detector
==32006== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==32006== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==32006== Command: ./012
==32006== Parent PID: 29454
==32006==
valgrind: the 'impossible' happened:
Unsupported arch_prctl option
host stacktrace:
==32006== at 0x580441BA: show_sched_status_wrk (m_libcassert.c:355)
==32006== by 0x580442D4: report_and_quit (m_libcassert.c:426)
==32006== by 0x58044517: panic (m_libcassert.c:502)
==32006== by 0x58044517: vgPlain_core_panic_at (m_libcassert.c:507)
==32006== by 0x5804454A: vgPlain_core_panic (m_libcassert.c:512)
==32006== by 0x580DAE22: vgSysWrap_amd64_linux_sys_arch_prctl_before (syswrap-amd64-linux.c:286)
==32006== by 0x580A0C23: vgPlain_client_syscall (syswrap-main.c:1857)
==32006== by 0x5809D48A: handle_syscall (scheduler.c:1126)
==32006== by 0x5809EBB6: vgPlain_scheduler (scheduler.c:1443)
==32006== by 0x580AED50: thread_wrapper (syswrap-linux.c:103)
==32006== by 0x580AED50: run_a_thread_NORETURN (syswrap-linux.c:156)
sched status:
running_tid=1
Thread 1: status = VgTs_Runnable (lwpid 32006)
==32006== at 0x401A1C5: ??? (in /usr/lib/ld-2.28.so)
==32006== by 0xBFEBFBFE: ???
Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.
If that doesn't help, please report this bug to: www.valgrind.org
In the bug report, send all the above text, the valgrind
version, and what OS and version you are using. Thanks.
The original bug I am trying to diagnose occurs when I am trying to insert uint64_ts into a std::set based on values in a std::vector (not a pointer in sight).
Try installing valgrind-git package from AUR and it may work
I have compiled a program using Code::Blocks. I have turned on "produce debugging symbols" under the "Debug" target, and also turned off "strip all symbols..." But when I run the program with Valgrind I get question marks in the output:
$ valgrind --leak-check=yes --track-origins=yes --log-file=valgrind_output.txt
~/bin/myprg
==3766== Memcheck, a memory error detector
==3766== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==3766== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==3766== Command: /home/xxxxxx/bin/myprg
==3766== Parent PID: 3209
==3766==
==3766== Warning: client switching stacks? SP change: 0xffefff978 --> 0xffed13da0
==3766== to suppress, use: --max-stackframe=3062744 or greater
==3766== Invalid write of size 4
==3766== at 0x40892B: ??? (in /home/xxxxxx/bin/myprg)
==3766== by 0x40275C: ??? (in /home/xxxxxx/bin/myprg)
==3766== by 0x56FB82F: (below main) (libc-start.c:291)
==3766== Address 0xffed13ddc is on thread 1's stack
==3766==
==3766== Invalid write of size 4
==3766== at 0x408931: ??? (in /home/xxxxxx/bin/myprg)
==3766== by 0x40275C: ??? (in /home/xxxxxx/bin/myprg)
==3766== by 0x56FB82F: (below main) (libc-start.c:291)
==3766== Address 0xffed13dd4 is on thread 1's stack
==3766==
...
What is the meaning of this output, and how do I find the piece of code that is causing this error?
Update: Solution
The problem was with Code::Blocks. It is necessary to correctly configure the project build options for the whole project and not just the "Debug" target. So all flags except "-std=c++11" were removed from the "whole project" options, so nothing was overriding the "Debug" options. Also the linker ".o" files need to be deleted when the options are changed, to force Code::Blocks to rebuild the executable.
The code needs to be compiled and linked with debug info (-g command line option) and -fno-omit-frame-pointer for valgrind to show correct stack traces.
See The stack traces given by Memcheck (or another tool) aren't helpful. How can I improve them? for more details.
I recently had this problem and was able to resolve it by using the "--keep-debuginfo=yes" option, as suggested by the FAQ on this page.
When running Valgrind's memcheck, occasionally valgrind report error like this:
==2745== Memcheck, a memory error detector
==2745== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==2745== Using Valgrind-3.6.0 and LibVEX; rerun with -h for copyright info
==2745== Command: ./HSFramework
==2745==
==2745== Invalid write of size 8
==2745== at 0x3B81C097C0: do_lookup_x (in /lib64/ld-2.12.so)
==2745== by 0x1C31032D: ???
==2745== by 0x3B81C09E19: _dl_lookup_symbol_x (in /lib64/ld-2.12.so)
==2745== Address 0x7feffee78 is on thread 1's stack
==2745==
platform: Linux 2.6.32-220.el6.x86_64 x86_64 x86_64 x86_64 GNU/Linux
There is not clue about my code from this error report.
I had no idea about this error report.
What reasons will lead to this error?
This error means you are getting a buffer overrun in do_lookup_x, if you got its source look at that or share with us.
http://valgrind.org/docs/manual/quick-start.html
This means that the do_lookup_x function has performed an invalid write access. That function is part of the runtime library (and not likely the origin of the issue). I would contact the author of HSFramework to see if they can fix this issue by running valgrind as you did