Issue on RHEL4 release using boost 1.36 and C++ - c++

I am struggling with a mysterious problem
I only see on my RHEL4 release build. Some of my unit tests (using boost 1.36 unit test framework) fail on RHEL4 (gcc 3.4.6) and using release build-type. I do not see the problem using RHEL5 release or debug build types (gcc 4.1.2, boost-1.39); neither do I
see it on Windows 32 bit or 64 bit using either Visual Studio 2005 (using boost-1.36) or 2008 (using boost-1.39).
Suspecting that this may be due to some subtle memory issue, I proceeded to run valgrind on the test application (minimal case that preserved the problem). Here is what I got when I ran valgrind using "full, no-reachable" mode:
==12285== Memcheck, a memory error detector.
==12285== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==12285== Using LibVEX rev 1575, a library for dynamic binary translation.
==12285== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==12285== Using valgrind-3.1.1, a dynamic binary instrumentation framework.
==12285== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==12285== For more details, rerun with: -v
==12285==
==12285== My PID = 12285, parent PID = 12284. Prog and args are:
==12285== ./myprojd
==12285==
==12285== Syscall param sigaltstack(ss) points to uninitialised byte(s)
==12285== at 0x3AD682EDA9: sigaltstack (in /lib64/tls/libc-2.3.4.so)
==12285== by 0x6488638: boost::detail::signal_handler::~signal_handler()
(in /<path_to>/libboost_unit_test_framework-gcc34-mt-1_36.so.1.36.0)
==12285== by 0x648975E: boost::execution_monitor::catch_signals
(boost::unit_test::callback0<int> const&)
(in /<path_to>/libboost_unit_test_framework-gcc34-mt-1_36.so.1.36.0)
==12285== by 0x6489813: boost::execution_monitor::execute
(boost::unit_test::callback0<int> const&)
(in /<path_to>/libboost_unit_test_framework-gcc34-mt-1_36.so.1.36.0)
==12285== by 0x648F2E4: boost::unit_test::framework::run(unsigned long, bool)
(in /<path_to>/libboost_unit_test_framework-gcc34-mt-1_36.so.1.36.0)
==12285== by 0x649BD02: boost::unit_test::unit_test_main(bool (*)(), int, char**)
(in /<path_to>/libboost_unit_test_framework-gcc34-mt-1_36.so.1.36.0)
==12285== by 0x4147F0: main (init.cpp:132)
==12285== Address 0x7FEFFF3B0 is on thread 1's stack
==12285==
==12285== ERROR SUMMARY: 6 errors from 1 contexts (suppressed: 4 from 1)
==12285== malloc/free: in use at exit: 190,112 bytes in 1,869 blocks.
==12285== malloc/free: 23,128 allocs, 21,259 frees, 2,520,845 bytes allocated.
==12285== For counts of detected errors, rerun with: -v
==12285== searching for pointers to 1,869 not-freed blocks.
==12285== checked 2,184,272 bytes.
==12285==
==12285== LEAK SUMMARY:
==12285== definitely lost: 0 bytes in 0 blocks.
==12285== possibly lost: 0 bytes in 0 blocks.
==12285== still reachable: 190,112 bytes in 1,869 blocks.
==12285== suppressed: 0 bytes in 0 blocks.
==12285== Reachable blocks (those to which a pointer was found) are not shown.
==12285== To see them, rerun with: --show-reachable=yes
Ofcourse, I ran this in debug mode (although as I mentioned the error occurs only in release mode). If I run valgrind in release mode, I get the same output (with perhaps
less detail such as line #s). From this it appears that the problem is somehow in boost-1.36, or perhaps my definition of init_unit_test_suite? Clearly one thing I can try is
to run using boost-1.39 on all platforms; but unfortunately, we're currently on boost-1.36 for RHEL4 and VS2005, and so this may not be practical yet.
I also observe that forcing a certain logger output to console at a point where the test fails, enables the test to pass (not good, I know)! Suspecting that this might be due that I commented all logger output and ran valgrind - so that's what's posted above. If you need some code snippets of the init_unit_test_suite function; I can post that, if it helps. Any ideas to resolve this are welcome and greatly appreciated.
05/26/2011 Edit:
Here's the init_unit_test_suite - appreciate if somebody could take a look.
std::ofstream log_stream;
std::ofstream report_stream;
const_string retrieve_framework_parameter( const_string cla_name, int argc, char** argv ) {
//- first try to find parameter among command line arguments if present
if( argc ) {
//- locate corresponding cla name
if( !cla_name.is_empty() ) {
for( int i = 1; i < argc; ++i ) {
if( cla_name == const_string( argv[i], cla_name.size() ) && argv[i][cla_name.size()] == '=' ) {
const_string result = argv[i] + cla_name.size() + 1;
for( int j = i; j < argc; ++j ) {
argv[j] = argv[j+1];
}
--argc;
return result;
}
}
}
}
return std::getenv( cla_name.begin() );
}
//! Format results to CPP UNIT xml
class simple_report_formatter : public results_reporter::format {
public:
virtual void results_report_start( std::ostream&) {
}
virtual void results_report_finish( std::ostream&) {
}
virtual void test_unit_report_start(test_unit const&, std::ostream&) {
}
virtual void test_unit_report_finish(test_unit const& tu, std::ostream& out) {
if( tu.p_type == tut_case ) {
const test_results& r = results_collector.results(tu.p_id);
if( r.passed() ) {
out<<"[PASS] ";
} else {
out<<"[FAIL] ";
}
out<<"Test Case <unit_"<<tu.p_name.get()<<"> ";
if( !r.passed() ) {
out<<" - ";
out<<"!! Assertions failed: "<<r.p_assertions_failed;
out<<" - See log files for details on failures !!";
}
out<<std::endl;
#if defined(MYPROJ_WINDOWS) && defined(MYPROJ_DEBUG)
if( !r.passed() ) {
std::ostringstream msg;
msg<<"!! "<<tu.p_name.get()<<" FAILED !!"<<std::endl;
OutputDebugStringA(msg.str().c_str());
}
#endif
}
}
virtual void do_confirmation_report(test_unit const&, std::ostream&) {
}
};
bool init_unit_test_suite() {
const_string log_file = retrieve_framework_parameter(
"--log_file",
framework::master_test_suite().argc,
framework::master_test_suite().argv
);
if( !log_file.empty() ) {
log_stream.open(log_file.begin());
unit_test_log.set_stream(log_stream);
}
const_string report_file = retrieve_framework_parameter(
"--report_file",
framework::master_test_suite().argc,
framework::master_test_suite().argv
);
if( !report_file.empty() ) {
report_stream.open(report_file.begin());
results_reporter::set_stream(report_stream);
}
if( runtime_config::report_format() == CLF ) {
results_reporter::set_format(new simple_report_formatter);
}
// This is providing the sensible default configuration when the test is being run
// without any input parameters whatsoever: print the final report to console
if( framework::master_test_suite().argc <= 1 ) {
results_reporter::set_stream(std::cout);
results_reporter::set_format(new simple_report_formatter);
results_reporter::set_level(DETAILED_REPORT);
}
framework::master_test_suite().p_name.set(MYPROJ_TEST_SUITE_NAME);
return true;
}

This "error" in valgrind is not necessarily a problem. When a system call is called in Linux the memory often has to be copied from users space to kernel space. In doing so the it may copy uninitialized bytes.
Just because it copies these bytes into kernel space does not mean the kernel will actually do anything with the uninitialized bytes. What the kernel does with the bytes depends heavily on the semantics of the system call in question.
Valgrind has no simple way of knowing if it is OK that some of the bytes passed in the system call are uninitialized so it always signals an error. Many of similar errors are suppressed by valgrind.
You can see some of valgrind's default suppressions in this file: /usr/lib/valgrind/default.supp You can also create your own suppression files if you like and use them in your unit test to suppress the error message. If you are not experiencing any actual problems on this system them suppressing the error is probably a good idea. See valgrind's command line options.

Related

valgrind leak error after deletion of all dynamic memory

I have tried to learn CPP pointer as well as to free all the memory about which I have used valgrind. But unfortunately I am getting leak error and I don't know where I am making the mistake. Also not so much idea about finding error as a human-readable way from valgrind. Any guidance to find the leak is highly appreciable.
compiler: g++ (Ubuntu 7.5.0-3ubuntu1~16.04) 7.5.0
related info regarding snippet
smart pointer is not used intentionally
it is a minimal example code. So, some portion might seem to be unnecessary.
file.cpp
#include <iostream>
void algo_fun(unsigned int* ip_ptr_array_,
unsigned int ip_size_,
unsigned int** op_ptr_array_,
unsigned int* op_size_)
{
*(op_size_) = ip_size_ + 2;
// following approach is good as it allocate dynamic memory
unsigned int* local = new unsigned int[*(op_size_)];
for (unsigned int i = 0; i< *(op_size_); i++)
{
local[i]=i+1*3;
}
*op_ptr_array_ = &local[0];
local[3] = 87;
}
int main()
{
// input array's contetnt
unsigned int ip_size = 10;
unsigned int* ip_ptr_array = new unsigned int[ip_size];
// output data
unsigned int op_size;
unsigned int* op_ptr_array;
// filling input array
for(unsigned int i = 0; i < ip_size; i++)
{
ip_ptr_array[i] = i+2*2;
}
// function calling to get output data
algo_fun(ip_ptr_array,
ip_size,
&op_ptr_array,
&op_size);
delete [] ip_ptr_array;
delete [] op_ptr_array;
return 0;
}
Working version will be found here.
command used to test: valgrind --leak-check=full --show-leak-kinds=all -v ./file
Leak Summary from valgrind
==23138== LEAK SUMMARY:
==23138== definitely lost: 0 bytes in 0 blocks
==23138== indirectly lost: 0 bytes in 0 blocks
==23138== possibly lost: 0 bytes in 0 blocks
==23138== still reachable: 72,704 bytes in 1 blocks
==23138== suppressed: 0 bytes in 0 blocks
==23138==
==23138== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==23138== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
tl;dr Please check that you are using a recent Valgrind.
With a combination of Valgrind and gdb you should be able to see what is going on.
I get the following (FreeBSD 12.2, g++ 10.3.0, Valgrind built from git HEAD, [OS and compiler version are not relevant]). I'm using the --trace-malloc=yes option to see all the mallloc/free calls. Don't do that on a large application.
$ valgrind --leak-check=full --trace-malloc=yes ./test
--61886-- malloc(72704) = 0x5800040
--61886-- calloc(1984,1) = 0x5811C80
--61886-- calloc(104,1) = 0x5812480
--61886-- calloc(224,1) = 0x5812530
--61886-- calloc(80,1) = 0x5812650
--61886-- calloc(520,1) = 0x58126E0
--61886-- calloc(88,1) = 0x5812930
--61886-- _Znam(40) = 0x58129D0
--61886-- _Znam(48) = 0x5812A40
--61886-- _ZdaPv(0x58129D0)
--61886-- _ZdaPv(0x5812A40)
--61886-- free(0x5800040)
==61886==
==61886== HEAP SUMMARY:
==61886== in use at exit: 3,000 bytes in 6 blocks
==61886== total heap usage: 9 allocs, 3 frees, 75,792 bytes allocated
_Znam is the mangled version of array new and _ZdaPv the mangled version of array delete from your code. The other calls to malloc/calloc/free come from libc/libstc++. You will probably see different traces on Linux or with libc++.
You could use --show-reachable=yes to get information on the reachable memory.
If I now run under gdb.
$ gdb ./test
(gdb) b malloc
(gdb) b malloc
(gdb) r
Breakpoint 1, malloc (nbytes=96) at /usr/src/libexec/rtld-elf/rtld.c:5877
This is a call to malloc in the link loader, ld.so. I then executed several more 'r's until
Breakpoint 1, __je_malloc_initialized () at jemalloc_jemalloc.c:208
Here the link loader has loaded libstdc++.so and global malloc has been replaced by jemalloc.
Getting the callstack
(gdb) bt
#0 __je_malloc_initialized () at jemalloc_jemalloc.c:208
#1 imalloc (sopts=<optimized out>, dopts=<optimized out>) at jemalloc_jemalloc.c:1990
#2 __malloc (size=72704) at jemalloc_jemalloc.c:2042
#3 0x00000008006eb6f4 in ?? () from /usr/local/lib/gcc10/libstdc++.so.6
#4 0x000000080060e2fd in objlist_call_init (list=<optimized out>, lockstate=<optimized out>) at /usr/src/libexec/rtld-elf/rtld.c:2820
#5 0x000000080060d03d in _rtld (sp=0x7fffffffe408, exit_proc=0x7fffffffe3d0, objp=0x7fffffffe3d8) at /usr/src/libexec/rtld-elf/rtld.c:811
#6 0x000000080060a8c9 in rtld_start () at /usr/src/libexec/rtld-elf/amd64/rtld_start.S:39
#7 0x0000000000000000 in ?? ()
(gdb)
Note the same allocation size on line #2.
I'm not going to dig into what this memory is for, something part of the C++ runtime.
libstdc++ (and libc) deliberately do not free this memory (presumably this is either difficult or just not worth it). In order to filter out these allocations, Valgrind uses a special function to ask libstc++/libc to free this memory.
The option in this case is
--run-cxx-freeres=no|yes free up libstdc++ memory at exit on Linux
(strictly speaking, not just Linux).
Looking at the Valgrind release notes
Release 3.12.0 (20 October 2016)
...
* New option --run-cxx-freeres=<yes|no> can be used to change whether
__gnu_cxx::__freeres() cleanup function is called or not. Default is
'yes'.
So I presume that you are using version 3.11 or earlier.
Finally, if I use clang++/libc++ there is no runtime allocation (and also no freeres function).

C++ Valgrind FIX Conditional jump or move depends on uninitialised value(s)

I run my program using valgrind like this:
valgrind --leak-check=full ./a.out < in0.txt > out0.txt
and get the following output:
==13077== Memcheck, a memory error detector
==13077== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==13077== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==13077== Command: ./a.out
==13077==
==13077== Conditional jump or move depends on uninitialised value(s)
==13077== at 0x40491F: OnGetMostViewedClasses (in /home/user/a.out)
==13077== by 0x404498: parser (in /home/user/a.out)
==13077== by 0x404263: main (in /home/user/a.out)
==13077==
==13077==
==13077== HEAP SUMMARY:
==13077== in use at exit: 0 bytes in 0 blocks
==13077== total heap usage: 90 allocs, 90 frees, 6,200 bytes allocated
==13077==
==13077== All heap blocks were freed -- no leaks are possible
==13077==
==13077== For counts of detected and suppressed errors, rerun with: -v
==13077== Use --track-origins=yes to see where uninitialised values come from
==13077== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
-bash-4.2$
while it refers to StatusType res; used in here:
static errorType OnGetMostViewedClasses(void* DS, const char* const command) {
int numOfClasses;
int *courses = NULL, *classes = NULL;
StatusType res;
ValidateRead(sscanf(command, "%d", &numOfClasses), 1, "%s failed.\n", commandStr[GETMOSTVIEWEDCLASSES_CMD]);
if (numOfClasses > 0) {
courses = (int *)malloc(numOfClasses * sizeof(int));
classes = (int *)malloc(numOfClasses * sizeof(int));
if (courses == NULL || classes == NULL) {
res = ALLOCATION_ERROR;
}
}
if (res != ALLOCATION_ERROR) {
res = GetMostViewedClasses(DS, numOfClasses, courses, classes);
}
if (res != SUCCESS) {
printf("%s: %s\n", commandStr[GETMOSTVIEWEDCLASSES_CMD], ReturnValToStr(res));
if (courses != NULL) free(courses);
if (classes != NULL) free(classes);
return error_free;
}
printf("%s: %s\n", commandStr[GETMOSTVIEWEDCLASSES_CMD], ReturnValToStr(res));
printf("Course\t|\tClass\n");
for (int i = 0; i < numOfClasses; i++)
{
printf("%d\t|\t%d\n", courses[i], classes[i]);
}
printf("--End of most viewed classes--\n");
if (courses != NULL) free(courses);
if (classes != NULL) free(classes);
return error_free;
}
and declared like this:
typedef enum {
SUCCESS = 0,
FAILURE = -1,
ALLOCATION_ERROR = -2,
INVALID_INPUT = -3
} StatusType;
So my question is, how can I solve this problem without changing the above code since it was given to us and my professor told: "you aren't allowed to change it" and "any errors found by valgrind will get you a mark of 0"?
Any ideas or suggestions are welcomed.
please note: except this problem I have 0 memory leak issue (According to 32 input files I tested)
Neither res nor numOfClasses is initialized. There are code paths that can result in these values being used in if statements without having a value assigned to them. scanf can fail, leaving numOfClasses uninitialized, and unless an error occurs in memory allocation res will not be assigned a value.
The solution is to initialize both:
int numOfClasses = 0;
StatusType res = SUCCESS;
As #1201ProgramAlarm points out, res is not initialized, and since it only gets set if numOfClasses is greater than zero and one of the malloc calls fails, you would need to somehow force the code into that code path. So, make sure the sscanf reads a value from command (command should be "1" or something like that), and then ... somehow cause malloc to return NULL. That's not trivial, but maybe you can add a clever line like #define malloc(x) NULL somewhere above that function? This entire exercise seems pretty terrible, TBH.
Edit: Alternatively, make malloc fail by asking for TOO MUCH memory. If numOfClasses is something like SIZE_MAX/sizeof(int), there's a pretty good chance you won't be able to allocate two blocks of memory that big, and one or both malloc calls will return NULL. That sets the res variable, and there won't be an uninitialized read. Stupid, stupid, stupid.
This is tagged C++, but the code, with typedef enum, void*, malloc, printf and scanf smells badly of C. That is not a good way to learn C++.
You don't provide a definition of ValidateRead. You might just be able to subvert the preprocessor to do what you need, but this will be tortuous.
What the code really should do is res = ValidateRead(... where ValidateRead is a function that returns SUCCESS or INVALID_INPUT.

throw() with clang++ shows possible memory leak

I have a piece of code, which when compiled with g++ does not show any memory leaks.
Whereas, the same when compiled with clang++ shows, possible memory leak.
Here's the trace,
==7115==
==7115== HEAP SUMMARY:
==7115== in use at exit: 16 bytes in 1 blocks
==7115== total heap usage: 2,324 allocs, 2,323 frees, 2,166,060 bytes allocated
==7115==
==7115== 16 bytes in 1 blocks are still reachable in loss record 1 of 1
==7115== at 0x4C2BFB9: calloc (vg_replace_malloc.c:762)
==7115== by 0x4129830: __cxxabiv1::__calloc_with_fallback(unsigned long, unsigned long) (in /opt/xxx/lib64/libc++abi.so.1)
==7115== by 0x4128946: __cxa_get_globals (in /opt/xxx/lib64/libc++abi.so.1)
==7115== by 0x412B287: __cxa_throw (in /opt/xxx/lib64/libc++abi.so.1)
==7115== by 0x4E712AE: Lib::GenCmd::RaiseException(Status, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) (LibBase.cpp:291)
==7115==
==7115== LEAK SUMMARY:
==7115== definitely lost: 0 bytes in 0 blocks
==7115== indirectly lost: 0 bytes in 0 blocks
==7115== possibly lost: 0 bytes in 0 blocks
==7115== still reachable: 16 bytes in 1 blocks
==7115== suppressed: 0 bytes in 0 blocks
==7115==
==7115== For counts of detected and suppressed errors, rerun with: -v
Well, its not possible to share the code snippet, but I can tell you RaiseException() is the function where I have a call made to throw() (at line 291) an exception. Here's the function snippet:
void GenCmd::RaiseException(Status status, std::string AdditionalMsg) throw(Status) {
s_last_error = GetStatusString(status);
if (false == AdditionalMsg.empty()) {
s_last_error = s_last_error + AdditionalMsg;
}
throw(status);
}
Status is a structure, defined as below (along with default, parameterized & copy constructors)
typedef struct _Status {
const u64_t m_status : 8;
const u64_t ReservedByte1 : 8;
const u64_t m_action : 8;
const u64_t ReservedByte3 : 5;
const u64_t m_testbit1 : 1;
const u64_t m_testbit2 : 1;
const u64_t m_cmd_failure : 1;
const u64_t m_module_code : 4;
const u64_t m_file_code : 8;
const u64_t ReservedByte7 : 4;
const u64_t m_line_no : 16;
}Status
The fact, that no leaks are seen with GCC, but only with Clang makes me think this to be some issue with Clang. (With Clang, I mean it could be libcxxabi as well)
I was navigating through the source for clang, & __cxa_get_globals() is the function where a calloc() call is made. I am not yet sure of the execution flow for clang.
Any idea or any inputs which could confirm this to be a Clang issue & not my code issue?
Here's the clang version I am using. The code is compiled with C++11, additionally with '-stdlib=libc++, '-lc++', '-lc++abi'.
[user~]$ clang --version
clang version 7.1.0
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin
Update: This exception is raised from the constructor.
Update
I wrote another dummy code to see the behaviour with Clang, & it seems the issue is actually with the Clang (libc++abi).
Have a look at the below naive code
#include <iostream>
#include <stdexcept>
class Positive {
int m_number1;
public:
Positive() : m_number1(10) {
}
Positive(int no) {
if (no < 0) {
// throw 100;
throw std::invalid_argument("Send positive nu");
} else {
m_number1 = no;
}
}
~Positive() {
}
void print() {
std::cout<< "Value of member is: " <<m_number1 <<std::endl;
}
};
int main() {
try {
Positive p1;
p1.print();
Positive p2(100);
p2.print();
Positive p3(-10);
p3.print();
} catch(...) {
std::cout << "Some Exception occured" <<std::endl;
}
return 0;
}
Even on executing the above code, I saw the same result on Valgrind. Here's the output:
[user]$ valgrind --leak-check=full --leak-resolution=high --show-leak-kinds=all ./a.out
==119789== Memcheck, a memory error detector
==119789== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==119789== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==119789== Command: ./a.out
==119789==
Value of member is: 10
Value of member is: 100
Some Exception occured
==119789==
==119789== HEAP SUMMARY:
==119789== in use at exit: 16 bytes in 1 blocks
==119789== total heap usage: 3 allocs, 2 frees, 201 bytes allocated
==119789==
==119789== 16 bytes in 1 blocks are still reachable in loss record 1 of 1
==119789== at 0x4C2BFB9: calloc (vg_replace_malloc.c:762)
==119789== by 0x40FF830: __cxxabiv1::__calloc_with_fallback(unsigned long, unsigned long) (in /usr/local/lib/libc++abi.so.1.0)
==119789== by 0x40FE946: __cxa_get_globals (in /usr/local/lib/libc++abi.so.1.0)
==119789== by 0x4101287: __cxa_throw (in /usr/local/lib/libc++abi.so.1.0)
==119789== by 0x4014B0: Positive::Positive(int) (in /home/user/test/a.out)
==119789== by 0x4010F9: main (in /home/user/test/a.out)
==119789==
==119789== LEAK SUMMARY:
==119789== definitely lost: 0 bytes in 0 blocks
==119789== indirectly lost: 0 bytes in 0 blocks
==119789== possibly lost: 0 bytes in 0 blocks
==119789== still reachable: 16 bytes in 1 blocks
==119789== suppressed: 0 bytes in 0 blocks
==119789==
==119789== For counts of detected and suppressed errors, rerun with: -v
==119789== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
[user]$
Interestingly, it shows 3 allocations made. Which I am assuming to be something related to the 3rd object, but how can I ensure that is cleared (or not allocated itself)?
Probably, the same thing can help me fix my original code.
I also found this problem. It actually occurs if your code throws anything at least once, so the simplest reproduction scenario is just running this code:
int
main()
{
try {
throw 42;
} catch (int) {}
}
However, if you throw several times there is still just one 16-bytes "leaked" block (which is not really leaked, since it is still reachable from some global variable). Digging into libc++abi library sources shows that such blocks are actually allocated once per thread to hold some exception processing context, and each one is freed when a thread is destroyed since it uses TLS and have proper destructor registered. So after all, it looks totally safe and is not an issue.

How to correctly use and free asn1c SEQUENCE_OF?

I'm using the asn1c lib for many projects, but I never found how to use the free member of a SEQUENCE_OF. I always set it to nullptr because of that, and when I use Valgrind, I see (of course) that my lists members are not freed when using ASN_STRUCT_FREE on the element containing the list.
So my question is how can I use that free member?
Here is a simple example of how I use my lists with asn1c.
ListItem_t *li = nullptr;
StructWList_t swl;
swl.list.count = 0;
swl.list.size = 0;
swl.list.free = nullptr; // How can I feed it properly?
swl.list.array = reinterpret_cast<ListItem_t**>(calloc(1, sizeof *swl.list.array));
for(int i = 0 ; i < 5 ; i++)
{
li = reinterpret_cast<ListItem_t*>(calloc(1, sizeof *li));
*li = i;
// Valgrind says that the calloc below is definitly lost
swl.list.array[i] = reinterpret_cast<ListItem_t*>(calloc(1, sizeof *swl.list.array[i]));
ASN_SEQUENCE_ADD(&swl, li);
}
...
ASN_STRUCT_FREE(ASN_DEF_StructWList, &swl);
Does anyone know how to feed it properly?
EDIT
My version of asn1c is the v0.9.29 from git repository in AUR (on Archlinux).
The above ASN.1 is as follow:
Example
DEFINITIONS AUTOMATIC TAGS ::=
BEGIN
StructWList ::= SEQUENCE OF ListItem
ListItem ::= INTEGER
END
Thanks in advance,
Emilien
// Valgrind says that the calloc below is definitly lost
swl.list.array[i] = reinterpret_cast<ListItem_t*>(calloc(1, sizeof *swl.list.array[i]));
ASN_SEQUENCE_ADD(&swl, li);
The ASN_SEQUENCE_ADD will overwrite the pointer you stored on the previous line. You should either store it manually as on the first line or call ASN_SEQUENCE_ADD but not both.
Also you should fully initialize swl as it contains more members (_asn_ctx) and use ASN_STRUCT_FREE_CONTENTS_ONLY as swl is allocated on the stack and cannot be freed.
--- main.cpp.orig 2019-05-07 20:49:25.880336931 +0300
+++ main.cpp 2019-05-07 20:59:10.192431926 +0300
## -3,7 +3,7 ##
int main()
{
ListItem_t *li = nullptr;
- StructWList_t swl;
+ StructWList_t swl = {0};
swl.list.count = 0;
swl.list.size = 0;
## -15,8 +15,8 ##
li = reinterpret_cast<ListItem_t*>(calloc(1, sizeof *li));
*li = i;
// Valgrind says that the calloc below is definitly lost
- swl.list.array[i] = reinterpret_cast<ListItem_t*>(calloc(1, sizeof *swl.list.array[i]));
+ //swl.list.array[i] = reinterpret_cast<ListItem_t*>(calloc(1, sizeof *swl.list.array[i]));
ASN_SEQUENCE_ADD(&swl, li);
}
- ASN_STRUCT_FREE(ASN_DEF_StructWList, &swl);
+ ASN_STRUCT_FREE_CONTENTS_ONLY(asn_DEF_StructWList, &swl);
}
Compile with g++ -Wall -I. -ggdb -O0 -o test main.cpp libasncodec.a
valgrind --tool=memcheck ./test
==29555== Memcheck, a memory error detector
==29555== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==29555== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==29555== Command: ./test
==29555==
==29555==
==29555== HEAP SUMMARY:
==29555== in use at exit: 0 bytes in 0 blocks
==29555== total heap usage: 9 allocs, 9 frees, 72,848 bytes allocated
==29555==
==29555== All heap blocks were freed -- no leaks are possible
==29555==
==29555== For counts of detected and suppressed errors, rerun with: -v
==29555== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Memory leak when using regex.h?

Minimal code example is as follows:
#include <cstdlib>
#include <iostream>
#include <vector>
#include <regex.h>
using namespace std;
class regex_result {
public:
/** Contains indices of starting positions of matches.*/
std::vector<int> positions;
/** Contains lengths of matches.*/
std::vector<int> lengths;
};
regex_result match_regex(string regex_string, const char* string) {
regex_result result;
regex_t* regex = new regex_t;
regcomp(regex, regex_string.c_str(), REG_EXTENDED);
/* "P" is a pointer into the string which points to the end of the
previous match. */
const char* pointer = string;
/* "n_matches" is the maximum number of matches allowed. */
const int n_matches = 10;
regmatch_t matches[n_matches];
int nomatch = 0;
while (!nomatch) {
nomatch = regexec(regex, pointer, n_matches, matches, 0);
if (nomatch)
break;
for (int i = 0; i < n_matches; i++) {
int start,
finish;
if (matches[i].rm_so == -1) {
break;
}
start = matches[i].rm_so + (pointer - string);
finish = matches[i].rm_eo + (pointer - string);
result.positions.push_back(start);
result.lengths.push_back(finish - start);
}
pointer += matches[0].rm_eo;
}
delete regex;
return result;
}
int main(int argc, char** argv) {
string str = "this is a test";
string pat = "this";
regex_result res = match_regex(pat, str.c_str());
cout << res.positions.size() << endl;
return 0;
}
So I have written a function that parses a given string for regular expression matches. The result is held in a class that is essentially two vectors, one for the positions of the matches and one for the corresponding match lengths.
This works fine, but when I ran valgrind over it, it shows some substantial memory leaks.
When using valgrind --leak-check=full on the code above I get:
==24843== Memcheck, a memory error detector
==24843== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==24843== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
==24843== Command: ./test
==24843==
1
==24843==
==24843== HEAP SUMMARY:
==24843== in use at exit: 11,688 bytes in 37 blocks
==24843== total heap usage: 54 allocs, 17 frees, 12,868 bytes allocated
==24843==
==24843== 256 bytes in 1 blocks are definitely lost in loss record 14 of 18
==24843== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24843== by 0x543549A: regcomp (regcomp.c:487)
==24843== by 0x400ED0: match_regex(std::string, char const*) (in <path>)
==24843== by 0x4010CA: main (in <path>)
==24843==
==24843== 11,432 (224 direct, 11,208 indirect) bytes in 1 blocks are definitely lost in loss record 18 of 18
==24843== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24843== by 0x4C2CF1F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24843== by 0x5434BAF: re_compile_internal (regcomp.c:760)
==24843== by 0x54354FF: regcomp (regcomp.c:506)
==24843== by 0x400ED0: match_regex(std::string, char const*) (in <path>)
==24843== by 0x4010CA: main (in <path>)
==24843==
==24843== LEAK SUMMARY:
==24843== definitely lost: 480 bytes in 2 blocks
==24843== indirectly lost: 11,208 bytes in 35 blocks
==24843== possibly lost: 0 bytes in 0 blocks
==24843== still reachable: 0 bytes in 0 blocks
==24843== suppressed: 0 bytes in 0 blocks
==24843==
==24843== For counts of detected and suppressed errors, rerun with: -v
==24843== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Is my code wrong or is there really a bug in those files?
Your regex_t management is not required to be dynamic, and though that isn't directly related to you problem, it is a little odd. The real problem is you never regfree() your resulting expression if compiled successfully (which you should verify). You should setup your regular expression like this:
regex_t regex;
int res = regcomp(&regex, regex_string.c_str(), REG_EXTENDED);
if (res == 0)
{
// use your expression via &regex
....
// and eventually free it when done.
regfree(&regex);
}
If your implementation supports them, I strongly advise using the C++11 provided <regex> library, as it has nice RAII solutions to much of this.
You must call regfree() to free memory allocated by regcomp().