valgrind reporting invalid read with std::string - c++

I'm working on code that runs on a raspberry pi 3. And got the following error on my logging classes.
==1297== Invalid read of size 8
==1297== at 0x4865D1C: ??? (in /usr/lib/arm-linux-gnueabihf/libarmmem.so)
==1297== Address 0x4c8d45c is 100 bytes inside a block of size 107 alloc'd
==1297== at 0x4847DA4: operator new(unsigned int) (vg_replace_malloc.c:328)
==1297== by 0x49C3D9B: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::reserve(unsigned int) (in /usr/lib/arm-linux-gnueabihf/libstdc++.so.6.0.22)
==1297== by 0x4AE65: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > std::operator+<char, std::char_traits<char>, std::allocator<char> >(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (basic_string.tcc:1155)
==1297== by 0xF82B5: Log::Book::addField(std::unique_ptr<Log::Entry, std::default_delete<Log::Entry> >&, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (LogBook.cpp:149)
==1297== by 0xF7CCB: Log::Book::record(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long long, std::ratio<1ll, 1000000000ll> > >) (LogBook.cpp:87)
GCC version: gcc version 6.3.0 20170516 (Raspbian 6.3.0-18+rpi1+deb9u1)
valgrind version: valgrind-3.13.0
I can't seem to locate the problem since the function Log::Book::record() get it's value via pass-by-value. I can also say that this error isn't always shown when the function is called. It is deterministic in the sense of on what line the error shows and on what line not. Can anybody direct my in direction of what this problem is and the solution for it? Code snippet below with a comment of the indicated lines.
/** log message */
void Book::record(std::string file, const int line, const unsigned int level, Identifier id, const std::string message,
const std::chrono::high_resolution_clock::time_point timeStamp)
{
if (!(fileLevels & level) && !(consoleLevels & level)) { return; }
auto now = Time::keeper->now();
auto duration = std::chrono::duration_cast<std::chrono::milliseconds>(timeStamp - Time::globalEpoch);
//generate message
auto entry = std::make_unique<Entry>(level);
// Time since startup
addField(entry, 0, std::to_string(duration.count()));
//UTC Time
addField(entry, 1, now.dateTime());
// File
std::string stringFile;
if (!file.empty())
{
stringFile = URL{file}.lastPathComponent();
}
addField(entry, 2, stringFile);
//Line number
addField(entry, 3, std::to_string(line));
//ID
addField(entry, 4, id);
//Message
std::string stringMessage;
if(!message.empty())
{
addField(entry, 5, message); //this is line LogBook.cpp:87
}
else
{
addField(entry, 5, " empty message.");
}
*entry << ";";
//queue message
this->append(std::move(entry));
}
void Book::addField(std::unique_ptr<Entry> &entry, unsigned int index, const std::string &text)
{
std::string textOutput;
if ((spacings.at(index) != 0) && (text.length() > (spacings.at(index) - 1)))
{
spacings.at(index) = (uint8_t) (text.length() + 2);
}
entry->setWidth(spacings.at(index));
if(entry->empty())
textOutput = text;
else
textOutput = ";" + text; //This is line LogBook.cpp:149
if(!textOutput.empty())
(*entry) << textOutput;
}
The code where this function gets called and this problem occurs.
auto node = child(items, "item", index);
auto enabled = boolValue(node, "enabled", false);
auto file = pathValue(node, key::path);
auto name = stringValue(node, "name", "");
auto type = stringValue(node, "type");
CLOG(CLOG::WARNING, "Yard item " + name + " not enabled, path:" + file.path());
Update 1:
I compile with cmake with the options. And added extra options. These didn't solve the problem.
add_compile_options(-ggdb)
add_compile_options(-O1)
#Extra disable vectorization
add_compile_options(-fno-tree-vectorize)
add_compile_options(-fno-tree-loop-vectorize)
add_compile_options(-fno-tree-slp-vectorize)
Update 2:
I've found an other place where string concatenation is used and valgrind reports te same errors
Update 3:
Some time and interesting discoveries later.
Error happens in a shared library libarmmem.so. This gets dynamically loaded and is by this reason always on a different address. Used gdb and valgrind combination to break when the error happens.
gdb loaded shared libraries with starting address.
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x0483246c 0x04832750 Yes /usr/local/lib/valgrind/vgpreload_core-arm-linux.so
0x04846e60 0x04850c10 Yes /usr/local/lib/valgrind/vgpreload_memcheck-arm-linux.so
0x04863588 0x048672fc Yes (*) /usr/lib/arm-linux-gnueabihf/libarmmem.so
...
Error reported by valgrind.
==9442== Invalid read of size 8
==9442== at 0x4865D34: ??? (in /usr/lib/arm-linux-gnueabi/libarmmem.so)
We know from readelf of libarmmem.so that the .text section begins on 588. and that memcpy sits on 710. The disassembly on this breakpoint shows we are in memcpy on address 0x04863710. If we check the range like : 0x04863588 - 0x04863710 = 188. 188 + 588(starting adress of .text) = 710.
Disassembly shows it happens on a assembly line. vldmia is a instruction for Load Vector Floating Point registers.
0x04865d34 <+9764>: vldmia r1!, {d9}
No solution yet.

Most probly the code inside libarmem.so has been vectorized in such a way that it realizes that there's a terminating char only after reading full 8-byte chunk. This will not trigger a processor exception (as algorithm ensures that pointer is aligned and thus stays in the same page) but will cause tools like Valgrind to report false positives.
Problems like this are getting worse over time and making Valgrind less useful in practice. See Valgrind vs Optimising Compilers for an in-depth discussion or this bug in diff for real-world example (or my Debian suppression list for even more examples).

Related

Creating 2D std::vector as input vector for Tensor Flow Lite results in crashing ESP although there is enough heap memory

I want to create a 2D input vector for my machine learning model. The model runs on a ESP32 but I am running into issues when it comes to setting up such an vector.
I initialise a vector by std::vector<std::vector<float>> testing_vector;
and reserve memory in my setup routine by testing_vector.reserve(1000);
This works fine when I reserve for 1000 elements. When I reserve for over 4700 elements though,
my ESP crashes although ESP.getFreeHeap() and ESP.getMaxAllocHeap() shows me that there should be enough heap memory available.
Edit:
Thanks to the answers of "Some programmer dude" and "molbdnilo" below I realised that my calculations for required space first was wrong. When I reserve 4700 elements of 12Byte vector objects this would now result in 56.400 Bytes which is still under Max Alloc Heap of 110580 though.
Input
When I reserve memory for 4.700 elements by testing_vector.reserve(4700);the ESP crashes
Outcome
My serial monitor with enabled "ESP32 Exception Decoder" gives me:
Rebooting...
ets Jul 29 2019 12:21:46
rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:1184
load:0x40078000,len:13132
load:0x40080400,len:3036
entry 0x400805e4
Total heap: 298076
Free heap: 274232
Total PSRAM: 0
Free PSRAM: 0
Max Alloc Heap: 110580
Max Vector Size: 357913941
abort() was called at PC 0x4013457b on core 1
Backtrace:0x400834e1:0x3ffb26900x40088c5d:0x3ffb26b0 0x4008d6b5:0x3ffb26d0 0x4013457b:0x3ffb2750 0x401345c2:0x3ffb2770 0x401346bb:0x3ffb2790 0x4013461a:0x3ffb27b0 0x400d1b16:0x3ffb27d0 0x400d1d95:0x3ffb27f0 0x40129fe7:0x3ffb2820
#0 0x400834e1:0x3ffb2690 in panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:402
#1 0x40088c5d:0x3ffb26b0 in esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/esp_system.c:128
#2 0x4008d6b5:0x3ffb26d0 in abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/abort.c:46
#3 0x4013457b:0x3ffb2750 in __cxxabiv1::__terminate(void (*)()) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:47
#4 0x401345c2:0x3ffb2770 in std::terminate() at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:57
#5 0x401346bb:0x3ffb2790 in __cxa_throw at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:95
#6 0x4013461a:0x3ffb27b0 in operator new(unsigned int) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/new_op.cc:54
#7 0x400d1b16:0x3ffb27d0 in __gnu_cxx::new_allocator<std::vector<float, std::allocator<float> > >::allocate(unsigned int, void const*) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\ext/new_allocator.h:111
(inlined by) std::allocator_traits<std::allocator<std::vector<float, std::allocator<float> > > >::allocate(std::allocator<std::vector<float, std::allocator<float> > >&, unsigned int) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/alloc_traits.h:436
(inlined by) std::_Vector_base<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >::_M_allocate(unsigned int) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/stl_vector.h:296
(inlined by) std::vector<float, std::allocator<float> >* std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >::_M_allocate_and_copy<std::move_iterator<std::vector<float, std::allocator<float> >*> >(unsigned int, std::move_iterator<std::vector<float, std::allocator<float> >*>, std::move_iterator<std::vector<float, std::allocator<float> >*>) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/stl_vector.h:1398
(inlined by) std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >::reserve(unsigned int) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/vector.tcc:74
#8 0x400d1d95:0x3ffb27f0 in setup() at src/main.cc:104
#9 0x40129fe7:0x3ffb2820 in loopTask(void*) at C:/Users/d.fluhr/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:42
Conclusion
I don't understand this output very well but I understand the result is invoked by my "reserve" method.
But what could be the reason for this crash?
Code
unfortunately the tensorflow lib is very big. This is why I just present my main.cc here.
I reduced the main loop part but kept the setup part as it is for a better understanding whats going on in the background.
/* Copyright 2020 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/lite/micro/all_ops_resolver.h"
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/micro/system_setup.h"
#include "tensorflow/lite/schema/schema_generated.h"
// #include "main_functions.h"
#include "model.h"
#include "constants.h"
#include "output_handler.h"
// additional libraries by Daniel
#include "Arduino.h"
#include <chrono>
#include <iostream>
#include <vector>
#include "input_data.h"
//setting timer
using namespace std::chrono;
unsigned long interval = 2000;
auto t_0 = high_resolution_clock::from_time_t(0);
auto now = high_resolution_clock::now();
auto previousMillis = duration_cast<milliseconds>(now - t_0).count();
bool debug_flag = true;
// Globals, used for compatibility with Arduino-style sketches.
namespace {
const tflite::Model* model = nullptr;
tflite::MicroInterpreter* interpreter = nullptr;
TfLiteTensor* input = nullptr;
TfLiteTensor* output = nullptr;
int inference_count = 0;
// increase if esp spits out "Failed to resize buffer"
constexpr int kTensorArenaSize = 55000;
uint8_t tensor_arena[kTensorArenaSize];
} // namespace
// initialise test vector
std::vector<std::vector<float>> testing_vector;
// The name of this function is important for Arduino compatibility.
void setup() {
// Map the model into a usable data structure. This doesn't involve any
// copying or parsing, it's a very lightweight operation.
model = tflite::GetModel(g_model);
if (model->version() != TFLITE_SCHEMA_VERSION) {
MicroPrintf("Model provided is schema version %d not equal to supported "
"version %d.", model->version(), TFLITE_SCHEMA_VERSION);
return;
}
// This pulls in all the operation implementations we need.
// NOLINTNEXTLINE(runtime-global-variables)
static tflite::AllOpsResolver resolver;
// Build an interpreter to run the model with.
static tflite::MicroInterpreter static_interpreter(
model, resolver, tensor_arena, kTensorArenaSize);
interpreter = &static_interpreter;
// Allocate memory from the tensor_arena for the model's tensors.
TfLiteStatus allocate_status = interpreter->AllocateTensors();
if (allocate_status != kTfLiteOk) {
MicroPrintf("AllocateTensors() failed");
return;
}
// Obtain pointers to the model's input and output tensors.
input = interpreter->input(0);
output = interpreter->output(0);
// Keep track of how many inferences we have performed.
inference_count = 0;
//debugg information
std::cout << "Total heap: " << ESP.getHeapSize() << "\n";
std::cout << "Free heap: " << ESP.getFreeHeap() << "\n";
std::cout << "Total PSRAM: " << ESP.getPsramSize() << "\n";
std::cout << "Free PSRAM: " << ESP.getFreePsram() << "\n";
std::cout << "Max Alloc Heap: " << ESP.getMaxAllocHeap() << "\n";
std::cout << "Max Vector Size: " << testing_vector.max_size() << "\n";
// reserve memory for vector
testing_vector.reserve(4700);
}
// The name of this function is important for Arduino compatibility.
void loop() {
// setting up timer for test ouput
now = high_resolution_clock::now();
auto mseconds = duration_cast<milliseconds>(now - t_0).count();
if (mseconds- interval > previousMillis){
std::cout << "test ouput: I am in main loop";
previousMillis = duration_cast<milliseconds>(now - t_0).count();
}
}
Further Information:
I have a partition table and already played with configure different sizes
Partition Table
partition table image
Hardware
ESP32-S (NODEMCU-32)
SPI Flash 32Mbit

Why does boost split cause double free or corruption issue

I developed a web server with C++ and here is a function, which caused a coredump issue but I don't know why.
bool MyClass::hasFamilyAdminPermission(uint32_t uid) {
ReadMutex mutex(&m_mtx); // this is read lock to lock m_familyOwner and m_familyAdmins
if (uid == m_familyOwner) {
return true;
}
std::vector<std::string> fields;
boost::split(fields, m_familyAdmins, boost::is_any_of(","));
std::string uidStr = boost::lexical_cast<string>(uid);
for (std::vector<std::string>::iterator itor = fields.begin(); itor != fields.end(); ++itor) {
if (uidStr == *itor) {
return true;
}
}
return false;
}
After executing gdb ./myServer coredump_file, I got the output as below:
warning: Unexpected size of section `.reg-xstate/8717' in core file.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./myServer'.
Program terminated with signal SIGABRT, Aborted.
warning: Unexpected size of section `.reg-xstate/8717' in core file.
#0 0x00007f627b2c5428 in __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
[Current thread is 1 (Thread 0x7f62277fe700 (LWP 8717))]
(gdb) where
#0 0x00007f627b2c5428 in __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007f627b2c702a in __GI_abort () at abort.c:89
#2 0x00007f627b3077ea in __libc_message (do_abort=do_abort#entry=2, fmt=fmt#entry=0x7f627b420ed8 "*** Error in `%s': %s: 0x%s ***\n")
at ../sysdeps/posix/libc_fatal.c:175
#3 0x00007f627b31037a in malloc_printerr (ar_ptr=<optimized out>, ptr=<optimized out>,
str=0x7f627b420fa0 "double free or corruption (fasttop)", action=3) at malloc.c:5006
#4 _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:3867
#5 0x00007f627b31453c in __GI___libc_free (mem=<optimized out>) at malloc.c:2968
#6 0x00007f627be650b4 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() ()
from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x000000000046acc1 in boost::as_literal<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > (r=...)
at /usr/local/include/boost/range/as_literal.hpp:102
#8 boost::algorithm::iter_split<std::vector<std::string, std::allocator<std::string> >, std::string, boost::algorithm::detail::token_finderF<boost::algorithm::detail::is_any_ofF<char> > > (Result=..., Input="604679400,1430691907,2792989999",
Finder=<error reading variable: DWARF-2 expression error: DW_OP_reg operations must be used either alone or in conjunction with DW_OP_piece or DW_OP_bit_piece.>) at /usr/local/include/boost/algorithm/string/iter_find.hpp:153
#9 0x0000000000464c17 in boost::algorithm::split<std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::string, boost::algorithm::detail::is_any_ofF<char> > (eCompress=<optimized out>, Pred=..., Input=..., Result=...) at /usr/local/include/boost/algorithm/string/split.hpp:149
#10 server::myServer::MyClass::hasFamilyAdminPermission (this=0x7f621c066dd0, uid=<optimized out>) at MyClass.cpp:202
#11 0x0000000000435678 in server::myServer::ServerAPI::onSetMicDataReq (this=0x7fff6da7e750, uid=1430691907, req=0x7f621c094200)
at ServerAPI.cpp:2291
So it seems that boost::split caused the coredump.
In gdb, I've done as below:
(gdb) frame 10
#10 server::myServer::MyClass::hasFamilyAdminPermission (this=0x7f621c066dd0, uid=<optimized out>) at MyClass.cpp:202
202 MyClass.cpp: No such file or directory.
(gdb) print fields
$1 = std::vector of length 0, capacity 0
For now, fields is an empty vector.
(gdb) frame 8
#8 boost::algorithm::iter_split<std::vector<std::string, std::allocator<std::string> >, std::string, boost::algorithm::detail::token_finderF<boost::algorithm::detail::is_any_ofF<char> > > (Result=..., Input="604679400,1430691907,2792989999",
Finder=<error reading variable: DWARF-2 expression error: DW_OP_reg operations must be used either alone or in conjunction with DW_OP_piece or DW_OP_bit_piece.>) at /usr/local/include/boost/algorithm/string/iter_find.hpp:153
153 /usr/local/include/boost/algorithm/string/iter_find.hpp: No such file or directory.
Here, the input is "604679400,1430691907,2792989999", which looks like ok.
However, when I execute info loclas (I'm still at the frame 8), I got a segmentation fault.
(gdb) info locals
Tmp = std::vector of length 1891892, capacity 1891893 = {<error reading variable Tmp (Cannot access memory at address 0x0)>
itBegin = {<boost::iterator_adaptor<boost::transform_iterator<boost::algorithm::detail::copy_iterator_rangeF<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, __gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, boost::algorithm::split_iterator<__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, boost::use_default, boost::use_default>, boost::algorithm::split_iterator<__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::use_default, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::use_default>> = {<boost::iterator_facade<boost::transform_iterator<boost::algorithm::detail::copy_iterator_rangeF<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, __gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, boost::algorithm::split_iterator<__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, boost::use_default, boost::use_default>, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::forward_traversal_tag, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, long>> = {<No data fields>},
m_iterator = {<boost::iterator_facade<boost::algorithm::split_iterator<__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, boost::iterator_range<__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const, boost::forward_traversal_tag, boost::iterator_range<__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, long>> = {<No data fields>}, <boost::algorithm::detail::find_iterator_base<__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >> = {
m_Finder = {<boost::function_base> = {vtable = 0x89bd9dd2, functor = {obj_ptr = 0x7f627b31453c <__GI___libc_free+76>, type = {
type = 0x7f627b31453c <__GI___libc_free+76>, const_qualified = false, volatile_qualified = false},
func_ptr = 0x7f627b31453c <__GI___libc_free+76>, bound_memfunc_ptr = {
memfunc_ptr = (void (boost::detail::function::X::*)(boost::detail::function::X * const,
int)) 0x7f627b31453c <__GI___libc_free+76>, obj_ptr = 0xe8436c057b9e1300}, obj_ref = {obj_ptr = 0x7f627b31453c <__GI___libc_free+76>,
is_const_qualified = false, is_volatile_qualified = false},
data = 60 '<'}}, <std::binary_function<__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, __gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::iterator_range<__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >> = {<No data fields>},
/usr/bin/sudo: line 11: 75240 Segmentation fault (core dumped) ${SUDO} $#
I don't know what caused the coredump, I don't know either why I got the segmentation fault after executing info locals...
So, boost::split doesn't crash. You have undefined behavior elsewhere.
Regardless, why are you parsing through a string, allocating a vector of strings, comparing to a temporary string etc. all the time? You could do this on the integer-domain.
Four takes. Starting from a simple skeleton:
#include <shared_mutex>
#include <string>
struct MyClass1 {
MyClass1(uint32_t owner, std::string admins)
: m_familyOwner(owner)
, m_familyAdmins(std::move(admins)) {}
bool hasFamilyAdminPermission(uint32_t uid) const;
private:
mutable std::shared_mutex m_mtx; // guards m_familyOwner and m_familyAdmins
uint32_t m_familyOwner;
std::string m_familyAdmins;
};
1. Comparing Ints, No Allocations
I'll use Boost Spirit X3:
#include <boost/spirit/home/x3.hpp>
bool MyClass1::hasFamilyAdminPermission(uint32_t uid) const {
std::shared_lock mutex(m_mtx);
if (uid == m_familyOwner)
return true;
bool matched = false;
auto element = boost::spirit::x3::uint32;
auto check = [uid, &matched](auto& ctx) {
if (_attr(ctx) == uid) {
matched = true;
_pass(ctx) = false; // short circuit for perf
}
};
parse(begin(m_familyAdmins), end(m_familyAdmins), element[check] % ',');
return matched;
}
This still does quite a lot of work under the lock, but certainly never allocates. Also, it does early-out, which helps if the collection of owners can be very large.
2. Comparing Text, But Without Allocations
With a nifty regex you can match the number as text on a constant string (or string view). The overhead here is the allocation(s) for the regex. But arguably, it's much simpler:
#include <regex>
bool MyClass2::hasFamilyAdminPermission(uint32_t uid) const {
std::shared_lock mutex(m_mtx);
if (uid == m_familyOwner)
return true;
return regex_search(m_familyAdmins, std::regex("(^|,)" + std::to_string(uid) + "(,|$)"));
}
3. Parse Once, At Construction
Why are we dealing with text? We could keep the admins in a set:
#include <set>
struct MyClass3 {
MyClass3(uint32_t owner, std::string_view admins) : m_familyOwner(owner) {
parse(admins.begin(), end(admins), boost::spirit::x3::uint32 % ',', m_familyAdmins);
}
bool hasFamilyAdminPermission(uint32_t uid) const;
private:
mutable std::shared_mutex m_mtx; // guards m_familyOwner and m_familyAdmins
uint32_t m_familyOwner;
std::set<uint32_t> m_familyAdmins;
};
bool MyClass3::hasFamilyAdminPermission(uint32_t uid) const {
std::shared_lock mutex(m_mtx);
return uid == m_familyOwner || m_familyAdmins.contains(uid);
}
That's even simpler. However, there's some overhead in the set which can be optimized.
4. Parse Once, No Allocations, Speed
std::set has the right semantics. However, for small sets it's sad that there's no locality of reference, and relatively high node allocation overhead. We could replace with:
boost::container::flat_set< //
uint32_t, //
std::less<>, //
boost::container::small_vector<uint32_t, 10>>
m_familyAdmins;
This makes it so that sets <= 10 elements do not allocate at all, and lookup benefits from contiguous storage. However, at this rate - unless you want to deal with duplicate entries - you might keep a linear search and store:
boost::container::small_vector<uint32_t, 10>
m_familyAdmins;
Combined Demo
Showing all the subtle edge cases. Note that only with the X3 parser
it will be easy to perform input validation on the comma-separated string
it will be easy to reliably compare differently formatted uid numbers
I snuck in one number that has a leading 0 (089 instead of 89) just to highlight this issue with the std::regex approach. Note that your original code has the same problem.
Live On Coliru/Compiler Explorer
#include <shared_mutex>
#include <string>
struct MyClass1 {
MyClass1(uint32_t owner, std::string admins)
: m_familyOwner(owner)
, m_familyAdmins(std::move(admins)) {}
bool hasFamilyAdminPermission(uint32_t uid) const;
private:
mutable std::shared_mutex m_mtx; // guards m_familyOwner and m_familyAdmins
uint32_t m_familyOwner;
std::string m_familyAdmins;
};
#include <boost/spirit/home/x3.hpp>
bool MyClass1::hasFamilyAdminPermission(uint32_t uid) const {
std::shared_lock mutex(m_mtx);
if (uid == m_familyOwner)
return true;
bool matched = false;
auto element = boost::spirit::x3::uint32;
auto check = [uid, &matched](auto& ctx) {
if (_attr(ctx) == uid) {
matched = true;
_pass(ctx) = false; // short circuit for perf
}
};
parse(begin(m_familyAdmins), end(m_familyAdmins), element[check] % ',');
return matched;
}
struct MyClass2 {
MyClass2(uint32_t owner, std::string admins)
: m_familyOwner(owner)
, m_familyAdmins(std::move(admins)) {}
bool hasFamilyAdminPermission(uint32_t uid) const;
private:
mutable std::shared_mutex m_mtx; // guards m_familyOwner and m_familyAdmins
uint32_t m_familyOwner;
std::string m_familyAdmins;
};
#include <regex>
bool MyClass2::hasFamilyAdminPermission(uint32_t uid) const {
std::shared_lock mutex(m_mtx);
if (uid == m_familyOwner)
return true;
return std::regex_search(m_familyAdmins, std::regex("(^|,)" + std::to_string(uid) + "(,|$)"));
}
#include <set>
struct MyClass3 {
MyClass3(uint32_t owner, std::string_view admins) : m_familyOwner(owner) {
parse(admins.begin(), end(admins), boost::spirit::x3::uint32 % ',', m_familyAdmins);
}
bool hasFamilyAdminPermission(uint32_t uid) const;
private:
mutable std::shared_mutex m_mtx; // guards m_familyOwner and m_familyAdmins
uint32_t m_familyOwner;
std::set<uint32_t> m_familyAdmins;
};
bool MyClass3::hasFamilyAdminPermission(uint32_t uid) const {
std::shared_lock mutex(m_mtx);
return uid == m_familyOwner || m_familyAdmins.contains(uid);
}
#include <boost/container/flat_set.hpp>
#include <boost/container/small_vector.hpp>
struct MyClass4 {
MyClass4(uint32_t owner, std::string_view admins) : m_familyOwner(owner) {
parse(admins.begin(), end(admins), boost::spirit::x3::uint32 % ',', m_familyAdmins);
}
bool hasFamilyAdminPermission(uint32_t uid) const;
private:
mutable std::shared_mutex m_mtx; // guards m_familyOwner and m_familyAdmins
uint32_t m_familyOwner;
#ifdef LINEAR_SEARCH
// likely faster with small sets, anyways
boost::container::small_vector<uint32_t, 10> m_familyAdmins;
#else
boost::container::flat_set< //
uint32_t, //
std::less<>, //
boost::container::small_vector<uint32_t, 10>>
m_familyAdmins;
#endif
};
bool MyClass4::hasFamilyAdminPermission(uint32_t uid) const {
std::shared_lock mutex(m_mtx);
return uid == m_familyOwner ||
#ifndef LINEAR_SEARCH
std::find(begin(m_familyAdmins), end(m_familyAdmins), uid) != end(m_familyAdmins);
#else
m_familyAdmins.contains(uid);
#endif
}
#include <iostream>
int main() {
MyClass1 const mc1{42, "21,377,34,233,55,089,144"};
MyClass2 const mc2{42, "21,377,34,233,55,089,144"};
MyClass3 const mc3{42, "21,377,34,233,55,089,144"};
MyClass4 const mc4{42, "21,377,34,233,55,089,144"};
std::cout << "uid\tdynamic\tregex\tset\tflat_set\n"
<< "\t(x3)\t-\t(x3)\t(x3)\n"
<< std::string(5 * 8, '-') << "\n";
auto compare = [&](uint32_t uid) {
std::cout << uid << "\t" << std::boolalpha
<< mc1.hasFamilyAdminPermission(uid) << "\t"
<< mc2.hasFamilyAdminPermission(uid) << "\t"
<< mc3.hasFamilyAdminPermission(uid) << "\t"
<< mc4.hasFamilyAdminPermission(uid) << "\n";
};
compare(42);
// https://en.wikipedia.org/wiki/Fibonacci_number
for (auto i = 3, j = 5; i < 800; std::tie(i, j) = std::tuple{j, i + j}) {
compare(i);
}
}
Prints
id dynamic regex set flat_set
(x3) - (x3) (x3)
----------------------------------------
42 true true true true
3 false false false false
5 false false false false
8 false false false false
13 false false false false
21 true true true true
34 true true true true
55 true true true true
89 true false true true
144 true true true true
233 true true true true
377 true true true true
610 false false false false

How to use Void in C++ without getting an Error about Argument [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I am trying to create a simple tree using C++ to run in ROOT using this tutorial here: https://www.niser.ac.in/sercehep2017/notes/RootTutorial_TTree.pdf.
However, Why do I get this error:
Processing examples/tree_example1.C("../trisignal/Events/run_01/tag_1_delphes_events.root")...
input_line_9:2:2: error: no matching function for call to 'tree_example1'
tree_example1("../trisignal/Events/run_01/tag_1_delphes_events.root") /* invoking function corresponding to '.x' */
^~~~~~~~~~~~~
/mnt/c/1/MG5_aMC_v2_6_6/Delphes/examples/tree_example1.C:12:6: note: candidate function not viable: requires 0 arguments, but 1 was provided
void tree_example1 ()
I am just trying to run a simple code where I am creating a TTree.
Here is my code:
/*
#ifdef __CLING__
R__LOAD_LIBRARY(libDelphes)
#include "classes/DelphesClasses.h"
#include "external/ExRootAnalysis/ExRootTreeReader.h"
#include "external/ExRootAnalysis/ExRootResult.h"
#else
class ExRootTreeReader;
class ExRootResult;
#endif
*/
void tree_example1 ()
{
//gSystem->Load("libDelphes");
// Create chain of root trees
//TChain chain("Delphes");
//chain.Add(inputFile);
//Open the input file and Create a simple tree
TFile *f = new TFile("eventstrial.root", "RECREATE");
TTree *tree = new TTree("T", "simple tree");
//TTree *tree = &chain;
TRandom r;
//Filling Histograms
TH1F *hist = new TH1F("hist", "", 100, 0., 2.);
//Set up the variables
Float_t px, py, pz, pt;
Double_t random;
UShort_t i;
//Set Variables to Tree's Branches
tree->Branch("px", &px, "px/F");
tree->Branch("py", &py, "py/F");
tree->Branch("pz", &py, "py/F");
tree->Branch("pt", &pt, "pt/F");
tree->Branch("random", &random, "random/D");
for (i = 0; i < 10000; i++)
{
r.Rannor(px,py);
pt = std::sqrt(px*px + py*py);
tree->Fill();
}
f->Write();
f->Close();
tree->Draw("pt>>hist");
hist->SetLineColor(3);
hist->Draw("same");
}
I type in my ROOT terminal: root -l 'examples/tree_example1.C("../trisignal/Events/run_01/tag_1_delphes_events.root")'
If I write
root -l tree_example1.C
I get this error:
root [0]
Processing tree_example1.C...
*** Break *** segmentation violation
Generating stack trace...
0x00007fdcdda680a0 in cling::IncrementalExecutor::executeWrapper(llvm::StringRef, cling::Value*) const + 0x380 from /home/cucip/builddir/lib/libCling.so
0x00007fdcdd9fae77 in cling::Interpreter::RunFunction(clang::FunctionDecl const*, cling::Value*) + 0xa7 from /home/cucip/builddir/lib/libCling.so
0x00007fdcdd9fc4df in cling::Interpreter::EvaluateInternal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::CompilationOptions, cling::Value*, cling::Transaction**, unsigned long) + 0x1df from /home/cucip/builddir/lib/libCling.so
0x00007fdcddac61f1 in cling::MetaSema::actOnxCommand(llvm::StringRef, llvm::StringRef, cling::Value*) + 0x591 from /home/cucip/builddir/lib/libCling.so
0x00007fdcddad6084 in cling::MetaParser::isXCommand(cling::MetaSema::ActionResult&, cling::Value*) + 0x194 from /home/cucip/builddir/lib/libCling.so
0x00007fdcddad7476 in cling::MetaParser::isCommand(cling::MetaSema::ActionResult&, cling::Value*) + 0xa6 from /home/cucip/builddir/lib/libCling.so
0x00007fdcddabf64b in cling::MetaProcessor::process(llvm::StringRef, cling::Interpreter::CompilationResult&, cling::Value*, bool) + 0x10b from /home/cucip/builddir/lib/libCling.so
0x00007fdcdd97b68e in <unknown> from /home/cucip/builddir/lib/libCling.so
0x00007fdcdd98fe72 in <unknown> from /home/cucip/builddir/lib/libCling.so
0x00007fdcdd984537 in <unknown> from /home/cucip/builddir/lib/libCling.so
0x00007fdce3112908 in TApplication::ExecuteFile(char const*, int*, bool) at /home/cucip/root-6.18.04/core/base/src/TApplication.cxx:1156 from /home/cucip/builddir/lib/libCore.so
0x00007fdce31120ac in TApplication::ProcessLine(char const*, bool, int*) at /home/cucip/root-6.18.04/core/base/src/TApplication.cxx:1007 from /home/cucip/builddir/lib/libCore.so
0x00007fdce35e18e2 in TRint::ProcessLineNr(char const*, char const*, int*) at /home/cucip/root-6.18.04/core/rint/src/TRint.cxx:762 from /home/cucip/builddir/lib/libRint.so
0x00007fdce35e31d9 in TRint::Run(bool) at /home/cucip/root-6.18.04/core/rint/src/TRint.cxx:421 from /home/cucip/builddir/lib/libRint.so
0x00007fdce3c00a0c in <unknown> from /home/cucip/builddir/bin/root.exe
0x00007fdce2571b97 in __libc_start_main + 0xe7 from /lib/x86_64-linux-gnu/libc.so.6
0x00007fdce3c00a6a in _start + 0x2a from /home/cucip/builddir/bin/root.exe
Root >
as you compiler says :
requires 0 arguments, but 1 was provided
void tree_example1 ()
you have called tree_example1 with one argument instead of zero.
redifine the function to accept the path

Is there something wrong with my GDB?

My program hangs hard (nigh freezes my whole computer), and I'm trying to figure out where it's happening. I've gone through some gdb tutorials and other things—and followed them pretty well. Whenever I see gdb output, though, it's somewhat clean and readable (that is, its output seems to at least relate to the file which it is debugging).
Here's a small chunk of my gdb's output:
23 std::map<std::string, int> Item::getStats() { return enhancements_; };
(gdb) step
Python Exception <class 'ValueError'> Cannot find type const std::map<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::less<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_Rep_type:
map (__x=std::map with 2 elements, this=0x7fffffffd0c0)
at src/item.cpp:23
23 std::map<std::string, int> Item::getStats() { return enhancements_; };
(gdb) step
_Rb_tree (__x=..., this=0x7fffffffd0c0) at src/item.cpp:23
23 std::map<std::string, int> Item::getStats() { return enhancements_; };
(gdb) step
_Rb_tree_impl (__a=<optimized out>, __comp=..., this=0x7fffffffd0c0)
at /usr/include/c++/4.9/bits/stl_tree.h:474
474 _M_header(), _M_node_count(0)
(gdb) step
475 { _M_initialize(); }
(gdb) step
_M_initialize (this=0x7fffffffd0c0)
at /usr/include/c++/4.9/bits/stl_tree.h:484
484 this->_M_header._M_left = &this->_M_header;
(gdb)
485 this->_M_header._M_right = &this->_M_header;
(gdb) step
_Rb_tree (__x=..., this=0x7fffffffd0c0)
at /usr/include/c++/4.9/bits/stl_tree.h:674
674 if (__x._M_root() != 0)
(gdb) step
_M_root (this=0x7fffffffd038)
at /usr/include/c++/4.9/bits/stl_tree.h:498
498 { return this->_M_impl._M_header._M_parent; }
Is there something up with my gdb or is this how it's generally supposed to read? I'm not seeing anything from the other header files or code outside of main.cpp. Only three of the above lines seem to relate to my code at all.
Edit:
I'm using G++ 4.9.2 and GDB 7.7.1 on Ubuntu 14.04 w/ Python 2.7.6
The output of 'where' is:
(gdb) where
#0 _int_malloc (av=0x7ffff7840760 <main_arena>, bytes=48) at malloc.c:3775
#1 0x00007ffff75047b0 in __GI___libc_malloc (bytes=48) at malloc.c:2891
#2 0x00007ffff7aebf18 in operator new(unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00000000004016f7 in allocate (__n=1, this=0x7fffffffd0c0) at /usr/include/c++/4.9/ext/new_allocator.h:104
#4 allocate (__n=1, __a=...) at /usr/include/c++/4.9/bits/alloc_traits.h:357
#5 _M_get_node (this=0x7fffffffd0c0) at /usr/include/c++/4.9/bits/stl_tree.h:385
#6 _M_create_node<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> const&> (this=0x7fffffffd0c0)
at /usr/include/c++/4.9/bits/stl_tree.h:417
#7 _M_clone_node (__x=0xa7b82fa0, this=0x7fffffffd0c0) at /usr/include/c++/4.9/bits/stl_tree.h:445
#8 std::_Rb_tree<std::string, std::pair<std::string const, int>, std::_Select1st<std::pair<std::string const, int> >, std::less<std::string>, std::allocator<std::pair<std::string const, int> > >::_M_copy (this=this#entry=0x7fffffffd0c0, __x=0xa7b82fa0, __p=0xa7b82fa0,
__p#entry=0x7fffffffd0c8) at /usr/include/c++/4.9/bits/stl_tree.h:1219
#9 0x0000000000401cc5 in _Rb_tree (__x=..., this=0x7fffffffd0c0) at /usr/include/c++/4.9/bits/stl_tree.h:676
Python Exception <class 'ValueError'> Cannot find type const std::map<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::less<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_Rep_type:
#10 map (__x=std::map with 2 elements, this=0x7fffffffd0c0) at /usr/include/c++/4.9/bits/stl_map.h:183
#11 Item::getStats (this=0x7fffffffd020) at src/item.cpp:23
#12 0x00000000004011a5 in main () at src/main.cpp:26
I can walk up and down the stack, backtrace, set breaks, and things like this—but all the output is similar in its format. I was expecting to see something a little cleaner, with more relevance to the code that I'm debugging. Something like what's in this tutorial:
Breakpoint 1, LinkedList<int>::remove (this=0x40160,
item_to_remove=#0xffbef014) at main.cc:52
52 Node<T> *marker = head_;
(gdb) step
53 Node<T> *temp = 0; // temp points to one behind as we iterate
(gdb)
55 while (marker != 0) {
(gdb)
56 if (marker->value() == item_to_remove) {
(gdb)
Node<int>::value (this=0x401b0) at main.cc:30
30 const T& value () const { return value_; }
(gdb)
LinkedList<int>::remove (this=0x40160, item_to_remove=#0xffbef014)
at main.cc:75
75 marker = 0; // reset the marker
(gdb)
76 temp = marker;
(gdb)
77 marker = marker->next();
(gdb)
Node<int>::next (this=0x0) at main.cc:28
28 Node<T>* next () const { return next_; }
(gdb)
Program received signal SIGSEGV, Segmentation fault.
Node<int>::next (this=0x0) at main.cc:28
28 Node<T>* next () const { return next_; }
(gdb)
Mine seems unintelligible compared to that.
I have no idea why your program freezes your whole computer, but the problem is obvious right here:
75 marker = 0; // reset the marker
76 temp = marker;
77 marker = marker->next();
If you ignore line 76, and just read the code, it appears to be clearly wrong. Perhaps you meant to write:
marker = marker->next();

crash at c++ STL map.find() when using boost::asio::ip::address as the key

I have following program which crashed at map.find(). I am not getting why there is a crash. Any reason why I am having a crash. Thanks for your help and time.
Please pay attention to 4th line in the function
void UploadStatisticModule::SubmitUploadSpeedInfo(boost::asio::ip::address address, boost::uint32_t size)
{
upload_speed_info_.SubmitUploadedBytes(size);
boost::uint32_t upload_speed = upload_speed_info_.GetSpeedInfo().NowUploadSpeed / 1024;
DACStatisticModule::Inst()->SubmitP2PUploadSpeedInKBps(upload_speed);
std::map<boost::asio::ip::address, SpeedInfoStatistic>::iterator iter = m_upload_map.find(address);
if (iter != m_upload_map.end())
{
iter->second.SubmitUploadedBytes(size);
}
else
{
m_upload_map.insert(std::make_pair(address, SpeedInfoStatistic()));
m_upload_map[address].Start();
m_upload_map[address].SubmitUploadedBytes(size);
}
}
Below is the crash stack,it seems finally crashed at boost::asio::ip::operator<()
Thread : Crashed: Thread
0 0x0071e37c boost::asio::ip::operator<(boost::asio::ip::address const&, boost::asio::ip::address const&) (address.hpp:233)
1 0x0073dcb9 std::_Rb_tree<boost::asio::ip::address, std::pair<boost::asio::ip::address const, statistic::SpeedInfoStatistic>, std::_Select1st<std::pair<boost::asio::ip::address const, statistic::SpeedInfoStatistic> >, std::less<boost::asio::ip::address>, std::allocator<std::pair<boost::asio::ip::address const, statistic::SpeedInfoStatistic> > >::find(boost::asio::ip::address const&) (stl_function.h:227)
2 0x0073c29b statistic::UploadStatisticModule::SubmitUploadSpeedInfo(boost::asio::ip::address, unsigned int) (stl_map.h:542)