Segmentation fault in boost file system path - c++

I have a member function within a verify class that generates a hash for a set of files.
The code compiles correctly though when the software reaches the hashing function it throws a seg fault. I have been looking for ages and I just can't see it. It seems to be something to do with the path vector and maybe the copy function.
Program received signal SIGSEGV, Segmentation fault.
0x000055c84fbbc058 in std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> >::push_back(boost::filesystem::path const&) ()
(gdb)
Single stepping until exit from function _ZNSt6vectorIN5boost10filesystem4pathESaIS2_EE9push_backERKS2_,
which has no line number information.
Program terminated with signal SIGSEGV, Segmentation fault.
output from gdb
code below:
#pragma once
#include <boost/filesystem.hpp>
#include <cryptopp/cryptlib.h>
#define CRYPTOPP_ENABLE_NAMESPACE_WEAK 1
#include <cryptopp/md5.h>
#include "cryptopp/hex.h"
#include "cryptopp/files.h"
class Verify {
private:
typedef std::vector<boost::filesystem::path> vec;
vec v;
public:
bool verify_files();
bool generate_hash(boost::filesystem::path source);
};
function implementation
bool Verify::generate_hash(boost::filesystem::path source) {
try {
if (boost::filesystem::exists(source)) { //make sure path is valid
if (boost::filesystem::is_regular_file(source)) { //no regular files
std::cout << "Please enter base path of directory only." << std::endl;
return false;
}
else if (boost::filesystem::is_directory(source)){ //process directory here
std::cout << "Scanning " << source << std::endl;
std::copy(boost::filesystem::recursive_directory_iterator(source), boost::filesystem::recursive_directory_iterator(), std::back_inserter(this->v));
std::sort(v.begin(), v.end());
for (vec::const_iterator it (v.begin()); it != v.end(); ++it){
std::string result;
if (boost::filesystem::is_regular_file(*it)) {
Weak::MD5 hash;
FileSource((*it).string().c_str(), true, new HashFilter(hash, new HexEncoder(new StringSink(result), false)));
}
std::cout << "Hashing: " << *it << " - " << result << std::endl;
}
}
}
} catch (const boost::filesystem::filesystem_error& e){
std::cout << e.what() << std::endl;
return false;
}
return true;
}

Your error is on this line:
std::copy(boost::filesystem::recursive_directory_iterator(source), boost::filesystem::recursive_directory_iterator(), std::back_inserter(this->v));
You should do a manual loop here instead to investigate further (if that doesn't eliminiate the issue)

Related

C++ 17 copy and delete files with special characters in their names on Windows 7 x64?

I wrote a program that copy a file from a given path(s) to another. It runs well until it meets special character in directory names or in file names. At that moment it stops and throws the error that "No such file or directory".
This is what I done until now:
#include <iostream>
#include <vector>
#include <filesystem>
#include <cxxabi.h>
#include <string>
#include <sstream>
#include <memory>
#include <windows.h>
#include <chrono>
#include <thread>
using namespace std;
namespace fs = std::filesystem;
int main(int argc, char **argv) {
vector<string> args(argv + 1, argv + argc);
auto target = args[args.size() - 1];
fs::path path = target;
cout << "Destination path: " << target << endl;
args.erase(args.end());
for (const auto &source : args) {
try {
for (const auto &entry : fs::recursive_directory_iterator(source)) {
std::string new_path = target + "\\" + entry.path().relative_path().string();
//if entry is directory:
while (true) {
if (GetDriveType(const_cast<char *>(path.root_path().string().c_str())) != DRIVE_NO_ROOT_DIR) {
if (fs::is_directory(entry)) {
//only if it NOT exists:
if (!fs::exists(new_path)) {
//create it only if not empty:
if (!fs::is_empty(entry)) {
//Creating directory tree structure with empty folders:
try {
fs::create_directories(new_path);
} catch (const std::exception &e) // caught by reference to base
{
std::cout << "When trying to create directory" << new_path
<< "A standard exception was caught, with message '"
<< e.what() << "'\n";
}
}
}
}
break;
} else {
cout << "Destination path is not available. Sleeping for 3 minutes!" << endl;
std::this_thread::sleep_for(180000ms);
}
}
while (true) {
if (GetDriveType(const_cast<char *>(path.root_path().string().c_str())) != DRIVE_NO_ROOT_DIR) {
if ((fs::is_regular_file(entry)) && (fs::exists(entry))) {
if (!fs::is_empty(entry)) {
if (!fs::exists(new_path)) {
//file does NOT exists in new path:
try {
fs::copy_file(entry.path().string(), new_path);
cout << "Copy file: " << entry.path().string() << endl;
fs::remove(entry);
} catch (const std::exception &e) // caught by reference to base
{
std::cout
<< "When trying to get file size and source a standard exception was caught, with message '"
<< e.what() << "'\n";
}
} else {
//if it exists in new path:
//first try to get file size and if this gives an error then do not copy:
if (fs::file_size(entry.path().string()) >
fs::file_size(entry.path().string())) {
try {
fs::copy_file(entry.path().string(), new_path);
cout << "Replacing file: " << entry.path().string() << endl;
fs::remove(entry);
} catch (const std::exception &e) // caught by reference to base
{
std::cout
<< "When trying to get file size and source a standard exception was caught, with message '"
<< e.what() << "'\n";
}
}
}
}
}
break;
} else {
cout << "Destination path is not available. Sleeping for 3 minutes!" << endl;
std::this_thread::sleep_for(180000ms);
}
}//end while!
}
} catch (const std::exception &e) // caught by reference to base
{
std::cout << "When recursive through directory tree a standard exception was caught, with message '"
<< e.what() << "'\n";
}
}
return 0;
}
After searching on Google and mostly on stackoverflow for a solution conclusion is that none works.
I tried adding #define UNICODE and #define _UNICODE at the top of it but it gives even more errors.
I also added -municode flag in CMakeLists in CLion but also not working (it compiles but gives runtime error).
Also tried to replace all possible string to wstring or wchar_t * with L"where possible" and to convert this entry.path().relative_path().string() to entry.path().relative_path().wstring() and also cout to wcout. Still not working.
Also changed main to wmain( int argc, wchar_t *argv[ ]) or to wmain( int argc, wchar_t *argv[ ], wchar_t *envp[ ] ) and still not working.
Also added setlocale(LC_ALL, ""); after the main function as the other article on stackoverflow says and still no improvement.
I am asking for help because I didn't find a solution for this problem and also, more of the other solution are for printing special unicode characters to console but I need more to work with them (read files names and paths that contain special unicode characters) instead of printing them.
More than this, after I tried all of these possible not working solutions I am talking about above and reverted my program back to the original code that I just posted above now it is not working at all. It says "no such file or directory" even for normal latin characters and doesn't copy or delete anything at all anymore.
Go to the header file in which std::filesystem::path is defined.
(possibly in: PATH_TO_MINGW/usr/include/c++/YOUR_VERSION/bits/fs_path)
Look for using value_type =
Look for compiler macros that define which value_type is ultimately used.
an example from the version from my system:
#ifdef _GLIBCXX_FILESYSTEM_IS_WINDOWS
using value_type = wchar_t;
static constexpr value_type preferred_separator = L'\\';
#else
When the macro _GLIBCXX_FILESYSTEM_IS_WINDOWS is set to 1 then a wchar_t will be used, which should solve your issue.

Boost Property Tree fails to retrieve simple JSON in multi-threaded context

I am trying to parse a simple JSON string using Boost.PropertyTree in my C/C++ application.
{"header":{"version":42,"source":1,"destination":2},"coffee":"colombian"}
Here is how I've set it up in my C/C++ multi-threaded application (manually defining the JSON string to demonstrate the issue).
ParseJson.cpp
#ifdef __cplusplus
extern "C"
{
#endif
#include "ParseJson.hpp"
#ifdef __cplusplus
}
#endif
#include <iostream>
#include <sstream>
#include <string>
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
using boost::property_tree::ptree;
using boost::property_tree::read_json;
using boost::property_tree::write_json;
extern "C" MyStruct * const parseJsonMessage(char * jsonMessage, unsigned int const messageLength) {
MyStruct * myStruct = new MyStruct();
// Create empty property tree object.
ptree tree;
if (myStruct != nullptr) {
try {
// Create an istringstream from the JSON message.
std::string jsonMessageString("{\"header\":{\"version\":42,\"source\":1,\"destination\":2},\"coffee\":\"colombian\"}"); // doesn't work
std::istringstream isStreamJson(jsonMessageString);
// Parse the JSON into the property tree.
std::cout << "Reading JSON ..." << jsonMessageString << "...";
read_json(isStreamJson, tree);
std::cout << " Done!" << std::endl;
// Get the values from the property tree.
printf("version: %d\n", tree.get<int>("header.version"));
printf("source: %d\n", tree.get<int>("header.source"));
printf("coffee: %s\n", tree.get<std::string>("coffee").c_str());
}
catch (boost::property_tree::ptree_bad_path badPathException) {
std::cout << "Exception caught for bad path: " << badPathException.what() << std::endl;
return nullptr;
}
catch (boost::property_tree::ptree_bad_data badDataException) {
std::cout << "Exception caught for bad data: " << badDataException.what() << std::endl;
return nullptr;
}
catch (std::exception exception) {
std::cout << "Exception caught when parsing message into Boost.Property tree: " << exception.what() << std::endl;
return nullptr;
}
}
return myStruct;
}
The read_json() call appears to complete but the get() calls to retrieve the parsed data from the property tree fail:
Reading JSON ...{"header":{"version":42,"source":1,"destination":2},"coffee":"colombian"}... Done!
Exception caught for bad path: No such node (header.version)
I am using Boost 1.53 on RHEL 7 (compiler is gcc/g++ version 4.8.5), and I've tried both suggestions mentioned in this post related to Boost.PropertyTree and multi-threading. I've defined the BOOST_SPIRIT_THREADSAFE compile definition globally for the project. I also tried the atomic swap solution suggested for that post. Neither of these had any effect on the symptoms.
Oddly enough, I can use the other public methods for Boost.Property tree to grab the values manually:
std::cout << "front.key: " << tree.front().first << std::endl;
std::cout << "front.front.key: " << tree.front().second.front().first << std::endl;
std::cout << "front.front.value: " << tree.front().second.front().second.get_value_optional<std::string>() << std::endl;
which shows the JSON was actually parsed:
front.key: header
front.front.key: version
front.front.value: 42
Note, I had to use std::string to grab the header.version value, as trying to use get_value_optional<int>() crashes also.
However, this manual approach is not scalable; my application needs to accept several, more complicated JSON structures.
When I tried more complicated JSON strings, these were also successfully parsed, but accessing values using the get() methods similarly failed, this time crashing the program. Here is one of the GDB backtraces I pulled from the crashes, but I wasn't familiar enough with Boost to get anything useful from this:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffebfff700 (LWP 7176)]
0x00007ffff5aa8200 in std::locale::locale(std::locale const&) () from /lib64/libstdc++.so.6
Missing separate debuginfos, use: debuginfo-install boost-system-1.53.0-28.el7.x86_64 boost-thread-1.53.0-28.el7.x86_64 bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.176-5.el7.x86_64 elfutils-libs-0.176-5.el7.x86_64 glibc-2.17-292.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-37.el7_7.2.x86_64 libattr-2.4.46-13.el7.x86_64 libcap-2.22-10.el7.x86_64 libcom_err-1.42.9-16.el7.x86_64 libgcc-4.8.5-39.el7.x86_64 libselinux-2.5-14.1.el7.x86_64 libstdc++-4.8.5-39.el7.x86_64 openssl-libs-1.0.2k-19.el7.x86_64 pcre-8.32-17.el7.x86_64 systemd-libs-219-67.el7_7.2.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-18.el7.x86_64
(gdb) bt
#0 0x00007ffff5aa8200 in std::locale::locale(std::locale const&) () from /lib64/libstdc++.so.6
#1 0x00007ffff5ab6051 in std::basic_ios<char, std::char_traits<char> >::imbue(std::locale const&) () from /lib64/libstdc++.so.6
#2 0x000000000041e322 in boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int>::get_value(std::string const&) ()
#3 0x000000000041c5b2 in boost::optional<int> boost::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get_value_optional<int, boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int> >(boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int>) const ()
#4 0x000000000041aa61 in boost::enable_if<boost::property_tree::detail::is_translator<boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int> >, int>::type boost::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get_value<int, boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int> >(boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int>) const ()
#5 0x000000000041985d in int boost::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get_value<int>() const ()
#6 0x0000000000418673 in int boost::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get<int>(boost::property_tree::string_path<std::string, boost::property_tree::id_translator<std::string> > const&) const ()
#7 0x0000000000414f4a in parseJsonMessage ()
#8 0x000000000040d8cd in ProcessThread () at ../../src/Processing.c:906
#9 0x00007ffff7bc6ea5 in start_thread () from /lib64/libpthread.so.0
#10 0x00007ffff55538cd in clone () from /lib64/libc.so.6
FWIW, I tried putting this code into a simple (single-threaded) main.cpp:
#include <iostream>
#include <sstream>
#include <string>
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
using boost::property_tree::ptree;
using boost::property_tree::read_json;
using boost::property_tree::write_json;
int main(int numArgs, char * const * const args) {
ptree tree;
try {
// Create an istringstream from the JSON message.
std::string jsonMessageString("{\"header\":{\"version\":42,\"source\":1,\"destination\":2},\"coffee\":\"colombian\"}");
std::istringstream isStreamJson(jsonMessageString);
// Parse the JSON into the property tree.
std::cout << "Reading JSON..." << jsonMessageString << "...";
read_json(isStreamJson, tree);
std::cout << " Done!" << std::endl;
// Print what we parsed.
std::cout << "version: " << tree.get<int>("header.version") << std::endl;
std::cout << "source: " << tree.get<int>("header.source") << std::endl;
std::cout << "coffee: " << tree.get<std::string>("coffee") << std::endl;
}
catch (boost::property_tree::ptree_bad_path badPathException) {
std::cout << "Exception caught for bad path: " << badPathException.what() << std::endl;
return -1;
}
catch (boost::property_tree::ptree_bad_data badDataException) {
std::cout << "Exception caught for bad data: " << badDataException.what() << std::endl;
return -1;
}
catch (std::exception exception) {
std::cout << "Exception caught when parsing message into Boost.Property tree: " << exception.what() << std::endl;
return -1;
}
std::cout << "Program completed!" << std::endl;
return 0;
}
This code works just fine:
bash-4.2$ g++ -std=c++11 main.cpp -o main.exe
bash-4.2$ ./main.exe
Reading JSON...{"header":{"version":42,"source":1,"destination":2},"coffee":"colombian"}... Done!
version: 42
source: 1
coffee: colombian
Program completed!
So, why won't the Boost.PropertyTree get() methods work for a multi-threaded application? Could the fact that the multi-threaded application is a mix of C and C++ code be causing an issue? I see that my specific compiler version (GCC 4.8.5) hasn't been explicitly verified with this Boost library... Could this be a compiler issue? Or is the Boost 1.53 version buggy?
An update based on provided answer:
Admittedly, my original code for the parseJsonMessage method was messy (a product of dozens of debugging iterations and ripping out code that wasn't relevant to the problem). A more condensed version that is free of distractions (and possible red herrings) is the following:
#ifdef __cplusplus
extern "C"
{
#endif
#include "DirectIpRev3.hpp"
#ifdef __cplusplus
}
#endif
#include <iostream>
#include <sstream>
#include <string>
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
using boost::property_tree::ptree;
using boost::property_tree::read_json;
using boost::property_tree::write_json;
extern "C" void parseJsonMessage2() {
// Create empty property tree object.
ptree tree;
std::string jsonMessageString("{\"header\":{\"version\":42,\"source\":1,\"destination\":2},\"coffee\":\"colombian\"}"); //doesn't work
std::istringstream isStreamJson(jsonMessageString);
try {
read_json(isStreamJson, tree);
std::cout << tree.get<int>("header.version") << std::endl;
std::cout << tree.get<int>("header.source") << std::endl;
std::cout << tree.get<std::string>("coffee") << std::endl;
}
catch (boost::property_tree::ptree_bad_path const & badPathException) {
std::cerr << "Exception caught for bad path: " << badPathException.what() << std::endl;
}
catch (boost::property_tree::ptree_bad_data const & badDataException) {
std::cerr << "Exception caught for bad data: " << badDataException.what() << std::endl;
}
catch (std::exception const & exception) {
std::cerr << "Exception caught when parsing message into Boost.Property tree: " << exception.what() << std::endl;
}
}
Running this condensed function in my multi-threaded program yields an exception:
Exception caught when parsing message into Boost.Property tree: <unspecified file>(1): expected object or array
Without the exception handling, it prints a bit more info:
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::property_tree::json_parser::json_parser_error> >'
what(): <unspecified file>(1): expected object or array
I'm still not too sure what might be causing the failure here, but leaning towards using nlohmann as suggested.
Please, don't use Property Tree to "parse" "JSON". See nlohmann or Boost.JSON
Further
you're using raw new and delete apparently without good reason
You have unused parameters
you're catching polymorphic exceptions by value
you're leaking memory on any exception and returning null pointers when there are allocation errors
Combining these, I'm 99% certain that your crashes are caused by something else: Undefined Behaviour has a tendency to show up elsewhere after memory corruption (e.g. stack thrashing or use-after-delete, out-of-bounds etc.).
Using My Crystal Ball
A guess: you didn't show, but the struct probably looks like
typedef struct MyStructT {
int version;
int source;
char const* coffee;
} MyStruct;
A naive mistake would be to assign coffee the same way you print it:
myStruct->coffee = tree.get<std::string>("coffee").c_str();
The "obvious"(?) problem here is that c_str() points to memory owned by the value node, and transitively by the ptree. When the function returns that pointer is stale. OOPS. UB
You are allocating the struct with new (even though it is likely POD because of the extern "C", so it gives you a false sense of security, since all the members have indeterminate values anyways).
Another naive mistake would be to pass that C code that de-allocates with ::free (like it does with everything malloc-ed, right). That's another potential source of UB.
If you had "nailed" the first idea e.g. with strdup you might run into problems with more leaked memory. Even if you use delete myStruct correctly (or start using malloc instead), you will have to remember to ::free the string allocated with strdup.
Your API is typical C-style (which is probably on purpose) but leaves the door wide open to passing the wrong messageLength causing out-of-bounds read. The chance of this happening is raised due to the observation that you're not even using the arguments in your own example code above.
MULTI-THREADED STRESS TEST
Here's a multi-threaded stress test live on Coliru. It does 1000 iterations on 25 threads.
Live On Coliru
#ifdef __cplusplus
extern "C"
{
#endif
typedef struct MyStructT {
int version;
int source;
char* coffee;
} MyStruct;
//#include "ParseJson.hpp"
#ifdef __cplusplus
}
#endif
#include <iostream>
#include <sstream>
#include <string>
#define BOOST_BIND_GLOBAL_PLACEHOLDERS
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
using boost::property_tree::ptree;
using boost::property_tree::read_json;
using boost::property_tree::write_json;
extern "C" MyStruct* parseJsonMessage(char const* jsonMessage, unsigned int const messageLength) {
auto myStruct = std::make_unique<MyStruct>(); // make it exception safe
// Create empty property tree object.
ptree tree;
if (myStruct != nullptr) {
try {
// Create an istringstream from the JSON message.
std::istringstream isStreamJson(std::string(jsonMessage, messageLength));
// Parse the JSON into the property tree.
//std::cout << "Reading JSON ..." << isStreamJson.str() << "...";
read_json(isStreamJson, tree);
//std::cout << " Done!" << std::endl;
// Get the values from the property tree.
myStruct->version = tree.get<int>("header.version");
myStruct->source = tree.get<int>("header.source");
myStruct->coffee = ::strdup(tree.get<std::string>("coffee").c_str());
return myStruct.release();
}
catch (boost::property_tree::ptree_bad_path const& badPathException) {
std::cerr << "Exception caught for bad path: " << badPathException.what() << std::endl;
}
catch (boost::property_tree::ptree_bad_data const& badDataException) {
std::cerr << "Exception caught for bad data: " << badDataException.what() << std::endl;
}
catch (std::exception const& exception) {
std::cerr << "Exception caught when parsing message into Boost.Property tree: " << exception.what() << std::endl;
}
}
return nullptr;
}
#include <cstdlib>
#include <string>
#include <thread>
#include <list>
int main() {
static std::string_view msg = R"({"header":{"version":42,"source":1,"destination":2},"coffee":"colombian"})";
auto task = [] {
for (auto i = 1000; --i;) {
auto s = parseJsonMessage(msg.data(), msg.size());
::printf("version: %d\n", s->version);
::printf("source: %d\n", s->source);
::printf("coffee: %s\n", s->coffee);
::free(s->coffee);
delete s; // not ::free!
}
};
std::list<std::thread> pool;
for (int i = 0; i < 25; ++i)
pool.emplace_back(task);
for (auto& t : pool)
t.join();
}
The output (sorted and uniq-ed):
24975 coffee: colombian
24975 source: 1
24975 version: 42

Trying to compile example code from Octave's Standalone Programs example, getting segfault on first line

I am trying to learn how to embed Octave in my C++ code. When running the second example from here, the code compiles fine, but when running the code, a segmentation fault appears in the first line, when trying to initialize the interpreter. I'm not extremely adept at C++ but even when looking it up I can't find any answers.
The original code had octave::feval instead of feval, that threw a different, namespace error, so I just got rid of that and added the parse.h in the includes. I doubt this is at all related to the issue but that is a modification I did do.
#include <iostream>
#include <octave/oct.h>
#include <octave/octave.h>
#include <octave/parse.h>
#include <octave/interpreter.h>
int
main (void)
{
// Create interpreter.
octave::interpreter interpreter;
try
{
int status = interpreter.execute ();
if (status != 0)
{
std::cerr << "creating embedded Octave interpreter failed!"
<< std::endl;
return status;
}
octave_idx_type n = 2;
octave_value_list in;
for (octave_idx_type i = 0; i < n; i++)
in(i) = octave_value (5 * (i + 2));
octave_value_list out = feval ("gcd", in, 1);
if (out.length () > 0)
std::cout << "GCD of ["
<< in(0).int_value ()
<< ", "
<< in(1).int_value ()
<< "] is " << out(0).int_value ()
<< std::endl;
else
std::cout << "invalid\n";
}
catch (const octave::exit_exception& ex)
{
std::cerr << "Octave interpreter exited with status = "
<< ex.exit_status () << std::endl;
}
catch (const octave::execution_exception&)
{
std::cerr << "error encountered in Octave evaluator!" << std::endl;
}
return 0;
}
The actual output is supposed to be:
GCD of [10, 15] is 5
I am using Linux Ubuntu 18.04 with Octave 4.2.2
The documentation looked at is a different version than the version I have installed on my computer. I have 4.2, but I was looking at 4.4 docs, which has different code for the task I was trying to accomplish.

Why does boost's managed_mapped_file :: shrink_to_fit behave differently on Windows and Linux?

This is about C ++ library boost.
The managed_mapped_file :: shrink_to_fit function works differently on Linux and Windows.
On Linux, this function succeeds even if the target instance exists.
However, on Windows, this function will fail if the target instance exists.
Is this correct behavior?
It seems correct to do the same behavior, is this a bug?
I put the sample code below.
Compilation environment
boost:version.1.65.1
Windows
VisualStudio2017
WSL(Ubuntu16.04)
Linux
UbuntuServer17.10,
Clang++5.0,
g++7.2.0
Compile with
clang++-5.0 -std=c++1z ./test.cpp -o test -lpthread
#define BOOST_DATE_TIME_NO_LIB
#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/file_mapping.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <vector>
#include <iostream>
namespace bip = boost::interprocess;
using intAlloc = bip::allocator<int, bip::managed_mapped_file::segment_manager>;
using intVec = std::vector<int, intAlloc>;
int main() {
bip::managed_mapped_file *p_file_vec;
intVec *vecObj;
std::string fileName = "tmp.dat";
size_t fileSize = 1024 * 1024 * 1;
bip::file_mapping::remove(fileName.c_str());
p_file_vec = new bip::managed_mapped_file(bip::create_only, fileName.c_str(), fileSize);
vecObj = p_file_vec->construct<intVec>("myVecName")(p_file_vec->get_allocator<int>());
for (size_t i = 0; i < 10; i++)
{
vecObj->push_back(1 + 100);
}
p_file_vec->flush();
try
{ //Fail when execute on Windows(WSL),but Success on Linux(Ubuntu17.10).
std::cout << "try to shrink:pointer has existed yet!" << std::endl;
bip::managed_mapped_file::shrink_to_fit(fileName.c_str());
std::cout << "success to shrink!" << std::endl;
}
catch (const boost::interprocess::interprocess_exception &ex)
{
std::cerr << "fail to shrink!" << std::endl;
std::cerr << ex.what() << std::endl;;
}
std::cout <<"please pless enter key."<< std::endl;
std::cin.get();
try
{ //Success when execute on Windows(WSL) and Linux(Ubuntu17.10).
delete p_file_vec;
std::cout << "try to shrink:pointer has deleted!" << std::endl;
bip::managed_mapped_file::shrink_to_fit(fileName.c_str());
std::cout << "success to shrink!" << std::endl;
}
catch (const std::exception& ex)
{
std::cerr << "fail to shrink!" << std::endl;
std::cerr << ex.what() << std::endl;;
}
std::cout << "please pless enter key." << std::endl;
std::cin.get();
}
Don't use new and delete in C++ (rule of thumb).
Apart from that
delete p_file_vec;
does NOT delete anything physical. It effectively disconnects from the mapped file. This is also why shrink_to_fit works: the documentation explicitly says:
If the application can find a moment where no process is attached it can grow or shrink to fit the managed segment.
And here
So, in short: the behaviour is correct on both platforms. It's just UNDEFINED what happens in your case when you shrink while the mapped file is in use (on Ubuntu).
Fixed Code:
Live On Coliru
#include <boost/interprocess/managed_mapped_file.hpp>
#include <iostream>
#include <vector>
namespace bip = boost::interprocess;
using intAlloc = bip::allocator<int, bip::managed_mapped_file::segment_manager>;
using intVec = std::vector<int, intAlloc>;
int main() {
std::string const fileName = "tmp.dat";
bip::file_mapping::remove(fileName.c_str());
{
bip::managed_mapped_file file_vec(bip::create_only, fileName.c_str(), 1l << 20);
auto *vecObj = file_vec.construct<intVec>("myVecName")(file_vec.get_allocator<int>());
for (size_t i = 0; i < 10; i++) {
vecObj->push_back(1 + 100);
}
}
try { // Success when execute on Windows(WSL) and Linux(Ubuntu17.10).
std::cout << "try to shrink:pointer has deleted!" << std::endl;
bip::managed_mapped_file::shrink_to_fit(fileName.c_str());
std::cout << "success to shrink!" << std::endl;
} catch (const std::exception &ex) {
std::cerr << "fail to shrink!" << std::endl;
std::cerr << ex.what() << std::endl;
;
}
}

std::ifstream setting fail() even when no error

Using GCC 4.7.3 on Cygwin 1.7.24. Compiler options include: -std=gnu++11 -Wall -Wextra
I am working on a command line application and I needed to be able to load and save a set of strings so I wrote a quick wrapper class around std::set to add load and save methods.
// KeySet.h
#ifndef KEYSET_H
#define KEYSET_H
#include <cstdlib>
#include <sys/stat.h>
#include <cerrno>
#include <cstring>
#include <string>
#include <set>
#include <iostream>
#include <fstream>
inline bool file_exists (const std::string& filename)
{
/*
Utility routine to check existance of a file. Returns true or false,
prints an error and exits with status 2 on an error.
*/
struct stat buffer;
int error = stat(filename.c_str(), &buffer);
if (error == 0) return true;
if (errno == ENOENT) return false;
std::cerr << "Error while checking for '" << filename << "': " << strerror(errno) << std::endl;
exit (2);
}
class KeySet
{
private:
std::string filename;
std::set<std::string> keys;
public:
KeySet() {}
KeySet(const std::string Pfilename) : filename(Pfilename) {}
void set_filename (const std::string Pfilename) {filename = Pfilename;}
std::string get_filename () {return filename;}
auto size () -> decltype(keys.size()) {return keys.size();}
auto cbegin() -> decltype(keys.cbegin()) {return keys.cbegin();}
auto cend() -> decltype(keys.cend()) {return keys.cend();}
auto insert(const std::string key) -> decltype(keys.insert(key)) {return keys.insert(key);}
void load ();
void save ();
};
void KeySet::load ()
{
if (file_exists(filename)) {
errno = 0;
std::ifstream in (filename, std::ios_base::in);
if (in.fail()) {
std::cerr << "Error opening '" << filename << "' for reading: " << strerror(errno) << std::endl;
exit (2);
}
std::string token;
if (token.capacity() < 32) token.reserve(32);
while (in >> token) keys.insert(token);
if (!in.eof()) {
std::cerr << "Error reading '" << filename << "': " << strerror(errno) << std::endl;
exit (2);
}
in.clear(); // need to clear flags before calling close
in.close();
if (in.fail()) {
std::cerr << "Error closing '" << filename << "': " << strerror(errno) << std::endl;
exit (2);
}
}
}
void KeySet::save ()
{
errno = 0;
std::ofstream out (filename, std::ios_base::out);
if (out.fail()) {
std::cerr << "Error opening '" << filename << "' for writing: " << strerror(errno) << std::endl;
exit (2);
}
for (auto key = keys.cbegin(), end = keys.cend(); key != end; ++key) {
out << *key << std::endl;
}
out.close();
if (out.fail()) {
std::cerr << "Error writing '" << filename << "': " << strerror(errno) << std::endl;
exit (2);
}
}
#endif
//
Here's a quick program to test the load method.
// ks_test.cpp
#include "KeySet.h"
int main()
{
KeySet test;
std::string filename = "foo.keys.txt";
test.set_filename(filename);
test.load();
for (auto key = test.cbegin(), end = test.cend(); key != end; ++key) {
std::cout << *key << std::endl;
}
}
The data file just has "one two three" in it.
When I go to run the test program, I get the following error from my test program:
$ ./ks_test
Error closing 'foo.keys.txt': No error
Both cppreference.com and cplusplus.com say that the close method should set the fail bit on error. The save method works fine, and the load method works correctly if I comment out the error check after the close. Should this really work or have I misunderstood how close is supposed to work? Thanks in advance.
Edited to clarify, fix typo's and adjust code per Joachim Pileborg's and Konrad Rudolph's comments.
Edited to add solution to the code.
You have two errors here: The first is about how you do your reading, more specifically the loop for reading. The eof flag will not be set until after you tried to read and the read failed. Instead you should do like this:
while (in >> token) { ... }
Otherwise you will loop one time to many and try to read beyond the end of the file.
The second problem is the one you notice, and it depends on the the first problem. Since you try to read beyond the end of the file, the stream will set failbit causing in.fail() to return true even though there is no real error.
As it turns out, the close method for ifstream (and I assume all other IO objects) DOES NOT clear the error flags before closing the file. This means you need to add an explicit clear() call before you close the stream after end of file if you are checking for errors during the close. In my case, I added in.clear(); just before the in.close(); call and it is working as I expect.