Crash while calling boost io_context run() - c++

I'm using boost::process library to run different sub-processes from my main process. Recently we have updated to boost 1.77 version from previous 1.71 version. After the update my tool is crashing when io_context run() function is called. The call is made like below.
namespace bp = boost::process;
namespace ba = boost::asio;
ba::io_context _ios;
_child = new bp::child(execPath,
_args,
bp::on_exit=[this](int exit_code, const std::error_code& ec){onExit(exit_code, ec);},
bp::std_out > _outHandler,
bp::std_err > _errHandler,
bp::std_in.close(),
_ios);
_ios.run();
Going deep into the trace it shows that it crashes while calling free() in boost::asio::aligned_delete funtion. Also the crash occurs before the exit handler is called. Any pointers would be very helpful.

Because the code is not enough to diagnose the problem off, here's what I imagined the surrounding code to be like, so you can compare notes.
This code has no problems:
Live On Coliru
#include <boost/asio.hpp>
#include <boost/process.hpp>
#include <boost/process/async.hpp>
#include <iomanip>
#include <iostream>
#include <string>
#include <vector>
namespace bp = boost::process;
namespace ba = boost::asio;
using boost::system::error_code;
struct X {
boost::filesystem::path execPath{"/bin/bash"};
std::vector<std::string> _args{
"-c",
"sort -R /etc/dictionaries-common/words | head -30 | nl"};
std::future<std::string> _outHandler, _errHandler;
void foo() {
ba::io_context _ios;
bp::child child(
execPath,
_args,
bp::on_exit =
[this](int exit_code, error_code ec)
{
onExit(exit_code, ec);
},
bp::std_out > _outHandler,
bp::std_err > _errHandler,
bp::std_in.close(),
_ios);
std::cout << "Before run" << std::endl;
_ios.run();
std::cout << "After run" << std::endl;
}
void onExit(int exit_code, error_code ec)
{
std::cout << "Exited with: " << exit_code << " ("
<< ec.message() << ")" << std::endl;
}
};
int main()
{
X x;
x.foo();
std::cout << "err: " << std::quoted(x._errHandler.get()) << std::endl;
std::cout << "out: " << std::quoted(x._outHandler.get()) << std::endl;
}
Prints e.g.
Before run
Exited with: 0 (Success)
After run
err: ""
out: " 1 mainframe
2 congenital
3 entrusted
4 employing
5 sheepishly
6 denuding
7 abjuration
8 descant
9 brawn's
10 happier
11 activism
12 Cheops
13 Taiwan's
14 sublimated
15 cardigans
16 Triton's
17 atlas's
18 penning
19 stink
20 forefeet
21 fusses
22 spectrum's
23 reoccupy
24 replicate
25 rectors
26 preventives
27 catacomb's
28 untainted
29 eightieth
30 testing
"
Like I said, there's not much use extending the lifetime of child but you can. If you replace that line with just
new bp::child(
// ... the same
everything works the same (though obviously the bp::child instance is leaked)

Related

Problem with boost::process with long (>260) start_dir (windows)

I am on Windows 10, and want to use boost process to start a child. When the child's working directory is too long, I get an exception:
CreateProcess failed: The directory name is invalid.
I wrote a test program to debug this:
#include <boost/process.hpp>
#include <boost/process/async.hpp>
#include <boost/asio/high_resolution_timer.hpp>
#include <iostream>
#include <iomanip>
#include <thread>
#include <string>
#include <filesystem>
namespace bp = boost::process;
void test(const std::filesystem::path& wdir)
{
boost::asio::io_context io;
bp::child child;
try
{
std::filesystem::create_directories(wdir);
std::cout << "exists " << std::filesystem::exists(wdir) << " len " << wdir.generic_wstring().size() << '\n';
std::cout << "start proc\n";
child = bp::child(
"C:/WINDOWS/System32/WindowsPowerShell/v1.0/powershell.exe",
std::vector<std::string> { "ls" },
io,
boost::process::start_dir = LR"(\\?\)" + wdir.generic_wstring());
std::this_thread::sleep_for(std::chrono::seconds{ 3 });
child.wait();
}
catch (const std::exception& e)
{
std::cout << "EXCEPTION " << e.what() << "\n";
}
std::cout << "done\n";
}
int main()
{
std::cout << "short path\n";
test("D:/tmp/10378020400asdfasdfqw4retf");
std::cout << "\nlong path\n";
test("D:/tmp/10378020400826168668/unicode/qwe/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiertzertz/aaaqqqwwwiiiiertzertz/eertz/"
"iiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiii"
"iiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiii"
"iiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii"
"i/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqw"
"wwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiii"
"iiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiii"
"iiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiii"
"iiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/a"
"aaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwi"
"iiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiii"
"iiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiii"
"iiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiii"
"iiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaq"
"qqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiii"
"iiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii/aaaqqqwwwiiiiiiiiiiiii"
"iiiiiiiiiiiiiiiiiii/");
return 0;
}
I get this output:
short path
exists 1 len 33
start proc
done
long path
exists 1 len 2177
start proc
EXCEPTION CreateProcess failed: The directory name is invalid.
done
It seems the problem comes from the internally called CreateProcessW.
The documentation does not mention any limit for lpCurrentDirectory, and also allows UNC paths (hence I tried adding the prefix \\?\), but it does not make any difference whether I use the UNC syntax or not.
My questions are:
Is this an inherent limitation of Windows?
Is there any way to circumvent this limitation using boost?
Is there any way to circumvent this limitation using some other Win32 API function?
What I tired: using normal and UNC paths.

Cannot send and execute correct command through pipes using Boost library in C++

Use the answer in the question: simultaneous read and write to child's stdio using boost.process,
I refactored the code and hybridized the new method using the Boost library. I've been successful in making a pipes connection with Stockfish, but this is also where I get errors I've never seen before, not even Google helps.
Here is what I have tried:
#include <stdio.h>
#include <time.h>
#include <string>
#include <memory.h>
#include <unistd.h>
#include <iostream>
#include <stddef.h>
#include <execinfo.h>
#include <errno.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <fstream>
#include </usr/local/include/backtrace.h>
#include </usr/local/include/backtrace-supported.h>
#include <boost/process.hpp>
#include <boost/asio.hpp>
#include <boost/process/async.hpp>
#include <vector>
#include <iomanip>
#include <stdlib.h>
#include <string.h>
using namespace std;
namespace bp = boost::process;
using boost::system::error_code;
using namespace std::chrono_literals;
string errDetails = "Error Details: ";
void delay(int number_of_seconds) {
int ms = 1000 * number_of_seconds;
clock_t start_time = clock();
while (clock() < start_time + ms)
;
}
static void full_write(int fd, const char* buf, size_t len) {
while (len > 0) {
ssize_t ret = write(fd, buf, len);
if ((ret == -1) && (errno != EINTR)) {
break;
}
buf += (size_t) ret;
len -= (size_t) ret;
}
}
void print_backtrace() {
static const char start[] = "--------BACKTRACE--------\n\n";
static const char end[] = "-------------------------\n\n";
void *bt[1024];
int bt_size;
char **bt_syms;
int i;
bt_size = backtrace(bt, 1024);
bt_syms = backtrace_symbols(bt, bt_size);
full_write(STDERR_FILENO, start, strlen(start));
full_write(STDERR_FILENO, errDetails.c_str(), strlen(errDetails.c_str()));
for (i = 1; i < bt_size; i++) {
size_t len = strlen(bt_syms[i]);
full_write(STDERR_FILENO, bt_syms[i], len);
full_write(STDERR_FILENO, "\n", 1);
}
full_write(STDERR_FILENO, end, strlen(end));
free(bt_syms);
}
void abort_application() {
size_t memLeakCount, staticMemLeakCount;
uint64_t memLeakSize, staticMemLeakSize;
for (int i = 0; i < 3; i++) {
/**
* Delay
*/
delay(1);
}
print_backtrace();
abort();
}
inline bool stockfish_check_exists(const std::string& name) {
struct stat buffer;
return (stat(name.c_str(), &buffer) == 0);
}
int main() {
std::future<std::string> data;
boost::asio::io_service svc;
bp::async_pipe in{svc}, out{svc};
string proc = "";
char command[64];
string output = "";
if (stockfish_check_exists("stockfish")) {
proc = "stockfish"; } else {
errDetails = "Stockfish not found!\n\n";
abort_application();
}
std::string const program_dir = proc;
auto on_exit = [](int code, std::error_code ec) {
std::cout << "Exited " << code << "(" << ec.message() << ")\n";
};
bp::child process(proc, bp::std_in < in, svc);
boost::asio::streambuf recv_buffer;
std::cout << "uci send" << std::endl;
boost::asio::async_write(in, boost::asio::buffer("uci\n"),
[&](boost::system::error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << "\n" << std::endl;
in.close();
}
);
std::cout << "isready send" << std::endl;
boost::asio::async_write(in, boost::asio::buffer("isready\n"),
[&](boost::system::error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << "\n" << std::endl;
in.close();
}
);
cout << "Enter your command: ";
cin >> command;
cout << "Your command is: " << command << endl;
if (strcmp(command, "quit") == 0) {
cout << "Quiting......." << endl;
boost::asio::async_write(in, boost::asio::buffer("quit"),
[&](boost::system::error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << std::endl;
in.close();
cout << "Engine quit!" << endl;
}
);
}
svc.run();
return 0;
}
To make it easier to follow, I left out std::std_out > out at the line:
bp::child process(proc, bp::std_in < in, svc);
so that the engine results are immediately displayed in the Terminal window, so I'll know if I've gone astray. And this is when I discovered the strange thing
When I launch the application, it outputs on Terminal as follows:
[2022-01-14 20:25:55]
duythanh#DuyThanhs-MacBook-Pro:/Volumes/Data/ChessGUI$ ./ChessGUI
uci send
isready send
Enter your command: Stockfish 120122 by the Stockfish developers (see AUTHORS file)
id name Stockfish 120122
id author the Stockfish developers (see AUTHORS file)
option name Debug Log File type string default
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 33554432
option name Clear Hash type button
option name Ponder type check default false
option name MultiPV type spin default 1 min 1 max 500
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 10 min 0 max 5000
option name Slow Mover type spin default 100 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
option name UCI_LimitStrength type check default false
option name UCI_Elo type spin default 1350 min 1350 max 2850
option name UCI_ShowWDL type check default false
option name SyzygyPath type string default <empty>
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 7 min 0 max 7
option name Use NNUE type check default true
option name EvalFile type string default nn-ac07bd334b62.nnue
uciok
Unknown command: isready
Contrasting with the code above, the two commands were sent through pipes. is uci and isready, this is fine. The first uci command runs successfully, but the isready command, instead of returning readyok, it returns:
Unknown command: isready
I keep trying to type quit, which sends a quit command to the pipe as the exit engine, and it also fails:
Your command is: quit
Quiting.......
Write: 5
Write: 9
Unknown command: quit
Write: 5
Engine quit!
The program will then exit with the engine. I'm still wondering what was going on at the time, but the clues are really hazy as to what was going on behind the scenes.
Please help me. Any help is highly appreciated. Thank you so much everyone
UPDATE: The error continued when Unknown Command: Quit appeared. I typed these commands in Terminal while running Stockfish directly through Terminal, they work as a result, but my program still can't
You are printing to cout as if the async operations happen immediately. That's not the case. The async operations only happen when the io service runs.
svc.run();
Is at the very end of your code. So no async_ operation ever completes (or even starts) before that.
Other problems:
Your out async pipe is never used (not even connected). It's unclear to me how you intend to communicate with the child process that way.
In fairness, you only every write to the child process, so maybe you're not at all interested in the output. (But then perhaps recv_buffer can be deleted just as well).
Your buffers include the terminating NUL characters. (asio::buffer("uci\n") sends {'u','c','i','\n','\0'}). That's going to mess up the child processes's parsing.
You do in.close() in response to every single async_write completion. This guarantees that subsequent writes never can happen, as you closed the pipe.
Then when you send quit you fail to include the '\n' as well
You are reading into a char[64] with operator>> which makes no sense at all. Maybe you are using c++20 (so width of 64 might be assumed) but you never set a width. Most likely you would want to read into a string instead.
However, doing so cannot accept commands with whitespace (because std::ios::skipws is set by default). So, likely you wanted std::getline instead...
The fact that you include a boatload of C headers makes me think you're porting some C code (badly). That's also exemplified by the strcmp use and others, e.g. no need to use ::stat
Don't use using namespace std; (Why is "using namespace std;" considered bad practice?)
Don't use global variables (errDetails)
Don't use loops to wait for a time delay
No need to manually print backtraces. Instead, use Boost:
void abort_application(std::string const& errDetails) {
std::cerr << errDetails << "\n";
std::cerr << boost::stacktrace::stacktrace{} << std::endl;
std::this_thread::sleep_for(3s);
abort();
}
Existing Stockfish Client: Playing Games
You're in luck: I have a written full demo using stockfish on this site: Interfacing with executable using boost in c++.
This example shows how to correctly await and parse expected replies from the child process(es).
You will note that I chose coroutines for the async version:
Just for completeness, I thought I'd try an asynchronous implementation. Using the default Asio callback style this could become unwieldy, so I thought to use Boost Coroutine for the stackful coroutines. That makes it so the implementation can be 99% similar to the synchronous version
Just for comparison, here's what your code should look like if you didn't use coroutines:
Fixing Up Your Code
Live On Coliru
#include <boost/asio.hpp>
#include <boost/process.hpp>
#include <boost/process/async.hpp>
#include <boost/stacktrace/stacktrace.hpp>
#include <chrono>
#include <iomanip>
#include <iostream>
namespace bp = boost::process;
using boost::system::error_code;
using namespace std::literals;
static void abort_application(std::string const& errDetails) {
std::cerr << errDetails << "\n";
std::cerr << boost::stacktrace::stacktrace{} << std::endl;
std::this_thread::sleep_for(3s);
abort();
}
inline static bool stockfish_check_exists(std::string& name) {
return boost::filesystem::exists(name);
}
int main() {
boost::asio::io_service svc;
bp::async_pipe in{svc};
std::string proc = "/usr/games/stockfish";
if (!stockfish_check_exists(proc)) {
abort_application("Stockfish not found!");
}
auto on_exit = [](int code, std::error_code ec) {
std::cout << "Exited " << code << "(" << ec.message() << ")\n";
};
bp::child process(proc, bp::std_in < in, svc, bp::on_exit = on_exit);
std::function<void()> command_loop;
std::string command_buffer;
command_loop = [&] {
std::cout << "Enter your command: " << std::flush;
// boost::asio::streambuf recv_buffer;
if (getline(std::cin, command_buffer)) {
std::cout << "Your command is: " << command_buffer << std::endl;
command_buffer += '\n';
async_write( //
in, boost::asio::buffer(command_buffer),
[&](error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << " (" << ec.message() << ")" << std::endl;
if (command_buffer == "quit\n") {
std::cout << "Quiting......." << std::endl;
// in.close();
std::cout << "Engine quit!" << std::endl;
} else {
command_loop(); // loop
}
});
}
};
std::cout << "uci send" << std::endl;
async_write(
in, boost::asio::buffer("uci\n"sv),
[&](error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << "\n" << std::endl;
std::cout << "isready send" << std::endl;
async_write(in, boost::asio::buffer("isready\n"sv),
[&](error_code ec, size_t n) {
std::cout << "Write: " << n << std::endl;
command_loop(); // start command loop
});
});
svc.run(); // only here any of the operations start
}
Prints, e.g.
Or if Stockfish is in fact installed:

Tasks on asio::strand are running on a single thread

I modified an asio strand example using the standalone version of the library from 4a here
#include <iostream>
#include <asio.hpp>
#include <future>
#include <thread>
#include <mutex>
#include <chrono>
using namespace std::chrono_literals;
namespace util
{
static std::mutex s_mtx_print;
// Default argument value
// https://en.cppreference.com/w/cpp/language/default_arguments
template <typename... Args>
void sync_print(const bool log_thread_id, Args &&... args)
{
std::lock_guard<std::mutex> print_lock(s_mtx_print);
if (log_thread_id)
{
std::cout << "[" << std::this_thread::get_id() << "] ";
}
(std::cout << ... << args) << '\n';
}
}
void Worker(std::unique_ptr<asio::io_service> &ios)
{
util::sync_print(true, " Started...");
if(ios) {ios->run();}
util::sync_print(true, " End");
}
void PrintNum(int n)
{
std::cout << "[" << std::this_thread::get_id() << "] " << n << '\n';
std::this_thread::sleep_for(300ms);
}
void OrderedInvocation(std::unique_ptr<asio::io_service::strand> &up_strand)
{
if(up_strand)
{
up_strand->post(std::bind(&PrintNum, 1));
up_strand->post(std::bind(&PrintNum, 2));
up_strand->post(std::bind(&PrintNum, 3));
up_strand->post(std::bind(&PrintNum, 4));
up_strand->post(std::bind(&PrintNum, 5));
up_strand->post(std::bind(&PrintNum, 6));
up_strand->post(std::bind(&PrintNum, 7));
up_strand->post(std::bind(&PrintNum, 8));
up_strand->post(std::bind(&PrintNum, 9));
}
else{
std::cerr << "Invalid strand" << '\n';
}
}
int main()
{
util::sync_print(true, "section 4 started ...");
auto up_ios = std::make_unique<asio::io_service>();
auto up_work = std::make_unique<asio::io_service::work>(*up_ios);
auto up_strand = std::make_unique<asio::io_service::strand>(*up_ios);
std::vector<std::future<void>> tasks;
constexpr int NUM_TASK = 3;
for(int i = 0; i< NUM_TASK; ++i)
{
tasks.push_back(std::async(std::launch::async, &Worker, std::ref(up_ios)));
}
std::cout << "Task size " << tasks.size() << '\n';
std::this_thread::sleep_for(500ms);
OrderedInvocation(up_strand);
up_work.reset();
for(auto &t: tasks){ t.get(); }
return 0;
}
The problem is: when I run the code, it appears that the function PrintNum only runs on a single thread
as the console output is
[140180645058368] section 4 started ...
Task size 3
[140180610144000] Started...
[140180626929408] Started...
[140180618536704] Started...
[140180610144000] 1
[140180610144000] 2
[140180610144000] 3
[140180610144000] 4
[140180610144000] 5
[140180610144000] 6
[140180610144000] 7
[140180610144000] 8
[140180610144000] 9
[140180610144000] End
[140180626929408] End
[140180618536704] End
My question is, do I need to configure the strand to let the tasks spread to all threads? Or maybe I missed something here?
[Edit]
Ideally, the output should be something like
[00154F88] The program will exit when all work has finished.
[001532B0] Thread Start
[00154FB0] Thread Start
[001532B0] x: 1
[00154FB0] x: 2
[001532B0] x: 3
[00154FB0] x: 4
[001532B0] x: 5
[00154FB0] Thread Finish
[001532B0] Thread Finish
Press any key to continue . . .
In the expected output, both thread 00154FB0 and 001532B0 executed the PrintNum(), but in the modified version, only one thread executed the PrintNum().
If the strand is not been used, the output is:
[140565152012096] section 4 started ...
[140565133883136] Started...
Task size 3
[140565117097728] Started...
[140565125490432] Started...
[[140565133883136] [140565117097728]] 12
3
[140565133883136] [4
[140565117097728140565125490432] 6
] 5
[140565133883136] 7
[140565125490432] 8
[140565117097728] 9
[140565125490432] End
[140565117097728] End
[140565133883136] End
Thanks
Here is the cpu info from the machine I am using
$lscpu
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
The OS is Ubuntu 18.04
Rong
That's the purpose of a strand:
A strand is defined as a strictly sequential invocation of event handlers (i.e. no concurrent invocation). Use of strands allows execution of code in a multithreaded program without the need for explicit locking (e.g. using mutexes).
If you want parallel invocation, you will need to remove the strand, post() directly to io_service and invoke io_service::run from a number of threads (you're doing that already).
An unrelated note: there is no point in passing unique pointers around; make your life easier and just pass raw pointers or references.
This may be a bit late. However, I ran into the same issue, following the same example as above. It turns out the current way of using a strand is a bit different, as hinted here. Here is my revision on the original code:
#include <boost/asio/io_context.hpp>
#include <boost/asio/strand.hpp>
#include <boost/asio/post.hpp>
#include <boost/asio/executor_work_guard.hpp>
#include <memory>
#include <mutex>
#include <thread>
#include <chrono>
#include <vector>
#include <iostream>
namespace asio = boost::asio;
std::mutex global_stream_lock;
void
worker_thread(std::shared_ptr<asio::io_context> ioc) {
global_stream_lock.lock();
std::cout << "[" << std::this_thread::get_id() << "] Thread start"
<< std::endl;
global_stream_lock.unlock();
ioc->run();
global_stream_lock.lock();
std::cout << "[" << std::this_thread::get_id() << "] Thread finished"
<< std::endl;
global_stream_lock.unlock();
}
void
print_num(int x) {
std::cout << "[" << std::this_thread::get_id() << "] x = " << x
<< std::endl;
}
int
main() {
auto ioc = std::make_shared<asio::io_context>();
auto strand = asio::make_strand(*ioc);
auto work = asio::make_work_guard(*ioc);
global_stream_lock.lock();
std::cout << "[" << std::this_thread::get_id()
<< "] This thread will exit when all work is finished "
<< std::endl;
global_stream_lock.unlock();
std::vector<std::thread> thread_group;
for (int i = 0; i < 4; ++i) {
thread_group.emplace_back(std::bind(worker_thread, ioc));
}
for (int i = 0; i < 4; ++i) {
std::this_thread::sleep_for(std::chrono::milliseconds(100));
asio::post(strand, std::bind(print_num, 2 * i + 1));
asio::post(strand, std::bind(print_num, 2 * i + 2));
}
work.reset();
for (auto &t : thread_group) {
t.join();
}
}
This produces the following output:
[139877509977920] This thread will exit when all work is finished
[139877509973568] Thread start
[139877501580864] Thread start
[139877493188160] Thread start
[139877484795456] Thread start
[139877509973568] x = 1
[139877509973568] x = 2
[139877493188160] x = 3
[139877493188160] x = 4
[139877501580864] x = 5
[139877501580864] x = 6
[139877484795456] x = 7
[139877484795456] x = 8
[139877509973568] Thread finished
[139877493188160] Thread finished
[139877484795456] Thread finished
[139877501580864] Thread finished

Boost::Process pipe chain

Using the results of this question simultaneous read and write to child's stdio using boost.process, I am trying to modify the code so that a file is read, piped through gzip, the output of gzip piped through bzip2, and finally the output of bzip2 written to a file.
My first attempt was
/*
* ProcessPipe.cpp
*
* Created on: Apr 17, 2018
* Author: dbetz
*/
//#define BOOST_ASIO_ENABLE_HANDLER_TRACKING 1
#include <boost/asio.hpp>
#include <boost/asio/high_resolution_timer.hpp>
#include <boost/process.hpp>
#include <boost/process/async.hpp>
#include <iostream>
#include <fstream>
#include <functional>
namespace bp = boost::process;
using boost::system::error_code;
using namespace std::chrono_literals;
using Loop = std::function<void()>;
using Buffer = std::array<char, 500>;
int main() {
boost::asio::io_service svc;
auto gzip_exit = [](int code, std::error_code ec) {
std::cout << "gzip exited " << code << " (" << ec.message() << ")\n";
};
auto bzip2_exit = [](int code, std::error_code ec) {
std::cout << "bzip2 exited " << code << " (" << ec.message() << ")\n";
};
bp::async_pipe file_to_gzip_pipe{svc}, gzip_to_bzip_pipe{svc}, bzip_to_file_pipe{svc};
bp::child process_gzip("/usr/bin/gzip", "-c", bp::std_in < file_to_gzip_pipe, bp::std_out > gzip_to_bzip_pipe, svc, bp::on_exit(gzip_exit));
bp::child process_bzip2("/usr/bin/bzip2", "-c", bp::std_in < gzip_to_bzip_pipe, bp::std_out > bzip_to_file_pipe, svc, bp::on_exit(bzip2_exit));
std::ifstream ifs("src/ProcessPipe2.cpp");
Buffer file_to_gzip_buffer;
Loop file_to_gzip_loop;
file_to_gzip_loop = [&] {
if (!ifs.good())
{
error_code ec;
file_to_gzip_pipe.close(ec);
std::cout << "Read file, write gzip: closed stdin (" << ec.message() << ")\n";
return;
}
ifs.read(file_to_gzip_buffer.data(), file_to_gzip_buffer.size());
boost::asio::async_write(file_to_gzip_pipe, boost::asio::buffer(file_to_gzip_buffer.data(), ifs.gcount()),
[&](error_code ec, size_t transferred) {
std::cout << "Read file, write gzip: " << ec.message() << " sent " << transferred << " bytes\n";
if (!ec) {
file_to_gzip_loop(); // continue writing
}
});
};
Buffer gzip_to_bzip_buffer;
Loop gzip_to_bzip_loop;
gzip_to_bzip_loop=[&] {
gzip_to_bzip_pipe.async_read_some(boost::asio::buffer(gzip_to_bzip_buffer),
[&](error_code ec, size_t transferred){
// duplicate buffer
std::cout << "Read gzip, write bzip: " << ec.message() << " got " << transferred << " bytes\n";
if (!ec)
gzip_to_bzip_loop();
else
gzip_to_bzip_pipe.close();
}
);
};
std::ofstream ofs("src/ProcessPipe2.gz");
Buffer bzip_to_file_buffer;
Loop bzip_to_file_loop;
bzip_to_file_loop = [&] {
bzip_to_file_pipe.async_read_some(boost::asio::buffer(bzip_to_file_buffer),
[&](error_code ec, size_t transferred) {
std::cout << "Read bzip, write file: " << ec.message() << " got " << transferred << " bytes\n";
ofs << std::string(bzip_to_file_buffer.data(),transferred);
if (!ec)
bzip_to_file_loop(); // continue reading
});
};
file_to_gzip_loop(); // async
gzip_to_bzip_loop();
bzip_to_file_loop(); // async
svc.run(); // Await all async operations
}
but this gives an error:
Read gzip, write bzip: Bad file descriptor got 0 bytes
The problem seems to be that gzip_to_bzip_pipe is opened for writing by gzip and for reading by bzip. Any ideas?
I'd write that code simply like:
#include <boost/process.hpp>
#include <iostream>
namespace bp = boost::process;
int main() {
bp::pipe intermediate;
bp::child process_gzip("/bin/gzip", "-c", bp::std_in<"src/ProcessPipe2.cpp", bp::std_out> intermediate);
bp::child process_bzip2("/bin/bzip2", "-c", bp::std_in<intermediate, bp::std_out> "src/ProcessPipe2.gz.bz2");
process_bzip2.wait();
process_bzip2.wait();
}
BONUS
You can do without sub processes entirely and just use boost::iostreams::copy:
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/filter/bzip2.hpp>
#include <boost/iostreams/filter/gzip.hpp>
#include <boost/iostreams/copy.hpp>
#include <iostream>
#include <fstream>
namespace io = boost::iostreams;
int main() {
std::ifstream ifs("src/ProcessPipe2.cpp");
io::filtering_stream<io::output> os;
os.push(io::gzip_compressor());
os.push(io::bzip2_compressor());
std::ofstream ofs("src/ProcessPipe2.gz.bz2");
os.push(ofs);
io::copy(ifs, os);
}

How To Create TimerHandler Using Boost Library

I'm working on a project using C++.
I want a TimerHandler to be called after a specified time, but at the same time I don't want to block the current thread or any code after io.run() in the following code:
#include <iostream>
#include <string>
#include <boost/format.hpp>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
class TimerTest
{
public:
static void PrintOutTimerHandler(const boost::system::error_code&, const std::string& message)
{
std::cout << "PrintOutTimerHandler called: " << ", message: " << message << std::endl;
}
void run()
{
boost::asio::io_service io;
boost::asio::deadline_timer dt(io, boost::posix_time::seconds(5));
std::cout << "Start:\t" << std::endl;
dt.async_wait(boost::bind(PrintOutTimerHandler, boost::asio::placeholders::error, std::string("here is the message")));
// Do some job here
for (int i = 0; i < 1000000; ++i)
++i, --i;
std::cout << "End:\t" << std::endl;
io.run();
std::cout << "When to reach here 1: " << std::endl;
}
};
int main()
{
TimerTest tt;
tt.run();
std::cout << "When to reach here 2: " << std::endl;
return 0;
}
/* Current output:
Start:
End:
PrintOutTimerHandler called: , message: here is the message
When to reach here 1:
When to reach here 2:
*/
/* Expected output:
Start:
End:
When to reach here 1:
When to reach here 2:
PrintOutTimerHandler called: , message: here is the message
*/
I think I made myself clear. My questions are:
If this can be solved without
introducing a new thread, like Flex
ActionScript, that's is the best, but
I guess not (I guess ActionScript is
using a hidden thread);
If we have to
introduce an extra thread to do the
job, would you mind writing down the
pseudo code for me?
Thanks.
Peter
Here is an example . Run the io_service in a separate thread
asio::io_service io_service;
asio::thread t(boost::bind(&asio::io_service::run, &io_service));
or run it in a thread group
boost::thread_group threads;
for (std::size_t i = 0; i < my_thread_count; ++i)
threads.create_thread(boost::bind(&asio::io_service::run, &io_service));
Remember that your main thread should always run because when it exists all threads spawned will also exit.
I hope this helps.
I misunderstood what OrcunC said, but actually he is correct. Here is the modified version for your reference:
#include <iostream>
#include <string>
#include <boost/format.hpp>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/thread.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
class TimerTest
{
public:
static void PrintOutTimerHandler(const boost::system::error_code&, const std::string& message)
{
std::cout << "PrintOutTimerHandler called: " << ", message: " << message << std::endl;
}
TimerTest(unsigned int timeout)
: dt(io, boost::posix_time::milliseconds(timeout))
{
}
void run()
{
std::cout << "Start:\t" << std::endl;
dt.async_wait(boost::bind(PrintOutTimerHandler, boost::asio::placeholders::error, std::string("here is the message")));
boost::thread thrd(boost::bind(&boost::asio::io_service::run, &io));
// Do some job here
for (int i = 0; i < 1000000; ++i)
++i, --i;
std::cout << "End:\t" << std::endl;
std::cout << "When to reach here 1: " << std::endl;
}
boost::asio::io_service io;
boost::asio::deadline_timer dt;
};
int main()
{
TimerTest tt(5000);
tt.run();
std::cout << "When to reach here 2: " << std::endl;
// Keep the main thread active for testing purpose. Otherwise,
// once the TimerTest object is destroyed when exiting the main() function,
// the sub thread spawed in tt.run() will also exit;
Sleep(10000);
}
/* Current output and Expected output:
Start:
End:
When to reach here 1:
When to reach here 2:
PrintOutTimerHandler called: , message: here is the message
*/