Can we create unnamed socket with boost.asio to emulate anonymous pipe? - c++

socketpair() on Linux lets you create unnamed socket. Is something similar in boost.asio library possible? I'm trying to emulate anonymous pipe with boost.asio library. I know boost.process supports this but I want to use boost.asio library. By-the-way why is anonymous pipe missing from boost.asio?

I wrote the code below to emulate pipe using boost.asio library. Its only demo code and there is no message boundary checking, error checking etc.
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <iostream>
#include <cctype>
#include <boost/array.hpp>
using boost::asio::local::stream_protocol;
int main()
{
try
{
boost::asio::io_service io_service;
stream_protocol::socket parentSocket(io_service);
stream_protocol::socket childSocket(io_service);
//create socket pair
boost::asio::local::connect_pair(childSocket, parentSocket);
std::string request("Dad I am your child, hello!");
std::string dadRequest("Hello son!");
//Create child process
pid_t pid = fork();
if( pid < 0 ){
std::cerr << "fork() erred\n";
} else if (pid == 0 ) { //child process
parentSocket.close(); // no need of parents socket handle, childSocket is bidirectional stream socket unlike pipe that has different handles for read and write
boost::asio::write(childSocket, boost::asio::buffer(request)); //Send data to the parent
std::vector<char> dadResponse(dadRequest.size(),0);
boost::asio::read(childSocket, boost::asio::buffer(dadResponse)); //Wait for parents response
std::cout << "Dads response: ";
std::cout.write(&dadResponse[0], dadResponse.size());
std::cout << std::endl;
} else { //parent
childSocket.close(); //Close childSocket here use one bidirectional socket
std::vector<char> reply(request.size());
boost::asio::read(parentSocket, boost::asio::buffer(reply)); //Wait for child process to send message
std::cout << "Child message: ";
std::cout.write(&reply[0], request.size());
std::cout << std::endl;
sleep(5); //Add 5 seconds delay before sending response to parent
boost::asio::write(parentSocket, boost::asio::buffer(dadRequest)); //Send child process response
}
}
catch (std::exception& e)
{
std::cerr << "Exception: " << e.what() << "\n";
std::exit(1);
}
}

Related

Is waiting for action completion in MQTT async_client necessary?

I am working on an cpp MQTT async_client utilizing the paho library. I am trying to fully understand the workings of the asynchronous client, but I am not sure how to correctly utilize the features.
As I understand it, messages with QoS > 0 follow a handshake procedure specified by the standard. The delivery to the server is regarded as complete if either PUBACK (QoS 1) or PUBCOMP (QoS 2) has been received. All of this seems to be handled by the library. The user can interact with the client either through callbacks or through tokens associated with specific actions (e.g. publish).
I have the following questions regarding these tools:
The user can either register a mqtt::iaction_listener callback to an action that is invoked when the action fails and/or succeeds. Simultaneously, there exists a callback when the last message of a handshake has been successfully received (PUBACK and PUBCOMP, respectively). Are the success callbacks not redundant?
Waiting for message tokens to complete in an asynchronous client does not make sense for me. If I wait for message to be published, I block the sending thread. Surely, I could use separate thread for waiting, but that defeats the purpose of asynchronous clients. So do I just omit the wait, especially when sending messages in rapid succession? (see code below)
Test client code:
#include <iostream>
#include <cstdlib>
#include <string>
#include <cstring>
#include <cctype>
#include <thread>
#include <chrono>
#include "mqtt/async_client.h"
const std::string SERVER_ADDRESS("tcp://localhost:1883");
const std::string CLIENT_ID("client1");
const std::string TOPIC("hello");
class Callback
: public virtual mqtt::callback
{
// the mqtt client
mqtt::async_client& cli_;
void delivery_complete(mqtt::delivery_token_ptr tok) override
{
std::cout << "delivery_complete for token: " << tok->get_message_id() << std::endl;
}
public:
Callback(mqtt::async_client& cli)
: cli_(cli) {}
};
int main(int argc, char* argv[])
{
mqtt::async_client cli(SERVER_ADDRESS, CLIENT_ID);
mqtt::connect_options connOpts;
connOpts.set_clean_session(false);
Callback cb(cli);
cli.set_callback(cb);
// Connect; waiting makes sense here as publishing without a connection is nonsense
try {
cli.connect(connOpts);
}
catch (const mqtt::exception& exc) {
std::cerr << "\nERROR: Unable to connect to MQTT server: '"
<< SERVER_ADDRESS << "'" << exc << std::endl;
return 1;
}
// Press 'm' to send messages or 'q' to exit
while (true)
{
char c = std::tolower(std::cin.get());
if (c == 'q') {
break;
}
else if (c == 'm')
{
for (int i = 0; i < 10; ++i)
{
mqtt::message_ptr pubmsg = mqtt::make_message(TOPIC, "TESTMSG");
pubmsg->set_qos(1);
auto tok = cli.publish(pubmsg);
tok->wait(); // how is this done correctly with QoS > 0 ?
}
}
}
// Disconnect
try {
cli.disconnect()->wait();
std::cout << "OK" << std::endl;
}
catch (const mqtt::exception& exc) {
std::cerr << exc << std::endl;
return 1;
}
return 0;
}

boost::process::child will not exit after closing input stream

In the following example I try to write some data to a child process, which processes the data and writes it to a file. After closing the stream the parent process waits indefinitely for the child to finish. I am at a loss to know how to indicate that I’m done writing the data and would like the child process to stop reading and finish whatever it is doing. According to the documentation calling terminate would send a SIGKILL which I don’t think is what I want.
What am I missing? I checked this question but I would rather try to make the actual code work with synchronous IO first.
#include <boost/process.hpp>
#include <iostream>
namespace bp = boost::process;
int main(int argc, char **argv)
{
boost::process::opstream in{};
boost::process::child child("/path/to/test.py", bp::std_in < in);
in << "test1\n";
in << "test2\n";
in << "test3\n";
in << std::flush;
std::cerr << "Closing the stream…\n";
in.close();
std::cerr << "Waiting for the child to exit…\n";
child.wait(); // Parent seems to hang here.
return 0;
}
test.py just writes the data to a file like so:
#!/usr/local/homebrew/opt/python#3.8/bin/python3
import sys
with open("/tmp/test.txt", "w") as f:
for line in sys.stdin:
f.write(line)
After inspecting the source code, I found out that closing the stream did not close the associated pipe at least in this case. Doing that manually did solve the issue:
...
in.close();
in.pipe().close();
child.wait(); // Does not hang.
The documentation warns that using synchronous IO to child processes is prone to deadlock.
Here's a minimal reword to async IO:
#include <boost/process.hpp>
#include <iostream>
namespace bp = boost::process;
int main() {
boost::asio::io_context ioc;
bp::async_pipe in{ioc};
bp::child child("./test.py", bp::std_in < in, bp::std_out.close());
for (auto msg : { "test1\n", "test2\n", "test3\n" }) {
write(in, bp::buffer(msg, strlen(msg)));
}
std::cerr << "Closing the pipe…\n";
in.close();
std::cerr << "Waiting for the child to exit…\n";
ioc.run(); // already awaits completion
child.wait(); // Parent seems to hang here.
}
You can make it more realistic by doing some delays:
#include <boost/process.hpp>
#include <iostream>
using namespace std::chrono_literals;
namespace bp = boost::process;
int main() {
boost::asio::io_context ioc;
bp::async_pipe in{ioc};
bp::child child("./test.py", bp::std_in < in, bp::std_out.close());
std::thread th([&] {
for (auto msg : { "test1\n", "test2\n", "test3\n" }) {
write(in, bp::buffer(msg, strlen(msg)));
std::this_thread::sleep_for(1s);
}
std::cerr << "Closing the pipe…\n";
in.close();
});
std::cerr << "Waiting for the child to exit…\n";
ioc.run(); // already awaits completion
th.join();
child.wait(); // Parent seems to hang here.
}
For fullblown async IO see other examples:
simultaneous read and write to child's stdio using boost.process
How to retrieve program output as soon as it printed?
Running a process using boost process in async mode with timeout

Proper way to handle SIGTERM with multiple threads

I have a multi threaded program on Raspberry in which I want to handle SIGTERM and shut everything down gracefully. The issue is that I have a background thread that has called recvfrom() on a blocking socket. As per my understanding from the man pages, if I exit my handler all the system calls should be woken up and return with -1 and errno set to EINTR. However in my case the recvfrom call keeps hanging.
1) In general am I understanding this right, that all threads that have blocking system calls that are able to be woken up by a signal should wake up in this scenario?
2) Could it be that the operating system is setting some special signal mask on my thead?
The interresting part is that I am using the VideoCore primitives, not pthread, maybe that could be the cause? Here is a small test example:
#include <iostream>
#include <cstdlib>
#include <cstring>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#include <signal.h>
#include "interface/vcos/vcos.h"
void SignalHandler(int nSignalNumber)
{
std::cout << "received signal " << nSignalNumber << std::endl;
}
void* ThreadMain(void* pArgument)
{
int nSocket = socket(AF_INET, SOCK_DGRAM, 0);
if (nSocket >= 0)
{
sockaddr_in LocalAddress;
memset(&LocalAddress, 0, sizeof(LocalAddress));
LocalAddress.sin_family = AF_INET;
LocalAddress.sin_addr.s_addr = INADDR_ANY;
LocalAddress.sin_port = htons(1234);
if (bind(nSocket, reinterpret_cast<sockaddr *>(&LocalAddress), sizeof(LocalAddress)) == 0)
{
sockaddr_in SenderAddress;
socklen_t nSenderAddressSize = sizeof(SenderAddress);
unsigned char pBuffer[512];
std::cout << "calling recvfrom()" << std::endl;
int nBytesReceived = recvfrom(nSocket, pBuffer, sizeof(pBuffer), 0, reinterpret_cast<struct sockaddr *>(&SenderAddress), &nSenderAddressSize);
if (nBytesReceived == -1)
{
if (errno == EINTR)
{
std::cout << "recvfrom() was interrupred by a signal" << std::endl;
}
else
{
std::cout << "recvfrom() failed with " << errno << std::endl;
}
}
}
else
{
std::cout << "bind() failed with " << errno << std::endl;
}
close(nSocket);
}
else
{
std::cout << "socket() failed with " << errno << std::endl;
}
return NULL;
}
int main(int argc, char** argv)
{
struct sigaction SignalAction;
memset(&SignalAction, 0, sizeof(SignalAction));
SignalAction.sa_handler = SignalHandler;
sigaction(SIGTERM, &SignalAction, NULL);
VCOS_THREAD_T Thread;
VCOS_STATUS_T nVcosStatus = vcos_thread_create(&Thread, "", NULL, ThreadMain, NULL);
if (nVcosStatus == VCOS_SUCCESS)
{
void* pData = NULL;
vcos_thread_join(&Thread, &pData);
}
else
{
std::cout << "vcos_thread_create() failed with " << nVcosStatus << std::endl;
}
return EXIT_SUCCESS;
}
It can be compiled like this:
g++ test.cpp -I/opt/vc/include -L/opt/vc/lib -lvcos -o test
When I run it and then call kill on the running instance the output is:
calling recvfrom()
received signal 15
and the process hangs. I'll try if a pthread behaves differently.
UPDATE
Ok I updated the sample to spawn a pthread thread as well and that one is not quitting as well. So I assume the signals are not populated to all threads?
#include <iostream>
#include <cstdlib>
#include <cstring>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#include <signal.h>
#include <pthread.h>
#include "interface/vcos/vcos.h"
void SignalHandler(int nSignalNumber)
{
std::cout << "received signal " << nSignalNumber << std::endl;
}
void* ThreadMain(void* pArgument)
{
const char* pThreadType = reinterpret_cast<const char*>(pArgument);
int nSocket = socket(AF_INET, SOCK_DGRAM, 0);
if (nSocket >= 0)
{
sockaddr_in LocalAddress;
memset(&LocalAddress, 0, sizeof(LocalAddress));
LocalAddress.sin_family = AF_INET;
LocalAddress.sin_addr.s_addr = INADDR_ANY;
LocalAddress.sin_port = htons(pThreadType[0] * 100);
if (bind(nSocket, reinterpret_cast<sockaddr *>(&LocalAddress), sizeof(LocalAddress)) == 0)
{
sockaddr_in SenderAddress;
socklen_t nSenderAddressSize = sizeof(SenderAddress);
unsigned char pBuffer[512];
std::cout << "calling recvfrom()" << std::endl;
int nBytesReceived = recvfrom(nSocket, pBuffer, sizeof(pBuffer), 0, reinterpret_cast<struct sockaddr *>(&SenderAddress), &nSenderAddressSize);
if (nBytesReceived == -1)
{
if (errno == EINTR)
{
std::cout << "recvfrom() was interrupred by a signal" << std::endl;
}
else
{
std::cout << "recvfrom() failed with " << errno << std::endl;
}
}
}
else
{
std::cout << "bind() failed with " << errno << std::endl;
}
close(nSocket);
}
else
{
std::cout << "socket() failed with " << errno << std::endl;
}
std::cout << pThreadType << " thread is exiting" << std::endl;
return NULL;
}
int main(int argc, char** argv)
{
struct sigaction SignalAction;
memset(&SignalAction, 0, sizeof(SignalAction));
SignalAction.sa_handler = SignalHandler;
sigaction(SIGTERM, &SignalAction, NULL);
VCOS_THREAD_T VcosThread;
VCOS_STATUS_T nVcosStatus = vcos_thread_create(&VcosThread, "", NULL, ThreadMain, const_cast<char*>("vcos"));
bool bJoinVcosThread = false;
if (nVcosStatus == VCOS_SUCCESS)
{
bJoinVcosThread = true;
}
else
{
std::cout << "vcos_thread_create() failed with " << nVcosStatus << std::endl;
}
pthread_t PthreadThread;
int nPthreadStatus = pthread_create(&PthreadThread, NULL, ThreadMain, const_cast<char*>("pthread"));
bool bJoinPthreadThread = false;
if (nPthreadStatus == 0)
{
bJoinPthreadThread = true;
}
else
{
std::cout << "pthread_create() failed with " << nPthreadStatus << std::endl;
}
if (bJoinVcosThread)
{
void* pData = NULL;
vcos_thread_join(&VcosThread, &pData);
}
if (bJoinPthreadThread)
{
void* pData = NULL;
pthread_join(PthreadThread, &pData);
}
return EXIT_SUCCESS;
}
A signal such as SIGTERM is submitted to one thread in the process only. The only precondition is that the chosen thread must either have not masked the signal, or must wait for it using sigwait. The other threads will not be directly notified that the signal has been delivered.
A common approach to combine signals with threads is to have a separate thread which handles signals only and notifies the other threads using thread synchronization mechanisms such as condition variables.
For interrupting file I/O, this may not be sufficient because there is a race condition between checking for a termination request and making the system call to perform the I/O operation. Some language run-time libraries use non-blocking I/O with poll or epoll with a special file descriptor which becomes ready on signal delivery (either using the previously-mentioned thread-based approach, or something Linux-specific like signalfd). Others try to avoid this overhead by using the read and write system calls directly with a complicated dance which uses dup2 to replace the file descriptor with one that always causes I/O to fail, thereby avoiding the race condition (but the bookkeeping needed for that is fairly complicated).
The manpage for signal reads:
If a signal handler is invoked while a system call or library function call is blocked, then either:
the call is automatically restarted after the signal handler returns; or
the call fails with the error EINTR.
Which of these two behaviors occurs depends on the interface and whether or not the signal handler was established using the SA_RESTART flag (see sigaction(2)). The details vary across UNIX systems<...>
A few lines below, recvfrom is listed among the functions that use SA_RESTART behavior by default. (Note: this behavior is disabled if there's a timeout on the socket, though.)
Thus, you should fill the sa_flags field of the sigaction structure to carefully avoid setting the SA_RESTART flag.
A good way to deal with blocking sockets -see socket(7)- (and even non blocking ones) is to use a multiplexing syscall like poll(2) (or the obsolete select(2)....)
Regarding signals, be sure to read signal(7) and signal-safety(7).
A common way to handle signals with some event loop (using poll(2)) is to have a signal handler which simply write(2)-s a byte on a pipe(7) to self (you'll setup the pipe at initialization, and you'll poll it in your event loop). The Qt documentation explains how and why. You might also use the Linux specific signalfd(2).

Reading ftrace pipes with Boost::asio posix stream_descriptor

I'm trying to build an application that reads from the ftrace pipes at the debug fs.
It seems that when trying to read asynchronously from trace_pipe or
trace_pipe_raw using boost::asio API, the events waiting in pipe are being
processed and printed to screen by the async_read handle, but new events that arrive after the program started don't trigger the async_read handle.
Running the sample code below, i'm getting a print of all events waiting in queue but i don't get any print for new events that arrive later.
The same sample works perfectly if i'm trying to read from manually created pipe using mkfifo but doesn't work for the ftrace pipes.
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <string>
#include <iostream>
namespace asio = boost::asio;
#ifdef BOOST_ASIO_HAS_POSIX_STREAM_DESCRIPTOR
typedef asio::posix::stream_descriptor stream_descriptor;
#endif
class PipeReader
{
typedef std::shared_ptr<PipeReader> PipeReaderPtr;
typedef std::weak_ptr<PipeReader> PipeReaderWeakPtr;
public:
static PipeReaderWeakPtr Create(asio::io_service& io_service, const std::string& path);
void HandleRead(PipeReaderPtr me, const boost::system::error_code &error);
private:
PipeReader(asio::io_service& io_service, const std::string& path);
stream_descriptor m_pipe;
char buf[4096];
};
PipeReader::PipeReaderWeakPtr PipeReader::Create(asio::io_service& io_service, const std::string& path)
{
PipeReaderPtr ptr(new PipeReader(io_service, path));
ptr->m_pipe.async_read_some(boost::asio::buffer(ptr->buf),
boost::bind(&PipeReader::HandleRead,
ptr.get(),
ptr,
asio::placeholders::error));
return ptr;
}
PipeReader::PipeReader(asio::io_service& io_service, const std::string& path)
: m_pipe(io_service)
{
int dev = open(path.c_str(), O_RDWR);
if (dev == -1) {
std::cout << "failed to open path - " << path << std::endl;
}
else
{
m_pipe.assign(dev);
}
}
void PipeReader::HandleRead(PipeReaderPtr me, const boost::system::error_code &error)
{
if (!error) {
std::string str(me->buf);
std::cout << "got message: " << str << std::endl;
m_pipe.async_read_some(boost::asio::buffer(me->buf),
boost::bind(&PipeReader::HandleRead,
this,
me,
asio::placeholders::error));
}
else
{
std::cout << "got error - " << error.message() << std::endl;
}
}
int main()
{
boost::asio::io_service io_service;
boost::asio::io_service::work dummy(io_service);
PipeReader::Create(io_service, "/sys/kernel/debug/tracing/trace_pipe");
io_service.run();
return 0;
}
I found the problem. it was a bug in the implementation of ftrace that caused the epoll to hang.
The bug was fixed at kernel 3.16.
correspondence thread,
commit in git hub

Boost.Asio - polling a named pipe

I am trying to listen for input on a named pipe. I'm using Boost.Asio's stream_descriptor and async_read under Linux. The problem is, the call to io_service::run() only blocks like I want it to until the first read. After that, it just keeps calling the handler immediately with the "End of file" error, even though I try to attach more async_reads to it. The code I have is equivalent to the following:
boost::asio::io_service io_service;
int fifo_d = open("/tmp/fifo", O_RDONLY);
boost::asio::posix::stream_descriptor fifo(io_service, fifo_d);
while (true)
{
// buffer and handler probably aren't important for the purposes of this question
boost::asio::async_read(fifo, buffer, handler);
io_service.run();
}
Only the first async_read works as I expect it to. Subsequent async_reads just return immediately. The only way I found to make it work like I want is to close and reopen the named pipe, but it seems like a hack:
boost::asio::io_service io_service;
while (true)
{
int fifo_d = open("/tmp/fifo", O_RDONLY);
boost::asio::posix::stream_descriptor fifo(io_service, fifo_d);
boost::asio::async_read(fifo, buffer, handler);
io_service.run();
close(fifo_d);
}
Can anyone tell me what am I doing wrong?
UPDATE: Here's a simple "read" version, which allowed for some code simplification, the problem remains the same:
int fifo_d = open("/tmp/fifo", O_RDONLY);
boost::asio::posix::stream_descriptor fifo(io_service, fifo_d);
while (true) {
try {
boost::asio::read(fifo, boost::asio::buffer(buffer));
}
catch (boost::system::system_error& err) {
// It loops here with "read: End of file" error
std::cout << err.what() << std::endl;
}
}
This is not how works. run() is not intended to be called in a loop. If you insist, you need to call reset() in between (as per the documentation).
Also, if you /want/ blocking behaviour, why are you using the async_* interface?
Demos
Consider using a simple iostream to read the fd:
Live On Coliru
#include <iostream>
#include <fstream>
int main() {
std::ifstream fifo("/tmp/fifo");
std::string word;
size_t lineno = 0;
while (fifo >> word) {
std::cout << "word: " << ++lineno << "\t" << word << "\n";
}
}
Or if you must attach to some fd you get from somewhere else, use file_descriptor from Boost IOstreams:
Live On Coliru
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
#include <iostream>
#include <fcntl.h>
int main() {
namespace io = boost::iostreams;
using src = io::file_descriptor_source;
io::stream<src> fifo(src(open("./fifo", O_RDONLY), io::file_descriptor_flags::close_handle));
std::string word;
size_t number = 0;
while (fifo >> word) {
std::cout << "word: " << ++number << "\t" << word << "\n";
}
}
Both examples print the expected:
word: 1 hello
word: 2 world
As also sehe reported, that's not the way boost::asio works.
The ioservice::run() method runs in blocking mode while it has some work. When the ioservice goes out of work you have to call the reset() method before putting other work, so that's why in your first code the async_read is done only once.
A common pattern in this case would look something like:
void handler(...) {
if (!error) {
// do your work
boost::asio::async_read(fifo, buffer, handler); // <-- at the end of the handler a subsequent async_read is put to the ioservice, so it never goes out-of-work
}
}
boost::asio::io_service io_service;
int fifo_d = open("/tmp/fifo", O_RDONLY);
boost::asio::posix::stream_descriptor fifo(io_service, fifo_d);
boost::asio::async_read(fifo, buffer, handler); // <-- you call async_read only once here.
io_service.run(); //<-- this blocks till an error occurs