I am writing some binary data to a device fie like /dev/itun.
void ahaConnector::asyncWriteData(vector<uint8_t> packedMessage) {
cout<<"\n async write data packed message";
deviceStreamDescriptor.assign(device);
boost::asio::write (
deviceStreamDescriptor,
boost::asio::buffer(packedMessage)
);
readbuffer.resize(1024);
deviceStreamDescriptor.async_read_some(boost::asio::buffer(readbuffer),
boost::bind(&ahaConnector::readHeader, this,
boost::asio::placeholders::error(),
boost::asio::placeholders::bytes_transferred()
));
io_service.run();
}
void ahaConnector::readHeader(const boost::system::error_code &ec, std::size_t bytes_transferred) {
if(!ec) {
std::cout<<"\n Bytes transfereed :"<<bytes_transferred<<" "<<readbuffer.size();
deviceStreamDescriptor.async_read_some(boost::asio::buffer(readbuffer),
boost::bind(&ahaConnector::readHeader, this,
boost::asio::placeholders::error(),
boost::asio::placeholders::bytes_transferred()
));
Callbacks callbacks;
callbacks.OnReceivingPackedMessage();
io_service.run();
}
else {
cout<<"\n System Error Code "<<ec;
}
}
The callback function readhandler is getting executed successfully, however I am not able to transfer the control from my Callback function to another class.
Is something wrong from the design perspective. I need to handle the message received from the callback function for further logic. Should I use another thread here ?
Looking at this code you might just want to replace the read(device,...) by boost Asio's support for Posix streams:
#include <boost/asio.hpp>
#include <boost/asio/posix/stream_descriptor.hpp>
#include <boost/function.hpp>
#include <iostream>
static int device = 0;
using namespace boost;
int main() {
boost::asio::io_service io_svc;
boost::asio::posix::stream_descriptor iodevice(io_svc, device);
char buffer[1024];
function<void(system::error_code const&, size_t)> callback;
callback = [&](boost::system::error_code const& ec, size_t bytes_transferred) {
if (ec)
{
std::cout << "Error '" << ec.message() << "' during asynchronous operation\n";
}
else
{
std::cout << "Read exactly " << bytes_transferred << " bytes\n";
std::cout << "Data: '";
std::cout.write(buffer, bytes_transferred);
std::cout << "'\n";
iodevice.async_read_some(asio::buffer(buffer), callback);
}
};
iodevice.async_read_some(asio::buffer(buffer), callback);
io_svc.run();
}
See it Live On Coliru.
Sadly on Coliru it can't work because input is redirected from a non-stream. But if you run it interactively it will work and print the first 10 characters entered.
The answer depends on exactly what are the properties of the device. Check the documentation for the device driver you're trying to use. If the device supports non-blocking I/O, open the device with O_NONBLOCK, and use poll() to wait for device to be available for reading or writing.
If the device does not support non-blocking I/O, the only viable option would be to use a separate thread to read and/or write to the device, and use the background thread to construct facade that pretends and behaves like a non/blocking data source and sink.
Related
I'm converting an application from using Juce asynchronous i/o to asio. The first part is to rewrite the code that receives traffic from another application on the same machine (it's a Lightroom Lua plugin that sends \n delimited messages on port 58764). Whenever I successfully connect to that port with my C++ program, I get a series of error codes, all the same:
An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.
Can someone point out my error? I can see that the socket is successfully opened. I've reduced this from my full program to a minimal example. I also tried it with connect instead of async_connect and had the same problem.
#include <iostream>
#include "asio.hpp"
asio::io_context io_context_;
asio::ip::tcp::socket socket_{io_context_};
void loop_me()
{
asio::streambuf streambuf{};
while (true) {
if (!socket_.is_open()) {
return;
}
else {
asio::async_read_until(socket_, streambuf, '\n',
[&streambuf](const asio::error_code& error_code, std::size_t bytes_transferred) {
if (error_code) {
std::cerr << "Socket error " << error_code.message() << std::endl;
return;
}
// Extract up to the first delimiter.
std::string command{buffers_begin(streambuf.data()),
buffers_begin(streambuf.data()) + bytes_transferred};
std::cout << command << std::endl;
streambuf.consume(bytes_transferred);
});
}
}
}
int main()
{
auto work_{asio::make_work_guard(io_context_)};
std::thread io_thread_;
std::thread run_thread_;
io_thread_ = std::thread([] { io_context_.run(); });
socket_.async_connect(asio::ip::tcp::endpoint(asio::ip::address_v4::loopback(), 58764),
[&run_thread_](const asio::error_code& error) {
if (!error) {
std::cout << "Socket connected in LR_IPC_In\n";
run_thread_ = std::thread(loop_me);
}
else {
std::cerr << "LR_IPC_In socket connect failed " << error.message() << std::endl;
}
});
std::this_thread::sleep_for(std::chrono::seconds(1));
socket_.close();
io_context_.stop();
if (io_thread_.joinable())
io_thread_.join();
if (run_thread_.joinable())
run_thread_.join();
}
You are trying to start an infinite number of asynchronous read operations at the same time. You shouldn't start a new asynchronous read until the previous one finished.
async_read_until returns immediately, even though the data hasn't been received yet. That's the point of "async".
I have this requirement where my app have to connect to another app via sockets and will have to maintain persistent connection for quiet long time. My app will be a TCP client and the other is a TCP server. My app will send commands and the server will respond accordingly.
The problem am facing right now is how to read the whole data from server a string and return for app which will issue the next command. Reading synchronously (with asio::read) looked like a good option up until I observed socket hanging up until I terminate the server. Looking at the documentation I found that the library is correctly working.
his function is used to read a certain number of bytes of data from a stream. The call will block until one of the following conditions is true:
1. The supplied buffers are full. That is, the bytes transferred is equal to the sum of the buffer sizes.
2. An error occurred.
The problem is I don't know correct buffer size as the response from the server varies. So If I put a too small buffer it returns fine but missing some data. If I put too big it will hang forever until server quits.
So I thought I would do the async reading. It works only once and I don't know how to make it fetch data until whole data it read.
here is the relevant async code
#define ASIO_STANDALONE 1
#include <iostream>
#include <asio.hpp>
int main()
{
asio::io_context context;
size_t reply_length;
size_t length = 1024;
std::vector<char> buffer;
//create socket
asio::ip::tcp::socket socket(context);
socket.connect(asio::ip::tcp::endpoint(asio::ip::address::from_string("127.0.0.1"), 8088));
std::string dataOut = "list --files"; //some command to write
std::error_code error;
asio::write(socket, asio::buffer(dataOut), error);
if (!error)
{
std::cout << "Receiving...!" << std::endl;
buffer.resize(length);
asio::async_read(socket, asio::buffer(buffer), [&buffer, &context](const asio::error_code &ec, std::size_t bytes_transferred) {
std::copy(buffer.begin(), buffer.end(), std::ostream_iterator<char>(std::cout, ""));
std::cout << "\nRead total of:" << bytes_transferred << "\n";
context.run();
});
}
else
{
std::cout << "send failed: " << error.message() << std::endl;
}
context.run();
}
Searching didn't help much solving my issue.
So my question is, how can I read all the data in a persistent socket with asio? Am not using boost.
You need to loop async_read calls. If you don't want your client to hang on read operation you can define the smallest possible buffer i.e. 1 byte.
Define function which takes socket, buffer and two additional parameters according to async_read's handler signature, and this function calls itself with async_read to make the loop of async_read calls - it reads until some error occures:
void onRead (
asio::ip::tcp::socket& socket,
std::array<char,1>& buf,
const system::error_code& ec,
std::size_t bytes)
{
if (ec)
{
if (ec == asio::error::eof && bytes == 1)
std::cout << buf[0];
return;
}
std::cout << buf[0];
asio::async_read(socket,asio::buffer(buf),
std::bind(onRead, std::ref(socket), std::ref(buf),
std::placeholders::_1, // error code
std::placeholders::_2)); // transferred bytes
}
and the changes in main:
std::array<char,1> buf;
asio::write(socket, asio::buffer(dataOut), error);
if (!error)
{
std::cout << "Receiving...!" << std::endl;
asio::async_read(socket, asio::buffer(buf),
std::bind(onRead, std::ref(socket), std::ref(buf),
std::placeholders::_1,
std::placeholders::_2));
context.run();
}
else
{
std::cout << "send failed: " << error.message() << std::endl;
}
(I am using Boost, so you should replace system::error_code on asio::error_code).
I am using boost::process::child and boost::process::async_pipe to start an application and read asynchronously (through the means of boost::asio) everything that app outputs on screen whenever this happens.
I want to check also if the application is alive by using child::running() method; if not running I'd like to read the exit code using child::exit_code.
This is very useful ESPECIALLY as it is a way to be notified about an application crashing or exiting unexpectedly (I could not find a better way); when the app exits the callback is called with boost::system::error_code set.
Do you know if I can use these two methods inside the callback called by async_pipe::async_read_some ?
In general the much more simple question would be if child::running() and child::exit_code() are thread safe (in both Windows and Linux).
namespace bp = boost::process;
char my_buffer[1024];
boost::asio::io_service io;
bp::async_pipe in_pipe(io);
void handle_pipe_read(const boost::system::error_code &ec, std::size_t bytes_transferred);
void schedule_read() {
in_pipe.async_read_some(
boost::asio::buffer(my_buffer),
boost::bind(&handle_pipe_read,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
void handle_pipe_read(
const boost::system::error_code &ec,
std::size_t bytes_transferred
)
{
// Q: Is this call possible? 'handle_pipe_read' can run in any thread
if(c->running())
std::cout << "I am alive" << std::endl;
else
std::cout << "EXIT CODE:" << c->exit_code() << std::endl;
if(ec) return; //app probably exit
// Do something with buffer and re-schedule
schedule_read();
}
int main() {
bp::child c("my_program_url", bp::std_out > in_pipe);
any_c = &c;
schedule_read();
io.run();
}
Since you only run the io_service::run() on the main thread, all completion handlers also run there. There's no threading.
Remember to pass the io_service to the child, and use the on_exit handler:
Live On Coliru
#include <boost/process.hpp>
//#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <system_error>
#include <utility>
#include <iostream>
namespace bp = boost::process;
char my_buffer[1024];
boost::asio::io_service io;
bp::async_pipe in_pipe(io);
void handle_pipe_read(const boost::system::error_code &ec, std::size_t bytes_transferred);
void schedule_read() {
in_pipe.async_read_some(
boost::asio::buffer(my_buffer),
boost::bind(&handle_pipe_read, _1, _2));
}
void handle_pipe_read(const boost::system::error_code &ec, std::size_t bytes_transferred) {
if (ec)
return; // app probably exit
// Do something with buffer and re-schedule
std::cout.write(my_buffer, bytes_transferred);
if (in_pipe.is_open())
schedule_read();
}
int main() {
bp::child c("/bin/ls", bp::std_out > in_pipe,
bp::on_exit([](int code, std::error_code ec) {
std::cout << "Child exited (" << code << "): " << ec.message() << std::endl;
in_pipe.close();
}), io);
schedule_read();
io.run();
std::cout << "Service done (" << c.exit_code() << ")" << std::endl;
}
Prints:
a.out
main.cpp
Child exited (0): Success
Service done (0)
The only solution that worked for me is the following
schedule a completion read
always check calling child::running()
on error_code set don't reschedule
When the pipe gets broken (because of a crash) the completion handler for a read has the boost::error_code argument set to true. Despite this I've seen cases where child::running() is false also when boost::error_code is NOT set.
I found this question which asks how to read input asynchronously, but will only work with POSIX stream descriptors, which won't work on Windows. So, I found this tutorial which shows that instead of using a POSIX stream descriptor I can use a boost::asio::windows::stream_handle.
Following both examples I came up with the code below. When I run it, I cannot type anything into the command prompt, as the program immediately terminates. I'd like it to capture any input from the user, possibly into a std::string, while allowing other logic within my program to execute (i.e. perform asynchronous I/O from a Windows console).
Essentially, I'm trying to avoid blocking my program when it attempts to read from stdin. I do not know if this is possible in Windows, as I also found this post which details problems another user encountered when trying to do the same thing.
#define _WIN32_WINNT 0x0501
#define INPUT_BUFFER_LENGTH 512
#include <cstdio>
#include <iostream>
#define BOOST_THREAD_USE_LIB // For MinGW 4.5 - (https://svn.boost.org/trac/boost/ticket/4878)
#include <boost/bind.hpp>
#include <boost/asio.hpp>
class Example {
public:
Example( boost::asio::io_service& io_service)
: input_buffer( INPUT_BUFFER_LENGTH), input_handle( io_service)
{
// Read a line of input.
boost::asio::async_read_until( input_handle, input_buffer, "\r\n",
boost::bind( &Example::handle_read, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
void handle_read( const boost::system::error_code& error, std::size_t length);
void handle_write( const boost::system::error_code& error);
private:
boost::asio::streambuf input_buffer;
boost::asio::windows::stream_handle input_handle;
};
void Example::handle_read( const boost::system::error_code& error, std::size_t length)
{
if (!error)
{
// Remove newline from input.
input_buffer.consume(1);
input_buffer.commit( length - 1);
std::istream is(&input_buffer);
std::string s;
is >> s;
std::cout << s << std::endl;
boost::asio::async_read_until(input_handle, input_buffer, "\r\n",
boost::bind( &Example::handle_read, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
else if( error == boost::asio::error::not_found)
{
std::cout << "Did not receive ending character!" << std::endl;
}
}
void Example::handle_write( const boost::system::error_code& error)
{
if (!error)
{
// Read a line of input.
boost::asio::async_read_until(input_handle, input_buffer, "\r\n",
boost::bind( &Example::handle_read, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
}
int main( int argc, char ** argv)
{
try {
boost::asio::io_service io_service;
Example obj( io_service);
io_service.run();
} catch( std::exception & e)
{
std::cout << e.what() << std::endl;
}
std::cout << "Program has ended" << std::endl;
getchar();
return 0;
}
I just spent an hour or two investigating this topic so decided to post to prevent others to waste their time.
Windows doesn't support IOCP for standard input/output handles. When you take the handle by GetStdHandle(STD_INPUT_HANDLE), the handle doesn't have FILE_FLAG_OVERLAPPED set so it doesn't support overlapped (async) IO. But even if you
CreateFile(L"CONIN$",
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING,
NULL);
WinAPI just ignore dwFlagsAndAttributes and again returns the handle that doesn't support overlapped IO. The only way to get async IO of console input/output is to use the handle with WaitForSingleObject with 0 timeout so you can check if there's anything to read non-blocking. Not exactly async IO but can avoid multithreading if it's a goal.
More details about console API: https://msdn.microsoft.com/en-us/library/ms686971(v=VS.85).aspx
What's the difference between handles returned by GetStdHandle and CreateFile is described here: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682075(v=vs.85).aspx. In short the difference is only for a child processes when CreateFile can give access to its console input buffer even if it was redirected in the parent process.
You need to invoke io_service::run() to start the event processing loop for asynchronous operations.
class Example {
public:
Example( boost::asio::io_service& io_service )
: io_service(io_service), input_buffer( INPUT_BUFFER_LENGTH), input_handle( io_service)
{
}
void start_reading();
void handle_read( const boost::system::error_code& error, std::size_t length);
void handle_write( const boost::system::error_code& error);
private:
boost::asio::io_service& io_service;
boost::asio::streambuf input_buffer;
boost::asio::windows::stream_handle input_handle;
};
int main( int argc, char * argv)
{
boost::asio::io_service io_service;
Example obj( io_service );
obj.start_reading();
io_service.run();
return 0;
}
You need to initialize your stream_handle to the console input handle. You can't use the same stream_handle for input and for output because those are two different handles.
For input:
Example()
: /* ... */ input_handle( io_service, GetStdHandle(STD_INPUT_HANDLE) )
For output you would use CONSOLE_OUTPUT_HANDLE. But that is probably overkill, you're unlikely to be pushing that much data into stdout on windows that you'd need to use an async write.
I created a console application that sends data on a network link. I used the boost library, both the thread and the asio ones; currently i'm running it under Windows. If I run a single application it works perfectly, but if I open two instances on two different consoles, the CPU load goes to 100%, if I close one of the application it goes back to normal. I just used a simple socket with async reads and writes, and threads with condition variables and mutexes. Is there any special thingh to do when dealing with such a situation? I can show you some code, but I think it's nothing special:
socket->connect(tcp::endpoint(address::from_string(getAddress()),getPort()));
for connecting
and
socket->async_read_some(buffer(receiveData),bind(&NetworkLink::handle_response, this,placeholders::error,placeholders::bytes_transferred));
inside the handle_response function for async reading.
For the thread I use
boost::unique_lock<boost::mutex> messages_lock(message_received_mutex);
Before deleting everything and starting a simple test project from scratch I would like to know if there are any special care to be taken in this situation.
Ok it seems I've tracked down the problem. First of all, the 100% CPU usage was due to the fact that each instance was using 50% of the CPU (I'm on a dual core PC). So I run all over the code and found out this. I had this in my code, inside the NetworkLink::handle_response function:
socket->async_read_some(
boost::asio::buffer(receiveData),
boost::bind(&NetworkLink::handle_response, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
io_service.run();
I used the io_service.run(); because before the software was not receiving data. Now I removed the line, I don't get the 50% CPU usage, but the handler response is not called so I cannot receive any data. Any thought about this?
Thanks
PS: I created a small app that shows this problem:
#include <iostream>
#include <boost/asio.hpp>
#include "boost/thread.hpp"
#include "boost/thread/mutex.hpp"
#ifdef _WIN32
#include "Windows.h"
#endif
using namespace boost::asio::ip;
using namespace std;
std::vector<uint8_t> buf;
boost::asio::io_service io_service;
boost::asio::ip::tcp::socket mysocket(io_service);
int handle_response(const boost::system::error_code &err,
size_t bytes_transferred)
{
// cout << bytes_transferred << ' ';
if (bytes_transferred > 0)
cout << buf.data() << ' ';
boost::asio::async_read(mysocket, boost::asio::buffer(buf),
boost::asio::transfer_at_least(1), &handle_response);
}
int mythread()
{
boost::asio::async_read(mysocket, boost::asio::buffer(buf),
boost::asio::transfer_at_least(1), &handle_response);
io_service.run();
}
int main()
{
int m_nPort = 12345;
buf.resize(100, '0');
boost::condition_variable message_received_condition;
boost::thread message_receiver_thread(&mythread);
boost::mutex messages_mutex;
tcp::endpoint endpoint(boost::asio::ip::address::from_string("127.0.0.1"),
m_nPort);
boost::unique_lock<boost::mutex> messages_lock(messages_mutex);
message_received_condition.notify_one();
cout << "Waiting for connection..." << endl;
Sleep(10000);
mysocket.connect(endpoint);
cout << "connection accepted" << endl;
try
{
while (true)
{
boost::system::error_code ec;
boost::asio::socket_base::bytes_readable command(true);
mysocket.io_control(command);
std::size_t bytes_readable = command.get();
mysocket.async_read_some(boost::asio::buffer(buf),
&handle_response);
io_service.run();
}
} catch (exception &e)
{
cerr << e.what() << endl; //"The parameter is incorrect" exception
}
}
If you remove the comment from th line
// cout << bytes_transferred << ' ';
in the handle response functionyou get a lower CPU usage, I guess because of the delay for writing to the screen.
You are not checking the error. If there is a failure, you continue to read anyway, which will probably immediately post back a completion with a failure, ad infinitum.