How to discard data as it is sent with boost::asio? - c++

I'm writing some code that reads and writes to serial device using boost::asio class. However, when sending several strings between programs, I've noticed that on the receiving program the data is read in the order as it was written to the serial port, and not as the data is sent from the other program - If I start reading data some seconds later, I don't get the values that I am sending at the moment but those that were sent previously. I'm assuming this is caused by how I am setting up my boost::asio::serial_port:
int main(int argc, char const *argv[]){
int baud=atoi(argv[1]);
std::string pty=argv[2];
printf("Virtual device: %s\n",pty.data());
printf("Baud rate: %d\n",baud);
boost::asio::io_service io;
boost::asio::serial_port port(io, pty);
port.set_option(boost::asio::serial_port_base::baud_rate(baud));
// counter that writes to serial port in 1s intervals
int val=0;
while (1){
std::string data=std::to_string(val);
data+='\n';
std::cout << data;
write(port,boost::asio::buffer(data.c_str(),data.size()));
sleep(1);
val++;
data.clear();
}
port.close();
return 0;
}
Is there a way to force past data to be discarded as soon as a new value is sent to the serial port (which I assume should be done on the write() part of the code)?

Boost.Asio does not provide a higher-level abstraction for flushing a serial port's buffers. However, this can often be accomplished by having platform specific calls, such as tcflush() or PurgeComm(), operate on a serial port's native_handle().
Each serial port has a receive and transmit buffer, and flushing operates on one or both of the buffers. For example, if two serial ports are connected (/dev/pts/3 and /dev/pts/4), and program A opens and writes to /dev/pts/3, then it can only flush the buffers associated with /dev/pts/3 (data received on /dev/pts/3 but not read, and data written to /dev/pts/3 but not transmitted). Therefore, if program B starts, opens /dev/pts/4, and wants to read non-stale data, then program B needs to flush the receive buffer for /dev/pts/4 after opening the serial port.
Here is a complete example running on CentOs. When the example runs as a writer, it will write a sequentially increasing number to the serial port once a second. When the example runs as a writer, it will read five numbers, sleep for 5 seconds and flush its read buffer every other iteration:
#include <iostream>
#include <vector>
#include <boost/asio.hpp>
#include <boost/thread.hpp>
/// #brief Different ways a serial port may be flushed.
enum flush_type
{
flush_receive = TCIFLUSH,
flush_send = TCOFLUSH,
flush_both = TCIOFLUSH
};
/// #brief Flush a serial port's buffers.
///
/// #param serial_port Port to flush.
/// #param what Determines the buffers to flush.
/// #param error Set to indicate what error occurred, if any.
void flush_serial_port(
boost::asio::serial_port& serial_port,
flush_type what,
boost::system::error_code& error)
{
if (0 == ::tcflush(serial_port.lowest_layer().native_handle(), what))
{
error = boost::system::error_code();
}
else
{
error = boost::system::error_code(errno,
boost::asio::error::get_system_category());
}
}
/// #brief Reads 5 numbers from the serial port, then sleeps for 5 seconds,
/// flushing its read buffer every other iteration.
void read_main(boost::asio::serial_port& serial_port)
{
std::vector<unsigned char> buffer(5);
for (bool flush = false;; flush = !flush)
{
std::size_t bytes_transferred =
read(serial_port, boost::asio::buffer(buffer));
for (std::size_t i = 0; i < bytes_transferred; ++i)
std::cout << static_cast<unsigned int>(buffer[i]) << " ";
boost::this_thread::sleep_for(boost::chrono::seconds(5));
if (flush)
{
boost::system::error_code error;
flush_serial_port(serial_port, flush_receive, error);
std::cout << "flush: " << error.message() << std::endl;
}
else
{
std::cout << "noflush" << std::endl;
}
}
}
/// #brief Write a sequentially increasing number to the serial port
/// every second.
void write_main(boost::asio::serial_port& serial_port)
{
for (unsigned char i = 0; ; ++i)
{
write(serial_port, boost::asio::buffer(&i, sizeof i));
boost::this_thread::sleep_for(boost::chrono::seconds(1));
}
}
int main(int argc, char* argv[])
{
boost::asio::io_service io_service;
boost::asio::serial_port serial_port(io_service, argv[2]);
if (!strcmp(argv[1], "read"))
read_main(serial_port);
else if (!strcmp(argv[1], "write"))
write_main(serial_port);
}
Create virtual serial ports with socat:
$ socat -d -d PTY: PTY:
2014/03/23 16:22:22 socat[12056] N PTY is /dev/pts/3
2014/03/23 16:22:22 socat[12056] N PTY is /dev/pts/4
2014/03/23 16:22:22 socat[12056] N starting data transfer loop with
FDs [3,3] and [5,5]
Starting both the read and write examples:
$ ./a.out read /dev/pts/3 & ./a.out write /dev/pts/4
[1] 12238
0 1 2 3 4 noflush
5 6 7 8 9 flush: Success
14 15 16 17 18 noflush
19 20 21 22 23 flush: Success
28 29 30 31 32 noflush
33 34 35 36 37 flush: Success
As demonstrating in the output, numbers are only skipped in the sequence when the reader flushes its read buffer: 3 4 noflush 5 6 7 8 9 flush 14 15.

Related

boost::asio: register fd for EPOLLIN / EPOLLOUT once and leave it registered

I have a tcp client which is serviced by a boost::asio::io_context running on a single thread. It is configured non-blocking.
Reads/writes to this client are only ever done within the context of this thread.
I am using async_wait to await for the socket to become readable/writeable.
void Client::awaitReadable()
{
_socket.async_wait(tcp::socket::wait_read, std::bind_front(&Client::onReadable, this));
}
Whenever the socket becomes readable, my onReadable callback is fired, and I read all available data until receive asio::error::would_block.
void Client::onReadable(boost::system::error_code ec)
{
if (!ec)
{
while (1) // drain the socket
{
const std::size_t len = _socket.read_some(_read_buf.writeBuf(), ec);
if (ec)
break;
else
_read_buf.advance(len);
}
}
if (ec == asio::error::would_block)
{
const std::size_t read = _read_cb(*this, _read_buf.readBuf();
_read_buf.dataRead(read);
awaitReadable(); // I have to await readable again
}
else
{
onDisconnected(ec);
}
}
Once I've drained the socket I then need to call awaitReadable again to re-register my onReadable callback.
This necessarily involves a call to epoll_ctl, which effectively changes absolutely nothing.
When writing to the socket, the process if similar.
First, if the socket is currently writeable, I will attempt to send the data immediately. If, during the write, the I receive asio::error::would_block, I buffer the remaining unsent data and call my awaitWriteable function
void Client::write(Data buf)
{
if (_writeable)
{
const auto [ sent, ec ] = doWrite(buf); // calls awaitWriteable if would_block
if (ec == asio::error::would_block)
_write_buf.add(buf.data() + sent, buf.size() - sent);
}
else
{
_write_buf.add(buf); // will be sent when socket becomes writeable
}
}
The awaitWriteable function is very similar to the awaitReadable version
void Client::awaitWriteable()
{
_socket.async_wait(tcp::socket::wait_write, std::bind_front(&Client::onWriteable, this));
}
When the socket becomes writeable again I will be notified, and I will write more data to the socket.
void Client::onWriteable(boost::system::error_code ec)
{
if (!ec)
{
_writeable = true;
if (!_write_buf.empty())
{
const auto [ sent, ec ] = doWrite(_write_buf.writeBuf());
if (!ec)
_write_buf.sent(sent);
}
}
else
{
onDisconnected(ec);
}
}
The actual writing is factored out into a separate function as it is called both by the "synchronous write" function, and from the OnWriteable callback
std::pair<std::size_t, boost::system::error_code> Client::doWrite(Data buf)
{
boost::system::error_code ec;
std::size_t sent = _socket.write_some(buf + sent, ec);
if (ec)
{
if (ec == asio::error::would_block)
awaitWriteable();
else
onDisconnected(ec);
}
return {sent, ec};
}
So the way reads work is
awaitReadable.
when readable, read everything until would_block.
repeat.
and the way writes work is
once connected awaitWriteable.
when writeable, set a flag true, and if any data is pending, send as much as possible.
if the send results in would_block then awaitWriteable again.
when a client wants to send data, if the socket is currently writeable then "synchronously" send as much as possible.
if the send results in would_block then buffer any unsent data and awaitWriteable again.
Question:
I would like to register my socket file descriptor with epoll, and leave it registered forever.
Is there any way to side-step this need to continually call awaitReadable/awaitWriteable?
You're mixing sync/async primitives. So at least the blanket claim "It is configured non-blocking" is inaccurate, because asio is having to switch it for you when you mix sync primitives.
Note: not all Asio-aware IO objects support this. E.g. Beast's tcp_stream (and ssl_stream) object do explicitly not support mixing synchronous and asynchronous operations.
This necessarily involves a call to epoll_ctl, which effectively changes absolutely nothing.
Have you checked? Because it's up to the service implementation to decide how your handlers are serviced. It might be the case that fds are added and removed from the pollfd set. It might do cleverer things. It might not even use (e)poll on a particular system.
Regardless, is there something stopping you from using read operations directly in a loop. You can even used composed read operations, such as asio::async_read_until or asio::async_read with a CompletionCondition.
E.g. in to read incoming data in a loop, returning whenever 1024 bytes or more have been received:
void read_loop() {
net::async_read(
_socket, _read_buf, net::transfer_at_least(1024),
[this](error_code ec, size_t xferred) {
std::cout << "Received " << xferred //
<< " (" << ec.message() << ")" << std::endl;
if (!ec)
read_loop();
});
}
Here's a live demo reading itself:
Live On Coliru
#include <boost/asio.hpp>
#include <iostream>
namespace net = boost::asio;
using boost::system::error_code;
using net::ip::tcp;
using namespace std::chrono_literals;
struct Client {
Client(net::any_io_executor ex, tcp::endpoint ep) : _socket(ex) {
_socket.connect(ep);
assert(_socket.is_open());
std::cout << "Connected " << ep << " from " << _socket.local_endpoint() << "\n";
}
void read_loop() {
net::async_read(
_socket, _read_buf, net::transfer_at_least(1024),
[this](error_code ec, size_t xferred) {
std::cout << "Received " << xferred //
<< " (" << ec.message() << ")" << std::endl;
if (!ec)
read_loop();
});
}
auto get_histo() const {
std::array<unsigned, 256> histo {0};
auto f = buffers_begin(_read_buf.data()),
l = buffers_end(_read_buf.data());
while (f != l)
++histo[uint8_t(*f++)];
return histo;
}
private:
net::streambuf _read_buf;
tcp::socket _socket;
};
int main() {
net::io_context ioc;
Client c(ioc.get_executor(), {{}, 8989});
c.read_loop();
ioc.run_for(10s); // time limit for online compilers
// do something witty with the result
auto histo = c.get_histo();
for (uint8_t ch : {'a','q','e','x'})
std::cout << "Frequency of '" << ch << "' was " << histo[ch] << "\n";
}
Prints
Connected 0.0.0.0:8989 from 127.0.0.1:48730
Received 1024 (Success)
Received 447 (End of file)
Frequency of 'a' was 38
Frequency of 'q' was 2
Frequency of 'e' was 92
Frequency of 'x' was 8
In about 10ms.
BONUS: Profling epoll_ctl calls
Here is the same program eating a dictionay on my machine, while counting calls to epoll_ctl:
Note how only 3 epoll_ctl calls are ever issued:
Connected 0.0.0.0:8989 from 127.0.0.1:52974
Received 1024 (Success)
Received 1024 (Success)
Received 2048 (Success)
Received 4096 (Success)
Received 8192 (Success)
Received 16384 (Success)
Received 16384 (Success)
Received 16384 (Success)
Received 49152 (Success)
...
Received 65536 (Success)
Received 53562 (Success)
Received 0 (End of file)
Frequency of 'a' was 65630
Frequency of 'q' was 1492
Frequency of 'e' was 90579
Frequency of 'x' was 2139
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
0.00 0.000000 0 3 epoll_ctl
------ ----------- ----------- --------- --------- ----------------
100.00 0.000000 3 total
Summary
Measure. Use async primitives to do the scheduling for you. The only reason to use async_wait in principle is when you have to call third-party code using the native_handle of the socket in response.

Unable to find the reason for "Broken Pipe" error while sending continuous data chunks through Beast websocket

I am working on streaming audio recognition with IBM Watson speech to text web service API. I have created a web-socket with boost (beast 1.68.0) library in C++(std 11).
I have successfully connected to the IBM server, and want to send a 231,296 bytes of raw audio data to server in following manner.
{
"action": "start",
"content-type": "audio/l16;rate=44100"
}
websocket.binary(true);
<bytes of binary audio data 50,000 bytes>
<bytes of binary audio data 50,000 bytes>
<bytes of binary audio data 50,000 bytes>
<bytes of binary audio data 50,000 bytes>
<bytes of binary audio data 31,296 bytes>
websocket.binary(false);
{
"action": "stop"
}
Expected Result from IBMServer is :
{"results": [
{"alternatives": [
{ "confidence": xxxx,
"transcript": "call Rohan Chauhan "
}],"final": true
}], "result_index": 0
}
But I am not getting the desired result: rather the error says
"Broken pipe"
DataSize is: 50000 | mIsLast is : 0
DataSize is: 50000 | mIsLast is : 0
what : Broken pipe
DataSize is: 50000 | mIsLast is : 0
what : Operation canceled
DataSize is: 50000 | mIsLast is : 0
what : Operation canceled
DataSize is: 31296 | mIsLast is : 0
what : Operation canceled
Here is my code which is an adaptation of the sample example given in beast library.
Foo.hpp
class IbmWebsocketSession: public std::enable_shared_from_this<IbmWebsocketSession> {
protected:
char binarydata[50000];
std::string TextStart;
std::string TextStop;
public:
explicit IbmWebsocketSession(net::io_context& ioc, ssl::context& ctx, SttService* ibmWatsonobj) :
mResolver(ioc), mWebSocket(ioc, ctx) {
TextStart ="{\"action\":\"start\",\"content-type\": \"audio/l16;rate=44100\"}";
TextStop = "{\"action\":\"stop\"}";
/**********************************************************************
* Desc : Send start frame
**********************************************************************/
void send_start(beast::error_code ec);
/**********************************************************************
* Desc : Send Binary data
**********************************************************************/
void send_binary(beast::error_code ec);
/**********************************************************************
* Desc : Send Stop frame
**********************************************************************/
void send_stop(beast::error_code ec);
/**********************************************************************
* Desc : Read the file for binary data to be sent
**********************************************************************/
void readFile(char *bdata, unsigned int *Len, unsigned int *start_pos,bool *ReachedEOF);
}
Foo.cpp
void IbmWebsocketSession::on_ssl_handshake(beast::error_code ec) {
if(ec)
return fail(ec, "connect");
// Perform the websocket handshake
ws_.async_handshake_ex(host, "/speech-to-text/api/v1/recognize", [Token](request_type& reqHead) {reqHead.insert(http::field::authorization,Token);},bind(&IbmWebsocketSession::send_start, shared_from_this(),placeholders::_1));
}
void IbmWebsocketSession::send_start(beast::error_code ec){
if(ec)
return fail(ec, "ssl_handshake");
ws_.async_write(net::buffer(TextStart),
bind(&IbmWebsocketSession::send_binary, shared_from_this(),placeholders::_1));
}
void IbmWebsocketSession::send_binary(beast::error_code ec) {
if(ec)
return fail(ec, "send_start");
readFile(binarydata, &Datasize, &StartPos, &IsLast);
ws_.binary(true);
if (!IsLast) {
ws_.async_write(net::buffer(binarydata, Datasize),
bind(&IbmWebsocketSession::send_binary, shared_from_this(),
placeholders::_1));
} else {
IbmWebsocketSession::on_binarysent(ec);
}
}
void IbmWebsocketSession::on_binarysent(beast::error_code ec) {
if(ec)
return fail(ec, "send_binary");
ws_.binary(false);
ws_.async_write(net::buffer(TextStop),
bind(&IbmWebsocketSession::read_response, shared_from_this(), placeholders::_1));
}
void IbmWebsocketSession::readFile(char *bdata, unsigned int *Len, unsigned int *start_pos,bool *ReachedEOF) {
unsigned int end = 0;
unsigned int start = 0;
unsigned int length = 0;
// Creation of ifstream class object to read the file
ifstream infile(filepath, ifstream::binary);
if (infile) {
// Get the size of the file
infile.seekg(0, ios::end);
end = infile.tellg();
infile.seekg(*start_pos, ios::beg);
start = infile.tellg();
length = end - start;
}
if ((size_t) length < 150) {
*Len = (size_t) length;
*ReachedEOF = true;
// cout << "Reached end of File (last 150 bytes)" << endl;
} else if ((size_t) length <= 50000) { //Maximumbytes to send are 50000
*Len = (size_t) length;
*start_pos += (size_t) length;
*ReachedEOF = false;
infile.read(bdata, length);
} else {
*Len = 50000;
*start_pos += 50000;
*ReachedEOF = false;
infile.read(bdata, 50000);
}
infile.close();
}
Any suggestions here?
From boost's documentation we have the following excerpt on websocket::async_write
This function is used to asynchronously write a complete message. This
call always returns immediately. The asynchronous operation will
continue until one of the following conditions is true:
The complete message is written.
An error occurs.
So when you create your buffer object to pass to it net::buffer(TextStart) for example the lifetime of the buffer passed to it is only until the function returns. It could be that even after the function returns you the async write is still operating on the buffer as per the documentation but the contents are no longer valid since the buffer was a local variable.
To remedy this you could, make your TextStart static or declare that as a member of your class and copy it to boost::asio::buffer there are plenty of examples on how to do that. Note I only mention TextStart in the IbmWebsocketSession::send_start function. The problem is pretty much the same throughout your code.
From IBM Watson's API definition, the Initiate a connection requires a certain format which can then be represented as a string. You have the string but missing the proper format due to which the connection is being closed by the peer and you are writing to a closed socket, thus a broken pipe.
The initiate connection requires :
var message = {
action: 'start',
content-type: 'audio/l16;rate=22050'
};
Which can be represented as string TextStart = "action: 'start',\r\ncontent-type: 'audio\/l16;rate=44100'" according to your requirements.
Following on from the discussion in the chat, the OP resolved the issue by adding the code:
if (!IsLast ) {
ws_.async_write(net::buffer(binarydata, Datasize),
bind(&IbmWebsocketSession::send_binary, shared_from_this(),
placeholders::_1));
}
else {
if (mIbmWatsonobj->IsGstFileWriteDone()) { //checks for the file write completion
IbmWebsocketSession::on_binarysent(ec);
} else {
std::this_thread::sleep_for(std::chrono::seconds(1));
IbmWebsocketSession::send_binary(ec);
}
}
Which from discussion stems from the fact that more bytes were being sent to the client before a file write was completed on the same set of bytes. The OP now verifies this before attempting to send more bytes.

Linux poll on serial transmission end

I'm implementing RS485 on arm developement board using serial port and gpio for data enable.
I'm setting data enable to high before sending and I want it to be set low after transmission is complete.
It can be simply done by writing:
//fd = open("/dev/ttyO2", ...);
DataEnable.Set(true);
write(fd, data, datalen);
tcdrain(fd); //Wait until all data is sent
DataEnable.Set(false);
I wanted to change from blocking-mode to non-blocking and use poll with fd. But I dont see any poll event corresponding to 'transmission complete'.
How can I get notified when all data has been sent?
System: linux
Language: c++
Board: BeagleBone Black
I don't think it's possible. You'll either have to run tcdrain in another thread and have it notify the the main thread, or use timeout on poll and poll to see if the output has been drained.
You can use the TIOCOUTQ ioctl to get the number of bytes in the output buffer and tune the timeout according to baud rate. That should reduce the amount of polling you need to do to just once or twice. Something like:
enum { writing, draining, idle } write_state;
while(1) {
int write_event, timeout = -1;
...
if (write_state == writing) {
poll_fds[poll_len].fd = write_fd;
poll_fds[poll_len].event = POLLOUT;
write_event = poll_len++
} else if (write == draining) {
int outq;
ioctl(write_fd, TIOCOUTQ, &outq);
if (outq == 0) {
DataEnable.Set(false);
write_state = idle;
} else {
// 10 bits per byte, 1000 millisecond in a second
timeout = outq * 10 * 1000 / baud_rate;
if (timeout < 1) {
timeout = 1;
}
}
}
int r = poll(poll_fds, poll_len, timeout);
...
if (write_state == writing && r > 0 && (poll_fds[write_event].revent & POLLOUT)) {
DataEnable.Set(true); // Gets set even if already set.
int n = write(write_fd, write_data, write_datalen);
write_data += n;
write_datalen -= n;
if (write_datalen == 0) {
state = draining;
}
}
}
Stale thread, but I have been working on RS-485 with a 16550-compatible UART under Linux and find
tcdrain works - but it adds a delay of 10 to 20 msec. Seems to be polled
The value returned by TIOCOUTQ seems to count bytes in the OS buffer, but NOT bytes in the UART FIFO, so it may underestimate the delay required if transmission has already started.
I am currently using CLOCK_MONOTONIC to timestamp each send, calculating when the send should be complete, when checking that time against the next send, delaying if necessary. Sucks, but seems to work

boost::asio async server design

Currently I'm using design when server reads first 4 bytes of stream then read N bytes after header decoding.
But I found that time between first async_read and second read is 3-4 ms. I just printed in console timestamp from callbacks for measuring. I sent 10 bytes of data in total. Why it takes so much time to read?
I running it in debug mode but I think that 1 connection for debug is
not so much to have a 3 ms delay between reads from socket. Maybe I need
another approach to cut TCP stream on "packets"?
UPDATE: I post some code here
void parseHeader(const boost::system::error_code& error)
{
cout<<"[parseHeader] "<<lib::GET_SERVER_TIME()<<endl;
if (error) {
close();
return;
}
GenTCPmsg::header result = msg.parseHeader();
if (result.error == GenTCPmsg::parse_error::__NO_ERROR__) {
msg.setDataLength(result.size);
boost::asio::async_read(*socket,
boost::asio::buffer(msg.data(), result.size),
(*_strand).wrap(
boost::bind(&ConnectionInterface::parsePacket, shared_from_this(), boost::asio::placeholders::error)));
} else {
close();
}
}
void parsePacket(const boost::system::error_code& error)
{
cout<<"[parsePacket] "<<lib::GET_SERVER_TIME()<<endl;
if (error) {
close();
return;
}
protocol->parsePacket(msg);
msg.flush();
boost::asio::async_read(*socket,
boost::asio::buffer(msg.data(), config::HEADER_SIZE),
(*_strand).wrap(
boost::bind(&ConnectionInterface::parseHeader, shared_from_this(), boost::asio::placeholders::error)));
}
As you see unix timestamps differ in 3-4 ms. I want to understand why so many time elapse between parseHeader and parsePacket. This is not a client problem, summary data is 10 bytes, but i cant sent much much more, delay is exactly between calls. I'm using flash client version 11. What i do is just send ByteArray through opened socket. I don't sure that delays on client. I send all 10 bytes at once. How can i debug where actual delay is?
There are far too many unknowns to identify the root cause of the delay from the posted code. Nevertheless, there are a few approaches and considerations that can be taken to help to identify the problem:
Enable handler tracking for Boost.Asio 1.47+. Simply define BOOST_ASIO_ENABLE_HANDLER_TRACKING and Boost.Asio will write debug output, including timestamps, to the standard error stream. These timestamps can be used to help filter out delays introduced by application code (parseHeader(), parsePacket(), etc.).
Verify that byte-ordering is being handled properly. For example, if the protocol defines the header's size field as two bytes in network-byte-order and the server is handling the field as a raw short, then upon receiving a message that has a body size of 10:
A big-endian machine will call async_read reading 10 bytes. The read operation should complete quickly as the socket already has the 10 byte body available for reading.
A little-endian machine will call async_read reading 2560 bytes. The read operation will likely remain outstanding, as far more bytes are trying to be read than is intended.
Use tracing tools such as strace, ltrace, etc.
Modify Boost.Asio, adding timestamps throughout the callstack. Boost.Asio is shipped as a header-file only library. Thus, users may modify it to provide as much verbosity as desired. While not the cleanest or easiest of approaches, adding a print statement with timestamps throughout the callstack may help provide visibility into timing.
Try duplicating the behavior in a short, simple, self contained example. Start with the simplest of examples to determine if the delay is systamtic. Then, iteratively expand upon the example so that it becomes closer to the real-code with each iteration.
Here is a simple example from which I started:
#include <iostream>
#include <boost/array.hpp>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <boost/enable_shared_from_this.hpp>
#include <boost/make_shared.hpp>
#include <boost/shared_ptr.hpp>
class tcp_server
: public boost::enable_shared_from_this< tcp_server >
{
private:
enum
{
header_size = 4,
data_size = 10,
buffer_size = 1024,
max_stamp = 50
};
typedef boost::asio::ip::tcp tcp;
public:
typedef boost::array< boost::posix_time::ptime, max_stamp > time_stamps;
public:
tcp_server( boost::asio::io_service& service,
unsigned short port )
: strand_( service ),
acceptor_( service, tcp::endpoint( tcp::v4(), port ) ),
socket_( service ),
index_( 0 )
{}
/// #brief Returns collection of timestamps.
time_stamps& stamps()
{
return stamps_;
}
/// #brief Start the server.
void start()
{
acceptor_.async_accept(
socket_,
boost::bind( &tcp_server::handle_accept, this,
boost::asio::placeholders::error ) );
}
private:
/// #brief Accept connection.
void handle_accept( const boost::system::error_code& error )
{
if ( error )
{
std::cout << error.message() << std::endl;
return;
}
read_header();
}
/// #brief Read header.
void read_header()
{
boost::asio::async_read(
socket_,
boost::asio::buffer( buffer_, header_size ),
boost::bind( &tcp_server::handle_read_header, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred ) );
}
/// #brief Handle reading header.
void
handle_read_header( const boost::system::error_code& error,
std::size_t bytes_transferred )
{
if ( error )
{
std::cout << error.message() << std::endl;
return;
}
// If no more stamps can be recorded, then stop the async-chain so
// that io_service::run can return.
if ( !record_stamp() ) return;
// Read data.
boost::asio::async_read(
socket_,
boost::asio::buffer( buffer_, data_size ),
boost::bind( &tcp_server::handle_read_data, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred ) );
}
/// #brief Handle reading data.
void handle_read_data( const boost::system::error_code& error,
std::size_t bytes_transferred )
{
if ( error )
{
std::cout << error.message() << std::endl;
return;
}
// If no more stamps can be recorded, then stop the async-chain so
// that io_service::run can return.
if ( !record_stamp() ) return;
// Start reading header again.
read_header();
}
/// #brief Record time stamp.
bool record_stamp()
{
stamps_[ index_++ ] = boost::posix_time::microsec_clock::local_time();
return index_ < max_stamp;
}
private:
boost::asio::io_service::strand strand_;
tcp::acceptor acceptor_;
tcp::socket socket_;
boost::array< char, buffer_size > buffer_;
time_stamps stamps_;
unsigned int index_;
};
int main()
{
boost::asio::io_service service;
// Create and start the server.
boost::shared_ptr< tcp_server > server =
boost::make_shared< tcp_server >( boost::ref(service ), 33333 );
server->start();
// Run. This will exit once enough time stamps have been sampled.
service.run();
// Iterate through the stamps.
tcp_server::time_stamps& stamps = server->stamps();
typedef tcp_server::time_stamps::iterator stamp_iterator;
using boost::posix_time::time_duration;
for ( stamp_iterator iterator = stamps.begin() + 1,
end = stamps.end();
iterator != end;
++iterator )
{
// Obtain the delta between the current stamp and the previous.
time_duration delta = *iterator - *(iterator - 1);
std::cout << "Delta: " << delta.total_milliseconds() << " ms"
<< std::endl;
}
// Calculate the total delta.
time_duration delta = *stamps.rbegin() - *stamps.begin();
std::cout << "Total"
<< "\n Start: " << *stamps.begin()
<< "\n End: " << *stamps.rbegin()
<< "\n Delta: " << delta.total_milliseconds() << " ms"
<< std::endl;
}
A few notes about the implementation:
There is only one thread (main) and one asynchronous chain read_header->handle_read_header->handle_read_data. This should minimize the amount of time a ready-to-run handler spends waiting for an available thread.
To focus on boost::asio::async_read, noise is minimized by:
Using a pre-allocated buffer.
Not using shared_from_this() or strand::wrap.
Recording the timestamps, and perform processing post-collection.
I compiled on CentOS 5.4 using gcc 4.4.0 and Boost 1.50. To drive the data, I opted to send 1000 bytes using netcat:
$ ./a.out > output &
[1] 18623
$ echo "$(for i in {0..1000}; do echo -n "0"; done)" | nc 127.0.0.1 33333
[1]+ Done ./a.out >output
$ tail output
Delta: 0 ms
Delta: 0 ms
Delta: 0 ms
Delta: 0 ms
Delta: 0 ms
Delta: 0 ms
Total
Start: 2012-Sep-10 21:22:45.585780
End: 2012-Sep-10 21:22:45.586716
Delta: 0 ms
Observing no delay, I expanded upon the example by modifying the boost::asio::async_read calls, replacing this with shared_from_this() and wrapping the ReadHandlerss with strand_.wrap(). I ran the updated example and still observed no delay. Unfortunately, that is as far as I could get based on the code posted in the question.
Consider expanding upon the example, adding in a piece from the real implementation with each iteration. For example:
Start with using the msg variable's type to control the buffer.
Next, send valid data, and introduce parseHeader() and parsePacket functions.
Finally, introduce the lib::GET_SERVER_TIME() print.
If the example code is as close as possible to the real code, and no delay is being observed with boost::asio::async_read, then the ReadHandlers may be ready-to-run in the real code, but they are waiting on synchronization (the strand) or a resource (a thread), resulting in a delay:
If the delay is the result of synchronization with the strand, then consider Robin's suggestion by reading a larger block of data to potentially reduce the amount of reads required per-message.
If the delay is the result of waiting for a thread, then consider having an additional thread call io_service::run().
One thing that makes Boost.Asio awesome is using the async feature to the fullest. Relying on a specific number of bytes read in one batch, possibly ditching some of what could already been read, isn't really what you should be doing.
Instead, look at the example for the webserver especially this: http://www.boost.org/doc/libs/1_51_0/doc/html/boost_asio/example/http/server/connection.cpp
A boost triboolean is used to either a) complete the request if all data is available in one batch, b) ditch it if it's available but not valid and c) just read more when the io_service chooses to if the request was incomplete. The connection object is shared with the handler through a shared pointer.
Why is this superior to most other methods? You can possibly save the time between reads already parsing the request. This is sadly not followed through in the example but idealy you'd thread the handler so it can work on the data already available while the rest is added to the buffer. The only time it's blocking is when the data is incomplete.
Hope this helps, can't shed any light on why there is a 3ms delay between reads though.

Two-way C++ communication over serial connection

I am trying to write a really simple C++ application to communicate with an Arduino. I would like to send the Arduino a character that it sends back immediately. The Arduino code that I took from a tutorial looks like this:
void setup()
{
Serial.begin(9600);
}
void loop()
{
//Have the Arduino wait to receive input
while (Serial.available()==0);
//Read the input
char val = Serial.read();
//Echo
Serial.println(val);
}
I can communicate with the Arduino easily using GNU screen, so I know that everything is working fine with the basic communication:
$ screen /dev/tty.usbmodem641 9600
The (broken) C++ code that I have looks like this:
#include <fstream>
#include <iostream>
int main()
{
std::cout << "Opening fstream" << std::endl;
std::fstream file("/dev/tty.usbmodem641");
std::cout << "Sending integer" << std::endl;
file << 5 << std::endl; // endl does flush, which may be important
std::cout << "Data Sent" << std::endl;
std::cout << "Awaiting response" << std::endl;
std::string response;
file >> response;
std::cout << "Response: " << response << std::endl;
return 0;
}
It compiles fine, but when running it, some lights flash on the Arduino and the terminal just hangs at:
Opening fstream
Where am I going wrong?
There are three points:
First: You don't initialize the serial port (TTY) on the Linux side. Nobody knows in what state it is.
Doing this in your program you must use tcgetattr(3) and tcsetattr(3). You can find the required parameters by using these keywords at this site, the Arduino site or on Google. But just for quick testing I propose to issue this command before you call your own command:
stty -F /dev/tty.usbmodem641 sane raw pass8 -echo -hupcl clocal 9600
Especially the the missing clocal might prevent you opening the TTY.
Second: When the device is open, you should wait a little before sending anything. By default the Arduino resets when the serial line is opened or closed. You have to take this into account.
The -hupcl part will prevent this reset most of the time. But at least one reset is always necessary, because -hupcl can be set only when the TTY is already open and at that time the Arduino has received the reset signal already. So -hupcl will "only" prevent future resets.
Third: There is NO error handling in your code. Please add code after each IO operation on the TTY which checks for errors and - the most important part - prints helpful error messages using perror(3) or similar functions.
I found a nice example by Jeff Gray of how to make a simple minicom type client using boost::asio. The original code listing can be found on the boost user group. This allows connection and communication with the Arduino like in the GNU Screen example mentioned in the original post.
The code example (below) needs to be linked with the following linker flags
-lboost_system-mt -lboost_thread-mt
...but with a bit of tweaking, some of the dependence on boost can be replaced with new C++11 standard features. I'll post revised versions as and when I get around to it. For now, this compiles and is a solid basis.
/* minicom.cpp
A simple demonstration minicom client with Boost asio
Parameters:
baud rate
serial port (eg /dev/ttyS0 or COM1)
To end the application, send Ctrl-C on standard input
*/
#include <deque>
#include <iostream>
#include <boost/bind.hpp>
#include <boost/asio.hpp>
#include <boost/asio/serial_port.hpp>
#include <boost/thread.hpp>
#include <boost/lexical_cast.hpp>
#include <boost/date_time/posix_time/posix_time_types.hpp>
#ifdef POSIX
#include <termios.h>
#endif
using namespace std;
class minicom_client
{
public:
minicom_client(boost::asio::io_service& io_service, unsigned int baud, const string& device)
: active_(true),
io_service_(io_service),
serialPort(io_service, device)
{
if (!serialPort.is_open())
{
cerr << "Failed to open serial port\n";
return;
}
boost::asio::serial_port_base::baud_rate baud_option(baud);
serialPort.set_option(baud_option); // set the baud rate after the port has been opened
read_start();
}
void write(const char msg) // pass the write data to the do_write function via the io service in the other thread
{
io_service_.post(boost::bind(&minicom_client::do_write, this, msg));
}
void close() // call the do_close function via the io service in the other thread
{
io_service_.post(boost::bind(&minicom_client::do_close, this, boost::system::error_code()));
}
bool active() // return true if the socket is still active
{
return active_;
}
private:
static const int max_read_length = 512; // maximum amount of data to read in one operation
void read_start(void)
{ // Start an asynchronous read and call read_complete when it completes or fails
serialPort.async_read_some(boost::asio::buffer(read_msg_, max_read_length),
boost::bind(&minicom_client::read_complete,
this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
void read_complete(const boost::system::error_code& error, size_t bytes_transferred)
{ // the asynchronous read operation has now completed or failed and returned an error
if (!error)
{ // read completed, so process the data
cout.write(read_msg_, bytes_transferred); // echo to standard output
read_start(); // start waiting for another asynchronous read again
}
else
do_close(error);
}
void do_write(const char msg)
{ // callback to handle write call from outside this class
bool write_in_progress = !write_msgs_.empty(); // is there anything currently being written?
write_msgs_.push_back(msg); // store in write buffer
if (!write_in_progress) // if nothing is currently being written, then start
write_start();
}
void write_start(void)
{ // Start an asynchronous write and call write_complete when it completes or fails
boost::asio::async_write(serialPort,
boost::asio::buffer(&write_msgs_.front(), 1),
boost::bind(&minicom_client::write_complete,
this,
boost::asio::placeholders::error));
}
void write_complete(const boost::system::error_code& error)
{ // the asynchronous read operation has now completed or failed and returned an error
if (!error)
{ // write completed, so send next write data
write_msgs_.pop_front(); // remove the completed data
if (!write_msgs_.empty()) // if there is anthing left to be written
write_start(); // then start sending the next item in the buffer
}
else
do_close(error);
}
void do_close(const boost::system::error_code& error)
{ // something has gone wrong, so close the socket & make this object inactive
if (error == boost::asio::error::operation_aborted) // if this call is the result of a timer cancel()
return; // ignore it because the connection cancelled the timer
if (error)
cerr << "Error: " << error.message() << endl; // show the error message
else
cout << "Error: Connection did not succeed.\n";
cout << "Press Enter to exit\n";
serialPort.close();
active_ = false;
}
private:
bool active_; // remains true while this object is still operating
boost::asio::io_service& io_service_; // the main IO service that runs this connection
boost::asio::serial_port serialPort; // the serial port this instance is connected to
char read_msg_[max_read_length]; // data read from the socket
deque<char> write_msgs_; // buffered write data
};
int main(int argc, char* argv[])
{
// on Unix POSIX based systems, turn off line buffering of input, so cin.get() returns after every keypress
// On other systems, you'll need to look for an equivalent
#ifdef POSIX
termios stored_settings;
tcgetattr(0, &stored_settings);
termios new_settings = stored_settings;
new_settings.c_lflag &= (~ICANON);
new_settings.c_lflag &= (~ISIG); // don't automatically handle control-C
tcsetattr(0, TCSANOW, &new_settings);
#endif
try
{
if (argc != 3)
{
cerr << "Usage: minicom <baud> <device>\n";
return 1;
}
boost::asio::io_service io_service;
// define an instance of the main class of this program
minicom_client c(io_service, boost::lexical_cast<unsigned int>(argv[1]), argv[2]);
// run the IO service as a separate thread, so the main thread can block on standard input
boost::thread t(boost::bind(&boost::asio::io_service::run, &io_service));
while (c.active()) // check the internal state of the connection to make sure it's still running
{
char ch;
cin.get(ch); // blocking wait for standard input
if (ch == 3) // ctrl-C to end program
break;
c.write(ch);
}
c.close(); // close the minicom client connection
t.join(); // wait for the IO service thread to close
}
catch (exception& e)
{
cerr << "Exception: " << e.what() << "\n";
}
#ifdef POSIX // restore default buffering of standard input
tcsetattr(0, TCSANOW, &stored_settings);
#endif
return 0;
}
You should check if you have access to /dev/tty.usbmodem641. The usual way in Linux is to add the user to the proper group with adduser.
By the way, I know that to access the serial port, one needs to open /dev/ttyS0 (for COM1), until /dev/ttyS3. See for example this example in C.