Using boost::asio::async_wait_until with boost::asio::streambuf - c++

I have an application that I am currently developing for communicating with a device using serial communication. For this I am using the boost library basic_serial_port. Right now, I am just attempting to read from the device and am using the async_wait_until function coupled with a async_wait from the deadline_timer class. The code that sets up the read and wait look like this:
async_read_until(port,readData,io_params.delim,
boost::bind(&SerialComm::readCompleted,
this,boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
timer.expires_from_now(boost::posix_time::seconds(1));
timer.async_wait(boost::bind(&SerialComm::timeoutExpired,this,
boost::asio::placeholders::error));
The callback on the async_read_until looks like
void SerialComm::readCompleted(const boost::system::error_code& error,
const size_t bytesTransferred){
if (!error){
wait_result = success;
bytes_transferred = bytesTransferred;
}
else {
if (error.value() != 125) wait_result = error_out;
else wait_result = op_canceled;
cout << "Port handler called with error code " + to_string(error.value()) << endl;
}
}
and the following code is triggered on successful read
string msg;
getline(istream(&readData), msg, '\r');
boost::trim_right_if(msg, boost::is_any_of("\r"));
In the case of this device, all messages are terminated with a carriage return, so specifying the carriage return in the async_read_until should retrieve a single message. However, what I am seeing is that, while the handler is triggered, new data is not necessarily entered into the buffer. So, what I might see is, if the handler is triggered 20x
one line pumped into the buffer in the first call
none in the next 6 calls
6 lines in the next call
no data in the next 10
10 lines following
...
I am obviously not doing something correctly, but what is it?

async_read_until does not guarantee it only read up until the first delimiter.
Due to the underlying implementation details, it will just "read what is available" on most systems and will return if the streambuf contains the delimiter. Additional data will be in the streambuf. Moreover, EOF might be returned even if you didn't expect it yet.
See for background Read until a string delimiter in boost::asio::streambuf

So, found the problem here. The way this program is intended to work is that it should
Send a request for data
Start an async_read_until to read data on the port.
Start an async_wait so that it we don't wait forever.
Use io_service::run_one to wait for a timeout or a successful read.
The code for step four looked like this:
for (;;){
// This blocks until an event on io_service_ is set.
n_handlers = io_service_.run_one();
// Brackets in success case limit scope of new variables
switch(wait_result){
case success:{
char c_[1024];
//string msg;
string delims = "\r";
std::string msg{buffers_begin(readData.data()), buffers_begin(readData.data()) + bytes_transferred- delims.size()};
// Consume through the first delimiter.
readData.consume(bytes_transferred);
data_out = msg;
cout << msg << endl;
data_handler(msg);
return data_out;
}
case timeout_expired:
//Set up for wait and read.
wait_result = in_progress;
cout << "Time is up..." << endl;
return data_out;
break;
case error_out:
cout << "Error out..." << endl;
return data_out;
break ;
case op_canceled:
return data_out;
break;
case in_progress:
cout << "In progress..." << endl;
break;
}
}
Only two cases should trigger an exit from the loop - timeout_expired and success. But, as you can see, the system will exit if an operation is cancelled (op_canceled) or if there is an error (error_out).
The problem is that when an async operation is cancelled (i.e. deadline_timer::cancel()) it will trigger an event picked up by io_service::run_one which will set the state evaluated by the switch statement to op_canceled. This can leave async operations stacking up in the event loop. The simple fix is to just comment out the return statement in all cases except for success and timeout_expired.

Related

What's the best way to copy a file in a way that I can easily cancel the copy while it is in progress?

I am using ReadFileEx to read some bytes from a file and using WriteFileEx to write some bytes to a device. This action will repeat till all file bytes are read and written to the device.
Reason I use Ex APIs is because by requesting overlapped IOs to the OS can keep UI thread responsive updating a progress bar while the read/write function is doing their tasks.
The process begins with a ReadFileEx and a MY_OVERLAPPED structure and a ReadCompletionRoutine will be passed in together. Once the read is done, the read completion routine will be called. Inside the routine, a WriteFileEx will be emitted and WriteCompletionRoutine will be called. Inside the write completion routine, another ReadFileEx will be emitted after the offset of the MY_OVERLAPPED structure is reset to next position. That is, two completions will call each other once a read or write is done.
Notice that the above process will only be executed if the calling thread is under alertable state. I use a while loop to keep the thread alertable by keep checking a global state variable is set to TRUE or not. The state variable, completed, will be set to TRUE inside ReadCompletionRoutine once all procedure is done.
FYI, MY_OVERLAPPED structure is a self-define structure that inherits OVERLAPPPED structure so that I can add 2 more information I need to it.
Now, my question is I would like to add a cancel function so that the user can cancel all the process that has been started. What I do is pretty simple. I set the completed variable to TRUE when a cancel button is pressed, so the while loop will break and alertable state will be stoped so the completion routines won't be executed. But, I don't know how to cancel the overlapped request that sent by the Read/WriteFileEx and their completion routines along with the MY_OVERLAPPED structure(see the //******* part in code). Now my code will crash once the cancel button is pressed. The cancel part is the one causing the crash. Please help, thank you.
//In MyClass.h========================================
struct MY_OVERLAPPED: OVERLAPPED {
MyClass *event;
unsigned long long count;
};
//In MyClass.cpp - main===============================
MY_OVERLAPPED overlap;
memset(&overlap, 0,sizeof(overlap));
//point to this class (MyClass), so all variables can later be accessed
overlap.event = this;
//set read position
overlap.Offset = 0;
overlap.OffsetHigh = 0;
overlap.count = 0;
//start the first read io request, read 524288 bytes, which 524288 bytes will be written in ReadCompletionRoutine
ReadFileEx(overlap.event->hSource, overlap.event->data, 524288, &overlap, ReadCompletionRoutine);
while(completed != true) {
updateProgress(overlap.count);
SleepEx(0,TRUE);
}
//********
CancelIo(overlap.event.hSource);
CancelIo(overlap.event.hDevice);
//********
//In MyClass.cpp - CALLBACKs===============================
void CALLBACK ReadCompletionRoutine(DWORD errorCode, DWORD bytestransfered, LPOVERLAPPED lpOverlapped)
{
//type cast to MY_OVERLAPPED
MY_OVERLAPPED *overlap = static_cast<MY_OVERLAPPED*>(lpOverlapped);
//write 524288 bytes and continue to read next 524288 bytes in WriteCompletionRoutine
WriteFileEx(overlap->event->hDevice, overlap->event->data, 524288, overlap, WriteCompletionRoutine);
}
void CALLBACK WriteCompletionRoutine(DWORD errorCode, DWORD bytestransfered, LPOVERLAPPED lpOverlapped)
{
MY_OVERLAPPED *overlap = static_cast<MY_OVERLAPPED*>(lpOverlapped);
if(overlap->count<fileSize/524288) {
//set new offset to 524288*i, i = overlap->count for next block reading
overlap->count = (overlap->count)+1;
LARGE_INTEGER location;
location.QuadPart = 524288*(overlap->count);
overlap->Offset = location.LowPart;
overlap->OffsetHigh = location.HighPart;
ReadFileEx(overlap->event->hSource, overlap->event->data, 524288, overlap, ReadCompletionRoutine);
}
else {
completed = TRUE;
}
}
Note that I prefer not to use multi-thread programming. Other than that, any better way of accomplishing the same goals is appreciated. Please and feel free to provide detail code and explanations. Thanks.
I actually would use a background thread for this, because modern C++ makes this very easy. Much easier, certainly, than what you are trying to do at the moment. So please try to shed any preconceptions you might have that this is the wrong approach for you and please try to read this post in the spirit in which it is intended. Thanks.
First up, here's some very simple proof of concept code which you can compile and run for yourself to try it out. At first sight, this might look a bit 'so what?', but bear with me, I'll explain at the end:
#define _CRT_SECURE_NO_WARNINGS
#include <iostream>
#include <thread>
#include <chrono>
#include <memory>
#include <atomic>
int usage ()
{
std::cout << "Usage: copy_file infile outfile\n";
return 255;
}
void copy_file (FILE *infile, FILE *outfile, std::atomic_bool *cancel)
{
constexpr int bufsize = 32768;
std::unique_ptr <char []> buf (new char [bufsize]);
std::cout << "Copying: ";
while (1)
{
if (*cancel)
{
std::cout << "\nCopy cancelled";
break;
}
size_t bytes_read = fread (buf.get (), 1, bufsize, infile);
if (bytes_read == 0)
{
// Check for error here, then break out of the loop
break;
}
size_t bytes_written = fwrite (buf.get (), 1, bytes_read, outfile);
// Again, check for error etc
std::cout << ".";
}
std::cout << "\nCopy complete\n";
// Now probably something like PostMessage here to alert your main loop hat the copy is complete
}
int main (int argc, char **argv)
{
if (argc < 3) return usage ();
FILE *infile = fopen (argv [1], "rb");
if (infile == NULL)
{
std::cout << "Cannot open input file " << argv [1] << "\n";
return 255;
}
FILE *outfile = fopen (argv [2], "wb");
if (outfile == NULL)
{
std::cout << "Cannot open output file " << argv [2] << "\n";
fclose (infile);
return 255;
}
std::atomic_bool cancel = false;
std::thread copy_thread = std::thread (copy_file, infile, outfile, &cancel);
std::this_thread::sleep_for (std::chrono::milliseconds (200));
cancel = true;
copy_thread.join (); // waits for thread to complete
fclose (infile);
fclose (outfile); // + error check!
std::cout << "Program exit\n";
}
And when I run this on my machine, I get something like:
background_copy_test bigfile outfile
Copying:
.....................................................................................
..............
Copy cancelled
Copy complete
Program exit
So, what's noteworthy about this? Well, in no particular order:
It's dead simple.
It's standard C++. There are no Windows calls in there at all (and I did that deliberately, to try to make a point).
It's foolproof.
It 'just works'.
Now of course, you're not going to put your main thread to sleep while you're copying the file in real life. No no no. Instead, you're going to just kick the copy off via std::thread and then put up your 'Copying...' dialog with a Cancel button in it (presumably, this would be a modal dialog)
Then:
If that button is pressed, just set cancel to true and the magic will then happen.
Have copy_file send your 'copying' dialog a WM_APP+nnn message when it is done. It can also do that to have the dialog update its progress bar (I'm leaving all that stuff to you).
Don't omit that call to join() before you destroy copy_thread or it goes out of scope!
What else? Well, to get your head around this properly, study a bit of modern C++. cppreference is a useful site, but you should really read a good book. Then you should be able to apply the lessons learned here to your particular use-case.
Edit: It occurs to me to say that you might do better to create your thread in the WM_INITDIALOG handler for your 'Copying' dialog. Then you can pass the dialog's HWND to copy_file so that it knows where to send those messages to. Just a thought.
And you have a fair bit of reading to do if you're going to profit from this post. But then again, you should. And this post is going to achieve precisely nothing, I fear. Shame.

How to use asio buffer after async_read_until for consecutive reads

I am reading from a serial device where each message must be specifically requested. E.g. you send a request and get a response with the serialised payload.
Each message contains these parts in order:
PREAMBLE (2 bytes, "$M")
HEADER (3 bytes, containing payload length N)
PAYLOAD+CRC (N+1 bytes)
My approach with asio is to detect the start (PREAMBLE) of a message by using asio::async_read_until and afterwards using asio::async_read for reading the exact amount of bytes for HEADER and PAYLOAD+CRC. Since there is no static pattern at the end of the message, I cannot use async_read_until to read the full message.
After receiving PREAMBLE, the handler for async_read_until gets called and the buffer contains the PREAMBLE bytes and might contain additional bytes from HEADER and PAYLOAD+CRC.
The asio documentation for async_read_until says:
After a successful async_read_until operation, the streambuf may
contain additional data beyond the delimiter. An application will
typically leave that data in the streambuf for a subsequent
async_read_until operation to examine.
I interpret this as that you should only consume the requested bytes and leave all remaining bytes in the buffer for further reads. However, all consecutive reads block since the data is already in the buffer and there is nothing left on the device.
The reading is implemented as a small state machine processState, where different handlers are registered depending on which part of the message is to be read. All reading is done with the same buffer (asio::streambuf). processState is called in an infinite loop.
void processState() {
// register handler for incomming messages
std::cout << "state: " << parser_state << std::endl;
switch (parser_state) {
case READ_PREAMBLE:
asio::async_read_until(port, buffer, "$M",
std::bind(&Client::onPreamble, this, std::placeholders::_1, std::placeholders::_2));
break;
case READ_HEADER:
asio::async_read(port, buffer, asio::transfer_exactly(3),
std::bind(&Client::onHeader, this, std::placeholders::_1, std::placeholders::_2));
break;
case READ_PAYLOAD_CRC:
asio::async_read(port, buffer, asio::transfer_exactly(request_received->length+1),
std::bind(&Client::onDataCRC, this, std::placeholders::_1, std::placeholders::_2));
break;
case PROCESS_PAYLOAD:
onProcessMessage();
break;
case END:
parser_state = READ_PREAMBLE;
break;
}
// wait for incoming data
io.run();
io.reset();
}
The PREAMBLE handler onPreamble is called when receiving the PREAMBLE:
void onPreamble(const asio::error_code& error, const std::size_t bytes_transferred) {
std::cout << "onPreamble START" << std::endl;
if(error) { return; }
std::cout << "buffer: " << buffer.in_avail() << "/" << buffer.size() << std::endl;
// ignore and remove header bytes
buffer.consume(bytes_transferred);
std::cout << "buffer: " << buffer.in_avail() << "/" << buffer.size() << std::endl;
buffer.commit(buffer.size());
std::cout << "onPreamble END" << std::endl;
parser_state = READ_HEADER;
}
After this handler, no other handlers get called since the data is in the buffer and no data is left on the device.
What is the correct way to use asio::streambuf such that the handlers of consecutive async_read get called and I can process bytes in order of the state machine? I don't want to process the remaining bytes in onPreamble since it is not guaranteed that these will contain the full message.
You don't need the call to buffer.commit() in the onPreamble() handler. Calling buffer.consume() will remove the header bytes as you expect and leave the remaining bytes (if any were received) in the asio::streambuf for the next read. The streambuf's prepare() and commit() calls are used when you are filling it with data to send to the remote party.
I just finished a blog post and codecast about using the asio::streambuf to perform a simple HTTP GET with a few web servers. It might give you a better idea of how to use async_read_until() and async_read().

Close the stdin of boost::process child

I'm trying to call a process with a string to its stdin, with Boost-1.64.0.
The current code is :
bp::opstream inStream ;
bp::ipstream outStream;
bp::ipstream errStream;
bp::child child(
command, // the command line
bp::shell,
bp::std_out > outStream,
bp::std_err > errStream,
bp::std_in < inStream);
// read the outStream/errStream in threads
child.wait();
The problem is that the child executable is waiting for its stdin EOF. Here child.wait() is hanging indefinitely…
I tried to used asio::buffer, std_in.close(),… But no luck.
The only hack I found was to delete() the inStream… And that's not really reliable.
How am I supposed to "notify" the child process and close its stdin with the new boost::process library ?
Thanks !
I tried to used asio::buffer, std_in.close()
This works. Of course it only works if you pass it to a launch function (bp::child constructor, bp::system, etc).
If you need to pass data, and then close it, simply close the associated filedescriptor. I do something like this:
boost::asio::async_write(input, bp::buffer(_stdin_data), [&input](auto ec, auto bytes_written){
if (ec) {
logger.log(LOG_WARNING) << "Standard input rejected: " << ec.message() << " after " << bytes_written << " bytes written";
}
may_fail([&] { input.close(); });
});
Where input is
bp::async_pipe input(ios);
Also, check that the process is not actually stuck sending the output! If you fail to consume the output it would be buffering and waiting if the buffer is full.
Closing the pipe by calling inStream.close(); when you're done writing to it. You can also close it while launching with bp::std_in.close().
The asio solution of course also works and avoids the danger of deadlocks.

Clearing a read() buffer while using a socket

Recently I've been messing around with some sockets by trying to make a client/server program. So far I have been successful, but it seems I hit a roadblock. For some quick background information, I made a server that can accept a connection, and once everything is set up and a connection to a client is made, this block of code begins to exectue:
while(1){
read(newsockfd, &inbuffer, 256);
std::cout << "Message from client " << inet_ntoa(cli_addr.sin_addr) << " : ";
for(int i = 0; i < sizeof(inbuffer); i++){
std::cout << inbuffer[i];
}
std::cout << std::endl;
}
Now the client simply, when executed, connects to the server and writes to the socket, and then exits. So since one message was sent, this loop should only run once, and then wait for another message if what I read was correct.
But what ends up happenning is that this loop continues over and over, printing the same message over and over. From what I read (on this site and others) about the read() function is that after it is called once, it waits for another message to be recieved. I may be making a stupid mistake here, but is there any way I can have this read() function wait for a new message, instead of using the same old message over and over? Or is there another function that could replace read() to do what I want it to?
Thanks for any help.
You don't check the return value of read. So if the other end closes the connection or there's an error, you'll just loop forever outputting whatever happened to be in the buffer. You probably want:
while(1){
int msglen = read(newsockfd, &inbuffer, 256);
if (msglen <= 0) break;
std::cout << "Data from client " << inet_ntoa(cli_addr.sin_addr) << " : ";
for(int i = 0; i < msglen; i++){
std::cout << inbuffer[i];
}
std::cout << std::endl;
}
Notice that I changed the word "message" to "data". Here's why:
So since one message was sent, this loop should only run once, and then wait for another message if what I read was correct.
This is incorrect. The code above does not have any concept of a "message", and TCP does not preserve application message boundaries. So not only is this wrong, there's no way it could be correct because the word "message" has no meaning that could possibly apply in this context. TCP does not "glue together" the bytes that happend to be passed in a single call to a sending function.

How to prevent input text breaking while other thread is outputting to the console?

I have 2 threads: one of them is constantly cout'ing to the console some value, let's say increments an int value every second - so every second on the console is 1,2,3... and so on.
Another thread is waiting for user input - with the command cin.
Here is my problem: when I start typing something, when the time comes to cout the int value, my input gets erased from the input field, and put into the console with the int value. So when I want to type in "hello" it looks something like this:
1
2
3
he4
l5
lo6
7
8
Is there a way to prevent my input from getting put to the console, while other thread is writing to the console?
FYI this is needed for a chat app at client side - one thread is listening for messages and outputs this message as soon as it comes in, and the other thread is listening for user input to be sent to a server app.
Usually the terminal itself echos the keys typed. You can turn this off and get your program to echo it. This question will give you pointers on how to do it Hide password input on terminal
You can then just get the one thread to handle output.
If you are a slow typer, then the solution to your problem can be, as I said, making it a single thread, but that may make the app to receive only after it sends.
Another way is to increase your receiving thread's sleep time, which would provide you some more time to type (without interruption)
You could make a GUI (or use ncurses if you really want to work in the console). This way you avoid having std::cout shared by the threads.
I think you could solve this problem with a semaphore. When you have an incoming message you check to see if the user is writing something. If he does you wait until he finishes to print the message.
Is there a way to prevent my input from getting put to the console, while other thread is writing to the console?
It is the other way around. The other thread shouldn't interrupt the display of what you are typing.
Say you have typed "Hel" and then a new message comes in from the other thread. What do you do? How should it be displayed?
Totally disable echoing of what you type and only display it after you hit enter. In this way you can display messages from the different threads properly, in an atomic fashion. The big drawback is that you cannot see what you have typed already... :(
You immediately echo what you type. When the new message comes in, you undo the "Hel", print the new message and print again "Hel" on a new line and you can continue typing. Doable but a bit ugly.
You echo what you type in a separate place. That is, you split somehow the display. In one place you display the posted/received messages in order; and in another place you display what you are typing. You either need a GUI or at least some console library to do this. This would be the nicest solution but perhaps the most difficult to port to another OS due to the library dependencies.
In any case, you need a (preferably internally) synchronized stream that you can safely call from different threads and can write strings into it atomically. That is, you need to write your own synchronized stream class.
Hope this helps.
Well i recently solved this same issue with a basic workaround. This might not be the #1 solution but worked like a charm for me, as a newbie;
#include <iostream> // I/O
#include <Windows.h> // Sleep();
#include <conio.h> // _getch();
#include <string> // MessageBuffer
#include <thread> // Thread
using namespace std;
void ThreadedOutput();
string MessageBuffer; // or make it static
void main()
{
thread output(ThreadedOutput); // Attach the output thread
int count = 0;
char cur = 'a'; // Temporary at start
while (cur != '\r')
{
cur = _getch(); // Take 1 input
if (cur >= 32 && cur <= 126) // Check if input lies in alphanumeric and special keys
{
MessageBuffer += cur; // Store input in buffer
cout << cur; // Output the value user entered
count++;
}
else if (cur == 8) // If input key was backspace
{
cout << "\b \b"; // Move cursor 1 step back, overwrite previous character with space, move cursor 1 step back
MessageBuffer = MessageBuffer.substr(0, MessageBuffer.size() - 1); // Remove last character from buffer
count--;
}
else if (cur == 13) // If input was 'return' key
{
for (int i = 0; i < (signed)MessageBuffer.length(); i++) // Remove the written input
cout << "\b \b";
// "MessageBuffer" has your input, use it somewhere
MessageBuffer = ""; // Clear the buffer
}
}
output.join(); // Join the thread
}
void ThreadedOutput()
{
int i = 0;
while (true)
{
for (int i = 0; i < (signed)MessageBuffer.length(); i++) // Remove the written input
cout << "\b \b";
cout << ++i << endl; // Give parallel output with input
cout << MessageBuffer; // Rewrite the stored buffer
Sleep(1000); // Prevent this example spam
}
}