multi socket architecture with asio - c++

I have a Client - Server architecture with 10 Servers with permanent connections with a single Client, the software is written in C++ and uses boost asio libraries.
All the connections are created in the initialization phase, and they are always open during the execution.
When the client needs some information, sends a request to all of the servers. Each server finds the information needed and answers to the client.
In the client there is a single thread that is in charge of receiving the messages from all of the sockets, in particular, I use only one io_services, and one async_read from each of the sockets.
When a message arrives in one of the sockets, the async_read read the first N bit that are the header of the message and than call a function that uses read (synchronous) to read the rest of the message. To the server side, the header and the rest of the message are sent with a single write (synchronous).
Then, the architecture works properly, but I noticed that sometimes the synchronous readtakes more time (~0.24 sec) than the usual.
In theory the data is ready to be read because the synchronous read is called when the async_read has already read the header. I also saw that if I use only one server instead of 10, this problem doesn't occur. Furthermore, I noticed that this problem is not caused because of the dimension of the message.
Is it possible that the problem occurs because the io_service is not able to handle all the 10 async_read? In particular, if all the sockets receive a message at the same time, could the io_service lost some time to manage the queues and slows down my synchronous read?
I haven't posted the code, because is difficult to estract it from the project, but if you don't understand my description I could write an example.
Thank you.

1) When async.read completion handler gets invoked, it doesn't mean that some data is available, it means that all the available to that moment data has already been read (unless you specified a restricting completion-condition). So the subsequent sync.read might wait until some more data arrives.
2) Blocking a completion handler is a bad idea, because you actually block all the other completion handlers and other functors posted to that io_service. Consider changing your design.

If you go for an asynchronous design, don't mix in some synchronous parts. Replace all your synchronous reads and writes with asynchronous ones. Both reads and writes will block your thread while the asynchronous variants will not.
Further, if you know the number of expected bytes exactly after reading the header you should request exactly that number of bytes.
If you don't know it, you could go for a single async_read_some with the size of the biggest message you expect. async_read_some will notify you how many bytes were actually read.

Related

What should I do if boost::beast write_some doesn't write everything?

I am sending data on a boost::beast::websocket
I would like to send the data synchronously, so am trying to decide if I should use write or write_some.
From this SO answer (which is about asio rather than beast specifically, but I assume(!) the same rules apply?) I understand that write will block until the entire message is confirmed sent, whereas write_some may return early, and will return the number of bytes sent which may not be all the bytes which were requested be sent.
In my particular use-case I am using a single thread, and the write is done from within this thread's context (ie: from inside a callback issued after entering io_context.run())
Since I don't want to block the caller for some indeterminate amount of time, I want to avoid using write if there is a more elegant solution.
So if I then turn to async_write, I am uncertain what I should do if the number of bytes is less than the number of bytes I requested be sent?
How I would normally handle this with standard tcp sockets is use non-blocking mode, and when I get back EWOULDBLOCK, enqueue the data and carry on. When the socket becomes writeable again, only then complete the write (much akin to an asio async_write). Since non-blocking is not supported in beast, I'm wondering what the analogous approach is?
Presumably I need to perform some additional write operation to ensure the rest of the bytes are sent in due course?
The beast docs say
Callers are responsible for synchronizing operations on the socket
using an implicit or explicit strand, as per the Asio documentation.
The websocket stream asynchronous interface supports one of each of
the following operations to be active at the same time:
async_read or async_read_some
async_write or async_write_some
async_ping or async_pong
async_close
Is it ok to start an async write of the remaining bytes, so long as I ensure that a new synchronous write/write_some isn't started before the outstanding async write has completed?
If I cannot start an async write to complete the send of the remaining bytes, how is one supposed to handle a synchronous write_some which doesn't completely send all bytes?
As to why I don't just use async_write always, I have additional slow processing to do after the attempt to write, such as logging etc. Since I am using a single thread, and the call to async_write happens within that thread, the write will only occur after I return control to the event loop.
So what I'd like to do is attempt to write synchronously (which will work in 90% of the cases) so the data is sent, and then perform my slow tasks which would otherwise delay the write. In the 10% of cases where a sync write doesn't complete immediately, then an alternative async_write operation should be employed - but only in the fallback situation.
Possibly related: I see that write_some has a flag fin, which should be set to true if this is the last part of the message.
I am only ever attempting to write complete messages, so should I always use true for this?

Any way to know how many bytes will be sent on TCP before sending?

I'm aware that the ::send within a Linux TCP server can limit the sending of the payload such that ::send needs to be called multiple times until the entire payload is sent.
i.e. Payload is 1024 bytes
sent_bytes = ::send(fd, ...) where sent_bytes is only 256 bytes so this needs to be called again.
Is there any way to know exactly how many bytes can be sent before sending? If the socket will allow for the entire message, or that the message will be fragmented and by how much?
Example Case
2 messages are sent to the same socket by different threads at the same time on the same tcp client via ::send(). In some cases where messages are large multiple calls to ::send() are required as not all the bytes are sent at the initial call. Thus, go with the loop solution until all the bytes are sent. The loop is mutexed so can be seen as thread safe, so each thread has to perform the sending after the other. But, my worry is that beacuse Tcp is a stream the client will receive fragments of each message and I was thinking that adding framing to each message I could rebuild the message on the client side, if I knew how many bytes are sent at a time.
Although the call to ::send() is done sequentially, is the any chance that the byte stream is still mixed?
Effectively, could this happen:
Server Side
Message 1: "CiaoCiao"
Message 2: "HelloThere"
Client Side
Received Message: "CiaoHelloCiaoThere"
Although the call to ::send() is done sequentially, is the any chance that
the byte stream is still mixed?
Of course. Not only there's a chance of that, it is pretty much going to be a certainty, at one point or another. It's going to happen at one point. Guaranteed.
sent to the same socket by different threads
It will be necessary to handle the synchronization at this level, by employing a mutex that each thread locks before sending its message and unlocking it only after the entire message is sent.
It goes without sending that this leaves open a possibility that a blocked/hung socket will result in a single thread locking this mutex for an excessive amount of time, until the socket times out and your execution thread ends up dealing with a failed send() or write(), in whatever fashion it is already doing now (you are, of course, checking the return value from send/write, and handling the exception conditions appropriately).
There is no single, cookie-cutter, paint-by-numbers, solution to this that works in every situation, in every program, that needs to do something like this. Each eventual solution needs to be tailored based on each program's unique requirements and purpose. Just one possibility would be a dedicated execution thread that handles all socket input/output, and all your other execution threads sending their messages to the socket thread, instead of writing to the socket directly. This would avoid having all execution thread wedged by a hung socket, at expense of grown memory, that's holding all unsent data.
But that's just one possible approach. The number of possible, alternative solutions has no limit. You will need to figure out which logic/algorithm based solution will work best for your specific program. There is no operating system/kernel level indication that will give you any kind of a guarantee as to the amount of a send() or write() call on a socket will accept.

Boost Beast WebSockets with several indeterminate writes per read

Using boost/beast websockets in C++
I've read up on the issues with beast websockets not supporting non-blocking reads, and the fact that there's no way to check if data is available, and that doing reads and writes in separate threads is probably not thread safe.
The issue I have, then, is figuring out the correct approach to this problem:
The IBM Watson speech-to-text WebSockets API allows you to send chunks of audio data as they become available (or in pieces from an existing file.) However, you do not get text replies for each chunk.
Instead, you keep sending it audio data until it recognizes a pause or an end of utterance, and then it finally sends back some results.
In other words, you may have to do several writes before a read will return anything, and there's no way to predict how many writes you will have to do.
Without a non-blocking read function, and without putting the blocking read in a separate thread, how do I keep sending data and then only retrieving results when they're available?
Don't confuse the lack of thread safety with a lack of full-duplex capability. You can call async_read and then follow it with a call to async_write. This will result in two "pending" asynchronous operations. The write operation will complete shortly afterwards, and the read operation will remain pending until a message is received.
Asio's asynchronous model is "reactive." That means that your completion handler gets called when something happens. You don't "check to see if data is available." Beast doesn't reinvent the wheel here, it adopts the asynchronous model of Asio. If you understand how to write asynchronous network programs with Asio, this knowledge will transfer over to Beast.

recv() correct use C++

I'm working on my own FTP client in C++, but I'm stuck at function recv(). When I get data with recv(), they can be incomplete, because I'm using TCP protocol, so I have to use recv in loop. Problem is that when I call recv after everything that should be received was received server blocks, and my program is stuck.
I don't know how many bytes im going to recieve so I can't control it and stop it when its done. I found two not very elegant solutions right now:
is to use string.substr() (or TR1 regex) to find needed
expression and then stop calling recv before it blocks
second is to
set up timeval structure and then control socket through
setsockopt() function. Problem is long response time when i can get
incomplete corrupted data.
Question is, is there any clean and elegant solution for this?
The obvious thing to do is to transmit the length of the to-be-received message ahead (many protocols, including for example HTTP do that, to address the exact same issue). That way, you know that when you have received amount X, no more will come.
This will work fine 99.9% of the time and will catastrophically fail in the 0.1% of cases where the server is lying to you or where the server crashes unexpectedly or someone stumbles over the network cable (or something similar happens). Sadly, the "connection" established by TCP is an illusion, and you don't have much of a means to detect when the connection dies. The other end can go down, and you will not notice anything, unless you try to send and get an error (or until several hours later).
Therefore, you also need a backup strategy for when things don't go quite as good as expected. You might either use select or poll to know when data is available, so you don't block forever for a message that will never come.
Using threads to solve the block-at-end problem (as proposed in other answers) is not a very good option since blocking isn't the actual problem. The actual problem is that you don't know when you have reached the end of the transmission. Having a worker thread block at the end of the transmission will "work", but will leave the worker thread blocked indefinitely, consuming resources and with an uncertain, system-dependent fate.
You cannot join the thread before exiting, since it is blocked (so trying to join it would deadlock your main thread). When your process exits and the socket is closed, the thread will unblock, but will (at least on some operating systems, e.g. Windows) be terminated immediately after. This likely won't do much evil, but terminating a thread in an uncontrolled way is always less desirable than having it exit properly. On other operating systems, you may have a lingering thread remaining.
Since you are using C++, there are alternative libraries that greatly simplify network programming compared to stock C. My personal favourite is Boost::Asio, however others are available. These libraries not only save you the pain of coding in C, but also provide asynchronous capabilities to work around your blocking problem.
The typical approach is to use select()/pselect() or poll()/ppoll(). Both allow to specify a timeout in order to exit if there are no incoming data.
However I don't see how you should "call recv after everything that should be received". It would be extremely inefficient to rely on the timeout also when there are not network problems...
Or you send the size of data being sent, before the data, and that's what you read, or the data connection is terminated with an EOF. In this case read() will return -1 and you exit.
I can think of two options that will not require a major rewrite of your existing code and a third one which is more radical:
use non-blocking I/O and poll for data periodically. You can do other work while a message remains incomplete or no further data can be read from the socket.
use a separate worker thread to do the I/O. Even if it blocks on synchronous recv() calls, your main thread can continue to do work. The worker thread can transfer the data it receives to the main thread for processing once a complete message is received via TCP.
use an OS specific feature (I/O completion ports on Windows or aio on Linux), but these are far more complex and you should definitely consider Boost.Asio before going this route.
You can put the recv function in it's own thread and do the processing in another thread.

What does boost::asio::write guarantee (ACK, Checksum)?

I have a questions concerning boost::asio::ip::tcp::socket and the associated write functions. From reading the Wikipedia article on the TCP, I understand, that TCP contains acknowledgement messages as well as checksums. Unfortunaly, I can't find any information on this in the boost::asio reference. As far as I understand boost::asio uses the OS implementation of the TCP, which should contain both features.
My question is what do the functions boost::asio::write or boost::asio::async_write guarantee when called with an boost::asio::ip::tcp::socket. So what does it mean if the function returns/the callback function is called without error. I can imagine some possibilities:
Basically nothing, It only means that the program told the OS to send the data, but nothing more.
Data is underway, meaning that the OS acknowledged that it has send the data.
Data has arrived, meaning that a acknowledgement message from the other side was received.
Data has arrived and is not corrupted, same as 3. plus that the checksum adds up.
If it is not 4. is there a way to enforce this using boost::asio (I mean within boost::asio, not implementing it yourself)?
It is #1, which is the way it should be. There are no guarantees that the data will ever be sent.
You think you want #4, but you really don't. The fact that the remote peer's network stack received the correct data is probably irrelevant to your application. You really want to know whether the data was received and processed correctly, which is beyond the scope of TCP, but easy enough to implement on top of TCP. (I recommend reading up on the OSI Model for an introduction to what TCP can be expected to do. Basically, you want to ensure that your data gets to the right application, or perhaps more, and TCP only ensures that it gets as far as the computer that the application is running on.)
To do what you want, send an in-band acknowledgement over the TCP link. You can also put the SHA-2 or some other hash of the data in the acknowledgement. You can also wait to send the acknowledgement until the data has been processed — e.g., wait until it has been written to disk and fsync() has been called.
Locally detected errors will be reported. Connection error will also be reported.
If you are using TCP, tcp-ack failure will be reported, but maybe at a later read or write call (when the os is notified of the tcp-hack failure).
So you can't be sure that when you issue a write that it is actually received. no write error means that the os knows currently no errors on the tcp connection you are using and that he buffered your data internally to transmit it to the tcp peer.