I'am currently exploring the Asio library and have working code for regular TCP connections. I used asio::ip::tcp::iostream objects since stuff I want to transmit already can serialize to/deserialize from iostreams, so this was really handy and worked well for me.
I then tried to switch to SSL connections and that's when everything turned crazy. There is apparently no built-in support to get the same iostream interface that all other protocols support for a secured connection. From a design perspective this is really perplexing to me. Is there any reason why this is the case?
I am aware of the discussion in How to create a boost ssl iostream? which concludes with a wrapper class to provide iostream functionality using boost. Apart from that, according to a comment, the implementation is flawed, this also does not give the same interface as for the other protocols (a basic_socket_iostream) which also allows to e.g., set expiration times and close the connection. (I am also using asio in the non-boost version and want to avoid adding boost as an additional dependency if possible).
So, I guess my questions are:
What exactly would I need to implement to get a basic_socket_iostream for an SSL connection? I assume it would be a derivation of asio::basic_streambuf or asio::basic_socket_streambuf but I somehow can't figure out how they work and need to be tweaked.. there's just a bunch of weird pointer movement and buffer allocations and documentation to me is quite unclear on what happens when exactly to achieve what...
Why is this not already present in the first place? It seems very unreasonable to have this one protocol behave entirely different from any other and thus require major refactoring for changing a tcp::iostream based project to support secured connections
> Well, the problem I have is that the ssl::stream really does neither: I doesn't give a socket but it also doesn't give me a stream interface that would be compatible to those available from the other protocols and, yes, in that sense it behaves very differently from the others (for no apparent reason)
I don't think the stream behaves any differently from the other protocols (see
https://www.boost.org/doc/libs/1_66_0/doc/html/boost_asio/overview/core/streams.html):
Streams, Short Reads and Short Writes
Many I/O objects in Boost.Asio are stream-oriented. This means that:
There are no message boundaries. The data being transferred is a continuous sequence of bytes.
Read or write operations may transfer fewer bytes than requested. This is referred to as a short read or short write.
Objects that provide stream-oriented I/O model one or more of the following type requirements:
SyncReadStream, where synchronous read operations are performed using a member function called read_some().
AsyncReadStream, where asynchronous read operations are performed using a member function called async_read_some().
SyncWriteStream, where synchronous write operations are performed using a member function called write_some().
AsyncWriteStream, where synchronous write operations are performed using a member function called async_write_some().
Examples of stream-oriented I/O objects include ip::tcp::socket, ssl::stream<>, posix::stream_descriptor, windows::stream_handle, etc.
Perhaps the confusion is that you're comparing to the iostream interface, which is simply not the same concept (it comes from the standard library).
To the question how you could make a iostream compatible stream wrapper for the ssl stream, I cannot devise an answer without consulting the documentations more and using a compiler, which I don't have on hand at the moment.
I think there is room for improvement in the library here. If you read the ip::tcp::iostream class (i.e. basic_socket_iostream<ip::tcp>), you'll see that it has two base classes:
private detail::socket_iostream_base<ip::tcp>
public std::basic_iostream<char>
The former contains a basic_socket_streambuf<ip::tcp> (a derived class of std::streambuf and basic_socket<ip::tcp>), whose address is passed to the latter at construction-time.
For the most part, basic_socket_streambuf<ip::tcp> performs the actual socket operations via its basic_socket<ip::tcp> base class. However, there is the connect_to_endpoints() member function that jumps the abstraction and calls several low-level functions from the detail::socket_ops namespace directly on socket().native_handle(). (This seems to have been introduced in Git commit b60e92b13e.) Those functions will only work on TCP sockets, even though the class is a template for any protocol.
Until I discovered this issue, my plan to integrate SSL support as a iostream/streambuf was to provide a ssl protocol class and a basic_socket<ssl> template specialization to wrap the existing ssl::context and ssl::stream<ip::tcp::socket> classes. Something like this (won't compile):
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/basic_socket.hpp>
#include <boost/asio/ssl.hpp>
namespace boost {
namespace asio {
namespace ip {
class ssl
: public tcp // for reuse (I'm lazy!)
{
public:
typedef basic_socket_iostream<ssl> iostream;
// more things as needed ...
};
} // namespace ip
template <>
class basic_socket<ip::ssl>
{
class SslContext
{
ssl::context ctx;
public:
SslContext() : ctx(ssl::context::sslv23_client)
{
ctx.set_options(ssl::context::default_workarounds);
ctx.set_default_verify_paths();
}
ssl::context & context() { return ctx; }
} sslContext;
ssl::stream<ip::tcp::socket> sslSocket;
public:
explicit basic_socket(const executor & ex)
: sslSocket(ex, sslContext.context())
{}
executor get_executor() noexcept
{
return sslSocket.lowest_layer().get_executor();
}
void connect(const ip::tcp::endpoint & endpoint_)
{
sslSocket.next_layer().connect(endpoint_);
sslSocket.lowest_layer().set_option(ip::tcp::no_delay(true));
sslSocket.set_verify_mode(ssl::verify_peer);
sslSocket.set_verify_callback(
ssl::rfc2818_verification("TODO: pass the domain here through the stream/streambuf somehow"));
sslSocket.handshake(ssl::stream<ip::tcp::socket>::client);
}
void close()
{
sslSocket.shutdown();
sslSocket.next_layer().close();
}
};
} // namespace asio
} // namespace boost
But due to the design issue I'll have to specialize basic_socket_streambuf<ip::ssl> as well, to avoid the detail::socket_ops routines. (I should also avoid injecting the ssl protocol class into the boost::asio::ip namespace, but that's a side concern.)
Haven't spent much time on this, but it seems doable. Fixing basic_socket_streambuf<>::connect_to_endpoints() first should help greatly.
Related
The documentation doesn't seem to tell much: lowest_layer(), next_layer().
What is the difference between them and when to use each?
To answer this, first thing to remember is that boost::asio::ssl::stream is a template class. Usually it look like boost::asio::ssl::stream<boost::asio::ip::tcp::socket>. Thus is implemented using boost::asio::ip::tcp::socket. That will be next_layer for boost::asio::ssl::stream. On other side, lowest_layer is always will be a basic_socket (its described in the docs).
Its little ambiguous especially when you see in the headers tcp::socket is typedef to basic_stream_socket<Tcp>, which is directly inherited from basic_socket. And.. In OOP terms you can say "next_layer IS the lowest_layer"..
But lets take another case where you create a ssl::stream< MyOwnClass >. In this case next_layer is MyOwnClass, which should control data reads/writes. And lowest_layer will be be whatever MyOwnClass will say in its typedef.
UPD: When to use each. Use next_layer for read/writes (you don't need this for SSL connection, but before starttls session its required). And use lowest_layer to control underlying socket.
I've seen some people creating a "is-a" relationship like the following:
class TCPClient : public Socket
{
public:
TCPClient(const std::string& host, unsigned short port);
};
where the Socket class implements Winsock functions such as Connect(), Close(), Bind() etc.
Examples:
Example 1
Example 2
But this doesn't feel natural to me who is a newbie in socket programming.
Does the above hierarchy make more logical sense than the following "has-a" counterpart?
class TCPClient
{
public:
TCPClient(const std::string& host, unsigned short port);
....
private:
Socket m_socket;
};
A TCPClient uses a socket or has a socket, but is not itself a socket, and you wouldn't normally expect to be able to substitute a TCPClient anywhere a socket was expected. As such, public inheritance doesn't make sense.
You could use private inheritance for this case, but (at least in a typical case) it probably doesn't make much sense either. Private inheritance makes sense primarily when the base class provides at least one virtual function you plan to override in the child class. If you have a virtual function and need to override it, you have no real choice but to use inheritance. I wouldn't expect a Socket class to have an virtual functions though; that wouldn't normally apply here.
That basically leads to your second solution: the TCPClient should contain an instance of a Socket, rather than using inheritance at all.
I should add, however, that the Socket class you've shown seems to conflate the notion of an actual socket with the notion of an address. My first socket class (years ago) worked about like that, but since then I've concluded that it's not really an ideal design. I've become convinced that it's worthwhile to keep the notion of an address separate from the socket itself. Though mine is a bit less elaborate, I find it interesting that what I came up with looks almost like it could have been the prototype from which Boost ASIO was derived. It's a little smaller and simpler, but a lot of the basic ideas are generally pretty similar anyway.
That leads to my next recommendation: take a look at Boost ASIO. Lacking a fairly specific reason to do otherwise, it's what I'd advise (and generally use) in most new code. Although (as I said above) I've written several socket classes over the years, I haven't used any of them in much (any?) new code in quite a while now -- they really only have two possible advantages over ASIO. The first applies only to me: since I wrote and used them before ASIO existed, I already understand them and how they work. The second may be similar: at least to me, they seem a little bit smaller and simpler (but, again, that may be just because I used them first). Even so, the advantages of (for example) using something other people already understand trumps those quite easily.
Use has-a. A TCPClient uses a socket like a person uses a telephone. Would you derive a Person from a Telephone?
class TCPClient : public Socket
{
public:
TCPClient(const std::string& host, unsigned short port);
};
Network sockets are used not only in TCP/IP and the above design is more suitable if you plan to reuse your "Socket" class to implement other protocols using network sockets. For example:
class UDPClient : public Socket
{
};
I would say so. Socket is an abstraction, a file descriptor (UNIX) or handle (Windows), which has resources associated with it and is managed by the operating system. If we consider OSI model, the socket fits well into the presentation layer (it presents, or describes, a communication channel between two nodes), whereas a program that uses the socket sits on the application layer.
Considering this, I would prefer not to inherit from the socket, unless I implement a kind of advanced socket (by analogy: C-pointer vs. smart pointer) to present and handle a logical connection and somehow manage the resources associated with the socket. If XYZClient is an application, whose goal is to implement some business or data processing logic, I would not mix these two concepts together and use the second approach (has-a).
I would split infrastructure/resource-specific and business/application-specific logic.
I'm trying to code a wrapper over a boost::asio::ip::tcp::socket
Something like that :
class Socket {
public:
void async_read(AsyncReadStream & s,
const boost::asio::MutableBufferSequence & buffers,
CompletionCondition completion_condition,
ReadHandler handler) {};
};
So I would be able to use ssl and non-ssl stream seamlessly...
The only thing is that, I do not seems to find the definition of each parameters to pass them to boost::asio::async_read (namespaces, etc...)
Any help would be appreciated ! Thanks
Your main requirements seems to be "use SSL and non-SSL streams seamlessly." To do that, you can wrap a the various stream types in a way that exposes the functions you need to use.
Part of how you do that is deciding how you're going to do memory management. MutableBufferSequence is not a type, it defines a set of requirements for a type to be used on that context.
If you are going to use one of a smallish number of approaches you can just use them in the interface (as long as it meets the MutableBufferSequence/ConstBufferSequence requirements, appropriate). The downside of this is that buffer management becomes part of the interface.
If you want to maintain the asio buffer management flexibility then you could
Template your code on stream type in order to achieve the seamless SSL/non-SSL requirement.
Create a wrapper for the various stream types with templated methods on buffer type.
(Updated response; I shouldn't try to respond to a question like this when I have less than two minutes!)
I am in the process of starting to implement some proprietary communication protocol stack in software but not sure where to start. It is the kind of work I have not done before and I am looking for help in terms of resources for best/recommended approaches.
I will be using c/c++ and I am free to use use libraries (BSD/BOOST/Apache) but no GPL. I have used C++ extensively so using the features of C++ is not a problem.
The protocol stack has three layers and it is already fully specified and formally verified. So all I need to do is implemented and test it fully in the specified languages. Should also mention that protocol is very simple but can run on different devices over a reliable physical transport layer. I know the events, inputs, outputs, side effects and the behaviour of the protocol state machine(s). Generally, an interrupt is received to read the message received from the physical layer to read it and send to the waiting device. The receiving device can process and pass the response message to the protocol layer to send out on the physical layer.
Any help with references/recommendations will be appreciated. I am willing to use a different language if only to help me understand how to implement them but I will have to eventually resort to the language of choice.
Update: An example protocol I wish to implement is something like SNEP.
I do not need to worry about connection management. We can assume the connection is already establish and I the protocol does is data exchange where the protocol messages are already well defined in specifications
Start with interfaces and messages.
Declare the interfaces of session that allow peers to exchange messages. Declare the messages as C++ structs with simple types, like ints, doubles, std::string's and and std::vectors. For example:
// these are your protocol messages
struct HelloRequest {
uint32_t seq_no;
// more stuff
};
struct HelloResponse {
uint32_t seq_no;
// more stuff
};
// Session callback for received messages
struct SessionReceiver {
virtual void connected(Session*) = 0;
virtual void receive(Session* from, HelloRequest msg) = 0;
virtual void receive(Session* from, HelloResponse msg) = 0;
virtual void disconnected(Session*) = 0;
};
// Session interface to send messages
struct Session {
virtual void send(HelloRequest msg) = 0;
virtual void send(HelloResponse msg) = 0;
};
// this connects asynchronously and then calls SessionReceiver::connected() with a newly established session
struct SessionInitiator {
virtual void connect(SessionReceiver* cb, std::string peer) = 0;
};
// this accepts connections asynchronously and then calls SessionReceiver::connected() with a newly accepted session
struct SessionAcceptor {
virtual void listen(SessionReceiver* cb, std::string port) = 0;
};
Then test your interfaces by coding the business logic that uses these interfaces. Once you are confident that the interfaces allow you to implement the required logic implement the interfaces and serialization of your messages using your preferred event-driven framework, like libevent or Boost.Asio.
Edit:
Note that interfaces allow you to have mock or test implementations. Also the fact that serialization happens behind the interface means that for in-process peers you don't have to serialize and deserialize the messages, you can pass them as is.
Boost.ASIO is pretty cutting edge when it comes to Asynchronous (or synchronous) network communication in C++
Have a look at Google Protocol Buffers.
From the description:
Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format.
Protocol Buffers are language and platform neutral so should fit into your project. I couldn't find the license, but at least it doesn't say "GPL" anywhere that I could find.
This will help you with the protocols. With the actual data transmission, well, unless you are writing the OS yourself, there should be some primitives you should use. It's hard to give more exact help on implementation unless you provide a bit more detail. For instance, what communication channel are you using? Ethernet?
But as a rule of thumb, you should make the ISR as short as possible. In these kinds of solutions that usually means copying data to a ring buffer. This way you don't have to allocate memory in the ISR. ISR, after having copied the data, should inform upper layers of the package. If you can use DMA, use that. In that case it might be possible to send the notification before you even start the DMA transfer.
You might also want to check out Linux Device Drivers, chapter 10 in particular. Check out the part about Bottom and Top Halves.
This is a generic C++ design question.
I'm writing an application that uses a client/server model. Right now I'm writing the server-side. Many clients already exist (some written by myself, others by third parties). The problem is that these existing clients all use different protocol versions (there have been 2-3 protocol changes over the years).
Since I'm re-writing the server, I thought it would be a great time to design my code such that I can handle many different protocol versions transparently. In all protocol versions, the very first communication from the client contains the protocol version, so for every client connection, the server knows exactly which protocol it needs to talk.
The naive method to do this is to litter the code with statements like this:
if (clientProtocolVersion == 1)
// do something here
else if (clientProtocolVersion == 2)
// do something else here
else if (clientProtocolVersion == 3)
// do a third thing here...
This solution is pretty poor, for the following reasons:
When I add a new protocol version, I have to find everywhere in the source tree that these if statements are used, and modify them to add the new functionality.
If a new protocol version comes along, and some parts of the protocol version are the same as another version, I need to modify the if statements so they read if (clientProtoVersion == 5 || clientProtoVersion == 6).
I'm sure there are more reasons why it's bad design, but I can't think of them right now.
What I'm looking for is a way to handle different protocols intelligently, using the features of the C++ langauge. I've thought about using template classes, possibly with the template parameter specifying the protocol version, or maybe a class heirarchy, one class for each different protocol version...
I'm sure this is a very common design pattern, so many many people must have had this problem before.
Edit:
Many of you have suggested an inheritance heirarchy, with the oldest protocol version at the top, like this (please excuse my ASCII art):
IProtocol
^
|
CProtoVersion1
^
|
CProtoVersion2
^
|
CProtoVersion3
... This seems like a sensible thing to do, in terms of resuse. However, what happens when you need to extend the protocol and add fundamentally new message types? If I add virtual methods in IProtocol, and implement these new methods in CProtocolVersion4, how are these new methods treated in earlier protocol versions? I guess my options are:
Make the default implementation a NO_OP (or possibly log a message somewhere).
Throw an exception, although this seems like a bad idea, even as I'm typing it.
... do something else?
Edit2:
Further to the above issues, what happens when a newer protocol message requires more input than an older version? For example:
in protocl version 1, I might have:
ByteArray getFooMessage(string param1, int param2)
And in protocol version 2 I might want:
ByteArray getFooMessage(string param1, int param2, float param3)
The two different protocol versions now have different method signatures, which is fine, except that it forces me to go through all calling code and change all calls with 2 params to 3 params, depending on the protocol version being used, which is what I'm trying to avoid in the first place!
What is the best way of separating protocol version information from the rest of your code, such that the specifics of the current protocol are hidden from you?
Since you need to dynamically choose which protocol to use, using different classes (rather than a template parameter) for selecting the protocol version seems like the right way to go. Essentially this is Strategy Pattern, though Visitor would also be a possibility if you wanted to get really elaborate.
Since these are all different versions of the same protocol, you could probably have common stuff in the base class, and then the differences in the sub classes. Another approach might be to have the base class be for the oldest version of the protocol and then have each subsequent version have a class that inherits from the previous version. This is a somewhat unusual inheritance tree, but the nice thing about it is that it guarantees that changes made for later versions don't affect older versions. (I'm assuming the classes for older versions of the protocol will stabilize pretty quickly and then rarely ever change.
However you decide to organize the hierarchy, you'd then want to chose the protocol version object as soon as you know the protocol version, and then pass that around to your various things that need to "talk" the protocol.
I have used (and heard of others using) templates to solve this problem too. The idea is that you break your different protocols up into basic atomic operations and then use something like boost::fusion::vector to build protocols out of the individual blocks.
The following is an extremely rough (lots of pieces missing) example:
// These are the kind of atomic operations that we can do:
struct read_string { /* ... */ };
struct write_string { /* ... */ };
struct read_int { /* ... */ };
struct write_int { /* ... */ };
// These are the different protocol versions
typedef vector<read_string, write_int> Protocol1;
typedef vector<read_int, read_string, write_int> Protocol2;
typedef vector<read_int, write_int, read_string, write_int> Protocol3;
// This type *does* the work for a given atomic operation
struct DoAtomicOp {
void operator ()(read_string & rs) const { ... }
void operator ()(write_string & ws) const { ... }
void operator ()(read_int & ri) const { ... }
void operator ()(write_int & wi) const { ... }
};
template <typename Protocol> void doProtWork ( ... ) {
Protocl prot;
for_each (prot, DoAtomicOp (...));
}
Because the protocl version is dynamic you'll need a single top level switch statement to determine which protocl to use.
void doWork (int protocol ...) {
switch (protocol) {
case PROTOCOL_1:
doProtWork<Protocol1> (...);
break;
case PROTOCOL_2:
doProtWork<Protocol2> (...);
break;
case PROTOCOL_3:
doProtWork<Protocol3> (...);
break;
};
}
To add a new protocol (that uses existing types) you only need to do define the protocl sequence:
typedef vector<read_int, write_int, write_int, write_int> Protocol4;
And then add a new entry to the switch statement.
I'd tend towards to using different classes to implement adapters for the different protocols to the same interface.
Depending on the protocol and the differences, you might get some gain using TMP for state machines or protocol details, but generating six sets of whatever code uses the six protocol versions is probably not worth it; runtime polymorphism is sufficient, and for most cases TCP IO is probably slow enough not to want to hard code everything.
maybe oversimplified, but this sounds like a job for inheritance?
A base class IProtocol which defines what a protocol does (and possibly some common methods), and then one implementation for IProtocol for each protocol you have?
You need a protocol factory that returns a protocol handler for the appropraite version:
ProtocolHandler& handler = ProtocolFactory::getProtocolHandler(clientProtocolVersion);
You can then use inheritance to only update the parts of the protocol that have been changed between versions.
Notice that the ProtocolHandler (base class) is basically a stratergy pattern. As such it should not maitain its own state (if required that is passed in via the methods and the stratergy will update the property of the state object).
Becuase the stratergy does not maintain state we can share the ProtcolHandler between any number of threads and as such the ownership does not need to leave the factory object. Thus the factory just need to create one handler object for each protocol version it understands (this can even be done lazily). Becuase the factory object retains ownership you can return a reference of the Protocol Handler.
I'm gonna agree with Pete Kirkham, I think it would be pretty unpleasant to maintain potentially 6 different versions of the classes to support the different protocol versions. If you can do it, it seems like it would be better to have the older versions just implement adapters to translate to the latest protocol so you only have one working protocol to maintain. You could still use an inheritance hierarchy like shown above, but the older protocol implementations just do an adaption then call the newest protocol.