The goal is to be able to support both IPv4 and IPv6 for a rather complex application. At present, only IPv4 is handled. There is a class called Socket and a class called TlsSocket which extends from it. Let's say that Socket has a set of methods M1,M2,....,M9. TlsSocket overrides M7, M8 and M9.
Given the current design I was thinking of making Socket an abstract class and extending it twice - SocketIPv4 and SocketIPv6 which would implement methods M5 and M6 differently. However, then I would have to extend them twice again to have a TLS version for both IPv4 socket and IPv6 socket leading to code duplication. I was looking at the best design pattern for the problem at hand and I was convinced that the decorator design pattern would work best.
However, then TlsSocket would inherit from the abstract Socket class and then be composed of a concrete implementation of Socket (either IPv4 or IPv6). Therefore, I would essentially be initializing two Socket instances (one for composition and the other is TlsSocket itself) pointing to the same file descriptor. Everything should work fine but I am slightly uncomfortable initializing two socket instances pointing to the same file descriptor. Is there an alternative design pattern that I have missed and should consider?
Related
First time post. This is a general question about how to do something in C++. To not waste your time, I'll just ask it generalized. But if your response is something like, "Why do you want to do something so stupid like that?", I'll give the details below.
General Question
I have a base class with common code (SPIBase). I have two classes that inherit it that have significantly different implementations (SPIMaster, SPISlave). In each of these two classes, I expose public/protected functionality that is identical. I want to write a single class that optionally inherits from one of these two classes. How should I go about doing that?
Details
I am building a library for some microelectronics (ESP8266). I use the Arduino IDE, and the library is in C++. As you might be familiar, SPI is a methodology/protocol for communicating between two micro-controllers or a micro-controller and something like a sensor. A micro-controller is always a Master and the other device/sensor is always a Slave. The Master completely controls the communication process in SPI. The Slave is normally relatively passive. I've written the library to work only between two ESP8266 micro-controllers to form a TCP/IP like protocol where either can send a message to the other un-solicited. Basically, the Master/Slave issues are hidden and the developer using the library can setup their protocol any way they prefer and in either direction. (i.e. Client/Server/Peer messaging, send/response, Streams, etc). Although, Master/Slave issues are hidden, obviously, one ESP8266 must use one base implementation and the other ESP8266 must use the other.
I now want to write the layer that implements Streams / File transfers. This class will ride on one or the other of the two intermediate classes, but will be identical except in which class they inherit. They both will be both senders and receivers. I'd rather not have two complete classes that are really identical in their implementation. In this case, I don't really want any portion of one chain of code to be on the other device. My original thought proved after research not to be possible was to basically... have the library using developer use #define MASTER or #define SLAVE in their main program file and the streaming class would inherit one or the other classes based on this. I soon discovered that #defines aren't visible downstream.
Is there some C++ design methodology / pattern that will get the results I want?
Thanks for your time.
You can use templates. For example:
template <typename SPIImpl>
class SPIStream : public SPIImpl
{
// stream implementation using the common interface
};
You would then declare an instance of the class like this:
SPIStream<SPIMaster> spi_master_stream;
OR
SPIStream<SPISlave> spi_slave_stream;
The documentation doesn't seem to tell much: lowest_layer(), next_layer().
What is the difference between them and when to use each?
To answer this, first thing to remember is that boost::asio::ssl::stream is a template class. Usually it look like boost::asio::ssl::stream<boost::asio::ip::tcp::socket>. Thus is implemented using boost::asio::ip::tcp::socket. That will be next_layer for boost::asio::ssl::stream. On other side, lowest_layer is always will be a basic_socket (its described in the docs).
Its little ambiguous especially when you see in the headers tcp::socket is typedef to basic_stream_socket<Tcp>, which is directly inherited from basic_socket. And.. In OOP terms you can say "next_layer IS the lowest_layer"..
But lets take another case where you create a ssl::stream< MyOwnClass >. In this case next_layer is MyOwnClass, which should control data reads/writes. And lowest_layer will be be whatever MyOwnClass will say in its typedef.
UPD: When to use each. Use next_layer for read/writes (you don't need this for SSL connection, but before starttls session its required). And use lowest_layer to control underlying socket.
I've seen some people creating a "is-a" relationship like the following:
class TCPClient : public Socket
{
public:
TCPClient(const std::string& host, unsigned short port);
};
where the Socket class implements Winsock functions such as Connect(), Close(), Bind() etc.
Examples:
Example 1
Example 2
But this doesn't feel natural to me who is a newbie in socket programming.
Does the above hierarchy make more logical sense than the following "has-a" counterpart?
class TCPClient
{
public:
TCPClient(const std::string& host, unsigned short port);
....
private:
Socket m_socket;
};
A TCPClient uses a socket or has a socket, but is not itself a socket, and you wouldn't normally expect to be able to substitute a TCPClient anywhere a socket was expected. As such, public inheritance doesn't make sense.
You could use private inheritance for this case, but (at least in a typical case) it probably doesn't make much sense either. Private inheritance makes sense primarily when the base class provides at least one virtual function you plan to override in the child class. If you have a virtual function and need to override it, you have no real choice but to use inheritance. I wouldn't expect a Socket class to have an virtual functions though; that wouldn't normally apply here.
That basically leads to your second solution: the TCPClient should contain an instance of a Socket, rather than using inheritance at all.
I should add, however, that the Socket class you've shown seems to conflate the notion of an actual socket with the notion of an address. My first socket class (years ago) worked about like that, but since then I've concluded that it's not really an ideal design. I've become convinced that it's worthwhile to keep the notion of an address separate from the socket itself. Though mine is a bit less elaborate, I find it interesting that what I came up with looks almost like it could have been the prototype from which Boost ASIO was derived. It's a little smaller and simpler, but a lot of the basic ideas are generally pretty similar anyway.
That leads to my next recommendation: take a look at Boost ASIO. Lacking a fairly specific reason to do otherwise, it's what I'd advise (and generally use) in most new code. Although (as I said above) I've written several socket classes over the years, I haven't used any of them in much (any?) new code in quite a while now -- they really only have two possible advantages over ASIO. The first applies only to me: since I wrote and used them before ASIO existed, I already understand them and how they work. The second may be similar: at least to me, they seem a little bit smaller and simpler (but, again, that may be just because I used them first). Even so, the advantages of (for example) using something other people already understand trumps those quite easily.
Use has-a. A TCPClient uses a socket like a person uses a telephone. Would you derive a Person from a Telephone?
class TCPClient : public Socket
{
public:
TCPClient(const std::string& host, unsigned short port);
};
Network sockets are used not only in TCP/IP and the above design is more suitable if you plan to reuse your "Socket" class to implement other protocols using network sockets. For example:
class UDPClient : public Socket
{
};
I would say so. Socket is an abstraction, a file descriptor (UNIX) or handle (Windows), which has resources associated with it and is managed by the operating system. If we consider OSI model, the socket fits well into the presentation layer (it presents, or describes, a communication channel between two nodes), whereas a program that uses the socket sits on the application layer.
Considering this, I would prefer not to inherit from the socket, unless I implement a kind of advanced socket (by analogy: C-pointer vs. smart pointer) to present and handle a logical connection and somehow manage the resources associated with the socket. If XYZClient is an application, whose goal is to implement some business or data processing logic, I would not mix these two concepts together and use the second approach (has-a).
I would split infrastructure/resource-specific and business/application-specific logic.
I've written a number of networking systems and have a good idea of how networking works. However I always end up having a packet receive function which is a giant switch statement. This is beginning to get to me. I'd far rather a nice elegant object-oriented way to handle receiving packets but every time I try to come up with a good solution I always end up coming up short.
For example lets say you have a network server. It is simply waiting there for responses. A packet comes in and the server needs to validate the packet and then it needs to decide how to handle it.
At the moment I have been doing this by switching on the packet id in the header and then having a huge bunch of function calls that handle each packet type. With complicated networking systems this results in a monolithic switch statement and I really don't like handling it this way. One way I've considered is to use a map of handler classes. I can then pass the packet to the relevant class and handle the incoming data. The problem I have with this is that I need some way to "register" each packet handler with the map. This means, generally, I need to create a static copy of the class and then in the constructor register it with the central packet handler. While this works it really seems like an inelegant and fiddly way of handling it.
Edit: Equally it would be ideal to have a nice system that works both ways. ie a class structure that easily handles sending the same packet types as receiving them (through different functions obviously).
Can anyone point me towards a better way to handle incoming packets? Links and useful information are much appreciated!
Apologies if I haven't described my problem well as my inability to describe it well is also the reason I've never managed to come up with a solution.
About the way to handle the packet type: for me the map is the best. However I'd use a plain array (or a vector) instead of a map. It would make access time constant if you enumerate your packet types sequentially from 0.
As to the class structure. There are libraries that already do this job: Available Game network protocol definition languages and code generation. E.g. Google's Protocol Buffer seems to be promising. It generates a storage class with getters, setters, serialization and deserialization routines for every message in the protocol description. The protocol description language looks more or less rich.
A map of handler instances is pretty much the best way to handle it. Nothing inelegant about it.
In my experience, table driven parsing is the most efficient method.
Although std::map is nice, I end up using static tables. The std::map cannot be statically initialized as a constant table. It must be loaded during run-time. Tables (arrays of structures) can be declared as data and initialized at compile time. I have not encountered tables big enough where a linear search was a bottleneck. Usually the table size is small enough that the overhead in a binary search is slower than a linear search.
For high performance, I'll use the message data as an index into the table.
When you are doing OOP, you try to represent every thing as an object, right? So your protocol messages become objects too; you'll probably have a base class YourProtocolMessageBase which will encapsulate any message's behavior and from which you will inherit your polymorphically specialized messages. Then you just need a way to turn every message (i.e. every YourProtocolMessageBase instance) into a string of bytes, and a way to do reverse. Such methods are called serialization techniques; some metaprogramming-based implementations exist.
Quick example in Python:
from socket import *
sock = socket(AF_INET6, SOCK_STREAM)
sock.bind(("localhost", 1234))
rsock, addr = sock.accept()
Server blocks, fire up another instance for a client:
from socket import *
clientsock = socket(AF_INET6, SOCK_STREAM)
clientsock.connect(("localhost", 1234))
Now use Python's built-in serialization module, pickle; client:
import pickle
obj = {1: "test", 2: 138, 3: ("foo", "bar")}
clientsock.send(pickle.dumps(obj))
Server:
>>> import pickle
>>> r = pickle.loads(rsock.recv(1000))
>>> r
{1: 'test', 2: 138, 3: ('foo', 'bar')}
So, as you can see, I just sent over link-local a Python object. Isn't this OOP?
I think the only possible alternative to serializing is maintaining the bimap IDs ⇔ classes. This looks really inevitable.
You want to keep using the same packet network protocol, but translate that into an Object in programming, right ?
There are several protocols that allow you to treat data as programming objects, but it seems, you don't want to change the protocol, just the way its treated in your application.
Does the packets come with something like a "tag" or metadata or any "id" or "data type" that allows you to map to an specific object class ? If it does, you may create an array that stores the id. and the matching class, and generate an object.
A more OO way to handle this is to build a state machine using the state pattern.
Handling incoming raw data is parsing where state machines provide an elegant solution (you will have to choose between elegant and performance)
You have a data buffer to process, each state has a handle buffer method that parses and processes his part of the buffer (if already possible) and sets the next state based on the content.
If you want to go for performance, you still can use a state machine, but leave out the OO part.
I would use Flatbuffers and/or Cap’n Proto code generators.
I solved this problem as part of my btech in network security and network programming and I can assure it's not one giant packet switch statement. The library is called cross platform networking and I modeled it around the OSI model and how to output it as a simple object serialization. The repository is here: https://bitbucket.org/ptroen/crossplatformnetwork/src/master/
Their is a countless protocols like NACK, HTTP, TCP,UDP,RTP,Multicast and they all are invoked via C++ metatemplates. Ok that is the summarized answer now let me dive a bit deeper and explain how you solve this problem and why this library can help you out whether you design it yourself or use the library.
First, let's talk about design patterns in general. To make it nicely organized you need first some design patterns around it as a way to frame your problem. For my C++ templates I framed it initially around the OSI Model(https://en.wikipedia.org/wiki/OSI_model#Layer_7:_Application_layer) down to the transport level(which becomes sockets at that point). To recap OSI :
Application Layer: What it means to the end user. IE signals getting deserialized or serialized and passed down or up from the networking stack
Presentation: Data independence from application and network stack
Session: dialogues between sessions
Transport: transporting the packets
But here's the kicker when you look at these closely these aren't design pattern but more like namespaces around transporting from A to B. So to a end user I designed cross platform network with the following standardized C++ metatemplate
template <class TFlyWeightServerIncoming, // a class representing the servers incoming payload. Note a flyweight is a design pattern that's a union of types ie putting things together. This is where you pack your incoming objects
class TFlyWeightServerOutgoing, // a class representing the servers outgoing payload of different types
class TServerSession, // a hook class that represent how to translate the payload in the form of a session layer translation. Key is to stay true to separation of concerns(https://en.wikipedia.org/wiki/Separation_of_concerns)
class TInitializationParameters> // a class representing initialization of the server(ie ports ,etc..)
two examples: https://bitbucket.org/ptroen/crossplatformnetwork/src/master/OSI/Transport/TCP/TCPTransport.h
https://bitbucket.org/ptroen/crossplatformnetwork/src/master/OSI/Transport/HTTP/HTTPTransport.h
And each protocol can be invoked like this:
OSI::Transport::Interface::ITransportInitializationParameters init_parameters;
const size_t defaultTCPPort = 80;
init_parameters.ParseServerArgs(&(*argv), argc, defaultTCPPort, defaultTCPPort);
OSI::Transport::TCP::TCP_ServerTransport<SampleProtocol::IncomingPayload<OSI::Transport::Interface::ITransportInitializationParameters>, SampleProtocol::OutgoingPayload<OSI::Transport::Interface::ITransportInitializationParameters>, SampleProtocol::SampleProtocolServerSession<OSI::Transport::Interface::ITransportInitializationParameters>, OSI::Transport::Interface::ITransportInitializationParameters> tcpTransport(init_parameters);
tcpTransport.RunServer();
citation:
https://bitbucket.org/ptroen/crossplatformnetwork/src/master/OSI/Application/Stub/TCPServer/main.cc
I also have in the code base under MVC a full MVC implementation that builds on top of this but let's get back to your question. You mentioned:
"At the moment I have been doing this by switching on the packet id in the header and then having a huge bunch of function calls that handle each packet type."
" With complicated networking systems this results in a monolithic switch statement and I really don't like handling it this way. One way I've considered is to use a map of handler classes. I can then pass the packet to the relevant class and handle the incoming data. The problem I have with this is that I need some way to "register" each packet handler with the map. This means, generally, I need to create a static copy of the class and then in the constructor register it with the central packet handler. While this works it really seems like an inelegant and fiddly way of handling it."
In cross platform network the approach to adding new types is as follows:
After you defined the server type you just need to make the incoming and outgoing types. The actual mechanism for handling them is embedded with in the incoming object type. The methods within it are ToString(), FromString(),size() and max_size(). These deal with the security concerns of keeping the layers below the application layer secure. But since your defining object handlers now you need to make the translation code to different object types. You'll need at minimum within this object:
1.A list of enumerated object types for the application layer. This could be as simple as numbering them. But for things like the session layer have a look at session layer concerns(for instance RTP has things like jitter and how to deal with imperfect connection. IE session concerns). Now you could also switch from enumerated to a hash/map but that's just another way of dealing of the problem how to look up the variable.
Defining Serialize and de serialize the object(for both incoming and outgoing types).
After you serialized or deserialize put the logic to dispatch it to the appropriate internal design pattern to handle the application layer. This could possibly be a builder , or command or strategy it really depends on it's use case. In cross platform some concerns is delegated by the TServerSession layer and others by the incoming and outgoing classes. It just depends on the seperation of concerns.
Deal with performance concerns. IE its not blocking(which becomes a bigger concern when you scale up concurrent user).
Deal with security concerns(pen test).
If you curious you can review my api implementation and it's a single threaded async boost reactor implementation and when you combine with something like mimalloc(to override new delete) you can get very good performance. I measured like 50k connections on a single thread easily.
But yeah it's all about framing your server in good design patterns , separating the concerns and selecting a good model to represent the server design. I believe the OSI model is appropriate for that which is why i put in cross platform network to provide superior object oriented networking.
I'm writing an RPC middleware in C++. I have a class named RPCClientProxy that contains a socket client inside:
class RPCClientProxy {
...
private:
Socket* pSocket;
...
}
The constructor:
RPCClientProxy::RPCClientProxy(host, port) {
pSocket = new Socket(host, port);
}
As you can see, I don't need to tell the user that I have a socket inside.
Although, to make unit tests for my proxies it would be necessary to create mocks for sockets and pass them to the proxies, and to do so I must use a setter or pass a factory to the sockets in the proxies's constructors.
My question: According to TDD, is it acceptable to do it ONLY because the tests? As you can see, these changes would change the way the library is used by a programmer.
I don't adhere to a certain canon i would say if you think you would benefit from testing through a mock socket the do it, you could implement a parallel constructor
RPCClientProxy::RPCClientProxy(Socket* socket)
{
pSocket = socket
}
Another option would be to implement a host to connect to for testing that you can configure to expect certain messages
What you describe is a perfectly normal situation, and there are established patterns that can help you implement your tests in a way that won't affect your production code.
One way to solve this is to use a Test Specific Subclass where you could add a setter for the socket member and use a mock socket in the case of a test. Of-course you would need to make the variable protected rather than private but that's probably no biggie. For example:
class RPCClientProxy
{
...
protected:
Socket* pSocket;
...
};
class TestableClientProxy : public RPCClientProxy
{
TestableClientProxy(Socket *pSocket)
{
this->pSocket = pSocket;
}
};
void SomeTest()
{
MockSocket *pMockSocket = new MockSocket(); // or however you do this in your world.
TestableClientProxy proxy(pMockSocket);
....
assert pMockSocket->foo;
}
In the end it comes down to the fact that you often (more often than not in C++) have to design your code in such a way as to make it testable and there is nothing wrong with that. If you can avoid these decisions leaking out into the public interfaces that may be better sometimes, but in other cases it can be better to choose, for example, dependency inject through constructor parameters above say, using a singleton to provide access to a specific instance.
Side note: It's probably worth taking a look through the rest of the xunitpatterns.com site: there are a whole load of well established unit-testing patterns to understand and hopefully you can gain from the knowledge of those who have been there before you :)
Your issue is more a problem of design.
If you ever with to implement another behavior for Socket, you're toasted, as it involves rewriting all the code that created sockets.
The usual idea is to use an abstract base class (interface) Socket and then use an Abstract Factory to create the socket you wish depending on the circumstances. The factory itself could be either a Singleton (though I prefer Monoid) or passed down as arguments (according to the tenants of Dependency Injection). Note that the latter means no global variable, which is much better for testing, of course.
So I would advise something along the lines of:
int main(int argc, char* argv[])
{
SocketsFactoryMock sf;
std::string host, port;
// initialize them
std::unique_ptr<Socket> socket = sf.create(host,port);
RPCClientProxy rpc(socket);
}
It has an impact on the client: you no longer hide the fact that you use sockets behind the scenes. On the other hand, it gives control to the client who may wish to develop some custom sockets (to log, to trigger actions, etc..)
So it IS a design change, but it is not caused by TDD itself. TDD just takes advantage of the higher degree of control.
Also note the clear resource ownership expressed by the use of unique_ptr.
As others have pointed out, a factory architecture or a test-specific subclass are both good options in this situation. For completeness, one other possibility is to use a default argument:
RGCClientProxy::RPCClientProxy(Socket *socket = NULL)
{
if(socket == NULL) {
socket = new Socket();
}
//...
}
This is, perhaps somewhere between the factory paradigm (which is ultimately the most flexible, but more painful for the user) and newing up a socket inside your constructor. It has the benefit that existing client code doesn't need to be modified.