thread safe class implementation for two servers - c++

Question

I have a class which has 2 members:
RequestController {
public:
SendRequest(); // called by multiple threads synchronously
private:
server primary; // This is the primary server
server backup; // This is the back up server.
}
My logic is simply this:
In SendRequest(), I want to send the request to primary server,
if it fails, I want to send it to backup server, if it passes,
I want to swap primary server and backup server.
Here the problem comes: when I do the swapping, I have to lock the primary and
backup (this is the place where multiple threads can not do at the same time).
Actually I need to make sure when I swap, no threads are reading primary server.
How do I write this piece of code in an efficient way?
I don't want to lock the whole thing, as for most of the
case, primary server works and there's no need to lock.
I think generally this problem is language independent. Anyway I tag this
with C++.

Answer 1

Lets assume that the servers take some non-negligible amount of time to process a request. Then if the requests are coming fast enough, you will have the case where SendRequest is called a second time while it is waiting for one of the servers to processes a previous request.
As a designer, you have two choices.
If it is OK for a server to process multiple requests simultaneously, then you do nothing.
If a server can only process a single request at a time, then you will need to perform some kind of synchronization on the code.
In case 2, since you already have a lock on the servers, you can swap them with no ramifications.
For case 1, why not do the following:
std::mutex my_mutex;
...
// Select the server
server* selected = NULL;
my_mutex.lock();
selected = &primary;
my_mutex.unlock();
// Let the selected server process the message.
bool success = selected->process();
// If there was a primary failure, see if we can try the backup.
if (!success) {
my_mutex.lock();
if (selected == &primary) {
selected = &backup;
}
my_mutex.unlock();
// Now try again
success = selected->process();
// If the backup was used successfully, swap the primary and backup.
if (success) {
my_mutex.lock();
if (selected == &backup) {
backup = primary;
primary = selected;
}
my_mutex.unlock();
}
}
But this could have some problems. Say for example that primary fails on the first message, but is successful on the rest. If SendRequest() is called at the same time by 3 different threads, then you could have the following:
Thread 1 - sends with primary
Thread 2 - sends with primary
Thread 3 - sends with primary
Thread 1 - fails, sends with backup
Thread 2 - primary succeeds
Thread 1 - backup succeeds
Thread 1 - swaps primary and backup
Thread 3 - old primary (new backup) succeeds
Thread 3 - swaps primary and backup
If the messages keep coming fast enough, it is possible to remain in this state where you keep swapping primary and backup. The condition would resolve the moment there are no pending messages, and then the primary and backup would be set until there is another failure.
Perhaps a better way would be to never swap, but have a better selection method. For example:
...
// Select the server
server* selected = NULL;
selected = &primary;
if (!primary.last_message_successful) {
// The most recent attempt made with primary was a failure.
if (backup.last_message_successful) {
// The backup is thought to be functioning.
selected = &backup;
}
}
// Let the selected server process the message.
// If successful, process() will set the last_message_successful boolean.
bool success = selected->process();
// If there was a failure, try the other one.
if (!success) {
if (&primary == selected) {
selected = &backup;
} else {
selected = &primary;
}
}
// Try again with the other one.
selected->process();
In this example, the lock is not necessary. Primary will be used until it fails. Then the backup will be used. If other messages are processed in the mean time, it may result in the primary becoming usable again, in this case it will be used. Otherwise, backup will b used until it fails. If both fail, they will both be attempted, first primary, and then backup.