passing information to multiple thread - c++

I am working on MultiThreaded TCP Server(c++) which will serve multiple client to stream Video.
I have decided to go with "thread per client" approach. There're no more than 100 simultaneous clients.
I am multiplexing the client using select system call and accepting the (accept) and then creating thread to stream video.
Is this Correct approach in this scenario?
I need to pass certain information like (socket id and handle to thread,threadID etc and other information to steam the video). How can i pass these information to all the thread, as these information are not common to all thread. I dont want to take array of HANDLE for the thread and thread id.

Please don'r "reinvent the wheel".
For example, the boost::asio library gives enormous help in creating a server such as you describe. There are lots of tutorials and examples - perhaps the most immediately useful to you would be http://www.boost.org/doc/libs/1_46_0/doc/html/boost_asio/example/echo/async_tcp_echo_server.cpp

usually starting a thread,
ether using standard approach of void function or functor class:
class StreamInfo
{
public:
int _id;
std::string _ip;
int _port;
int status;
StreamInfo(int id, string ip, int p):_id(id), _ip(ip), _port(p){};
};
void myThread(void* datastruct)
{
StreamInfo* info = static_cast<StreamInfo*>(datastruct);
..
..
//busy wait..
while(...)
{
// check for some new info
if(info->status)....
}
}
main(..)
{
multimap<int, StreamInfo*> mmap;
SOCK s=null;
StramInfo* i=null;
while(s = accept(sock))
{
i=new StreamInfo(12, "localhost", 5000);
id = createThread(...myThread, i);
....
mmap.append(pair<..,..>(id, i);
};
};
//...some other thread
/// change StreamInfo status for thread 12
mmap.last(12).status = 134;
//... then thread 12 will automatically recon this change in busy wait.
its just one approach, you could use observer pattern as well.
But this would require more work.
I imagine streaming is quite bandwidth, cpu and memory consuming.
So switching between threads might be not efficient enough though.
Maybe look at the asynchronous approach.
Boost::ASIO is pretty decent for that purpose and works on all platforms.

"Thread per client" is not scalable approach, especially for video streaming server (usually high-performance required). Have a look at Boost.Asio library, especially at example "An HTTP server using a single io_service and a thread pool calling io_service::run()." (just ignore HTTP-specific stuff). Using Boost.Asio you'll receive high-performance scalable multithreaded server w/o headache.

Related

TIdTCPServer sharing of serial port

Recently I changed a program which acts as a TCP server to help share the traffic on a serial port connected to a device. Multiple clients connect and should have access to the Serial Port and act simultaneously.
Application is built using C++Builder, using TIdTCPServer in the server and TIdTCPClient in the client application.
Multiple clients need to connect and send commands to the serial port. The serial port will respond immediately after sending a command to it, as per the protocols of the device it is attached to.
There is also a background thread which occasionally accesses the serial port and updates a memory cache of data held in the server's memory. The commands for sending and receiving from the serial port have a mutex on them, so they are accessible from both the TIdTcpServer's OnExecute event and the background thread.
I'm having difficulty getting the TIdTCPServer's OnExecute event to work without overlapping.
It would be really nice if the OnExecute event were to execute fully without another request coming in from another client, causing the overlapping.
Here is the OnExecute event handler of the TIdTCPServer:
void __fastcall TfrmMain::IServerExecute(TIdContext *AContext)
{
int i;
int Len;
TIdBytes TRB, TSB;
unsigned char ARB[BUFFERLENGTH];
int NumbSent, NumbReceived;
// Read the command from the client. Send the length first then the actual data.
Len = AContext->Connection->Socket->ReadLongInt();
AContext->Connection->Socket->ReadBytes(TRB, Len, false);
memset(ARB,0,BUFFERLENGTH);
for(i=0;i<Len;i++) AOB[i]=TRB[i];
NumbSent=Len;
// Now send it out to the Serial port
ProcessSerialMessage(AOB, Len, ARB, &NumbReceived, false);
sending=false;
TSB.Length=NumbReceived;
for(i=0;i<TSB.Length;i++) TSB[i]=ARB[i];
AContext->Connection->Socket->Write(TSB.Length);
AContext->Connection->Socket->Write(TSB);
return;
}
Here is the routine for sending the data out over the serial port:
int ProcessSerialMessage(unsigned char *SendBuf, int NumbSBytes, unsigned char *ReceiveBuf, int *NumbRBytes, bool CalledFromThread)
{
// MMUtex is a global TMutex Object
// Mutex required to help with the background thread trying to update memory cache.
MMutex->Acquire();
// Ok now send the data out over the serial port and receive it.
// These routines are standard serial port I/O routines and aren't explained here.
rawsend(SendBuf, NumbSBytes);
rawreceive(ReceiveBuf, NumbRBytes);
RetValue=*NumbRBytes;
MMutex->Release();
return(RetValue);
}
TIdTCPServer is a multi-threaded component. Each connected client runs in its own independent thread. The OnExecute event runs in those threads. So, it is your responsibility to make sure your OnExecute code is thread-safe, by serializing access to any shared resources.
You are using a mutex inside of ProcessSerialMessage(), so you are serializing access to the serial port (assuming your other background thread is also entering the same mutex). So that should be fine (although, I would suggest protecting the mutex locking/unlocking using a try..__finally block, or a local RAII-style class, to ensure the mutex is unlocked properly even if an exception is thrown).
However, one major issue I see with this code is that your AOB (and sending) variable is not declared as a local variable to IServerExecute(), which means it must be a shared variable accessed across threads (UPDATE: you have confirmed that in comments: "[AOB is] declared globally."). But, it is not being protected from concurrent access by multiple threads, which means that multiple clients are free to overwrite each other's inbound data while it is being sent to the serial port.
You are reading the serial port's response into local variables, and then using them to send back to the requesting clients. So there is no concurrency issue in that code.
I would suggest passing your TRB and TSB arrays directly to ProcessSerialMessage(). The 2 loops you have to copy bytes from one array to another are not really necessary, thus you can eliminate the AOB and ARB variables from this code completely. That might be enough to solve your issue.
Try this:
void __fastcall TfrmMain::IServerExecute(TIdContext *AContext)
{
TIdBytes TRB, TSB;
int NumbSent, NumbReceived;
// Read the command from the client. Send the length first then the actual data.
NumbSent = AContext->Connection->Socket->ReadLongInt();
AContext->Connection->Socket->ReadBytes(TRB, NumbSent, false);
TSB.Length = BUFFERLENGTH;
// Now send it out to the Serial port
ProcessSerialMessage(&TRB[0], NumbSent, &TSB[0], &NumbReceived, false);
AContext->Connection->Socket->Write(NumbReceived);
AContext->Connection->Socket->Write(TSB, NumbReceived);
}

Concurrent TCP server in C++

I am trying to create a concurrent c++ TCP server using threads. In particular I was wondering if I could use std::async to accept connections and serve each one in its own thread.
So far I have created a rough mockup but can't really tell if I am on the correct path.
void networking::TCP_Server::acceptConnection() {
std::string stringToSend{"This is a test string to be replied to"};
int new_fd = accept(listeningFD, nullptr, nullptr);
send(new_fd, stringToSend.c_str(), stringToSend.size(), 0);
sleep(3);
std::cout << ("End of thread");
}
///LISTEN FOR CONNECTIONS ON listeningFD
///CREATE A LIST OF FILE DESCRIPTORS FOR POLL fds[]
(fds[i].fd == listeningFD) {
do {
std::cout << ("New incoming connection - %d\n", new_fd);
std::async(std::launch::async, acceptConnection)
} while (new_fd != -1);
} /* End of existing connection is readable */
} /* End of loop through pollable descriptors */
I am connecting at the same time to the server with two clients and would expect for the loop to run through both new connections and create a thread for each one. As of now it is as it runs in deferred mode, one gets accepted, the other waits until the first finishes.
Any ideas?
(Pardon any mistakes in the code)
std::async returns a std::future which the code doesn't save into a variable, hence its destructor is called immediately. std::future::~future() blocks the calling thread until the future becomes ready.
You may like to use (detached) std::thread instead of std::async.
There are more scalable strategies to handle many clients. I highly recommend reading old but instructive The C10K problem.
You may also like to get familar with Asio C++ Library.

C++ Boost Serialization: Input Stream Error

Hi fellow C++ developers,
I'm trying to send a C++ class over the network with zmq and boost::serialization.
The concept is to serialize the class PlayCommand on the client. Then send it to the server with zmq. And the deserialize it on the server.
This works fine in the rest of the application. For some reason I get input stream errors while deserializing the PlayCommand on the server from time to time. I can not figure out why it is sometimes throwing this exception and sometimes not.
It seems to be a time sensitive problem. Do I have to wait at some point to let boost do its thing ?
std::shared_ptr<PlayCommand> _exe(dynamic_cast<PlayCommand*>(_cmd.get()));
zmq::context_t _ctx(1);
zmq::socket_t _skt(_ctx, ZMQ_PUB);
_skt.connect("tcp://0.0.0.0:" + this->kinect_daemon_com_port);
std::stringstream _type_stream;
std::stringstream _exe_stream;
boost::archive::text_oarchive _type_archive(_type_stream);
boost::archive::text_oarchive _exe_archive(_exe_stream);
_type_archive << _type;
_exe_archive << *_exe.get();
std::string _type_msg_str = _type_stream.str();
std::string _exe_msg_str = _exe_stream.str();
zmq::message_t _type_msg(_type_msg_str.length());
zmq::message_t _exe_msg(_exe_msg_str.length());
memcpy(_type_msg.data(), _type_msg_str.data(), _type_msg_str.length());
memcpy(_exe_msg.data(), _exe_msg_str.data(), _exe_msg_str.length());
_skt.send(_type_msg, ZMQ_SNDMORE);
_skt.send(_exe_msg, 0);
void ZMQMessageResolver::resolve_message(std::shared_ptr<Event> _event, unsigned _unique_thread_id)
{
std::cout << "ZMQMessageResolver::resolve_message(std::shared_ptr<Event> _event, unsigned _unique_thread_id)" << std::endl;
std::shared_ptr<ZMQMessageEvent> _zmq_event = std::static_pointer_cast<ZMQMessageEvent>(_event);
//(static_cast<ZMQMessageEvent*>(_event.get()));
ZMQMessageType _type;
PlayCommand _cmd;
auto _messages = _zmq_event->get_data();
auto _type_string = std::string(static_cast<char*>(_messages->front()->data()), _messages->front()->size());
auto _cmd_string = std::string(static_cast<char*>(_messages->back()->data()), _messages->back()->size());
std::stringstream _type_stream{_type_string};
std::istringstream _cmd_stream{_cmd_string};
boost::archive::text_iarchive _type_archive{_type_stream};
boost::archive::text_iarchive _cmd_archive{_cmd_stream};
std::cout << "1" << std::endl;
_type_archive >> _type;
std::cout << "2" << std::endl;
_cmd_archive & _cmd;
std::cout << "3" << std::endl;
std::shared_ptr<ThreadEvent> _thread_event = std::make_shared<ThreadEvent>(_zmq_event->get_event_message());
_cmd.execute(_thread_event);
std::lock_guard<std::mutex> _lock{*this->thread_mutex};
this->finished_threads.push_back(_unique_thread_id);
}
The complete project is on github: rgbd-calib and rgbd-calib-py.
The important files are /framework/ZMQMessageResolver.cpp in rgbd-calib and /src/KinectDaemon.cpp in rgbd-calib-py.
I would appreciate any help.
First insights
I checked for shared zmq::socket_t instances. I could not find any so thread safety should be a non issue.
I found out that other developers are also experiencing problems with ZMQ multi part messages. Maybe that could be an issue in my case as well. Maybe someone as experiences with those. Do I have to take any safety measures when sending and receiving multi part messages ?
If it's timing sensitive, no doubt it's unrelated to boost: the boost code shown is completely synchronous and local. If you don't always receive the full streams you get this error. Likewise if there's a protocol error interpreting the received data you might get corrupt data.
Both cases would easily lead to "input stream error".
I have no experience with 0MQ so I don't know whether the code as shown could receive incomplete messages, but I'd look into that.
A minor note is that it's rather strange to have one stringstream and the other istringstream. There might be differences in seek behaviour.
Let me add a few cents on ZeroMQ part of the story:
Fact #1:ZeroMQ does never deliver a piece of trash it delivers either a complete message( as it was sent ) or nothing at all
this principal design feature helps to sort out one of the claimed potential issues.
If an application indeed receives a ZeroMQ message delivered, one can be sure of its being of a shape and sound copy of what has been dispatched from the remote process. It is simply the same. Pullstop.
Fact #2: ZeroMQ architects and evangelists do in every chapter since the very beginning of the ZeroMQ API v2.xx warn to never share
which seems from the code depicted above unclear.
If one instantiates a ZeroMQ-socket AccessPoint ( of a SUB-type in the above context ), the thread owning such AccessPoint resource is the only thread that may manipulate with such resource and never "let" any other touch this toy. Never. While there might be some recent talks and efforts to re-design the ZeroMQ core, so as to add a thread-safety ex-post, I remain skeptical as per these moves, being principally sure that a non-blocking high-performance + low-latency motivated designs in distributed computing should never share a common piece, right because of the costs of overheads and lost principal safety ( which is not easy to be got just ex-post bought back by any inter-thread signalling / locking / blocking ).
You may review the code, so as to confirm or deny any kind of sharing ZeroMQ instances ( a Context being another, separate subject in this ) and for cases, where shared pieces were detected, your team ought re-design the code so as to avoid it.
Yes. Avoid sharing and ZeroMQ tools will serve you as a hell.

How do I use select() and gRPC to create a server?

I need to use gRPC but in a single-threaded application (with additional socket channels). Naively, I'm thinking of using select() and depending on which file descriptor pops, calling gRPC to handle the message. My question is, can someone give me a rough (5-10 lines of code) outline skeleton on what I need to call after the select() pops?
Looking at Google's "hello world" example in the synchronous case implies a thread pool (which I can't use), and in the asynchronous case shows the main loop blocking -- which doesn't work for me because I need to handle other socket operations.
You can't do it, at this point (and probably ever).
One of the big weaknesses of event loops, including direct use of select()/poll() style APIs, is that they aren't composable in any natural way short of direct integration between the two.
We could theoretically add such functionality for Linux -- exporting an epoll_fd with a timerfd which becomes readable if it would be productive to call into a completion queue, but doing so would impose substantial constraints and architectural overhead on the rest of the stack just to support this usecase only on Linux. Everywhere else would require a background thread to manage that fd's readability.
This can be done using a gRPC async service along with grpc::Alarm to send any events that come from select or other polling APIs onto the gRPC completion queue. You can see an example using Epoll and gRPC together in this gist. The important functions are these two:
bool grpc_tick(grpc::ServerCompletionQueue& queue) {
void* tag = nullptr;
bool ok = false;
auto next_status = queue.AsyncNext(&tag, &ok, std::chrono::system_clock::now());
if (next_status == grpc::CompletionQueue::GOT_EVENT) {
if (ok && tag) {
static_cast<RequestProcessor*>(tag)->grpc_queue_tick();
} else {
std::cerr << "Not OK or bad tag: " << ok << "; " << tag << std::endl;
return false;
}
}
return next_status != grpc::CompletionQueue::SHUTDOWN;
}
bool tick_loops(int epoll, grpc::ServerCompletionQueue& queue) {
// Pump epoll events over to gRPC's completion queue.
epoll_event event{0};
while (epoll_wait(epoll, &event, /*maxevents=*/1, /*timeout=*/0)) {
grpc::Alarm alarm;
alarm.Set(&queue, std::chrono::system_clock::now(), event.data.ptr);
if (!grpc_tick(queue)) return false;
}
// Make sure gRPC gets at least 1 tick.
return grpc_tick(queue);
}
Here you can see the tick_loops function repeatedly calls epoll_wait until no more events are returned. For each epoll event, a grpc::Alarm is constructed with the deadline set to right now. After that, the gRPC event loop is immediately pumped with grpc_tick.
Note that the grpc::Alarm instance MUST outlive its time on the completion queue. In a real-world application, the alarm should be somehow attached to the tag (event.data.ptr in this example) so it can be cleaned up in the completion callback.
The gRPC event loop is then pumped again to ensure that any non-epoll events are also processed.
Completion queues are thread safe, so you could also put the epoll pump on one thread and the gRPC pump on another. With this setup you would not need to set the polling timeouts for each to 0 as they are in this example. This would reduce CPU usage by limiting dry cycles of the event loop pumps.

Multi-threaded Server handling multiple clients in one thread

I wanted to create a multi-threaded socket server using C++11 and standard linux C-Librarys.
The easiest way doing this would be opening a new thread for each incoming connection, but there must be an other way, because Apache isn't doing this. As far as I know Apache handles more than one connection in a Thread. How to realise such a system?
I thought of creating one thread always listening for new clients and assigning this new client to a thread. But if all threads are excecuting an "select()" currently, having an infinite timeout and none of the already assigned client is doing anything, this could take a while for the client to be useable.
So the "select()" needs a timeout. Setting the timeout to 0.5ms would be nice, but I guess the workload could rise too much, couldn't it?
Can someone of you tell me how you would realise such a system, handling more than one client for each thread?
PS: Hope my English is well enough for you to understand what I mean ;)
The standard method to multiplex multiple requests onto a single thread is to use the Reactor pattern. A central object (typically called a SelectServer, SocketServer, or IOService), monitors all the sockets from running requests and issues callbacks when the sockets are ready to continue reading or writing.
As others have stated, rolling your own is probably a bad idea. Handling timeouts, errors, and cross platform compatibility (e.g. epoll for linux, kqueue for bsd, iocp for windows) is tricky. Use boost::asio or libevent for production systems.
Here is a skeleton SelectServer (compiles but not tested) to give you an idea:
#include <sys/select.h>
#include <functional>
#include <map>
class SelectServer {
public:
enum ReadyType {
READABLE = 0,
WRITABLE = 1
};
void CallWhenReady(ReadyType type, int fd, std::function<void()> closure) {
SocketHolder holder;
holder.fd = fd;
holder.type = type;
holder.closure = closure;
socket_map_[fd] = holder;
}
void Run() {
fd_set read_fds;
fd_set write_fds;
while (1) {
if (socket_map_.empty()) break;
int max_fd = -1;
FD_ZERO(&read_fds);
FD_ZERO(&write_fds);
for (const auto& pr : socket_map_) {
if (pr.second.type == READABLE) {
FD_SET(pr.second.fd, &read_fds);
} else {
FD_SET(pr.second.fd, &write_fds);
}
if (pr.second.fd > max_fd) max_fd = pr.second.fd;
}
int ret_val = select(max_fd + 1, &read_fds, &write_fds, 0, 0);
if (ret_val <= 0) {
// TODO: Handle error.
break;
} else {
for (auto it = socket_map_.begin(); it != socket_map_.end(); ) {
if (FD_ISSET(it->first, &read_fds) ||
FD_ISSET(it->first, &write_fds)) {
it->second.closure();
socket_map_.erase(it++);
} else {
++it;
}
}
}
}
}
private:
struct SocketHolder {
int fd;
ReadyType type;
std::function<void()> closure;
};
std::map<int, SocketHolder> socket_map_;
};
First off, have a look at using poll() instead of select(): it works better when you have large number of file descriptors used from different threads.
To get threads currently waiting in I/O out of waiting I'm aware of two methods:
You can send a suitable signal to the thread using pthread_kill(). The call to poll() fails and errno is set to EINTR.
Some systems allow a file descriptor to be obtained from a thread control device. poll()ing the corresponding file descriptor for input succeeds when the thread control device is signalled. See, e.g., Can we obtain a file descriptor for a semaphore or condition variable?.
This is not a trivial task.
In order to achieve that, you need to maintain a list of all opened sockets (the server socket and the sockets to current clients). You then use the select() function to which you can give a list of sockets (file descriptors). With correct parameters, select() will wait until any event happen on one of the sockets.
You then must find the socket(s) which caused select() to exit and process the event(s). For the server socket, it can be a new client. For client sockets, it can be requests, termination notification, etc.
Regarding what you say in your question, I think you are not understanding the select() API very well. It is OK to have concurrent select() calls in different threads, as long as they are not waiting on the same sockets. Then if the clients are not doing anything, it doesn't prevent the server select() from working and accepting new clients.
You only need to give select() a timeout if you want to be able to do things even if clients are not doing anything. For example, you may have a timer to send periodic infos to the clients. You then give select a timeout corresponding to you first timer to expire, and process the expired timer when select() returns (along with any other concurrent events).
I suggest you have a long read of the select manpage.