Ours is a huge project. I need to call certain functions in my code asynchronously to avoid some circular function calls. Upon receiving a specific input, I can call my function asynchronously either by using Event or Timer.
Which way is preferable considering Performance ?
Sending events to Event manager and handling them with an Event handler ? Or
Starting a timer and provide timeout handler ?
For pure performance, event-driven model will be better. Use timers only if you cannot rely on one or more of your events to get set in a timely way by the worker code, and so need a backup means by which to continue processing. This may be the case if your worker code makes external calls to a database or other remote service whose reliability or performance is unproven.
Related
I wonder if anyone familiar with a synchronization mechanism in user-mode, by which an app can register a "callback" function that would be called when another app signals it ... i don't mind the callback to be in an arbitraty thread.
Suppose i'm having lots of "Worker" processes in parallel, And one wants to notify them of a change (no payloaded data needed), by which every process will have to do some internal updates.
The immediate approach to this was to create another thread in each of them, and have an infinite loop that waits for a global event and call the callback function right afterwards. To signal this, one process would only need to signal this global event.
The problem is that i'll have lots of parallel processes in this project, i don't want to add thread*nProcesses to the system just to implement this, even if they're mostly paused.
The current "workaround" i found for this would be to hold my own "dummy" registry key, and every process will "register registery notification callback", when one app wants to notify the others it will just trigger a write to this key... and windows will callback every process which registered to this notification.
Any other ideas?
The nicer solution, which doesn't pollute the registry, would be to use a shared pipe. All workers can connect to the named pipe server, and do an async read. When the server wants to kick the workers, it just writes a byte. This triggers the completion routine of the worker. Basic example
Still, this notification has the same drawback as most other Windows notifications. If all of your worker threads are running worker code, there's no thread on which your notification can arrive - and you didn't create a special thread for that purpose either. The only solution around that is CreateRemoteThread, but that's a very big hammer.
thank you all for the useful ideas,
Eventually, I accidentally came across RegisterWaitForSingleObject which seems to do just that.
I'm still taking in account #MSalters comment about not having enough free worker threads at a given time since i'm assuming this callback mechanism relies on the same callback mechanism most Win32API does
I have implemented my own Timer/Callback classes in C/C++ in Linux, wherein a process requiring a timer to fire either ONE_SHOT or PERIODICally instantiates a timer, and instantiates a callback object and associates the callback with previously created Timer object. The Callback class implements a triggered () method, and when the timer fires at the appointed timeout, the triggered () method is executed. (Nothing new in terms of functionality.) The way my Timer class works is I maintain a minheap of Timer objects and thus always know which timer to fire next. There is a timer task (TimerTask) which itself runs as a separate process (created using fork ()) and shares the memory pools from which the Timer objects and the Callback objects are created. The TimerTask has a main while (1) loop which keeps checking if the root of the Timer object minheap has a time since epoch that is LEQ the current time since epoch. If so, the timer at root has "fired."
Currently, when the timer fires, the callback is executed in the TimerTask process context. I am currently changing this behavior to run the callback processing on other tasks (send them the information that the Timer object has fired via a POSIX message queue. For example, send the message to the Timer object creating process), but my question to SO is what are the principles behind this? Executing a callback in the TimerTask context seems like a bad idea if I expect to service a large number of timers. It seems like a good idea to dispatch the callback processing over to other processes.
What are the general rules of thumb for processing the callback in one task/process over the other? My intention is to process the callback in the receiving task using a pthread like so:
void threadFunctionForTimerCallback (void* arg)
{
while (1)
{
if ((mq_receive (msg_fd, buffer, attr.mq_msgsize, NULL)) == -1)
exit (-1);
else
printf ("Message received %s\n", buffer);
}
}
Would this be a reasonable solution? But never mind the actual way of receiving the message from the TimerTask (threads or any other method, doesn't matter), any discussion and insight into the problem of assigning a task for the callback is appreciated.
There is no need to busy spin while(1) to implement a timer. One traditional and robust way of implementing timers has been using minheap as you do to organize times to expiry and then pass the time till the next timer expiry as a timeout argument to select() or epoll(). Using select() call a thread can watch for file descriptor readiness, signals and timers all at the same time.
Recent kernels support timerfd that delivers timer expiry events as file descriptor readiness for read which again can be handled using select()/epoll(). It obviates the need to maintain the minheap, however, requires a system call for each add/modify/delete a timer.
Having timer code in another process requires processes to use inter-process communication mechanisms, thereby introducing more complexity, so it can actually make the system less robust, especially when the processes communicate via shared memory and can corrupt it.
Anyway, one can use Unix domain sockets to send messages back and forth between communicating processes on the same host. Again, select()/epoll() are your best friends. Or a more high level framework can be used for message passing, such as 0MQ.
We have an API that handles event timers. This API says that it uses OS callbacks to handle timed events (using select(), apparently).
The api claims this order of execution as well:
readable events
writable events
timer events
This works by creating a point to a Timer object, but passing the create function a function callback:
Something along these lines:
Timer* theTimer = Timer::Event::create(timeInterval,&Thisclass::FunctionName);
I was wondering how this worked?
The operating system is handling the timer itself, and when it sees it fired how does it actually invoke the callback? Does the callback run in a seperate thread of execution?
When I put a pthread_self() call inside the callback function (Thisclass::FunctionName) it appears to have the same thread id as the thread where theTimer is created itself! (Very confused by this)
Also: What does that priority list above mean? What is a writable event vs a readable event vs a timer event?
Any explanation of the use of select() in this scenario is also appreciated.
Thanks!
This looks like a simple wrapper around select(2). The class keeps a list of callbacks, I guess separate for read, write, and timer expiration. Then there's something like a dispatch or wait call somewhere there that packs given file descriptors into sets, calculates minimum timeout, and invokes select with these arguments. When select returns, the wrapper probably goes over read set first, invoking read callback, then write set, then looks if any of the timers have expired and invokes those callbacks. This all might happen on the same thread, or on separate threads depending on the implementation of the wrapper.
You should read up on select and poll - they are very handy.
The general term is IO demultiplexing.
A readable event means that data is available for reading on a particular file descriptor without blocking, and a writable event means that you can write to a particular file descriptor without blocking. These are most often used with sockets and pipes. See the select() manual page for details on these.
A timer event means that a previously created timer has expired. If the library is using select() or poll(), the library itself has to keep track of timers since these functions accept a single timeout. The library must calculate the time remaining until the first timer expires, and use that for the timeout parameter. Another approach is to use timer_create(), or an older variant like setitimer() or alarm() to receive notification via a signal.
You can determine which mechanism is being used at the OS layer using a tool like strace (Linux) or truss (Solaris). These tools trace the actual system calls that are being made by the program.
At a guess, the call to create() stores the function pointer somewhere. Then, when the timer goes off, it calls the function you specified via that pointer. But as this is not a Standard C++ function, you should really read the docs or look at the source to find out for sure.
Regarding your other questions, I don't see mention of a priority list, and select() is a sort of general purpose event multiplexer.
Quite likely there's a framework that works with a typical main loop, the driving force of the main loop is the select call.
select allows you to wait for a filedescriptor to become readable or writable (or for an "exception" on the filedeescriptor) or for a timeout to occur. I'd guess the library also allow you to register callbacks for doing async IO, if it's a GUI library it'll get the low primitive GUI events via a file descriptor on unixes.
To implement timer callbacks in such a loop, you just keep a priority queue of timers and process them on select timeouts or filedescriptor events.
The priority means it processes the file i/o before the timers, which in itself takes time, could result in GUI updates eventually resulting in GUI event handlers being run, or other tasks spending time servicing I/O.
The library is more or less doing
for(;;) {
timeout = calculate_min_timeout();
ret = select(...,timeout); //wait for a timeout event or filedescriptor events
if(ret > 0) {
process_readable_descriptors();
process_writable_descriptors();
}
process_timer_queue(); //scan through a timer priority queue and invoke callbacks
}
Because of the fact that the thread id inside the timer callback is the same as the creator thread I think that it is implemented somehow using signals.
When a signal is sent to a thread that thread's state is saved and the signal handler is called which then calls the event call back.
So the handler is called in the creator thread which is interrupted until the signal handler returns.
Maybe another thread waits for all timers using select() and if a timer expires it sends a signal to the thread the expired timer was created in.
I'm programming an online game for two reasons, one to familiarize myself with server/client requests in a realtime environment (as opposed to something like a typical web browser, which is not realtime) and to actually get my hands wet in that area, so I can proceed to actually properly design one.
Anywho, I'm doing this in C++, and I've been using winsock to handle my basic, basic network tests. I obviously want to use a framelimiter and have 3D going and all of that at some point, and my main issue is that when I do a send() or receive(), the program kindly idles there and waits for a response. That would lead to maybe 8 fps on even the best internet connection.
So the obvious solution to me is to take the networking code out of the main process and start it up in its own thread. Ideally, I would call a "send" in my main process which would pass the networking thread a pointer to the message, and then periodically (every frame) check to see if the networking thread had received the reply, or timed out, or what have you. In a perfect world, I would actually have 2 or more networking threads running simultaneously, so that I could say run a chat window and do a background download of a piece of armor and still allow the player to run around all at once.
The bulk of my problem is that this is a new thing to me. I understand the concept of threading, but I can see some serious issues, like what happens if two threads try to read/write the same memory address at the same time, etc. I know that there are already methods in place to handle this sort of thing, so I'm looking for suggestions on the best way to implement something like this. Basically, I need thread A to be able to start a process in thread B by sending a chunk of data, poll thread B's status, and then receive the reply, also as a chunk of data., ideally without any major crashing going on. ^_^ I'll worry about what that data actually contains and how to handle dropped packets, etc later, I just need to get that happening first.
Thanks for any help/advice.
PS: Just thought about this, may make the question simpler. Is there a way to use the windows event handling system to my advantage? Like, would it be possible to have thread A initialize data somewhere, then trigger an event in thread B to have it pick up the data, and vice versa for thread B to tell thread A it was done? That would probably solve a lot of my problems, since I don't really need both threads to be able to work on the data at the same time, more of a baton pass really. I just don't know if this is possible between two different threads. (I know one thread can create its own messages for the event handler.)
The easiest thing
for you to do, would be to simply invoke the windows API QueueUserWorkItem. All you have to specify is the function that the thread will execute and the input passed to it. A thread pool will be automatically created for you and the jobs executed in it. New threads will be created as and when is required.
http://msdn.microsoft.com/en-us/library/ms684957(VS.85).aspx
More Control
You could have a more detailed control using another set of API's which can again manage the thread pool for you -
http://msdn.microsoft.com/en-us/library/ms686980(VS.85).aspx
Do it yourself
If you want to control all aspects of your thread creation and the pool management you would have to create the threads yourself, decide how they should end , how many to create etc (beginthreadex is the api you should be using to create threads. If you use MFC you should use AfxBeginThread function).
Send jobs to worker threads - Io completion Ports
In this case, you would also have to worry about how to communicate your jobs - i would recommend IoCOmpletionPorts to do that. It is the most scalable notification mechanism that i currently know of made for this purpose. It has the additional advantage that it is implemented in the kernel so you avoid all kinds of dead loack sitautions you would encounter if you decide to handroll something yourself.
This article will show you how with code samples -
http://blogs.msdn.com/larryosterman/archive/2004/03/29/101329.aspx
Communicate Back - Windows Messages
You could use windows messages to communicate the status back to your parent thread since it is doing the message wait anyway. use the PostMessage function to do this. (and check for errors)
ps : You could also allocate the data that needs to be sent out on a dedicated pointer and then the worker thread could take care of deleting it after sending it out. That way you avoid the return pointer traffic too.
BlodBath's suggestion of non-blocking sockets is potentially the right approach.
If you're trying to avoid using a multithreaded approach, then you could investigate the use of setting up overlapped I/O on your sockets. They will not block when you do a transmit or receive, but have the added bonus of giving you the option of waiting for multiple events within your single event loop. When your transmit has finished, you will receive an event. (see this for some details)
This is not incompatible with a multithreaded approach, so there's the option of changing your mind later. ;-)
On the design of your multithreaded app. the best thing to do is to work out all of the external activities that you want to be alerted to. For example, so far in your question you've listed network transmits, network receives, and user activity.
Depending on the number of concurrent connections you're going to be dealing with you'll probably find it conceptually simpler to have a thread per socket (assuming small numbers of sockets), where each thread is responsible for all of the processing for that socket.
Then you can implement some form of messaging system between your threads as RC suggested.
Arrange your system so that when a message is sent to a particular thread and event is also sent. Your threads can then be sent to sleep waiting for one of those events. (as well as any other stimulus - like socket events, user events etc.)
You're quite right that you need to be careful of situations where more than one thread is trying to access the same piece of memory. Mutexes and semaphores are the things to use there.
Also be aware of the limitations that your gui has when it comes to multithreading.
Some discussion on the subject can be found in this question.
But the abbreviated version is that most (and Windows is one of these) GUIs don't allow multiple threads to perform GUI operations simultaneously. To get around this problem you can make use of the message pump in your application, by sending custom messages to your gui thread to get it to perform gui operations.
I suggest looking into non-blocking sockets for the quick fix. Using non-blocking sockets send() and recv() do not block, and using the select() function you can get any waiting data every frame.
See it as a producer-consumer problem: when receiving, your network communication thread is the producer whereas the UI thread is the consumer. When sending, it's just the opposite. Implement a simple buffer class which gives you methods like push and pop (pop should be blocking for the network thread and non-blocking for the UI thread).
Rather than using the Windows event system, I would prefer something that is more portable, for example Boost condition variables.
I don't code games, but I've used a system similar to what pukku suggested. It lends nicely to doing things like having the buffer prioritize your messages to be processed if you have such a need.
I think of them as mailboxes per thread. You want to send a packet? Have the ProcessThread create a "thread message" with the payload to go on the wire and "send" it to the NetworkThread (i.e. push it on the NetworkThread's queue/mailbox and signal the condition variable of the NetworkThread so he'll wake up and pull it off). When the NetworkThread receives the response, package it up in a thread message and send it back to the ProcessThread in the same manner. Difference is the ProcessThread won't be blocked on a condition variable, just polling on mailbox.empty( ) when you want to check for the response.
You may want to push and pop directly, but a more convenient way for larger projects is to implement a toThreadName, fromThreadName scheme in a ThreadMsg base class, and a Post Office that threads register their Mailbox with. The PostOffice then has a send(ThreadMsg*); function that gets/pushes the messages to the appropriate Mailbox based on the to and from. Mailbox (the buffer/queue class) contains the ThreadMsg* = receiveMessage(), basically popping it off the underlying queue.
Depending on your needs, you could have ThreadMsg contain a virtual function process(..) that could be overridden accordingly in derived classes, or just have an ordinary ThreadMessage class with a to, from members and a getPayload( ) function to get back the raw data and deal with it directly in the ProcessThread.
Hope this helps.
Some topics you might be interested in:
mutex: A mutex allows you to lock access to specific resources for one thread only
semaphore: A way to determine how many users a certain resource still has (=how many threads are accessing it) and a way for threads to access a resource. A mutex is a special case of a semaphore.
critical section: a mutex-protected piece of code (street with only one lane) that can only be travelled by one thread at a time.
message queue: a way of distributing messages in a centralized queue
inter-process communication (IPC) - a way of threads and processes to communicate with each other through named pipes, shared memory and many other ways (it's more of a concept than a special technique)
All topics in bold print can be easily looked up on a search engine.
I have an ActiveX control written using the MS ATL library and I am firing events via pDispatch->Invoke(..., DISPATCH_METHOD). The control will be used by a .NET client and my question is this - is the firing of the event a synchronous or asynchronous call? My concern is that, if synchronous, the application that handles the event could cause performance issues unless it returns immediately.
It is synchronous from the point of view of the component generating the event. The control's thread of execution will call out into the receivers code and things are out of its control at that point.
Clients receiving the events must make sure they return quickly. If they need to do some significant amount of work then they should schedule this asynchronously. For example by posting a windows message, or using a separate thread.