I'm thinking of how to property port a "ant-farm simulator" from to Erlang. Here's the basic rundown:
1) Define a 100x100 world of "slots"
2) Ants occupy one slot
3) The ant colony occupies location 50,50
4) Food is placed randomly around the map
5) Ants move one space at a time to seek food and bring it back to the colony
6) Only one object can be in a slot at a time.
The goal of this problem is to keep the system as concurrent as possible. In Clojure, the above problem is solved by having a thread pool of Agents that each run the AI for a single ant. Then these ants update the global state via a transaction.
It's that global state that I keep thinking about. How do we go about constructing the "game world"?
My first thought is to have a Erlang process for each ant, and then a process for each slot in the map. To move, a ant does the following:
1) The ant tells it's current slot "I want to move North"
2) The slot calls the slot to the north and says "Please update your contents to now contain ant "pid""
3) If the north slot already has an ant, it sends a "denied" response that trickles down to the slot containing the ant (and then to the ant). If the update works, then "granted" is sent down the chain and the ant updates its internal state.
The only thing I don't like about this method is that during a move process, the ant, its slot and the target slot are all "locked" until the whole transaction is completed. This then opens itself up for deadlocks. That is, two ants could be trying to swap places at the same time, each slot would be waiting for the other.
Can anyone suggest a better solution?
---EDIT----
Let me step through the deadlock problem:
1) Ant 1 asks slot A to "transfer north" to slot 2
2) Ant 2 asks slot B to "transfer south" to slot 1
3) Slot 1 sends a transfer request to slot 2 and waits for a reply
4) Slot 2 sends a transfer request to slot 1 and waits for a reply
From a code perspective this would be simple to implement but would also deadlock as each slot is only listening for replies from the other slot. I suppose the "correct way" might be to automatically reject all transfer requests while in the process of a transfer.
Have your ant make a cast to the slot he is moving to, asking for permission to move. The ant then waits for a cast response, telling it whether the move was successful or not. If the move was successful, the ant updates his own state to indicate that he is in the new slot. If it failed, he once again executes his search logic. If you end up with A and B trying to exchange slots, you won't have a deadlock, but they'll both think they have to look for other options.
If your grid is heavily occupied, you might want the slots to execute a polling logic, handing out permission to neighboring ants, telling them that if their logic leads them there, they may enter. (Imagine a grid with 50x50-2 ants on it, and you'll see why this would be a good change of logic.)
Never use calls unless you absolutely, positively, without question can't survive without them. And then, try very hard to get rid of them if there is a chance of processes of the same type calling each other, or of types that can make calls to each other.
I assume that when it fails to go north, it will try a different direction next? Where is the deadlock then?
I can imagine starvation if the field is pretty crowded, but not deadlocks.
Related
Hello I am looking for a signal for gtkmm. Basically I am doing some simulations and what I want is something like this :
I assume I do 5 simulations :
progressBar.set_fraction(0);
1 simulation
progressBar.set_fraction(progressBar.get_fraction()+1/5)
2 simulation
progressBar.set_fraction(progressBar.get_fraction()+1/5)
3 simulation
progressBar.set_fraction(progressBar.get_fraction()+1/5)
4 simulation
progressBar.set_fraction(progressBar.get_fraction()+1/5)
5 simulation
progressBar.set_fraction(progressBar.get_fraction()+1/5)
But I don't know which signal I have to use and how to translate to this.
Thank you a lot for your help !!!
The pseudo code which you presented in your question should actually work - no signal is necessary. However, you could introduce a signal into your simulation for update of the progress bar. IMHO this will not solve your problem and I will try to explain why and what to do to solve it:
You provided a little bit too less context, so, that I will introduce some more assumptions: You have a main window with a button or toolbar item or menu item (or even all of them) which start the simulation.
Let's imagine you set a breakpoint at Gtk::ProgressBar::set_fraction().
Once the debugger stopped at this break point you will find the following calls on the stack trace (probably with many other calls in between):
Gtk::Main::run()
the signal handler of the widget or action which started the simulation
the function which runs the five simulations
and last the call of Gtk::ProgressBar::set_fraction().
If you could inspect the internals of Gtk::ProgressBar you would notice that everything in Gtk::ProgressBar::set_fraction() is done properly. So what's wrong?
When you call Gtk::ProgressBar::set_fraction() it probably generates an expose event (i.e. adds an event to the event queue inside of Gtk::Main with a request for its own refresh). The problem is that you probably do not process the request until all five runs of the simulation are done. (Remember that Gtk::Main::run() which is responsible for this is the uppermost/outmost call of my imaginery stack trace.) Thus, the refresh does not happen until the simulation is over - that's too late. (Btw. the authors of Gtk+ stated somewhere in the manual about their cleverness to optimize events. I.e. there might be finally only one expose event for the Gtk::ProgressBar in the event queue but this does not make your situation better.)
Thus, after you called Gtk::ProgressBar::set_fraction() you must somehow flush the event queue before doing further progress with your simulation.
This sounds like leaving the simulation, leaving the calling widget signal handler, returning to Gtk::Main::run() for further event processing and finally coming back for next simulation step - terrible idea. But we did it much simpler. For this, we use essentially the following code (in gtkmm 2.4):
while (Gtk::Main::events_pending()) Gtk::Main::iteration(false);
(This should hopefully be the same in the gtkmm version you use but if in doubt consult the manual.)
It should be done immediately after updating the progress bar fraction and before simulation is continued.
This recursively enters (parts of) the main loop and processes all pending events in the event queue of Gtk::Main and thus, the progress bar is exposed before the simulation continues. You may be concerned to "recursively enter the main loop" but I read somewhere in the GTK+ manual that it is allowed (and reasonable to solve problems like this) and what to care about (i.e. to limit the number of recursions and to grant a proper "roll-back").
What in your case is the simulation we call in general long running functions. Because such long running functions are algorithms (in libraries for anything) which shall not be polluted with any GUI stuff, we built some administrational infra structure around this basic concept including
a progress "proxy" object with an update(double) method and a signal slot
a customized progress dialog which can connect a signal handler to such a progress object (i.e. its signal slot).
The long running function gets a progress object (as argument) and is responsible to call the Progress::update() method in appropriate intervals with an appropriate progress factor. (We simply use values in the range [0, 1].)
One issue is the interval of calling the progress update. If it is called to often the GUI will slow down your long running function significantly. The opposite case (calling it not often enough) results in less responsiveness of GUI. Thus, we decided for more often progress update. To lower the time consuming of GUI, we remember the time of last update in our progress dialog and skip the next refreshs until a certain duration since last refresh is measured. Thus, the long running function has still some extra effort for progress update but it is not recognizable anymore. (A good refresh interval is IMHO 0.1 s - the perception threshold of humans but you may choose 0.05 s if in doubt.)
Flushing all pending events results in processing of mouse events (and other GTK+ signals) also. This allows another useful feature: aborting the long running function.
When the "Cancel" button of our progress dialog is pressed it sets an internal flag. If the progress is updated next time it checks the flag. If the flag became true it throws a special exception. The throw aborts the caller of the progress update (the long running function) immediately. This exception must be catched in the signal handler of the button (or whatever called the long running function). Otherwise, it would "fall through" to the event dispatcher in Gtk::Main where it is catched definitely which would abort your application. (I saw it often enough whenever I forgot to catch.) On the other hand: catching the special exception tells clearly that the long running function has been aborted (in opposition to ended by regulary return). This may or may not be something which can be stated on GUI also.
Finally, the above solution can cause another issue: It enables to start the simulation (via GUI) while a simulation is already running. This is possible because button presses for simulation start could be processed while in progress update. To prevent this, there is actually a simple solution: set a flag at start of simulation in the GUI until it has finished and prevent further starts while the flag is set. Another option can be to make the widget/action insensitive when simulation is started. This topic becomes more complicated if you have multiple distinct long running functions in your application which may or may not exclude each other - leads to something like an exclusion matrix. Well, we solved it pragmatically... (but without the matrix).
And last but not least I want to mention that we use a similar concept for output of log views (e.g. visual logging of infos, warnings, and errors while anything long running is in progress). IMHO it is always good to provide some visual action for end users. Otherwise, they might get bored and use the telephone to complain about the (too) slow software which actually steals you the time to make it faster (a vicious cycle you have to break...)
I plan to hook up a raspberry pi to a 64x64 led matrix, and writing a sort-of boot loader software written in c++. I would like the software to be plug-and-play with custom games I make. That is, I put all the compiled code for the games into a directory and the boot loader will recognize them and, using fork() and execvp() calls, run the selected games. I want there to be different states to the loader such as: start_state, game_selection_state, preview_state, idle_state to name a few. Each state would do their own thing and a transition from state to state based on input from a input device or idle time.
Now I do not know the best way as far as the architecture to set this up. I am not even sure that a state machine is the best way to handle this. But so far, what I have come up with is I would have 2 threads.
Thread 1:
Handles all the functionality and sanity of the states. This would include starting, stopping, and data of each state. It would also include function calls to transition from state to state.
Thread 2:
Parses input from the input Device (game controller) and makes sure a transition is valid. Then using a thread 1's function calls change the state accordingly. I planned on using a message queue to send any input that is not reserved for transitioning to the current state. (E.g A & B signify transitioning, but Y is pressed and would be sent to current state) This way the state can do what ever it wants with these button presses.
Now where this gets a little fuzzy is when the user picks a game a fork() call will be made inside a thread and I have read this can cause issues. If I did do it this way I would need a way to terminate or halt thread 2 until execvp() call ends.
Now what I want to know is this a valid or best way of implementing something like this.
At the moment I am writing a turn based game for the iOS platform. The client is written in Objective-C with CocoaTouch, and the server is written in C++ for the Ubuntu Server OS. The server is connected to a MySQL database, in which it stores user & game data.
Right now I wish to implement a time-per-turn restriction, and this has to be done on the server side. When a user takes a turn, the next user will have a maximum of 24 hours to answer, otherwise I want the game to skip this user's turn and move on to the next player. I have some ideas about how to do this, but I am not sure if they are any good. What I've been thinking of is storing the date&time of the last turn taken as an entity related to the Game table on my SQL database. Then I'm thinking of launching a thread on the server which runs until termination, and looks up current time minus every game's last turn, say every minute or so. If it's been more than 24hrs since the last turn was taken, this thread allows the next player in the queue to take their turn, and skips the lazy player.
Does it sound over-complicated? Is there another, more simple way to do this? I know it's been done in many games before, I just don't know how. Thanks in advance!
I don't think you need any threads or background processes at all in this case.
What I see here is a simple algorithm:
When a user logs in to the game/match - check up the last turn ending time in the database,
If the elapsed time from the last turn ending time is greater than 24h, get the current time, substract the time from the database (obviously you need to convert both times into hours) and divide it by 24,
If the division yelds an odd number, it's the turn of the other player (player A)
If the division yelds an even number, it's the turn of the player B.
Set the database time to databaseTime+division*24
This algorithm can skip multiple turns. When player A finishes his move, and 48h passed, it's players B turn.
You probably just want a background process that has a schedule of "next actions" to take, a sort of priority queue you can work through as the events should be triggered.
A single process can handle a lot of independent games if you design the server properly. The architecture would pick up an event, load any associated data, dispatch accordingly, and then go back to waiting for new events.
C++ does have frameworks for this, but you could prototype it in NodeJS or Python's Twisted really quickly.
Please look at the reactor pattern (boost.asio, ACE). These frameworks are asynchronous, use an event-driven model and require no threads. Below is pseudo code on how you can solve it:
reactor.addTCPListener(acceptSock(), Handler::AcceptSock) // calls AcceptSock when accepting a new TCP connection
rector.addTCPListener(clientSock, Handler::ClientData) // calls ClientData when user is sending the server game stats (its move, status etc)
.
.
.
later on somewhere
.
.
.
for(set<Game>::Iterator it = games.begin(); it != games.end(); ++it) {
(it*)->checkTurn() // this call can be responsible for checking the timestamps from the ClientData function
}
Summary:
With the reactor pattern you will be able to have a non-blocking server that can do cleanup tasks when it is not handling IO. That cleanup can be comparing timestamps to switch/pass turns.
I have encountered the need to use multithreading in my windows form GUI application using C++. From my research on the topic it seems background worker threads are the way to go for my purposes. According to example code I have
System::Void backgroundWorker1_DoWork(System::Object^ sender, System::ComponentModel::DoWorkEventArgs^ e)
{
BackgroundWorker^ worker = dynamic_cast<BackgroundWorker^>(sender);
e->Result = SomeCPUHungryFunction( safe_cast<Int32>(e->Argument), worker, e );
}
However there are a few things I need to get straight and figure out
Will a background worker thread make my multithreading life easier?
Why do I need e->Result?
What are the arguments passed into the backgroundWorker1_DoWork function for?
What is the purpose of the parameter safe_cast(e->Argument)?
What things should I do in my CPUHungryFunction()?
What if my CPUHungryFunction() has a while loop that loops indefinitely?
Do I have control over the processor time my worker thread gets?
Can more specifically control the number of times the loop loops within a set period? I don’t want to be using up cpu looping 1000s of times a second when I only need to loop 30 times a second.
*Is it necessary to control the rate at which the GUI is updated?
Will a background worker thread make my multithreading life easier?
Yes, very much so. It helps you deal with the fact that you cannot update the UI from a worker thread. Particularly the ProgressChanged event lets you show progress and the RunWorkerCompleted event lets you use the results of the worker thread to update the UI without you having to deal with the cross-threading problem.
Why do I need e->Result?
To pass back the result of the work you did to the UI thread. You get the value back in your RunWorkerCompleted event handler, e->Result property. From which you then update the UI with the result.
What are the arguments passed into the function for?
To tell the worker thread what to do, it is optional. Otherwise identical to passing arguments to any method, just more awkward since you don't get to chose the arguments. You typically pass some kind of value from your UI for example, use a little helper class if you need to pass more than one. Always favor this over trying to obtain UI values in the worker, that's very troublesome.
What things should I do in my CPUHungryFunction()?
Burn CPU cycles of course. Or in general do something that takes a long time, like a dbase query. Which doesn't burn CPU cycles but takes too long to allow the UI thread to go dead while waiting for the result. Roughly, whenever you need to do something that takes more than a second then you should execute it on a worker thread instead of the UI thread.
What if my CPUHungryFunction() has a while loop that loops indefinitely?
Then your worker never completes and never produces a result. This may be useful but it isn't common. You would not typically use a BGW for this, just a regular Thread that has its IsBackground property set to true.
Do I have control over the processor time my worker thread gets?
You have some by artificially slowing it down by calling Thread.Sleep(). This is not a common thing to do, the point of starting a worker thread is to do work. A thread that sleeps is using an expensive resource in a non-productive way.
Can more specifically control the number of times the loop loops within a set period? I don’t want to be using up cpu looping 1000s of times a second when I only need to loop 30 times a second.
Same as above, you'd have to sleep. Do so by executing the loop 30 times and then sleep for a second.
Is it necessary to control the rate at which the GUI is updated?
Yes, that's very important. ReportProgress() can be a fire-hose, generating many thousands of UI updates per second. You can easily get into a problem with this when the UI thread just can't keep up with that rate. You'll notice, the UI thread stops taking care of its regular duties, like painting the UI and responding to input. Because it keeps having to deal with another invoke request to run the ProgressChanged event handler. The side-effect is that the UI looks frozen, you've got the exact problem back you were trying to solve with a worker. It isn't actually frozen, it just looks that way, it is still running the event handler. But your user won't see the difference.
The one thing to keep in mind is that ReportProgress() only needs to keep human eyes happy. Which cannot see updates that happen more frequently than 20 times per second. Beyond that, it just turns into an unreadable blur. So don't waste time on UI updates that just are not useful anyway. You'll automatically also avoid the fire-hose problem. Tuning the update rate is something you have to program, it isn't built into BGW.
I will try to answer you question by question
Yes
DoWork is a void method (and need to be so). Also DoWork executes
in a different thread from the calling one, so you need to have a
way to return something to the calling thread. The e->Result
parameter will be passed to the RunWorkerCompleted event inside
the RunWorkerCompletedEventArgs
The sender argument is the backgroundworker itself that you can use
to raise events for the UI thread, the DoWorkEventArgs eventually
contains parameters passed from the calling thread (the one who has
called RunWorkerAsync(Object))
Whatever you have need to do. Paying attention to the userinterface
elements that are not accessible from the DoWork thread. Usually, one
calculate the percentage of work done and update the UI (a progress
bar or something alike) and call ReportProgress to communicate with
the UI thread. (Need to have WorkerReportProgress property set to
True)
Nothing runs indefinitely. You can always unplug the cord.
Seriously, it is just another thread, the OS takes care of it and
destroys everything when your app ends.
Not sure what do you mean with this, but it is probably related
to the next question
You can use the Thread.Sleep or Thread.Join methods to release the
CPU time after one loop. The exact timing to sleep should be fine
tuned depending on what you are doing, the workload of the current
system and the raw speed of your processor
Please refer to MSDN docs on BackgroundWorker and Thread classes
I have a situation where I have a single Emitter object and a set of Receivers. The receivers are of the same class, and actually represent a set of devices of the same type. I'm using the Qt framework.
The Emitter itself first gets a signal asking for information from one of the devices.
In the corresponding slot, the Emitter has to check to see which of the Receivers are 'ready', and then send its own signal to request data to one of the devices (whichever is ready first).
The Emitter receives signals very quickly, on the order of milliseconds. There are three ways I can think of safely requesting data from only one of the devices (the devices live in their own threads, so I need a thread-safe mechanism). The number of devices isn't static, and can change. The total number of devices is quite small (definitely under 5-6).
1) Connect to all the devices when they are added or removed. Emit the one request and have the devices objects themselves filter out whether the request is for them using some specific device tag. This method is nice because the request slot where the check occurs will execute in a dedicated thread's context, but wasteful as the number of devices go up.
2) Connect and disconnect from the object within the Emitter on the fly when it's necessary to send a request.
3) Use QMetaObject::invokeMethod() when its necessary to send a request.
Performance is important. Does anyone know which method is the 'best', or if there's a better one altogether?
Regards
Pris
Note: To clarify: Emitter gets a signal from the application, to get info by querying the device. Crazy ASCII art go:
(app)<---->(emitter)<------>(receivers)<--|-->physical devices
Based on the information you have provided I would still recommend a Reactor implementation. If you don't use ACE then you can implement your own. The basic architecture is as follows:
use select to wake up when signal or data is received from the App.
If there is a socket ready on the sending list then you just pick one and send it data
When data is sent the Receiver removes itself from the set of sockets/handlers that are available
When data is processed the Reciever re-registers itself to the list of available recipients.
The reason I suggested ACE is because it has one of the simplest to use implementations of the Reactor pattern.
I'm amusing here this is multi thread environment.
If you are restricted to Qt signal / slot system between then the answer for your specific questions:
1) is definitely not the way to go. On an emit from the Emitter a total number of events equal to the number of Receivers will be queued for the thread(s) event loops of the devices, then the same number of slot calls will occur once the thread(s) reach those event. Even if most of the lost just if(id!=m_id) return; on their first line, its a significant amount of things going on in the core of Qt. Place a breakpoint in one of your slots that is evoked by a Qt::QueuedConnection signal and validate this looking at the actual stack trace. Its usually at least 4 call deep from the xyEventLoop::processEvents(...), so "just returning" is definitely not "free" in terms of time.
2) Not sure how Qt's inner implementation actually is, but from what I know connecting and disconnecting most likely include inserting and removing the sender and receiver into some lists, which are most likely accessed with QMutex locking. - might also be "expensive" time-wise, and rapidly connecting and disconnecting is definitely not a best practice.
3) Probably the least "expensive time-wise" solution you can find that is still using Qt's singnal-slot system.
optionally) Take a look at QSignalMapper. It is designed exactly for what you planned to do in option 1).
There are more optimal solutions to communicate between your Emitter and Receivers, but as a best practice I'd first choose the option that is most easy to use and fast to implement, yet has a chance of being fast enough run-time (that is option 3). ). Then, when its done, see if it meets your performance requirements. If it does not, and only then, consider using shared memory with mutexes in a data provider - data consumer architecture (Emitter thread rapidly post request data in a circular list, while the Receiver thread(s) reads them whenever have time, then post results back a similar way, while the Emitter thread constantly polls for done results.)