We have a windows service in C++/ MFC which has to carry out a number of tasks on the host workstation some of which may be long running and may fail a few times before they are completed. Each task will only need to be completed once and sequentially.
I was of thinking of some form of callback initially to retry the failed task but each function has totally different parameters and the code has already been written and tested and just needs a re-queuing method.
I thought we could write the failed task to the registry, sleep() for a while and then restart the service. Is there a better approach?
TIA..
I'm doing quite the same thing in my professional project. My server component is getting runnable objects from different sources and execute them sequentially in a separated thread. All my runnable objects are using different parameters but they all have one function run(void* pUserParam). the void* parameters is a special object that contains a collection of field with different type (double, string, etc...).
My component is queuing the runnable object and launch a new one each time the thread is freed. Of course my component is sleeping when queue is empty and wake up when an object arrives. In your case when a task fail you just need to re-queue it and it will automatically retry the task later.
To achieve these you need:
a Pool mechanism that manage a queue
of tasks,
a task object that contains all information about the runnable object to launch and the parameters,
a runnable object that contains your action to execute.
How it works:
Your service is listening for demands,
When a demand arrives, it give it to the Pool mechanism,
The Pool take the runnable object and its parameter(s) and create a task. This task is queued,
(2b. If the queue was empty, the pool wakes up the execution thread,)
The Thread pick up one task from the queue and execute it calling the Run() function of the runnable object and passing to it the parameters previously stored in the task,
(3b. If the task failed, the thread re-queue a task with the runnable object and its parameter(s),)
The thread picks up a new task or sleeps if queue is empty.
This is my approach and I know this works fine. I know with this method you need to rewrite a part of your application but then the only thing to modify when adding a sort of task is to create a new runnable object (one sort of task => one runnable object that inherit from the abstract one).
Hope this will help you
Related
I am trying to understand just how to use semaphores in general. I have chosen a mutex type semaphore instead of a binary in order to maintain task priorities. The synopsis is:
I have two tasks, WebTask and DBTask.
WebTask is a simple task that takes (can you guess) web requests and does something with them.
DBTask is a (you got it) database task that runs on a messaging queue type system, so in order to work with it, you must send a message to its queue with n arguments.
Here is how things need to go down
WebTask receives an input and wants to get data from DBTask.
WebTask constructs a message for DBTask and sends it along the main messaging system with DBTask's id and all arguments
WebTask must wait for DBTask to
Receive the message
Talk to the database
Format the result
Without semaphores, here's what is happening
WebTask constructs and sends msg
WebTask continues on
DBTask gets msg
DBTask talks to database, gets data, etc
So obviously they are out of sync (should be 1,3,4,2)
This is just one thing I tried that I thought would work..
WebTask create a new mutex semaphore
WebTask constructs and sends a msg, while also telling DBTask about the semaphore ID
WebTask takes the semaphore
DBTask gets msg
DBTask processes msg
DBTasks gives back semaphore
This changed nothing.
What I don't understand at all (after many attempts).
Who should be in charge of the semaphore? Who should create it?
When is the semaphore taken?
When is the semaphore given?
(Notes: I am on VxWorks)
If you have to do your tasks in such a synchronous manner, it's easy enough.
The web task creates a semaphore at startup. The web task gets data, loads it into some request/response object instance, along with its semaphore reference/pointer. It fires it off to the queue and waits on the semaphore. The DB task gets the object, reads the request, processes it, loads the response field/s with results, (or error messages:), and signals the semaphore. The web task then runs on, knowing that the response is available.
That way, several web tasks can use the DB and it's input queue.
Forget the mutex stuff, you don't need any mutex, (well, there may well be one protecting the DB input producer-consumer queue but, presumably, that is an internal matter).
In WebTask, after sending the msg to DBTask, it waits on a semephore (semTake) while in DBTask, after processing the Database request and data formating, it release a semephore(semGive). I think that should make your 2 tasks synchronized.
I have a class that I'm using to manage creation of and destruction of threads that are responsible for sending and receiving CAN messages. I don't know if this is the best way to go about it, so I'm looking for advice on how to manage my threads for send messages and receive messages.
Basically I want spawnThread() to spawn a thread for the object passed to it.
so, something to the effect of
spawnThread(T obj)
{
std::thread (&T::obj, this);
}
My expectation was that I would use the Thread class to manage starting and ending the thread for two separate classes SendMessage and ReceiveMessage. Is there a better way to handle threading for sending and receive messages?
There is no such thing as a Manager pattern in OO. A thread either does a continuous job like waiting for connections or does a single shot job. The last type typically are worker-threads that are reused.
Now coming to the question. Threading does not scale. If you have send/receive tasks, process them in a fixed-size worker-thread pool. As a consequence, your application will be react slower if the workload extends your pool size as new request have to wait for a worker, but it will continue to work.
I am writing my first threaded application for an industrial machine that has a very fast line speed. I am using the MFC for the UI and once the user pushes the "Start" machine button, I need to be simultaneously executing three operations. I need to collect data, process it and output results very quickly as well as checking to see if the user has turned the machine "off". When I say very quickly, I expect the analyze portion of the execution to take the longest and it needs to happen in well under a second. I am mostly concerned about overhead elimination associated with threads. What is the fastest way to implement the loop below:
void Scanner(CString& m_StartStop) {
std::thread Collect(CollectData);
while (m_StartStop == "Start") {
Collect.join();
std::thread Analyze(AnalyzeData);
std::thread Collect(CollectData);
Analyze.join();
std::thread Send(SendData);
Send.join();
}
}
I realize this sample is likely way off base, but hopefully it gets the point across. Should I be creating three threads and suspending them instead of creating and joining them over and over? Also, I am a little unclear if the UI needs its own thread since the user needs to able to pause or stop the line at anytime.
In case anyone is wondering why this needs to be threaded as opposed to sequential, the answer is that the line speed of the machine will cause the need to be collecting data for the second part while the first part is being analyzed. Every 1 second equates to 3 ft of linear part movement down this machine.
Think about functionnal problem before thinking about implementation.
So we have a continuous flow of data that need to be collected, analyzed and sent elsewhere, with a supervision point to be able to stop of pause the process.
collection should be limited by the input flow
analyze should only be cpu limited
sending should be io bound
You just need to make sure that the slowest part must be collection.
That is a correct use case for threads. Implementation could use:
a pool of input buffers that would be filled by collect task and used by analyze task
one thread that continuously:
controls if it should exit (a dedicated variable)
takes an input object from the pool
fills it with data
passes it to analyze task
one thread that continuously
waits for the first of an input object from collect task and a request to exit
analyzes the object and prepares output
send the output
Optionnaly, you can have a separate thread for processing the output. In that case, the last lines becomes
passes an output object to the sending task
and we must add:
one thread that continuously
waits for the first of an output object from analze task and a request to exit
send the output
And you must provide a way to signal the request for pause or exit, either with a completely external program and a signalisation mechanism, or a GUI thread
Any threads you need should already be running, waiting for work. You should not create or join threads.
If job A has to finish before job B can start, the completion of job A should trigger the start of job B. That is, when the thread doing job A finished doing job A, it should either do job B itself or trigger the dispatch of job B. There shouldn't need to be some other thread that's waiting for job A to finish so that it can start job B.
I wonder if anyone familiar with a synchronization mechanism in user-mode, by which an app can register a "callback" function that would be called when another app signals it ... i don't mind the callback to be in an arbitraty thread.
Suppose i'm having lots of "Worker" processes in parallel, And one wants to notify them of a change (no payloaded data needed), by which every process will have to do some internal updates.
The immediate approach to this was to create another thread in each of them, and have an infinite loop that waits for a global event and call the callback function right afterwards. To signal this, one process would only need to signal this global event.
The problem is that i'll have lots of parallel processes in this project, i don't want to add thread*nProcesses to the system just to implement this, even if they're mostly paused.
The current "workaround" i found for this would be to hold my own "dummy" registry key, and every process will "register registery notification callback", when one app wants to notify the others it will just trigger a write to this key... and windows will callback every process which registered to this notification.
Any other ideas?
The nicer solution, which doesn't pollute the registry, would be to use a shared pipe. All workers can connect to the named pipe server, and do an async read. When the server wants to kick the workers, it just writes a byte. This triggers the completion routine of the worker. Basic example
Still, this notification has the same drawback as most other Windows notifications. If all of your worker threads are running worker code, there's no thread on which your notification can arrive - and you didn't create a special thread for that purpose either. The only solution around that is CreateRemoteThread, but that's a very big hammer.
thank you all for the useful ideas,
Eventually, I accidentally came across RegisterWaitForSingleObject which seems to do just that.
I'm still taking in account #MSalters comment about not having enough free worker threads at a given time since i'm assuming this callback mechanism relies on the same callback mechanism most Win32API does
I'm somehow stuck with implementing a reporting functionailty in my Log-Parser Application.
This is what I did so far:
I'm writing an Application that reads Logfiles and searches the strings for multiple regular Expressions that can be defined in a user-configuration file. For every so called "StringPipe"-defintion that is parsed from the configuration the Main-Process spawns a worker thread that will search for a single regex. The more definitons the user creates, the more worker threads are spawned. The Main Function reads a bunch of Logstrings and then sends the workers to process the strings and so on.
Now I want every single worker thread that is spawned to report information about the number of matches it has found, how long it took, what it did with those strings and so on. These Information are used to export as csv, write to DB and so on.
Now I'm stuck at the point where I created a Class "Report". This Class provides member functions that are called by the worker threads to make the Report-Class gather the Infos needed for generating the report.
For that my workers (which are boost::threads / functors) have to create a Report-Object which they can call those reporting functions for.
The problem is in my Design, because when a worker-thread finishes his job, it is destroyed and for the next bunch of strings that needs to be processed a new instance of this worker functor is spawned and so it needs to create a new Report Object.
This is a problem from my understanding, because I need some kind of container where every worker can store it's reported infos into and finally a global report that contains such infos as how long the whole processing has taken, which worker was slowest and so on.
I just need to collect all these infos together, but how can I do this? Everytime a worker stops, reports, and then starts again, it will destroy the Report-Object and it's members, so all the infos from previous work is gone.
How can I solve this problem or how is such a thing handled in general?
First, I would not spawn a new thread do the RE searching and such. Rather, you almost certainly want a pool of threads to handle the jobs as they arise.
As far as retrieving and processing the results go, it sounds like what you want are Futures. The basic idea is that you create an object to hold the result of the computation, and a Future to keep track of when the computation is complete. You can either wait for the results to be complete, or register a call-back to be called when a future is complete.
Instead of having the worker thread create the report object, why don't you have the main thread create the empty report and pass a pointer to the worker thread when created. Then the worker thread can report back when it has completed the report, then the main thread can add the data from that report to some main report.
So, the worker thread will never have ownership of the actual report, it fill just populate its data fields and report back to the main thread.