I want to run several concurrent jobs from QtScript script:
function job1() { ... }
function job2() { ... }
runConcurrentJobs(job1, job2)
The jobs a essentially sequences of remote procedure calls (ZeroC Ice), which need to synchronize at several points.
The Qt 4.8.0 documentation says nothing about QScriptEngine thread-safety. My questions:
Is it safe to use single QScriptEngine to exectute QtScript functions from multiple thread concurrently?
What approach would you recommend to accomplish the task?
Notes:
Scripts are edited not by programmers but also by electric engineers and I want to keep the script as simple and clean as possible.
QScriptEngine is documented as "reentrant", meaning, essentially, you can use it multi-threaded, but only one QScriptEngine per thread.
Now, if functions job1() and job2() can be run concurrently at all, in principle, it should be possible to separate them into two distinct QScriptEngines (easy, if neither the functions use local variables, only, more difficult, if globals are involved).
Implement runConcurrentJobs() as a Q_INVOKABLE function (or slot) in C++.
In there, do something like
void runConcurrently (const QString &functionname1, QString &functionname2) {
MyScriptThread thread1 (functionname1);
MyScriptThread thread2 (functionname2);
thread1.start();
thread2.start();
thread1.wait ();
thread2.wait ();
// optionally fetch return values from the threads and return them
}
Where MyScriptThread is derived from QThread and implements QThread::run() roughly like this:
void MyScriptThread::run () {
QScriptEngine engine;
engine.evaluate (common_script_code);
result = engine.evaluate (the_threads_function);
// the_threads_function passed as a QScriptProgram or QString
}
In general, if documentation says nothing about threading, it is not thread safe.
I would rewrite to use asynchronous requests. Just kick them both off, then wait for them both.
Related
I have this class inside a qt application that has some data defined:
class TableView:public QWidget
{
Q_OBJECT
public:TableView (QWidget * parent = 0);
std::vector < float > arr;
and some class functions and so on..
Now I want to add a socket server in another thread (posibly from inside this class) that is able to access data from this class when it recives something from the socket, calculate
new data and return.
For the test I used:
//defines
DWORD WINAPI SocketHandler(void* lp);
DWORD WINAPI starttremeshserver(void* lp);
CreateThread(0,0,&starttremeshserver, (void*)csock , 0,0);
This works. Now I need something easy and platform independent. Maybe something with qthread since I am working in qt.
So the question is, can I make a new thread function inside a class that can access class data. ty
Qt provides everything you need. Your needs are as follows:
Thread-safe (serialized) access to the data members of the TableView class.
Networking primitives.
Flexibility to move network access to a separate thread.
You are probably receiving data from the network, and want to update the arr member. You'd do it like so:
Create a QObject-derived class, say Processor, that has a slot that sets up the network connection (probably a QTcpServer). Upon a connection, use QTcpSocket to exchange data. Make sure all your data handling is done in slots in the class. When you have new values to update in the vector, simply emit some signal, like hasFloat(int,float).
Add a setFloat(int,float) slot in the TableView.
Connect the setFloat signal from the instance of Processor to your TableView.
At this point, everything runs in the GUI thread, but the code is non-blocking since you never wait for network data; you respond to the signals emitted by QTcpServer and QTcpSocket. You can leave it like that if you wish.
Having the Processor class run in a separate thread, if your benchmarking shows that your main thread is CPU bound, is then trivial:
int main(int argc, char** argv) {
bool separateThread = true;
QApplication app(argc, argv);
TableView view;
Processor proc;
connect(&proc, SIGNAL(hasFloat(int,float)), &view, SLOT(setFloat(int,float)));
QThread thread;
if (separateThread) {
thread.start();
proc.moveToThread(&thread);
}
view.show();
const bool rc = app.exec();
if (thread.isRunning()) {
thread.exit(); // tells the event loop in the thread to bail out
thread.wait(); // waits for the above to finish
}
return rc;
}
There's this misconception that spreading things across threads somehow magically makes them better. Threads are a solution to a particular problem: CPU-boundedness of the computations being done, and blocking APIs. If your processing is trivial, you are not likely to be CPU bound. Qt provides nonblocking, asynchronous networking. Thus, usually, spinning a second thread is entirely unnecessary.
You must show real numbers first to show you otherwise. Else you're buying into the cargo cult of threading: oh, it's networking, it must go into a separate thread. Nope, not necessarily. Measure first. Understand what you're doing.
The reason for thread safety in the above code is as follows: when you move the Processor instance to a different thread, Qt will reconnect all signal-slot connections using Qt::QueuedConnection type. Thus, when Processor emits hasFloat, it will internally cause an event to be queued in the event queue of the thread where TableView lives -- the GUI thread, in this case. When the event loop spins (here it'd be the application's event loop) -- it'll pick up the event and execute a call to TableView::setFloat. This ensures that the access to the arr data member is serialized and there are is no possibility of concurrent access from multiple threads.
I'd personally look for more high level socket support, something based on boost perhaps, anyway for threads you can use <thread> in C++11.
To answer your specific question:
class Foo
{
private:
void spinthread()
{
thread t([this] {
this->bar = 12;
});
}
private:
int bar;
}
If you're using Win32 threads API, the thread proc has a parameter which you can use to pass your instance to during the CreateThread call.
Qt has threading support as can be found here.
In addition to threads and accessing the data, you'll need synchronization for thread-safe code. This is called pthread mutex in linux, and in Qt its called QMutex as mentioned here.
You can find the Qt networking primitives here.
Using the Qt implementation of these threading and networking primitives will be portable, so should work for you on windows.
Regarding your question about creating a thread function from a class that accesses data on that class, the answer is yes. Instead of creating a thread function, it would be better to create a thread object, namely a QThread. When you create the thread object, you'll need to pass a reference to the instance of said class, which will allow the thread to access the class instance.
I'm programming in C++, but I'm only using pthread.h, no boost or C++11 threads.
So I'm trying to use threads but based on one of my previous questions (link), this doesn't seem feasible since threads terminate right after completion of its task, and one of the more prevalent reasons to use a thread-pool implementation is to reduce thread-creation overhead by reusing these threads for multiple tasks.
So is the only other way to implement this in C to use fork(), and create a pipe from the main to child processes? Or is there a way to set up a pipe between threads and their parent that I don't know about?
Many thanks in advance!
Yes, you can create a thread-safe queue between the threads. Then the threads in the pool will sit in a loop retrieving an item from the queue, executing whatever it needs, then going back and getting another.
That's generally a bit easier/simpler in C++ because it's a little easier to agree on some of the interface (e.g., overload operator() to execute the code for a task), but at a fundamental level you can do all the same things in C (e.g., each task struct you put in the queue will contain a pointer to a function to carry out the work for that task).
In your case, since you are using C++, it's probably easier to use an overload of operator() to do the work though. The rest of the task struct (or whatever you choose to call it) will contain any data needed, etc.
From the POSIX standard:
int pthread_create(pthread_t *restrict thread,
const pthread_attr_t *restrict attr,
void *(*start_routine)(void*), void *restrict arg);
(...) The thread is created executing start_routine with arg as its sole argument.
So, you should create a bunch of threads with this function, and have them all execute a function that goes something like
void *consumer(void *arg)
{
WorkQueue *queue = static_cast<WorkQueue *>(arg);
for (task in queue) {
if (task == STOP_WORKING)
break;
do work;
}
return WHATEVER;
}
(At the end of input, push n STOP_WORKING items to the queue where n is the number of threads.)
Mind you, pthreads is a very low-level API that offers very little type-safety (all data is passed as void pointers). If you're trying to parallelize CPU-intensive tasks, you might want to look at OpenMP instead.
'doesn't seem feasible since threads terminate right after completion of its task' what??
for(;;){
Task *myTask=theCommonProducerConsumerQueue->pop();
myTask->run();
}
.. never return anything, in fact, never return.
You may find it helpful to look at the source code for libdispatch, which is the basis for Apple's Grand Central Dispatch and uses thread pools.
I would suggest using Threaded Building Blocks from Intel to accomplish work-queue/threadpool like tasks. A fairly contrived example using TBB 3.0:
class PoorExampleTask : public tbb::task {
PoorExampleTask(int foo, tbb::concurrent_queue<float>& results)
: _bar(foo), _results(results)
{ }
tbb::task* execute() {
_results.push(pow(2.0, foo));
return NULL;
}
private:
int _bar;
tbb::concurrent_queue<float>& _results;
}
Used later on like so:
tbb::concurrent_queue<float> powers;
for (int ww = 0; ww < LotsOfWork; ++ww) {
PoorExampleTask* tt
= new (tbb::task::allocate_root()) PoorExampleTask(ww, powers);
tbb::task::enqueue(*tt);
}
http://people.clarkson.edu/~jmatthew/cs644.archive/cs644.fa2001/proj/locksmith/code/ExampleTest/threadpool.c
I used google a couple months ago, you should try it.
Edit: it seems maybe you want a group instead. I was able to create one with some minor alteration of the above so that the worker didn't perform work, but just joined threads.
I am using Qt and wish to write a class that will perform some network-type operations, similar to FTP/HTTP. The class needs to connect to lots of machines, one after the other but I need the applications UI to stay (relatively) responsive during this process, so the user can cancel the operation, exit the application, etc. My first thought was to use a separate thread for network stuff but the built-in Qt FTP/HTTP (and other) classes apparently avoid using threads and instead rely on signals and slots. So, I'd like to do something similar and was hoping I could do something like this:
class Foo : public QObject
{
Q_OBJECT
public:
void start();
signals:
void next();
private slots:
void nextJob();
};
void Foo::start()
{
...
connect(this, SIGNAL(next()), this, SLOT(nextJob()));
emit next();
}
void Foo::nextJob()
{
// Process next 'chunk'
if (workLeftToDo)
{
emit next();
}
}
void Bar::StartOperation()
{
Foo* foo = new Foo;
foo->start();
}
However, this doesn't work and UI freezes until all operations have completed. I was hoping that emitting signals wouldn't actually call the slots immediately but would somehow be queued up by Qt, allowing the main UI to still operate.
So what do I need to do in order to make this work? How does Qt achieve this with the multitude of built-in classes that appear to perform lengthy tasks on a single thread?
If you're doing a length job in the UI thread the UI is going to freeze. One way to avoid this is to call once in a while QCoreApplication::processEvents().
You should be VERY careful however to understand what this does before you decide to do it. Calling this function means that a GUI event can fire in the middle of your operation. If this event can in turn create some more jobs you can end up starting a new job while in the middle of the old job.
I wouldn't be so quick dismissing the worker thread approach. It has the advantage of completely separating the work from the GUI so you are certain that something that began is going to finish.
You should also consider that Windows especially sometimes introduces non trivial delays to GUI loops. If the host is somewhat busy or in a memory thrashing state you'll find that GUI events may take up to several long seconds to finish and return control to your processing.
Use QThread with a run method like this:
void run(){ exec(); }
this will provide another execution loop, where you can do your "hard work" without acutally freezing the UI.
Note: Make sure to actually use the thread's execution loop by adding
moveToThread(this);
at the end of the Constructor of your QThread derived Thread class (the doc doesn't say much about it)
I have gone through similar questions on Stackoverflow but still can't get a good answer:
how boost implements signals and slots
How signal and slots are implemented
I am quite puzzled on how this signal/slot is achieved.
Q1: From the following code, sig is connected to two function(Hello() and World()), and it seems that the functions are called in a serialized manner, which also implies that, one function(Hello()) need to be completed before going into another function(World())? => Single thread program
Q2: Are there anyway to enable multi-threaded signal/slot?(=> World() will start instantly, don't need to wait for Hello() to complete.) Or if it's not recommended, would you mind tell me why?
Sample codes on Boost website:
struct Hello
{
void operator()() const { std::cout << "Hello";}
};
struct World
{
void operator()() const { std::cout << ", World!" << std::endl;}
};
boost::signal<void ()> sig;
sig.connect(Hello());
sig.connect(World());
sig();
Output:
Hello, World!
Q1:
The calls are serialized. What signals are doing internally is, greatly simplified:
foreach connection:
call handler
Therefore you don't want to block in the handlers for long. If you need to do much work you can invoke it from there though, for example by creating a thread for it.
Q2:
boost signals 1 isn't even thread-safe; signals 2 is, but still does serialized calls. As signals are mostly used for event handling it is common style to not actually do much work in the handlers.
Thus there is no real benefit in calling them 'in parallel', the benefits would not in general justify the overhead of the neccessary thread invocations.
Q1: you are correct. Fixed my answer to the question you referenced to reflect that.
Q2: it seems you're confused by what should be threaded. In emitting/capturing process what contains code is the slot. So if you want to run the code concurrently, you should place the slots in different threads.
Such behavior is supported by Qt (don't know about boost, actually), and there's a chapter in qt manual that explains, that you most likely need "queued processing" for such behavior. But then you'll have to have the notion of "event loop" in the thread that executes the slot code (because you can't just tell the working thread "hey, stop doing your stuff, do this instead!").
If you don't want to wait, you'll have to spawn threads in slot codes directly. And you should not forget to use some kind of "wait" function in the code both slots have access to. By the way, both boost and Qt have nice wrappers around system threading libraries to do it easilly.
Follow up question to:
This question
As described in the linked question, we have an API that uses an event look that polls select() to handle user defined callbacks.
I have a class using this like such:
class example{
public:
example(){
Timer* theTimer1 = Timer::Event::create(timeInterval,&example::FunctionName);
Timer* theTimer2 = Timer::Event::create(timeInterval,&example::FunctionName);
start();
cout<<pthread_self()<<endl;
}
private:
void start(){
while(true){
if(condition)
FunctionName();
sleep(1);
}
}
void FunctionName(){
cout<<pthread_self()<<endl;
//Do stuff
}
};
The idea behind this is that you want FunctionName to be called both if the condition is true or when the timer is up. Not a complex concept. What I am wondering, is if FunctionName will be called both in the start() function and by the callback at the same time? This could cause some memory corruption for me, as they access a non-thread safe piece of shared memory.
My testing tells me that they do run in different threads (corruption only when I use the events), even though: cout<<pthread_self()<<endl; says they have the same thread id.
Can someone explains to me how these callbacks get forked off? What order do they get exectued? What thread do they run in? I assume they are running in the thread that does the select(), but then when do they get the same thread id?
The real answer would depend on the implementation of Timer, but if you're getting callbacks run from the same thread, it's most likely using signals or posix timers. Either way, select() isn't involved at all.
With signals and posix timers, there is very little you can do safely from the signal handler. Only certain specific signal safe calls, such as read() and write() (NOT fread() and fwrite(), or even new and cout) are allowed to be used. Typically what one will do is write() to a pipe or eventfd, then in another thread, or your main event loop running select(), notice this notification and handle it. This allows you to handle the signal in a safe manner.
Your code as written won't compile, much less run. Example::FunctionName needs to be static, and needs to take an object reference to be used as a callback function.
If the timers run in separate threads, it's possible for this function to be called by three different threads.