I am using Qt SQL which is blocking API so I have to execute SQL code in Separate thread (QtConcurrent::run) and return (Q)future.
something like this:-
QFuture<QString> future = QtConcurrent::run( []() { /* some SQL code */ } );
auto watcher = new QFutureWatcher<QString>();
watcher.setFuture(future);
connect(watcher,&QFutureWatcher<QString>::finished,
[future](){ /* code to execute after future is finished */ });
But I learned that Threading is costly. every context switch is expensive. So it looks like CPU wastage to create new Thread just for waiting for result from MySQL server. My application is going to run on single core Virtual Machine on Google Cloud anyways . it there any way I can execute Qt SQL code asynchronusly without possibly creating new thread ?
I was also wondering how other APIs like Qt Networking implement asynchronus API without create new thread ? or i am wrong and they do create new thread under the hood ?
Many threaded applications run on a single core. Flushing cache to run on a separate core is also expensive. Use the right tool for the job. There's nothing wrong with threads.
That said, if you really want to run on a single thread use a workqueue to keep track of async task progress. The libevent library does this for you, but there are others. You just run a polling loop adding work onto the queue and executing callbacks when a task needs attention or completes.
By using QtConcurrent::run you already solved one problem - cost of creating thread because it use a thread pool.
When comes to context switches, first you could try to measure them with perf stat. And depends on situation, optimize it. If its just simple queries then probably vast majority of context switches comes from the system, not your app.
Doing something async means that you can execute task and move forward with your current code without waiting for results. But usually such task i.e sql query will spawn thread/process or will make request to OS.
Qt Networking make i.e read request and OS signals (epoll) when data will arrive. But in case of single core OS will interrupt your thread anyway.
If you have many many small queries you could try optimize them to make less queries, do caching.
Related
I have a single threaded asynchronous tcp server written using boost asio. Each incoming request will go through several processing steps (synchronous and asynchronous) and finally send back the response using async write.
For small loads with 10 concurrent requests, it works decently. However, when I test using a parallelism of 100, things start worsening. Response latency starts increasing as time progresses. So, I want to try with some multi-threaded processing for handling requests.
I am looking for a decent example / help on creating and running multiple threads for asynchronous reading/writing to clients. I have the following doubts:
Should I use a single IOS object and call its run method in all of the threads of the thread pool, or should I use a separate IOS per thread?
If I use a single IOS, is there a possibility that part of the tcp data goes to one thread, while another part going to another thread and so on.. Is this understanding correct?
Is there any other better way?
Thanks for any help and pointers here.
Without seeing your code I can only guess what goes wrong. Most probably you're running long actions inside async completion handlers. The completion handlers should be fast - get the data, hand it off for further processing, done.
As a first priority, I would go full-asynchronous and run all processing in a thread pool. You can find an example here, where a new thread is started for every new client, which you can replace with a thread pool.
Use a single io_service. A single io_service can handle a lot of parallelism, provided you don't delay it inside completion handlers. This simplifies the implementation because you don't have to worry about completion handlers running in parallel, which will happen if you run multiple IOS in multiple threads.
Q1: Should I use a single IOS object and call its run method in all of the threads of the thread pool, or should I use a separate IOS per thread?
Either you can
HTTP Server 2 - IOS per thread
HTTP Server 3 - single IOS with thread pool
Q2: If I use a single IOS, is there a possibility that part of the tcp data goes to one thread, while another part going to another thread and so on.. Is this understanding correct?
Yes, there is a race condition, but boost.asio support strand to avoid it.
Q3: Is there any other better way?
To me, not find a better way, if you find, tell me or past here, thank you.
BTW, as #rustyx said, your program is blocked at sync calls, turn to full-asynchronous calls will help.
I am creating application which uses sqlite to store some key value pairs.Now I dont want to block the main thread for performing sqlite operations.I have created separate thread for sqlite operations and created a queue for all operations. Main thread tell sqlite thread to do various oprerations. For each sqlite operation sqlite thread create a task and add to its queue .Main loop of sqlite thread takes task from queue and process it.
Now issue is that main thread cannot proceed until its get data from sqlite.So does it makes sense to have separate thread for sqlite operations?
Can I do this is some better way so that my main thread remains unblocked and it can get sqlite data also?
Yes you can. For example, you can have some function getResult() which will return concurrent container which you will use in your main thread. It allows to add new data as long as it is extracted from a DB and on the other side to get data as soon as it is available and do not wait until whole result is ready.
For concurrent containers you can try the following libraries: PPL, TBB or Boost.Lockfree.
Also you can use some event-driven programming by sending events from your work thread to your main thread. Boost.Signals could be of help in such a case.
One more thing: you can use asynchronous programming by using PPL::task, for example. You create task to get result from the DB and set continuation to handle this result. No block.
I believe there are other variants as well. So it is up to you to choose what is better suited for the task
As stated clearly in the documentation, Qt GUI must be accessed from main thread only. For complex app with multiple large and busy tables, this can be a bottleneck just from all the font-size text metrics calculations Qt likes to do. The only alternative I can think of is multi-tasking with separate processes. The tables are currently about as fast as you can get, custom model that is direct-mapped to a cache that is fed from another thread using dataChanged() calls on the most conservative set of changed cells. I've already profiled with vTune, 70% of the app time is now in Qt rendering code. Any suggestions?
I havn't used QT, but accessing GUI from only one thread (the GUI thread) is a known matter in almost any GUI I'm familiar with. I used 2 solutions for this case, of which I prefer the first one:
1) Your form will update the GUI (table, in this case) at timer intervals. The timer is activated on the GUI thread's events. At those timer events you read the data from global vars and update your table. The global vars can be updated by as many threads as you wish. You might need to synch (semaphores, for examples) the access to the global vars.
2) In many GUI APIs threads can update the GUI by handing the GUI thread a function (or an object) and ask it to execute it ASAP on its context. The calling thread meanwhile blocks, until the GUI fulfilled the action. I can recall three such functions - Invoke, InvokeLater from Java and C#, or wx.CallAfter of wxPython.
Use a variant of MVC pattern and make the model multithread
If your table entries and methods are done in several steps, you can call QCoreApplication::processEvents() to update the qt ui in between the calculations. Another thing you can do is run everything on different thread and emit signals from the thread when the calculations are complete. At the end, the updates are done on the ui from the main thread but asynchronously. To connect to a signal from a different thread you'll have to use qRegisterMetaType<>.
I'm designing a component to be run as a backend of a website. The component will take care of some AI logic, and I'm building it under C++. Would it be best if I let each session start a new EXE address space, or the EXE would be up and running and each session will start a new thread?
Or is there is any better suggestions alltogether?
I would be better to keep a process alive and create a new thread for each 'session': if you are looking for good performances under heavy load, starting a new process (fork, initialization of your app, etc.) will be really slow and can constitute a bottleneck.
Compared to that, creation of a new thread (in user space) is much lighter.
Even better, you can also keep the process running, and create a pool of threads. Then a 'manager' thread will process new connections, assign it to an existing thread and start it. In that case, you don't even need to create a new thread for each new connection. And if needed, the manager thread can adapt the number of existing threads to the load of your application.
Edit:
This can be useful: Apache MPM model
I'm having trouble keeping my app responsive to user actions. Therefore, I'd like to split message processing between multiple threads.
Can I simply create several threads, reading from the same message queue in all of them, and letting which ever one is able process each message?
If so, how can this be accomplished?
If not, can you suggest another way of resolving this problem?
You cannot have more than one thread which interacts with the message pump or any UI elements. That way lies madness.
If there are long processing tasks which can be farmed out to worker threads, you can do it that way, but you'll have to use another thread-safe queue to manage them.
If this were later in the future, I would say use the Asynchronous Agents APIs (plug for what I'm working on) in the yet to be released Visual Studio 2010 however what I would say given todays tools is to separate the work, specifically in your message passing pump you want to do as little work as possible to identify the message and pass it along to another thread which will process the work (hopefully there isn't Thread Local information that is needed). Passing it along to another thread means inserting it into a thread safe queue of some sort either locked or lock-free and then setting an event that other threads can watch to pull items from the queue (or just pull them directly). You can look at using a 'work stealing queue' with a thread pool for efficiency.
This will accomplish getting the work off the UI thread, to have the UI thread do additional work (like painting the results of that work) you need to generate a windows message to wake up the UI thread and check for the results, an easy way to do this is to have another 'work ready' queue of work objects to execute on the UI thread. imagine an queue that looks like this: threadsafe_queue<function<void(void)> basically you can check if it to see if it is non-empty on the UI thread, and if there are work items then you can execute them inline. You'll want the work objects to be as short lived as possible and preferably not do any blocking at all.
Another technique that can help if you are still seeing jerky movement responsiveness is to either ensure that you're thread callback isn't executing longer that 16ms and that you aren't taking any locks or doing any sort of I/O on the UI thread. There's a series of tools that can help identify these operations, the most freely available is the 'windows performance toolkit'.
Create the separate thread when processing the long operation i.e. keep it simple, the issue is with some code you are running that is taking too long, that's the code that should have a separate thread.