Working with threads in Qt5, how can I set a single thread's CPU affinity?
I want to specify a mask of available CPU cores that the thread may run under.
In other words, what is the Qt5 equivalent to Posix thread's pthread_setaffinity_np() ?
Can I do this for threads managed by QThreadPool?
Usually that sort of things is done by extracting the native thread handle and then doing whatever system specific stuff necessary, as no accepted cross-platform API exists for low level thread management.
Indeed, if we inspect the source for the qthread_unix.cpp we will see the following:
Qt::HANDLE QThread::currentThreadId() Q_DECL_NOTHROW
{
// requires a C cast here otherwise we run into trouble on AIX
return to_HANDLE(pthread_self());
}
And in qthread_win.cpp the implementation will differ in the expected way:
Qt::HANDLE QThread::currentThreadId() Q_DECL_NOTHROW
{
return reinterpret_cast<Qt::HANDLE>(quintptr(GetCurrentThreadId()));
}
So, it is responsibility of the application code to do the proper low level actions pertaining to each platform it is expected to run on.
Qt doesn't provide any public APIs for setting processor affinity. There's definitely no way to do this for a QThreadPool, which doesn't even provide an API for accessing the QThreads within the pool.
For explicitly-created QThread objects, you could try using the QThread::getCurrentThreadId() function to return a native "handle" to the thread, and then pass that to your system's thread-management library calls. But I would strongly recommend against this. The docs explicitly state:
Warning: The handle returned by this function is used for internal purposes and should not be used in any application code.
If you're building for a specific platform, you might be able to mix calls to your OS's library (e.g., pthreads), but that will not be portable and I have no idea if it will work.
Your best bet is probably to manage the CPU affinity of the whole application from the command-line. If you're on Linux, taskset is the way to go. For Windows, take a look at this SO answer. Unfortunately, Apple seems pretty determined to prevent users from setting thread affinity, at least from the command line.
Related
For example I have code
while (!something} {
//waiting
}
it does wait for something, but it uses a lot of CPU. C++ have things like thread join, condition variable wait, mutex lock - which allow to wait, so it does check some condition, but it behaves like idle process - not consuming CPU time. How it is done and is there way to make while loop (or any other code) behave like this?
These are features necessarily backed by the operating system.
The OS is responsible for allocating time to your processes and threads, and you require these features to control (or, rather, make requests of) that mechanism.
Your C++ standard library implementation calls platform-specific functions provided by your operating system.
There is no way to replicate that yourself without using the kind of C++ types/functions you've already listed: mutexes & timers (or without calling those OS functions yourself, like we did in the olden days!). All you can do is a spin lock, like the one you already demonstrated.
You can achieve this by calling system calls that block the execution of the thread.
You don't usually call the system directly, but use a wrapper function instead, which is an abstraction over system specific details. This approach allows your program to be portable to different systems.
In fact, this is exactly what the standard functions such as std::condition_variable::wait are: an abstraction over system interface, which in this case blocks the thread. std::cin::operator>> is another example of function call that blocks the execution of the thread.
is there any way to set Processor affinity for ITK (3.20.1) threads. I was looking into ITK::MultiThreader class which has "SetGlobalDefaultNumberOfThreads". but i am not finding any function/method which is saying anything about affinity.
any help would be appreciated..
In the version that I have installed (3.18), it seems that this feature is not supported. However, looking at the MultiThreader header file, I noticed that the implementation relies on the native OS thread mechanisms. This means that on linux, pthreads are actually spawned, and pthread_t are directly used for ThreadProcessIdType, which means that you might be able to use the linux proprietary extensions for core affinities.
Quoting myself from this answer to another question:
Unfortunately, the posix thread API doesn't provide a way to set cpu affinity for threads. You may use the non portable extension provided on the linux platform pthread_attr_setaffinity_np, with the cpuset family of functions to configure a thread affinity.
Since the thread will be already running, you will need to get the thread attributes before being able to set its affinity. Linux provides one more non portable function for that: pthread_getattr_np.
Also, the cpuset feature must be built in the kernel, however this should be the case on most platforms with multicore support.
references:
cpuset
pthread_attr_setaffinity_np
pthread_getattr_np
I know that you can't technically chroot a thread, but I'm looking for functional alternatives that would limit the amount of permissions to an existing set of tasks within a thread.
Edit: This is for a Linux environment, however Windows related techniques would be useful as well (even if they have no *NIX equivalent.
There are few differences between threads and processes on Linux. Use the clone syscall with the flag CLONE_THREAD and without the flag CLONE_FS, so that the new thread does not share its root directory/current directory/umask with the calling thread. Alternatively, you can use the unshare syscall with CLONE_FS after the fact. Now you can use chroot as normal and it will not affect other threads in the process.
Setting permissions for a thread in windows is trivial using the impersonation functions in the Win32 API.
I'm not an expert in GNU/Linux system programming so there may be extensions that allow modification of the per thread UID, but as far as I know Unix systems have per process security not per thread as in Windows.
This is because historically Unix has used processes for parallel processing (fork) while Windows uses threads (CreateThread), and (historically?) creating new processes in Unix is comparable in speed to creating new threads in Windows.
I'm giving your question +1 since a cursory Google search seems to indicate the preferred way to restrict permissions is to fork the different tasks to processes by security and use setuid to set the permissions and optionally then chroot them.
I'd like to hear whether there are other options since, as mentioned above, this use case is rather trivial to implement in windows.
I'm writing portable code for multicore machines and I want kernel level threads so the threads can use more than one cpu. After reading QThread documentation on Qt Assistant I still haven't found any hints.
On Windows XP the multithreading example (mandelbrot) from the QtSDK used only one core. So I guess on XP only user level threads are possible. I haven't tested that on Linux or OSX so far since there isn't the full SDK installed.
EDIT: The example given in the SDK is stupid - it only uses one thread for those calculation so the binding to only one core was misleading. Buildig a sample myself I could use all cores, so on XP with mingw/GCC Qt uses kernel level threads.
So, what kind of threads are used by QThread? Is it possible to specify what kind of thread to use?
Multiple processes are also an option in combination with shared memory.
Edit
http://doc.qt.io/qt-4.8/thread-basics.html gives a nice introduction.
I don't know about Windows, but on Unix it is using pthreads. QT isn't exposing API for CPU affinity because it needs to be platform- and hardware-independent. The QThread distribution across CPUs is left to the OS scheduler, you can't hint it via some QT API.
From QThread Class Reference:
A QThread represents a separate thread of control within the program; it shares data with all the other threads within the process but executes independently in the way that a separate program does on a multitasking operating system.
In your terms, it's a "kernel" thread.
Also, the conclusion that "only user-level threads are possible" on Windows XP is surely incorrect.
I have a program that I would like to run on just one CPU so it doesn't take up too much system resources. The problem is, it makes a call into an external DLL that automatically uses all available CPU cores. I do not have the source code to the external DLL. How can I limit the DLL to only using one CPU?
EDIT: Thanks for the help, here is the code I used to limit to one CPU (Windows):
// Limit the process to only 1 thread so we don't chew up system resources
HANDLE ProcessHandle = GetCurrentProcess();
DWORD ProcessAffinityMask;
DWORD SystemAffinityMask;
if(GetProcessAffinityMask(ProcessHandle,&ProcessAffinityMask,&SystemAffinityMask)
&& SystemAffinityMask != 0)
{
// Limit to 1 thread by masking all but 1 bit of the system affinity mask
DWORD NewProcessAffinityMask = ((SystemAffinityMask-1) ^ SystemAffinityMask) & SystemAffinityMask;
SetProcessAffinityMask(ProcessHandle,NewProcessAffinityMask);
}
EDIT: Turns out Brannon's approach of setting process priority works even better for what I want, which is to keep the process from chewing up resources. Here's that code (Windows):
// Make the process low priority so we don't chew up system resources
HANDLE ProcessHandle = GetCurrentProcess();
SetPriorityClass(ProcessHandle,BELOW_NORMAL_PRIORITY_CLASS);
You could set the CPU affinity of your program. Try the SetProcessAffinityMask function on Windows or sched_setaffinity on Linux.
Setting processor affinity is the wrong approach. Let the OS handle scheduling.
If the machine is sitting idle, you want to use as much processor as you can. Otherwise you're doing less work for no reason. If the machine is busy, then you want to make use of "free" cycles and not adversely affect other processes.
Windows has this functionality built-in. The proper solution for this is to set the base priority of the process.
See http://msdn.microsoft.com/en-us/library/ms686219(VS.85).aspx for details on SetPriorityClass().
If you want to test this without writing any code, use Task Manager to change the priority of your process.
Normally, a dll lives in the same thread/memory space as the code that calls it. The act of calling a DLL itself should not create threads. If calling the DLL seems to create more threads, that means that the DLL itself is creating the threads somewhere in it's code. If you don't have any source code or documentation for the DLL, there isn't much you can do about it (and if you want the DLL to do its job, there isn't much you should do about this).
You might try playing with the priority of your application - setting it to low might change the CPU usage even if it doesn't change what threads are created. But it seems likely that what you will really want is to get documentation for this beast. Without knowing how code works, in general, there isn't much you can do to change how it works. No super genius can change that.
Er...... why? Seriously, why would you limit a library that's capable of giving you additional performance in such a way? Are you trying to access a shared resource or something? One would think that a multi-threaded library would be capable of handling this safely.
Unless there's something you're not mentioning, I can't see any valid reason for even trying to limit a multi-threaded library to a single thread.
So your program uses one thread, but you don't want the external DLL to use more than one thread? You don't have much control over what the external DLL does, but some approaches might be:
Use the "Half-Sync/Half-Async: An Architectural Pattern for Efficient and Well-structured Concurrent I/O" design pattern to queue work items from your one thread to the external DLL.
Or since the other DLL will be loaded in your process, you might be able to set your process or threads' processor affinity to just one CPU. See Raymond Chen's "Psychic debugging: Why your expensive four-processor machine is ignoring three of its processors".
You didn't say what platform this is for. I'm going to assume windows here.
Create the child process and associate it with a Job Object. You can then set the processor affinity for that job object to include only one of the available CPU cores. The child process is not able to change the affinity mask to anything that is not a subset of the processor affinity for the job object. Additionally, you should remember not to set the limit JOB_OBJECT_LIMIT_BREAKAWAY_OK or the extended limit JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK on the job or else the child process will be able to break out of the job.
Additionally, you can set the priority and scheduling class for the job. Perhaps it will be sufficient to create the child process at a lower CPU and/or IO priority level?