This is my understanding how things are supposed to work . Warning: the last time I dealt intensively with such questions was 6 years ago, so I can't guarantee to get everything right.
POSIX differentiates between threads subject to process contention (PCS) and system contention scope (SCS). PCS threads corresponds to the user level threads that are scheduled by the thread library, whereas SCS threads to kernel level threads that are scheduled by the OS (I think, Solaris calls SCS threads LWP).
We may set different scheduling policy and priority to these PCS; the scheduling shall be always relative to the other PCS threads within the process. We may for instance have 3 PCS threads; thread 1 with SCHED_FIFO policy and priority 1, thread 2 with priority 2, and thread 3 with the default time sharing policy (SCHED_OTHER). Assume further that these 3 PCS threads are mapped onto 1 SCS thread. When this SCS thread gets scheduled by the OS, the thread library shall look which PCS thread should run: thread 2 shall be scheduled if runnable, otherwise thread 1, otherwise thread 3.
Now, how does nice() affects the whole thing? First nice() only operates at the process level, and doesn't really make sense for real-time policy. We expect therefore nice() to affect only SCS threads that are not subject to real-time policy. Indeed, POSIX states:
Quote:
[PS|TPS]
Calling the
nice() function has no effect on the priority of processes or threads with policy SCHED_FIFO or SCHED_RR. The effect on processes or threads with other scheduling policies is implementation-defined.
The nice value set with
nice() shall be applied to the process. If the process is multi-threaded, the nice value shall affect all system scope threads in the process.
Back to our previous example. If I renice my process, and if my process is not subject to real-time policy, the corresponding SCS shall be scheduled less frequently (assuming I increased the nice level) compared to other SCS running on the system. So will the 3 PCS threads. From a system perspective, we can say that all 3 PCS have been impacted simultaneously by this renice operation. From the perspective of our PCS threads however, nothing has changed since the scheduling is always relative to other PCS threads (except perhaps that it feels like running on a slower CPU).
HTH, Loïc