org.hd.d.pg2k.svrCore
Class ThreadUtils

java.lang.Object
  org.hd.d.pg2k.svrCore.ThreadUtils

public final class ThreadUtils
extends java.lang.Object

Utilities to assist with threading/concurrency.

PROCESSOR_COUNT_OVERRIDE_SYSPNAME

public static final java.lang.String PROCESSOR_COUNT_OVERRIDE_SYSPNAME

If this system property is set and parsable, then the system behaves as if the specified number of CPUs is available at start-up. This can be used to stress-test the system, or to avoid grabbing all the CPUs in a shared system, or when the JVM/system notion of CPU is not very suitable for us, eg with early HyperThreading.

Any value set for the property should be strictly positive when parsed as if by Integer.getInteger().

See Also:

Constant Field Values

FORCED_CPU_COUNT

private static final java.lang.Integer FORCED_CPU_COUNT

If non-null (ie the system property is set and parsable and we are allowed access), then the system behaves as if the specified number of CPUs is available at start-up.

AVAILABLE_PROCESSORS

public static final int AVAILABLE_PROCESSORS

Estimated number of CPUs available; strictly positive. Usually the JVM/system count of available processors when this class is initialised, but can be overridden manually.

MAX_LO_PRI_THREADS

private static final int MAX_LO_PRI_THREADS

Approximate maximum concurrent low-priority pool threads; strictly positive. We may refuse to queue some (discardable) threads when at least this many of our threads are running.

MAX_THREADS_NCTP

private static final int MAX_THREADS_NCTP

Maximum number of threads that may run in nonCPUThreadPool(); strictly positive. We limit the amount of threading by:

• The likely limit on back-end connectivity (eg HTTP-limited).
• The likely strain on other resources, such as disc and memory.

Observation suggests that up to 8 threads should be deployed to extract maximum throughput from a single magnetic disc spindle given I/O delays from seeking and from I/O bandwidth delays, so we should always deploy something like that (though solid-state storage is essentially seek-free with less latency), rising slowly with the number of CPUs because I/O capacity probably grows somewhat with CPU count.

Note also that we may place network-based work and other I/O blocked items in this pool.

This starts at 4 for uniprocessors.

_computeIntensiveThreadPool

private static final java.util.concurrent.ThreadPoolExecutor _computeIntensiveThreadPool

Shared non-reconfigurable thread pool for compute-bound activities. Suitable for almost-entirely CPU-bound threads (though we allow for a little parallel slackness), and so the pool size is bounded to a value roughly proportional to the number of CPUs available to this JVM (at creation of this pool).

Work cannot be queued for this pool (ie tasks are always executed immediately), and excess work (with no free pool threads available) will be run directly in the caller's thread.

This puts an upper bound on Thread resources (eg memory for stack) consumed by threads in this pool.

The threads in the pool are daemon threads, so should not prevent the JVM from exiting.

computeIntensiveThreadPool

public static final java.util.concurrent.ExecutorService computeIntensiveThreadPool

Shared non-reconfigurable thread pool for compute-bound activities. Suitable for almost-entirely CPU-bound threads (though we allow for a little parallel slackness), and so the pool size is bounded to a value roughly proportional to the number of CPUs available to this JVM (at creation of this pool).

By preference, this should be used for tasks that may block user interaction, or that really need to use every available CPU for some critical work; anything else should use a lower-priority pool if possible.

Work cannot be queued for this pool (ie tasks are always executed immediately), and excess work (with no free pool threads available) will be run directly in the caller's thread.

This puts an upper bound on Thread resources (eg memory for stack) consumed by threads in this pool.

The threads in the pool are daemon threads, so should not prevent the JVM from exiting.

_nonCPUThreadPool

private static final java.util.concurrent.ThreadPoolExecutor _nonCPUThreadPool

Shared thread pool for I/O-bound activities.

nonCPUThreadPool

public static final java.util.concurrent.ExecutorService nonCPUThreadPool

Shared non-reconfigurable thread pool for I/O-bound activities. Suitable for relatively short-lived and/or I/O-bound threads, thus we have a fixed thread/resource limit.

Work cannot be queued for this pool (ie tasks are always executed immediately), and excess work (with no free pool threads available) will be run directly in the caller's thread.

We always try to keep one or two worker threads alive for fast response.

The threads in the pool are daemon threads, so should not prevent the JVM from exiting.

_nonCPUThreadPoolDiscardable

private static final java.util.concurrent.ThreadPoolExecutor _nonCPUThreadPoolDiscardable

Shared non-reconfigurable thread pool for discardable low-priority I/O-bound activities. Suitable for non-critical relatively I/O-bound threads, thus we have a fixed thread/resource limit.

Work cannot be queued for this pool (ie tasks are always executed immediately or silently discarded).

The threads in the pool are daemon threads, so should not prevent the JVM from exiting.

nonCPUThreadPoolDiscardable

public static final java.util.concurrent.ExecutorService nonCPUThreadPoolDiscardable

Shared non-reconfigurable thread pool for discardable I/O-bound activities. Suitable for non-critical relatively I/O-bound threads, thus we have a fixed thread/resource limit.

Work cannot be queued for this pool (ie tasks are always executed immediately or silently discarded).

The threads in the pool are daemon threads, so should not prevent the JVM from exiting.

_lowPriorityThreadPool

private static final java.util.concurrent.ThreadPoolExecutor _lowPriorityThreadPool

Mutable implementation of thread pool for low-priority compute-bound activities.

lowPriorityThreadPool

public static final java.util.concurrent.ExecutorService lowPriorityThreadPool

Shared non-reconfigurable thread pool for low-priority compute-bound activities. Suitable for almost-entirely CPU-bound threads and so the pool size is bounded to a value roughly proportional to (less than) the number of CPUs available to this JVM (at creation of this pool).

Work cannot be queued for this pool (ie tasks are always executed immediately), and excess work (with no free pool threads available) will be run directly in the caller's thread.

These threads run with minimum priority.

Tasks submitted to this pool should not consume heavy resources (such as memory) other than CPU so that we can run many of them at once, though we try not to use more than half the CPUs at once when there are multiple CPUs available.

This puts an upper bound on Thread resources (eg memory for stack) consumed by threads in this pool.

The threads in the pool are daemon threads, so should not prevent the JVM from exiting.

_lowPriorityThreadPoolDiscardable

private static final java.util.concurrent.ThreadPoolExecutor _lowPriorityThreadPoolDiscardable

Mutable implementation of thread pool for discardable low-priority compute-bound activities.

lowPriorityThreadPoolDiscardable

public static final java.util.concurrent.ExecutorService lowPriorityThreadPoolDiscardable

Shared non-reconfigurable thread pool for discardable low-priority compute-bound activities. Most suitable for almost-entirely CPU-bound threads, and so the pool size is bounded to a value roughly proportional to (but less than) the number of CPUs available to this JVM (at creation of this pool).

This prefers to queue tasks and execute them on a small number of CPUs, only potentially increasing the pool size if the queue fills. (This will only queue tasks on a machine with a significant number of CPUs, where it could run more than one of these tasks at once, eg not on a uniprocessor.)

When this pool and its queue are full, or other CPU-bound pools appear to be using all available CPUs, then this silently discards new tasks submitted.

These threads run with minimum priority.

Tasks submitted to this pool should not generally consume heavy resources (such as memory) other than CPU so that we can safely run many of them at once. This pool tries not to use more than half the CPUs at once when there are multiple CPUs available.

A bounded (small) amount of work can be queued for this pool (ie tasks are usually executed immediately or should not wait long), and excess work (with no free pool threads or queue space available) will be silently discarded.

This puts an upper bound on Thread resources (eg memory for stack) consumed by threads in this pool.

The threads in the pool are daemon threads, so should not prevent the JVM from exiting.

CHECK_UPTIME

private static final boolean CHECK_UPTIME

If false, then it seems to be unsafe to try to check system load. Avoid uptime check on some early 1.6.0 pre-releases that crash...

_lastHeavilyLoadSampleTime

private static volatile long _lastHeavilyLoadSampleTime

Private flag for cpuHeavilyLoaded() to note time when we last sampled. Given that the underlying value is on the scale of a minute, we need only sample on a similar timescale, though to detect and respond quickly to overload a few seconds is probably best.

Initially zero.

Marked volatile for thread-safe lock-free access.

_lastHeavilyLoadSample

private static volatile boolean _lastHeavilyLoadSample

Private cache for cpuHeavilyLoaded() to note if system was heavily loaded at last sample. Initially false.

Marked volatile for thread-safe lock-free access.

ThreadUtils

private ThreadUtils()

Prevent creation of instances.

computeIntensiveThreadPoolSpace

public static int computeIntensiveThreadPoolSpace()

Returns approximate space (spare threads) currently available in the computeIntensivePool; non-negative.

nonCPUThreadPoolSpace

public static int nonCPUThreadPoolSpace()

Returns approximate space (spare threads) currently available in the nonCPUThreadPool; non-negative.

nonCPUThreadPoolDiscardableSpace

public static int nonCPUThreadPoolDiscardableSpace()

Returns approximate space (spare threads) currently available in the nonCPUThreadPoolDiscardable; non-negative.

lowPriorityThreadPoolSpace

public static int lowPriorityThreadPoolSpace()

Returns approximate space (spare threads) currently available in the lowPriorityThreadPool; non-negative.

_dontDiscardLowPriCPUBoundTask

private static boolean _dontDiscardLowPriCPUBoundTask()

If true then a low-priority CPU-bound task need not be discarded if the thread pool has room and the system is not too busy. Conversely, when this is false, discard such tasks even when the pool could take them.

Note that the test of system load may be too expensive.

lowPriorityThreadPoolDiscardableSpace

public static int lowPriorityThreadPoolDiscardableSpace()

Returns approximate space (spare threads) currently available in the lowPriorityThreadPoolDiscardable; non-negative.

couldRunLowPriorityDiscardableTask

public static boolean couldRunLowPriorityDiscardableTask()

Returns true if it may be possible to have a low-priority discardable task run immediately. This also indicates in general that the thread pools are not very busy and a true return can be regarded as an indication to that effect.

This returning true is no guarantee that a low-priority discardable task that is immediately submitted will be run, however, since other threads may attempt to (and manage to) start a task that would make this false.

_cpuIntensiveThreadsRunningNotOverLimit

private static boolean _cpuIntensiveThreadsRunningNotOverLimit(int limit)

Returns true if the count of (non-I/O-bound) threads running in our pools is no higher than the specified limit. This is only likely to be an approximation since there is no synchronisation, etc, and statistical parameters are being combined here.

Parameters:

limit - expected to be non-negative (but may be negative resulting in always returning false)

makeCompletedFuture

public static <T> java.util.concurrent.Future<T> makeCompletedFuture(T r)

Return a Future that already has its result computed.

loadFraction

public static double loadFraction()

Get load fraction in queued jobs per CPU over last minute (like UNIX load average, but normalised); -1 if unavailable otherwise non-negative. A value of 0 indicates an idle system in CPU terms, a value of 1 indicates fully-utilised CPUs, and a value greater than one indicates more work available than CPUs to run it.

Like the UNIX load average, but divided by the number of CPUs to normalise it.

cpuHeavilyLoaded

public static boolean cpuHeavilyLoaded()

Returns true if CPU load known to be heavily loaded over the last minute or so. May be cached for efficiency, eg to avoid expensive system calls and floating-point arithmetic.