org.hd.d.pg2k.svrCore
Class MemoryTools

java.lang.Object
  org.hd.d.pg2k.svrCore.MemoryTools

public final class MemoryTools
extends java.lang.Object

This contains utilities for memory intensive/sensitive operations. For example, this can veto or partly serialise memory-intensive operations such as image resizing in order to prevent the JVM running out of memory in a resource-constrained environment such as a Web server.

_iMS_SR

private static volatile java.lang.ref.SoftReference<java.lang.Object> _iMS_SR

Used by (and private to) isMemoryStressed() to detect memory stress; never null. We use clearing of this reference as an indication of possible memory stress, or at least of fast memory turnover wrt the rate of poll of this value.

Marked volatile for thread-safe lock-free access.

_iMS_lastCheck

private static volatile long _iMS_lastCheck

Time of last check of _iMS_SR; the least significant bit indicates state detected. If the last bit is 1 it indicates that stress was detected, else none was.

Marked volatile for thread-safe lock-free access.

The initial state indicates that stress was not detected.

_IMS_CHECK_MS

private static final int _IMS_CHECK_MS

Minimum interval for check/update of _iMS_SR in ms; strictly positive. If this is too small then the LRU mechanism that should be built into SoftReferences may mask all but the most severe shortage just before an OOM would have been thrown if isMemoryStressed() is being called very frequently.

We probably want to be able to detect milder stress than this, so a poll on the order of seconds is probably reasonable, but we try to avoid making this an obvious (sub-)multiple of any regular activity.

See Also:

Constant Field Values

_tMU_registered

private static final java.util.Collection<java.lang.ref.WeakReference<MemoryTools.Compactable>> _tMU_registered

Thread-safe Collection of registered Compactable instances to containers trimmable at run-time without change to their logical state; never null (except possibly during initialisation). All Compactable instances are held via WeakReference wrappers to avoid preventing their GC.

The Collection implementation is unbounded, should make insertion/deletion O(1), and must allow safe/sensible concurrent use of iterator() with other uses/modification and without throwing ConcurrentModificationException so that we can avoid any locking/blocking.

This is intended only to hold a smallish number of potentially-large collections/containers.

INCR_HOUSEKEEP_EFQ

private static final boolean INCR_HOUSEKEEP_EFQ

If true then attempt to incrementally clear the _emergencyFreeQueue in the background. The aim is to prevent the _emergencyFreeQueue itself retaining memory uselessly when many items in it are dead.

See Also:

Constant Field Values

_emergencyFreeQueue

private static final java.util.Queue<java.lang.ref.WeakReference<java.lang.Runnable>> _emergencyFreeQueue

Queue of one-shot entries that can be called in an emergency to free memory; never null. Entries are held by weak reference to ensure that they don't prevent GC.

Whenever we detect that memory is under great stress we work through this queue in order calling the target (unless the WeakReference has already expired, in which case we simply discard the entry).

We may periodically purge this queue of expired items to avoid it becoming a problem itself.

Anything that is called by this would need to re-register in order to be called again, ie on the next emergency. It is safe to re-register the same Runnable instance in such a case.

Thread-safe (and its iterators won't ever throw exceptions).

_emergencyFreeQueueRecurrent

private static final java.util.concurrent.CopyOnWriteArrayList<java.lang.ref.WeakReference<MemoryTools.RecurrentEmergencyFreeHandle>> _emergencyFreeQueueRecurrent

Concurrent and thread-safe read-mainly store of permanent emergency-free callbacks; never null once class is initialised.

_tMU_lock

private static final java.util.concurrent.locks.ReentrantLock _tMU_lock

Private lock for _trimMemoryUse(); never null (except possibly during initialisation).

_dEF_lock

private static final java.util.concurrent.locks.ReentrantLock _dEF_lock

Private lock for _doEmergencyFree(); never null (except possibly during initialisation).

_eTMUATimer

private static final java.util.Timer _eTMUATimer

Timer used to process _emergencyTrimMemoryUseAsync() activity; never null. Runs as a daemon to allow the JVM to exit.

Avoids having more than one Thread run this concurrently.

We may also run other periodic tasks with this timer. TODO: move expensive/periodic checks to this so that isMemoryStressed() calls all just poll a volatile flag

TARGET_FREE_FORCE_POLL_MS

private static final int TARGET_FREE_FORCE_POLL_MS

Interval between attempts to keep target memory free, ms, strictly positive.

See Also:

Constant Field Values

NO_MEM_INTENSIVE_CONCURRENCY

private static final boolean NO_MEM_INTENSIVE_CONCURRENCY

If true, only allows one memory-intensive operation to run at once. This reduces the chance of blowing up the heap, in return for reducing concurrency on multi-CPU machines.

We must not hold a lock to enforce this as doing so may cause deadlock; we veto any concurrent call immediately when true.

See Also:

Constant Field Values

_rMIO_commitedBytes

private static long _rMIO_commitedBytes

Bytes of memory ``committed'' to runMemoryIntensiveOperation(). Private to runMemoryIntensiveOperation() and accessed by it under the _rMIO_lock lock.

Initially zero.

_rMIO_noGCBefore

private static volatile long _rMIO_noGCBefore

Time before which System.gc() should not be retried; private to _makeMemoryAvailable(). Based on how long was spent running System.gc() on a previous attempt, this prevents system resources being significantly impacted by us continually running System.gc().

Initialised with 'now' to postpone the first attempt to force GC.

Is volatile to allow it to be read/written safely without a lock.

_rMIO_lock

private static final java.lang.Object _rMIO_lock

Private lock for runMemoryIntensiveOperation().

_rMIO_MAX_GC_FRAC

private static final int _rMIO_MAX_GC_FRAC

Maximum fraction of wall-clock time we will spend running GC; strictly positive. If this is N we will spend at most 1/(1+N) of wall-clock time running System.gc().

TODO: we might additionally abort if we've spent more than some fixed time, eg 30s, trying to collect garbage already.

This is used to limit impact on global system performance.

A value between 10 and 100 is probably reasonable.

See Also:

Constant Field Values

MIN_WAIT_AFTER_FAILED_GC

private static final int MIN_WAIT_AFTER_FAILED_GC

Minimum time before attempting forced GC after previous attempt (ms); strictly positive. This can save us from churning the heap too hard, or wasting time when System.gc() does nothing at all.

This can be a few seconds through to a few minutes.

_rMIO_MAX_GC_CALLS

private static final int _rMIO_MAX_GC_CALLS

Maximum number of times that we will call System.gc() to free memory; strictly positive. This is per call to runMemoryIntensiveOperation(); we may do less than this.

It may be worth calling System.gc() more than once because it may in fact take several passes in some JVMs to liberate all the heap from several different heap areas and across several threads, but calling it lots of times is probably just a huge drain on system resources or may simply be ignored by the JVM.

A value from 1 to 3 inclusive is probably good.

See Also:

Constant Field Values

_rMIO_FREE_MEM_MULT

private static final int _rMIO_FREE_MEM_MULT

Multiple of excess free memory there should be before attempting memory-intensive task; strictly positive. Bigger numbers are more conservative and reduce the probability of a crash due to running out of memory, but increase the chance of not running such as task when we otherwise could.

If we eliminate concurrency between memory-intensive operations then we can be less conservative than we otherwise would be, and indeed can possibly drop this down to 1.

A value in the range 1--4 is probably good.

See Also:

Constant Field Values

FORCE_SINGLE_CPU_MODE

private static final boolean FORCE_SINGLE_CPU_MODE

If true then force single-CPU mode.

See Also:

Constant Field Values

SINGLE_CPU_MODE

private static final boolean SINGLE_CPU_MODE

If true then run in single-CPU mode to save some space and time overheads by giving up some concurrency.

MAX_INTERN_CONC_POWER

private static final int MAX_INTERN_CONC_POWER

Power of maximum concurrency available in intern(); non-negative (0 in single-CPU mode).

WHM_Internables

private static final java.util.WeakHashMap<MemoryTools.Internable,java.lang.ref.WeakReference<MemoryTools.Internable>>[] WHM_Internables

Private to intern(): for holding Internable values; never null nor zero-size nor containing nulls. Each map must be accessed only under a lock on that map to serialise access as WeakHashMap is not itself thread-safe.

We start this off reasonably large, and with a low load factor, for access performance.

Unless in SINGLE_CPU mode we have many (low-load-factor) maps to usually allow concurrent (and fast) intern() operations.

WHM_Internables_0

private static final java.util.WeakHashMap<MemoryTools.Internable,java.lang.ref.WeakReference<MemoryTools.Internable>> WHM_Internables_0

Private to intern(); reference to 0 element of WHM_Internables as optimisation for single-CPU systems; never null.

_USE_STRING_INTERN

private static final boolean _USE_STRING_INTERN

If true, we trust String.intern() to be sensible and to allow GC of unused entries, and thus we delegate handling of at least some String values to it.

See Also:

Constant Field Values

MAX_STRING_INTERN

private static final int MAX_STRING_INTERN

Strings no longer than this use our storage rather than String.intern(). If this threshold is less than Integer.MAX_VALUE then this has the effect of keeping big Strings out of the PermGen in Sun's JVMs, which probably would not meld with any JVM-intern()ed value anyway. Avoiding String.intern() may save time as well as heap-management problems. (Allowing some short values to use String.intern() may increase available concurrency slightly.)

A reasonable value for this threshold is probably in the range -1--1024, for example:

• -1 prevents use of String.intern(),
• 0 allows only "" use of String.intern(),
• Integer.MAX_VALUE to allow all values to use String.intern().

See Also:

Constant Field Values

LOWER_PC_THRESHOLD_FOR_MIN_CAPACITY

private static final int LOWER_PC_THRESHOLD_FOR_MIN_CAPACITY

Lower percentage threshold of free space to force reported capacity cap to minimum; strictly positive and less than 99. A value a little greater than 1 helps quickly flush out residual entries when very short of memory, which may yield a disproportionate reduction in GC costs.

See Also:

Constant Field Values

MemoryTools

private MemoryTools()

Prevent creation of instances of this class.

_isMemoryStressedQ

private static boolean _isMemoryStressedQ()

Very quick non-blocking internal poll of stress state. Only to be used internally, and only valid if isMemoryStressed() is polled regularly.

Can be used to bias allocators, etc.

Can be used in a short-cut conditional such as (_isMemoryStressedQ() && isMemoryStressed()) to avoid running the full-blown isMemoryStressed() unless we were recently stressed, ie is quick if not recently stressed and doesn't give false positives.

isMemoryStressed

public static final boolean isMemoryStressed()

Returns true if the memory system may be under (great) stress, eg not enough free. This can help the application to avert a shortage by avoiding discretionary allocation, as well as in reacting to an unexpected shortage/stress.

This may work best if polled reasonably regularly.

This is likely to be at best stochastic, ie more likely to return true under severe memory stress, and less likely to return true when not under significant memory stress.

This checks the clearing of a SoftReference as one measure of memory stress:

• On a system with plenty of free memory this should rarely/never be cleared.
• On a system under severe memory stress this may be cleared frequently.

This also makes sure that at least a reasonable fraction of the current heap 'total' memory is reported as free.

This may attempt to initiate release of some memory asynchronously when it detects memory stress.

TODO: make use of MemoryPoolMXBean (etc) facilities for more robust detection.

_targetFreeMemory

private static long _targetFreeMemory()

Target amount of free heap space (a decent fraction of current heap size). Small enough not to be rejected by _makeMemoryAvailable() outright, currently 25% of total heap size.

lotsFree

public static boolean lotsFree()

True when there's currently lots of memory free and memory is not otherwise stressed.

percentFreeWithinTarget

public static int percentFreeWithinTarget()

Estimate of percentage of free space remaining as a fraction of the system target; in range[0,100]. If at least the target amount of memory is free such that lotsFree() could be true then this returns 100.

Nominally, if less than 1% of the target amount of free space is available the this returns 0, though an OOME is likely before this is reached.

Lower numbers indicate increasing memory pressure.

Returns:

value in range zero to 100 inclusive with 0 indicating complete heap exhaustion and 100 indicating that there is a comfortable portion of heap left.

registerCompactable

public static final void registerCompactable(MemoryTools.Compactable c)

Allows a local Compactable instance to register itself. This is intended only to hold a smallish number of potentially-large collections/containers that may each be repeatedly compacted over time.

Should be largely wait-free.

This may fail to register an item during initialisation, but in any case no Compactable instance should rely on this to call compact() but should also be doing the equivalent itself, eg on most accessors as per WeakHashMap.

Parameters:

c - fully-constructed non-null Compactable instance

registerEmergencyFreeHandle

public static void registerEmergencyFreeHandle(java.lang.Runnable r)

Queue up a handle to be called at most once to release memory on demand when detected to be very low. Entries are held by weak reference to ensure that they don't prevent GC.

Anything that is called by this would need to re-register in order to be called again, ie on the next emergency. It is safe to re-register the same Runnable instance in such a case, though registering multiple times would be inadvisable.

These handles should ideally run without blocking at all, or at least be quick and avoid deadlock.

Thread-safe.

registerRecurrentEmergencyFreeHandle

public static void registerRecurrentEmergencyFreeHandle(MemoryTools.RecurrentEmergencyFreeHandle r)

Queue up a handle to be called back as often as necessary to release memory on demand when detected to be very low. Entries are held by weak reference to ensure that they don't prevent GC, and need not unregister themselves but instead may simply be released to be GCed.

These entries need not and must not attempt to re-register themselves or be otherwise registered multiple times.

We try very hard to make sure that any significant memory required is allocated (and thus may possibly fail) during registration rather than when attempting to actually use the callbacks, thus maximising the chance of working correctly!

These handles should ideally run without blocking at all, or at least be quick and avoid deadlock.

These are assumed to be added (and expire) infrequently.

Thread-safe.

_trimMemoryUse

private static void _trimMemoryUse()

Internal routine, thread-safe, to try to (fairly firmly) trim some memory use on demand. May take some time and resources of its own, so possibly best only called if there is significant memory stress evident.

Doesn't force any emergency memory release.

Thread-safe: silently returns if routine already running (lock already held) or in the (unusual) case of the lock not yet initialised/set-up.

_doEmergencyFree

private static int _doEmergencyFree()

Do emergency free/release of memory. Processes all non-recurrent items in the queue by draining it into another collection first then processing all the moved items. (This prevents items that re-register themselves from being re-run continuously!)

FIXME: danger of 'losing' handles if we get another OOM here while moving them around.

Thread-safe: silently returns (-1) if already running (lock already held) or in the (unusual) case of the lock not yet initialised/set-up.

_emergencyTrimMemoryUseAsync

private static void _emergencyTrimMemoryUseAsync(boolean doEmergencyFree)

Run _trimMemoryUse(), _doEmergencyFree() if requested plus forced GC, asynchronously.

Parameters:

doEmergencyFree - if true then call the emergency-free handlers and possibly force GC

_makeMemoryAvailable

private static boolean _makeMemoryAvailable(long bytes)

Attempt to make sure that the specified amount of memory is available; return true if so. If this cannot be done (eg the requested space is apparently not available, or it is too soon since the last GC to try to force another full GC) then this returns false.

This does not hold any locks.

We try very hard to avoid actually calling System.gc() as doing so may upset the ergonomics of almost any GC implementation, and in particular is quite likely to result in an unwanted long stop-the-world pause.

Parameters:

bytes - approximate number of bytes of heap free required; non-negative

Returns:

true if requested memory is apparently available, false if not available

runMemoryIntensiveOperation

public static <V,E extends java.lang.Exception> V runMemoryIntensiveOperation(java.util.concurrent.Callable<V> task,
                                                                              int estimatedMinBytesRequired)
                                     throws java.lang.Exception

Wrapper round runMemoryIntensiveOperation(Runnable, ...) for value-returning tasks that must always be run. Returns a pair of a Boolean (true if the task ran, false if not) and the value returned by the Callable (always false if the task is not run).

Throws:

java.lang.Exception - any exception thrown by the Callable or our handler routine

runMemoryIntensiveOperation

public static <V> Tuple.Pair<java.lang.Boolean,V> runMemoryIntensiveOperation(java.util.concurrent.Callable<V> task,
                                                                              boolean unlimitedMemory,
                                                                              int estimatedMinBytesRequired)
                                                                   throws java.lang.Exception

Wrapper round runMemoryIntensiveOperation(Runnable, ...) for value-returning tasks. Returns a pair of a Boolean (true if the task ran, false if not) and the value returned by the Callable (always false if the task is not run).

Throws:

java.lang.Exception - any exception thrown by the Callable or our handler routine

runMemoryIntensiveOperation

public static boolean runMemoryIntensiveOperation(java.lang.Runnable task,
                                                  boolean unlimitedMemory,
                                                  int estimatedMinBytesRequired)
                                           throws java.lang.IllegalArgumentException,
                                                  java.lang.IllegalStateException

Run (or veto by refusing to run) a memory-intensive operation. This is passed a task to be run, with an estimate of the amount of extra memory required to run it.

The task is run, possibly after a delay, if resources seem to be available for it, and this routine returns true.

If sufficient resource (ie memory) does not seem to be available, then the task is not return and this routine returns false.

This routine may attempt to free memory by calling System.gc() one or more times, but because this may be slow and may impact system performance (eg concurrency) badly, this waits a significant multiple of the time taken to perform GC between such attempts, so this routine may refuse to try to free memory for a task even if it would work.

This keeps a static variable to track memory use between separate (concurrent) callers and is thus implicitly counting memory shared in this JVM or class loader. This routine tries to be robust enough to work reasonably well even with multiple separate copies of this class in separate class loaders within one JVM.

If the unlimitedMemory parameter is true then the task is always run immediately, and can be used for testing and for situations where heap space is freely available, eg the heap may be easily expanded.

No checked exception is propagated to the caller, but a non-checked exception may be.

This routine holds no lock for its duration to avoid deadlocks, but may "baulk" at (ie refuse to run) concurrent calls so as to limit concurrency.

Parameters:

task - the task to be run

estimatedMinBytesRequired - estimate of peak number of bytes of heap that the task will use (overestimate rather than underestimate this value); must be non-negative

unlimitedMemory - if true, the task is always run on the assumption that at the moment resources are effectively unlimited (or at least that this task must be run) but other memory-intensive tasks may be denied

Returns:

true if the task was run, false if it was not run because of apparent memory shortage or because too many other memory-intensive tasks are running

Throws:

java.lang.IllegalArgumentException - if the task is null or a negative amount of memory is specified

java.lang.IllegalStateException - if too much aggregate memory is being requested between all the concurrent tasks

preemptiveGC

public static boolean preemptiveGC()

Request a preemptive garbage collection, for example while the system is (too) quiet. This request may be ignored (eg if too soon after a previous one) and the system may ignore any attempts made here, or may do it asynchronously, or may do a partial GC.

This should be called when the system seems to be idle (in terms of human user, eg Web site visitor, activity) and is likely to idle for (say) a minute or so, to allow us to get in a full GC run while no one is looking, so as we can last as long as possible without an intrusive GC when the system is in use later. This may help recover from (for example) leaked file handles preventing any clients contacting us.

An alternative use is when we can't see a comfortable margin of free memory (possibly even though we've just trimmed significant fat) and we want to try to force such a margin to be visibly free.

Returns:

true if we apparently managed to free up some memory, else false

heldInternableCount

private static int heldInternableCount()

Get current number of Internable items held, and purge any dead entries too.

intern

public static java.lang.String intern(java.lang.String s)

Specific overloading of intern() for String to avoid runtime instanceof cost where possible.

intern

public static <E extends MemoryTools.Internable> E intern(E o)

Specific overloading of intern() for Internable to avoid runtime instanceof cost where possible.

intern

public static <E extends java.lang.Number> E intern(E o)

Specific overloading of intern() for Number to avoid runtime instanceof cost where possible.

intern

public static <E> E intern(E o)

Generic equivalent of String.intern(), for more immutable objects that we care about. Any object which is immutable (and probably final) and has functional equals() and hashcode() methods where only apparently-identical instances are equal can potentially be cached/canonicalised/deduped via this routine.

This generic routine is for where we do not know the type until run-time; the compiler may select a better overloading at compile-time otherwise. Thanks to Peter Ahé of Sun for the suggestion!

Note that lots of immutable/final objects are not suitable for canonicalisation with this routine. Caveat programmer.

A unique instance is returned of the same type.

When the intern method is invoked, if the pool already contains an Object equal to this Object o as determined by the equals(Object) method, then the object from the pool is returned. Otherwise, this Object is added to the pool and a reference to this Object is returned.

It follows that for any two Objects s and t, intern(s) == intern(t) is true if and only if s.equals(t) is true.

Entries are automatically dropped sometime after the last strong/soft reference to them has gone, possibly on the next call to this routine.

Unlike String.intern(), this applies to more than just String elements, and memory is guaranteed to be recoverable from non-referenced items (whereas String.intern() has not always been so and is still not clear in the spec).

It is probably not especially useful to put String constants or other static items in here routinely as their entries are unlikely to become unreferenced.

This routine is likely to be quite slow and have significant overhead, so only use it when not doing so would cause huge memory bloat or churn, and when you expect the intern()ed values to live for a long time.

May not in fact be able to cache/canonicalise all or any objects; mutable objects should never be intern()ed.

Calling this with null returns null.

Returns:

an Object that has the same contents as this Object, but is guaranteed to be from a pool of unique Objects.

computeCapacityCap

private static int computeCapacityCap(int minSize,
                                      int maxCapacity)

Computes target map size cap based on system memory availability; in range [minSize,maxCapacity]. If free heap falls below the system target this computes a reduced cap between the two hard limits supplied.

Always returns Integer.MAX_VALUE if maxCapacity is Integer.MAX_VALUE.