org.hd.d.pg2k.svrCore
Class GenUtils

java.lang.Object
  org.hd.d.pg2k.svrCore.GenUtils

public final class GenUtils
extends java.lang.Object

Some general utility methods of use throughout the application.

systemOutLogger

@Deprecated
public static final SimpleLoggerIF systemOutLogger

Deprecated. 

Simple logger instance to log to System.out.

systemErrLogger

@Deprecated
public static final SimpleLoggerIF systemErrLogger

Deprecated. 

Simple logger instance to log to System.err.

nullLogger

public static final SimpleLoggerIF nullLogger

Simple logger instance to throw away log output.

MAX_gLTD_cache_SIZE

private static final int MAX_gLTD_cache_SIZE

Maximum capacity for _gLTD_cache; strictly positive. The larger this capacity the more memory that we may consume but the greater the chance we have of making a cache hit.

If this cache is to help significantly with building the by-word index then this should be easily large enough to handle the fact that keys will at best be looked up in sorted order on part of the exhibit name, and that that may not correspond exactly with the way that AKA/desc data is organised.

If this is to help significantly with normal catalogue-page lookup then this should allow for the typical distribution of page views and of locales used by visitors.

This suggests a minimum size of at least several hundred entries under normal circumstances.

Aim to allow ~10000 entries per 1GB of heap space (at initialisation)...

_gLTD_cache

private static final LRUMapAutoSizeForHitRate<CS8Bit,java.lang.Object> _gLTD_cache

An LRU cache for lookups in getLocalisedTreeDesc() and private to it; never null. This is a map from a composite key based on the data hash, exhibit name, lookup type and locale to the result of the lookup.

The result text can be extracted with toString().

(The data hash that we use from the AEP is from just the treedesc data so that changes in other parts of the AEP do not invalidate cache entries. This is useful because the treedesc data usually changes much more slowly than the entire AEP (and the AEP longHash does not always capture the treedesc state) and because we can use a shorter hash for quicker key construction.)

The keys are essentially unique and this not intern()ed, but the result texts may not be unique, and thus are intern()ed to try to economise on memory.

This cache can in principle support multiple AEPs simultaneously (assuming that their hashes do not collide) and thus will cope with an update to a new AEP in the Web server.

We are prepared to empty the cache entirely under memory stress.

FIXME: convert this from being a static cache, eg to an opaque value passed in by caller

TREE_TEXT_PREFIX_DESCRIPTION

public static final java.lang.String TREE_TEXT_PREFIX_DESCRIPTION

Prefix of descriptive-text keys in tree-text i18n bundles.

See Also:

Constant Field Values

TREE_TEXT_PREFIX_AKA

public static final java.lang.String TREE_TEXT_PREFIX_AKA

Prefix of Also-Known-As-keyword-list keys in tree-text i18n bundles.

See Also:

Constant Field Values

DEFAULT_MAX_DESCAKA_GAP

public static final int DEFAULT_MAX_DESCAKA_GAP

Default maximum acceptable gap between aka/desc entries for search to continue; non-negative.

See Also:

Constant Field Values

SECTION_LOOKUP_COMMON_FALLBACK

private static final boolean SECTION_LOOKUP_COMMON_FALLBACK

If true, fall back to a lookup in the common i18n bundles for section titles and description.

See Also:

Constant Field Values

SECTION_DESC_PROPNAME_PREFIX

private static final java.lang.String SECTION_DESC_PROPNAME_PREFIX

The prefix to look up the section description in the common message bundle, if any.

See Also:

Constant Field Values

SECTION_TITLE_PROPNAME_PREFIX

private static final java.lang.String SECTION_TITLE_PROPNAME_PREFIX

The prefix to look up the prettied/i18ned version of a section title in the common message bundle, if any.

See Also:

Constant Field Values

MAX_PARENTS_COUNTED

private static final int MAX_PARENTS_COUNTED

Number of parents (above child hot spot) to count in in stack trace; strictly positive. If 1 this counts only the closest interesting parent.

If significantly greater than 1 then can capture recursion for example, at the cost of extra work.

A reasonable value is probably 1 to 32.

See Also:

Constant Field Values

MAX_SUPPORTED_COMPRESSION_LEVEL

public static final CompressionLevel MAX_SUPPORTED_COMPRESSION_LEVEL

Maximum compression level currently supported by compressData(); never null. Until 20070811 this has been BZIP2.

DATA_PUMP_BLOCK_SIZE

private static final int DATA_PUMP_BLOCK_SIZE

Basic dataPump() read size (bytes) for efficient operations; strictly positive.

See Also:

Constant Field Values

DEFAULT_LOWPOWER_FLAG_FILENAME

public static final java.io.File DEFAULT_LOWPOWER_FLAG_FILENAME

Presence of a file in this location is an indication of power shortage by default. The file is relative to the JVM's working directory.

If any local-props value is supplied then this is ignored.

_lp_lastCheck

private static volatile long _lp_lastCheck

Time and result of last check of system power state; the least significant bit indicates the state detected. When the last bit is 1 it indicates that power is low, else none was.

Marked volatile for thread-safe lock-free access.

The initial state indicates that the system should start in power-conserving mode unless this instance is in fast-start mode.

_LP_MIN_RECHECK_MS

private static final int _LP_MIN_RECHECK_MS

Minimum interval for check/update of file flags in ms; strictly positive. Set to avoid expensive and redundant tests.

Should probably be of the order of a few seconds.

See Also:

Constant Field Values

_LP_MIN_RECHECK_URI_MS

private static final int _LP_MIN_RECHECK_URI_MS

Minimum interval for check/update of URI flags in ms; strictly positive. Chosen to avoid expensive and redundant tests.

Should probably be of the order of a few tens of seconds as network connections may have all sorts of costs and be uncacheable, though checking some factors such as local grid overload (via mains frequency) could profitably be done more often.

See Also:

Constant Field Values

_LP_MAX_RECHECK_URI_MS

private static final int _LP_MAX_RECHECK_URI_MS

Maximum interval for check/update of URI flags in ms; strictly positive. Much larger than _LP_MIN_RECHECK_URI_MS to allow a large dynamic range and thus efficient adaptation to the actual underlying information rate.

Should probably be of the order of a an hour or more.

See Also:

Constant Field Values

_LP_MAX_CHECK_TIME_URI_MS

private static final int _LP_MAX_CHECK_TIME_URI_MS

Maximum time that a check/update of URI flags may take in ms; strictly positive. Long enough to only allow a new check to be launched if a previous one has failed in some way not otherwise caught.

Should probably be several minutes at least.

See Also:

Constant Field Values

_mcp_lock

private static final java.lang.Object _mcp_lock

Private lock for mustConservePower() while working.

lowerPowerFileURIStatus

private static final java.util.concurrent.ConcurrentMap<java.net.URI,GenUtils.RemoteFlagInfo> lowerPowerFileURIStatus

Low-power URI flags and their status; never null. Is thread-safe to allow async updates.

Is periodically purged of stale URI keys to avoid 'leaking' memory.

extremeLowPower

private static volatile boolean extremeLowPower

Extreme-low-power flag status, true if extreme conservation is required; initially false. Marked volatile for lock-free thread-safe access.

MIN_LONG_TERM_LOW_POWER_MS

public static final long MIN_LONG_TERM_LOW_POWER_MS

Minimum time that system has to be continuously in low-power mode to trigger 'long term' measures (ms); strictly positive. Somewhat more than one or more 24h periods to allow for daily energy cycles for systems powered by renewables and for clock-driven notions of low-power mode (eg driven by ToD/HH grid load and carbon-intensity).

Not so long as to remove responsiveness to actual energy issues, especially after system (re)start.

Small random factor to help avoid collisions between different servers.

lastInNormalPowerMode

private static volatile long lastInNormalPowerMode

Last time system was not in low-power mode. Initially 'now' so that system does not start in long-term low-power measures.

Marked volatile to allow lock-free thread-safe access.

GenUtils

private GenUtils()

Prevent construction of an instance.

getStandardDigest

public static java.security.MessageDigest getStandardDigest()

Get instance of our standard good/secure digester (a new instance each time). This is the SHA1 digest.

Instances returned by this call must not be shared between threads.

appVersion

public static java.lang.String appVersion()

Get application build version in form x.y.z (eg 1.57.15); null if not available.

log2Approx

public static int log2Approx(long v)

Compute the approximate rounded-down log-base-2 of the input. The input must be non-negative; input zero returns -1.

In effect this returns a 1-significant-bit answer, which is useful for "clumping" values that have a large dynamic range.

Returns:

floor(log₂(v))

getLocalisedTreeDesc

public static final java.lang.CharSequence getLocalisedTreeDesc(AllExhibitProperties aep,
                                                                java.lang.CharSequence exhibitName,
                                                                LocaleBeanBase localeBean,
                                                                boolean getDesc,
                                                                boolean getAKA,
                                                                boolean akaLinks,
                                                                boolean mostSpecificOnly)

Extract (immutable) description tree information as a list; returns "" if none; never null. Returns an HTML fragment, zero or more <li>key (AKA keywords) = text</li> values in order, with localised i18n text extracted about the exhibit.

This will stop searching after a certain number of failures, ie nodes where no description/AKA text is available. This is to reduce search times on long exhibit names or where there is no suitable text at all.

Behaviour is undefined if the exhibitName is not syntactically valid. Note that the exhibitName need not be a current/extant exhibit, and only the main-words portion is used case-insensitively.

Note that the returned text is almost always 7-bit or 8-bit, and thus may be returned as (for example) a Name or Compact7BitString.

Parameters:

aep - complete exhibit metadata/properties; never null

exhibitName - syntactically-valid (not necessarily extant) full exhibit name; never null

getDesc - if true, include descriptive text from the tree

getAKA - if true, include "Also-Known-As" synonym text from the tree

akaLinks - if true, insert hyperlinks from synonyms to appropriate places in the catalogue tree where possible

mostSpecificOnly - if true, show only the most-specific text,

getLocalisedSectionDesc

public static final java.lang.CharSequence getLocalisedSectionDesc(AllExhibitProperties aep,
                                                                   java.lang.CharSequence sectionDir,
                                                                   LocaleBeanBase localeBean)

Find the localised section description for the given category name; returns null if none. This is sufficiently specialised that we don't follow our normal convention of returning the request key if localised text is not found.

Parameters:

aep - current AEP; never null

sectionDir - category/section/directory name; never null

localeBean - current locale; never null

Returns:

section description, or null if none

computeSectionTitle

public static java.lang.String computeSectionTitle(AllExhibitProperties aep,
                                                   java.lang.CharSequence sectionDir,
                                                   LocaleBeanBase localeBean)

Get section title (as HTML); never null. This attempts to get or compute a section title given the top-level section directory name from an exhibit (such as "brain_scan").

The fall-back algorithm is for this to convert all non-alphanumeric characters in the name to spaces and capitalise the first letter of the title.

However, if passed a valid AEP and locale bean then this routine attempts to look up the directory name (suitably prefixed and suffixed) in appropriate message catalogue(s) to find a sugared and/or i18ned version.

Parameters:

aep - the exhibit properties containing the treedesc i18n text; may be null

sectionDir - section/category/directory; never null

localeBean - locale; can be null

isSensitive

public static boolean isSensitive(java.lang.CharSequence nameOrURI,
                                  GenProps gp)

Returns true iff the full/short exhibit name or URI supplied is "sensitive" in some way. Can be used to suppress "promotion" of (or advertising against) an exhibit, or other accidentally-tasteless activity.

Simply matches (case-sensitively) potentially-problematic substrings, so has to be used with care to avoid false/silly matches, the main defence against which is to use sufficiently long and specific substrings.

TODO: consider switching to regex expression and/or cached lookup

startThreadPerfMonitor

public static java.lang.Thread startThreadPerfMonitor(java.lang.Thread toObserve,
                                                      java.util.concurrent.ConcurrentHashMap<java.lang.StackTraceElement,java.util.concurrent.atomic.AtomicInteger> counts,
                                                      java.lang.String prefix,
                                                      long stopBy,
                                                      int sampleTimeMS)

Create a performance monitor attached to the given thread, filling in the given map. This adds data to an existing map, and multiple performance monitors may concurrently update the same map.

This thread can be interrupt()ed to stop it collecting, and it will also stop when the thread being monitored exits.

This runs as a high-priority thread sampling at a high rate (~1ms).

Parameters:

toObserve - the thread to be sampled; never null

counts - the map from code sample site to sample count; never null

prefix - if non-null then the lowest method on the stack whose fully-qualified class name starts with this prefix is logged, else the lowest method is logged

stopBy - the time by which we should stop sampling

sampleTimeMS - approximate minimum milliseconds to sleep between samples; strictly positive

startThreadPerfMonitor

public static java.lang.Thread startThreadPerfMonitor(java.lang.Thread toObserve,
                                                      java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,java.util.concurrent.atomic.AtomicInteger> counts,
                                                      java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,java.util.concurrent.atomic.AtomicInteger>> parentCounts,
                                                      java.lang.String prefix,
                                                      long stopBy,
                                                      int sampleTimeMS)

Create a performance monitor attached to the given thread, filling in the given map. This adds data to an existing map, and multiple performance monitors may concurrently update the same map.

This thread can be interrupt()ed to stop it collecting, and it will also stop when the thread being monitored exits.

This runs as a high-priority thread sampling at a high rate (~1ms).

Parameters:

toObserve - the thread to be sampled; never null

counts - the map from (prefix-matching) code sample site to sample count; never null

parentCounts - map from (prefix-matching) code sample site to all its (prefix-matching) parent/caller sites; can be null if not required

prefix - if non-null then the lowest method on the stack whose fully-qualified class name starts with this prefix is logged, else the lowest method is logged

stopBy - the time by which we should stop sampling

sampleTimeMS - approximate minimum milliseconds to sleep between samples; strictly positive

stopPerfMonitorandDumpSamples

public static void stopPerfMonitorandDumpSamples(java.lang.Thread monitorThread,
                                                 java.lang.String title,
                                                 java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,java.util.concurrent.atomic.AtomicInteger> counts,
                                                 int maxToShow,
                                                 SimpleLoggerIF logger)
                                          throws java.lang.InterruptedException

Stop a monitor thread and dump its samples.

Parameters:

monitorThread - thread being profiled; never null

title - title to display; can be null

counts - map from sample site to counts at that site; never null

maxToShow - max (top) entries to display; strictly positive

logger - if null no dump is generated, else is log sink

Throws:

java.lang.InterruptedException - if this thread is interrupted

stopPerfMonitorandDumpSamples

public static <SM extends java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,java.util.concurrent.atomic.AtomicInteger>> void stopPerfMonitorandDumpSamples(java.lang.Thread monitorThread,
                                                                                                                                                                        java.lang.String title,
                                                                                                                                                                        java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,java.util.concurrent.atomic.AtomicInteger> counts,
                                                                                                                                                                        java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,SM> parentCounts,
                                                                                                                                                                        int maxToShow,
                                                                                                                                                                        SimpleLoggerIF logger)
                                          throws java.lang.InterruptedException

Stop a monitor thread and dump its samples. Note that we only show parent traces for a small top fraction of the hot sites, and show a fraction of maxToShow parent sites on the grounds that we otherwise have a huge explosion in output size for relatively little extra value!

Parameters:

monitorThread - thread being profiled; never null

title - title to display; can be null

counts - map from sample site to counts at that site; never null

parentCounts - map from sample site to parents and counts at that site; can be null if not required

maxToShow - max (top) entries to display; strictly positive

logger - if null no dump is generated, else is log sink

Throws:

java.lang.InterruptedException - if this thread is interrupted

dumpPerfSamples

public static void dumpPerfSamples(java.lang.String title,
                                   java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,java.util.concurrent.atomic.AtomicInteger> counts,
                                   int maxToShow,
                                   SimpleLoggerIF logger)

Dump a set of performance-monitoring site samples. We take a snapshot of the current profile stats when we start, so that it is safe to dump from a map that is still being updated (though keys/entries must not be removed while we are working) and an iterator over the counts map and get() must be thread-safe.

We take a snapshot of the current profile stats when we start, so that it is safe to dump from a map that is still being updated (though keys/entries must not be removed while we are working) and an iterator over the counts map and get() must be thread-safe.

Parameters:

title - title to display; can be null

counts - map from sample site to counts at that site; never null

maxToShow - max (top) entries to display; strictly positive

logger - destination of profile results; never null

dumpPerfSamples

public static <SM extends java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,java.util.concurrent.atomic.AtomicInteger>> void dumpPerfSamples(java.lang.String title,
                                                                                                                                                          java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,java.util.concurrent.atomic.AtomicInteger> counts,
                                                                                                                                                          java.util.concurrent.ConcurrentMap<java.lang.StackTraceElement,SM> parentCounts,
                                                                                                                                                          int maxToShow,
                                                                                                                                                          SimpleLoggerIF logger)

Dump a set of performance-monitoring site samples. We take a snapshot of the current profile stats when we start, so that it is safe to dump from a map that is still being updated (though keys/entries must not be removed while we are working) and an iterator over the counts map and get() must be thread-safe.

We take a snapshot of the current profile stats when we start, so that it is safe to dump from a map that is still being updated (though keys/entries must not be removed while we are working) and an iterator over the counts map and get() must be thread-safe.

Parameters:

title - title to display; can be null

counts - map from sample site to counts at that site; never null

parentCounts - map from sample site to parents and counts at that site; can be null if not required

maxToShow - max (top) entries to display; strictly positive

logger - destination of profile results; never null

isValidGalleryTimestamp

public static final boolean isValidGalleryTimestamp(long timestamp)

Validate internal Gallery timestamp; returns true if sensible, false otherwise. Such timestamps should not be before the Gallery was created, nor much in the future (allowing for some clock skew between consenting participants).

wrapForCompression

public static java.io.OutputStream wrapForCompression(java.io.OutputStream os,
                                                      CompressionLevel level)
                                               throws java.io.IOException

Stream compress/filter; never null. Especially valuable for very large amounts of data to large to handle uncompressed.

The output stream is wrapped with the selected compression filter that should be byte-for-byte compatible with the output of compressData() for the given compression level, generally maximal, unless flush() is called mid-stream in which case there may be reduced compression.

Compression may not be finished and all data pushed through until close() is called; flush() may not be sufficient and may produced less-compressed output.

Errors from the compressor may be transformed into IOException for easier handling and recovery by the caller.

This may increase the size of the data in some cases, especially where the input array is short.

A compression level of NONE returns the supplied stream unchanged.

Parameters:

os - output stream of uncompressed data; never null

level - compression level; never null and no higher than MAX_SUPPORTED_COMPRESSION_LEVEL

Throws:

java.io.IOException

compressData

public static Tuple.Pair<CompressionLevel,byte[]> compressData(byte[] in,
                                                               CompressionLevel maxLevel)
                                                        throws java.lang.InterruptedException

Compress the supplied binary data as well as possible up to the given compression level; never null. Tries all the supported compression methods up to the specified level returning whichever results in the smallest result.

(Resource constraints may prevent some of the compression types being applied, usually the highest levels.)

This does NOT assume that higher compression levels always compress given data better, not does it assume a lower bound on data size worth compressing; this tries all the possibilities exhaustively if it can.

This is likely to be very memory- and CPU- intensive; use with care.

This may attempt compressions in parallel on a multi-CPU machine, though each compression is likely to thrash the cache so this may or may not be good for compression and system performance.

This routine does not alter the content of the input array.

TODO: WRAP AS MEMORY-INTENSIVE OP FOR MORE EXTREME COMPRESSION METHODS

Parameters:

in - the data to be compressed; never null

maxLevel - the maximum level at which to attempt to compress data, using a value greater than MAX_SUPPORTED_COMPRESSION_LEVEL is allowed but ineffective; never null

Returns:

non-null data compressed as well as possible; if compression was not possible the level is NONE and the input array is returned in the result as-is

Throws:

java.lang.InterruptedException

compressData

public static Tuple.Pair<CompressionLevel,byte[]> compressData(byte[] in,
                                                               CompressionLevel maxLevel,
                                                               boolean maxOnly)
                                                        throws java.lang.InterruptedException

Compress the supplied binary data as well as possible up to the given compression level; never null. Tries all the supported compression methods up to the specified level returning whichever results in the smallest result.

(Resource constraints may prevent some of the compression types being applied, usually the highest levels.)

This does NOT assume that higher compression levels always compress given data better, not does it assume a lower bound on data size worth compressing; this tries all the possibilities exhaustively if it can.

This is likely to be very memory- and CPU- intensive; use with care.

This may attempt compressions in parallel on a multi-CPU machine, though each compression is likely to thrash the cache so this may or may not be good for compression and system performance.

This routine does not alter the content of the input array.

TODO: WRAP AS MEMORY-INTENSIVE OP FOR MORE EXTREME COMPRESSION METHODS

Parameters:

in - the data to be compressed; never null

maxLevel - the maximum level at which to attempt to compress data, using a value greater than MAX_SUPPORTED_COMPRESSION_LEVEL is allowed but ineffective; never null

maxOnly - if true then only try the maximum compression level, and implicitly 'NONE'

Returns:

non-null data compressed as well as possible; if compression was not possible the level is NONE and the input array is returned in the result as-is

Throws:

java.lang.InterruptedException

compressObject

public static Tuple.Pair<CompressionLevel,byte[]> compressObject(java.io.Serializable in,
                                                                 CompressionLevel maxLevel)
                                                          throws java.io.IOException,
                                                                 java.lang.InterruptedException

Serialise the given object and compress the result as well as possible up to the given compression level; never null. Tries all the supported compression methods up to the specified level returning whichever results in the smallest result.

(Resource constraints may prevent some of the compression types being applied, usually the highest levels.)

This does NOT assume that higher compression levels always compress given data better, not does it assume a lower bound on data size worth compressing; this tries all the possibilities exhaustively if it can.

This is likely to be very memory- and CPU- intensive; use with care.

This may attempt compressions in parallel on a multi-CPU machine, though each compression is likely to thrash the cache so this may or may not be good for compression and system performance.

This routine does not alter the content of the input array.

Parameters:

in - the object to be compressed; never null

maxLevel - the maximum level at which to attempt to compress data, using a value greater than MAX_SUPPORTED_COMPRESSION_LEVEL is allowed but ineffective; never null

Returns:

non-null data compressed as well as possible; if compression was not possible the level is NONE and the input array is returned

Throws:

java.io.IOException

java.lang.InterruptedException

compressObject

public static Tuple.Pair<CompressionLevel,byte[]> compressObject(java.io.Serializable in,
                                                                 CompressionLevel maxLevel,
                                                                 boolean maxOnly)
                                                          throws java.io.IOException,
                                                                 java.lang.InterruptedException

Serialise the given object and compress the result as well as possible up to the given compression level; never null. Tries all the supported compression methods up to the specified level returning whichever results in the smallest result.

(Resource constraints may prevent some of the compression types being applied, usually the highest levels.)

This does NOT assume that higher compression levels always compress given data better, not does it assume a lower bound on data size worth compressing; this tries all the possibilities exhaustively if it can.

This is likely to be very memory- and CPU- intensive; use with care.

This may attempt compressions in parallel on a multi-CPU machine, though each compression is likely to thrash the cache so this may or may not be good for compression and system performance.

This routine does not alter the content of the input array.

Parameters:

in - the object to be compressed; never null

maxLevel - the maximum level at which to attempt to compress data, using a value greater than MAX_SUPPORTED_COMPRESSION_LEVEL is allowed but ineffective; never null

maxOnly - if true then only try the maximum compression level, and implicitly 'NONE'

Returns:

non-null data compressed as well as possible; if compression was not possible the level is NONE and the input array is returned

Throws:

java.io.IOException

java.lang.InterruptedException

wrapForDecompression

public static final java.io.InputStream wrapForDecompression(java.io.InputStream in,
                                                             CompressionLevel cl)
                                                      throws java.io.IOException

Wrap input stream for decompression according to the compression level; never null. If an unsupported compression level is selected then an exception is thrown.

If NONE is chosen as the compression level then the input stream is returned as-is.

Can decompress data compressed with compressData()/compressObject().

Throws:

java.io.IOException

dataPump

public static final java.io.InputStream dataPump(java.io.InputStream is,
                                                 int bufferSize,
                                                 boolean aggressive)

Wraps InputStream with a data pump (if possible); never null. This can be used to keep data flowing in a pipeline with "lumpy" CPU use, such as a stream decompressor with large input blocks.

This also allows stages on either side of the pump to run in separate threads which can improve throughput further on a multi-CPU JVM.

In general this should be transparent to the user, with close()s, exceptions, etc, being propagated much as usual.

In particular, a close() on the returned stream will cause a close() on the argument input stream.

This routine may return the input stream if it cannot create a pump.

Instances of the returned class may not be thread-safe. This is so that we can maximise performance, especially for small reads.

If flagged as aggressive then this routine always forces the creation of an extra thread to do read-ahead, else this only does so when an attempted read from down-stream cannot be immediately (partly or fully) satisfied. Thus a "non-aggressive" data pump may save some overhead where it is not needed, eg on short/fast streams.

Parameters:

is - input data stream; never null

bufferSize - approximate maximum amount of data to read-ahead; strictly positive

aggressive - if true then always start another thread, else possibly do so only to avoid blocking

cast

public static <T> T cast(java.lang.Object x)

Avoid compile-time complaints about casts believed to be correct. C/o http://weblogs.java.net/blog/emcmanus/archive/2007/03/getting_rid_of.html.

mustConservePower

public static final boolean mustConservePower()

If this returns true then the system must temporarily conserve power as much as possible. This may be because, for example, we are running on battery power, and/or the (remaining) battery capacity may be very limited.

This may also be because the system is running too hot and/or too noisy for example.

This cap the rate at which underlying data source(s) are polled.

This may block (very briefly) to check filesystem flags, but URI (ie probably-remote) flags are checked asynchronously and much less frequently and only if the local filesystem flags do not request a low-power state, so as to save bandwidth and time and effort.

This condition is usually assumed to be transient, not permanent.

Note that if more than one filesystem flag is listed, the last one is taken to indicate 'extreme' energy shortage/conservation.

mustConservePowerLongTerm

public static boolean mustConservePowerLongTerm()

If true then the system has been in a low-power state for a long time. Typically implies power has been low for (much) more than 24 hours, and thus longer-term changes in behaviour to save energy are in order.

approxMinMSToStartLongTermLowPowerMode

public static long approxMinMSToStartLongTermLowPowerMode()

Approximate minimum time to long-term low-power mode (ms). Can be negative to indicate how long the system has been in long-term low-power mode already.

Will stay at approximately maximum time if not currently in low-power mode.

mustConservePowerExtreme

public static boolean mustConservePowerExtreme()

If true then the system is in an extreme low-power state and may sacrifice some functionality to conserve. This is true if either the system has been in a low-power state for much longer than the MIN_LONG_TERM_LOW_POWER_MS limit, eg at least twice that, OR if the 'extreme low power' (last local filesystem) flag is set.

mustConserveCPU

public static boolean mustConserveCPU()

If this returns true then the system must permanently conserve CPU or is has extreme energy shortage. This may be because, for example, we are running on extremely limited battery reserves, or because we are running in a CPU-metered (eg "cloud"/VPS) environment, or possibly because of issues such as system over-heating.

This may block (very briefly).

leastN

public static <T extends java.lang.Comparable<T>> java.util.SortedSet<T> leastN(int n,
                                                                                java.util.Iterator<T> it,
                                                                                java.util.Comparator<T> comparator)

Returns immutable SortedSet of the n best items retrieved via the supplied iterator; never null but may be empty. Only retains at most n items at one time.

If less then n items are available then all are returned else the least n by the supplied Comparator are returned.

Minimises memory overhead by only retaining at most n items while working.

Parameters:

n - maximum size of head (least-value-items) set to return; non-negative

it - iterator over items to consider for inclusion in the result; non-null

comparator - Comparator over items to include; non-null

getIntHashedSimple

public static final int getIntHashedSimple(long seed,
                                           int n)

Generates a number in the range 0 to n-1 based mainly on the low-order bits of seed; non-negative. This is a simple and cheap alternative to (say) (new Random(seed)).nextInt(n)) to get a reasonably-evenly-distributed value based on a much larger one, such as currentTimeMillis().

The seed value should have much more than 2^n different values in its low-order bits.

A basic modulo function is used in this implementation.

Parameters:

seed - input whose value will be used to generate n.

n - small strictly positive exclusive upper bound on returned value

Returns:

non-negative value less than n