org.hd.d.pg2k.webSvr.exhibit
Class DataSourceBean

java.lang.Object
  org.hd.d.pg2k.webSvr.exhibit.DataSourceBean

All Implemented Interfaces:

java.io.Serializable, SimpleExhibitPipelineIF, SimpleLoggerIF, BasicVarMgrInterface, SimpleVariablePipelineIF

public final class DataSourceBean
extends java.lang.Object
implements SimpleExhibitPipelineIF, SimpleLoggerIF, java.io.Serializable

JavaBean encapsulating access to exhibit data and meta-data. This is expected to exist in one copy for the exhibit servlet, enabling (cached) access exhibit data and meta data.

This also provides access to shared data-pipeline and cache facilities, and is expected to be created on demand at application level. This, therefore, is designed to be used as a bean, and is suitable for direct instantiation from a JSP.

We have a public no-arg constructor to allow use from JSPs.

This object is thread-safe, and maximises concurrency while synchronising where necessary. (Quick calls are synchronized on this instance; longer-running calls may be synchronized on a private lock.)

This creates an internal thread to maintain the cache, and the thread is destroyed when the object is GCed. We only hold a SoftReference back to this instance from the background thread.

External callers can call the poll() method, though it is not recommended. They can also call the destroy method to close down the cache and stop the background thread, though this object must not be used thereafter.

When initialised for first use a number of parameters need to be set. (For JSP, these should only be done upon creation, eg between the useBean ... /useBean tags, to save cycles.) Many of the actual values passed are not stored but just used to extract other necessary parameters. Some parameters, notably contextPath and servletContext, should be set just before each use after retrieving the data from attributes, in case this instance has been passivated and restored.

(The servlet context parameter should be set every time the bean is used so that we always have a valid value to hand; we don't persist this if the bean is serialised.)

If an attempt to made to use the bean without initialising it, or to reinitialise it with conflicting values, an exception is thrown.

If an attempt is made to serialise/deserialise is made, the internal cache is reconstructed; careless use may result in multiple cache copies per servlet context which is wasteful, and may be unsafe.

One item created here is the by-word inverted index of the exhibits, created on first use.

This supports simple logging; by preference to the servlet context log, but failing that to System.out.

This is Serializable so as to be able to be stored in a servlet session; nothing especially long-lived or sensitive.

See Also:

Serialized Form

BEAN_NAME

public static final java.lang.String BEAN_NAME

Name that should be used for unique application-level instance.

See Also:

Constant Field Values

FORCE_INDEX_REBUILD_IN_POLL

private static final boolean FORCE_INDEX_REBUILD_IN_POLL

If true, ensure that index is up-to-date on each poll() call if possible. If true, this minimises delay for users, but can seriously impact start-up time and may perform significant unnecessary work when new items are being loaded into the database.

We may try to avoid a forced rebuild if the system is busy or else do each (re)build in a low-priority thread.

See Also:

Constant Field Values

logger

private final WebUtils.ServletLoggerWithFallback logger

Our logger which falls back to System.out if servlet log not available; never null.

iterEPCM

private final DataSourceBean.AEPLinkedKey iterEPCM

Key for systematic EPCM recomputation (linked to AEP), Iterator; never null.

deadIt

private static final java.util.Iterator<Name.ExhibitFull> deadIt

'Dead' iterator with hasNext() always false; never null.

SCORER_BG_EMERGENCY_CUTOFF

private static final boolean SCORER_BG_EMERGENCY_CUTOFF

If true then we stop all poll()-driven background/speculative Scorer processing. This will reduce system functionality (and predictive ability), but may be necessary where the strain of the extra processing is too high for a given machine, or where resources are shared with other heavy load.

A "true" value will have to be supplied on the the command line to effect this stop.

PROPS_REPOST_INTERVAL_MS

private final int PROPS_REPOST_INTERVAL_MS

How often do we repost properties (ms, approx); strictly positive. This should be at least once per minute.

_lastGoodPostProps

private long _lastGoodPostProps

Note of the last time that we successfully ran raw properties repost without error. Private to _postProps().

destroyed

private volatile boolean destroyed

Set true when the servlet is destroyed.

_observable

private transient volatile DataSourceBean.EDVHObservable _observable

Can be observed by an Observer that wants to know when the underlying data has changed; never null. Observers are notified when our hash changes, for example so that they can clear any now-stale cache and help GC.

Volatile so that access to the reference itself does not need a lock.

May be cleared once a notification has been delivered during a poll(), ie may be at most one-shot.

Missed notifications are not considered very important.

_slave

private volatile java.lang.Boolean _slave

Indicates if this is a master or a slave Web server, or not known. "Not known" is indicated by a null value.

This value may be persisted.

Volatile to be accessed without a lock by isSlave() and setSlave().

Must be set non-null to initialise the bean for use.

May be able to be set immediately from the context.

_servletContext

private transient volatile javax.servlet.ServletContext _servletContext

A valid servlet context, or null. We don't persist the value if the bean is serialised.

Declared volatile to allow safe lock-free access.

Should only be accessed by setServletContext() for write and generally by _getPipeline() and _postProps() and getServletContext() for read.

aggressive

private transient volatile boolean aggressive

The aggressiveness flag; defaulting to false. Is set when we set the context; we do not bother to persist it.

Is volatile so that no lock is needed to access it.

Private to setServletContext() and isAggressive().

_dataPipeline_build_lock

private static final java.lang.Object _dataPipeline_build_lock

Private lock to ensure only one thread tries to build _dataPipeline.

_dataPipeline

private transient volatile SimpleExhibitPipelineIF _dataPipeline

Non-persistable data pipeline. This is reconstructed under the pipeline lock if found to be null and if slave is non-null.

Should be accessed only by _getPipeline(), and (without a lock) nulled out in destroy().

MEAN_POLL_INTERVAL_MS

private static final int MEAN_POLL_INTERVAL_MS

Approximate target mean interval between poll() calls (ms); strictly positive. Polling may run (much) slower than this in energy-conserving mode.

Normally polling is just about once per second.

See Also:

Constant Field Values

NAMEDESC_ENCODING

private static final java.lang.String NAMEDESC_ENCODING

Encoding we use to convert name or name+description to byte stream for indexing. We can assume 7-bit ASCII or 8-bit ISO-8859-1.

See Also:

Constant Field Values

_rebuildJIB_lock

private final java.util.concurrent.locks.ReentrantLock _rebuildJIB_lock

Private lock for _rebuildJIB. Attempts by another thread to enter the lock are vetoed immediately, preventing other threads from blocking pointlessly.

CACHE_INDEX_AS_FILE

private static final boolean CACHE_INDEX_AS_FILE

If true then attempt to cache index in a file for potentially-faster system restart.

See Also:

Constant Field Values

CACHED_BYWORD_INDEX_FNAME

private static final java.lang.String CACHED_BYWORD_INDEX_FNAME

Name under which we attempt to cache the by-word index on disc (relative to servlet temp directory) if we do; not null. The format is implementation-dependent, and could be a serialised object or a saved index file.

See Also:

Constant Field Values

_getJIB_cache

private transient volatile ORG.hd.d.jIndexer.server.JIndexBean _getJIB_cache

Cache for, and private to, _getJIB(); initially null. We do not attempt to persist this data, so it is transient.

Is volatile so that access without a lock is safe.

FEBY_MATCH_TYPE_ANY

public static final int FEBY_MATCH_TYPE_ANY

Query type or findExhibitsByWord() to return entries matching any word.

See Also:

Constant Field Values

FEBY_MATCH_TYPE_MOST

public static final int FEBY_MATCH_TYPE_MOST

Query type or findExhibitsByWord() to return entries matching most words.

See Also:

Constant Field Values

FEBY_MATCH_TYPE_ALL

public static final int FEBY_MATCH_TYPE_ALL

Query type or findExhibitsByWord() to return entries matching all words.

See Also:

Constant Field Values

LOG_QUERIES

public static final boolean LOG_QUERIES

If true, log queries made on the system for analysis.

See Also:

Constant Field Values

storeLinked

private final java.util.concurrent.atomic.AtomicReference<java.util.concurrent.ConcurrentHashMap<DataSourceBean.AEPLinkedKey,java.lang.Object>> storeLinked

Thread-safe (highly-concurrent) AEP-linked store; initialised on demand; never null after construction/deserialisation. May be cleared if the system becomes very short of memory.

storeUnlinked

private final java.util.concurrent.atomic.AtomicReference<java.util.concurrent.ConcurrentHashMap<DataSourceBean.UnlinkedKey,java.lang.Object>> storeUnlinked

Thread-safe (highly-concurrent) non-AEP-linked store; initialised on demand; never null after construction/deserialisation. May be cleared if the system becomes very short of memory.

_efh

private final DataSourceBean.EFH _efh

Set up emergency to clear these stores under extreme memory stress.

scorerKey

private static final DataSourceBean.UnlinkedKey scorerKey

The non-AEP-linked key for the Scorer cache; never null.

serialVersionUID

private static final long serialVersionUID

Unique Serialisation class ID generated by http://random.hd.org/.

See Also:

Constant Field Values

DataSourceBean

public DataSourceBean()

Public no-arg constructor for ease of use as a JavaBean. This defers as much work as it reasonably can.

However, if the system property CoreConsts.WAR_SYSPROPNAME_DISPOSITION_PRESET is set to a usable value, this may allow us to start the cache sooner.

getApplicationInstance

public static DataSourceBean getApplicationInstance(javax.servlet.ServletContext ctxt)

Factory method to create/return the unique application-level instance; never returns null. Makes an instance at application level, or returns an extant one if present.

Grabs appropriate locks to ensure that there are no data races and that there is at most one instance per servlet context.

The instance has the name ``dataSource'' and can be accessed from JSPs by that name.

Parameters:

ctxt - current servlet context; must not be null

getLogger

public SimpleLoggerIF getLogger()

Get logger; never null. Allow external users to use our managed logger.

log

public void log(java.lang.String message)

Log the given message. Allow external users to use our managed logger.

Specified by:

log in interface SimpleLoggerIF

poll

public final void poll(GenProps gp)
                throws java.io.IOException

Poll periodically (of the order of a second) to do background tasks. One job we do is post some properties where they can be found by other system components...

Not expected to be called by more than one thread at a time.

Specified by:

poll in interface SimpleExhibitPipelineIF

Throws:

java.io.IOException - in case of difficulty, but even if a sub-ordinate call throws IOException then poll() call should continue to do as much of its remaining work as reasonably possible

_precomputePopularPages

private void _precomputePopularPages(GenProps gp,
                                     long startTime)
                              throws java.io.IOException

Attempt to to precompute or keep like some pages likely to be popular/busy.

Throws:

java.io.IOException

_keepEPCMUpToDate

private void _keepEPCMUpToDate(GenProps gp,
                               long startTime,
                               boolean lightlyLoaded,
                               boolean overloaded)
                        throws java.io.IOException

Attempt to keep EPCM values up to date from within poll.

Throws:

java.io.IOException

_preloadBestAndNewest

private void _preloadBestAndNewest(GenProps gp,
                                   long startTime,
                                   SimpleExhibitPipelineIF pipeline,
                                   javax.servlet.ServletContext servletContext)
                            throws java.io.IOException

Attempt to preload the best and newest exhibits and/or keep them fresh/live.

Throws:

java.io.IOException

_postProps

private void _postProps()
                 throws java.io.IOException

Update and post properties to application level as required. Operates under the instance lock for thread safety.

Throws:

java.io.IOException

getStratum

public Stratum getStratum()
                   throws java.io.IOException

Description copied from interface: SimpleExhibitPipelineIF

Gets information about our stratum and upstream server; never null.

Specified by:

getStratum in interface SimpleExhibitPipelineIF

Throws:

java.io.IOException

destroy

public final void destroy()

Free up some system resources and make our poller thread go away. This instance is unusable once this methid has been called.

Specified by:

destroy in interface SimpleExhibitPipelineIF

isDestroyed

private boolean isDestroyed()

Check if destroyed...

addObserver

public void addObserver(java.util.Observer obs)

Register Observer to be told when exhibit data changes. This will probably be poll driven.

We may discard Observers at will, and will certainly do so if serialised and deserialised.

deleteObserver

public void deleteObserver(java.util.Observer o)

Remove an Observer. An Observer may have no further need to keep observing, eg because its job is done and it wants to be GCed.

isSlave

public java.lang.Boolean isSlave()

Get status of this server; master, slave or unknown. Once initialised for use the status is known and is either TRUE or FALSE; until then it is null indicating unknown. Once set non-null its value cannot be changed.

setSlave

private void setSlave(boolean isASlave)
               throws java.lang.IllegalStateException

Set disposition. If the parameter is true, the disposition is set to slave mode. If the parameter is false, the disposition is set to master mode.

If value is passed that conflicts with a non-null value already set, an IllegalStateException is thrown, ie the disposition cannot be changed once set.

Throws:

java.lang.IllegalStateException

setSlave

private void setSlave(java.lang.String masterSlave)
               throws java.lang.IllegalStateException

Set slave from context or property value. The value passed is ignored if null, and is not case-insensitively-equal to ``master'' or ``slave'' or ``waronly'', else it is set to slave or master mode as appropriate.

Throws:

java.lang.IllegalStateException

setContextPath

public void setContextPath(java.lang.String cPath)
                    throws java.lang.IllegalStateException

We examine the context path to see if we can tell if we are slave or master. We are definitely a master server if our context path is not the empty string. All slaves definitely have an empty-string context path, though a ``waronly'' master does too.

This must be set before calling any of the get...() methods of the bean (since we may need this to set up our cache, for example) and is idempotent so long as we call it with the same value. If we call it with a different value the call may be aborted.

Throws:

java.lang.IllegalStateException

isAggressive

public boolean isAggressive()

Check if we are going to be aggressive in cacheing. Defaults to false; is only valid once setServletContext() has been called.

setServletContext

public void setServletContext(javax.servlet.ServletContext context)

Set a valid servlet context. We don't really care which it is so long as is valid (eg not null).

We don't persist the value if the bean is serialised.

Note that if we are going to be aggressive and we already know if we are master or slave then we automatically start the data pipeline and thread if necessary, but not inside the instance lock.

If the context path is set before the servlet context and we are in aggressive mode then that will force an immediate pipeline start.

getServletContext

public javax.servlet.ServletContext getServletContext()

Get the servlet context associated with this bean; may be null. Thread-safe access.

_startBackgroundThread

private static void _startBackgroundThread(DataSourceBean dsb)

Start background thread...

_getPipeline

private SimpleExhibitPipelineIF _getPipeline()
                                      throws java.lang.IllegalStateException

Get data pipeline; never null. Will return any extant pipeline, or try to create one providing that the bean has been initialised and knows if this is a master (getting its data directly from EJB) or a slave (getting data from the master server).

Operates under the instance lock except to start any background thread.

Should be small enough to inline, statically or with JIT compiler.

Throws:

java.lang.IllegalStateException - if the cache cannot be created due to incorrect initialisation

_createPipeline

private SimpleExhibitPipelineIF _createPipeline()
                                         throws java.lang.IllegalStateException

Do the heavy-lifting of creating the data pipeline; never null. Should be called (about) once during the life of any DSB instance.

Returns:

the non-null pipeline

Throws:

java.lang.IllegalStateException - if the cache cannot be created due to incorrect initialisation

getStaticAttr

@Deprecated
public ExhibitStaticAttr getStaticAttr(java.lang.CharSequence name)
                                throws java.io.IOException

Deprecated. use ExhibitFull version if possible for efficiency

Get the static attributes for a given exhibit. Returns null if the named exhibit does not exist.

Convenience method supporting wider variety of types including older String-valued argument.

Throws:

java.io.IOException

getStaticAttr

public ExhibitStaticAttr getStaticAttr(Name.ExhibitFull name)
                                throws java.io.IOException

Get the static attributes for a given exhibit. Returns null if the named exhibit does not exist.

Specified by:

getStaticAttr in interface SimpleExhibitPipelineIF

Throws:

java.io.IOException - if the operation cannot be completed due to I/O restrictions or failure

getRawFile

public void getRawFile(java.nio.ByteBuffer buf,
                       Name.ExhibitFull exhibitName,
                       int position,
                       boolean dontCache)
                throws java.io.IOException

Get a chunk of the raw exhibit binary. The start index and the index after the last required byte is supplied. The start value must be non-negative and the afterEnd value no smaller than start and no larger than the exhibit data length.

Specified by:

getRawFile in interface SimpleExhibitPipelineIF

Parameters:

buf - the buffer into which to read the data; must be non-null, in put()able state, and with remaining capacity of at least the requested number of bytes

exhibitName - the full name of the exhibit to read from; never null and must be syntactically valid

position - position/index of first byte in exhibitFile to read; non-negative

dontCache - if true, this is a hint not to attempt to cache this or displace anything from extant caches for this data as it may for example be precaching or random activity; by default callers should leave this false to allow cacheing

Throws:

java.io.IOException - for requests that cannot be fulfilled because of I/O restrictions or problems, such as link failure or an upper bound on the length of a request

getAllExhibitImmutableData

public AllExhibitImmutableData getAllExhibitImmutableData(long oldStamp)
                                                   throws java.io.IOException

Gets all static exhibit data if its timestamp is not that specified. If the time specified is negative the object will be returned unconditionally.

If no exhibits are currently installed a default set with a zero timestamp is returned.

If the caller's copy appears to be up-to-date (eg the oldStamp matches that that would have been returned) null is returned.

Specified by:

getAllExhibitImmutableData in interface SimpleExhibitPipelineIF

Throws:

java.io.IOException - if the operation cannot be completed due to I/O restrictions or failure

getAllExhibitProperties

public AllExhibitProperties getAllExhibitProperties(long oldHash)
                                             throws java.io.IOException

Gets set of all exhibit properties if its hash is not that specified. If the hash specified is negative the object will be returned unconditionally.

If no exhibits are currently installed a default set with a zero timestamp is returned.

If the caller's copy appears to be up-to-date (eg the oldHash matches that that would have been returned then null is returned.

Specified by:

getAllExhibitProperties in interface SimpleExhibitPipelineIF

Throws:

java.io.IOException - if the operation cannot be completed due to I/O restrictions or failure

exhibitDataVersionHash

public final int exhibitDataVersionHash()
                                 throws java.io.IOException

Computes a hash on the versions of immutable and mutable exhibit data currently held. This value is computed on the timestamps/hashes of the exhibit data and properties available through this bean. If this hash changes then some of the underlying exhibit data has changed and users of the bean should recompute values based on this bean.

Throws:

java.io.IOException

_getJIB

private ORG.hd.d.jIndexer.server.JIndexBean _getJIB()
                                             throws java.io.IOException

Get the by-word-index, possibly recreating it (without blocking if possible) if not cached or if out of date; null if not available. This is created from the full exhibit names and any extant description.

We know all names to be 7-bit (printable) ASCII, and assume all descriptions to be either 7-bit ASCII or at most 8-bit ISO-8859-1 (Latin-1) text, and we use that when converting to a byte stream for indexing.

This may not force a synchronous rebuild since this may take too long, and therefore may return a stale or null index while a new one is being computed. This may also defer a build temporarily if the system is heavily loaded.

THUS: keys/docnames held in the index must be relatively stable and sane even across a (minor) change of AEP, eg exhibit short names or equivalent, so that a lookup result is still very likely to make sense.

Throws:

java.io.IOException

byWordIndexIsAvailable

public boolean byWordIndexIsAvailable()

Returns true if there is a by-word index available. Returns false until one is built or loaded.

If this returns true it does not guarantee that the index is up-to-date, simply that one exists and can be used, probably without blocking.

byWordIndexIsAvailableAndUpToDate

public boolean byWordIndexIsAvailableAndUpToDate()

Returns true if there is a by-word index available and it is up-to-date. Returns false until one is (re)built or loaded.

_rebuildJIB

private void _rebuildJIB()
                  throws java.io.IOException

Rebuild the JIB (index bean) if necessary and possible, avoiding blocking if possible. Thread safe, and synchronized on a private lock in order to avoid wasting CPU cycles with multiple update attempts.

Directly updates the cache used by _getJIB() if the index needs recomputing, ie if it is null or out of date wrt the AEP.

If a second thread tries to enter while one is already rebuilding the index, then the second thread returns immediately (does not block), but meaning that the cache will not be guaranteed rebuilt by the time that the second thread returns, so any caller has to be prepared to deal with the cache still null when this routine returns.

If the system is too busy then we will not attempt to (re)build the index.

Note that normally a rebuild is attempted in the background and at low priority and may be vetoed if the system is busy. However, were there is no index present at all this will force a higher-priority build and may possibly block until done.

Throws:

java.io.IOException

computeByWordIndex

public static ORG.hd.d.jIndexer.server.JIndexBean computeByWordIndex(AllExhibitProperties aep)
                                                              throws java.io.IOException,
                                                                     java.lang.InterruptedException

Computes a (compact) by-word index of the current exhibits; never null. This is based on the exhibit names, descriptions, and any "AKA" (Also-Known-As) keyword data.

Returns:

compacted by-word index

Throws:

java.io.IOException

java.lang.InterruptedException

canonicaliseSimpleByWordQuery

public static java.lang.CharSequence canonicaliseSimpleByWordQuery(java.lang.CharSequence s,
                                                                   int maxLength)

Canonicalises a simple by-word query string and sanitises it; useful for potentially-unsafe user input. This not only potentially makes the query string small(er), it also shows the query that is actually being performed, whatever the user thinks that they are doing!

This also converts the query string to pure ASCII, having first converted any non-ISO-Latin-1 characters to spaces, trimmed any obvious whitespace, and truncated to the given length (if non-negative).

This can be directly handed the result of an HTTP request parameter, for example. The output is suitable to hand to findExhibitsByWord().

This keeps words in the order supplied especially the first.

This returns an empty sequence (eg "") if the input is null.

If the input string does not need transforming then it is returned as-is.

Any new CharSequence returned is immutable.

findExhibitsByWord

public java.util.List<Name.ExhibitFull> findExhibitsByWord(java.lang.String query,
                                                           int matchType,
                                                           int maxResults)
                                                    throws java.io.IOException

Perform simple query on the exhibit data without filtering. The query should be a simple whitespace-separated string of query words.

This returns an immutable ranked list of (full) exhibit names; possibly zero-length if no matches (but never null).

The underlying index is (re)built on first use or when the underlying data has changed.

This may take longer than we'd like and use more memory than we'd like.

Parameters:

query - a plain-text query of search words, space-separated

matchType - the number of input words to match; -1 means any word, 0 means most words, 1 means all words.

maxResults - is the maximum number of results returned; should be at least twice the maximum shown to reduce the risk of having abandoned good results prematurely

Throws:

java.io.IOException

findExhibitsByWord

public java.util.List<Name.ExhibitFull> findExhibitsByWord(java.lang.CharSequence query,
                                                           int matchType,
                                                           int maxResults,
                                                           ORG.hd.d.jIndexer.server.JIndexBean.SearchFilterByName docFilter)
                                                    throws java.io.IOException

Perform simple query on the exhibit data with candidate filtering; never null. The query should be a simple whitespace-separated string of query words.

This returns an immutable ranked list of (full) exhibit names; possibly zero-length if no matches (but never null).

The underlying index is (re)built on first use or when the underlying data has changed.

This may take longer than we'd like and use more memory than we'd like.

Parameters:

query - a plain-text query of search words, space-separated

matchType - the number of input words to match; -1 means any word, 0 means most words, 1 means all words.

maxResults - is the maximum number of results returned; should be at least twice the maximum shown to reduce the risk of having abandoned good results prematurely

docFilter - if non-null, any documents for which the accept method returns false are excluded from the search

Throws:

java.io.IOException

getGenProps

public GenProps getGenProps(long oldStamp)
                     throws java.io.IOException

Gets the general properties as a GenProps object if its timestamp is not that specified. If the time specified is negative the object will be returned unconditionally.

If no props are currently installed/available then a default set with a zero timestamp is returned.

If the caller's copy appears to be up-to-date (eg the oldStamp matches that that would have been returned) then null is returned.

Specified by:

getGenProps in interface SimpleExhibitPipelineIF

Throws:

java.io.IOException - if the operation cannot be completed due to I/O restrictions or failure

getGenSecProps

public java.util.Properties getGenSecProps(long oldStamp)
                                    throws java.io.IOException

Gets the security properties as a Properties object if its timestamp is not that specified. If the time specified is negative the object will be returned unconditionally.

If no props are currently installed/available a default set with a zero timestamp is returned.

If the caller's copy appears to be up-to-date (eg the oldStamp matches that that would have been returned) then null is returned.

Specified by:

getGenSecProps in interface SimpleExhibitPipelineIF

Throws:

java.io.IOException - if the operation cannot be completed due to I/O restrictions or failure

getThumbnails

public ExhibitThumbnails getThumbnails(Name.ExhibitFull name,
                                       boolean create)

Gets the thumbnails for an exhibit. A data source is at liberty to refuse to compute thumbnails in which case it may return null, else it returns a non-null value which may include the `could-not-compute' value to indicate that a thumbnail/sample cannot be made for this exhibit and no attempt need be made in future.

As a kindness to callers, we convert any IOException thrown to returning a null instead, to fold the two cases together.

Specified by:

getThumbnails in interface SimpleExhibitPipelineIF

Parameters:

create - if true, and no thumbnail yet exists, try to create one if possible, else only return an existing one

setVariable

public void setVariable(SimpleVariableValue newValue)
                 throws java.io.IOException

Description copied from interface: BasicVarMgrInterface

Set variable to the given value (the variable name and definition are implicit). The value set should be immediately readable with getVariable() as the "main" value of the variable even for global variables.

This is also used to send events, the last event value being the variable's value.

Specified by:

setVariable in interface BasicVarMgrInterface

Throws:

java.io.IOException - in case of I/O difficulty

setVariables

public int setVariables(SimpleVariableValue[] newValues)
                 throws java.io.IOException

Description copied from interface: BasicVarMgrInterface

Update a number of variables at once for efficiency; returns the number of variables set. Is passed a set of SimpleVariableValues and behaves as if it operates on all of them by calling setVariable() for each item in the set.

Furthermore it behaves as if the set operations are done in order, from low index to high index, unless variables are coalesced (multiple sets of the same simple variable can keep just the last value). Set operations of different variables may be re-ordered with respect to one another in order that redundancy may reduced for transmission and storage, so this implicitly allows, for example, events to be re-ordered with respect to non-event variables.

An implementation may "fail fast" on the first error/exception, or may attempt to continue and do as much as possible.

An implementation may throw an IllegalArgumentException on attempt to: set a variable with value of wrong type or incompatible definition, set a non-existent or read-only variable (or these may be ignored)

Specified by:

setVariables in interface BasicVarMgrInterface

Returns:

the number of variable values set (the length of the array); never negative, never more than the number passed in

Throws:

java.io.IOException

getVariable

public SimpleVariableValue getVariable(SimpleVariableDefinition var)
                                throws java.io.IOException

Description copied from interface: BasicVarMgrInterface

Get a single variable value; returns null if no such value or wrong type. Implementations should, where possible, trim from the globalMap of returned values any apparently-stale data, so as to return only reasonably-live data to the caller.

Only the last event value for a given event variable is returned by this method.

Specified by:

getVariable in interface BasicVarMgrInterface

Parameters:

var - definition of variable to fetch; never null

Throws:

java.io.IOException - in case of I/O difficulty

getVariables

public SimpleVariableValue[] getVariables(long changedSince)
                                   throws java.io.IOException

Description copied from interface: BasicVarMgrInterface

Get set of variable values altered on or after specified time, or get all values with -1; never null. This may be slow if there are many live variables.

Only the last event value of each event variable is returned by this method.

Specified by:

getVariables in interface BasicVarMgrInterface

Throws:

java.io.IOException - in case of I/O difficulty

getEventValue

public EventVariableValue getEventValue(SimpleVariableDefinition def,
                                        EventPeriod intervalSelector,
                                        boolean current)

Get the current partial, or previous full, event set at the specified interval; never null. This is a simplified interface to return either the current event set that is being collected, or the previous completed set.

The current set is the most timely, but may not contain enough data to be meaningful if the new interval has just started.

The previous set is complete and thus most likely to have enough samples to be useful, but is not completely current.

Specified by:

getEventValue in interface BasicVarMgrInterface

Parameters:

def - event definition (must be for an event); never null

intervalSelector - one of EVENT_INTERVAL_SELECTOR_xxx values

current - if true the current event set is returned, else the previous complete set is returned

Returns:

requested event set; may be empty but not null if requested set not available

getEventValues

public EventVariableValue[] getEventValues(SimpleVariableDefinition def,
                                           EventPeriod intervalSelector,
                                           long intervalNumber,
                                           java.util.BitSet whichValues)

Get the specified event sets for the specified intervals; never null. This allows retrieval of zero or more event sets for the specified interval size.

Requests for more than SystemVariables.EVENT_SAMPLES_RETAINED in the past (or for the future!) cannot be satisfied and data will not be returned for them.

Usually not more than SystemVariables.EVENT_SAMPLES_RETAINED samples will be returned in response to any one request as a safety measure.

(An implementation that is not an end-point may go upstream to fetch missing values and cache them to satisfy future requests.)

Specified by:

getEventValues in interface BasicVarMgrInterface

Parameters:

def - event definition (must be for an event); never null

intervalSelector - one of EVENT_INTERVAL_SELECTOR_xxx values

intervalNumber - a time (as from System.currentTimeMillis()) which identifies the first interval for which data is potentially required; if too far in the past or future then possibly no data will be available, zero is used to access the "all" bucket

whichValues - each true bit represents a slot for which data is required, bit 0 indicating data from the slot within which firstIntervalTime is located, bit 1 the previous slot, etc

Returns:

as many of the requested values as available, at least long enough to return all the available values, with [0] corresponding to bit 0 in the BitSet; may contain nulls or be zero-length but is never null

syncVariables

public void syncVariables(boolean force)
                   throws java.io.IOException

Synchronise variables with upstream values. Pushes updated values upstream to the source, calls sync on the source if called with the "force" argument true, and then retrieves changed values from upstream.

When called with force==true, this acts like a full "memory barrier", flushing all write-cached items downstream immediately and afterwards getting the value of all upstream values with getVariables(-1), but may be expensive in terms of CPU or bandwidth, so use sparingly.

When called with force=false, this incrementally flushes outstanding writes and will then fetch all, or only new, values from upstream, so is potentally much less resource-intensive. In particular, this does not propagate the sync() upstream.

In any case, it is rarely the right thing for a casual user to vall this as it may be very expensive.

Specified by:

syncVariables in interface SimpleVariablePipelineIF

Parameters:

force - if true, this will force a full write flush, a full sync upstream, then full read with getVariables(-1), to get the effect of a full "barrier"; otherwise, in general, a more incremental and non-propagating mode is used which still does a write flush but may chose to do a partial read of "new" upstream values

Throws:

java.io.IOException - if one is received from upstream

getProperties

public java.util.Properties getProperties(SimpleExhibitPipelineIF.PropsKey key,
                                          long versionID)
                                   throws java.io.IOException

Get requested Properties selected by key and versionID. Fetches a Properties set unconditionally (versionID == -1) else if the versionID presented is not current.

Specified by:

getProperties in interface SimpleExhibitPipelineIF

Parameters:

key - selector (with possible embedded sub-key) for desired properties set; never null

versionID - if -1 then map is always returned if available, else must be non-negative and null is returned if the versionID presented matches that of the current version (ie if the caller has presumably got the up-to-date version); may be a timestamp or a hash or other value, and by convention is zero only for an empty properties set

Returns:

null, or Properties map guaranteed to contain only String keys and values

Throws:

java.io.IOException

getStoreLinked

private java.util.concurrent.ConcurrentHashMap<DataSourceBean.AEPLinkedKey,java.lang.Object> getStoreLinked()

Get AEP-linked store, creating if necessary; never null.

getAEPLinkedValue

public java.lang.Object getAEPLinkedValue(DataSourceBean.AEPLinkedKey key)

Get value against supplied key in AEP-linked store; result may be null. May be cleared at any time, and will be cleared when the AEP changes.

This is designed for good concurrency and as little locking as possible.

putAEPLinkedValue

public java.lang.Object putAEPLinkedValue(DataSourceBean.AEPLinkedKey key,
                                          java.lang.Object value)

Store value against supplied key in AEP-linked store. A null value is used to clear any extant entry.

This is designed for good concurrency and as little locking as possible.

This may be cleared automatically if we become very short of memory.

Returns:

any previous value, or null if none

replaceAEPLinkedValue

public boolean replaceAEPLinkedValue(DataSourceBean.AEPLinkedKey key,
                                     java.lang.Object oldValue,
                                     java.lang.Object newValue)

Replace value for supplied key in AEP-linked store. This (atomically) replaces the value for the given key only if the extant value is that supplied.

Null values are not allowed.

This is designed for good concurrency and as little locking as possible.

This may be cleared automatically if we become very short of memory.

Returns:

true if the value was replaced

removeAEPLinkedValue

public java.lang.Object removeAEPLinkedValue(DataSourceBean.AEPLinkedKey key)

Remove any value associated with the supplied key in AEP-linked store.

putIfAbsentAEPLinkedValue

public java.lang.Object putIfAbsentAEPLinkedValue(DataSourceBean.AEPLinkedKey key,
                                                  java.lang.Object value)

Store value against supplied key in AEP-linked store only if no value already mapped for that key. A null value is used to clear any extant entry.

This is designed for good concurrency and as little locking as possible.

This may be cleared automatically if we become very short of memory.

Returns:

any previous value, or null if none

getStoreUnlinked

private java.util.concurrent.ConcurrentHashMap<DataSourceBean.UnlinkedKey,java.lang.Object> getStoreUnlinked()

Get non-AEP-linked store, creating if necessary; never null.

getUnlinkedValue

public java.lang.Object getUnlinkedValue(DataSourceBean.UnlinkedKey key)

Get value against supplied key in non-AEP-linked store; result may be null. This is designed for good concurrency and as little locking as possible.

putUnlinkedValue

public java.lang.Object putUnlinkedValue(DataSourceBean.UnlinkedKey key,
                                         java.lang.Object value)

Store value against supplied key in non-AEP-linked store. A null value is used to clear any extant entry.

This is designed for good concurrency and as little locking as possible.

This may be cleared automatically if we become very short of memory.

Returns:

any previous value, or null if none

replaceUnlinkedValue

public boolean replaceUnlinkedValue(DataSourceBean.UnlinkedKey key,
                                    java.lang.Object oldValue,
                                    java.lang.Object newValue)

Replace value for supplied key in non-AEP-linked store. This (atomically) replaces the value for the given key only if the extant value is that supplied.

Null values are not allowed.

This is designed for good concurrency and as little locking as possible.

This may be cleared automatically if we become very short of memory.

Returns:

true if the value was replaced

removeUnlinkedValue

public java.lang.Object removeUnlinkedValue(DataSourceBean.AEPLinkedKey key)

Remove any value associated with the supplied key in non-AEP-linked store.

putIfAbsentUnlinkedValue

public java.lang.Object putIfAbsentUnlinkedValue(DataSourceBean.UnlinkedKey key,
                                                 java.lang.Object value)

Store value against supplied key in non-AEP-linked store only if no value already mapped for that key. A null value is used to clear any extant entry.

This is designed for good concurrency and as little locking as possible.

This may be cleared automatically if we become very short of memory.

Returns:

any previous value, or null if none

getScorerCache

public ScorerCacheIF getScorerCache()

Get the Scorer cache; never null. The cache may be discarded under extreme memory stress and will be atomically (re)created as necessary.

We may restrict access in future.

readObject

private void readObject(java.io.ObjectInputStream ois)
                 throws java.io.IOException,
                        java.lang.ClassNotFoundException

Deserialise.

Throws:

java.io.IOException

java.lang.ClassNotFoundException

validateObject

public void validateObject()
                    throws java.io.InvalidObjectException

Validate fields/state. Called in the constructor and possibly after de-serialising.

Barf if something bad is found. (Maybe allow some extra info in debug version.)

Throws:

java.io.InvalidObjectException