org.hd.d.pg2k.ai.scorer
Interface ScorerCacheIF

All Known Implementing Classes:

AbstractScorerCache, MiniScorerCacheImpl, ScorerCacheImpl

public interface ScorerCacheIF

Base interface to compute (and cache) the score and confidence for exhibits. Note: since the result of this computation may be used in computing (EPCM) the ExhibitPropsComputableMutable value for an exhibit, then any implementation of this must avoid forcing recaclulation of any EPCM value to avoid danger of infinite recursion (other than the static calcVoteFactor() method). Ideally the value computed will not depend on any EPCM value.

Methods that take an allowStale parameter will generally try to reduce compute and I/O effort and increase robustness at a smallish cost in currency. Typically such methods will return data up to about one day old if present in cache, else the newest available from cache however old it is if there is a problem computing an up-to-date value (eg because of connectivity issues).

TRIVIAL

static final ScorerCacheIF TRIVIAL

Trivial implementation that always returns "no comment"/empty/null values; non-null.

computeCompositeScoreAndConfidence

ScoreAndConf computeCompositeScoreAndConfidence(java.lang.String exhibitName,
                                                boolean allowStale)
                                                throws java.io.IOException

Computes a weighted composite score [-1,+1] and confidence [0,+1] for the specified exhibit with the best available scorers/parameters; never null but may be (0,0). This is the highest-level available entry to the cache.

Parameters:

exhibitName - valid full exhibit name

allowStale - if true then allow a stale value from cache, else throw an exception if nothing is currently available

Returns:

(0,0) if named scorer is not available, cannot be used with the specified parameters, or cannot be applied to the indicated exhibit (eg because of the exhibit type or the exhibit does not exist); else returns a non-null ScoreAndConf value

Throws:

java.io.IOException

computeUnweightedScoreAndConfidence

ScoreAndConf computeUnweightedScoreAndConfidence(java.lang.String exhibitName,
                                                 ScorerIF scorer,
                                                 boolean allowStale)
                                                 throws java.io.IOException

Computes the raw score [-1,+1] and confidence [0,+1] for the specified exhibit with the specified scorer and parameters; never null but may be (0,0). This is NOT moderated/weighted in the light of how well the scorer's predictions match reality; that requires an extra step.

Parameters:

exhibitName - valid full exhibit name

scorer - instance of the Scorer; never null

allowStale - if true then allow a stale value from cache, else throw an exception if nothing is currently available

Returns:

(0,0) if named scorer is not available, cannot be used with the specified parameters, or cannot be applied to the indicated exhibit (eg because of the exhibit type or the exhibit does not exist); else returns a non-null ScoreAndConf value

Throws:

java.io.IOException

computeScorerWeighting

ScoreAndConf computeScorerWeighting(java.lang.String scorerNameAndParameters,
                                    boolean allowStale,
                                    java.lang.String source)
                                    throws java.io.IOException

ScoreAndConfidence for the given Scorer over all exhibit types; never null but may be (0,0) where the scorer is unknown or untested. Essentially the result of this should be multiplied by the result for each exhibit (for the same scorer and parameters) to normalise the predicted score and confidence for the exhibit.

Some results will be cached (typically those from the base/current Scorer lists) while other may have to be computed each time, which may be slow.

The source parameter can be used to track which mechanisms are most effective at generating improvements in the population, and could, for example, to spend more time on those that are most effective right now.

Parameters:

scorerNameAndParameters - the name and parameters of the scorer; never null

allowStale - if true then allow a stale or low-confidence value from cache, else throw an exception if nothing is currently available and we cannot quickly compute enough points to increase our confidence

source - the name of the mechanism used to generate this Scorer value, or null if none

Returns:

the score represents the correlation with the underlying votes (and whatever the scoring is measured against) with MAX meaning perfect correlation, 0 meaning no correlation, and -MAX meaning perfectly wrong answers all the time, and the confidence 0 if we have no (or very/too few) data points and approaching MAX as we have a large (enough) number of data points

Throws:

java.io.IOException

computeScorerWeighting

ScoreAndConf computeScorerWeighting(ScorerIF scorer,
                                    boolean allowStale,
                                    java.lang.String source)
                                    throws java.io.IOException

ScoreAndConfidence for the given Scorer over all exhibit types; never null but may be (0,0) where the scorer is unknown or untested. Essentially the result of this should be multiplied by the result for each exhibit (for the same scorer and parameters) to normalise the predicted score and confidence for the exhibit.

Some results will be cached (typically those from the base/current Scorer lists) while other may have to be computed each time, which may be slow.

The source parameter can be used to track which mechanisms are most effective at generating improvements in the population, and could, for example, to spend more time on those that are most effective right now.

The main value of this variant over the String 'name-and-parameters' method is that the Scorer instance supplied is used, which avoids constructing a new instance and may enable state to be shared better.

Parameters:

scorer - instance of the Scorer; never null

allowStale - if true then allow a stale or low-confidence value from cache, else throw an exception if nothing is currently available and we cannot quickly compute enough points to increase our confidence

source - the name of the mechanism used to generate this Scorer value, or null if none

Returns:

the score represents the correlation with the underlying votes (and whatever the scoring is measured against) with MAX meaning perfect correlation, 0 meaning no correlation, and -MAX meaning perfectly wrong answers all the time, and the confidence 0 if we have no (or very/too few) data points and approaching MAX as we have a large (enough) number of data points

Throws:

java.io.IOException

getBaseScorersWithoutParameters

java.util.Set<java.lang.String> getBaseScorersWithoutParameters()

Base set of available Scorers' names (no parameters); never null but may be empty. The values returned are of the form ScorerName.

getBaseScorerByName

ScorerIF getBaseScorerByName(java.lang.String baseName)

Get base non-parameterised Scorer by name; null if no such base Scorer supported.

Parameters:

baseName - base (no parameters) name of Scorer; must not be null

getCurrentScorersWithParameters

java.util.Set<java.lang.String> getCurrentScorersWithParameters(boolean allowStale)

Current set of available Scorers name and parameters (where applicable); never null but may be empty. The values returned are of the form ScorerName{:name=value}*.

The scorers returned by this will generally be the best available, usually the best one or two per base-Scorer type, allowing in particular that different Scorer types may have different domains.

This call should not be desperately expensive, eg will not generally be doing any evolution/scoring, but may not be very quick/cheap either as it may require some search and sort.

extractCalibrationSet

java.util.Map<java.lang.String,ScoreAndConf> extractCalibrationSet(java.lang.String baseName,
                                                                   int maxSamples,
                                                                   java.lang.Boolean difficult,
                                                                   boolean allowStale)
                                                                   throws java.io.IOException

Compute exemplar exhibit sub-set to calibrate Scorers with given base name against; never null but may be empty. These may be exhibits for which we get particularly good or bad predictions, or a random sub-set.

An implementation may return an empty result (not null) if it cannot compute this value.

It may be possible to tune or pre-test new Scorers against the results of this as a fast filter.

If a base name is specified that is invalid, it is treated as if null.

Parameters:

baseName - base name of Scorer to extract calibration set for, or null for a generic all-Scorers calibration set

maxSamples - the maximum number of samples to return; strictly positive

difficult - if TRUE the return the difficult cases that we do not predict well, if FALSE then return the easy cases that we predict well, else return a mixure of good, bad, and other random cases

allowStale - if true then allow slightly older data for speed and robustness

Returns:

map of zero-or-more exhibits (short names) to calibration-data values; non-null

Throws:

java.io.IOException

size

int size()

Get the current population size; non-negative.

getScorerInstance

ScorerIF getScorerInstance(java.lang.String nameAndParameters)

Get Scorer instance given the Scorer{:value=name}* format; null if no such Scorer available. Any instance returned may be a shared/cached instance rather than a new instance.

offerExternalScorer

boolean offerExternalScorer(java.lang.String externalScorerNameAndParameters)

Non-blocking attempt to queue an externally-supplied Scorer value; returns true if accepted. The input values must be fully validated and canonicalised before be used, but it is useful if some minimal screening is done before submitting values to this routine to prevent (for example) pointless excessive resource consumption.

A typical implementation that accepts inbound Scorer values would bound this with a bounded-size non-blocking queue.

A given implementation may always return false (ie never accept inbound Scorer values).

canAcceptMoreExternalScorers

boolean canAcceptMoreExternalScorers()

Returns true if this cache can definitely accept (many) more externally-supplied Scorer values. Even if this returns false we may in practice be able to accept one or more new values: this is indicative.

An implementation that cannot accept any external Scorer values must always return false.

Typically this is true if any internal bounded-size queue has a lot of space left, eg is half empty, but this is not a guarantee that another value will actually be accepted.

hasQueuedExternalScorer

boolean hasQueuedExternalScorer()

Returns true if at least once external Scorer is queued waiting to be processed.