org.hd.d.pg2k.webSvr.ads
Class AdUtils

java.lang.Object
  org.hd.d.pg2k.webSvr.ads.AdUtils

public final class AdUtils
extends java.lang.Object

Advertising-related utility functions. One advantage of having code here rather than in-line in a JSP is that it is pre-compiled off-line for speed and robustness; code here is also potentially easier to test.

KHIT_THRESHOLD

private static final int KHIT_THRESHOLD

Threshold (in thousands) when we switch from logging individual hits/clicks to thousands; strictly positive. Raising this value improves accuracy when stats come in slowly and/or servers are frequently restarted, but increases stats traffic and log size.

Ideally this should be large enough that loss of a nearly full "k-click" should not be hugely significant, which implies a value of maybe 10 or more.

We should set this as low as we can bear in terms of potential statistical inaccuracy to keep log traffic reasonable.

See Also:

Constant Field Values

LOCAL_AD_STATS_WEIGHTING

public static final int LOCAL_AD_STATS_WEIGHTING

Extra weighting in ad layout/format decisions given to local hits/stats; non-negative. Although all ad layout and click-through stats are recorded globally, some of them are also recorded locally, in a somewhat volatile form. (Such local stats might be lost at an upgrade for example, or be less robust in the face of local disc errors.)

A weighting of zero means to not treat local stats specially, and just use the global stats (which includes the local stats).

A weighting of 1 gives local stats equal weight to global stats, though clicks (etc) are recorded once in the global stats already, so that means that local stats are effectively given somewhat higher weight.

Note that the weighting is usually in terms of the entire eCPM/eCTR/etc (providing that there are enough local pageviews to keep noise acceptable, else the local stats may be ignored or used in a more-noise-tolerant way), and thus even low-traffic mirrors can show significant favour to local mores.

See Also:

Constant Field Values

MIN_LOCAL_STATS_PAGEVIEWS

private static final int MIN_LOCAL_STATS_PAGEVIEWS

Minimum number of pageviews to accept one of the local stats values; strictly positive. Because the stats system switches from recording individual page views and clicks to recording thousands of page views and clicks at a round 1000 mark, this value should be either 1000 or less (though that may be too small) or a multiple of 1000 (but not so high as to exclude local stats entirely), or enough below the KHIT_THRESHOLD to be immune to the large-count lumpiness.

NB: Google AdWords seems to regard 1000 as a minimum sensible sample size.

A value of 1000 up to some fraction of our KHIT_THRESHOLD may be good.

TOP_N_RANK_AD_LIMIT

private static final int TOP_N_RANK_AD_LIMIT

Ranking threshold assumed to indicate "well" ranked amongst many peers; strictly positive.

LITE_AD_SHOW_FRACTION

private static final int LITE_AD_SHOW_FRACTION

What fraction of ad slots are shown in "lite" mode; strictly positive.

ADSLOTMONIOR_TIME_CONST

private static final int ADSLOTMONIOR_TIME_CONST

Time constant to use for ageing any memory of ad slot run times; strictly positive.

See Also:

Constant Field Values

MAX_SIGNIFICANT_URI_PREFIX

private static final int MAX_SIGNIFICANT_URI_PREFIX

Maximum leading portion of request URI that we will consider significant. This is based on the maximum legitimate exhibit name length (for when all or part of an exhibit name is embedded in a URI) plus an allowance for overhead.

See Also:

Constant Field Values

CLICKTHRU_MIN_WAIT_TIME_MS

private static final int CLICKTHRU_MIN_WAIT_TIME_MS

Time we wait for a user to click through on ads (of the order of minutes), in ms.

BEST_AD_LAYOUT_SAMPLE_PERIOD_MIN_MS

private static final long BEST_AD_LAYOUT_SAMPLE_PERIOD_MIN_MS

Minimum sample period to decide which is best-performing layout, ms; strictly positive. This should be long enough to cover:

• At least one rotation of all possible major ad-layout styles.
• At least one rotation of all major/minor ad-layout styles together.
• At least one day to allow for daily traffic cycles/patterns.
• At least one week/month to allow for traffic cycles/patterns.
• Long enough to survive glitches such as frequent restarts that "lose" low-frequency events.

but not so long as to incur a huge calculation penalty nor make it very slow to respond to changes.

(It may be wise to factor in a small component of the all-time numbers.)

A length of just over a week-multiple (to give some extra emphasis to patterns pertient to the day just coming up) may be good.

See Also:

Constant Field Values

BEST_AD_LAYOUT_SAMPLE_PERIOD_MIN_VLONG_UNITS

private static final int BEST_AD_LAYOUT_SAMPLE_PERIOD_MIN_VLONG_UNITS

Ad-layout selection period expressed in VLONG units; strictly positive.

See Also:

Constant Field Values

WT_LAST_WEEK

private static final int WT_LAST_WEEK

Extra weighting given to "within last week" clickthrough stats (normal==1); strictly positive. This is based on the assumption that recent behaviour is the most telling of the neat future.

See Also:

Constant Field Values

WT_THIS_DAY_LAST_WEEK

private static final int WT_THIS_DAY_LAST_WEEK

Extra weighting given to "this day last week" clickthrough stats (normal==1); strictly positive. This is based on the assumption that surfers' behaviour shows a weekly cycle.

See Also:

Constant Field Values

WT__VLONG_UNITS

private static final int WT__VLONG_UNITS

Recent-stats weighting period (1 week) expressed in VLONG units; strictly positive.

pvTowerInternalFormatHits

private static final java.util.EnumMap<AdTowerInternalLayout,java.util.concurrent.atomic.AtomicInteger> pvTowerInternalFormatHits

Counters of page/ad basic tower format used, indexed by AdTowerInternalLayout; never null. All elements are initialised to 0 at start-up, so no element is null and the map itself is never changed, and is implicitly thread-safe when used read-only this way.

All values will be non-negative, and usually less than 1000.

FIXME: should be keyed from DataSourceBean to allow for multiple sites in one VM.

_cache_sAMPIL_key

private static final DataSourceBean.UnlinkedKey _cache_sAMPIL_key

Key for thread-safe cache from Web site (DataSourceBean) to computed results for selectAdMidPageInternalLayout(); never null.

PREF_MP_FORMAT

public static final AdMidPageInternalLayout PREF_MP_FORMAT

Preferred mid-page ad format.

AMPI_NFORMATS

private static final int AMPI_NFORMATS

Number of distinct mid-page ad formats; strictly positive.

pvMPInternalFormatHits

private static final java.util.EnumMap<AdMidPageInternalLayout,java.util.concurrent.atomic.AtomicInteger> pvMPInternalFormatHits

Counters of page/ad basic tower format used, indexed by AdMInternalLayout; never null. All elements are initialised to 0 at start-up, so no element is null and the map itself is never changed, and is implicitly thread-safe when used read-only this way.

All values will be non-negative, and usually less than 1000.

FIXME: should be keyed from DataSourceBean to allow for multiple sites in one VM.

_cache_sATIL_key

private static final DataSourceBean.UnlinkedKey _cache_sATIL_key

Key for thread-safe cache from Web site (DataSourceBean) to computed results for selectAdTowerInternalLayout(); never null.

_cache_sPAL_key

private static final DataSourceBean.UnlinkedKey _cache_sPAL_key

Key for thread-safe cache from Web site (DataSourceBean) to computed results for selectPageAdLayout(); never null.

pvLayoutHits

private static final java.util.EnumMap<AdTowerPos,java.util.concurrent.atomic.AtomicInteger> pvLayoutHits

Counters of page/ad basic layout-style used, indexed by AdTowerPos; never null. All elements are initialised to 0 at start-up, so no element is null and the map itself is never changed, and is implicitly thread-safe when used read-only this way.

All values will be non-negative, and usually less than 1000.

FIXME: should be keyed from DataSourceBean to allow for multiple sites in one VM.

DA_MAX_EXHIBITS_TO_SHOW

private static final int DA_MAX_EXHIBITS_TO_SHOW

Maximum exhibits to show in drop-in ad-tower replacement text; strictly positive.

See Also:

Constant Field Values

AdUtils

private AdUtils()

Prevent construction of an instance.

showAds

public static final boolean showAds(DataSourceBean dsb,
                                    javax.servlet.http.HttpServletRequest request)

Show ads or not on the current page and/or in the current slot? Returns false if ads are generally disabled, etc, though even in the case that the user session has been set to "lite" mode, will return true a small fraction of the time so as to show occasional ads.

We don't show ads if they've been explicitly disabled system-wide, or if the URI seems to indicate possibly-sensitive content...

Doesn't monitor a particular ad slot, ie is yes/no for the whole page, and thus nominally the result is the same whenever and however often called on a page, though we might start saying no if it has been a long time since this page's generation started.

Returns:

true if it is OK to show an ad

showAds

public static final boolean showAds(DataSourceBean dsb,
                                    javax.servlet.http.HttpServletRequest request,
                                    AdUtils.AdSlotMonitor monitor)

Show ads or not on the current page and/or in the current slot? Returns false if ads are generally disabled, etc, though even in the case that the user session has been set to "lite" mode, will return true a small fraction of the time so as to show occasional ads.

We don't show ads if they've been explicitly disabled system-wide, or if the URI seems to indicate possibly-sensitive content...

Parameters:

monitor - if non-null should be a unique monitor token associated with one ad slot (or group of slots) to time how long it takes to generate content for that slot and decide whether to avoid showing the ad at all; only appropriate for server-side inserted ads

Returns:

true if it is OK to show an ad

skipUnderperformingAdSlot

public static boolean skipUnderperformingAdSlot(SimpleVariablePipelineIF vars,
                                                SimpleVariableDefinition history,
                                                EventPeriod eventPeriod,
                                                java.lang.Object slotIdentifier)

Decides if an underperforming ad slot should be skipped (ie not filled). This uses the system variable info indicated to decide how well the ad slot is performing, and then chooses randomly with an appropriate probability whether it should be filled or skipped.

Reducing the number of (non-performing) ads shown to users probably improves the user experience in various ways and probably also increases the effective revenue from other slots.

The system variable may represent something such as page views or CTR.

This must not throw any exceptions so as to avoid killing any JSP that it happens to be called from.

TODO: May want to factor in local stats specially at some point.

Parameters:

vars - handle on the system variables; never null

history - event history (definition) to be examined; never null

eventPeriod - period we are interested in from event history; must be valid for the given event and non-null

slotIdentifier - the unique ID for advertising "events" for this slot; must be valid value and type for specified history and non-null

Returns:

true if the slot should be skipped this time, false if it should be filled

createAndFileClickThruListener

public static java.lang.String createAndFileClickThruListener(javax.servlet.http.HttpServletRequest request,
                                                              SimpleVariablePipelineIF vars,
                                                              java.lang.String... otherTags)

Create and post the handler for a click-through if possible and returns the unique listenerID. This replaces any existing listener for this users, eg if the user goes back and clicks then we won't capture it.

This returns null if we could not create a listener.

Parameters:

request - client's HTTP request; never null

vars - where stats updates are posted; never null

otherTags - any non-null (and non-empty) values (which should be distinctive but short) are recorded to GENSTATS_STRING_GLOBAL_EVENT upon a click-through

Returns:

listenerID, or null if one could not be generated

canonicaliseURI

public static java.lang.String canonicaliseURI(java.lang.String uri)

Canonicalise URI (lower-case and truncate if need be; null is returned as null.

computeECPM

public static int computeECPM(GenProps gp,
                              java.lang.String statName)

Compute value (in arbitrary units) of named thousand CPM-ad-display impressions; non-negative. This will apply a default value if no specific one applies.

computeECPC

public static int computeECPC(GenProps gp,
                              java.lang.String statName)

Compute value (in arbitrary units) of named click-though; non-negative. This will apply a default value if no specific one applies.

_computeOptimalLayout

private static <E extends java.lang.Enum<E>> java.util.List<E> _computeOptimalLayout(java.lang.String callerName,
                                                                                     DataSourceBean dsb,
                                                                                     java.lang.String layoutKHitsEventPrefix,
                                                                                     java.lang.String ctEventPrefix,
                                                                                     DataSourceBean.UnlinkedKey cacheKey,
                                                                                     E defaultValue)
                                                                          throws java.lang.InterruptedException

Return optimal value(s) best-first of parameter based on past data, recomputing/cacheing as needed; never null. Any recomputation is done asynchronously. We may wait a very short time for the computation to finish but if the result isn't very quickly available then we return a default (and cache it for a short while to help avoid concurrent recomputations).

Parameters:

defaultValue - the default value to use when we cannot (immediately) compute the optimum; never null

Returns:

immutable computed optimum values best first, or the (non-null) default as a singleton if there is no (valid) cached optimum value and we are still (re)computing it asynchronously; never null, never empty, never containing nulls nor duplicates

Throws:

java.lang.InterruptedException

selectAdMidPageInternalLayout

public static final AdMidPageInternalLayout selectAdMidPageInternalLayout(DataSourceBean dsb,
                                                                          javax.servlet.http.HttpServletRequest request)
                                                                   throws java.lang.InterruptedException

Chose internal layout of ad mid-age slot to be shown on page; never null. Uses algorithm to ensure that all servers should generally choose the same layout at any one time so that, for example, switching between servers should not be disorienting.

May collect stats on layout to help monitor/select the most effective.

Parameters:

request - the RQPNAME_MIDPAGEINTLAYOUT attribute is set to indicate which format has been selected

Throws:

java.lang.InterruptedException

countPageImpressions

private static void countPageImpressions(java.util.concurrent.atomic.AtomicInteger count,
                                         DataSourceBean dsb,
                                         java.lang.String kHitName)

Count page impressions in this layout/ad-style. Logs thousands of impressions (kHits) to reduce event traffic, but can log single hits initially for good resolution and robustness for short-running servers and/or sparse events.

make1HitName

private static java.lang.String make1HitName(java.lang.String kHitName)

Make single-hit event name from k-hit name, or return null if not possible. This is done by replacing the initial "k" with a "1".

selectAdTowerInternalLayout

public static final AdTowerInternalLayout selectAdTowerInternalLayout(DataSourceBean dsb,
                                                                      javax.servlet.http.HttpServletRequest request)
                                                               throws java.lang.InterruptedException

Chose internal layout of ad tower(s) to be shown on page; never null. Uses algorithm to ensure that all servers should generally choose the same layout at any one time so that, for example, switching between servers should not be disorienting.

May collect stats on layout to help monitor/select the most effective.

Parameters:

request - the RQPNAME_TOWERINTLAYOUT attribute is set to indicate which format has been selected

Throws:

java.lang.InterruptedException

selectPageAdLayout

public static final AdTowerPos selectPageAdLayout(DataSourceBean dsb,
                                                  javax.servlet.http.HttpServletRequest request,
                                                  boolean siteIsBusy,
                                                  boolean probeOnly)
                                           throws java.lang.InterruptedException

Choose general ad/page layout; never null. Uses algorithm to ensure that all servers should generally choose the same layout at any one time so that, for example, switching between servers should not be disorienting.

May collect stats on layout to help monitor/select the most effective.

Note that most effective combo seems to be:

• Wide tower on left.
• Horizontal banner at top.

Computed values may be cached wrt the DataSourceBean variable, so should consistently be called with the same instance for one site.

Parameters:

probeOnly - this call is to find out what the layout should be, and should not count as a page view

Throws:

java.lang.InterruptedException

computeCTEventName

public static java.lang.String computeCTEventName(AdTowerPos towerPos)

Compute the full event name for recording a clickthrough with a particular layout style.

generateAdTowerFallbackHTML

public static java.lang.String generateAdTowerFallbackHTML(javax.servlet.ServletContext application,
                                                           LocaleBean localeBean,
                                                           int pxWidth,
                                                           int pxHeight)

Get the HTML content for a fall-back/fill-in ad-tower filler; never null but may be "". Returns "" if nothing appropriate to show, or if nothing available quickly (since we don't want to delay page loading pointlessly), or in case of error.