org.hd.d.pg2k.svrCore.datasource
Class ExhibitDataHTTPTunnelSource

java.lang.Object
  org.hd.d.pg2k.svrCore.datasource.ExhibitDataTunnelSource
      org.hd.d.pg2k.svrCore.datasource.ExhibitDataHTTPTunnelSource

All Implemented Interfaces:

SimpleExhibitPipelineIF, BasicVarMgrInterface, SimpleVariablePipelineIF

public final class ExhibitDataHTTPTunnelSource
extends ExhibitDataTunnelSource

Exhibit pipeline stage that fetches its data across an HTTP[S] tunnel. Derived from the abstract tunnel source base class.

This class does two important things:

• Sets up an HTTP connection with the correct characteristics (such as with sensible timeouts on open and read).
• Controls the concurrency of the connection to avoid overwhelming the upstream server.

A limited amount of concurrency (ie more than one connection) may help overcome latency and improve throughput, especially over a lossy network or during long operations such as fetching the AEP or exhibit data blocks.

Concurrency should not be so large as to cause the master to start rejecting connections, eg no more than (say) the 8 connections that a normal Web browser might open to overcome latency.

Note also that the master may itself have limited concurrency available internally to service many types of operation.

Our concurrency limit may be by any type of RPC call, or at most one of any particular operation type (on the grounds that the master may only be able to service limited numbers of calls by any particular operation type). (At one point we limited concurrency using a simple Java monitor that was selected by a hash on the RPC type. Now we have one overall limit on the number of connections upstream and abort rather than waiting indefinitely to get a connection.)

We may vary this value between instances to help prevent collisions between different slaves, etc.

We may limit concurrency to 1 after any IOException until one or more successful RPCs have completed.

serverURL

private final java.net.URL serverURL

The full master-Web-server endpoint URL; never null.

userID

private transient volatile java.lang.String userID

User ID authentication info, never serialised; null if not set.

passwd

private transient volatile java.lang.String passwd

User ID password, never serialised; null if not set.

HEADER_AUTH_ID

public static final java.lang.String HEADER_AUTH_ID

Name of HTTP header for (optional) authentication user ID.

See Also:

Constant Field Values

HEADER_AUTH_PW

public static final java.lang.String HEADER_AUTH_PW

Name of HTTP header for (optional) authentication password.

See Also:

Constant Field Values

HEADER_MAC

public static final java.lang.String HEADER_MAC

Name of HTTP header for (optional) MAC and timestamp.

See Also:

Constant Field Values

MAX_CONCURRENT_CONNECTIONS

public static final int MAX_CONCURRENT_CONNECTIONS

Maximum number of concurrent connections to allow back to the master; strictly positive. We make this value public in case other modules would like to base their load regulation on this connection limit.

If >1 then we can potentially prevent any one RPC type hogging all the available connections.

Any value between 1 and 8 inclusive is probably reasonable; in practice there is rarely more than one connection open at once.

See Also:

Constant Field Values

ALLOW_CONNECTION_QUEUEING

private static final boolean ALLOW_CONNECTION_QUEUEING

If true allow excess connection attempts to queue for a while. Setting this false reduces threads blocking.

See Also:

Constant Field Values

USE_FAIR_LOCKS

private static final boolean USE_FAIR_LOCKS

If true, use "fair" lock access to try to reduce variance in connection service time. May be slightly slower (and is fatally buggy under JDK 1.5.0_05 or earlier).

Largely irrelevant if we don's allow connection queueing anyway.

See Also:

Constant Field Values

countingSemaphore

private final java.util.concurrent.Semaphore countingSemaphore

Counting semaphore to limit RPC concurrency by total number of callers; never null

DEFAULT_HTTP_CONN_TIMEOUT_MS

private final int DEFAULT_HTTP_CONN_TIMEOUT_MS

Default HTTP connection timeout (ms); strictly positive. This is set relatively low so that we don't block downstream/callers too long if the Net is lossy or slow.

We assume that the default HTTP connection timeout would be longer than the value chosen here.

We take failure to establish a connection quickly as indicative of generally poor connectivity.

(We may need to allow extra time for HTTPS or tunnelled connections to be set up.)

We may vary this value a little between instances to help prevent collisions between different clients at one master.

A value of a few seconds to a few tens of seconds is probably suitable, especially given the fact that connections that don't happen quickly rarely happen at all.

As at 20070430 from the UK approximate RTTs (over DSL which accounts for ~14ms) to: cn-bj1 RTT 600ms, 330ms au-nsw1, 270ms sg-wv1, 160ms in-bom1, 110ms us-ga1; thus we must allow for (say) at least 3*RTT for HTTP/TCP connection to establish, ie at least (say) 2s.

This should probably be somewhat lower than CoreConsts.MAX_RPC_RTT_MS since this connection phase is only one component of the total RPC time.

(Note that in Windows NT the hard-wired initial TCP timeout was 3s, which seems to have been a little too short for legitimate long-delay paths.)

DEFAULT_HTTP_MAX_READ_TIMEOUT_MS

private static final int DEFAULT_HTTP_MAX_READ_TIMEOUT_MS

Default maximum HTTP read timeout (ms); strictly positive. This is set high enough not to wrongly time-out long-running operations, such as fetching the AEP, but short enough not to block clients indefinitely if something goes seriously wrong, eg a filesystem on the master hangs or dies.

This should be (much) larger than the CoreConsts.MAX_RPC_RTT_MS time.

This must probably be in the region of a few minutes to a few hours.

USE_WATCHDOG

private static final boolean USE_WATCHDOG

If true then have a watchdog thread ensure that no connection hangs indefinitely. This is only needed as a fallback because the read and connect timeouts on the HTTP URLConnection are not totally reliable as of JDK 1.5.0.

See Also:

Constant Field Values

openConnections

private final java.util.concurrent.ConcurrentMap<java.net.HttpURLConnection,Tuple.Triple<java.lang.Thread,java.lang.Long,ExhibitDataTunnelSource.RawPacket.OpCode>> openConnections

Thread-safe Map of connections to times they should have terminated. This may get checked in poll(), or where another inbound thread might otherwise be blocked.

This holds the Thread with the connection, the time that the connection should have been closed by, and the RPC type of the connection.

_kdcLock

private final java.util.concurrent.locks.Lock _kdcLock

Private lock to allowing one thread into killDeadConnections(); never null.

ENFORCE_MAC

public static final boolean ENFORCE_MAC

If true then we enforce MACs when we have a local xfer key. We may still try to send a request without a MAC (iff we don't have a key), but we won't accept a response without a MAC unless we have no key.

All but a safe subset of ops is rejected with an OP__RUNTEX (RuntimeException) on incoming requests without a verifiable MAC.

See Also:

Constant Field Values

messageIDs

private final ExhibitDataTunnelSource.DuplicateIDChecker<ROByteArray> messageIDs

Record of unique response packet IDs around our acceptance window; never null. We use the final MAC segment ID, as that covers the entire frame.

We actually remember IDs for about twice the age implied by the skew so that if our clock is wrong by, or slips by, the maximum skew, we won't start admitting very old duplicate messages.

We should only ad IDs if possible when we are already fairly sure of the source and when we have, for example, already checked for acceptable skew, to make any sort of DoS attack against us harder.

We should reject otherwise-acceptable messages that have an ID already present in this Map.

Thread-safe.

LIMIT_BUSY_CONS_BY_RPC_TYPE

private static final boolean LIMIT_BUSY_CONS_BY_RPC_TYPE

If true, then limit outgoing connections by RPC type when busy. This may help prevent a misbehaviour of one system component shutting out communications for the rest of the system.

See Also:

Constant Field Values

EMPTY_INPUTSTREAM

private static final java.io.InputStream EMPTY_INPUTSTREAM

Empty, immutable/unclosable InputStream that we can return to all callers.

EMPTY_RESPONSE_STREAM

private static final Tuple.Pair<java.lang.Integer,java.io.InputStream> EMPTY_RESPONSE_STREAM

Empty, immutable/unclosable response stream that we can return to all callers.

localAddr

private static final java.net.InetAddress localAddr

Address to bind to for outbound connections, or null if any available address is OK. At the moment we only accept this in a backdoor setting via a system property (and don't complain if property access is not allowed).

If using this, eg with -Dorg.hd.pg2k.tunnelLocalAddr=85.158.46.82 (and if HTTPClient V0.3-3 is still in use for these 'local' connections) then it may be wise to work round a bug with -DHTTPClient.disableKeepAlives=true.

ExhibitDataHTTPTunnelSource

public ExhibitDataHTTPTunnelSource(java.lang.String masterURL,
                                   SimpleLoggerIF logger)

Construct a new HTTP tunnel data source to the given URL. Does not necessarily try to build the tunnel immediately.

Parameters:

masterURL - URL to reach the server's tunnel, of the form http://serverName/tunnelMountPoint; never null

Throws:

java.lang.IllegalArgumentException - if the URL is invalid or unusable.

getServerURL

public java.net.URL getServerURL()

Get the endpoint URL of this tunnel; never null. We assume that it is safe to return the URL directly, ie that it behaves as if immutable.

setAuthenticationInfo

public void setAuthenticationInfo(java.lang.String userID,
                                  char[] passwd)

Set (or clear) additional authentication data in form of ID and password. When the server requires explicit authentication from the client, this form may be sufficient to allow full or partial (eg read-only) access.

To set authentication arguments must be non-null and non-empty, else all arguments must be null to clear authentication data.

killDeadConnections

private void killDeadConnections()

Closes any connections that have been alive too long. Thread-safe.

We only allow one thread into this (per instance) at once, eg to limit the scale of any damage if one of these hangs due to problems with the underlying HTTP implementation.

doRPCRaw

protected ExhibitDataTunnelSource.RawPacket doRPCRaw(ExhibitDataTunnelSource.RawPacket packetOut)
                                              throws java.io.IOException

Make an RPC call over HTTP[S] with the given outgoing packet. This controls concurrency (number of connections to the master).

If we cannot get a connection, because:

• we have reached the self-imposed concurrency limit, or
• there is a problem at the HTTP level (or in the upstream server),

then we will try to time out and throw an IOException rather than blocking indefinitely.

Specified by:

doRPCRaw in class ExhibitDataTunnelSource

Parameters:

packetOut - request packet; never null

Returns:

response packet; never null

Throws:

java.io.IOException - in case of I/O difficulties

_doRPCRawInternal

private java.lang.Object _doRPCRawInternal(ExhibitDataTunnelSource.RawPacket packetOut,
                                           boolean asStream,
                                           boolean allowBigReadTimeout)
                                    throws java.io.IOException

Make an RPC call over HTTP[S] with the given outgoing packet. This controls concurrency (number of connections to the master).

If we cannot get a connection, because:

• we have reached the self-imposed concurrency limit, or
• there is a problem at the HTTP level (or in the upstream server),

then we will eventually time out and throw an IOException rather than blocking indefinitely.

If we have a data-protecting "XferKey" then we always send a MAC. This enables the receiver to detect more errors in transmission.

Parameters:

packetOut - request packet; never null

asStream - if true, return as streamed result rather than packet

allowBigReadTimeout - if true then the RPC may take a lot of work at the master so we should be prepared to wait a long time before timing out

Returns:

response packet; never null

Throws:

java.io.IOException - in case of I/O difficulties

doRPCRawWithStreamResponse

protected Tuple.Pair<java.lang.Integer,java.io.InputStream> doRPCRawWithStreamResponse(ExhibitDataTunnelSource.RawPacket packetOut,
                                                                                       boolean allowBigReadTimeout)
                                                                                throws java.io.IOException

Description copied from class: ExhibitDataTunnelSource

Optimised RPC call with the given outgoing packet and returning packet body as an InputStream; never null nor with a null stream. This is an optimisation for performance-critical cases only, and foregoes some error checking/handling for speed (and thus the caller should ensure that it performs integrity checks). There is an onus on the streaming-related code to behave safely even if fed bogus/corrupt data, and it must be able to safely/easily undo any work done if the message is found to be bogus as late as the final byte(s).

This streams the content of the response packet and will object if it sees any IOException or if the packets come back with the wrong op code.

A terminating trailer byte may or may not be visible on the returned stream thus allowing the implementation to be as efficient as possible.

This may return after all the input data has been collected, or while some or all is still to come, and thus the returned stream may fail and throw an exception.

The first element of the result is the length of the response data (not including any non-data trailer bytes from the packet even if present) but may be null if this length is not known when the packet header is seen, eg because the packet body was compressed.

This may be implemented/overridden by the deriving class to suit its transmission medium, and as an optimisation to reduce copying and allow streaming, ie starting to process the input before it is all received.

The data stream is always of uncompressed data, regardless of whether the data was sent compressed on the wire, ie this routine will correctly decompress data on the fly as/when needed.

The caller must close the stream promptly to release resources such as file handles and non-Java memory.

Overrides:

doRPCRawWithStreamResponse in class ExhibitDataTunnelSource

Parameters:

packetOut - request packet; never null

allowBigReadTimeout - TODO

Returns:

response length and data stream; never null

Throws:

java.io.IOException - in case of I/O difficulties

poll

public void poll(GenProps gp)

Poll periodically. We call super.poll(gp) to ensure that base-class polling work gets done, including any upstream poll() calls.

Specified by:

poll in interface SimpleExhibitPipelineIF

Overrides:

poll in class ExhibitDataTunnelSource