org.archive.wayback.resourceindex.cdx.dynamic
Class DynamicCDXIndex

java.lang.Object
  org.archive.wayback.resourceindex.CompositeSearchResultSource
      org.archive.wayback.resourceindex.cdx.dynamic.DynamicCDXIndex

All Implemented Interfaces:

SearchResultSource

public class DynamicCDXIndex
extends CompositeSearchResultSource

A CompositeSearchResultSource that autmatically manages it's list of sources based on 3 configuration files, and a background thread: Config 1: Mapping of ranges to hosts responsible for that range this class is aware of the local host name, so uses this file to determin which range(s) should be local Config 2: Mapping of ranges to one or more MD5s that compose that range when all of these MD5s have been copied local, this index becomes active, and each request uses a composite of these local files Config 3: Mapping of MD5s to locations from which they can be retrieved when a file that should be local is missing, these locations will be used to retrieve a copy of that file Background Thread: compares current set of files to the various configurations files, gets files local that need to be and updates the composite set searched when the correct set of MD5s are localized. The Thread maintains the state of the sychronization with the desired file set: UNKNOWN: If the desired state is unknown SYNCHING: If the local state does not match the desired state SYNCHED: If the local stat matches the desired state This class forwards all method requests to the superclass, if the state is SYNCHED, otherwise throws a ResourceIndexNotAvailableException.

Version:

$Date: 2010-09-29 05:28:38 +0700 (Wed, 29 Sep 2010) $, $Revision: 3262 $

Author:

brad

Field Summary

protected static String	MD5_PATTERN
protected static Pattern	MD5_REGEX
protected static int	STATE_SYNCHED
protected static int	STATE_SYNCHING
protected static int	STATE_UNKNOWN

Fields inherited from class org.archive.wayback.resourceindex.CompositeSearchResultSource

sources

Constructor Summary

DynamicCDXIndex(Object[] nodeNames, int interval, File dataDir, RangeAssignmentFile rangeFile, CDXDefinitionFile definitionFile, MD5LocationFile md5File)

Method Summary

protected File	dataFileForMD5(String md5)
protected Object[]	getLocalMD5s()
CloseableIterator<CaptureSearchResult>	getPrefixIterator(String prefix)
CloseableIterator<CaptureSearchResult>	getPrefixReverseIterator(String prefix)
protected int	getState()
protected void	setCDXFiles(Object[] md5s)
protected void	setState(int newState)

Methods inherited from class org.archive.wayback.resourceindex.CompositeSearchResultSource

addSource, cleanup, getCDXSources, getSources, setCDXSources, setSources, shutdown

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STATE_UNKNOWN

protected static int STATE_UNKNOWN

STATE_SYNCHING

protected static int STATE_SYNCHING

STATE_SYNCHED

protected static int STATE_SYNCHED

MD5_PATTERN

protected static String MD5_PATTERN

MD5_REGEX

protected static final Pattern MD5_REGEX

Constructor Detail

DynamicCDXIndex

public DynamicCDXIndex(Object[] nodeNames,
                       int interval,
                       File dataDir,
                       RangeAssignmentFile rangeFile,
                       CDXDefinitionFile definitionFile,
                       MD5LocationFile md5File)

Parameters:

nodeNames -

interval -

dataDir -

rangeFile -

definitionFile -

md5File -

Method Detail

getLocalMD5s

protected Object[] getLocalMD5s()

dataFileForMD5

protected File dataFileForMD5(String md5)

setCDXFiles

protected void setCDXFiles(Object[] md5s)

setState

protected void setState(int newState)

getState

protected int getState()

getPrefixIterator

public CloseableIterator<CaptureSearchResult> getPrefixIterator(String prefix)
                                                         throws ResourceIndexNotAvailableException

Specified by:

getPrefixIterator in interface SearchResultSource

Overrides:

getPrefixIterator in class CompositeSearchResultSource

Returns:

CleanableIterator that will return SearchResults beginning with prefix argument, with subsequent next() calls returning subsequent results.

Throws:

ResourceIndexNotAvailableException

getPrefixReverseIterator

public CloseableIterator<CaptureSearchResult> getPrefixReverseIterator(String prefix)
                                                                throws ResourceIndexNotAvailableException

Specified by:

getPrefixReverseIterator in interface SearchResultSource

Overrides:

getPrefixReverseIterator in class CompositeSearchResultSource

Returns:

CleanableIterator that will return SearchResults starting *before* prefix argument, with subsequent next() calls returning previous results.

Throws:

ResourceIndexNotAvailableException