org.archive.wayback.resourcestore.indexer
Class IndexWorker

java.lang.Object
  org.archive.wayback.resourcestore.indexer.IndexWorker

All Implemented Interfaces:

Shutdownable

public class IndexWorker
extends java.lang.Object
implements Shutdownable

Simple worker, which gets tasks from an IndexQueue, in the case, the name of ARC/WARC files to be indexed, retrieves the ARC/WARC location from a ResourceFileLocationDB, creates the index, which is serialized into a file, and then hands that file off to a ResourceIndex for merging, using an IndexClient.

Version:

$Date$, $Revision$

Author:

brad

Field Summary

static java.lang.String	ARC_EXTENSION
static java.lang.String	ARC_GZ_EXTENSION
static java.lang.String	WARC_EXTENSION
static java.lang.String	WARC_GZ_EXTENSION

Constructor Summary

IndexWorker()

Method Summary

boolean	doWork()
UrlCanonicalizer	getCanonicalizer()
ResourceFileLocationDB	getDb()
long	getInterval()
IndexQueue	getQueue()
IndexClient	getTarget()
CloseableIterator<CaptureSearchResult>	indexFile(java.lang.String pathOrUrl)
void	init()
void	setCanonicalizer(UrlCanonicalizer canonicalizer)
void	setDb(ResourceFileLocationDB db)
void	setInterval(long interval)
void	setQueue(IndexQueue queue)
void	setTarget(IndexClient target)
void	shutdown()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ARC_EXTENSION

public static final java.lang.String ARC_EXTENSION

See Also:

Constant Field Values

ARC_GZ_EXTENSION

public static final java.lang.String ARC_GZ_EXTENSION

See Also:

Constant Field Values

WARC_EXTENSION

public static final java.lang.String WARC_EXTENSION

See Also:

Constant Field Values

WARC_GZ_EXTENSION

public static final java.lang.String WARC_GZ_EXTENSION

See Also:

Constant Field Values

Constructor Detail

IndexWorker

public IndexWorker()

Method Detail

init

public void init()

shutdown

public void shutdown()

Specified by:

shutdown in interface Shutdownable

doWork

public boolean doWork()
               throws java.io.IOException

Throws:

java.io.IOException

indexFile

public CloseableIterator<CaptureSearchResult> indexFile(java.lang.String pathOrUrl)
                                                 throws java.io.IOException

Throws:

java.io.IOException

getInterval

public long getInterval()

Returns:

the interval

setInterval

public void setInterval(long interval)

Parameters:

interval - the interval to set

getQueue

public IndexQueue getQueue()

Returns:

the queue

setQueue

public void setQueue(IndexQueue queue)

Parameters:

queue - the queue to set

getDb

public ResourceFileLocationDB getDb()

Returns:

the db

setDb

public void setDb(ResourceFileLocationDB db)

Parameters:

db - the db to set

getTarget

public IndexClient getTarget()

Returns:

the target

setTarget

public void setTarget(IndexClient target)

Parameters:

target - the target to set

getCanonicalizer

public UrlCanonicalizer getCanonicalizer()

Returns:

the canonicalizer

setCanonicalizer

public void setCanonicalizer(UrlCanonicalizer canonicalizer)

Parameters:

canonicalizer - the canonicalizer to set