org.apache.hadoop.util
Class CopyFiles.HTTPCopyFilesMapper

java.lang.Object
  org.apache.hadoop.mapred.MapReduceBase
      org.apache.hadoop.util.CopyFiles.CopyFilesMapper
          org.apache.hadoop.util.CopyFiles.HTTPCopyFilesMapper

All Implemented Interfaces:

Closeable, JobConfigurable, Mapper

Enclosing class:

CopyFiles

public static class CopyFiles.HTTPCopyFilesMapper
extends CopyFiles.CopyFilesMapper
implements Mapper

Constructor Summary

CopyFiles.HTTPCopyFilesMapper()

Method Summary

void	cleanup(Configuration conf, JobConf jobConf, String srcPath, String destPath) Interface to cleanup *distcp* specific resources
void	configure(JobConf job) Default implementation that does nothing.
void	map(WritableComparable key, Writable val, OutputCollector out, Reporter reporter) Maps a single input key/value pair into intermediate key/value pairs.
void	setup(Configuration conf, JobConf jobConf, String[] srcPaths, String destPath, boolean ignoreReadFailures) Initialize HTTPCopyFileMapper specific job.

Methods inherited from class org.apache.hadoop.util.CopyFiles.CopyFilesMapper

getMapCount, makeRelative

Methods inherited from class org.apache.hadoop.mapred.MapReduceBase

close

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.io.Closeable

close

Constructor Detail

CopyFiles.HTTPCopyFilesMapper

public CopyFiles.HTTPCopyFilesMapper()

Method Detail

setup

public void setup(Configuration conf,
                  JobConf jobConf,
                  String[] srcPaths,
                  String destPath,
                  boolean ignoreReadFailures)
           throws IOException

Initialize HTTPCopyFileMapper specific job.

Specified by:

setup in class CopyFiles.CopyFilesMapper

Parameters:

conf - : The dfs/mapred configuration.

jobConf - : The handle to the jobConf object to be initialized.

srcPaths - : The source URI.

destPath - : The destination URI.

ignoreReadFailures - : Ignore read failures?

Throws:

IOException

cleanup

public void cleanup(Configuration conf,
                    JobConf jobConf,
                    String srcPath,
                    String destPath)
             throws IOException

Description copied from class: CopyFiles.CopyFilesMapper

Interface to cleanup *distcp* specific resources

Specified by:

cleanup in class CopyFiles.CopyFilesMapper

Parameters:

conf - : The dfs/mapred configuration.

jobConf - : The handle to the jobConf object to be initialized.

srcPath - : The source uri.

destPath - : The destination uri.

Throws:

IOException

configure

public void configure(JobConf job)

Description copied from class: MapReduceBase

Default implementation that does nothing.

Specified by:

configure in interface JobConfigurable

Overrides:

configure in class MapReduceBase

Parameters:

job - the configuration

map

public void map(WritableComparable key,
                Writable val,
                OutputCollector out,
                Reporter reporter)
         throws IOException

Description copied from interface: Mapper

Maps a single input key/value pair into intermediate key/value pairs. Output pairs need not be of the same types as input pairs. A given input pair may map to zero or many output pairs. Output pairs are collected with calls to OutputCollector.collect(WritableComparable,Writable).

Specified by:

map in interface Mapper

Parameters:

key - the key

val - the values

out - collects mapped keys and values

Throws:

IOException