org.apache.hadoop.dfs
Class DistributedFileSystem

java.lang.Object
  org.apache.hadoop.conf.Configured
      org.apache.hadoop.fs.FileSystem
          org.apache.hadoop.fs.FilterFileSystem
              org.apache.hadoop.fs.ChecksumFileSystem
                  org.apache.hadoop.dfs.DistributedFileSystem

All Implemented Interfaces:

Configurable

public class DistributedFileSystem
extends ChecksumFileSystem

Implementation of the abstract FileSystem for the DFS system. This object is the way end-user code interacts with a Hadoop DistributedFileSystem.

Author:

Mike Cafarella

Field Summary

Fields inherited from class org.apache.hadoop.fs.FilterFileSystem

fs

Fields inherited from class org.apache.hadoop.fs.FileSystem

LOG

Constructor Summary

DistributedFileSystem()

DistributedFileSystem(InetSocketAddress namenode, Configuration conf)
Deprecated.

Method Summary

void	finalizeUpgrade() Finalize previously upgraded files system state.
long	getContentLength(Path f) Return the number of bytes of the given path If f is a file, return the size of the file; If f is a directory, return the size of the directory tree
DatanodeInfo[]	getDataNodeStats() Return statistics for each datanode.
long	getRawCapacity() Return the total raw capacity of the filesystem, disregarding replication .
long	getRawUsed() Return the total raw used space in the filesystem, disregarding replication .
void	metaSave(String pathname)
void	refreshNodes()
boolean	reportChecksumFailure(Path f, FSDataInputStream in, long inPos, FSDataInputStream sums, long sumsPos) We need to find the blocks that didn't match.
boolean	setSafeMode(FSConstants.SafeModeAction action) Enter, leave or get safe mode.

Methods inherited from class org.apache.hadoop.fs.ChecksumFileSystem

completeLocalOutput, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, delete, getApproxChkSumLength, getBytesPerSum, getChecksumFile, getChecksumFileLength, getRawFileSystem, isChecksumFile, listPaths, listPaths, lock, mkdirs, open, release, rename, setReplication, startLocalOutput

Methods inherited from class org.apache.hadoop.fs.FilterFileSystem

checkPath, close, exists, getBlockSize, getConf, getDefaultBlockSize, getDefaultReplication, getFileCacheHints, getLength, getName, getReplication, getUri, getWorkingDirectory, initialize, isDirectory, makeQualified, setWorkingDirectory

Methods inherited from class org.apache.hadoop.fs.FileSystem

closeAll, copyFromLocalFile, copyToLocalFile, create, create, create, create, create, create, create, createNewFile, get, get, getLocal, getNamed, getUsed, globPaths, globPaths, isFile, listPaths, listPaths, moveFromLocalFile, moveToLocalFile, open, parseArgs

Methods inherited from class org.apache.hadoop.conf.Configured

setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DistributedFileSystem

public DistributedFileSystem()

DistributedFileSystem

public DistributedFileSystem(InetSocketAddress namenode,
                             Configuration conf)
                      throws IOException

Deprecated.

Throws:

IOException

Method Detail

getContentLength

public long getContentLength(Path f)
                      throws IOException

Description copied from class: FileSystem

Return the number of bytes of the given path If f is a file, return the size of the file; If f is a directory, return the size of the directory tree

Overrides:

getContentLength in class FileSystem

Throws:

IOException

getRawCapacity

public long getRawCapacity()
                    throws IOException

Return the total raw capacity of the filesystem, disregarding replication .

Throws:

IOException

getRawUsed

public long getRawUsed()
                throws IOException

Return the total raw used space in the filesystem, disregarding replication .

Throws:

IOException

getDataNodeStats

public DatanodeInfo[] getDataNodeStats()
                                throws IOException

Return statistics for each datanode.

Throws:

IOException

setSafeMode

public boolean setSafeMode(FSConstants.SafeModeAction action)
                    throws IOException

Enter, leave or get safe mode.

Throws:

IOException

See Also:

ClientProtocol.setSafeMode(FSConstants.SafeModeAction)

refreshNodes

public void refreshNodes()
                  throws IOException

Throws:

IOException

finalizeUpgrade

public void finalizeUpgrade()
                     throws IOException

Finalize previously upgraded files system state.

Throws:

IOException

metaSave

public void metaSave(String pathname)
              throws IOException

Throws:

IOException

reportChecksumFailure

public boolean reportChecksumFailure(Path f,
                                     FSDataInputStream in,
                                     long inPos,
                                     FSDataInputStream sums,
                                     long sumsPos)

We need to find the blocks that didn't match. Likely only one is corrupt but we will report both to the namenode. In the future, we can consider figuring out exactly which block is corrupt.

Overrides:

reportChecksumFailure in class ChecksumFileSystem

Parameters:

f - the file name containing the error

in - the stream open on the file

inPos - the position of the beginning of the bad data in the file

sums - the stream open on the checksum file

sumsPos - the position of the beginning of the bad data in the checksum file

Returns:

if retry is neccessary