org.apache.hadoop.dfs
Class DataNode

java.lang.Object
  org.apache.hadoop.dfs.DataNode

All Implemented Interfaces:

Runnable, FSConstants

public class DataNode
extends Object
implements FSConstants, Runnable

DataNode is a class (and program) that stores a set of blocks for a DFS deployment. A single deployment can have one or many DataNodes. Each DataNode communicates regularly with a single NameNode. It also communicates with client code and other DataNodes from time to time. DataNodes store a series of named blocks. The DataNode allows client code to read these blocks, or to write new block data. The DataNode may also, in response to instructions from its NameNode, delete blocks or copy blocks to/from other DataNodes. The DataNode maintains just one critical table: block-> stream of bytes (of BLOCK_SIZE or less) This info is stored on a local disk. The DataNode reports the table's contents to the NameNode upon startup and every so often afterwards. DataNodes spend their lives in an endless loop of asking the NameNode for something to do. A NameNode cannot connect to a DataNode directly; a NameNode simply returns values from functions invoked by a DataNode. DataNodes maintain an open server socket so that client code or other DataNodes can read/write data. The host/port for this server is reported to the NameNode, which then sends that information to clients or other DataNodes that might be interested.

Author:

Mike Cafarella

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.dfs.FSConstants

FSConstants.NodeType, FSConstants.SafeModeAction, FSConstants.StartupOption

Field Summary

static org.apache.commons.logging.Log

LOG

Fields inherited from interface org.apache.hadoop.dfs.FSConstants

BLOCK_INVALIDATE_CHUNK, BLOCKREPORT_INTERVAL, BUFFER_SIZE, CHUNKED_ENCODING, COMPLETE_SUCCESS, HEARTBEAT_INTERVAL, LAYOUT_VERSION, LEASE_HARDLIMIT_PERIOD, LEASE_SOFTLIMIT_PERIOD, MAX_PATH_DEPTH, MAX_PATH_LENGTH, MIN_BLOCKS_FOR_WRITE, OP_ACK, OP_BLOCKRECEIVED, OP_BLOCKREPORT, OP_CLIENT_ABANDONBLOCK, OP_CLIENT_ABANDONBLOCK_ACK, OP_CLIENT_ADDBLOCK, OP_CLIENT_ADDBLOCK_ACK, OP_CLIENT_COMPLETEFILE, OP_CLIENT_COMPLETEFILE_ACK, OP_CLIENT_DATANODE_HINTS, OP_CLIENT_DATANODE_HINTS_ACK, OP_CLIENT_DATANODEREPORT, OP_CLIENT_DATANODEREPORT_ACK, OP_CLIENT_DELETE, OP_CLIENT_DELETE_ACK, OP_CLIENT_EXISTS, OP_CLIENT_EXISTS_ACK, OP_CLIENT_ISDIR, OP_CLIENT_ISDIR_ACK, OP_CLIENT_LISTING, OP_CLIENT_LISTING_ACK, OP_CLIENT_MKDIRS, OP_CLIENT_MKDIRS_ACK, OP_CLIENT_OBTAINLOCK, OP_CLIENT_OBTAINLOCK_ACK, OP_CLIENT_OPEN, OP_CLIENT_OPEN_ACK, OP_CLIENT_RAWSTATS, OP_CLIENT_RAWSTATS_ACK, OP_CLIENT_RELEASELOCK, OP_CLIENT_RELEASELOCK_ACK, OP_CLIENT_RENAMETO, OP_CLIENT_RENAMETO_ACK, OP_CLIENT_RENEW_LEASE, OP_CLIENT_RENEW_LEASE_ACK, OP_CLIENT_STARTFILE, OP_CLIENT_STARTFILE_ACK, OP_CLIENT_TRYAGAIN, OP_ERROR, OP_FAILURE, OP_HEARTBEAT, OP_INVALIDATE_BLOCKS, OP_READ_BLOCK, OP_READ_RANGE_BLOCK, OP_READSKIP_BLOCK, OP_TRANSFERBLOCKS, OP_TRANSFERDATA, OP_WRITE_BLOCK, OPERATION_FAILED, READ_TIMEOUT, RUNLENGTH_ENCODING, STILL_WAITING, WRITE_COMPLETE

Method Summary

static InetSocketAddress	createSocketAddr(String target) Util method to build socket addr from either: : ://:/
static DataNode	getDataNode() Return the DataNode object
String	getNamenode() Return the namenode's identifier
InetSocketAddress	getNameNodeAddr()
static void	main(String[] args)
void	offerService() Main loop for the DataNode.
void	run() No matter what kind of exception we get, keep retrying to offerService().
static DataNode	run(Configuration conf) Start datanode daemon.
void	shutdown() Shut down this instance of the datanode.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

Method Detail

createSocketAddr

public static InetSocketAddress createSocketAddr(String target)
                                          throws IOException

Util method to build socket addr from either: : ://:/

Throws:

IOException

getDataNode

public static DataNode getDataNode()

Return the DataNode object

getNameNodeAddr

public InetSocketAddress getNameNodeAddr()

getNamenode

public String getNamenode()

Return the namenode's identifier

shutdown

public void shutdown()

Shut down this instance of the datanode. Returns only after shutdown is complete.

offerService

public void offerService()
                  throws Exception

Main loop for the DataNode. Runs until shutdown, forever calling remote NameNode functions.

Throws:

Exception

run

public void run()

No matter what kind of exception we get, keep retrying to offerService(). That's the loop that connects to the NameNode and provides basic DataNode functionality. Only stop when "shouldRun" is turned off (which can only happen at shutdown).

Specified by:

run in interface Runnable

run

public static DataNode run(Configuration conf)
                    throws IOException

Start datanode daemon.

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

main

public static void main(String[] args)