org.apache.hadoop.tools
Class Logalyzer

java.lang.Object
  org.apache.hadoop.tools.Logalyzer

public class Logalyzer
extends Object

Logalyzer: A utility tool for archiving and analyzing hadoop logs.

This tool supports archiving and anaylzing (sort/grep) of log-files. It takes as input a) Input uri which will serve uris of the logs to be archived. b) Output directory (not mandatory). b) Directory on dfs to archive the logs. c) The sort/grep patterns for analyzing the files and separator for boundaries. Usage: Logalyzer -archive -archiveDir -analysis -logs -grep -sort -separator

Author:

Arun C Murthy

Nested Class Summary

static class	Logalyzer.LogComparator A WritableComparator optimized for UTF8 keys of the logs.
static class	Logalyzer.LogRegexMapper A Mapper that extracts text matching a regular expression.

Constructor Summary

Logalyzer()

Method Summary

void	doAnalyze(String inputFilesDirectory, String outputDirectory, String grepPattern, String sortColumns, String columnSeparator) doAnalyze:
void	doArchive(String logListURI, String archiveDirectory) doArchive: Workhorse function to archive log-files.
static void	main(String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Logalyzer

public Logalyzer()

Method Detail

doArchive

public void doArchive(String logListURI,
                      String archiveDirectory)
               throws IOException

doArchive: Workhorse function to archive log-files.

Parameters:

logListURI - : The uri which will serve list of log-files to archive.

archiveDirectory - : The directory to store archived logfiles.

Throws:

IOException

doAnalyze

public void doAnalyze(String inputFilesDirectory,
                      String outputDirectory,
                      String grepPattern,
                      String sortColumns,
                      String columnSeparator)
               throws IOException

doAnalyze:

Parameters:

inputFilesDirectory - : Directory containing the files to be analyzed.

outputDirectory - : Directory to store analysis (output).

grepPattern - : Pattern to *grep* for.

sortColumns - : Sort specification for output.

columnSeparator - : Column separator.

Throws:

IOException

main

public static void main(String[] args)