Intro to Harvest for Webmasters at MIT

November 20, 1996

1. Why to choose Harvest
   1. Lycos, etc. are costly
   2. WAIS, etc. have limitations
   3. with Harvest, indexing info. can easily be shared
   4. FAQ: Compare Harvest with related work
2. How to use Harvest
   1. A site with httpd already installed should be able to get Harvest installed and running in about 30 minutes using the above installation instructions and the binary software distributions.
3. Harvest and web.mit.edu
   1. why WWWWAIS won't work
      1. Unacceptable: find /afs/athena -name '*.html' ...
      2. There are about 150 gigs under /afs/athena
   2. why robots won't work
      1. Normally, most files are cached
      2. Robots flush the AFS cache
   3. mods to harvest to make it less robot-like
      1. Harvest does simple Local-Mapping
      2. httpd does more sophisticated mapping
      3. Mods I've made to the Harvest gatherer