Hands-on 5: Log File Systems

In this hands-on assignment we will experiment with some of the same benchmarks used to measure the performance of the log-structured file system (LFS). We'll run these same benchmarks on file systems in use today.

This assignment should be done on an otherwise unused workstation that you have root access to. Athena workstations are acceptable (the root password can be found by typing tellme root at the athena prompt). To become root type su at the athena prompt and give the root password when prompted. If you use athena to complete this assignment you'll likely be testing the ext3 filesystem (running on Linux) or the UFS filesystem (running on Sun). Both of these filesystems are relatives of the Fast File System described in the LFS paper but ext3 actually borrows a few ideas from LFS. You can read a high-level overview of ext3 here.

Step 0: Preparation You will need to collect some information about the machine you are using before beginning the assignment: determine the name of the raw disk device for the machine's hard drive. You can find this info by looking at the output of the df -k command which lists the available space on each partition in KBytes. (Type man df for more details.) Either pick a partition that corresponds to the whole disk or one that is at least 100MB in size.

For example, running df -k on SunOS, we get:

athena% df -k

Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0t0d0s0    17542945  760860 16606656     5%    /
/proc                       0       0        0     0%    /proc
fd                          0       0        0     0%    /dev/fd
mnttab                      0       0        0     0%    /etc/mnttab
swap                  1368168       8  1368160     1%    /var/run
/dev/dsk/c0t0d0s3      634350  383323   193936    67%    /var/usr/vice
AFS                   9000000       0  9000000     0%    /afs

You will need to build some test software to complete this assignment. This software is found here. Untar the source and build it with the following commands:

athena% add -f gnu
athena% gtar zxvf lfs.tgz
athena% cd lfs
athena% gmake

If you have problems compiling the software on certain older Linux systems, try a statically-linked, precompiled version of the software here.

Step 1: Raw Data Transfer Rate/Latency To begin this hands-on assignment we determine the maximum data transfer rate and average latency of the fixed disk attached to the machine we are using. The maximum data transfer rate is the maximum data rate that can be sustained when reading many consecutive sectors. We will measure disk performance by accessing the disk in "raw" format. This avoids the overhead imposed by accessing the disk through the file system. Later we will compare these results to those obtained by accessing the disk via the file system.

1.A Data Rate There are two ways to find the maximum data transfer rate of your disk:

1) Using the supplied benchmarking program: bw. This program expects two arguments: the name of the raw disk device (collected above), and the number of megabytes to read from disk. Specify the -b option to measure the data transfer rate. The command must be run as root on most systems. Results are most reliable if at least 64 MB are read. Experiment with increasing sizes until the transfer rate is stable. For example,

athena# ./bw -b /dev/dsk/c0t0d0s0 64

2) Using the dd and time commands. Type man dd for more details. For example,

athena# time dd if=/dev/dsk/c0t0d0s0 of=/dev/null bs=1024k count=64

1.B Latency Find the average latency using the bw command (with the -l option):

In this mode, bw expects two arguments: the name of the device and the number of trials to run. For each trial, bw will seek to a random location on disk and read a small amount of data. It will report the average latency to begin a read (this latency includes both the time to position the disk heads and the rotational latency). Increasing the number of trials will produce better results. Experiment until you are confident in your answer. Make sure that the specified partition is at least 100MB in size. For example,

athena# ./bw -l /dev/dsk/c0t0d0s0 1000

Step 2: LFS Benchmarks We will now run a version of the benchmarks from the LFS paper. The results of these benchmarks (as run by the authors) can be seen in the graphs on pages 45 and 46 of the LFS paper. Please review section 5.1 of the paper which describes these tests in detail.

2.A First, run the small file benchmark using smallfb. This program expects two arguments. The number of files to create (10000 in the paper) and the size of each file (1024 bytes in the paper). Run smallfb with these values and report your results. This command may take a long time (minutes) to complete. You may experiment with lower values, but if you set the number of files too low you may end up testing your disk cache and not your disk. Note: make sure that you run this command from a directory which resides on a disk local to your workstation (smallfb will test the current working directory). /var/tmp is a good choice on most Athena machines. Do not run this command from your home directory or another network file system. The smallfb command will create a large file (and periodically read it) in an attempt to flush the disk cache: make sure that the partition you run this command on has enough free disk space (about 1 GB) to allow space for this file (again, check with df -k). For example,

athena% cd /var/tmp
athena% [path to lfs]/smallfb 10000 1024

This command can take a long time (minutes) to complete. The grinding noise your hard drive is making indicates that the test is running.

2.B Now run the large file benchmarking program: largefb. Largefb takes one argument: the size of the output file in Megabytes. Run largefb and report your results. For example,

athena% cd /var/tmp
athena% [path to lfs]/largefb 64

Step 3: Analysis Given what you know about the (non-log-structured) file system running on your test machine, explain the performance of your file system on these benchmarks. Questions to consider:

1. What is the ratio of the actual data transfer rate achieved in the small file benchmark to the raw data transfer rate of the disk? In other words, are you able to take advantage of the raw performance of your disk? What accounts for the discrepancy?

2. What is the limiting factor in each of the two tests (large/small file benchmarks)?

3. Would you expect the log file system perform better in these tests? Why?

4. Optional: Are there other aspects of the file system that can affect the performance of these tests? Those who tested on the Linux ext2 file system should consider the fact that ext2 relaxes the consistency guarantees of FFS by not synchronously writing meta-data (inodes, superblock, freelist etc.) to disk. Similarly, ext3's use of a log may offer advantages over FFS.

There may not be obvious answers to these questions. Disk caches, file system policies, and system overhead can confuse the numbers. Do your best to explain the numbers you see.