• archiving: keeping old data values in case the user needs to look at them again.
• recoverability: the system may crash, and invidividual in-progress transactions may abort, but committed transactions are always remembered.
• storage modularity: data may cross page boundaries.
• storage efficiency: pages may contain data of different transactions.
• coordination and isolation: the locks aren't mentioned in this paper, but they must have complicated the design.)

• Shadowed files
• Log copy of new/old record values [the EXOR log was a later idea]
• Log record of transaction start/end/save
• Log record of system checkpoint
• archive copy from disk -> tape
• RSS actions are individually atomic
• Log is duplexed)

                        RAID          DEC        IBM
multiple, identical
copies                 mirror       shadow     duplex

modify a copy, not
the original                                   shadow

unit of disk
storage                block                    page

• every incomplete transaction
• the latest checkpoint
• the latest time when data was archived to tape

Archiving:

keep a record of old data values, for future auditing

Replication:

fight back against hardware decay of current data values

Journaling:

group updates to different records into recoverable transactions)

• They allocate a separate disk partition--called an "extent"--to hold the log. (p. 240) If duplexed, there are two such partitions.
• There is a log manager program that directly reads and writes to this partition; the log is not a file in the file system.
• The log partition is managed as a big ring buffer of disk sectors. (p. 236) The next log page to be written goes into the next sector of the partition.
• There is a single RAM buffer for the log; that is where things are "written to the log". (p. 232) When the buffer fills, it is written to the disk into the current log sector, the buffer is cleared, and the record of the current log sector advanced by one. When checkpoints and commit records are written, the current buffer is simply written to the disk in the current log sector. This is what "forcing the log" means.

Now consider four different ways System R could be organized to take advantage of this property of shadow files.

(In that case, the system might run all day, logging planned writes and then doing those writes, and then logging commits. Then, a crash occurs. All writes are lost; the disk looks just like it did when the system started--except for the log, which is not a shadowed file, it is in a partition of its own. But since the log contains all the writes and commit records, we can recover by REDO'ing every committed transaction.)

(It now needs to REDO only transactions that committed after the FILESAVE. The log from before that point can be discarded.)

(All writes up to that commit are permanent, even writes for uncommitted transactions that happened to be in progress at the time the most recent transaction committed. All writes after that commit are as if they had never happened. From this state, we can recover by UNDO'ing the logged writes of every transaction that was pending at the time of the last commit.)

(If a crash occurs after a checkpoint, all writes up to the checkpoint are permanent (perhaps including some writes for transactions that never got to commit), all writes after the checkpoint are lost (perhaps including some writes for transactions that did commit). So the recovery procedure is to REDO any writes after the checkpoint that belong to committed transactions, and UNDO any writes before the checkpoint that belong to losers.)

At this point it is appropriate to sketch a log scenario on the board such as the one of figure 11 on page 235 of the paper, with the addition of pointers to show the status of shadow versus current pages for various files that the five transactions may happen to have open. For an example, there is a scan of a nearly unintelligible hand-drawn figure. (Karen Sollins contributed easier-to-read Acrobat and PowerPoint versions of most of that same figure.

The idea is to examine each of the transactions T1 - T5 in turn to see how it is affected by a system crash at the point indicated. Those five transactions include every possible combination of {completed, incomplete} at crash time and {preceding, spanning, following} the latest checkpoint.)

1. write a checkpoint record to the log.
2. force the log to disk
3. force all shadow page maps to disk
4. force to disk new directory entries that refer to the new page maps
5. force to disk a new copy of the file system directory, that contains these new directory entries.
6. write a one-sector root record (that contains the pointer to the file system directory and the pointer to the log address of the checkpoint) twice, using a recoverable-put algorithm like the one described in chapter 8.

The assignment question from 1992: When system R takes a system checkpoint it waits until the system state is action consistent but not transaction consistent. Discuss briefly the reasons for this decision and its implications for recovery.

There are several very useful questions that might be used to spur discussion on pages 7-101 and 7-102. The following two additional discussion questions were suggested by Eric McDonald:

1) System R possesses the property of "consistency," essentially transforming concurrent transactions on a shared object (i.e. a file) into a serial set of transactions. Discuss the reasoning behind this decision. Describe a situation involving concurrency and shared objects where System R could cause problems if it didn't serialize concurrent transactions on shared objects.

(2) In section 3.9, the authors describe their failure to include a recovery system for output messages as a "major design error." Do you think they're being a little hard on themselves? Try to identify, by giving two significantly different real-world examples, why they're so concerned with message recovery.