@device(PostScript)
@make(Plan)
@planhead(versiondate="13 November 1986",
          plansection="Section E.3.1",
          copyrightdate="1986",
          title="Remote Virtual Disk Upgrade",
          Author="by J. H. Saltzer")

Technical Plan Section D describes the requirements for and planned
uses of remote virtual disk and remote file systems.  This section
extends that discussion by describing in detail a plan for upgrade of
the Remote Virtual Disk System originally developed by the M.I.T.
Laboratory for Computer Science.  A separate System Manual describes
operation and use of the Remote Virtual Disk System.

This upgrade plan is organized in terms of a series of releases, with
release-by-release feature lists.  All those upgrade ideas that are
not actually scheduled for implementation are collected together in a
single release description dubbed the outplan release.

For purposes of discussing release, this document describes the
Remote Virtual Disk system in four major parts:
@tabclear()@tabset(0.5inch, 1inch, 4inch)
@begin(format, use sanserif)
@\I.@\the server 
@\II.@\the client driver
@\III.@\the client commands
@\IV.@\the documentation

@end(format)

Versions 1.x of the various parts are uncoordinated; versions
starting with 2.0 consist of coordinated releases of all four parts.
Current target release dates:

@begin(format, use sanserif)

@\2.0@\Initial fall release@\7/29/86
@\2.1@\Cleanups, ioctl interface@\8/29/86
@\2.2@\UID's, nonces, and physical ops@\10/3/86
@\3.0@\initial Kerberos Integration@\11/14/86
@\3.1@\scheduled wrapups@\11/30/86
@\3.2@\everything else@\outplan

@end(format)

@Majorheading(I.  RVD server)


@Heading[Server version 1.0/1.1.]
Server obtained from L.C.S. in June, 1985.


@Heading[Server version 1.2 (1/1/86)]
@enumerate[
Bug fix@Y<M>avoid crashing on delete_virtual operations.
]

@heading[Server version 1.3 (5/22/86)]
@enumerate[
Comes up with no spinups allowed; requires an allow_spinups
     operation to start it.

Logs all spindowns, including those from spindown_host
     and spindown_virtual.

All logging to the 4.3 syslogd.

Ignores log_truncate requests; crontab-triggered manipulation
     of syslog moves old logs aside.

Code to use RVDMASTER (old data base format) removed.

Display_virtual returns total number of connections and only
     one packet full of connection descriptions; can be told which
     description to start with.  Can return either connections of
     a client or connections of a pack.

Allow_spinups, a new operation, controls the mode of
     allowable spinups on server as a whole or on specific packs.

Set_message and get_message permit a message-of-the-day to
     be posted by operations and to be read by clients.

Require_authorization, a new operation, tells server to
     read its authorization file for a password and begin accepting
     requests from other network nodes.  If invoked while running,
     server refreshes its stored password from the authorization file.
]

@Heading[Server version 1.4 (6/10/86)]
@enumerate[
Display_virtual has a new option to return a list of all
     packs that have one or more spinups, with time since most recent
     use.

Logging operations now require operations password.

Accepts the operations password to allow spinup of packs
     in otherwise disallowed modes.

Logs shutdowns.

Bughalt on receiving "network unreachable" changed to
    increment a statistics counter, and carry on.
]

@Heading[Server version 1.5 (6/15/86)]
@enumerate[
Ported to IBM RT PC.
]

@Heading[Server version 1.6 (6/30/86)]
Never deployed@Y<M>changes integrated in 2.0.


@Heading[Server version 2.0@Y<M>Initial Fall release)]
@enumerate[
Has a version number that goes into log at startup time.

Client errors are distinguished from server errors so that
     server errors can be directed to a separate log.

Logs attempts to do control operations with wrong password.

General code review.
]

@Heading[Server version 2.1@Y<M>Minor cleanups and test suite]
@enumerate[
Gathers and logs server queue length statistics.

Bughalts now log all statistics before exiting.

Rvdexchanges are logged correctly.

A server test suite provides a client package that pounds
     the server in any of several different ways and can be run on
     several clients at once.  It also tests every RVD
     control function and every RVD and RVDCTL error condition that a
     client can generate.  A checkout mode in the server causes it
     to think it is getting all possible disk error conditions from the
     kernel and from the network.
]

@Heading[Server version 2.2@Y<M>uid, nonces, and physical ops]
@enumerate[
Most calls to bughalt are replaced with more sensible
     responses.

The server accepts a -r option to mean that if a bughalt
     is encountered, the server should automatically restart itself.

Pack unique id's are provided to make rvdexch atomic.  The
     pack UID is returned in the spinup-ack packet.  The operations
     add_virtual, and exchange_virtual require pack UID's, and modify_
     virtual allows them, as a way of renaming packs.  A new respinup
     packet type accepts spinup with pack uid rather than name.

Server responds to nonces in control packets by including them
     in response packets.

Server provides delete_physical, use_physical, and 
     disuse_physical control operations.

Server defines and returns a new error code ("requested mode
     temporarily unavailable") when allow_spinups has restricted
     spinup modes more tightly than that specified in add_virtual.
]

@Heading[Server version 3.0@Y<M>initial Kerberos integration]
@enumerate[
Server recognizes authenticated-spinup packet type, containing
     a Kerberos ticket, and interprets capability of pack as the name
     of an access control list, and owner field of pack as name of a
     user with full access.

Server accepts Kerberos tickets in control operations, and
     maintains separate access control lists for operations,
     maintenance, and shutdown.

Server no longer logs "display_virtual: no such connection"
     incidents.
]

@Heading[Server version 3.1@Y<M>wrapups]
@enumerate[
Server scheduling allows control operations during heavy
     read/write activity.

Server provides get_load function.
]

@Heading[Server Release 3.2 (outplan)]
@enumerate[
Spinup should report pack in use in incompatible mode even
     if password isn't supplied.

A way is needed to extract rvddb information from a running
     server.

Should have general set/get control operation pairings.

Server should catch and handle the signal that is associated
     with shutdown, by doing a graceful shutdown.

Measure performance:  maximum service rates and fanout.

Server test suite:  write the man page, finish writing tests,
     and put rvdtest into the release tree.
]

@Majorheading[II.  Client driver]


@Heading[Client version 1.0.]
Driver obtained from L.C.S. in June, 1985


@Heading[Client version 1.1/1.2.  (1/1/86)]
@enumerate[
New state (misnamed "server crashed") added to client driver.
     Stops a workstation from continually retrying to access a formerly
     spunup pack after a server is restarted.
]

@Heading[Client version 1.3  (6/15/86)]
@enumerate[
Ported to IBM RT PC.
]

@heading[Client version 1.4 (6/30/86)]
Never deployed!@Y<M>changes integrated in 2.0.


@Heading[Client version 2.0@Y<M>Initial Fall Release]
@begin(enumerate)
Fix spinup to try only 5 times, then give up.
     
Adjust burst size from 32 to 16.

Spinup: if user tries to spinup an already spunup pack,
     resend the request.  (Allows recovery in many cases if server
     crashed.  In future should check returned pack uid to verify
     that it didn't change.)

Recognize hard error return code, retry a few times, then
     return hard error status, rather than retrying forever.
     [N.B.: latest version in castor:~philipp/rvd tries to return all
     readable data in this burst; that version should be shaken down
     and installed, so that rvdcopy can take advantage of it.]

Redesign retry timeout on read/write.  On reads and writes,
     retry once quickly (e.g. 500 ms.) then if no response, repeatedly
     double the timeout, up to a maximum of 10 seconds.  Statistics
     should show separately the number of short and long timeouts.

Fix spinup code so that it passes along (and stores) all 32
     allowed bytes of passwords.

Find and fix [vdcopy: 0 pte] bug.  Occurs under heavy load
     with multiple processes using the client driver; generates
     kernel panic & crash.
@end(enumerate)

@Heading[Client version 2.1@Y<M>new ioctl interface]
@enumerate[
Redesign the client interface to use ioctl calls on
     /dev/rvdctl rather than additional supervisor entry points.

Add an ioctl that returns the version number of the
     structure declarations of the client call interface.

Change client to store pack name as part of drive state, and
     return it on vdstats ioctl.

Add an ioctl that returns the number of rvd devices
     implemented in the driver.

Driver should store and vdstats should return the (not yet
     returned) pack uid.

Change spindown call to check for busy file system and
     reject call, but allow user to force a spindown anyway.

Don't send spindown packets for drives that aren't spun up.
]

@heading[Client version 2.2@Y<M>uid, nonces, physical ops]
@enumerate[
spinup: stores returned pack uids (and also the originally
     supplied password) as part of the client driver state.  If user
     tries to spinup an already spunup pack, sends a respinup request
     based on uid.

read/write: if "no such connection" error comes back from
     server, sends a respinup packet and prepares to retry the
     read/write if the respinup is successful.

Hard spinup option; if the spinup succeeds, then on
     reads and writes, the client retries forever if the host
     stops responding.  On soft spinup, retries give up
     after 2 minutes.

Total timeout time on spinup/down is shortened to five
     retries of one second each.
]

@heading[Client version 3.0@Y<M>Kerberos integration]
@enumerate[
Sends authenticated-spinup packet type containing
     a Kerberos ticket.
]

@heading[Client version 3.1@Y<M>scheduled wrapups]
@enumerate[
Review all panics; eliminate where possible.
]

@heading[Client version 3.2 (outplan)]
@enumerate[
Create pseudo server that returns all possible error
     conditions, to test client driver responses.  Develop
     standard test suite for client driver.

Logging via syslog to a central log service.
]


@Majorheading[III.  Client commands]

@heading[Commands Version 1.0]

  Client commands obtained from L.C.S. in June, 1985.

@heading[Commands version 1.1  (6/1/86)]
@enumerate[
New commands rvdsetm (invokes set_message) rvdgetm (invokes
     get_message) and rvdallow (invokes allow_spinups).  rvdshow has
     an option to get list of spunup packs and can cope with more
     than 12 spinups.

Up command reprogrammed in C, changed to invoke rvdgetm once
     for each server used and rfsck for each writeable file system
     mounted.  Also allows multiple entries on different servers for
     a single drive/directory combination in /etc/rvdtab, and cycles
     through them till one responds.

Vddb now has a list operation and an exchange operation.

Vdstats output is a little more readable.

Most client commands have a -d option for debugging
     (displays the request and response packets.)

Vddb can manage a server running on a different host.  (Permits
     central management of several servers.)

New rvdcopy command for fast transfer of pack contents.
]

@heading[Commands version 2.0@Y<M>Initial Fall release]
@enumerate[

Redesign Up/Down commands for better user interface.

Magic number checking subroutine that can detect attempts to
     mount IBM file systems on DEC processors and vice-versa.

Up command: change to use magic number checker, and give
     user-friendly comment about what is probably wrong if the bytes are
     out of order.

Up command: change to allow one-time spinups if running on a
     workstation or running as root.  (requires new system type
     variable.)

Up command: change to look for .rvdtab in user's directory
     if running on a workstation.

Up command: when interrupted, report what state it is
     leaving the client in.

Up command: shouldn't suggest server is down if user gives
     wrong pack password.

Down command:  occasionally spins down wrong disk!

Make sure up and spinup allow 32-byte passwords.

Add option -f to rvdsetm that suppresses prompting for a
     password, so that it can be used in /etc/rc before
     require_authorization has been issued.
]

@Heading[Commands version 2.1@Y<M>new ioctl interface]
@enumerate[
Redesign to use driver ioctl interface; check structure
     version number.

Merge the commands into two programs that look to see what
     name they were invoked under, to avoid needing multiple binary
     copies of all the libraries.

Review error response and user interface design of all
     client commands; take advantage of pack name stored by client driver

Spinup no longer tries to change owner of virtual device.
]

@Heading[Commands version 2.2@Y<M>uid, nonces, and physical ops]
@enumerate[
Rvdexch command uses pack uid's.

Vddb:  allows pack rename based on uid.

New spinup options: allows choice of hard/soft spinups,
     and respinup.

newvd:  sets mode of newly created root to 555, not 500.
]

@Heading[Commands version 3.0@Y<M>Kerberos integration]
@enumerate[
Integrate with Kerberos.

vddb checks for write access to named rvddb before starting
     work.  Doesn't require authorization password when talking to a
     remote system.
]

@Heading[Commands version 3.1@Y<M>scheduled wrapups]
@enumerate[
vddb exchange request uses nonces.

New control command allows use/disuse of disk partitions.

New control command to allow_spinups.

Can now install a null password with vddb.

vddb:  add delete_physical feature.

up/down commands replaced with bind-based attach/detach
     commands.  rvdflush flushes servers mentioned in attach table.

]

@Heading[Commands version 3.2 (outplan)]
@enumerate[
rvdcopy:  add option to suppress progress reports.

vddb:  should do rcp of rvddb if working for a remote site.

vddb:  figure out way to verify operations password of remote
     system at outset rather than at first modification.

Fix /etc/shutdown to shutdown rvd server, too.

Design method for user to change passwords and exchange
     packs, giving only user passwords and changing the data base.

Add options on savervd/restorervd.

newvd: verify that file system is not mounted.

Add nonce support to rest of RVD library.

Find and fix segmentation faults that occur in vddb.
]

@Majorheading[IV.  Documentation]

@Heading[Documentation release 2.0.]
@enumerate[
RVDCTL protocol document expanded and updated.

Complete review of all man pages.

Available documentation brought together into a single
     notebook; overview, installation notes, operations guide, and
     example standard server configuration added.
]

@Heading[Documentation release 2.1.]
@enumerate[
RVDCTL protocol document updated.

RVD protocol document reviewed and updated.  

Expanded operations guide.

Server and client statistics documentation added.
]

@Heading[Documentation release 2.2.]
@enumerate[
Document limits:  maximum password length, maximum pack name
     length, maximum number of connections, maximum number of packs,
     maximum burstsize allowed by server, etc.

Cookbook for replacing a disk used by an RVD server.
]

@Heading[Documentation release 3.0.]
@enumerate[
Document Kerberos integration.
]