6.033 Lab 2 - A Remote Procedure Call Library
Preliminary Version

This assignment assumes that you are comfortable with the concepts of remote procedure calls. A good primer on remote procedure calls can be found in Birrell and Nelson's paper (see reading #12).

For this assignment, you will create a library which implements a remote procedure call protocol. The functions you export from that library are described in a later section. In addition, you are to write up your inital design in a preliminary design document, due on February 27th. A write-up of your final design is due on March 20th and can be used in place of your 6.033 case study.

Design Constraints

The goal of the remote procedure call is to make procedure calls over the network act in a similar fashion to in-process procedure calls. To accomplish this goal, the RPC protocol must support reliable, at-most once delivery. Reliable delivery means the remote procedure call is guaranteed to go through uncorrupted if the network, computer, and server on the other end are all operational. At-most once means that, despite the possibility of packet duplication in the network or computer restart, the procedure call gets executed either once or not at all on the server side.

To make the lab more challenging and realistic, your RPC protocol will be implemented on top of an unreliable network protocol known as the user datagram protocol (UDP). Unlike TCP, UDP exposes a packet or datagram interface to the user. A packet is a lot like a letter - it contains a finite amount of information, called the payload and a destination and source. Much like the post office, UDP is allowed to re-order, drop, duplicate, and corrupt packets. So, why would you want to use such a beast? Well, it's easy to implement and not all protocols need the guarantees provided by a reliable protocol such as TCP.

The UDP protocol is one of standard Internet protocols and is documented in RFC 768. For information on how to use the UDP protocol with the socket interface, type man udp.

UDP only allows packets of up to 64k in size. However, to make the protocol as flexible as possible, we require that the RPC protocol be able to reliably deliver arbitrarily large amounts of data.

Desirable Properties of your RPC Design

There are several desirable but not absolutely required properties in an RPC design. First, small RPC should run quickly, but large RPCs should occur efficiently. The protocol should be a good citizen on the network and avoid wasting network bandwidth by sending too many packets or retransmitting unnecessary data.

The Programmatic Interface

You must implement at least the following interfaces in your 6.033-compliant RPC library.

The functions below use three different types, which you may define as you wish. The SVCDAT type encapsulates any state that may be associated with a given RPC service. For example, the SVCDAT will probably contain the socket from which the server is reading requests as well as additional information needed to prevent re-play of RPC messages. The CLIREQ type encapsulates any state that must persist between the calling of get_request and put_reply. On the client side, the SRVINFO type represents the information necessary to contact the server.

These types, plus the prototypes for the functions below, are to be placed in a header file called myrpc.h.

Server Side

SVCDAT *bind_service ( const struct sockaddr *service_addr );

Requires: A sockaddr structure that describes the UDP port the service will be bound to.
Returns: NULL if the attempt to bind a service to the port failed. Otherwise, a pointer to a SVCDAT type
which can be used to communicate using the functions below

int get_request( SVCDAT *svc, CLIREQ **call, const char **buffer, unsigned int *size);

Requires: svc be a valid SVCDAT * returned by bind_service
Effects: Returns with a client request when one arrives
Returns: Zero in case of success, otherwise a negative integer. The *buffer variable will point to a sequence of bytes of size *size. The sequence is not necessarily NULL terminated. The *buffer pointer returned by get_request must be explicitly freed - otherwise, memory leaks may occur. Finally, *call points to a CLIREQ * type which contains state specific to this RPC call. It should be passed unmolested to put_reply.

int put_reply ( SVCDAT *svc, CLIREQ *call, const char *buffer, unsigned int size);

Requires: svc be a valid SVCDAT * returned by bind_service and call be a valid CLIREQ * returned by get_request and size be at most the allocated size of the buffer.
Effects: Sends back the bytes in the buffer as the response to the client request. Does not return until receipt of reply is acknowledge by the client. The client request state is then freed and invalidated
Returns: Zero in case of success, otherwise a negative integer.

void release_service ( SVCDAT *svc );

Requires: svc be a valid non-null SVCDAT returned by bind_service
Effects: Releases any state associated with the service, including allocated memory and UDP port bindings
Returns: Nothing.

 

Client Side

SRVINFO *setup_srvinfo ( const char *hostname, unsigned short port);

Requires: The service named by hostname and port be running and contactable via the network
Effects: Returns a SRVINFO structure with the necessary information to contact the server
Returns: NULL if the setup failed, otherwise a valid pointer to a SRVINFO

int rpc( SRVINFO *srv, const void *request, unsigned int req_size,
const char **reply, unsigned int *repl_size);


Effects: See definition of RPC above. Does not return until the reply has been received and acknowledged.
Returns: A negative integer on failure, otherwise zero for success. If successful, *reply points to the server reply of size *repl_size. These replies must be explicitly free'd by the caller.

void teardown_srvinfo (SRVINFO *srv);

Requires: srv be SRVINFO * returned by setup_srvinfo
Effects: Cleans up any state associated with contacting a server for a given RPC protocol

Stub Generation

The RPC library exposes an interface for passing around buffers of bytes. However, we'd like RPC calls to looks as much like standard C function calls as possible. To accomplish this feat, we write stub functions convert arguments and return values into byte buffers. For example, on the client side, a stub for the add function might look like this:

int cli_add (SRVINFO *srv, int a, int b)

{
      int  buf[3], res;
      int  *reply;
      unsigned int size;

      buf[0] = htonl(FN_ADD);
      buf[1] = htonl(a);
      buf[2] = htonl(b);

      if (rpc(srv, buf, sizeof(buf), &reply, &size) < 0)
         goto err_exit;

      if (size < 4) goto err_exit;

      res = ntohl(reply[0]);
      free(reply);

      return (res);

err_exit:
      return (rpc_error_handler(FN_ADD));   /* Some global error handler */
     
}    
The exact form of the buffer doesn't matter - just as long as the server and client agree.

How to Design and Build the Protocol

First, you should do some more reading past the Birrell paper. The Modern Operating Systems book by Andy Tanenbaum has an extensive section on building RPC protocols in chapter 10. Other recommend reading includes the Sun RPC spec, RFC1050, as well as the eXternal Data Representation spec, RFC1014. Note that the Sun RPC protocol does not have any provisions for arbitrarily long requests. For further tips on protocol design, you may want to check out Computer Networks, Third Edition, by Andy Tanenbaum, sections 3.3 and 3.4.

The first step is to design the wire protocol - that is, figure out when you will be sending packets and for what reasons. Then, in every place in your design where you send a packet, figure out the effects of dropping that packet, duplicating that packet, or reordering it with another packet. In addition, see what happens if the computer is restarted both before and after the packet arrives. If undesirable effects occur, iterate.

Once you have convinced yourself of the correctness of your design, you should implement the protocol on top of TCP. This will make sure that you can debug the data representation part of the protocol.

Then, move the protocol to UDP and make certain the RPC calls are still making it reliably, especially for larger transfers. Most likely, you will not see any packet loss when you test UDP between a server and client process that are both on your computer.

Finally, to simulate a horrible network, I recommend wrapping the code for sending out packets with code that randomly drops or duplicates packets it is sent. First do this on small RPCs, so that you do not have to deal with the extra protocol overhead of multi-packet RPC requests. Then, debug large RPCs under network lossage.

Once you have debugged a pathological network, you are ready to test your protocol between two machines on the network.

What to Hand In, Due Dates and Hand-in Procedure

A preliminary design document describing your RPC protocol is due on Thursday, February 27. It should contain a description of the protocol with arguments for why it meets the design constraints. In addition, the paper should describe any additional twists you may have decided to add to the protocol design.

The final code for the library is due on Tuesday, March 18. Place your assignment in your locker in a subdirectory called 6.033/lab2 and e-mail the lab TA, csapuntz@mit.edu when you are finished.

The final write-up for the RPC protocol is due on Thursday, March 20. This design paper should be comparable in length (maximum 10 pages) and quality to the first design projects for 6.033.

Collaboration

This is also an individual project. The design documents and code must be yours, but you are otherwise free to discuss the design and implementation details with others.