Reading 6: Specifications

Many of the nastiest bugs in programs arise because of misunderstandings about behavior at the interface between two pieces of code. Although every programmer has specifications in mind, not all programmers write them down. As a result, different programmers on a team have different specifications in mind. When the program fails, it’s hard to determine where the error is. Precise specifications in the code let you apportion blame (to code fragments, not people!), and can spare you the agony of puzzling over where a fix should go.

Specifications are good for the client of a method because they spare the task of reading code. If you’re not convinced that reading a spec is easier than reading code, take a look at some of the standard Java specs and compare them to the source code that implements them.

Here’s an example of one method from BigInteger — a class for representing integers up to arbitrary size without the size limit of primitive int — next to its code:

if (val.signum == 0)
    return this;
if (signum == 0)
    return val;
if (val.signum == signum)
    return new BigInteger(add(mag, val.mag), signum);

int cmp = compareMagnitude(val);
if (cmp == 0)
    return ZERO;
int[] resultMag = (cmp > 0 ? subtract(mag, val.mag)
                   : subtract(val.mag, mag));
resultMag = trustedStripLeadingZeroInts(resultMag);

return new BigInteger(resultMag, cmp == signum ? 1 : -1);

The spec for BigInteger.add is straightforward for clients to understand, and if we have questions about corner cases, the BigInteger class provides additional human-readable documentation. If all we had was the code, we’d have to read through the BigInteger constructor, compare­Magnitude, subtract, and trusted­StripLeadingZero­Ints just as a starting point.

Specifications are good for the implementer of a method because they give the implementer freedom to change the implementation without telling clients. Specifications can make code faster, too. We’ll see that using a weaker specification can rule out certain states in which a method might be called. This restriction on the inputs might allow the implementer to skip an expensive check that is no longer necessary and use a more efficient implementation.

The contract acts as a firewall between client and implementer. It shields the client from the details of the workings of the unit — you don’t need to read the source code of the procedure if you have its specification. And it shields the implementer from the details of the usage of the unit; he doesn’t have to ask every client how she plans to use the unit. This firewall results in decoupling, allowing the code of the unit and the code of a client to be changed independently, so long as the changes respect the specification — each obeying its obligation.

Consider these two methods. Are they the same or different?

static int findFirst(int[] arr, int val) {
    for (int i = 0; i < arr.length; i++) {
        if (arr[i] == val) return i;
    }
    return arr.length;
}

static int findLast(int[] arr, int val) {
    for (int i = arr.length -1 ; i >= 0; i--) {
        if (arr[i] == val) return i;
    }
    return -1;
}

Of course the code is different, so in that sense they are different; and we’ve given them different names, just for the purpose of discussion. To determine behavioral equivalence, our question is whether we could substitute one implementation for the other.

Not only do these methods have different code, they actually have different behavior:

• when val is missing, findFirst returns the length of arr and findLast returns -1;
• when val appears twice, findFirst returns the lower index and findLast returns the higher.

But when val occurs at exactly one index of the array, the two methods behave the same: they both return that index. It may be that clients never rely on the behavior in the other cases. Whenever they call the method, they will be passing in an arr with exactly one element val. For such clients, these two methods are the same, and we could switch from one implementation to the other without issue.

The notion of equivalence is in the eye of the beholder — that is, the client. In order to make it possible to substitute one implementation for another, and to know when this is acceptable, we need a specification that states exactly what the client depends on.

In this case, our specification might be:

static int find(int[] arr, int val)

static int findFirst(int[] a, int val) {
    for (int i = 0; i < a.length; i++) {
        if (a[i] == val) return i;
    }
    return a.length;
}

static int findLast(int[] a, int val) {
    for (int i = a.length - 1 ; i >= 0; i--) {
        if (a[i] == val) return i;
    }
    return -1;
}

A specification of a method consists of several clauses:

• a precondition, indicated by the keyword requires
• a postcondition, indicated by the keyword effects

The precondition is an obligation on the client (i.e., the caller of the method). It’s a condition over the state in which the method is invoked.

The postcondition is an obligation on the implementer of the method. If the precondition holds for the invoking state, the method is obliged to obey the postcondition, by returning appropriate values, throwing specified exceptions, modifying or not modifying objects, and so on.

The overall structure is a logical implication: if the precondition holds when the method is called, then the postcondition must hold when the method completes.

If the precondition does not hold when the method is called, the implementation is not bound by the postcondition. It is free to do anything, including not terminating, throwing an exception, returning arbitrary results, making arbitrary modifications, etc.

static int find(int[] arr, int val)

Specifications in Java

static int find(int[] arr, int val)

/**
 * Find a value in an array.
 * @param arr array to search, requires that val occurs exactly once
 *            in arr
 * @param val value to search for
 * @return index i such that arr[i] = val
 */
static int find(int[] arr, int val)

static boolean isPalindrome(String word)

Null references

In Java, references to objects and arrays can also take on the special value null, which means that the reference doesn’t point to an object. Null values are an unfortunate hole in Java’s type system.

Primitives cannot be null:

int size = null;     // illegal
double depth = null; // illegal

and the compiler will reject such attempts with static errors.

On the other hand, we can assign null to any non-primitive variable:

String name = null;
int[] points = null;

and the compiler happily accepts this code at compile time. But you’ll get errors at runtime because you can’t call any methods or use any fields with one of these references:

name.length()   // throws NullPointerException  
points.length   // throws NullPointerException

Note, in particular, that null is not the same as an empty string "" or an empty array. On an empty string or empty array, you can call methods and access fields. The length of an empty array or an empty string is 0. The length of a string variable that points to null isn’t anything: calling length() throws a NullPointer­Exception.

Also note that arrays of non-primitives and collections like List might be non-null but contain null as a value:

String[] names = new String[] { null };
List<Double> sizes = new ArrayList<>();
sizes.add(null);

These nulls are likely to cause errors as soon as someone tries to use the contents of the collection.

Null values are troublesome and unsafe, so much so that you’re well advised to remove them from your design vocabulary. In 6.031 — and in fact in most good Java programming — null values are implicitly disallowed in parameters and return values. So every method implicitly has a precondition on its object and array parameters that they be non-null. Every method that returns an object or an array implicitly has a postcondition that its return value is non-null. If a method allows null values for a parameter, it should explicitly state it, or if it might return a null value as a result, it should explicitly state it. But these are in general not good ideas. Avoid null.

There are extensions to Java that allow you to forbid null directly in the type declaration, e.g.:

static boolean addAll(@NonNull List<T> list1, @NonNull List<T> list2)

where it can be checked automatically at compile time or runtime.

Google has their own discussion of null in Guava, the company’s core Java libraries. The project explains:

Careless use of null can cause a staggering variety of bugs. Studying the Google code base, we found that something like 95% of collections weren’t supposed to have any null values in them, and having those fail fast rather than silently accept null would have been helpful to developers.

Additionally, null is unpleasantly ambiguous. It’s rarely obvious what a null return value is supposed to mean — for example, Map.get(key) can return null either because the value in the map is null, or the value is not in the map. Null can mean failure, can mean success, can mean almost anything. Using something other than null makes your meaning clear.

(Emphasis added.)

reading exercises

NullPointerException accessing exercise.name()

Which of the following can be null?

If you’re not sure, try it yourself in a small Java program.

Check all that apply:

int a;(missing answer)

char b;(missing answer)

double c;(missing answer)

int[] d;(missing answer)

String e;(missing answer)

String[] f;(missing answer)

Double g;(missing answer)

List<Integer> h;(missing answer)

final MouseTrap i;(missing answer)

static final String j;(missing answer)

(missing explanation)

check explain

There are null exercises remaining

public static String none() {
    return null;          // (1)
}

public static void main(String[] args) {
    String a = none();    // (2)
    String b = null;      // (3)
    if (a.length() > 0) { // (4)
        b = a;            // (5)
    }
    return b;             // (6)
}

Which line contains a static error?

1,2,3,4,5,6(missing answer)

(missing explanation)

If we comment out that line and run main…

Which line contains a dynamic error?

1,2,3,4,5,6(missing answer)

(missing explanation)

check explain

What a specification may talk about

In testing, we talk about black box tests that are chosen with only the specification in mind, and glass box tests that are chosen with knowledge of the actual implementation (Testing). But it’s important to note that even glass box tests must follow the specification. Your implementation may provide stronger guarantees than the specification calls for, or it may have specific behavior where the specification is undefined. But your test cases should not count on that behavior. Test cases must obey the contract, just like every other client.

For example, suppose you are testing this specification of find, slightly different from the one we’ve used so far:

static int find(int[] arr, int val)

This spec has a strong precondition in the sense that val is required to be found; and it has a fairly weak postcondition in the sense that if val appears more than once in the array, this specification says nothing about which particular index of val is returned. Even if you implemented find so that it always returns the lowest index, your test case can’t assume that specific behavior:

int[] array = new int[] { 7, 7, 7 };
assertEquals(0, find(array, 7));  // bad test case: violates the spec
assertEquals(7, array[find(array, 7)]);  // correct

Similarly, even if you implemented find so that it (sensibly) throws an exception when val isn’t found, instead of returning some arbitrary misleading index, your test case can’t assume that behavior, because it can’t call find() in a way that violates the precondition.

So what does glass box testing mean, if it can’t go beyond the spec? It means you are trying to find new test cases that exercise different parts of the implementation, but still checking those test cases in an implementation-independent way.

Testing units

/** @return the contents of the web page downloaded from url */
public static String getWebPage(URL url) { ... }

/** @return the words in string s, in the order they appear,
 *          where a word is a contiguous sequence of
 *          non-whitespace and non-punctuation characters */
public static List<String> extractWords(String s) { ... }

/** @return an index mapping a word to the set of URLs
 *          containing that word, for all webpages in the input set */
public static Map<String, Set<URL>> makeIndex(Set<URL> urls) { 
    ...
    calls getWebPage and extractWords
    ...
} 

We previously discussed mutable vs. immutable objects, but our specifications of find didn’t give us the opportunity to illustrate how to describe side-effects — changes to mutable data — in the postcondition.

Here’s a specification that describes a method that mutates an object:

static boolean addAll(List<T> list1, List<T> list2)

We’ve taken this, slightly simplified, from the Java List interface. First, look at the postcondition. It gives two constraints: the first telling us how list1 is modified, and the second telling us how the return value is determined.

Second, look at the precondition. It tells us that the behavior of the method if you attempt to add the elements of a list to itself is undefined. You can easily imagine why the implementer of the method would want to impose this constraint: it’s not likely to rule out any useful applications of the method, and it makes it easier to implement. The specification allows a simple implementation in which you take an element from list2 and add it to list1, then go on to the next element of list2 until you get to the end. The sequence of snapshot diagrams at right illustrate this behavior. So if list1 and list2 are the same list, this simple algorithm will not terminate — or practically speaking it will throw a memory error when the list object has grown so large that it consumes all available memory. Either outcome, infinite loop or crash, is permitted by the specification because of its precondition.

Remember also our implicit precondition that list1 and list2 must be valid objects, rather than null. We’ll usually omit saying this because it’s virtually always required of object references.

Here is another example of a mutating method:

static void sort(List<String> lst)

And an example of a method that does not mutate its argument:

static List<String> toLowerCase(List<String> lst)

Just as we’ve said that null is implicitly disallowed unless stated otherwise, we will also use the convention that mutation is disallowed unless stated otherwise. The spec of to­Lower­Case could explicitly state as an effect that “lst is not modified”, but in the absence of a postcondition describing mutation, we demand no mutation of the inputs.

public static int gcd(int a, int b) {
    if (a > b) {
        return gcd(a-b, b);
    } else if (b > a) {
        return gcd(a, b-a);
    }
    return a;
}

@Test public void gcdTest() {
    assertEquals(6, gcd(24, 54));
}

Now that we’re writing specifications and thinking about how clients will use our methods, let’s discuss how to handle exceptional cases in a way that is safe from bugs and easy to understand.

A method’s signature — its name, parameter types, return type — is a core part of its specification, and the signature may also include exceptions that the method may trigger.

Exceptions for signaling bugs

Exceptions for special results

class BirthdayBook {
    LocalDate lookup(String name) { ... }
}

LocalDate lookup(String name) throws NotFoundException {
    ...
    if ( ...not found... )
        throw new NotFoundException();
    ...

BirthdayBook birthdays = ...
try {
    LocalDate birthdate = birthdays.lookup("Alyssa");
    // we know Alyssa's birthday
} catch (NotFoundException nfe) {
    // her birthday was not in the birthday book
}

try {
    LocalDate birthdate = birthdays.lookup("Elliot");
}

try {
    LocalDate birthdate = birthdays.lookup("Elliot");
} catch (NotFoundException nfe) {
    birthdate = LocalDate.now();
}

try {
    LocalDate birthdate = birthdays.lookup("Elliot");
} catch (NotFoundException nfe) {
    throw new DateTimeException("Missing reference birthday", nfe);
}

We’ve seen two different purposes for exceptions: special results and bug detection. As a general rule, you’ll want to use checked exceptions to signal special results and unchecked exceptions to signal bugs. In a later section, we’ll refine this a bit.

Some terminology: checked exceptions are called that because they are checked by the compiler:

• If a method might throw a checked exception, the possibility must be declared in its signature. Not­Found­Exception would be a checked exception, and that’s why the signature ends throws Not­Found­Exception.
• If a method calls another method that may throw a checked exception, it must either handle it, or declare the exception itself, since if it isn’t caught locally it will be propagated up to callers.

So if you call BirthdayBook’s lookup method and forget to handle the Not­Found­Exception, the compiler will reject your code. This is very useful, because it ensures that exceptions that are expected to occur will be handled.

Unchecked exceptions, in contrast, are used to signal bugs. These exceptions are not expected to be handled by the code except perhaps at the top level. We wouldn’t want every method up the call chain to have to declare that it (might) throw all the kinds of bug-related exceptions that can happen at lower call levels: index out of bounds, null pointers, illegal arguments, assertion failures, etc.

As a result, for an unchecked exception the compiler will not check for try-catch or a throws declaration. Java still allows you to write a throws clause for an unchecked exception as part of a method signature, but this has no effect, and is thus a bit funny, and we don’t recommend doing it.

All exceptions may have a message associated with them. If not provided in the constructor, the reference to the message string is null.

Throwable hierarchy

To understand how Java decides whether an exception is checked or unchecked, let’s look at the class hierarchy for Java exceptions.

Throwable is the class of objects that can be thrown or caught. Throwable’s implementation records a stack trace at the point where the exception was thrown, along with an optional string describing the exception. Any object used in a throw or catch statement, or declared in the throws clause of a method, must be a subclass of Throwable.

Error is a subclass of Throwable that is reserved for errors produced by the Java runtime system, such as StackOverflow­Error and OutOfMemory­Error. For some reason Assertion­Error also extends Error, even though it indicates a bug in user code, not in the runtime. Errors should be considered unrecoverable, and are generally not caught.

Here’s how Java distinguishes between checked and unchecked exceptions:

• RuntimeException, Error, and their subclasses are unchecked exceptions. The compiler doesn’t require them to be declared in the throws clause of a method that throws them, and doesn’t require them to be caught or declared by a caller of such a method.
• All other throwables — Throwable, Exception, and all of their subclasses except for those of the RuntimeException and Error lineage — are checked exceptions. The compiler requires these exceptions to be caught or declared when it’s possible for them to be thrown.

When you define your own exceptions, you should either subclass RuntimeException (to make it an unchecked exception) or Exception (to make it checked). Programmers generally don’t subclass Error or Throwable, because these are reserved by Java itself.

reading exercises

Get to the point

Suppose we’re building a robot and we want to specify the function

public static List<Point> findPath(Point initial, Point goal)

which is responsible for path-finding: determining a sequence of Points that the robot should move through to navigate from initial to goal, past any obstacles that might be in the way.

In the postcondition, we say that findPath will search for paths only up to a bounded length (set elsewhere), and that it will throw an exception if it fails to find one.

Which of these would we choose?

throws a checked NoPathException(missing answer)

throws an unchecked NoPathException(missing answer)

throws a checked PathNotFoundException(missing answer)

throws an unchecked PathNotFoundException(missing answer)

(missing explanation)

check explain

Don’t point that thing at me

When we define our exception for findPath, which will we choose as our superclass?

Throwable(missing answer)

Exception(missing answer)

Error(missing answer)

RuntimeException(missing answer)

(missing explanation)

check explain

The rule we have given — use checked exceptions for special results (i.e., anticipated situations), and unchecked exceptions to signal bugs (unexpected failures) — makes sense, but it isn’t the end of the story. The snag is that exceptions in Java aren’t as lightweight as they might be.

Aside from the performance penalty, exceptions in Java incur another (more serious) cost: they’re a pain to use, in both method design and method use. If you design a method to have its own (new) exception, you have to create a new class for the exception. If you call a method that can throw a checked exception, you have to wrap it in a try-catch statement (even if you know the exception will never be thrown). This latter stipulation creates a dilemma. Suppose, for example, you’re designing a queue abstraction. Should popping the queue throw a checked exception when the queue is empty? Suppose you want to support a style of programming in the client in which the queue is popped (in a loop say) until the exception is thrown. So you choose a checked exception. Now some client wants to use the method in a context in which, immediately prior to popping, the client tests whether the queue is empty and only pops if it isn’t. Maddeningly, that client will still need to wrap the call in a try-catch statement.

This suggests a more refined rule:

• You should use an unchecked exception only to signal an unexpected failure (i.e. a bug), or if you expect that clients will usually write code that ensures the exception will not happen, because there is a convenient and inexpensive way to avoid the exception;
• Otherwise you should use a checked exception.

Here are some examples of applying this rule to hypothetical methods:

• Queue.pop() throws an unchecked Empty­Queue­Exception when the queue is empty, because it’s reasonable to expect the caller to avoid this with a call like Queue.size() or Queue.isEmpty().
• Url.getWebPage() throws a checked IOException when it can’t retrieve the web page, because it’s not easy for the caller to prevent this.
• int integerSquareRoot(int x) throws a checked Not­Perfect­Square­Exception when x has no integral square root, because testing whether x is a perfect square is just as hard as finding the actual square root, so it’s not reasonable to expect the caller to prevent it.

The cost of using exceptions in Java is one reason that many Java API’s use the null reference as a special value. It’s not a terrible thing to do, so long as it’s done judiciously, and carefully specified.

How to declare exceptions in a specification

/**
 * @param lst list of strings to convert to lower case
 * @return new list lst' where lst'[i] is lst[i] converted to lowercase
 */
static List<String> toLowerCase(List<String> lst)

/**
 * Compute the integer square root.
 * @param x value to take square root of
 * @returns square root of x
 * @throws NotPerfectSquareException if x is not a perfect square
 */
int integerSquareRoot(int x) throws NotPerfectSquareException;

/**
 * Pops a value from this queue.
 * @return next value in the queue, and removes the value from the queue
 * @throws EmptyQueueException if this queue is empty
 */
int pop();

static void analyzeEverything() {
    analyzeThingsInOrder();
}

static void analyzeThingsInOrder() {
    try {
        for (Thing t : ALL_THE_THINGS) {
            analyzeOneThing(t);
        }
    } catch (AnalysisException ae) {
        return;
    }
}

static void analyzeOneThing(Thing t) throws AnalysisException {
    // ...
    // ... maybe go off the end of an array
    // ...
}

Before we wrap up, check your understanding with one last example:

// Requires: tiles has length 7 & contains only uppercase letters.
//           crossings contains only uppercase letters, without duplicates.
// Effects: Returns a list of words where each word can be made by taking
//          letters from tiles and at most 1 letter from crossings.
public static List<String> scrabble(String tiles, String crossings) {
    if (tiles.length() != 7) { throw new RuntimeException(); }
    return new ArrayList<>();
}

A specification acts as a crucial firewall between the implementer of a procedure and its client. It makes separate development possible: the client is free to write code that uses the procedure without seeing its source code, and the implementer is free to write the code that implements the procedure without knowing how it will be used.

Let’s review how specifications help with the main goals of this course:

• Safe from bugs. A good specification clearly documents the mutual assumptions that a client and implementer are relying on. Bugs often come from disagreements at the interfaces, and the presence of a specification reduces that. Using machine-checked language features in your spec, like static typing and exceptions rather than just a human-readable comment, can reduce bugs still more.

• Easy to understand. A short, simple spec is easier to understand than the implementation itself, and saves other people from having to read the code.

• Ready for change. Specs establish contracts between different parts of your code, allowing those parts to change independently as long as they continue to satisfy the requirements of the contract.