6.031
6.031 — Software Construction
Fall 2018

Reading 24: Callbacks

Software in 6.031

Safe from bugsEasy to understandReady for change
Correct today and correct in the unknown future. Communicating clearly with future programmers, including future you. Designed to accommodate change without rewriting.

Objectives

In this reading we talk about callbacks, in which an implementer calls a function provided by the client.

We discuss this idea in two contexts, graphical user interfaces and web servers, in which the callbacks are used to respond to incoming input events.

But callbacks are an example of a bigger idea, first-class functions, or treating functions just like data: passing them as parameters, returning them as results, and storing them in variables and data structures. We’ll see more uses for first-class functions over the next few classes.

Input Handling in a Graphical User Interface

Input is handled differently in graphical user interfaces (GUIs) than we’ve been handling it in console user interfaces and servers. In those other systems, we’ve seen a single input loop that reads commands typed by the user or messages sent by the client, parses them, and decides how to direct them to different modules of the program.

If a GUI email client were written that way, it might look like this (in pseudocode):

while (true) {
    read mouse click
    if (clicked on Check Mail button) doRefreshInbox();
    else if (clicked on Compose button) doStartNewEmailMessage();
    else if (clicked on an email in the inbox) doOpenEmail(...);
    ...
}

But in a GUI, we don’t directly write this kind of method, because it’s not modular. GUIs are put together from a variety of library components – buttons, scrollbars, textboxes, menus – that need to be self-contained and handle their own input. For example, here is how you create a button:

JButton playButton = new JButton("Play");

which looks on screen something like this:

This button handles its own input from the mouse and keyboard. Deep inside the graphical user interface library is an input loop that is reading from the mouse and keyboard, and passing those input events to the appropriate components of the GUI.

In order to make your program do something when the button is clicked, you attach a listener to it:

playButton.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent event) {
        playSound();
    } 
});

This code creates an instance of an anonymous class implementing the ActionListener interface, which has exactly one method, actionPerformed. When this ActionListener instance is given to the button with addActionListener(), the button promises to call its actionPerformed method every time the user presses the button.

GUI event handling is an instance of the Listener pattern, also known as Publish-Subscribe. In the Listener pattern:

  • An event source generates (or publishes) a stream of discrete events, which correspond to state transitions in the source.
  • One or more listeners register interest (subscribe) to the stream of events, providing a function to be called when a new event occurs.

In this example:

  • the JButton is the event source;
  • its events are button presses;
  • the listener is the anonymous ActionListener instance
  • the function called when the event happens is actionPerformed

An event often includes additional information which might be bundled into an event object (like the ActionEvent here) or passed as parameters to the listener function.

When an event occurs, the event source distributes it to all subscribed listeners, by calling their listener methods.

So the control flow through a graphical user interface proceeds like this:

  • A top-level event loop reads input from mouse and keyboard. In Java and most graphical user interface toolkits, this loop is actually hidden from you. It’s buried inside the toolkit, often running on a separate thread created by the toolkit, and listeners appear to be called magically.
  • Each listener does its thing (which might involve e.g. modifying objects in the view tree), and then returns immediately to the event loop.

The last part – listeners return to the event loop as fast as possible – is very important, because it preserves the responsiveness of the user interface.

The Listener pattern isn’t just used for button presses. Every GUI object generates events, often as a result of some combination of low-level input events. For example:

  • JButton sends an action event when it is pressed (whether by mouse or keyboard)
  • JList sends a selection event when the selected element changes (whether by mouse or by keyboard)
  • JTextField sends change events when the text inside it changes for any reason

reading exercises

Listeners

Put the following items in order according to when they would happen during the setup and execution of a Java graphical user interface.

launchButton = new JButton("Launch the Missiles");

launchButton.addActionListener(launchMissiles);

launchMissilesactionPerformed() method is called

Mouse click on the launch button is received by the Java input event loop

(missing explanation)

Callbacks

The actionPerformed listener function we saw in the previous section is an example of a general design pattern, a callback. A callback is a function that a client provides to a module for the module to call. This is in contrast to normal control flow, in which the client is doing all the calling: calling down into functions that the module provides. With a callback, the client is providing a piece of code for the implementer to call.

Here’s one analogy for thinking about this idea. Normal function calling is like picking up the phone and calling a service, like calling your bank to find out the balance of your account. You give the information that the bank operator needs to look up your account, they read back the account balance to you over the phone, and you hang up. You are the client, and the bank is the module that you’re calling into.

Sometimes the bank is slow to give an answer. You’re put on hold, and you wait until they figure out the answer for you. This is like a function call that blocks until it is ready to return, which we saw when we talked about sockets and message passing.

But sometimes the task may take so long that the bank doesn’t want to put you on hold. Then the bank will ask you for a callback phone number, and they will promise to call you back with the answer. This is analogous to providing a callback function.

The kind of callback used in the Listener pattern is not an answer to a one-time request like your account balance. It’s more like a regular service that the bank is promising to provide, using your callback number as needed to reach you. A better analogy for the Listener pattern is account fraud protection, where the bank calls you on the phone whenever a suspicious transaction occurs on your account.

Route Handling in a Web Server

Let’s look at another example of a system that uses callbacks for input handling: a web server. A web server typically divides up the website it serves into sections, called routes. For example, the server for web.mit.edu might have routes for:

and so forth.

The control flow of a web server is an input loop waiting for incoming connections from web browsers. When a new connection arrives, the server reads and parses the request (using the HTTP wire protocol). The request is then routed to the handler registered for the route matching the request. Typically only a prefix of the request has to match the route, so the request http://web.mit.edu/community/topic/arts.html will be routed to the handler for /community unless there is a more specific (longer prefix) route registered.

The handler is then responsible for constructing the response to the request.

Oracle’s Java implementation includes a web server, HttpServer. You can create a new HttpServer like this:

int port = 8081;
HttpServer server = HttpServer.create(new InetSocketAddress(port), 0);

and then add routes to it using createContext:

server.createContext("/play/", new HttpHandler() {
    public void handle(HttpExchange exchange) throws IOException {
        handlePlay(exchange);
    }
});

Here, we are making an anonymous instance that implements the HttpHandler interface, which is an interface with one method, handle(). This handle() is the callback function. The server will call it anytime an incoming request starts with the /play/ prefix. The argument to the callback is an HttpExchange object, which provides observer methods with information about the request, and mutator methods for generating a response that goes back to the web browser.

As an example of examining the request, if the incoming request was http://localhost:8081/play/cello.wav, then:

String path = exchange.getRequestURI().getPath();

returns "/play/cello.wav", which the handler can then parse to extract the filename "cello.wav" that it should play:

// remove /play/ from the start of the path to get the filename
String soundFilename = path.substring(exchange.getHttpContext().getPath().length());

As an example of generating a response, the handler should first state the type of response it is returning (HTML, plain text, something else):

exchange.getResponseHeaders().add("Content-Type", "text/html; charset=utf-8");
exchange.sendResponseHeaders(200, 0);

And then write the response using the exchange object’s output stream:

PrintWriter out =
    new PrintWriter(new OutputStreamWriter(exchange.getResponseBody(),
                                           StandardCharsets.UTF_8),
                    true /* autoflush */);
out.println("playing " + soundFilename);
exchange.close();

Once again, this HttpHandler is an example of a callback: a piece of code that we as clients are handing to the module HttpServer, for the module to call whenever an event occurs, in this case the arrival of a request matching the route.

Clients

Since the Specifications reading, we’ve used the term client to refer to code that uses a class. In Sockets & Networking we started using client to mean one party in a network client/server communication. With the HTTP server, we need to keep track of both specific meanings of client:

  • A client of the web server is a web browser, sending it requests over the network. For example, a client of the web server sends a HTTP request to GET /play/cello.wav

  • A client of the HttpServer class is code we’ve written against the API that it provides. For example, a client of HttpServer calls createContext and passes in a HttpHandler.

First-class Functions

Using callbacks requires a programming language in which functions are first-class, which means they can be treated like any other value in the language: passed as parameters, returned as return values, and stored in variables and data structures.

Programming languages are full of things that are not first-class. For example, access control is not first-class – you can’t pass public or private as a parameter into a function, or store it in a data structure. The notion of public and private is an aspect of your program that Java offers no way to refer to or manipulate at runtime. Similarly, a while loop or if statement is not first-class. You can’t refer to that piece of code all by itself, or manipulate it at runtime.

In old programming languages, only data was first-class: built-in types (like numbers) and user-defined types. But in modern programming languages, like Python and JavaScript, both data and functions are first-class. First-class functions are a very powerful programming idea. The first practical programming language that used them was Lisp, invented by John McCarthy at MIT. But the idea of programming with functions as first-class values actually predates computers, tracing back to Alonzo Church’s lambda calculus. The lambda calculus used the Greek letter λ to define new functions; this term stuck, and you’ll see it as a keyword not only in Lisp and its descendants, but also in Python.

Guido van Rossum, the creator of Python, wrote a blog post about the design principle that led not only to first-class functions in Python, but first-class methods as well: First-class Everything.

In Java, the only first-class values are primitive values (ints, booleans, characters, etc.) and object references. But objects can carry functions with them, in the form of methods. So it turns out that the way to implement a first-class function, in an object-oriented programming language like Java that doesn’t support first-class functions directly, is to use an object with a method representing the function.

We’ve actually seen this before several times already:

  • The Runnable object that you pass to a Thread constructor is a first-class function, void run().
  • The Comparator<T> object that you pass to a sorted collection (e.g. SortedSet) is a first-class function, int compare(T o1, T o2).
  • The ActionListener object that we passed to the JButton above is a first-class function, void actionPerformed(ActionEvent e).
  • The HttpHandler object that we passed to the HttpServer above is a first-class function, void handle(HttpExchange exchange).

This design pattern is called a functional object, an object whose purpose is to represent a function. The spec for a functional object in Java is given by an interface, called a Single Abstract Method (SAM) interface because it contains just one method.

Lambda Expressions

Java’s lambda expression syntax provides a succinct way to create instances of functional objects. For example, instead of writing:

new Thread(new Runnable() {
    public void run() {
        System.out.println("Hello!");
    }
}).start();

we can use a lambda expression:

new Thread(() -> {
    System.out.println("Hello!");
}).start();

On the Java Tutorials page for Lambda Expressions, read Syntax of Lambda Expressions.

There’s no magic here: Java still doesn’t have first-class functions. So you can only use a lambda when the Java compiler can verify two things:

  1. It must be able to determine the type of the functional object the lambda will create. In this example, the compiler sees that the Thread constructor takes a Runnable, so it will infer that the type must be Runnable.
  2. This inferred type must be a functional interface: an interface with only one (abstract) method. In this example, Runnable indeed only has a single method — void run() — so the compiler knows the code in the body of the lambda belongs in the body of a run method of a new Runnable object.

reading exercises

Comparator<‌Dog>

In Java, suppose we have:

public interface Dog {
    public String name();
    public Breed breed();
    public int loudnessOfBark();
}

We have several Dog objects, and we’d like to keep a collection of them, sorted by how loud they bark.

Java provides an interface SortedSet for sorted sets of objects. TreeSet implements SortedSet, so we’ll use that.

The TreeSet constructor takes as an argument a Comparator that tells it how to compare two objects in the set; in this case, two Dogs.

Comparator is a functional interface: it has a single unimplemented method: int compare(...).

So our code will look like:

SortedSet<Dog> dogsQuietToLoud = new TreeSet<>(... /* a Comparator<Dog> that sorts by loudness */);
dogsQuietToLoud.add(...);
dogsQuietToLoud.add(...);
dogsQuietToLoud.add(...);
// ...

An instance of Comparator is an example of:

(missing explanation)

Barking up the wrong TreeSet

Which of these would create a TreeSet to sort our dogs from quietest bark to loudest?

Read the documentation for Comparator.compare(...) to understand what it needs to do.

new TreeSet<>(new Comparator<Dog>() {
  public int compare(Dog dog1, Dog dog2) {
    return dog2.loudnessOfBark() - dog1.loudnessOfBark();
  }
});

(missing explanation)

new TreeSet<>(new Comparator<Dog>() {
  public Dog compare(Dog dog1, Dog dog2) {
    return dog1.loudnessOfBark() > dog2.loudnessOfBark() ? dog1 : dog2;
  }
});

(missing explanation)

new TreeSet<>((dog1, dog2) -> {
  return dog1.loudnessOfBark() - dog2.loudnessOfBark();
});

(missing explanation)

public class DogBarkComparator implements Comparator<Dog> {
  public int compare(Dog dog1, Dog dog2) {
    return dog1.loudnessOfBark() - dog2.loudnessOfBark();
  }
}
// ...
new TreeSet<>(new DogBarkComparator());

(missing explanation)

Listener lambda

Consider this anonymous class from earlier in the reading:

playButton.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent event) {
        playSound();
    } 
});

Fill in the blanks to transform the anonymous class into a lambda expression:

playButton.addActionListener(▶▶A◀◀ -> ▶▶B◀◀);

(missing explanation)

Concurrency in Event Processing Systems

Using callbacks in a system inevitably forces the programmer to think about concurrency, because control flow is no longer under your control. Your callback might be called at any time, and in some systems, it might be called from a different thread than the client originally came from.

Java’s graphical user interface library automatically creates a single thread as soon as a GUI object is created. This event-handling thread is different from the program’s main thread. It runs the event loop that is reading from the mouse and keyboard, and calls listener callbacks.

Likewise, the HttpServer class creates a new thread to listen for incoming connections and parse their HTTP requests, and then call the route handler callbacks.

So both of these systems are concurrent, even though the additional threads are not visible to the user.

Implementing an Event Source

All our examples of callbacks so far have been from the client’s point of view. The client creates a callback function, passes it to the implementer, and when the desired event occurs, the implementer calls it.

What about the implementer’s side? How does an implementer keep track of callback functions, and make the calls? To illustrate this, we’ll implement a simple counter that calls its listeners whenever it increments.

Counter spec

Counter has several operations, plus the listener interface:

public class Counter {
    /** Make a counter initially set to zero. */
    public Counter() { ... }

    /** @return the value of this counter. */
    public synchronized BigInteger number() { ... }

    /** Increment this counter. */
    public synchronized void increment() { ... }

    /** Modifies this counter by adding a listener.
      * @param listener called by this counter when it changes. */
    public synchronized void addNumberListener(NumberListener listener) { ... }

    /** Modifies this counter by removing a listener.
      * @param listener will no longer be called by this counter. */
    public synchronized void removeNumberListener(NumberListener listener) { ... }

    /** A listener for this counter. */
    public interface NumberListener {
        /** Called when the counter changes.
          * @param number the new number */
        public void numberReached(BigInteger number); 
    }

}

So, from a client’s point of view, we can make a counter and attach listeners to it:

Counter counter = new Counter();
counter.addNumberListener(new NumberListener() {
    public void numberReached(BigInteger number) {
        System.out.println(number);
    }
});

or, using a lambda expression for the listener:

Counter counter = new Counter();
counter.addNumberListener(number -> System.out.println(number));

Whenever counter.increment() is called, Counter will call this listener and print the new number.

To make the counting happen, the client will spin up a new thread:

new Thread(() -> {
    while (true) {
        counter.increment();
    }
}).start();

Counter implementation

Let’s turn to the implementer’s view. Counter needs to track both the number it is counting and the set of listeners who want to be called back. So we’ll use this straightforward rep, and design for thread safety from the outset:

private BigInteger number = BigInteger.ZERO;
private Set<NumberListener> listeners = new HashSet<>();

// Abstraction function
//    AF(number, listeners) = a counter currently at 'number'
//        that sends events to the 'listeners' whenever it changes
// Rep invariant
//    true
// Thread safety argument
//    uses the monitor pattern -- the rep is guarded by this object's lock,
//    acquired on entering every public method

The counting itself is handled by increment():

public synchronized void increment() {
    number = number.add(BigInteger.ONE);
    callListeners();        
}

The essence of implementing the Listener pattern is the cooperation between the public mutators addNumberListener() and removeNumberListener(), which clients use to register their callback functions, and the private method callListeners() that the implementation uses to call those callback functions. The operations might then be implemented like this:

public synchronized void addNumberListener(NumberListener listener) {
    listeners.add(listener);
}

public synchronized void removeNumberListener(NumberListener listener) {
    listeners.remove(listener);
}

private void callListeners() {
    for (NumberListener listener : listeners) {
        listener.numberReached(number);
    }
}

In this code, addNumberListener()/removeNumberListener() maintain the set of listeners. Whenever the counter increments, it calls callListeners() to iterate through the set, calling each listener’s numberReached() method.

Our thread safety argument holds because all public methods are marked synchronized, guarded by the lock on the Counter object.

Unfortunately this straightforward implementation has a subtle bug. The following reading exercises uncover the bug and fix it.

reading exercises

A one-time listener

Suppose a client only cares to be called back once, and then removes itself to stop further calls:

counter.addNumberListener(new NumberListener() {
    public void numberReached(BigInteger number) {
        System.out.println(number);
        counter.removeNumberListener(this);
    }
});

Assume that addNumberListener(), removeNumberListener(), and callListeners() are implemented as shown above, and that the counter is being steadily incremented in a background thread. Which of the following is most likely to happen when this listener is called?

(missing explanation)

Fixing reentrancy

A solution to the reentrancy bug is our old friend defensive copying. Fill in the blank below, with as little code as possible, so that the iteration happens over a different Set object than addNumberListener() and removeNumberListener() can mutate.

private void callListeners() {
    for (NumberListener listener : ▶▶A◀◀) {
        listener.numberReached(number);
    }
}

(missing explanation)

You can find the full code for Counter on GitHub.

Summary

Callbacks are our first example of first-class functions, where we treat functions just as we treat data: passing them to and returning them from other functions, and storing them to call later.

Callbacks make code more ready for change by allowing clients to provide their own code to run when an event occurs, so that the behavior doesn’t have to be hard-coded into the implementation beforehand.

Writing a single giant input loop to handle every possible event in a large system is neither safe from bugs nor easy to understand: that single piece of code becomes an all-knowing all-controlling behemoth. Callbacks allow each module in the system to be responsible for their own events.

In future readings, we’ll see further benefits to all three of these properties of good software from the idea of first-class functions.