Extensible Dynamic Data Distribution System

6.853 Research Proposal for the
First MIT Student Symposium on Computer Systems (SOCS)

Erik Nygren (nygren@mit.edu)
September 18, 1996


Different classes of data require different distribution policies. Some data is idempotent, some can be lost without consequence, and some must be delivered exactly once. In addition, some data may have "inertia". For example, the position of a moving aircraft also has an associated velocity. Rather than constantly transmitting the position, the position and velocity could be sent together with updates only when necessary. Other data distribution policies might only forward on data after it has changed by a certain amount.

Rather than building a data distribution system that encompasses all possible data distribution policies, this project will attempt to develop a run-time extensible virtual network. Network consumer, producer, and router nodes will be implemented to a minimum level of functionality in the Java programming language. At run-time, data requiring additional or complex distribution policies will be distributed along with Java code for implementing those policies.

By making the virtual data distribution network extensible, classes of data which require new data distribution policies can be added without installing new versions of the network software on all node of the network. By distributing code for caching schemes out to consumer and router nodes, the amount of data being transmitted could be significantly decreased. The network traffic could be further decreased if consumer and router nodes could be sent code for handling data objects with "inertia".

One of the objectives is to provide a system for a large number of heterogeneous data producers to feed dynamic data to a large number of data consumers. For example, these consumers could be Java applets running inside of web browsers. Applications for this include real-time stock market data feeds, data feeds from sensors into visualizers, data feeds into network monitoring and visualization tools, and communications systems for remote collaboration and shared virtual environments.


Erik Nygren (nygren@mit.edu)