6.02 - The History of the Internet Lecture #25 Katrina LaCurts, lacurts@mit.edu ====================================================================== Sources ====================================================================== The pre-ARPAnet historical information that I give comes from: - Tom Standage, "The Victorian Internet: The Remarkable Story of the Telegraph and the Nineteenth Century's On-line Pioneers" - Brian Bowers, "Cooke and Wheatstone, and Morse: a Comparative View" (http://www.ieeeghn.org/wiki/images/5/53/Bowers.pdf) - Donovan A. Shilling, "Rochester's Remarkable Past" The post-ARPAnet historical information comes from: - David D. Clark, "The Design Philosophy of the DARPA Internet Protocols" - Mark Handley, "Why the Internet Only Just Works" ====================================================================== Introduction ====================================================================== This lecture contains relatively little technical content compared to previous lectures. Instead, it's a history of the Internet. What you should take away from this lesson is an understanding of the design decisions that led to the Internet, and how that continues to affect us today, both as users of the Internet, and as scientists/engineers interested in communication. ====================================================================== Part I: Pre-ARPAnet ====================================================================== Here is a sentence: "It is anticipated that the whole of the populous parts of the United States will, within two or three years, be covered with a net-work like a spider's web." If you know a little bit about Internet history, you might think this sentence was about the ARPAnet -- which we'll come to in a bit -- but surprisingly, this sentence was written in 1848. Indeed, the history of the Internet begins much earlier than you might have expected. The technology that this quote is reacting to is the electrical telegraph. Commercially, there were two important versions of the electrical telegraph 1. Cooke and Wheatstone telegraph, 1837. This telegraph doesn't look like the "typical" telegraph. Instead, it used a number of needles (typically five) which could be moved to indicate which was meant to be transmitted (see the slides for a picture of this; it's not the same as doing, say, a five-bit ASCII encoding). An operator would move the needles by winding the device, which would move the relevant pair of needles in the appropriate directions. At the receiving end, the operator could look at the positions of the needles, and read from the grid which letter was sent. Cooke and Wheatstone eventually got down to a one-needle design, but the encoding required up to three movements of the needle for a single letter. 2. The Morse-Vail (or perhaps Morse-Henry-Vail) telegraph, 1844 (but in development as early as 1836). This electric telegraph had some advantages over Cooke and Wheatstone's: It used Morse code (dots and dashes), and printed its response out on paper (it didn't print, exactly, but made indentations on a piece of paper). This meant that an operator could receive a message and then read it off of the paper. Interestingly, since the device also made a clicking sound, the operators learned to hear Morse code as a language, and the paper tape was eventually made unnecessary. Particularly interesting because Morse was originally very interested in this difference between his machine and the Cooke-Wheatstone machine, claiming that his device conveyed a "permanent" sign while the Cook-Wheatstone device conveyed an "evanescent" one. The electrical telegraph is the first practical application of communicating with electricity, which means it was the first device that people could use to communicate globally in real-time. This was an astounding shift in communications. But it had some problems. During the late 1800s/early 1900s, the world experience a communications arms race, in which no nation could trust its messages to a foreign power. This had unfortunate consequences: In 1914, the British cut the German overseas cables within hours of the start of WWI. In response, Germany cut England's Baltic cables and the overland lines to the Middle East through Turkey. To the rescue came wireless communications. ---------------------------------------------------------------------- Wireless ---------------------------------------------------------------------- Wireless communications got their start much earlier than WWI. Maxwell developed the theory of the electromagnetic field in 1864, and Hertz validated this theory in 1886/88 by demonstrating the wave character of an electrical transmission in space. Wireless communications came a bit later, in 1893/94, when Tesla and Bose were developing technologies such as the spark gap transmitter and the coherer. The latter anticipated the existence of semiconductors 60 years ahead of their time. This "wireless telegraphy" was commercialized in the late 1800s/early 1900s when Marconi invented the radio. Also around this time, we start to see modulation in practice: amplitude modulation in 1900 by Fessenden, and frequency modulation in 1918 by Armstrong. ---------------------------------------------------------------------- Telephones ---------------------------------------------------------------------- In the meantime, those back in the wired world were still working. In particular, in the second half of the 19th century, many engineers tried to improve the capacity of telegraph systems. This desire brought about the invention of the telephone, by Alexander Graham Bell. The patents surrounding the telephone are highly disputed and full of historical drama. In 1889, the telephone switch came along, with Strowger's invention of the Strowger switch. Built in response to Strowger's frustrations with a particular operator who didn't complete calls to him, it was advertised with the somewhat misogynistic tag, the "girl-less, cuss-less, out-of-order-less, wait-less telephone". ---------------------------------------------------------------------- Information Theory ---------------------------------------------------------------------- At this point in the world, real-time global communications are extremely popular. You can see that advances in physics/engineering propelled us to this point, and provided the underlying principles for how these devices should work. In some sense, they got the physical layer figured out for us. But what about the data that we send over that physical layer? When do the principles that underly how we communicate over a channel show up? They show up in the late 1930s/1940s, right here at MIT. Claude Shannon wrote his Master's Thesis in 1937, "A Symbolic Analysis of Relay and Switching Circuits", which introduced the application of Boolean algebra to logic circuits (and vice versa). It's arguably the most important master's thesis of this century. Shannon studied other areas of math during his time, but made huge advances in information theory, particularly in the findings of channel capacity. The capacity of a channel tells us how much information we can possibly get through a particular channel (with, e.g., a particular model of noise). Both of these notions -- information and channel capacity -- were introduced by Shannon. With these definitions, scientists and engineers now had an upper-bound for the performance of their codes, and could start developing better ones. A large portion of information theory has been dedicated to developing codes that approach the channel capacity: - Hamming code (1950) - Convolutional code (1955) - Low-density parity check codes (1960) - Sequential decoding - Viterbi decoding (1967) - Turbo codes (1993) What else was happening in the world during this Golden Age of Information Theory? The launch of Sputnik and the Cold War. ====================================================================== Part II: The Internet, pre-1993 ====================================================================== The launch of Sputnik inspired the creation of ARPA (now DARPA), for developing research projects that went beyond immediate military requirements. Given the era, survivable communications systems were of interest. And you know this part of the story! It's how we ended up with Baran's distributed network idea. 1970s: The ARPA network -- ARPAnet -- started off small. It combined addressing, the process by which we assign numerical addresses to endpoints, and transport, the process by which we get data to a particular address. Endpoints also had human-readable names, which were stored in a "hosts.txt" file on every machine. It did not scale. Aside: Think about why we might give an endpoint a numerical address *and* a human-readable address. It will be good preparation for 6.033. During the 70s, the ARPAnet started to grow from individual small networks to small networks connected to one another via gateways. Sliding-window protocols as well as distance-vector routing were invented (and used) during this period. ---------------------------------------------------------------------- Flexibility ---------------------------------------------------------------------- 1978: Now comes a very important design decision in the Internet's history: the desire to make it flexible. Flexibility encouraged the design of layering, which we've talked about. We can swap out various physical-layer technologies, e.g., for others, and everything that runs "on top" of them will still work. As a result, we got TCP in 1983 (this early version of TCP did not include congestion control). As you know, TCP provides reliable transport. Other applications run "on top" of TCP, and so don't have to re-implement reliable transport. This separation -- reliability from addressing/transport -- was done to make the Internet more flexible. As we continue, think about whether you believe the Internet to be a flexible place. ---------------------------------------------------------------------- Growth ---------------------------------------------------------------------- As this was happening, ARPAnet continued to grow, which led us to new protocols: 1978-79: Link-state routing and EGP. You already know that LS routing scales better than DV. EGP is the start of the hierarchical routing that I talked about at the end of our routing lectures, where certain portions of the network are isolated. 1982: DNS. "hosts.txt" had all sorts of issues with handling frequent updates, etc. It was not a scalable naming mechanism. DNS is how we deal with naming today. Machines can query servers for the mapping between an IP and a hostname. It's distributed, and updates can happen quickly, and without end users noticing (for the most part). In addition to responding to growth, DNS actually enabled growth by allowing sites to administer their own domains. This style of administration is also nice from a policy standpoint. Another thing we think about in networking is who should be in charge of what things. Both from a systems standpoint -- centralized systems have a single point of failure -- and a human standpoint -- do we want a single person or organization in charge of everything? ---------------------------------------------------------------------- Problems ---------------------------------------------------------------------- You should see at this point that the growth of the Internet is starting to effect changes. It was also starting to cause problems. 1. Mid 80s, Congestion Collapse: We talked about this briefly last week. Congestion collapse happens when there are so many packets in the network that delay is too high to be useful. Congestion control was added to TCP to address this problem. An interesting question, since you all know a bit about layering, is why we added congestion to TCP. Why not give it its own layer? Even in the 80s, it was very hard to deploy a congestion control layer. TCP congestion control "was backwards compatible, incrementally deployable, and did an excellent job of addressing the immediate problems at hand." So it was good enough. 2. Early 90's, policy routing. The Internet was already using link-state routing for scale, but in the early 90s the Internet was beginning to be commercialized. Different groups (think: universities, large companies) wanted to decide what routes they advertised and who could use them (e.g., NSFNet, the NSF-funded Internet backbone at the time, wanted to prohibit commercial traffic on its links). This led to the development of BGP. BGP is a path-vector routing protocol like I told you earlier this semester, but it is also a policy routing protocol. That means that portions of the Internet can decide what routes to advertise depending on who they're dealing with. Typical scenario: ISP A and ISP are peers; they share a route between them without charging the other. Neither ISP wants to let ISP C know about that route, since ISP A (e.g.) will not make any money by letting C use it. 3. Addressing. As many of you already know, machines on the Internet have numerical addresses. These addresses are given out to organizations in chunks of consecutive numbers. Originally you could get these chunks, or classes, in three different sizes: 16m, 65k, and 256. 65k is "just right" -- most organizations have more than 256 machines, and few have 16m -- but we quickly ran out of these "class B" networks. Now we use the protocol CIDR to divide up larger classes of addresses. Fact of the day: MIT owns one of the 16m "class A" address spaces. The most interesting thing about CIDR was that it was possible to make this change on the Internet. Changing the way addressing is done requires a substantial change to the switches in the network (basically, it changes some details in how forwarding happens). The reason we were able to make such a change is because at the time, all switches were made by Cisco, and forwarding was performed in software. Today, it'd be difficult to make such a change. Lots of switches do forwarding in hardware, and companies other than Cisco make switches. Summary so far: Internet got big, fast. Most problems were solved "just in time". ====================================================================== Part III: The Internet, 1993-today ====================================================================== In 1993, the Internet was commercialized. Changes stopped. And it wasn't for a lack of ideas. Some proposed changes: - ECN, DiffServ. Both are meant to change congestion control. - Mobile IP. Allows devices to roam. - Multicast. More scalable broadcasting. However, most of the proposed changes were more enhancements, not big fixes (in the way that congestion control or policy routing were). Why does that matter? "New technologies essentially get deployed for reasons of fear or greed." Commercialization also inspired another technology: NATs. NATs translate addresses so that you can have private portions of a network that can still access the outside. - Basically, everything behind a NAT has the same outward-facing IP address. When that IP address receives a packet, the NAT takes care of figuring out who it's really meant for. - They break a million things. How did commercialization inspire NATs? ISPs charge more for more IPs, so using a NAT lets you operate with fewer. We're talking about NATs in particular because they caused another barrier to deployment. New protocols won't be adopted if they don't work end-to-end. OSes won't implement it if there's no need for it. NATs won't add support if the protocol is not in common OSes. The new protocol won't work end-to-end because there won't be NAT support. But wasn't the Internet meant to be flexible? What happened? ====================================================================== Part IV: Impact ====================================================================== The history of the Internet -- in particular, the fact that it was designed to be robust to failure (and *not* for other things, e.g., security), the fact that a lot of the earliest improvements were "just-in-time" fixes, and the fact that it was commercialized -- has bearings on a lot of the problems we experience today. 1. DDoS. Problem: lack of accountability First up: DDoS. This is when a host, or a set of hosts, send a lot of unwanted traffic to a particular host (or set of hosts). One example (my favorite example) is "DNS reflection". A bot launches a DNS request, and spoofs its source as the victim's. The DNS server returns a response that is much larger than the request (as it should be). Get a lot of bots to do this, the victim is overrun with requests. Why is this even allowed? The Internet was not designed with accountability in mind! Why is it hard to prevent? The DNS request/response are totally valid traffic, and filtering is expensive (examine each packet). Filtering a botnet is very difficult. (This problem has gotten worse with the advent of botnets.) 2. Security. Problem: Internet was not designed for security. So many things on the Internet are insecure. Traffic isn't encrypted by default, BGP and DNS have no security (you could lie about a route in BGP; there's no infrastructure to support authenticity). Also think about social networks, NSA, .. 3. Mobility. Problem: No one ever imagined this When I travel with my phone, my IP address changes. Every time my IP changes, I have to re-start my connections. In TCP, that means they will slow down for a bit (they'll enter slow start). Think about what happens if I'm switching rapidly between access points. ====================================================================== Part V: The Internet, 2014 - ? ====================================================================== So, is the Internet flexible, as it was designed to be? The Internet was designed with different principles than we'd (probably) like today. It is flexible in some ways, but not in others; it's difficult to make changes to the core protocols, in part due to the commercialization of the Internet. Thus, there is some truth to the idea that problems aren't fixed until they are so bad that there's no other choice. Despite this, we've seen many changes in the Internet in recent years: - P2P traffic - wireless and mobile traffic - video streaming - cloud computing/datacenter networks - security threats and defenses Almost everything -- telephony, video, etc., -- has moved to the Internet. We expect to see continued changes: - more massive growth from entertainment - software-defined networks - sensors, mobile robots, embedded devices - network security and privacy; censorship Despite the fact that the core protocols in the Internet are difficult to change, there are still things to do! We can also still study how to make applications perform better on the Internet given its constraints. Full disclosure: that is what I did in grad school, so I'm totally biased. ====================================================================== The Takeaways ====================================================================== Aside from the fact that as engineers, you should know the history of the Internet -- because those who don't know history are doomed to repeat it -- what else should you have gotten from this lecture? As I've said over and over, the history of the Internet informs an incredible amount of what we do today, and how we design protocols and applications for it. Applications don't always "just work" on the Internet. As an example, Skype (at least at one time) required peer-to-peer networking to be reliable. Companies that do video-streaming usually do so via a Content Distribution Network such as Akamai. These design decisions are also important when we consider a re-design -- or "clean slate" design -- of the Internet. What might we design for now that we didn't originally?