6.829 Fall 2020 Problem Set 2 and Leaderboard

You have the option of working in teams of up to 2 people. You may collaborate and discuss ideas with others in the class, but your solutions and presentation must be your team's own. Do not look at other teams' solutions or copy them from anywhere. You must list your collaborators in your submission.

Commit your solutions before Monday, October 19th at 11:59 pm EST. You must also submit the submission form as soon as your repo and group are set up.


Introduction

In this Problem Set, you will compete to build a congestion-control algorithm to achieve hight throughput and low delay on an emulated cellular network. You will start with a simple UDP sender (client.py) and receiver (receiver.py), built on python networking classes that do most of the grunt work of opening sockets and reading and writing data. By modifying the congestion control behavior in client.py, you can control how the sender decides when to send packets. Sending packets faster will generally result in higher throughput but at the risk of building up a standing queue at the bottlneck link, increasing delay. We will therefore be concerned with two performance metrics: the average throughput (in Mbps) and the 95th percentile queuing delay (in ms).

The problem set starter code is available here


Setup

We recommend using Ubuntu 14.04 or later. In case you don't have Ubuntu installed on your computer, we recommend installing a virtual machine (e.g., running Ubuntu using VMware fusion from IS&T's website).
The first step is to install Mahimahi, a network emulation tool. Detailed instructions can be found here:
http://mahimahi.mit.edu/#getting.
Ensure that all the dependencies listed on this website are installed correctly, as they will not neccessarily be installed automatically.

The following files are of interest:

Before you begin please run "chmod +x run.sh". If you are using a different shell to the one specified in the first line, please alter this also.

The template for the command that runs the experiment is:
./run.sh $cellular_link_trace $output_log $duration_of_the_experiment $visualization_image
as an example, run "./run.sh traces/Verizon-LTE-short.up output.log 140 output.png".
The script also reports the average throughput and the 95th percentile per-packet queuing delay (performance metrics), used for the power score. Do not edit the argument format for the client.py and server.py files, or alter in your commits the way run.sh operates


Submission Instructions

After you optionally choose a teammate, come up with a team name, and enter that name into a file called `NAME.txt`. Enter the MIT ldaps of everyone in your team into `PARTNER.txt`, one name per line. You must submit to the leaderboard at least once before the deadline. Only one repository is required per team. In the `your_code` directory, commit a file `README.md` that includes a 1-2 paragraph writeup of your approach, why you chose it, and what you tried. This is only required for your final submission, not for leaderboard submissions.

Submit a link via the form below to your team's *private* github.mit.edu repository (everyone must do this so we can verify teammate reciprocity) as soon as possible. Your repository must be a copy of the starter repository. Ensure that the usernames "prateesh" and "jsalamy" as well as the username of your teammate are added as collaborators to your repository. We will only consider commits made before the submission deadline. If you would like to use an extension day, include the string "EXTENSION-DAY" in *all* commit messages for commits made after the deadline AND contact the staff individually once you are done committing. You must submit the form (which tells us where your repository is) as soon as you've set your repository up, even if you are using extension days. Extension days cannot be used for contest submissions.


Competition

This problem set involves a contest. This is the leaderboard, updated twice daily, so you can see how well your fellow congestion control developers are doing as well as test using our link conditions. To submit to the leaderboard, all you need to do is push your commits to github, ensure you have both TAs added as collaborators, and submit the form. We expect every team to continuously submit to the leaderboard as they improve their algorithms!

Every day, the contest server will choose a different cellular link trace and base RTT each day to evaluate every submission. For each configuration, we will evaluate all the submissions on a power score, defined as the ratio of average throughput to 95th percentile per-packet queueing delay. Your goal is to achieve the highest power score. Submissions will use the team name you choose, not your ldap (see Submission Instructions).

Final Powerscore Leaderboard:

  1. Miljkovic: 592.97 (19, 11.27)
  2. NetRunners: 462.24 (32, 14.79)
  3. splash: 358.69 (45, 16.14)
  4. FullMarks: 322.56 (33, 10.64)
  5. Tensorbro: 321.49 (34, 10.93)
  6. 3dots: 315.64 (26, 8.21)
  7. Panther: 301.23 (11, 3.31)
  8. Robusta: 286.03 (44, 12.58)
  9. covfefe: 258.58 (57, 14.74)
  10. pineapple2: 245.61 (15, 3.68)
  11. hiper: 240.29 (13, 3.12)
  12. 05greedy: 237.63 (53, 12.59)
  13. dianny: 224.36 (16, 3.59)
  14. SWTS: 206.12 (50, 10.31)
  15. coffeehour: 186.46 (10, 1.86)
  16. random_guy: 177.22 (106, 18.79)
  17. jonabox: 155.84 (133, 20.73)
  18. KatWicks: 103.61 (55, 5.70)
  19. WizardsofNetworkingPlace: 14.84 (22, 0.33)
*PScore (95% delay, Ave Throughput). High Link Capacity CC Algorithm Performance

Performance Graph

Graph of Relative Performance