6.S088 Modern Machine Learning: Simple Methods that WorkWelcome to 6.S088! IntroductionOver the past decade, interest in machine learning research has spiked drastically, with advancements in deep learning being a significant driving force. Indeed, deep learning has transformed many areas in computer science including computer vision, natural language processing, and reinforcement learning. Unfortunately, given the rapid pace of progress in deep learning, a newcomer looking for a simple set of guiding principles for building machine learning applications can be easily overwhelmed by the nuances of training deep networks. Thus, motivated by recent developments in machine learning, we present a simple class of machine learning methods that are easy to implement and which achieve competitive performance in practice. In particular, our methods rely on the recently established equivalence between kernel regression and infinite width neural networks given by the neural tangent kernel (NTK). In addition to being a theoretical tool for understanding neural networks, we demonstrate that the NTK is a simple method for achieving competitive results in a variety of machine learning applications including regression, classification, and matrix/image completion. We provide problem sets containing both theoretical and coding exercises with the aim of (1) providing newcomers to machine learning a simple toolkit for building effective machine learning models in practice and (2) preparing interested students for research in the area. OrganizationUnits: 105 Prereqs: Knowledge of linear algebra (level of 18.06 or 18.700), analysis (level of 18.100), and probability (level of 6.041 or 18.600) is generally assumed. Familiarity with Python (in particular, NumPy) is also assumed. Knowledge of Fourier analysis (18.103), functional analysis (18.102), random matrix theory (18.338), and complex analysis (18.112) is suggested for students who want to pursue research in this area. Schedule: Monday – Friday, January 18 – January 28, 12:30pm, room 32141 Instructor: Adityanarayanan Radhakrishnan, aradha@mit.edu Faculty Advisor: Prof Caroline Uhler TAs: Max Luyten, George Stefanakis, Cathy Cai Schedule
