6.S088 Modern Machine Learning: Simple Methods that Work

Introduction

Over the past decade, interest in machine learning research has spiked drastically, with advancements in deep learning being a significant driving force. Indeed, deep learning has transformed many areas in computer science including computer vision, natural language processing, and reinforcement learning. Unfortunately, given the rapid pace of progress in deep learning, a newcomer looking for a simple set of guiding principles for building machine learning applications can be easily overwhelmed by the nuances of training deep networks. Thus, motivated by recent developments in machine learning, we present a simple class of machine learning methods that are easy to implement and which achieve competitive performance in practice. In particular, our methods rely on the recently established equivalence between kernel regression and infinite width neural networks given by the neural tangent kernel (NTK). In addition to being a theoretical tool for understanding neural networks, we demonstrate that the NTK is a simple method for achieving competitive results in a variety of machine learning applications including regression, classification, and matrix/image completion. We provide problem sets containing both theoretical and coding exercises with the aim of (1) providing newcomers to machine learning a simple toolkit for building effective machine learning models in practice and (2) preparing interested students for research in the area.

Organization

Schedule