Week 0: ML Fundamentals

Week 0 is a bit different from the rest of the program: it's self-paced and optional, completed on your own. No other weeks will require time outside the 2-hour weekly session or standard assigned readings for the 1-hour sessions. The purpose is to ensure that all participants share a basic foundation in key machine learning concepts that we'll build on throughout the reading group.

Most participants will be able to skip this week. Since the program has no formal prerequisites, we're providing resources so we don't need to spend later sessions covering ML fundamentals. The goal of the reading group is conceptual understanding rather than hands-on expertise, but some technical context can be helpful for developing that understanding.

Learning Objectives

By the end of Week 0, you should be able to:

• Explain, at a high level, how neural networks are trained
• Describe the basic architecture of a transformer

If you're already comfortable with these topics, feel free to skip all resources below. If you're rusty or new to them, start with the main resource. A solid understanding of the 3Blue1Brown playlist is fully sufficient to meet the learning objectives; the additional resources are optional for anyone who wants deeper coverage.

• Neural Networks Chapters 1-6 (3Blue1Brown, 2017-2024) (~2 hours)
  • Feel free to skip videos of topics you are already familiar with

• What is self-supervised learning? (CodeBasics, 2021) (~5 minutes)
• Transformers from scratch (Rohrer, 2021) (~60 minutes)
• The spelled-out intro to neural networks and backpropagation: building micrograd (Karpathy, 2022) (~150 minutes)
• Machine Learning for Humans, Part 2.1: Supervised Learning (Maini and Sabri, 2017) (~15 minutes)