Natural Language Processing (6.806-864)
Generating and understanding human language remains one of the most exciting (and challenging) frontiers in artificial intelligence research. In this class, we'll survey contemporary prediction problems involving human language data, and introduce probabilistic modeling and representation learning tools that can be used to tackle them.
Course Staff
Instructors
Jacob Andreas (jda@mit.edu)
Yoon Kim (yoonkim@mit.edu)
TAs
Dylan Doblar
Deep Gupta
Pranav Krishna
Alex Liu
Joe O'Connor
Nitya Parthasarathy
Julia Wagner
Admin
Homework, announcements, etc. will be distributed on Canvas.
Class will meet on Tuesdays and Thursdays from 11 to 12:30 PM ET in 4-149 and simultaneously recorded on Zoom (see Canvas for link). Videos will be available on canvas after class.
Grading
You are encouraged to work together on homework assignments, but all submitted writeups and code should be done on your own. Students in 6.806 will complete extra communication-focused assignments, while students in 6.864 will have extra problems on homeworks 1–3.
50% homework, 50% project.
Grade scale: A [90, 100]; B [80, 90); C [70, 80); D [60, 70); F [0, 60).
Homework assignments lose 10% for every late day. Final projects will not be accepted late.
This is (still) not a normal semester! We want everyone to learn from this class, and we can almost certainly find a way to accommodate any issues that arise. If you're struggling, please reach out to Jacob or Yoon as soon as possible.
Syllabus
| Th 9 Sep | Introduction | [sp21 slides] |
| Tu 14 Sep | Classification 1: linear models and deep networks | [sp21 slides] |
| Th 16 Sep | Classification 2: recurrent networks | [sp20 slides] |
| Tu 21 Sep | Classification 3: attention | [sp21 slides] |
| Th 23 Sep | Classification 4: contextualized word representations | [sp20 slides] |
| Fr 24 Sep | HW1 (classification) due | |
| Tu 28 Sep | Classification 5: more modeling tools | [sp21 slides] |
| Th 30 Sep | Representation learning 1: distributed representations | [sp21 slides] |
| Tu 5 Oct | Representation learning 2: Deep word representations | [sp20 slides] |
| Th 7 Oct | Representation learning 3: contextualized word representations | [sp21 slides] |
| Fr 8 Oct | HW2 (representation learning) due | |
| Tu 12 Oct | Structured prediction 1: finite-state sequence models | [sp20 slides] |
| Th 14 Oct | Structured prediction 2: conditional random fields | [sp21 slides] |
| Fr 15 Oct | Project proposal due | |
| Tu 19 Oct | Structured prediction 3: trees | [sp21 slides] |
| Th 21 Oct | Latent variable models 1 | new! |
| Fr 22 Oct | HW3 (structured prediction) due | |
| Tu 26 Oct | Latent variable models 2 | new! |
| Th 28 Oct | How to write an NLP paper | new! |
| Tu 2 Nov | Speech 1 | [sp20 slides] |
| Th 4 Nov | Speech 2 | [sp20 slides] |
| Fr 5 Nov | HW4 (dataset design) due | |
| Tu 9 Nov | Interpreting NLP models | [sp21 slides] |
| Th 11 Nov | (no class) | |
| Tu 16 Nov | Guest lecture: Human language processing | |
| We 17 Nov | Project draft due | |
| Tu 18 Nov | Guest lecture: TBD | |
| Tu 23 Nov | Grounded language learning | [sp21 slides] |
| Th 25 Nov | (no class) | |
| Tu 30 Nov | Social and ethical considerations | [sp21 slides] |
| Th 2 Dec | The future of NLP research | new! |
| Tu 7 Dec | Project presentations | |
| Th 9 Dec | Project presentations Final project report due |