Week 8: Research & Careers in Safety
Empirical Research Workflow, Active Alignment Agendas, and Careers in Safety
Overview
Over the past several weeks, we surveyed both the technical landscape of AI safety and the institutional levers through which it might be governed. This closing session turns from understanding the field to working in it. We start with practical advice on the empirical research workflow — choosing problems, prioritizing experiments for information gain, and moving fast under uncertainty — alongside the broader mindset of explore, understand, distill. We then survey active alignment research agendas and discuss what next steps and careers in AI safety can look like.
Learning Objectives
By the end of Week 8, fellows should be able to:
- Describe the stages of an empirical research project — exploration, hypothesis-testing, distillation
- Apply practical heuristics for doing safety research effectively, including prioritizing for information gain and designing fast experimental loops
- Identify major active research agendas in AI alignment and articulate considerations for evaluating which problems are tractable, neglected, or high-impact
Core Readings
- Tips for Empirical Alignment Research (Perez, 2024)
- How I Think About My Research Process: Explore, Understand, Distill (Nanda, 2025)
- My Research Process: Key Mindsets - Truth-Seeking, Prioritisation, Moving Fast (Nanda, 2025)
- AI Safety Careers Guide for Undergraduates and Early-Career Students (MAIA, 2026)
Recommended Readings
- An Opinionated Guide to ML Research (Schulman, 2020)
- My Research Methodology (Christiano, 2021)
- Highly Opinionated Advice on How to Write ML Papers (Nanda, 2025)
- Bottom-Up Alignment Research (Perez, 2024)
- Research as a Stochastic Decision Process (Steinhardt, 2018)
- AI Safety Technical Research (80,000 Hours, 2024)