Phillip Isola

p h i l l i p i @ m i t . e d u
Google Scholar / GitHub / Social media

About me

I am an associate professor in EECS at MIT studying computer vision, machine learning, and AI.

Previously, I spent a year as a visiting research scientist at OpenAI, and before that I was a postdoctoral scholar with Alyosha Efros in the EECS department at UC Berkeley. I completed my Ph.D. in Brain & Cognitive Sciences at MIT, under the supervision of Ted Adelson, where I also frequently worked with Aude Oliva. I received my undergraduate degree in Computer Science from Yale, where I got my start on research working with Brian Scholl. A longer bio is here.

Quick links: Papers / Courses / Talks / Writing / Research Group

⭐

News

⭐

→ Slides from my keynote at WACV 2025: "Language as a Camera"

→ Slides and recording from a recent talk I gave on "Reasoning and search with deep nets"

Our computer vision textbook is finished!

Lots of things have happened since we started thinking about this book in November 2010; yes, it has taken us more than 10 years to write this book. Our initial goal was to write a large book that provided a good coverage of the field. Unfortunately, the field of computer vision is just too large for that. So, we decided to write a small book instead, limiting each chapter to no more than five pages. Writing a short book was perfect because we did not have time to write a long book and you did not have time to read it. Unfortunately, we have failed at that goal, too. This book covers foundational topics within computer vision, with an image processing and machine learning perspective. The audience is undergraduate and graduate students who are entering the field, but we hope experienced practitioners will find the book valuable as well.

Foundations of Computer Vision
Antonio Torralba, Phillip Isola, William F. Freeman
MIT Press

Research Group

The goal of our group is to scientifically understand intelligence. We are especially interested in human-like intelligence, which to us means intelligence that is built out of deep nets, is highly adaptive and general-purpose, and is emergent from embodied interactions in rich ecosystems.

Questions we are currently studying include the following, which you can click on to expand:

▸ Deep representation learning: What kinds of representations do deep nets learn? Why are these representations effective, and how are they limited?

Representative projects: Platonic Representation Hypothesis, Low-rank bias, Understanding contrastive learning

▸ Generative intelligence: How can we use generative models as mental simulation engines, supporting learning, inference, and control?

Representative projects: Learning from models, Learning from NeRFs, Denoised world models

▸ World representations for agents: How should an intelligent agent, such as a robot, represent the environment around it?

Representative projects: F3RM, Embodied representation learning, Mental imagery for robots

▸ Emergent intelligence: How can intelligence emerge from "scratch", without imitating another intelligence's cultural artifacts?

Representative projects: Learning without data, Neural MMO, PowderWorld

Our goal in studying these questions is to help equip the world with the tools necessary to bring about a positive integration of AI into society; to understand intelligence so we can prevent its harms and to reap its benefits.

The lab is part of the broader Embodied Intelligence and Visual Computing research communities at MIT.

PhD Students Caroline Chan Hyojin Bahng Akarsh Kumar Shobhita Sundaram Ishaan Preetam-Chandratreya Kaiya (Ivy) Zhao Yulu Gan Adam Rashid Ching Lam Choi	Postdocs Jeremy Bernstein Ge Yang Prafull Sharma MEng Students Laker Newhouse Undergraduates Uzay Girit Yajvan Ravan Sophie Wang

Former Members and Visitors Minyoung (Jacob) Huh (PhD), Tongzhou Wang (PhD), Alan Yu (UROP), Hannah Gao (UROP), Sage Simhon (MEng), Jeff Li (UROP, MEng), Joseph Suarez (PhD), Yen-Chen Lin (PhD), Lucy Chai (PhD), Swami Sankaranarayanan (Postdoc), Stephanie Fu (UROP, MEng), Kevin Frans (UROP, MEng), Yonglong Tian (PhD), Jerry Ngo (Visiting student), Taqiya Ehsan (Visiting student), Ali Jahanian (Research Scientist), Dillon Dupont (UROP), Kate Xu (UROP), Maxwell Jiang (UROP), Toru Lin (MEng), Kenny Derek (MEng), Yilun Du (UROP), Zhongxia Yan (Rotation)
Interested in joining the group? Please see info about applying here.

Recent Courses

6.7960: Deep Learning (Fall 2024)
6.s953: Embodied Intelligence (Spring 2024)
6.819/6.869: Advances in Computer Vision (Spring 2022)

New papers (All papers)

	Automating the Search for Artificial Life with Foundation Models Akarsh Kumar, Chris Lu, Louis Kirsch, Yujin Tang, Kenneth O. Stanley, Phillip Isola, David Ha arXiv 2024. [Paper (web)][Paper (pdf)][Code][Blog]
	Personalized Representation from Personalized Generation Shobhita Sundaram, Julia Chae, Yonglong Tian, Sara Beery, Phillip Isola ICLR 2025. [Paper][Website][Code][Data]
	Adaptive Length Image Tokenization via Recurrent Allocation Shivam Duggal, Phillip Isola, Antonio Torralba, William T. Freeman ICLR 2025. [Paper][Code]
	Learning Visual Parkour from Generated Images Alan Yu, Ge Yang, Ran Choi, Yajvan Ravan, John Leonard, Phillip Isola CoRL 2024. [Paper][Website][Code][Video][News Article]
	When Does Perceptual Alignment Benefit Vision Representations? Shobhita Sundaram, Stephanie Fu, Lukas Muttenthaler, Netanel Y. Tamir, Lucy Chai, Simon Kornblith, Trevor Darrell, Phillip Isola NeurIPS 2024. [Paper][Website][Code]
	Scalable Optimization in the Modular Norm Tim Large, Yang Liu, Minyoung Huh, Hyojin Bahng, Phillip Isola, Jeremy Bernstein NeurIPS 2024. [Paper][Code][Docs][Slides]
	The Platonic Representation Hypothesis Minyoung Huh, Brian Cheung, Tongzhou Wang, Phillip Isola ICML 2024 (Position Paper, Oral). [Paper][Website][Code]

...

All papers

Accessibility

	Automating the Search for Artificial Life with Foundation Models Akarsh Kumar, Chris Lu, Louis Kirsch, Yujin Tang, Kenneth O. Stanley, Phillip Isola, David Ha arXiv 2024. [Paper (web)][Paper (pdf)][Code][Blog]
	Personalized Representation from Personalized Generation Shobhita Sundaram, Julia Chae, Yonglong Tian, Sara Beery, Phillip Isola ICLR 2025. [Paper][Website][Code][Data]
	Adaptive Length Image Tokenization via Recurrent Allocation Shivam Duggal, Phillip Isola, Antonio Torralba, William T. Freeman ICLR 2025. [Paper][Code]
	Learning Visual Parkour from Generated Images Alan Yu, Ge Yang, Ran Choi, Yajvan Ravan, John Leonard, Phillip Isola CoRL 2024. [Paper][Website][Code][Video][News Article]
	When Does Perceptual Alignment Benefit Vision Representations? Shobhita Sundaram, Stephanie Fu, Lukas Muttenthaler, Netanel Y. Tamir, Lucy Chai, Simon Kornblith, Trevor Darrell, Phillip Isola NeurIPS 2024. [Paper][Website][Code]
	Scalable Optimization in the Modular Norm Tim Large, Yang Liu, Minyoung Huh, Hyojin Bahng, Phillip Isola, Jeremy Bernstein NeurIPS 2024. [Paper][Code][Docs][Slides]
	The Platonic Representation Hypothesis Minyoung Huh, Brian Cheung, Tongzhou Wang, Phillip Isola ICML 2024 (Position Paper, Oral). [Paper][Website][Code]