Phillip Isola

p h i l l i p i @ m i t . e d u
Google Scholar / GitHub / Twitter

About me

I am an associate professor in EECS at MIT studying computer vision, machine learning, and AI.

Previously, I spent a year as a visiting research scientist at OpenAI, and before that I was a postdoctoral scholar with Alyosha Efros in the EECS department at UC Berkeley. I completed my Ph.D. in Brain & Cognitive Sciences at MIT, under the supervision of Ted Adelson, where I also frequently worked with Aude Oliva. I received my undergraduate degree in Computer Science from Yale, where I got my start on research working with Brian Scholl. A longer bio is here.

Quick links: Papers / Courses / Talks / Writing / Research Group


News



→ I rewrote the description of our lab's research agenda. Like everyone else, we are reorienting, given the pace of recent progress. We will focus more on the science of intelligence, and on the positive integration of AI systems into society. 👇

Research Group

The goal of our group is to scientifically understand intelligence. We are especially interested in human-like intelligence, i.e. intelligence that is built out of deep nets, is highly adaptive and general-purpose, and is emergent from embodied interactions in rich ecosystems.

Questions we are studying include the following, which you can click on to expand:

Science of deep learning: Why do large models find solutions that generalize? What structures improve generalization? And when might these methods still fail?

Representative projects: Low-rank bias, Quasimetric learning, What makes for good views for contrastive learning

Emergent intelligence: How can intelligence emerge from data and tasks, and how can it emerge without imitating another intelligence's cultural artifacts?

Representative projects: Neural MMO, PowderWorld

Embodied intelligence: To what extent is physical embodiment, in interactive environments, useful or necessary for intelligence?

Representative projects: Embodied representation learning, Mental imagery for robots

Role of data and environments: What kinds of training data / environments are most instructive toward achieving robust, aligned, and general intelligence?

Representative projects: Generative data, Procedural data, NeRF data

Controllable AI: How can we make AI systems that can be steered, edited, and controlled by human users?

Representative projects: Visual prompting, GAN steering, Totems


Our goal in studying these questions is to help equip the world with the tools necessary to bring about a positive integration of AI into society; to understand intelligence so we can prevent its harms and to reap its benefits.

The lab is part of the broader Embodied Intelligence and Visual Computing research communities at MIT.

PhD Students
Yen-Chen Lin
Caroline Chan
Lucy Chai
Tongzhou Wang
Joseph Suarez
Minyoung (Jacob) Huh
Hyojin Bahng
Akarsh Kumar
Shobhita Sundaram
MEng Students
Kevin Frans
Stephanie Fu
Sage Simhon
Postdocs
Swami Sankaranarayanan


Undergraduates
Jeff Li
Alan Yu
Former Members and Visitors
Yonglong Tian (PhD), Jerry Ngo (Visiting student), Taqiya Ehsan (Visiting student), Ali Jahanian (Research Scientist), Dillon Dupont (UROP), Kate Xu (UROP), Maxwell Jiang (UROP), Toru Lin (MEng), Kenny Derek (MEng), Yilun Du (UROP), Zhongxia Yan (Rotation)
Interested in joining the group? Please see info about applying here.



Recent papers

Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning
Tongzhou Wang, Antonio Torralba, Phillip Isola, Amy Zhang
arXiv 2023.
[Paper][Website]
Persistent Nature: A Generative Model of Unbounded 3D Worlds
Lucy Chai, Richard Tucker, Zhengqi Li, Phillip Isola, Noah Snavely
CVPR 2023.
[Paper][Website]
Powderworld: A Platform for Understanding Generalization via Rich Task Distributions
Kevin Frans, Phillip Isola
ICLR 2023.
[Paper][Blog + Demo][Code]
The Low-Rank Simplicity Bias in Deep Networks
Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, Phillip Isola
TMLR 2023.
[Paper][Website][Code]
Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddings
Tongzhou Wang, Phillip Isola
NeurReps Workshop 2022.
[Paper][Website][Code]
MIRA: Mental Imagery for Robotic Affordances
Lin Yen-Chen, Pete Florence, Andy Zeng, Jonathan T. Barron, Yilun Du, Wei-Chiu Ma, Anthony Simeonov, Alberto Rodriguez Garcia, Phillip Isola
CoRL 2022.
[Paper][Website][Video]
Semantic uncertainty intervals for disentangled latent spaces
Swami Sankaranarayanan, Anastasios Angelopoulos, Stephen Bates, Yaniv Romoano, Phillip Isola
NeurIPS 2022.
[Paper][Website][Code][Video]
Procedural Image Programs for Representation Learning
Manel Baradad, Chun-Fu (Richard) Chen, Jonas Wulff, Tongzhou Wang, Rogerio Feris, Antonio Torralba, Phillip Isola
NeurIPS 2022.
[Paper][Website][Code & Datasets]
Offline Multi-Agent Reinforcement Learning with Knowledge Distillation
Wei-Cheng Tseng, Tsun-Hsuan Wang, Lin Yen-Chen, Phillip Isola
NeurIPS 2022.
[Paper]
Totems: Physical Objects for Verifying Visual Integrity
Jingwei Ma, Lucy Chai, Minyoung Huh, Tongzhou Wang, Sernam Lim, Phillip Isola, Antonio Torralba
ECCV 2022.
[Paper][Website][Code][Video]
Any-resolution Training for High-resolution Image Synthesis
Lucy Chai, Michaël Gharbi, Eli Shechtman, Phillip Isola, Richard Zhang
ECCV 2022.
[Paper][Website][Code][Video]
Denoised MDPs: Learning World Models Better Than The World Itself
Tongzhou Wang, Simon S. Du, Antonio Torralba, Phillip Isola, Amy Zhang, Yuandong Tian
ICML 2022.
[Paper][Website][Code]
Exploring Visual Prompts for Adapting Large-Scale Models
Hyojin Bahng, Ali Jahanian*, Swami Sankaranarayanan*, Phillip Isola
arXiv 2022.
[Paper][Website][Code]
Learning to generate line drawings that convey geometry and semantics
Caroline Chan, Frédo Durand, Phillip Isola
CVPR 2022.
[Paper][Website][Code][Demo]
On the Learning and Learnability of Quasimetrics
Tongzhou Wang, Phillip Isola
ICLR 2022.
[Paper][Website][Code]
Generative Models as a Data Source for Multiview Representation Learning
Ali Jahanian, Xavier Puig, Yonglong Tian, Phillip Isola
ICLR 2022.
[Paper][Website][Code][News article]
NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields
Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Tsung-Yi Lin, Alberto Rodriguez, Phillip Isola
ICRA 2022.
[Paper][Website][Video][Code][Colab]

Older papers


Accessibility