Phillip Isola

p h i l l i p i @ m i t . e d u
Google Scholar / GitHub / Twitter

About me

I am an associate professor in EECS at MIT studying computer vision, machine learning, and AI.

Previously, I spent a year as a visiting research scientist at OpenAI, and before that I was a postdoctoral scholar with Alyosha Efros in the EECS department at UC Berkeley. I completed my Ph.D. in Brain & Cognitive Sciences at MIT, under the supervision of Ted Adelson, where I also frequently worked with Aude Oliva. I received my undergraduate degree in Computer Science from Yale, where I got my start on research working with Brian Scholl. A longer bio is here.

Quick links: Papers / Courses / Talks / Writing / Research Group

Research Group

The goal of our group is to scientifically understand intelligence. We are especially interested in human-like intelligence, i.e. intelligence that is built out of deep nets, is highly adaptive and general-purpose, and is emergent from embodied interactions in rich ecosystems.

Questions we are studying include the following, which you can click on to expand:

▸ Science of deep learning: Why do large models find solutions that generalize? What structures improve generalization? And when might these methods still fail?

Representative projects: Low-rank bias, Quasimetric learning, What makes for good views for contrastive learning

▸ Emergent intelligence: How can intelligence emerge from data and tasks, and how can it emerge without imitating another intelligence's cultural artifacts?

Representative projects: Neural MMO, PowderWorld

▸ Embodied intelligence: To what extent is physical embodiment, in interactive environments, useful or necessary for intelligence?

Representative projects: Embodied representation learning, Mental imagery for robots

▸ Synthetic data and environments: What kinds of training data / environments are most instructive toward achieving robust, aligned, and general intelligence?

Representative projects: Generative data, Procedural data, NeRF data

▸ Controllable AI: How can we make AI systems that can be steered, edited, and controlled by human users?

Representative projects: Visual prompting, GAN steering, Totems

Our goal in studying these questions is to help equip the world with the tools necessary to bring about a positive integration of AI into society; to understand intelligence so we can prevent its harms and to reap its benefits.

The lab is part of the broader Embodied Intelligence and Visual Computing research communities at MIT.

PhD Students Caroline Chan Tongzhou Wang Joseph Suarez Minyoung (Jacob) Huh Hyojin Bahng Akarsh Kumar Shobhita Sundaram Ishaan Preetam-Chandratreya	MEng Students Sage Simhon Jeff Li Postdocs Jeremy Bernstein Ge Yang Undergraduates Alan Yu

Former Members and Visitors Yen-Chen Lin (PhD), Lucy Chai (PhD), Swami Sankaranarayanan (Postdoc), Stephanie Fu (UROP, MEng), Kevin Frans (UROP, MEng), Yonglong Tian (PhD), Jerry Ngo (Visiting student), Taqiya Ehsan (Visiting student), Ali Jahanian (Research Scientist), Dillon Dupont (UROP), Kate Xu (UROP), Maxwell Jiang (UROP), Toru Lin (MEng), Kenny Derek (MEng), Yilun Du (UROP), Zhongxia Yan (Rotation)
Interested in joining the group? Please see info about applying here.

Recent Courses

6.s898: Deep Learning (Fall 2023)
6.s898: Deep Learning (Fall 2022)
6.819/6.869: Advances in Computer Vision (Spring 2022)
6.s898: Deep Learning (Fall 2021)
6.036: Introduction to Machine Learning (Fall 2020)
6.882: Embodied Intelligence (Spring 2020)

New papers (All papers)

	Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation William Shen, Ge Yang, Alan Yu, Jansen Wong, Leslie Kaelbling, Phillip Isola CoRL 2023 (oral). [Paper][Website][Code][Video]
	Learning New Dimensions of Human Visual Similarity using Synthetic Data Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, Phillip Isola NeurIPS 2023 (spotlight)*. [Paper][Website][Code/Data][Colab]
	StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, Dilip Krishnan NeurIPS 2023. [Paper]
	Improving CLIP Training with Language Rewrites Lijie Fan, Dilip Krishnan, Phillip Isola, Dina Katabi, Yonglong Tian NeurIPS 2023. [Paper][Code]
	Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks Minyoung Huh, Brian Cheung, Pulkit Agrawal, Phillip Isola ICML 2023. [Paper][Website][Code]
	Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning Tongzhou Wang, Antonio Torralba, Phillip Isola, Amy Zhang ICML 2023. [Paper][Website][Code]
	Persistent Nature: A Generative Model of Unbounded 3D Worlds Lucy Chai, Richard Tucker, Zhengqi Li, Phillip Isola, Noah Snavely CVPR 2023. [Paper][Website][Code]
	Powderworld: A Platform for Understanding Generalization via Rich Task Distributions Kevin Frans, Phillip Isola ICLR 2023. [Paper][Blog + Demo][Code]

Accessibility