UROP Openings

Have a UROP opening you would like to submit?

Please fill out the form.

Submit your UROP opening

Learning from Humans in VirtualHome


IAP and Spring


CSAIL: Computer Science and Artificial Intelligence Lab

Faculty Supervisor:

Antonio Torralba

Faculty email:


Apply by:



Xavier Puig xpuig@mit.edu

Project Description

We want to build a database of household activities with the goal of creating better machine learning models for activity understanding, or developing agents that can learn to perform them. We provide a realistic simulator called VirtualHome (http://virtual-home.org/) that allows synthesizing human activities in a virtual environment. Our goal is to collect a large database of activities by asking people to interact and do tasks in the simulator. The collected activities will provide information about how humans plan tasks, and how they reason and interact with new environments, which will help build better models to understand human goals, or explore problems in Human-AI collaboration. Some of the challenges in this project will include 1) Designing an interface to interact with the simulator online, so that we can crowd-source human activities, 2) Defining the dataset and data collection strategies, 3) Building models to explore the collected dataset that reason about people's goals, preferences or knowledge of the environment. Check out http://virtual-home.org/ for an overview of related projects.


Python, Javascript. C#, Experience in Unity development, web development and Amazon Turk is preferred. Weekly availability and curiosity.