UROP Openings

Learning from Humans in VirtualHome

Project Description

We want to build a database of household activities with the goal of creating better machine learning models for activity understanding, or developing agents that can learn to perform them. We provide a realistic simulator called VirtualHome (http://virtual-home.org/) that allows synthesizing human activities in a virtual environment. Our goal is to collect a large database of activities by asking people to interact and do tasks in the simulator. The collected activities will provide information about how humans plan tasks, and how they reason and interact with new environments, which will help build better models to understand human goals, or explore problems in Human-AI collaboration. Some of the challenges in this project will include 1) Designing an interface to interact with the simulator online, so that we can crowd-source human activities, 2) Defining the dataset and data collection strategies, 3) Building models to explore the collected dataset that reason about people's goals, preferences or knowledge of the environment. Check out http://virtual-home.org/ for an overview of related projects.

Pre-requisites

Python, Javascript. C#, Experience in Unity development, web development and Amazon Turk is preferred. Weekly availability and curiosity.