UROP Openings

Have a UROP opening you would like to submit?

Please fill out the form.

Submit your UROP opening

Dissecting a Deep RL Network


Term:

Summer

Department:

6: Electrical Engineering and Computer Science

Faculty Supervisor:

Antonio Torralba

Faculty email:

torralba@mit.edu

Apply by:

5/4/2020

Contact:

David Bau: davidbau@csail.mit.edu, Put "UROP" in the subject line of your email.

Project Description

Help uncover the hidden structure of a deep reinforcement learning network. We will develop a framework for visualizing, understanding, and controlling the hidden layers of network trained by reinforcement learning. Our lab's previous work on network dissection in vision has revealed nontrivial emergent structure within the hidden layers of deep networks for vision, and this understanding has allowed us to develop new semantic image manipulation applications. For example, by identifying networks that act as switches for high-level visual concepts, we can allow a human user to directly control those neurons to add or remove objects in a photorealistic image. Now we seek to develop similar methods for deep reinforcement learning. We will train DRL systems in a setting that will allow us to characterize, visualize, and control their internal encoding of state, memory, and decision-making. You will help us design and train a reference setting for DRL networks, with the goal of understanding whether and how DRL systems represent emergent signals for high-level semantics.

Pre-requisites

You should have coursework in machine learning and should be familiar with the principles of deep reinforcement learning, with practical experience training models using pytorch or similar. You should have coursework that includes in-depth work on related mathematical topics such as linear algebra, signal processing, statistics, or control systems.