UROP Openings

Have a UROP opening you would like to submit?

Please fill out the form.

Submit your UROP opening

Rewriting a Mapping Network


Term:

Fall

Department:

6: Electrical Engineering and Computer Science

Faculty Supervisor:

Antonio Torralba

Faculty email:

torralba@mit.edu

Apply by:

9/7/2020

Contact:

davidbau@csail.mit.edu

Project Description

Why is it easy for a person to take a walk in unfamiliar terrain? After you bump into one prickly bush or slippery rock, you know right away not to step on a similar one again. Our lab has developed methods for rewriting rules inside generative networks for vision (see rewriting.csail.mit.edu), and we are investigating how to use these methods to quickly rewrite the rules inside networks used for perception, to help an agent more easily navigate a world full of novelty. In this project, you will help us build a visual occupancy network that builds maps of the visual world from a monocular view - we will work in a synthetic domain similar to minecraft - and we will investigate the structure of the network to find ways to write in changes. Our goal: to be able to add knowledge about specific new visual patterns that might not have been seeing during training, without retraining the whole network.

Pre-requisites

Machine learning and computer vision (e.g. 6.869 or similar), and comfort with linear algebra and pytorch.