Have a UROP opening you would like to submit?
Please fill out the form.
Painting Light and Style with a Generative Adversarial Network
6: Electrical Engineering and Computer Science
David Bau: firstname.lastname@example.org
How does a neural network, trained to imitate real photographs of the world, reason about light? In previous work (https://ganpaint.io/), we found that, when a generative adversarial network (GAN) is trained to synthesize real-looking photos, it learns sets of neurons that can be used to control the presence of object classes. Remarkably, manipulating neurons to add a light source such as a lamp or window, will also add reflected light, far from the object. In this project, we will investigate this phenomenon and improve our control of it. Our goal is to be able to improve our understanding of a GAN's model of attributes, and to create an interactive app that disentangles the factors of variation learned by a GAN (e.g., inspired by and collaborating with authors of https://ali-design.github.io/gan_steerability/). We want to be able to cleanly turn and off light sources, or change their color and intensity, without changing the presence of lamps or other objects in the scene. We also aim to cleanly control other attributes, such as style, size, and orientation of objects within a scene.