UROP Openings

Have a UROP opening you would like to submit?

Please fill out the form.

Submit your UROP opening

Combining touch and vision to learn robotics tasks




2: Mechanical Engineering

Faculty Supervisor:

Alberto Rodriguez

Faculty email:


Apply by:



Maria Bauza Villalonga: bauza@mit.edu

Project Description

The lack of tactile reasoning still remains one of the main limitations of dexterous robotic manipulation and a long-standing challenge in the robotics community. After decades of advances in sensing instrumentation and processing power, the basic question remains: How should robots make use of sensed contact information? To partially answer this question, we are exploring how tactile sensing can help to localize objects in hand and help to better manipulate them. The goal of this UROP project is to contribute to our long-term goal to allow robots to learn to interact with the world using all their senses. The project will be discussed with the student to accommodate her/his interests. Among the possible projects there are: - Combining computer vision techniques with touch to improve robotic perception - Learning in a supervised way how to solve complex tasks such as manipulating objects in hand and re-grasping them - Using multi-finger hands together with touch. This is a challenging task as controlling and efficiently learning useful behaviors with complex manipulators is still a wide-open problem. - Solve a small reinforcement learning task. While reinforcement in robotics is a hard problem, there are subsets of the manipulation problem that can be solved in simulation and then transferred to the real world. Some of them can be explored in the scope of this project For those interested in PhD programs, most of these projects are likely to result in scientific publications.


Python, general computer science background. Desirable: ROS Machine Learning: 6.036 or above Keras, Pytorch Interested in entering a PhD program Availability on summer is preferable, but not determinant.