UROP Openings

Have a UROP opening you would like to submit?

Please fill out the form.

Submit your UROP opening

Beyond TEASER: From Robust Point Cloud Registration to Unsupervised Language Translation (Remote)


Term:

Fall

Department:

LIDS: Laboratory for Information & Decision Systems

Faculty Supervisor:

Luca Carlone and Jean-Jacques E. Slotine

Faculty email:

lcarlone@mit.edu

Apply by:

ASAP

Contact:

Heng Yang, hankyang@mit.edu

Project Description

In this project, we are interested in extending a state-of-the-art algorithm for robust 3D registration, namely TEASER (https://arxiv.org/abs/2001.07715), to unsupervised language translation. Some brief background information: Point cloud registration (PCR) is one of the most important tasks in computer vision and robotics. Given two sets of points in 3D, PCR seeks to find the best rigid transformation (rotation and translation) to align the two sets of points. A most popular algorithmic way to solve PCR consists in three phases: (1) Feature Descriptor: for each point in both point clouds, compute a transformation-invariant high-dimensional geometric feature, (2) Feature Matching: using the geometric feature, establish point-to-point correspondences by fast nearest neighbor search, (3) Robust Registration: given putative correspondences, find the best rigid transformation, typically by solving an optimization problem. For PCR, the (deep-learning-free) state-of-the-art methods are (1) FPFH descriptor, (2) KD Tree based FLANN and (3) TEASER, proposed by MIT SPARK Lab. On the other hand, language translation is one of the most important tasks in Natural Language Processing (NLP). Given two languages (for example English and Spanish), unsupervised language translation seeks to translate English words to Spanish words, without supervision from human-labelled dictionary. While sounds like a drastically different problem than PCR, language translation can be formulated as a high-dimensional PCR problem. The reason is because existing techniques in NLP can reliably generate word embeddings for each language, that is, for each word in a language, we associate a high-dimensional vector (point). Therefore, the problem of language translation can be formulated as registration of two sets of high-dimensional point clouds, that is, to find a high-dimensional rigid transformation to align two sets of word embeddings. Hence, in this project, we aim to transfer the algorithmic insights we already have in 3D registration (basically (1)-(3)) to language translation.

Pre-requisites

- Strong implementation skills in Python and Pytorch are required, especially KD Tree FLANN search. - Basic knowledge about vision and language are preferred. - Passionate about research, willing to read related work and solve hard problems.