As far as the eye can see—and beyond

Our eyes—and the extensive human visual system behind them—have a remarkable ability to make sense of our three-dimensional world from two-dimensional information. Mimicking this ability with artificial means, through algorithms and computer-based technologies which analyze two-dimension images, is a major challenge. This is precisely the focus of the research of Dr. Tali Dekel, who joins the Department of Computer Science and Applied Mathematics in January.

Dr. Dekel comes to Weizmann after a postdoctoral fellowship, first at MIT’s Computer Science and Artificial Intelligence Lab followed by a four-year stint as a Senior Research Scientist at Google in Cambridge, Massachusetts.

Recreating spatial geometry from two-dimensional images is one of the fundamental challenges in the visual computational field. The object must be captured from two points of view at any given time, which requires either the subject or the camera to be stationary. Interpreting the three-dimensionality of the scene when both camera and object are in motion has remained a fundamental stumbling block in the field of computer vision.

At Google, Dr. Dekel developed a solution to this challenge through combining fundamental knowledge in computer vision and deep learning. Deep learning—a subset of artificial intelligence where machines can learn directly by observing large amounts of data—has been revolutionizing computer vision and graphics in recent years.

To “teach” a computer to predict the physical dimensions of a human from 2D images, Dr. Dekel discovered a surprising new source of data: thousands of YouTube videos of the “Mannequin Challenge”—an Internet video trend in which people imitate mannequins by freezing in place while a moving camera films them. Because the subjects are stationary, it is possible to accurately estimate the geometry of the entire scene, including the people in it.

Based on a large collection of such videos and geometry behind them, Dr. Dekel designed a deep learning-based model that could analyze nonstationary video footage and accurately interpret the geometry of any moving human in it.  Future potential applications of this technology are numerous—from advanced computer graphic effects for example, augmented reality, robotics, and automatic navigation.

Of more than 5,000 papers submitted to the 2019 Computer Vision and Pattern Recognition conference—the premier annual computer vision event held in Long Beach, California in June 2019—this work was one of two receiving honorable mention awards.

Mimicking movement

In another project, Dr. Dekel developed the Motion Sculpture system, or “MoSculp,” which can visualize the complex 3D movement of a human movement. For example, if presented with a video of an Olympic runner, MoSculp can recreate the 3D path traced by the subject as he or she moves through space—and render a 3D simulation. It can even physically print a sculpture of the runner in three dimensions.

The MIT Museum recently worked with Dr. Dekel to create a new exhibit, “In Motion,” which explores movement. Though the museum has temporarily closed its doors due to the COVID-19 pandemic, the exhibit displays pieces inspired by her video-to-sculpture projects and will be open to the public once the museum is back in operation.

These are only a sampling of Dr. Dekel’s deep-learning projects in computer vision and graphics. She has contributed to many models that are helping advance technology and greatly enhance our visual and auditory perception of the world.

“In the new era of deep learning, we can go beyond what we may think is possible today,” says Dr. Dekel. “A whole new world of research questions can be asked and addressed.”

Dr. Tali Dekel earned her BSc and MSc in electrical engineering, both cum laude, from Tel Aviv University. After interning at the Disney Research lab at ETH Zurich (the Swiss Federal Institute of Technology), she returned to Tel Aviv University to complete her PhD in electrical engineering and computer vision in 2015. She completed her postdoctoral fellowship with the support of a Rothschild Postdoctoral Fellowship and the Israel National Postdoctoral Award for Advancing Women in Science.

This year, she received the Alon Fellowship for outstanding young researchers. In 2019, Dr. Dekel was the first woman to win the prestigious Eric and Sheila Samson Prime Minister's Prize. She received the Prof. Norman W. Rosenberg Memorial Prize at Tel Aviv University in 2013, and an Excellence Scholarship from the Tel Aviv University School of Electrical Engineering in 2009. In 2005, she won a scholarship from RAD Data Communications, Ltd.

She is married and has three children.