The core of machine learning

Deep learning theory, a sub-field of machine learning, holds the promise of transforming artificial intelligence. This area of research focuses on fundamental mathematical questions about neural networks—a method in AI that uses layered structures of interconnected nodes or neurons to teach computers to process data in a way inspired by the human brain. 

Dr. Gal Vardi, a new member of the Department of Computer Science and Applied Mathematics, is working to understand the underlying mathematical principles that determine when deep learning works—and when it doesn’t, and what factors control the success or failure of training neural networks. 

At the core of machine learning is the concept of generalization—using real-world examples to “train” artificial neural networks. For example, a network can be trained to recognize whether an image shows a cat or a dog after it has been shown many images of the same animals. The challenge is that while the human brain excels at generalization from just a few examples, artificial neural networks need vast data to perform similarly. 

“The objective is to use existing examples to find an algorithm that performs well on new examples,” he says. 

Dr. Vardi explains that while generalization is fundamental to machine learning and has been studied for decades, we still don’t fully understand generalization in the context of deep learning. 

“We don’t understand what controls whether a neural network generalizes well or not,” he says.

Theory vs. real-world 

One of Dr. Vardi’s main research areas is probing the factors behind a neural network’s ability to generalize from a theoretical, mathematical perspective. 

“If we can gain good insights into what helps generalization, we might also be able to improve practical applications,” he says. “I like the whole field of deep learning theory because it’s intriguing and challenging from the mathematics perspective, but also connected to real-world applications.”  

Born in Rehovot, Dr. Vardi was drawn to math and computer science from a young age and completed his BSc at The Hebrew University of Jerusalem as a cadet in the IDF’s prestigious Talpiot Program, designed to nurture the country’s most talented young scientists. 

“I didn’t know that this was what I was going to do until a much later stage,” he says.

A new algorithm

After earning his MSc and PhD in computer science under Prof. Orna Kupferman at Hebrew University, he came to the Weizmann Institute as a postdoctoral fellow in the group of Prof. Ohad Shamir in the Department of Computer Science and Applied Mathematics, with whom he published numerous studies in the field of deep learning.

From 2022-2024, he was a joint postdoctoral fellow at the Toyota Technological Institute-Chicago and Hebrew University, where he was also supported by the National Science Foundation/Simons Collaboration on the Theoretical Foundations of Deep Learning. 

In June 2024, Dr. Vardi returned to the Weizmann Institute as a principal investigator. He is now collaborating with his department dean and colleague, Prof. Michal Irani, a leader in the study of computer vision. One of their joint studies has already yielded exciting insights: using theoretical results from deep learning, they developed an algorithm that identifies the examples a given network was trained on.

While many young scientists at Weizmann live on campus, Dr. Vardi and his wife, Liron, a gynecology resident at Hillel Yaffe Medical Center in Hadera, and their two young children have made Kfar Saba their home. 

While the decision to come to Weizmann, rather than his alma mater of Hebrew University, might not be an obvious choice, Dr. Vardi enjoys “the spirit of Weizmann,” as well as the people. “I was really happy to come back to the Institute,” he says. 

Education and select awards

• BSc, The Hebrew University of Jerusalem, Talpiot Program (2007)
• MBA, Tel Aviv University (2011)
• MSc (2015) and PhD (2019), Hebrew University
• Postdoctoral Fellow, Weizmann Institute of Science (2020-2022)
• Postdoctoral Fellow, Toyota Technological Institute-Chicago/Hebrew University (2022-2024)
• Koshland Prize from the Weizmann Institute (2020); Feinberg Graduate School Prize for Outstanding Achievements in Postdoctoral Research (2022); Postdoctoral Fellow, National Science Foundation/Simons Collaboration on the Theoretical Foundations of Deep Learning (2022-2024);
• Zuckerman Faculty Scholar (2024).
   

GAL VARDI IS SUPPORTED BY: 

• Shimon and Golde Picker - Weizmann Annual Grant
• Zuckerman Faculty Scholar