When math becomes music

Yuval Kaminka is determined to bring live music into homes around the globe—from one-handed piano melodies like Beethoven’s Ode to Joy to more complex renditions of today’s pop chart favorites. 

Kaminka, a Weizmann Institute computer science alumnus, is the CEO and co-founder of the Tel Aviv-based JoyTunes. JoyTunes’ piano-learning apps are its key products (pun intended) and the company plans to expand them to include other musical instruments. Placing a tablet or mobile phone on their piano’s music rack, users can launch one of three JoyTunes apps that instantly enable their device to recognize the notes they are playing.

The company’s three main products include Piano Maestro, an in-depth educational app used by teachers to accompany traditional lessons; Simply Piano, a step-by-step program for self-learners; and Piano Dust Buster, a playful introduction to piano for kids. All require a subscription.

“In every household, you’ll find someone who dreams about jamming on the guitar or someone who played piano at one point and wants to get back to it,” Kaminka says.  

While the apps teach piano students, they aren't meant to replace teachers. Instructors can use Piano Maestro in a variety of ways: as a special activity for their students during their lessons, for homework assignments, or as a digital method book. Simply Piano takes a linear approach—watch videos, complete exercises, play songs. About 30 seconds after learning their first three notes, students play a simple song, and about an hour later, they’re on to a version of Jingle Bells with two hands. 

Rather than disrupting traditional music education, JoyTunes aims to break down the mental block that often prevents people from casually trying out an instrument, says Kaminka. 

The app’s core technology is its acoustic recognition engine, which is able to distinguish each note that a user plays—transforming an ordinary smartphone into a powerful ear that sifts through sounds and provides instant feedback. Its main challenge is coping with “real life” conditions outside the laboratory, such as the restrictions posed by a single microphone, the presence of background noise, and the highly limited performance abilities of standard mobile devices. All the while, the app must precisely identify both single notes and chords (polyphonic recognition) through a “music sense” system that combines machine learning, neuro-linguistic programming, and acoustic recognition. 

Kaminka came up with the idea for JoyTunes after watching his nephew happily playing video games but protesting the idea of practicing piano. 

“We thought we’d take something very tedious and mechanical, and make it fun,” he says. 

Thinking like a scientist, he started brainstorming how computer algorithms might be able to help. Kaminka received his MSc from the Institute in 2009, under the supervision of Prof. Shimon Ullman from the Department of Computer Science and Applied Mathematics and Dr. Elad Schneidman from the Department of Neurobiology. 

During his studies, Kaminka says he read several published papers on human computation—how the mind does wondrous things but also benefits from extensive training. This sparked his interest in music learning and self expression through music, as well as how these skills could be cultivated. He started JoyTunes in 2010 with Roey Izkovsky, who became its Chief Technology Officer; Izkovsky is also a Weizmann Institute alumnus, who earned his MSc in 2007 under Prof. Itai Benjamini in the Department of Mathematics. 

At the company’s headquarters, visitors will find a glossy upright piano in an eclectic lobby festooned with musical trinkets. Along an exposed brick wall, a record stylus keeps time on a vintage vinyl clock, while a giant poster of cassette tapes hangs behind the communal dining table.

While Kaminka briefly dabbled in both violin and saxophone, Izkovsky has no musical background. But the concept was music to his ears, and he is responsible for the company’s acoustic recognition technology. The third co-founder and musical director, Kaminka’s brother Yigal, was the principal oboist in the Jerusalem Symphony Orchestra. 

Singing praise

The company has won accolades from experts in the field. Eran Egozy, co-founder of Harmonix Music Systems (which put out Guitar Hero and Rock Band), and currently a music professor at MIT, says he sees JoyTunes “as a shining example of a very successful music startup.”

“JoyTunes has created a powerful platform,” he adds. “The result is a fast growing community of piano students and teachers that is rekindling the joy of playing piano.”

Linda Christensen, a piano instructor at a school in Maryland, credits JoyTunes for changing “the face and culture of piano teaching.” She adds, “The Piano Maestro app in particular has allowed teachers to incorporate technology easily, allowing for dynamic engagement between teacher, student, and parent.”

Particularly appreciative of the collaborative culture of critical thinking on the Weizmann campus, Kaminka says he makes sure to encourage his employees to ask tough questions.

“I very much liked the idealism of science there—the openness of the community, of sharing knowledge, and this common goal of advancing science,” he adds. “We like to take that into the company.”