When Penn Medicine student Tim Soo realized he left his violin at home, he decided to build one. His new “violin” involved no strings, a bow made from a Wii remote and sounds from a music library on his Mac.
He used his invention for a music theory class project and has since created other so-called “Invisible Instruments.” The invisible instruments designed by Soo allow a computer to register movements from a Wii remote — either as strumming, bowing or drumming — and an iPod touch or iPhone, which can register finger placement and tilting along the wrist.
Although this started as “a gimmicky thing to get an A,” it has since earned Soo a Music Hack Day award and an invitation to the Awesome Foundation event in Boston. Now, the latest version is what one could expect “if a real guitar and guitar hero had a kid,” Soo said.
As a classically trained musician, Soo recognizes the shortcomings of his project — lag time and limited manipulation are problems he is still correcting. However, the invisible instruments available span drum set, guitar and violin, and the program incorporates more-advanced techniques such as vibrato. The freedom from a real instrument is also a benefit for Soo, who does not have to worry about violin-related injuries because there is nothing physical to limit his posture.
Soo described the programming process as “very intuitive” and by “trial and error.” As he incorporated new aspects — such as an iPod touch to act as the fingerboard or frets — and new instruments, Soo faced new challenges.
Due to Mac’s copyright setup, the Wii remote and the iPod cannot communicate directly. Instead, Soo had to indirectly connect them through four separate programs on his Mac laptop. These programs communicate with the remote to determine when to “start the note, stop the note and how loud,” Soo said.
Also, Soo has discovered that the accelerometer — which monitors the orientation of the iPod and controller in space — was not ideal, so instruments such as the harp would require extensive corrections. Otherwise, the invisible harp would be constantly moving away from the user, Soo said.
In the future, Soo hopes to take advantage of the microphone in the iPod touch by creating invisible woodwind instruments, such as clarinet or flute that would allow a user to manipulate the music with breath.
But invisible instruments isn’t Soo’s main project. “I built this in a weekend,” he admitted. Now, he is working to combine his programming and medical interests because in the future, he hopes to develop “a new way for patients to talk with each other online.”
