Piano Projector

The challenge

Can a Guitar Hero like concept for piano help you learn faster?

The details

I set up a projector to display the piano app Synthesia on top of my piano keyboard at home - aka projection mapping.

The goal was to investigate if it was easier to learn piano using this approach, where you can easily see what keys you need to play next, instead of having to look at sheet music or your computer screen / phone and then try to identify the right keys to play.

The piano keyboard was connected over USB to my laptop, running Synthesia. Synthesia is able to display the required keys for a particular music piece as well as register the piano key presses.

There are a variety of modes to use in the app, for example, continuous mode displays the whole song from start to finish like with Guitar Hero.

In melody practice, the app waits for you to play the correct note before moving on.

The findings

From my experience, if you already know how to play a bit of piano, this approach was useful to be able to quickly play a song "guitar hero style". Having some basic knowledge is necessary so you can quickly correlate the incoming notes with what to play next.

In terms of actually learning how to play a new song, I found it challenging and not very helpful, as you are too focused on what to play next rather than actually registering what you just played and trying to memorise it. Given the size of the keyboard, it can also be overwhelming to keep track of all the notes by looking at the projections, instead of a smaller screen. I found looking at a video for 10 seconds, trying to play a segment and repeating that until I got it right was more natural.

For beginners a combination of the projection to identify the right keys on the keyboard and looking at your laptop to be able to start / stop and loop the music to practice more thoroughly could be useful.