Although VR and AR have been around for a long time, now treated under a wider mixed (MR) or eXtended reality (XR) umbrella, most people still consider these experiences as complicated and unnecessary extensions of our computer monitors. After all, seeing digital objects floating in front of you gets old quickly if all you can do is to “click” on them or type on an invisible keyboard.
The technology has a lot more to offer, of course, especially given Apple’s push for its own brand of “spatial computing.” Unfortunately, the value of mixed reality can really only be appreciated if you have applications and experiences that truly showcase the power of this fusion of the real world and the virtual world, something that this educational music app pulls off with aplomb.
Designer: Simply
Augmented reality, as the name suggests, tries to augment or supplement what we can see in the real world by overlaying digital objects or markers over physical objects. Spatial computing takes it a bit further by making it a two-way street, where the real world’s physics, movement, and position can affect those virtual objects in turn. Of course, you’ll need specialized equipment to experience this, which is what the Apple Vision Pro brings to the table, or to your head, rather.
Simply Piano is an app that originally taught piano lessons using an iPhone or an iPad, but its new Vision Pro incarnation takes learning to the next level. Using Apple’s spatial computing platform, it not only pins a virtual score or guide above your keyboard, it can also detect your fingers, highlight the keys on the piano you need to hit, and let you know if you’ve made a mistake, just in case you out of tune note didn’t clue you in.
The magic happens when you find out that you can do all these things even if you don’t own a piano or electric keyboard. It projects a virtual keyboard that only you can see on any flat surface in front of you, allowing you to learn and practice anywhere without even making a sound. It uses all those same technologies and techniques to map your finger movement to the digital keyboard so you can use it on a plane, on your desk, or really anywhere you can tap comfortably on a table of sorts.
There are, of course, limitations to this learning technique, since Simply Piano won’t be able to check your hand posture, the strength of your key presses (which would be crucial to a real piano), and similar aspects of playing the instrument. It’s still a fun way to learn anywhere without even requiring a physical piano. Presuming, of course, you’re in a place where you can wear the Vision Pro headset.
The post Simply Piano Apple Vision Pro app teaches you to play even without a real keyboard first appeared on Yanko Design.