Pianoply is color coded karaoke for piano. In your browser, free. It layers gorgeous graphics as well as sounds right on top of the original YouTube music videos for the songs you love. Users supply the MIDI "piano rolls" for songs, which they can create and edit right in the app. The app is very interactive and talks to your MIDI piano, unlike all the zillions of "piano tutorials" on Youtube.

Above is a short intro, but if you'd rather just read about it (until you can make it back to your computer or to somewhere you can turn up the volume), read on. Also, scroll down for a whole bunch of links to complete songs that you can play along with right now, if you have an electronic piano attached. (first you'll want to read the first couple items in the FAQ though). Or, go to the grid (computer only for now...not for touch screens yet, sorry!).


(part 1, the essentials)

How do I play songs on Pianoply?

You simply connect a digital piano or MIDI controller to your computer (or fast Android smartphone or tablet), and click on any links to songs below. Currently only Chrome supports MIDI devices, but we understand that Firefox and Safari will have support soon. Even without a piano connected, you can enjoy the songs using the overlaid graphics and sounds as a "music appreciation device."

Pianorolls, also known as MIDI sequences, are named for player piano rolls.

More songs will arrive soon, since users contribute the "pianorolls". It helps if you have colored stickers (or marks from washable markers) on the white keys of your piano, but it isn't necessary. The colors are red, orange, yellow, green, blue, purple, and pink, for C D E F G A B. There isn't a lot to learn to be able to play songs. Black keys are "striped" with the colors of the two adjacent white keys. That's pretty much it.

Anything else I need to know?

You should learn a few of the "keyboard shortcuts," which are handy for doing things like changing the instrument, toggling the piano display (photoreal vs. minimal), adjusting the volume of the video, adjusting the volume of the instrument, jumping around in a song, changing the speed, transposing, turning on and off note names (D# etc), and turning on and off "autoplay." Most of them are two letters (or a letter and a number), so for instance to jump to the 40% mark in the video, simply type "j" followed by "4". A menu is shown after the first letter, so to see all the "jump" commands, press "j" and you'll see this:

To adjust video volume, press "v" and a number. To adjust instrument volume, press "i" and a number. To change the instrument, press up and down arrow keys. To change the speed, press "s" and look at the menu choices. To toggle settings, press "t" and look at the options. One of the most important toggles is autoplay, which you turn on and off by pressing "t" then "a". If you want to see all the keyboard shortcuts, just press "k".

Please note that keyboard shortcuts aren't going to work on a tablet unless you have a bluetooth keyboard. The first version of Pianoply is optimized for a computer rather than a touch screen device. We're working on it and intend to have full touchscreen support in the not so distant future.

Update: Here's a short video showing the basic keyboard shortcuts:

How do I submit/share a pianoroll for a music video?

It's pretty straightforward, however, right now we are requiring that people host their own pianoroll files either on their own server, or by simply putting it in a pastebin. It's both to assure you that "you own your own data," and because we are being careful regarding copyright issues. Note that you can even do short pianorolls that are not hosted, by creating long urls that contain all the information, and that you can easily paste into an email. There is currently a private beta that you should probably join if you want a bit more help with the process. (email rjbrown at gmail if you want to be in the beta) It will soon get easier, and tips and techniques will be published. However, for the next month or two, I expect that most casual users will want to just play along with existing songs rather than create their own.

I found a piano tutorial on YouTube I'd like to bring into Pianoply. Is there any easy way to do that?

Yep, if the tutorial is done with Synthesia or a similar app, anyway. There is a really powerful tool for automatically "scraping" the MIDI data from these videos, and then synching them up perfectly with the original videos. Many of the newest songs are done this way, and you can see it does it with a great deal of precision. Check out this video that gives an overview:

This seems like it would be perfect for kids. Is that the idea?

Pianoply was absolutely inspired by my daughter Stella's love of music. She was less than 2 when I started the project and now she's five. She loves music and she absolutely loves Pianoply. Whenever a new music video comes out that she likes (which is a couple times a week), she's on my case to "put colors on it." We often play duets Stella and I often play duets, where she'll play the bass line and I'll play the melody or vice versa. She also spends a lot of time listening and watching passively (in "autoplay" mode), which she finds much more satisfying than watching the music video by itself.

Here are several videos of her using Pianoply (or the experimental stuff that preceded it), or in one video where she is doing other things while appreciating Pianoply more passively. (there is probably 20 minutes of video here, but at least skipping around in them a bit can provide some meaningful insight into the color-enhanced music learning process of kids)

All that said, it doesn't mean Pianoply is for kids. Not at all. It's a ton of fun for adults too, whether they are new to piano, or are advanced (and possibly want to create pianorolls themselves and share them). Pianoply is for anyone who loves music. (Below is a short video from a few days ago of some adults having a blast with Pianoply at an event in San Francisco)

Are you really learning music when using Pianoply?

Absolutely. You just aren't learning traditional notation. But remember, a whole lot of modern music is composed using software (Ableton Live, Logic Pro, etc) that has a pianoroll display. Pianoroll display is simply more appropriate for computers than traditional notation, and is much more flexible and is better able to capture subtleties of timing. It's a perfectly valid way of representing music, even it seems to make some old-school piano teachers bristle. (sorry, Mom!)

In some ways, pianoroll display vs. traditional notation can be seen as analogous to the difference between learning decimals vs. fractions. Decimals have big advantages with computers, and they also have certain intuitive properties that fractions don't. For instance, you can much more easily tell that 0.42857 is larger than 0.41666, than you can tell that 3/7 is larger than 5/12. Also, decimals do not require such a tradeoff between accuracy and readability as fractions do. Regardless, fractions have their uses, and learning decimals should not damage your ability to understand fractions.

Likewise, learning to play with Pianoply should not damage your ability to learn traditional notation. It has one huge advantage though: you can start much earlier. I think the videos of my daughter Stella above should make it very clear how this is something that can be learned at a much younger age than learning traditional notation. There is no reason to wait to start learning the feel of a piano, the muscle memory, the way intervals and chords sound in relation to songs, the geometry of the keyboard, and so on.

How does copyright affect all of this?

It's complicated...

Of course, YouTube allows us to play music videos on our web site, unless the content owner decides they don't want us to. Almost all music videos on YouTube can be embedded into a web site this way. We also suspect that people who are actively engaged with the video, rather than just passively listening to the music, are more likely to click on the ads. The point is, they make money from videos being played Pianoply.

Pianorolls themselves are potentially subject to copyright, although it is not entirely clear what they actually are in terms of copyright law. Are they closer to a performance (i.e. a cover), or to something like sheet music? The tutorials on YouTube seem to be treated as if they are a performance, and prior to the scraper I described above, that might seem accurate. Once you bring the scraper into the picture though, it gets murkier because now users can easily get the "actual notes," not just a moving image of them with accompanying sound. Pianoply itself allows users to easily get to the data, and allows people to easily modify it interactively or with code. But they can also just watch and listen, like it is a performance. It's a whole new world.

We think the pianorolls fall under fair use, as they certainly qualify as "transformative" under US law. The DMCA's "safe harbor" provision may come into play as well, since they are user generated content. Meanwhile the EU is now in the process of changing things around with Article 13 (a.k.a. the "meme killer"). We're going to see how things shake out.

Note that we also are exploring the idea of allowing people to put pianorolls into videos that are displayed on our site, but simply by linking to offsite content, which can be hosted as a pastebin. We can do this without even having the contents ever passing through our server. Alternatively, we can allow people to embed pianoply videos onto their own web pages, with our involvement simply being that we serve them the app itself, in the form of a JavaScript file. This way we can do what we do best (make a really cool, useful app) and let other people worry about whether they have the legal right to share the content. In this sense we are like any number of desktop apps, for instance Adobe Photoshop, in which case the software developer is simply supplying software, rather than directly managing how their software interacts with copyrighted content.

We know that Pianoply is potentially very interesting with regard to the ongoing tension between the desire to freely create, and the right of people to control and monetize things they have created. We are excited to be part of the discussion.

Can YouTubers who do piano tutorials make money by putting their stuff on Pianoply?

We're going to try to figure something out so they can. Most of these YouTubers are already concerned about YouTube shutting them down due to copyright issues, especially if Article 13 goes through. We hope to provide them a viable alternative. We love their work and think it would shine here. I'm currently reaching out to all of them that I can, in hopes of bringing them into the beta and getting their ideas and input.

Where do the instrument sounds come from?

An excellent open source library called WebAudio TinySynth. (everything else in Pianoply is "vanilla JavaScript" and built without 3rd party libraries)

Does Pianoply have any other tricks up it's sleeve?

Here's a pretty big one: in addition to synching notes to videos, we can synch words, and highlight them syllable by syllable. (and very precisely) As great as this is for putting lyrics on music videos, it can also be used on non-music videos, as a powerful way for kids to improve their reading skills, or for people to learn a second language. Another thing we can do is censor the video by very precisely dropping out the audio during "bad words." In theory these capabilities could be a worthwhile enterprise all on their own -- think "parent curated content." Here is a video of Taylor Swift's new song ME! with synched lyrics. For censoring, see the Lil Dicky video "Earth" above or in the grid (a song and video which would otherwise be entirely inappropriate for kids).

Is Pianoply always going to be free?

That's the hope. It would be great to make some money off this (this wasn't exactly slapped together in a weekend), but I think the best route to that is getting as many people as possible using it and spreading the word, gently suggesting that people buy new pianos, and taking a little cut if they do. I'm generally not big on advertising, but this seems like a harmless model, especially if we aren't all pushy about it.

I want Pianoply to do X. Can you make it do it?

Maybe. But even better, Pianoply has an incredibly powerful "mod page" where you can easily write snippets of JavaScript and run them on the site, save them, etc. I'd be happy to help you get started. The hope is that hackers will figure out cool things to do that I haven't thought of, and if they are good we can integrate them into the product somehow. Who knows maybe we can pay them for it, if we are able to make a buck ourselves. We'll see how things go...