This seems like it would be perfect for kids. Is that the idea?

Pianoply was absolutely inspired by my daughter Stella's love of music. She was less than 2 when I started the project and now she's five. She loves music and she absolutely loves Pianoply. Whenever a new music video comes out that she likes (which is a couple times a week), she's on my case to "put colors on it." We often play duets Stella and I often play duets, where she'll play the bass line and I'll play the melody or vice versa. She also spends a lot of time listening and watching passively (in "autoplay" mode), which she finds much more satisfying than watching the music video by itself.

Here are several videos of her using Pianoply (or the experimental stuff that preceded it), or in one video where she is doing other things while appreciating Pianoply more passively. (there is probably 20 minutes of video here, but at least skipping around in them a bit can provide some meaningful insight into the color-enhanced music learning process of kids)

Are you really learning music when using Pianoply?

Absolutely. You just aren't learning traditional notation. But remember, a whole lot of modern music is composed using software (Ableton Live, Logic Pro, etc) that has a pianoroll display. Pianoroll display is simply more appropriate for computers than traditional notation, and is much more flexible and is better able to capture subtleties of timing. It's a perfectly valid way of representing music, even it seems to make some old-school piano teachers bristle. (sorry, Mom!)

In some ways, pianoroll display vs. traditional notation can be seen as analogous to the difference between learning decimals vs. fractions. Decimals have big advantages with computers, and they also have certain intuitive properties that fractions don't. For instance, you can much more easily tell that 0.42857 is larger than 0.41666, than you can tell that 3/7 is larger than 5/12. Also, decimals do not require such a tradeoff between accuracy and readability as fractions do. Regardless, fractions have their uses, and learning decimals should not damage your ability to understand fractions.

Likewise, learning to play with Pianoply should not damage your ability to learn traditional notation. It has one huge advantage though: you can start much earlier. I think the videos of my daughter Stella above should make it very clear how this is something that can be learned at a much younger age than learning traditional notation. There is no reason to wait to start learning the feel of a piano, the muscle memory, the way intervals and chords sound in relation to songs, the geometry of the keyboard, and so on.

How does copyright affect all of this?

It's complicated...

Of course, YouTube allows us to play music videos on our web site, unless the content owner decides they don't want us to. Almost all music videos on YouTube can be embedded into a web site this way. We also suspect that people who are actively engaged with the video, rather than just passively listening to the music, are more likely to click on the ads. The point is, they make money from videos being played Pianoply.

Pianorolls themselves are potentially subject to copyright, although it is not entirely clear what they actually are in terms of copyright law. Are they closer to a performance (i.e. a cover), or to something like sheet music? The tutorials on YouTube seem to be treated as if they are a performance, and prior to the scraper I described above, that might seem accurate. Once you bring the scraper into the picture though, it gets murkier because now users can easily get the "actual notes," not just a moving image of them with accompanying sound. Pianoply itself allows users to easily get to the data, and allows people to easily modify it interactively or with code. But they can also just watch and listen, like it is a performance. It's a whole new world.

We think the pianorolls fall under fair use, as they certainly qualify as "transformative" under US law. The DMCA's "safe harbor" provision may come into play as well, since they are user generated content. Meanwhile the EU is now in the process of changing things around with Article 13 (a.k.a. the "meme killer"). We're going to see how things shake out.

Note that we also are exploring the idea of allowing people to put pianorolls into videos that are displayed on our site, but simply by linking to offsite content, which can be hosted as a pastebin. We can do this without even having the contents ever passing through our server. Alternatively, we can allow people to embed pianoply videos onto their own web pages, with our involvement simply being that we serve them the app itself, in the form of a JavaScript file. This way we can do what we do best (make a really cool, useful app) and let other people worry about whether they have the legal right to share the content. In this sense we are like any number of desktop apps, for instance Adobe Photoshop, in which case the software developer is simply supplying software, rather than directly managing how their software interacts with copyrighted content.

We know that Pianoply is potentially very interesting with regard to the ongoing tension between the desire to freely create, and the right of people to control and monetize things they have created. We are excited to be part of the discussion.

Can YouTubers who do piano tutorials make money by putting their stuff on Pianoply?

We're going to try to figure something out so they can. Most of these YouTubers are already concerned about YouTube shutting them down due to copyright issues, especially if Article 13 goes through. We hope to provide them a viable alternative. We love their work and think it would shine here. I'm currently reaching out to all of them that I can, in hopes of bringing them into the beta and getting their ideas and input.

Where do the instrument sounds come from?

An excellent open source library called WebAudio TinySynth. (everything else in Pianoply is "vanilla JavaScript" and built without 3rd party libraries)

Does Pianoply have any other tricks up it's sleeve?

Here's a pretty big one: in addition to synching notes to videos, we can synch words, and highlight them syllable by syllable. (and very precisely) As great as this is for putting lyrics on music videos, it can also be used on non-music videos, as a powerful way for kids to improve their reading skills, or for people to learn a second language. Another thing we can do is censor the video by very precisely dropping out the audio during "bad words." In theory these capabilities could be a worthwhile enterprise all on their own -- think "parent curated content." Here is a video of Taylor Swift's new song ME! with synched lyrics. For censoring, see the Lil Dicky video "Earth" above or in the grid (a song and video which would otherwise be entirely inappropriate for kids).

Is Pianoply always going to be free?

That's the hope. It would be great to make some money off this (this wasn't exactly slapped together in a weekend), but I think the best route to that is getting as many people as possible using it and spreading the word, gently suggesting that people buy new pianos, and taking a little cut if they do. I'm generally not big on advertising, but this seems like a harmless model, especially if we aren't all pushy about it.

I want Pianoply to do X. Can you make it do it?

Maybe. But even better, Pianoply has an incredibly powerful "mod page" where you can easily write snippets of JavaScript and run them on the site, save them, etc. I'd be happy to help you get started. The hope is that hackers will figure out cool things to do that I haven't thought of, and if they are good we can integrate them into the product somehow. Who knows maybe we can pay them for it, if we are able to make a buck ourselves. We'll see how things go...