January 21, 2003

The Music Business and the Big Flip

By Clay Shirky

The first and last thirds of the music industry have been reconfigured by digital tools. The functions in the middle have not.

Thanks to software like ProTools and CakeWalk, the production of music is heavily digital. Thanks to Napster and its heirs like Gnutella and Kazaa, the reproduction and distribution of music is also digital. As usual, this digitization has taken an enormous amount of power formerly reserved for professionals and delivered it to amateurs. But the middle part -- deciding what new music should be available -- is still analog and still professionally controlled.

The most important departments at a record label are Artists & Repertoire, and Marketing. A&R's job is to find new talent, and Marketing's job is to publicize it. These are both genuinely hard tasks, and unlike production or distribution, there is no serious competition for those functions outside the labels themselves. Prior to its demise, Napster began publicizing itself as a way to find new music, but this was a fig leaf, since users had to know the name of a song or artist in advance. Napster did little to place new music in an existing context, and the current file-sharing networks don't do much better. In strong contrast to writing and photos, almost all the music available on the internet is there because it was chosen by professionals.

- Aggregate Judgments

The curious thing about this state of affairs is that in other domains, we now use amateur input for finding and publicizing. The last 5 years have seen the launch of Google, Blogdex, Kuro5in, Slashdot, and many other collaborative filtering sites that transform the simple judgments of a few participants into aggregate recommendations of remarkably high quality.

This is all part of the Big Flip in publishing generally, where the old notion of "filter, then publish" is giving way to "publish, then filter." There is no need for Slashdot's or Kuro5hin's owners to sort the good posts from the bad in advance, no need for Blogdex or Daypop to pressure people not to post drivel, because lightweight filters applied after the fact work better at large scale than paying editors to enforce minimum quality in advance. A side-effect of the Big Flip is that the division between amateur and professional turns into a spectrum, giving us a world where unpaid writers are discussed side-by-side with New York Times columnists.

The music industry is largely untouched by the Big Flip. The industry harvests the aggregate taste of music lovers and sells it back to us as popularity, without offering anyone the chance to be heard without their approval. The industry's judgment, not ours, still determines the entire domain in which any collaborative filtering will subsequently operate. A working "publish, then filter" system that used our collective judgment to sort new music before it gets played on the radio or sold at the record store would be a revolution.

- Core Assumptions

Several attempts at such a thing have been launched, but most are languishing, because they are constructed as extensions of the current way of producing music, not alternatives to it. A working collaborative filter would have to make three assumptions.

First, it would have to support the users' interests. Most new music is bad, and the users know it. Sites that sell themselves as places for bands to find audiences are analogous to paid placement on search engines -- more marketing vehicle than real filter. FarmFreshMusic, for example lists its goals as "1. To help artists get signed with a record label. 2. To help record labels find great artists efficiently. 3. To help music lovers find the best music on the Internet." Note who comes third.

Second, life is too short to listen to stuff you hate. A working system would have to err more on the side of false negatives (not offering you music you might like) rather than false positives (offering you music you might not like). With false negatives as the default, adventurous users could expand their preferences at will, while the mass of listeners would get the Google version -- not a long list of every possible match, but rather a short list of high relevance, no matter what has been left out.

Finally, the system would have to use lightweight rating methods. The surprise in collaborative filtering is how few people need to be consulted, and how simple their judgments need to be. Each Slashdot comment is moderated up or down only a handful of times, by only a tiny fraction of its readers. The Blogdex Top 50 links are sometimes pointed to by as few as half a dozen weblogs, and the measure of interest is entirely implicit in the choice to link. Despite the almost trivial nature of the input, these systems are remarkably effective, given the mass of mediocrity they are sorting through.

A working filter for music would similarly involve a small number of people (SMS voting at clubs, periodic "jury selection" of editors a la Slashdot, HotOrNot-style user uploads), and would pass the highest ranked recommendations on to progressively larger pools of judgment, which would add increasing degrees of refinement about both quality and classification.

Such a system won't undo inequalities in popularity, of course, because inequality appears whenever a large group expresses their preferences among many options. Few weblogs have many readers while many have few readers, but there is no professional "weblog industry" manipulating popularity. However, putting the filter for music directly in the hands of listeners could reflect our own aggregate judgments back to us more quickly, iteratively, and with less distortion than the system we have today.

- Business Models and Love

Why would musicians voluntarily put new music into such a system?

Money is one answer, of course. Several sorts of businesses profit from music without needing the artificial scarcity of physical media or DRM-protected files. Clubs and concert halls sell music as experience rather than as ownable object, and might welcome a system that identified and marketed artists for free. Webcasting radio stations are currently forced to pay the music industry per listener without extracting fees from the listeners themselves. They might be willing to pay artists for music unencumbered by per-listener fees. Both of these solutions (and other ones, like listener-supported radio) would offer at least some artists some revenues, even if their music were freely available elsewhere.

The more general answer, however, is replacement of greed with love, in Kevin Kelly's felicitous construction. The internet has lowered the threshold of publishing to the point where you no longer need help or permission to distribute your work. What has happened with writing may be possible with music. Like writers, most musicians who work for fame and fortune get neither, but unlike writers, the internet has not offered wide distribution to people making music for the love of the thing. A system that offered musicians a chance at finding an audience outside the professional system would appeal to at least some of them.

- Music Is Different

There are obvious differences here, of course, as music is unlike writing in several important ways. Writing tools are free or cheap, while analog and digital instruments can be expensive, and writing can be done solo, while music-making is usually done by a group, making coordination much more complex. Furthermore, bad music is far more painful to listen to than bad writing is to read, so the difference between amateur and professional music may be far more extreme.

But for all those limits, change may yet come. Unlike an article or essay, people will listen to a song they like over and over again, meaning that even a small amount of high-quality music that found its way from artist to public without passing through an A&R department could create a significant change. This would not upend the professional music industry so much as alter its ecosystem, in the same way newspapers now publish in an environment filled with amateur writing.

Indeed, the world's A&R departments would be among the most avid users of any collaborative filter that really worked. The change would not herald the death of A&R, but rather a reconfiguration of the dynamic. A world where the musicians already had an audience when they were approached by professional publishers would be considerably different from the system we have today, where musicians must get the attention of the world's A&R departments to get an audience in the first place.

Digital changes in music have given us amateur production and distribution, but left intact professional control of fame. It used to be hard to record music, but no longer. It used to be hard to reproduce and distribute music, but no longer. It is still hard to find and publicize good new music. We have created a number of tools that make filtering and publicizing both easy and effective in other domains. The application of those tools to new music could change the musical landscape.

(c) Creative Commons Attribution License. 2003, Clay Shirky.

The licensor permits others to copy, distribute, display, and perform the work. In return, licensees must give the original author credit.

To view a copy of this license, visit http://creativecommons.org/licenses/by/1.0

it seems to me that if an artist wants to get digital distribution through a p2p network... that network would have to contain famous mainstream artists as well or no one would ever sign up for the network in the first place.

It does not work. You have to be already mainstream for people to look for you on the net. Unless you're talking about people that spend all day looking for something 'different' and are not discouraged by all the crap they can get.

The only way you can get distribution is to get an audience. To get an audience you have to be networked. To be networked you either have to join the formal music biz people or play live, starting from your hometown and then going further.

Why do you say that GT? Seems like a music website would have to have major label artists to draw visitors as
well. Sure it would help to create traffic, but there are tons of people out there looking for alternatives to the major label offerings. This is one of the reasons DMusic exists, and it's getting bigger every day.

P2P is not a be all end all, it's just one part of the equation, one of the the tools. You can't build a house with just a hammer, you need saws, levels, and a host of other tools as well. Same with the music biz, P2P is one of the tools, you still should have a website, work radio, live venues, but what P2P does is allow one more avenue of access to the music public that has for so long been denied independent musicians.

olethros has a point, but as the article says, publishing is different from music. I don't know what kind of audience Blogs attract (either in terms of size or type), but perhaps there is a similar way the balance can be tipped. I can see DM is trying to kickstart the process, but it's obviously an uphill battle.

Profile-oriented audio analysis could provide a strong basis for such future distribution. Music could be profiled using a defined set of metrics: chord progression metrics, cadence metrics, harmonic metrics, etc. This acoustic profile could be created for any song (e.g. auto-processing MP3 files), and one could create a profile repository for a large library of music (e.g. gnutella network). An individual could select a set of profiles that represented some "genre of interest", and then search a library for music that had similar profiles. In this way, a person could quickly identify "new" music with an acoustically-validated probability of being interesting to the user, and eliminate text-based searches (which, for audio data, is technically absurd). A user could create a set of profiles ("My Old-Tyme Blues", "My Dethmetal", "My Country Fingerstyle") based on established favorites to search for new music, in a way that was not influenced by popularity perceptions. The technology to support this has existed for some time, but has not been applied in this way as there is no economic incentive for business development (it is however, perfect for a community-based software development project).

I suggest something like a database, for example a website like audiogalaxy or cddb, in which songs are sorted by either artist, genre or album. Or something like a pro version of a certain p2p program, which can collect info on which files are/were available. The normal p2p client program makes a small file in which all shared songs/files on the users are listed and sends the data to all other users on the network and, most importantly, the pro version. Incoming "contents files" will be added to a database file which containst a list of all files on the network available. Then it should sort the files by artist, song, genre and/or album (those parameters should be determined by users of both the pro and the normal version). It then should create a sort of browsing list. The normal versions should also hook up to those pro versions and download those lists or to be able to browse them. The pro version should be able to be nver put in a queue in order to download from the normal version, as a reward for the sacrificed bandwith.

This way, a user can view a huge list of bands which are e.g. rock, metal, house, club ect. They will the best bands on top but browsing further through the list he'll find the more unknown bands. there should also be a distinct list which contains all the newly emerged songs in a certain artist/album/genre list. For example: if you search for metallica, then you'll see a list of songs that are/were available, sorted by album or in alphabetical order. Then down the list there should be a link like "new artist that match this style of music" and link like "new songs added to the database", in which unknown songs are stored for a month or so and then be moved to their appropriate artist/album/genre.

dunno if it will work but it was just a thought.