Access to music information: The state of the art

Bulletin of the American Society for Information Science, Jun/Jul 2000 by Downie, J Stephen

This article is a synopsis of my contribution to The Sound of Information: Auditory Browsing and Audio Information Retrieval (SIG/VIS), a panel moderated by Abby Goodrum. The presentation date, November 1, 1999, is significant, for on that date I had been in possession of my recently conferred Ph.D. for exactly 9 days. At the panel session, I fell prey to the classic problem that afflicts most recent doctoral graduates: I went on, ad infinitum, about my thesis and never did get around to addressing any of the larger issues. This being the case, I welcome this opportunity to share with you both an outline of my thesis work and an overview of the community-building efforts that are currently being undertaken by myself and others interested in the burgeoning field of Music Information Retrieval (MIR) research and development. My thesis, entitled Evaluating a Simple Approach to Music Information Retrieval: Conceiving Melodic N-grams as Text, will soon be available for downloading at http://mir.lis.uiuc.edu/thesis.

Background

Before the advent of digital computers, the principal method of accessing music information was the thematic catalogue. Scholars and musicians have consulted these printed volumes for over a thousand years. In them, they have found fragments of musical works called incipits that represent the beginnings of a work or significant parts (i.e., themes). According to Barry S. Brook, these incipits have taken on various forms including "conventional notes, neumes, tablatures, numbers, letters or computer codes." The amount of information conveyed by incipits can vary depending on their representation. Sometimes incipits have been verbatim extracts from a musical score and contain pitch, harmonic, rhythmic, editorial, textual and timbral (i.e., sound "colour") information. Other times, the authors of thematic catalogues have seen fit to greatly reduce the amount of information presented by representing only select aspects of a melody, usually pitch names (e.g., Barlow and Morgenstern's Dictionary of Musical Themes). Information has been further reduced by representing incipits, not through the use of the pitches, but by intervals (i.e.,, the distance between notes). Denys Parsons' Directory of Tunes and Musical Themes is the best exemplar of this interval-only approach. Notwithstanding the method of representation used, the one thing that makes all thematic catalogues special is that they attempt to give users the ability to access music information on its own terms; that is, thematic catalogues provide the ability to answer music queries which have been framed musically.

Automating access to music information using digital computers has intrigued musicologists, computer scientists, librarians and music lovers alike. Each has had his/her own purpose in mind and thus there seem to be as many approaches to developing Music Information Retrieval (MIR) systems as there are users. A half-hour perusal of the back issues of Computing in Musicology

http://musedata.stanford.edu/publications/cm/ index.html

will bring this fact to the fore. Some have designed complex suites of computer tools to analyze all the varied facets of music information. One such suite is David Huron's, Humdrum

http://dactyl.som.ohio-state.edu/Humdrum/ Others have tried to automate the thematic catalogue by including incipit or thematic extracts as part of a bibliographic record. The Repertoire International Des Sources Musicales database is a case in point

www.rism.harvard.edu/rism/

Another interesting example of the thematic approach is Huron and Kornstaedt's Themefinder

http://musedata.stanford.edu/databases/themefinder/

Still others have explored the idea of using sophisticated approximate string matching techniques. The New Zealand Digital Library's Meldex

www.nzdl.org/cgi-bin/gw?c=meldex&a=page&p=coltitle

is a good example. Prechelt and Typke's Tuneserver

www.ipd.ira.uka.de/tuneserver/

based upon the work of Denys Parsons, also utilizes approximate string matching techniques. Despite the variety of approaches taken, all extant MIR systems are united by the fact that each has some kind of significant shortcoming. The more powerful analytic systems can be very difficult to use, incipit and thematic indexes both leave out large amounts of music that might be of interest, and approximate string matches can be computationally expensive without necessarily giving better results.

Summarizing My Thesis Research

Taking my cue from those printed thematic catalogues that have reduced the amount of music information represented (e.g., Keller and Rabson's National Tune Index), I developed, and then evaluated, an MIR system based upon the intervals found within the melodies of a collection of 9354 folk songs. I postulated that there is enough information contained within an interval-only representation of monophonic melodies that effective retrieval of music information could be achieved. I extended the thematic catalogue model by affording access to musical expressions found anywhere within a melody (i.e., a "full-text" approach). To achieve this extension, I fragmented the melodies into length-n subsections called n-grams. The length of these n-grams, and the degree to which I precisely represented the intervals, were variables analyzed in the thesis.