Intellectual Access to Images - image database systems

Library Trends, Fall, 1999 by Hsin-Liang Chen, Edie M. Rasmussen

In general, it is easier to determine a picture's content than to interpret what it is about, and this distinction has engaged many scholars. Krause (1998) distinguishes between "hard" indexing (the description of what an indexer can see in the frame), and "soft" indexing ("aboutness," the image as stimulus). He says:

   We know that pictures provoke reaction, stimulate ideas, rekindle memories.
   They are powerful instruments in story telling, teaching, propaganda, and
   numerous other fields. Therefore, it is important that libraries provide
   access to images which illustrate ideas, even abstract ones like hunger, or
   the experience of hunger.... If we can index this aspect of the picture, we
   make it easily available to users requiring such an image; we make our
   collection more accessible. (pp. 73-74)

A number of authors (such as Shatford, 1986) have based their analysis of image indexing on the theories of the art historian Panofsky (1939), who identified three levels of meaning in works of art. At the first, or pre-iconographic, level, subject matter was designated as factual ("ofness") or expressional ("aboutness"), and based on the objects and events in an image as it could be interpreted through everyday experience. At the second, or iconographic, level, interpretation requires some cultural knowledge of themes and concepts (not "a sailor" but "Ulysses"). The third or iconological level requires interpretation at a sophisticated level using world and cultural knowledge plus a deeper understanding of the history and background of the work. Shatford (1986) suggests that this third level cannot be indexed with any degree of consistency. Svenonius (1994) points out that "indexing aboutness at the iconographic level is equally problematic" (p. 603), since what is symbolized is not always evident, nor is there always a simple referent to it.

Shatford (1986) uses Panofsky's levels of meaning to explore the kinds of subjects an image might have, proposing "Generic Of, "Specific Of," and "About" with facets answering the questions Who? What? When? and Where? Interestingly, a preliminary attempt in the Hulton study (described later in this article) to categorize queries posed to an image database according to Panofsky's levels of meaning was not successful, suggesting that they did not translate well from the area of Renaissance art to a more general domain (Enser & McGregor, 1993).

Markey (1984) studied interindexer consistency by nonspecialists using a free vocabulary to index pictorial works of art, finding terminology consistency scores of 7 percent and concept consistency scores of 13 percent. While interindexer consistency has always been problematic, even in text, these figures do serve to illustrate the imperfect level of agreement in subject analysis of images. Clearly, image analysis can be carried out at many levels, from the primitive (What colors are present? What shapes?) to more analytical but general (What objects appear in the image? What is this a picture of?.) to a more culturally dependent interpretation (What specific individual or thing is portrayed? What is the mood? What metaphor or lesson is presented?). Choosing an approach to image indexing may require a compromise based on what the system is capable of delivering and what the users of the system would like in an optimal retrieval environment. The question of user need for images is at present relatively little studied.